High ISO = more highlight detail?

The place for questions about shooting with Blackmagic Cameras.
  • Author
  • Message
Offline
User avatar

Robert Niessner

  • Posts: 5025
  • Joined: Thu Feb 21, 2013 9:51 am
  • Location: Graz, Austria

Re: High ISO = more highlight detail?

PostWed Aug 31, 2016 5:02 pm

Eddie Barton wrote:Because, when you change the EI, you change how the image looks in terms of lightness. So, if you properly light for ISO 400 and the image has the correct lightness values but you notice your highlights are clipping, you either throw on ND or close down your aperture. This reduces the amount of light coming in, which is what preserves the highlights. This reduced signal is boosted digitally when you raise the EI to achieve the correct middle gray lightness. The only difference is that your highlights will still be intact in the raw file.

So, yeah, if you only changed the EI but didn't correct the exposure with NDs or aperture, then you'd have the exact same raw data at every EI. The point here is that we're exposing for an EI. So you change the light coming into the camera based on how much room you want in the highlights and shadows. It's like with film. The manufacturer rates a film stock 250 ISO or something. But the DP thinks it's too noisy and wants cleaner shadows, so they overexpose and pull the levels down after the fact. Except now, you don't have to perform the push or pull yourself, it's done for you.


No, this is not like with film. Shooting RAW means any camera setting is just meta data (except shutter speed and aperture). So if you do not change the amount of light hitting the sensor, any ISO setting has no effect whatsoever to the RAW image.
You could compare film to recording in ProRes where your ISO setting sets the transfer curve. You can push/pull later on in post.

And if change the amount of light entering by closing down the aperture or throwing on ND, then you cannot compare two ISO shots as you have changed the input.

I find it hard to believe that after over a decade with RAW capturing people still have troubles to understand what is going on.
Saying "Thx for help!" is not a crime.
--------------------------------
Robert Niessner
LAUFBILDkommission
Graz / Austria
--------------------------------
Blackmagic Camera Blog (German):
http://laufbildkommission.wordpress.com

Read the blog in English via Google Translate:
http://tinyurl.com/pjf6a3m
Offline
User avatar

Uli Plank

  • Posts: 21785
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: High ISO = more highlight detail?

PostWed Aug 31, 2016 5:23 pm

I'd follow this description for a Red or Arri camera any time. But what if BM is really applying different curves to the analog signal or the linear data coming from the sensor before storing digital RAW?
Now that the cat #19 is out of the bag, test it as much as you can and use the subforum.

Studio 18.6.6, MacOS 13.6.6, 2017 iMac, 32 GB, Radeon Pro 580
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G
Offline
User avatar

Robert Niessner

  • Posts: 5025
  • Joined: Thu Feb 21, 2013 9:51 am
  • Location: Graz, Austria

Re: High ISO = more highlight detail?

PostWed Aug 31, 2016 5:42 pm

Uli Plank wrote:I'd follow this description for a Red or Arri camera any time. But what if BM is really applying different curves to the analog signal or the linear data coming from the sensor before storing digital RAW?


At least for the BMCC there is no difference, as I have tested that years ago (and Tom has tested that too). I would not expect BMD changing their sensor philosophy without reason.
Saying "Thx for help!" is not a crime.
--------------------------------
Robert Niessner
LAUFBILDkommission
Graz / Austria
--------------------------------
Blackmagic Camera Blog (German):
http://laufbildkommission.wordpress.com

Read the blog in English via Google Translate:
http://tinyurl.com/pjf6a3m
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostWed Aug 31, 2016 8:28 pm

Robert Niessner wrote:No, this is not like with film. Shooting RAW means any camera setting is just meta data (except shutter speed and aperture). So if you do not change the amount of light hitting the sensor, any ISO setting has no effect whatsoever to the RAW image.
You could compare film to recording in ProRes where your ISO setting sets the transfer curve. You can push/pull later on in post.

And if change the amount of light entering by closing down the aperture or throwing on ND, then you cannot compare two ISO shots as you have changed the input.

I find it hard to believe that after over a decade with RAW capturing people still have troubles to understand what is going on.


I'm just telling you where the practice of rating sensitivity relative to dynamic range shift came from, whether you want to accept it or not is up to you. You are correct that the actual DPX scans are more like ProRes than raw, but that was not the focus of my statement, the practice was.

And you're missing the point trying to be made. We're not saying changing the EI changes light hitting the sensor or the sensitivity of the sensor. It just adjusts the digital push/pull (metadata) to change what you see on the scopes and screen as if the sensor were actually at that sensitivity. The point of the system is to always keep middle gray at 0.18f. That's our reference. So when we expose a sensor that is rated by the manufacturer as 800 at 1600, we let in one less stop of light. The result of this is that we protect one stop of highlight range, but we lose a stop of shadow range. The chart isn't saying "change your ISO to change how your DR is distributed." It's to be used as an exposure guide. Because there is no analog gain on the signal, we are able to say, "if you expose the camera at this EI, this is the dynamic range distribution you can expect." It's to help understand what is happening when you expose for different EI values. Not saying that a particular EI will change the image without any physical adjustment on the part of the operator.

I can show you the math if you'd like.
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostWed Aug 31, 2016 8:31 pm

Uli Plank wrote:I'd follow this description for a Red or Arri camera any time. But what if BM is really applying different curves to the analog signal or the linear data coming from the sensor before storing digital RAW?


They could be, but I highly doubt it for anything other than 1600 to keep extra steps in the shadows. All the inverse curves are stored in the metadata under the tag LinearizationTable and we are able to analyze them in something like Excel to see how they're manipulating the linear 16b signal.
Offline

Benjamin de Menil

  • Posts: 330
  • Joined: Wed Feb 04, 2015 9:04 pm

Re: High ISO = more highlight detail?

PostWed Aug 31, 2016 9:41 pm

Does this mean that shooting at higher ISO doesn't make the raw image grainier? ie, shoot at 1600 instead of 800, and give 1 stop more light, and you'll have the same image - no difference in graininess?
Offline

Steven Abrams

  • Posts: 275
  • Joined: Tue Feb 24, 2015 12:43 am
  • Location: LA La Land

Re: High ISO = more highlight detail?

PostWed Aug 31, 2016 10:17 pm

Eddie Barton wrote:I can show you the math if you'd like.


Yes please!
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostWed Aug 31, 2016 10:30 pm

Benjamin de Menil wrote:Does this mean that shooting at higher ISO doesn't make the raw image grainier? ie, shoot at 1600 instead of 800, and give 1 stop more light, and you'll have the same image - no difference in graininess?


If I'm understanding your question correctly, then yes, you're effectively exposing for ISO 800. But you would need to drop the exposure slider in Resolve by one stop in order to achieve correct exposure. So there would be no difference between the two.

Steven Abrams wrote:
Eddie Barton wrote:I can show you the math if you'd like.


Yes please!


Typing it up now. Anyone know if these forums support Markdown and LaTeX?
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostThu Sep 01, 2016 12:29 am

Steven Abrams wrote:
Eddie Barton wrote:I can show you the math if you'd like.


Yes please!


So I'm just gonna go through the thought and mathematical process I went through to come up with these charts.

I'll start off looking at the metadata of a BMCC DNG. The value we're interested in right now that allows the exposure compensation to happen is BaselineExposure. From the DNG specification, "BaselineExposure specifies by how much (in EV units) to move the zero point." The zero point being the value that middle gray falls at after the DNG data has been linearized and scaled according to the LinearizationTable, WhiteLevel, and BlackLevel (also found in DNG spec). What BaselineExposure does is tell Resolve, ACR, etc. how many stops to push or pull the data by in order to make a middle gray chart exposed at the selected EI give a value of 0.18.

Now the important thing is how we get this value. Okay, now lets assume we've taken our BMCC and shot a perfectly exposed middle gray card. We bring our DNG file into a program that allows us to view the untouched, unlinearized data. We apply our linearization table and scale the data. Now we have our 16b linear values. Let's say that our middle gray chart gives us an average value of 2235 on our 16b scale. And let's say that when we exposed the chart, we set the meter to EI 800 (not exactly important just yet, but will be in a moment).

So, our middle gray is 2235 / 65535. Now we want middle gray to be 0.18 (the rock of this system) to get scene referred linear data. So we set up our function such that we have our target of 0.18 on one side and our known value with some adjustment variable to make the mathematical statement true:

0.18 = (2235 / 65535) * q

Where "q" is our variable that represents the exposure adjustment as a linear multiplier. We find q with some simple algebra and get:

q = 0.18 * (65535 / 2235) ≈ 5.278

Now, to get our multiplier in the unit of photographic stops (EV), we have to take the base-2 logarithm of q.

log2(5.278) ≈ 2.39999 EV ≈ 2.4 EV

Now, if I remember correctly, BaselineExposure in an EI 800 file from the BMCC is 2.4. For EI 400 and 1600 we would expect the values to be 1.4 and 3.4, respectively, because they are one stop differences. To confirm this we do our test again, and we know exposing for a one stop lower ISO means we have twice the light. So our middle gray chart comes up with an average value of around 4470 / 65535. We apply our formula again:

q = 0.18 * (65535 / 4470) ≈ 2.639

If you multiply it by two, you'll see that it is 5.278; so we know we're on the right track. Now we take the base-2 logarithm and get:

log2(2.639) ≈ 1.39999 EV ≈ 1.4 EV

And a similar process would happen for 1600, except we would get half the linear middle gray value to being with, 1118. This would result in the multiplier being double and the EV adjustment value being one stop greater than what we had for 800, as expected.

Of course this only makes sense when you do correct exposure. So, now we know that shooting at EI 200 on the BMCC is effectively the same as shooting EI 800, exposing to the right by two stops, and dropping the "Exposure" slider to -2.00 in Resolve. Now the question posed to us is: "how do these various exposure settings affect our dynamic range distribution?"

Well, we know that BM claims a 13 stop dynamic range for the BMCC. We also know that highlight range is related to the ratio of light between clipping and middle gray. With these two pieces of knowledge, we can come up with distribution for a certain EI. The inverse LUT for BMD Film that BMD includes with Resolve also helps us set our upper limit for highlight range is. Let's return to our middle gray value for EI 800 that we got before, 2235 on a 16b scale. We know that clipping occurs at a linear value of 1.0, or 65535 on a 16b scale. The highlight range is defined as the base-2 logarithm of the ratio between the saturation point (clipping) and the middle gray value. So we get:

log2(65535 / 2235) ≈ 4.874 EV

So why is it not reaching 5 stops like we expected? Well, these calculations don't take highlight reconstruction (HLR) into account. Once HLR has its way with the image, we get linear values above the conventional clipping point of 1.0. This means our highlight range is going to be 5 stops plus a little. So around 5.1 - 5.2 ish. Now, why didn't I put 5.1 or 5.2 as the upper limit on the BMCC chart for 800? This comes from the BMD Film to Linear LUT included in Resolve. What I was looking for was the maximum scene referred linear value that BMD Film could encode before clipping. In other words I want to know what scene reffered linear value corresponds to a BMD Film value of 1.0.

In the 12b LUT we see the maximum value is 5.7661304310. Now we can apply the same formula to find the maximum highlight range BMD Film can store, since it was developed for the native EI.

log2(5.7661304310 / 0.18) ≈ 5.00 EV

And the shadow range must be the difference between the total and the highlight range.

13.0 EV - 5.0 EV = 8.0 EV

So there's our dynamic range distribution for EI 800, and we already proved that exposing for another EI shifts our middle gray point by the expected amount of stops. Since exposing for EI 400 set our middle gray point as twice of what it was at 800, that means we have half the amount of light we can record in the highlights, or one stop less, and twice the amount of light recorded in the shadows, or one stop more. Therefore, lowering the EI shifts the distribution down, and in the case of EI 400, it shifts by one stop, and similar for the opposite direction.

The values for 1600 should show an equal amount of highlight range since BMD Film can't encode that high before clipping, but I decided to put what the reasonable expected value for the raw data would be, 6.0 EV. And that is what you'll see if you test it. And if you try using another color space option via Resolve's color management or the color space plugin they include, you can use BMD Film as the input space and something like Log-C as your output. All of your highlight detail will come back for you to use. There's also the option of using the highlight slider in Resolve; I prefer not to because that causes nonlinearity in the underlying data.

I think I covered everything. If I missed something or if you have any questions, please feel free to ask. I'll answer to the best of my ability.
Offline
User avatar

PaulDelVecchio

  • Posts: 799
  • Joined: Wed Aug 22, 2012 5:33 am
  • Location: NY

Re: High ISO = more highlight detail?

PostThu Sep 01, 2016 6:53 am

My brain hurts.
Paul Del Vecchio - Director/Producer
http://www.pauldv.net
http://www.youtube.com/user/pdelvecchio814
http://www.facebook.com/pauldv
http://instagram.com/pdelv
Twitter: @pauldv
Offline

Soeren Mueller

  • Posts: 604
  • Joined: Tue Sep 04, 2012 2:21 pm
  • Location: Düsseldorf, Germany

Re: High ISO = more highlight detail?

PostThu Sep 01, 2016 9:23 am

Eddie Barton wrote:So, yeah, if you only changed the EI but didn't correct the exposure with NDs or aperture, then you'd have the exact same raw data at every EI. The point here is that we're exposing for an EI. So you change the light coming into the camera based on how much room you want in the highlights and shadows.


Eddie that's exactly what I mean - you guys are talking about changing exposure but a lot of postings make it sound like the camera "behaves" fundamentally different when you change ISO, which is definitely not the case. (besides the only very slightly different curve applied)

I mean it's not really rocket science it's basically saying I'll just lower exposure and pretend this native ISO800 sensor is really ISO1600 native so I can protect more of my highlights with the cost of having more noise in the shadows (if I have to lift them in post because I lowered exposure).

Maybe it comes from a point of people that took ETTR a little to the extreme and got frustrated by too many clipped highlights.. just trying to understand ;)
Offline
User avatar

PaulDelVecchio

  • Posts: 799
  • Joined: Wed Aug 22, 2012 5:33 am
  • Location: NY

Re: High ISO = more highlight detail?

PostThu Sep 01, 2016 11:55 am

So let me pose this question if you guys are using IRE or False Color to expose. Where are you placing skin tones typically?

I expose skin tones at around 35-45 or 40-50 typically for a normally lit scene. If you go by camera False Color, that would be approximately green instead of pink.

This gives me roughly a stop more up top, essentially like exposing skin at pink at 1600.
Paul Del Vecchio - Director/Producer
http://www.pauldv.net
http://www.youtube.com/user/pdelvecchio814
http://www.facebook.com/pauldv
http://instagram.com/pdelv
Twitter: @pauldv
Offline
User avatar

timbutt2

  • Posts: 2933
  • Joined: Tue Nov 12, 2013 10:32 am
  • Location: St. Petersburg, Florida, United States of Amercia

Re: High ISO = more highlight detail?

PostThu Sep 01, 2016 2:57 pm

PaulDelVecchio wrote:My brain hurts.

Yeah, me too. It starts to get to that "it's too much technical babble" and I couldn't even talk this with anyone that cares on set. All that matters is how it all applies to the creative craft.

Since I know from testing how the ISO effects the image in the grand scheme of things any director is only need me to say "I'm shooting at a lower ISO because I can push it in post with less noise, but yes, it makes it more difficult to see the image while shooting." Do you know what! I'm going to make a new low-light LUT for shooting in 200 ASA setting for on set monitoring. That should solve any director comments about not being able to see the image. Hmmm... Why didn't I think of that before.

Firmware 4.0 to the rescue!
"I'm well trained in the art of turning **** to gold." - Tim Buttner (timbutt2)

Cameras: URSA Mini Pro G2 & Pocket 6K Pro
Past: UM4.6K, P6K, BMCC 2.5K
Computers: iMac 5K (Mid 2020) & MacBook Pro Retina 15.4in (Mid 2018)
Offline
User avatar

Valentin Remy

  • Posts: 282
  • Joined: Thu Mar 24, 2016 3:52 pm
  • Location: Belgium

Re: High ISO = more highlight detail?

PostThu Sep 01, 2016 7:19 pm

I just tested the DR difference today in bright sunlight, between ISO200 and ISO1600.

I wasn't expecting such a huge difference :o Thanks to the thread op for the info !
http://www.instagram.com/valentinremy.be/
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostFri Sep 02, 2016 2:00 am

timbutt2 wrote:
PaulDelVecchio wrote:My brain hurts.

Yeah, me too. It starts to get to that "it's too much technical babble" and I couldn't even talk this with anyone that cares on set. All that matters is how it all applies to the creative craft.

Since I know from testing how the ISO effects the image in the grand scheme of things any director is only need me to say "I'm shooting at a lower ISO because I can push it in post with less noise, but yes, it makes it more difficult to see the image while shooting." Do you know what! I'm going to make a new low-light LUT for shooting in 200 ASA setting for on set monitoring. That should solve any director comments about not being able to see the image. Hmmm... Why didn't I think of that before.

Firmware 4.0 to the rescue!


Apologies. I wouldn't expect the contents of that post to actually be brought up in a conversation other than those who design the inner workings of the cameras. I just posted it as logical/mathematical proof that what I'm saying isn't just some crazy, out-of-nowhere BS.

And about the shooting at lower ISO having less noise and looking underexposed on screen, you're really just exposing for a higher ISO like 400 or 800 and you're not really going to have less noise. You'd have the same exact amount of noise as the ISO that would be considered correct exposure for the amount of light you're letting in. So creating a viewing LUT for underexposed ISO 200 is essentially what the camera already does when you switch it to 400, 800, or 1600, so it's ultimately redundant.

Soeren Mueller wrote:Eddie that's exactly what I mean - you guys are talking about changing exposure but a lot of postings make it sound like the camera "behaves" fundamentally different when you change ISO, which is definitely not the case. (besides the only very slightly different curve applied)

I mean it's not really rocket science it's basically saying I'll just lower exposure and pretend this native ISO800 sensor is really ISO1600 native so I can protect more of my highlights with the cost of having more noise in the shadows (if I have to lift them in post because I lowered exposure).

Maybe it comes from a point of people that took ETTR a little to the extreme and got frustrated by too many clipped highlights.. just trying to understand ;)


I never meant to use language that implied that changing EI changed something with the camera's behavior. If I did, I do apologize. I tried my best to use terms and phrases related to actual under/overexposure and things like ETTR/ETTL, which requires changing the amount of light coming into the camera. And you're right, it isn't rocket science, and I definitely didn't want to make it out to be, because then I'm failing at my job.

I'm just trying to help answer questions and understand a bit more stuff myself. Not trying to cause any trouble, but I will defend my arguments to the best of my ability. I hope no one thought I was malicious with my posts.

PaulDelVecchio wrote:So let me pose this question if you guys are using IRE or False Color to expose. Where are you placing skin tones typically?

I expose skin tones at around 35-45 or 40-50 typically for a normally lit scene. If you go by camera False Color, that would be approximately green instead of pink.

This gives me roughly a stop more up top, essentially like exposing skin at pink at 1600.


I don't know how precise BM's false color lets you get, but on BMD Film and BMD Film 4K middle gray is 392/1023, or 38.3%; 4.6K is 420/1023, or 40.6%. Average skin tone is maybe a stop more than that, and on the characteristic chart you'll see +1 falls around 510 for Film, 550 for Film 4K, and 520 for Film 4.6K. That would be 50.9%, 55.5%, and 52.1% respectively.

And those values stay the same no matter what EI you're at. That ensures proper exposure each time.

I can make a chart of relevant IRE values if that'd be something y'all are interested in.
Offline

Soeren Mueller

  • Posts: 604
  • Joined: Tue Sep 04, 2012 2:21 pm
  • Location: Düsseldorf, Germany

Re: High ISO = more highlight detail?

PostFri Sep 02, 2016 5:40 am

Eddie Barton wrote:I'm just trying to help answer questions and understand a bit more stuff myself. Not trying to cause any trouble, but I will defend my arguments to the best of my ability. I hope no one thought I was malicious with my posts.


Sorry I didn't want to imply anything like that - it's definitely the opposite of being malicious I think to explain the math behind it! I'm into that myself ;)

Just not always easy in this field with the mixture of hardcore tech/nerdism on one end and the 100% creative/artsy people on the other end and everything in between. It's like a bunch of people speaking completely different languages.

One example of this is now people starting to state that the DR is very different between the several ISO settings on camera. Which is not really true. Others are reading that sentence and save it as fact that the ISO is changing the way the camera records the picture. That's just what bothered me..
Offline
User avatar

Robert Niessner

  • Posts: 5025
  • Joined: Thu Feb 21, 2013 9:51 am
  • Location: Graz, Austria

Re: High ISO = more highlight detail?

PostFri Sep 02, 2016 8:35 am

@ Eddie & Soeren
100% agree with you.
BTW, Eddie - very nice, long, well written mathematical explanation!
Next time spice it up with some nice differential equation here and there, just for the fun ;)
Saying "Thx for help!" is not a crime.
--------------------------------
Robert Niessner
LAUFBILDkommission
Graz / Austria
--------------------------------
Blackmagic Camera Blog (German):
http://laufbildkommission.wordpress.com

Read the blog in English via Google Translate:
http://tinyurl.com/pjf6a3m
Offline
User avatar

Troy Turner

  • Posts: 95
  • Joined: Sat Nov 02, 2013 1:03 pm

Re: High ISO = more highlight detail?

PostSat Sep 03, 2016 1:43 am

If I recorded a clip underexposed by one stop at ISO 800, would the RAW file be the same if I recorded a second clip at ISO 1600 and no other changes? Or would the RAW file have a different distribution of data?

I'm trying to understand this thread. . .

:shock:
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostSat Sep 03, 2016 6:29 am

Soeren Mueller wrote:Sorry I didn't want to imply anything like that - it's definitely the opposite of being malicious I think to explain the math behind it! I'm into that myself ;)

Just not always easy in this field with the mixture of hardcore tech/nerdism on one end and the 100% creative/artsy people on the other end and everything in between. It's like a bunch of people speaking completely different languages.

One example of this is now people starting to state that the DR is very different between the several ISO settings on camera. Which is not really true. Others are reading that sentence and save it as fact that the ISO is changing the way the camera records the picture. That's just what bothered me..

Definitely agree that it is difficult to bridge the gap between creative and technical. And I understand what you're saying now, as I saw the misunderstanding in another thread as well. This is the kind of thing that would be easier to explain by example in a video showing the actual process.

Robert Niessner wrote:@ Eddie & Soeren
100% agree with you.
BTW, Eddie - very nice, long, well written mathematical explanation!
Next time spice it up with some nice differential equation here and there, just for the fun ;)

We'll save the DE's for gradient domain operations haha ;)

Troy Turner wrote:If I recorded a clip underexposed by one stop at ISO 800, would the RAW file be the same if I recorded a second clip at ISO 1600 and no other changes? Or would the RAW file have a different distribution of data?

I'm trying to understand this thread. . .

:shock:

That's exactly right. ISO 800 underexposed by one stop would have the same dynamic range distribution as properly exposed 1600. They are the same amounts of light, so they are technically the same data. Though, apparently, BMD has done something to the underlying 16b linear to 12b log curve to increase the amount of bits given to the shadows, but that's invisible to the user and doesn't affect the dynamic range distribution in the final BMD Film image.
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostSat Sep 03, 2016 6:34 am

Also, I think it's important to point out that the log curve never changes! It's the linear data that is shifted before the log curve is applied. Just thought I'd add that because I see this common misconception. The only camera whose log curve changes with EI that I know of is the Alexa, the rest (Sony, RED, Panasonic, Canon, BMD, Digital Bolex) use static, unchanging curves.
Offline
User avatar

timbutt2

  • Posts: 2933
  • Joined: Tue Nov 12, 2013 10:32 am
  • Location: St. Petersburg, Florida, United States of Amercia

Re: High ISO = more highlight detail?

PostSat Sep 03, 2016 7:44 am

Eddie Barton wrote:
timbutt2 wrote:
PaulDelVecchio wrote:My brain hurts.

Yeah, me too. It starts to get to that "it's too much technical babble" and I couldn't even talk this with anyone that cares on set. All that matters is how it all applies to the creative craft.

Since I know from testing how the ISO effects the image in the grand scheme of things any director is only need me to say "I'm shooting at a lower ISO because I can push it in post with less noise, but yes, it makes it more difficult to see the image while shooting." Do you know what! I'm going to make a new low-light LUT for shooting in 200 ASA setting for on set monitoring. That should solve any director comments about not being able to see the image. Hmmm... Why didn't I think of that before.

Firmware 4.0 to the rescue!


Apologies. I wouldn't expect the contents of that post to actually be brought up in a conversation other than those who design the inner workings of the cameras. I just posted it as logical/mathematical proof that what I'm saying isn't just some crazy, out-of-nowhere BS.

No need to apologize. I was merely pointing out that the technical level of all of this reached a point beyond what I was ever willing to discuss with clients. I appreciate all the work and research you put into this.

I myself am more creative than technical. However, I am somehow intertwined with the technical to a level that I've surpassed what I though was possible when I was a young pre-teen. I use ISO as it needs to be used. Above all else I translate as much to film as I can, and yet know how digital emulsion is different. When I said, create a LUT for 200 ISO it was more for the preview in camera purposes. I'm working on it still.

At this point my LUT is in a satisfactory place for me. However, I want to ensure it's fair enough to share. That rhymed. Hmm... Creative writing or coincidence?

Either way, we have to remember that the audience never is aware of these technical quandaries that we find ourselves debating. That's why I put story first when I discuss all technical aspects. Might also be why the people who've worked with me respect my opinion. After all, we're always trying to tell a story visually and audibly.

ISO is a tool within the camera, which is a tool for capturing the visual images. The key is always going to be understanding how each tool works. Thank you for all your hard work in helping us understand these tools with regards to the Blackmagic Cameras.
"I'm well trained in the art of turning **** to gold." - Tim Buttner (timbutt2)

Cameras: URSA Mini Pro G2 & Pocket 6K Pro
Past: UM4.6K, P6K, BMCC 2.5K
Computers: iMac 5K (Mid 2020) & MacBook Pro Retina 15.4in (Mid 2018)
Offline

Simon McLean

  • Posts: 141
  • Joined: Sat Jan 05, 2013 9:39 pm
  • Location: San Francisco

Re: High ISO = more highlight detail?

PostSat Sep 03, 2016 11:29 pm

Yes, thank you Eddie for all the technical explanations. Even for us less technically inclined it's very helpful. cheers.
Offline

Steven Abrams

  • Posts: 275
  • Joined: Tue Feb 24, 2015 12:43 am
  • Location: LA La Land

Re: High ISO = more highlight detail?

PostSat Sep 03, 2016 11:49 pm

Eddie Barton wrote:Also, I think it's important to point out that the log curve never changes! It's the linear data that is shifted before the log curve is applied. Just thought I'd add that because I see this common misconception. The only camera whose log curve changes with EI that I know of is the Alexa, the rest (Sony, RED, Panasonic, Canon, BMD, Digital Bolex) use static, unchanging curves.

Are you positive? For the 4.6K raws when you change ISO in resolve the log curve seems to change. I was told the others are meant to as well but for some reason resolve didn't add support. Its strange since its the same company but i think resolve team is in different country to the camera team. You can verify by shooting prores film at different ISO and a single raw, changing the ISO for the RAW in resolve doesn't match the prores curves so seems like resolve isn't matching what the camera does. But for the 4.6K it does seem to match the different isos curves of prores.
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostSun Sep 04, 2016 12:23 am

Steven Abrams wrote:
Eddie Barton wrote:Also, I think it's important to point out that the log curve never changes! It's the linear data that is shifted before the log curve is applied. Just thought I'd add that because I see this common misconception. The only camera whose log curve changes with EI that I know of is the Alexa, the rest (Sony, RED, Panasonic, Canon, BMD, Digital Bolex) use static, unchanging curves.

Are you positive? For the 4.6K raws when you change ISO in resolve the log curve seems to change. I was told the others are meant to as well but for some reason resolve didn't add support. Its strange since its the same company but i think resolve team is in different country to the camera team. You can verify by shooting prores film at different ISO and a single raw, changing the ISO for the RAW in resolve doesn't match the prores curves so seems like resolve isn't matching what the camera does. But for the 4.6K it does seem to match the different isos curves of prores.


How do you mean the log curve "seems to change" in regards to the raw panel in Resolve? I've done tests with BMPCC DNGs exposed at each ISO and the curve stayed exactly the same in each image (I didn't perform the exposures, just handled the processing).

The curve being static is the reason properly exposed ISO 1600 clips exactly the same as properly exposed ISO 800 when you don't shift the highlights slider in Resolve.

If you want to confirm it, record an 18% gray card and a 90% white card, or you can use a ColorChecker. Properly expose for each ISO recording ProRes and DNG. That removes the variables.

Sent from my VS986 using Tapatalk
Offline
User avatar

Uli Plank

  • Posts: 21785
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: High ISO = more highlight detail?

PostSun Sep 04, 2016 10:13 am

After careful testing I can confirm that changing ISO doesn't change the RAW recording at all – just like it should as a true RAW recording. Yes, it changes the curves for display and passes such changes on as metadata. Of course, Resolve can reproduce such curves. But it's all there just to help you to judge your lighting situation for what you may probably want to achieve when changing ISO. But the clipping point as an upper limit and the amount of noise as the lower one doesn't change at all! It'll just allow you to look deeper into the noise to judge it or into your highlights to check protection.

When digital cinematography with true RAW recording started, there were lots of misunderstandings about the use of ISO and endless discussions on Reduser (for example). Other than with film, where you can really change ISO by switching to another reel, you can't switch your sensor (yet – hey BM, what about the big Ursa?). If you go for a higher ISO it allows you to look deeper into your shadows to judge noise for low-light situations. If you switch to a lower one, it's the contrary: you can see better what happens in your highlights. BM tries to help you by adapting the curve, stretching the shadows for 1600 or stretching the highlights for 200 or 400. If you don't like that, stick to 800.

Of course, if you record ProRes or DNxHD, this is baked in, so expose accordingly. But for RAW switching ISO does NOTHING to your recording that you couldn't do in post! If you get the impression that it might change the amount of noise, please keep in mind that judging the lower limit of acceptable exposure is subjective and also depends on style and content, while clipping is a hard limit.

For me, it's another "case closed".
Now that the cat #19 is out of the bag, test it as much as you can and use the subforum.

Studio 18.6.6, MacOS 13.6.6, 2017 iMac, 32 GB, Radeon Pro 580
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G
Offline

Steven Abrams

  • Posts: 275
  • Joined: Tue Feb 24, 2015 12:43 am
  • Location: LA La Land

Re: High ISO = more highlight detail?

PostSun Sep 04, 2016 12:53 pm

Eddie Barton wrote:How do you mean the log curve "seems to change" in regards to the raw panel in Resolve? I've done tests with BMPCC DNGs exposed at each ISO and the curve stayed exactly the same in each image (I didn't perform the exposures, just handled the processing).

The curve being static is the reason properly exposed ISO 1600 clips exactly the same as properly exposed ISO 800 when you don't shift the highlights slider in Resolve.

If you want to confirm it, record an 18% gray card and a 90% white card, or you can use a ColorChecker. Properly expose for each ISO recording ProRes and DNG. That removes the variables.

Sent from my VS986 using Tapatalk

Yeah, I've done that and for bmcc and bmpcc and the RAW and ProRes did not match across different ISO's in resolve. But for the 4.6K they did. And I was told by someone I trust that Resolve doesn't have or use the different ISO curves for BMDFilm and the 4K like the camera does for ProRes, but it does have them for the 4.6K.
Offline
User avatar

rick.lang

  • Posts: 17278
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

Re: High ISO = more highlight detail?

PostSun Sep 04, 2016 2:47 pm

Here is the video test I did a week ago in which raw and ProRes are compared for noise. The shots with ISO 1600 illustrate the difference between raw and ProRes when the camera is set to ISO 1600 versus 800 or lower. ProRes controls the noise better than raw.



Password UM46K
Rick Lang
Offline
User avatar

Uli Plank

  • Posts: 21785
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: High ISO = more highlight detail?

PostSun Sep 04, 2016 3:28 pm

ProRes does some denoising of finest detail for better compression, that's why it also cancels some Moiré out.
Now that the cat #19 is out of the bag, test it as much as you can and use the subforum.

Studio 18.6.6, MacOS 13.6.6, 2017 iMac, 32 GB, Radeon Pro 580
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G
Offline

Denny Smith

  • Posts: 13131
  • Joined: Thu Aug 01, 2013 4:19 pm
  • Location: USA, Northern Calif.

Re: High ISO = more highlight detail?

PostSun Sep 04, 2016 4:34 pm

Interesting test Rick, thanks.
Cheers
Denny Smith
SHA Productions
Offline
User avatar

rick.lang

  • Posts: 17278
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

Re: High ISO = more highlight detail?

PostMon Sep 05, 2016 12:29 am

Thanks, for the comments, folks. I think it illustrates two important takeaways for me: ISO 1600 raw is not as clean as ISO 800 raw and ISO 1600 ProRes is much cleaner than ISO 1600 raw.

I can see that pushing the exposure of ISO 800 raw in post a stop would be a viable alternative to shooting ISO 1600 raw. Kinda wish I had tried that for the low light shots at my daughter’s wedding, but I hadn’t tested that aspect before the early July shoot and went with ISO 1600 raw.

I applied Frank’s Instant Cinema just add water LUT to the ungraded test above and it sure did wonders for the images of the furniture and so on. Reminded me of the lovely colours Frank achieves on his interior shots also with lots of wood in frame. It even seemed to reduce the appearance of noise in the ISO 1600 raw clips significantly which I didn’t expect.

Tried the same LUT on the wedding shoot and it’s too strong a treatment, but I can work on it later to make it more pleasing for ladies’ faces.
Rick Lang
Offline

John Paines

  • Posts: 5828
  • Joined: Tue Jul 28, 2015 4:04 pm

Re: High ISO = more highlight detail?

PostWed Sep 07, 2016 3:28 pm

PaulDelVecchio wrote:What I did was set the blacks to 0 on both and in the RAW tab, I set the Highlights to -50 for both and SATURATION to +30 for both..... 1600 clearly has more detail.


Forgive me, I've come late and haven't read every tortured word here, but this makes no sense! Or so it seems to me....

Are you saying that you could not make the 800 shot yield the same apparent highlight detail seen in the 1600 shot, by bringing down the highlights? The fact that you used the same settings for both shots, doesn't prove the camera yields better highlights at higher isos.

Or have I missed the point entirely? If the 4.6K is like the BMPCC, it clips at exactly the same light level (in raw), regardless of ISO setting.... 1600 doesn't actually gain you additional highlight head-room.
Offline
User avatar

PaulDelVecchio

  • Posts: 799
  • Joined: Wed Aug 22, 2012 5:33 am
  • Location: NY

Re: High ISO = more highlight detail?

PostWed Sep 07, 2016 4:54 pm

John Paines wrote:
PaulDelVecchio wrote:What I did was set the blacks to 0 on both and in the RAW tab, I set the Highlights to -50 for both and SATURATION to +30 for both..... 1600 clearly has more detail.


Forgive me, I've come late and haven't read every tortured word here, but this makes no sense! Or so it seems to me....

Are you saying that you could not make the 800 shot yield the same apparent highlight detail seen in the 1600 shot, by bringing down the highlights? The fact that you used the same settings for both shots, doesn't prove the camera yields better highlights at higher isos.

Or have I missed the point entirely? If the 4.6K is like the BMPCC, it clips at exactly the same light level (in raw), regardless of ISO setting.... 1600 doesn't actually gain you additional highlight head-room.


Yeah it's confusing in a way. Here it is simply:

Let's say you exposed skin at 45-50 IRE as a base and you're at 800 ISO. If you then bump it up to 1600, your skin tones move up to maybe 55-65 IRE so you'd have to stop down to bring them back to 45-50 IRE thus protecting the highlights.

Sure you could do it at 800 and underexpose by 1 stop. You'd have to play around with the curve a bit more because you'd be pushing a lot of the image into the bottom end but it will work once you get the hang of it.
Paul Del Vecchio - Director/Producer
http://www.pauldv.net
http://www.youtube.com/user/pdelvecchio814
http://www.facebook.com/pauldv
http://instagram.com/pdelv
Twitter: @pauldv
Offline

John Paines

  • Posts: 5828
  • Joined: Tue Jul 28, 2015 4:04 pm

Re: High ISO = more highlight detail?

PostWed Sep 07, 2016 5:22 pm

But that's just it.... I don't think you *are* protecting the highlights, or actually gaining anything, by shooting at 1600 and bringing the levels down in post.

I'd wager that you can make the highlights of the 800 shot looks exactly like those in the 1600 shot, all else being equal.
Offline
User avatar

Robert Niessner

  • Posts: 5025
  • Joined: Thu Feb 21, 2013 9:51 am
  • Location: Graz, Austria

Re: High ISO = more highlight detail?

PostWed Sep 07, 2016 5:52 pm

PaulDelVecchio wrote:
Yeah it's confusing in a way. Here it is simply:

Let's say you exposed skin at 45-50 IRE as a base and you're at 800 ISO. If you then bump it up to 1600, your skin tones move up to maybe 55-65 IRE so you'd have to stop down to bring them back to 45-50 IRE thus protecting the highlights.


Paul, the only confusing thing here in this thread with your example is that you haven't ever stated if you changed the amount of incoming light between the two shots or not.

EDIT: Just saw that you have edited your example description. So it is now clear that you changed the aperture and the result comes with no surprise.
Saying "Thx for help!" is not a crime.
--------------------------------
Robert Niessner
LAUFBILDkommission
Graz / Austria
--------------------------------
Blackmagic Camera Blog (German):
http://laufbildkommission.wordpress.com

Read the blog in English via Google Translate:
http://tinyurl.com/pjf6a3m
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostFri Sep 09, 2016 12:04 am

John Paines wrote:But that's just it.... I don't think you *are* protecting the highlights, or actually gaining anything, by shooting at 1600 and bringing the levels down in post.

I'd wager that you can make the highlights of the 800 shot looks exactly like those in the 1600 shot, all else being equal.

You physically can't do that. It'd be like breaking physics. What's happening is that he is exposing for a higher ISO which means less light hits the sensor. Therefore, when the metadata is read by Resolve and middle gray is set to 0.18, there will be extra information at the top since all the data was recorded at lesser values.

If you understand the concept of exposing to the right or left, then this isn't too far off. Instead of just ETTR until you see zebras, the ISO adjustments are ETTR/ETTL in 1 stop increments.

ISO 200: ETTR 2EV
ISO 400: ETTR 1EV
ISO 800: Neutral
ISO 1600: ETTL 1EV
Offline
User avatar

PaulDelVecchio

  • Posts: 799
  • Joined: Wed Aug 22, 2012 5:33 am
  • Location: NY

Re: High ISO = more highlight detail?

PostFri Sep 09, 2016 12:36 am

Robert Niessner wrote:
PaulDelVecchio wrote:
Yeah it's confusing in a way. Here it is simply:

Let's say you exposed skin at 45-50 IRE as a base and you're at 800 ISO. If you then bump it up to 1600, your skin tones move up to maybe 55-65 IRE so you'd have to stop down to bring them back to 45-50 IRE thus protecting the highlights.


Paul, the only confusing thing here in this thread with your example is that you haven't ever stated if you changed the amount of incoming light between the two shots or not.

EDIT: Just saw that you have edited your example description. So it is now clear that you changed the aperture and the result comes with no surprise.


Right. Sorry. Typo... and it seems to have caused a lot of confusion. Apologies.

But... it's still an important detail that some don't get. I just had a shoot yesterday where I used 1600 and as long as things are properly exposed (if you want to put a false color or IRE value on it - green or pink) then the noise shouldn't be an issue. Sure it's noisier, but it's MORE THAN usable.

Grade:
Image

Grade with Grain/Film:
Image

Raw:
Image
Paul Del Vecchio - Director/Producer
http://www.pauldv.net
http://www.youtube.com/user/pdelvecchio814
http://www.facebook.com/pauldv
http://instagram.com/pdelv
Twitter: @pauldv
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostFri Sep 09, 2016 12:54 am

Steven Abrams wrote:
Eddie Barton wrote:Also, I think it's important to point out that the log curve never changes! It's the linear data that is shifted before the log curve is applied. Just thought I'd add that because I see this common misconception. The only camera whose log curve changes with EI that I know of is the Alexa, the rest (Sony, RED, Panasonic, Canon, BMD, Digital Bolex) use static, unchanging curves.

Are you positive? For the 4.6K raws when you change ISO in resolve the log curve seems to change. I was told the others are meant to as well but for some reason resolve didn't add support. Its strange since its the same company but i think resolve team is in different country to the camera team. You can verify by shooting prores film at different ISO and a single raw, changing the ISO for the RAW in resolve doesn't match the prores curves so seems like resolve isn't matching what the camera does. But for the 4.6K it does seem to match the different isos curves of prores.

Could you be a little more specific? How does it seem to change? Do you have any example ProRes files you can share for analysis?
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostFri Sep 09, 2016 12:55 am

PaulDelVecchio wrote:I just had a shoot yesterday where I used 1600 and as long as things are properly exposed (if you want to put a false color or IRE value on it - green or pink) then the noise shouldn't be an issue. Sure it's noisier, but it's MORE THAN usable.

Grade:
Image

Grade with Grain/Film:
Image

Raw:
Image

A damn nice image Paul
Offline

John Paines

  • Posts: 5828
  • Joined: Tue Jul 28, 2015 4:04 pm

Re: High ISO = more highlight detail?

PostFri Sep 09, 2016 1:12 am

Eddie Barton wrote:You physically can't do that.


I hesitate to interject here -- the late, drunk party guest, who's talking about a different TV show -- not having read the entire thread. However, there is received wisdom on shooting raw with which these claims are apparently in conflict, unless I'm misreading what's being said.

In ProRes, I assume there's no disagreement that best results are achieved by using the native ISO, for reasons exhaustively explained in the past.

In RAW, whether you're metering for 1600 or 800 obviously matters, but the camera's ISO setting does not -- it's simply metadata. So what exactly is the claim being made here? That camera ISO settings in RAW are altering the curve, and that a camera setting of 1600 is gaining highlight headroom?
Offline

Eddie Barton

  • Posts: 31
  • Joined: Fri Aug 24, 2012 3:48 am
  • Location: Fairfield, CA, USA

Re: High ISO = more highlight detail?

PostFri Sep 09, 2016 1:24 am

John Paines wrote:
Eddie Barton wrote:You physically can't do that.


I hesitate to interject here -- the late, drunk party guest, who's talking about a different TV show -- not having read the entire thread. However, there is received wisdom on shooting raw with which these claims are apparently in conflict, unless I'm misreading what's being said.

In ProRes, I assume there's no disagreement that best results are achieved by using the native ISO, for reasons exhaustively explained in the past.

In RAW, whether you're metering for 1600 or 800 obviously matters, but the camera's ISO setting does not -- it's simply metadata. So what exactly is the claim being made here? That exposing to the right reduces noise? Okay, but that isn't exactly news.

What *is* being said? That ISO settings in RAW are altering the curve, and that a camera setting of 1600 is gaining headlight headroom?

The claim I've made is just that the exposure at a certain ISO shifts the middle gray point up and down the dynamic range because the sensor doesn't change analog ISO. So you're right, it's not news at all. I'm just showing what is happening when you expose for a certain ISO with the graphs. It's not the metadata that gives the headroom. The metadata just tells the program to digitally push/pull the recorded data to make middle gray = 0.18, i.e. make the exposure look correct; so it physically can't change how dynamic range is distributed. And you pointed that out.

I've been saying that the ISO in camera is basically calibrated ETTR/ETTL in increments of 1EV. That's literally all it is. It just makes ETTR/ETTL easy for us
Previous

Return to Cinematography

Who is online

Users browsing this forum: No registered users and 136 guests