Gen 4 vs Gen 5 Color Science (Question)

I hope after Camera Update 7.3 is released and DaVinci Resolve Studio 17 is in general release, we can put together a white paper describing best practices for Colour Science Gen 5 and Resolve YRGB, Resolve Colour Managed, and ACES for both delivering to HDR and SDR. I think I have an idea what is required and I’m comfortable with that, but it would be much better to have the best practices for various scenarios defined by BMD. The manuals don’t really seem to discuss best practices for web (computers/mobile devices), television (SDR/HDR), and Cinema deliverables.

It would seem Captain Hook is the best person to provide input or oversee the task. This thread has been extremely helpful, but a white paper seems to me the best place to put this for ease of the readership. Sorry to put a burden on anyone, but BMD is always so good at clarifying for their users what other vendors leave to the anointed and hallowed cognoscenti. In the meantime, like you, I’m writing notes to myself to get a handle on this.

I second Tom’s recommendation about using Resolve 17 beta 9. I love working with it and everything is there except the release of Camera Update 7.3 and Resolve 17 presumably in the next month.

This question CaptainHook:

Would setting Input to "bypass" for P6K-Pro, and Input to Gen-4 for P6K work to match in Prores, P6K-Pro/Gen5 to P6K/Gen4 without resorting to grading?

But back to my question, without resorting to grading, how can you match in Prores, P6K-Gen4 and P6K-Pro-Gen5 without any input transformations for Gen-5? Would setting Input to bypass for P6K-Pro, and Input to Gen-4 for P6K work?

Bypass won't work with Prores; at best, it'll just leave the footage as log, but there's no way it will guess "right". So you're back to normalizing it manually, if RCM doesn't yet support a gen5 normalization.

Video, Ext Video works fine, exactly as Captain Hook has described, and the discussion with Joe12South. Ext video is just BMD film gamut with a raw tab custom preset for gamma and saturation. Video is straight 709, no custom alterations available.

The discussion between Joe and Hook actually confirmed what I've seen so far on this initial gen5 version. Joe did point out the hard clipping with "video" and extended video isn't actually in a correct color space. A bit puzzling, this.

If I'm understanding correctly?

Camera Raw - DECODE OPTIONS

IF
Color Science: Gen 5
AND
Color Space:Blackmagic Design

Then:

(NON-REC709 Primaries - Custom BMD Profile)
--> BMD FILM
--> BMD EXTENDED VIDEO*
*Applies non-destructive contrast and saturation that can be reversed in the gamma controls to BMD FILM

(REC709 Primaries - Standardized (Generic) Profile**)
-> BMD DESIGN VIDEO*
*Gamma Controls Disabled
**UNIQUE to GEN 5

[CaptainHook]

RCM with RAW codecs (BRAW, DNG, ARRIRAW, etc) doesn't really care about colour science version as its generally irrelevant*. Same with ACES. For RCM Resolve can ask the RAW SDK to decode into Linear XYZ and then it can do any transform it likes from there - in the case of DNG the information on how to decode as Linear XYZ is in each DNG frame. For ACES its similar and the RAW SDK can decode in Linear ACES AP0 and then go through the rest of the pipeline. This is why for Blackmagic RAW in RCM or ACES those gamma/gamut options get greyed out - because Resolve forces the options it needs and gets the output it wants from the SDK so it can do its own transforms. I know there was a question on input space in regards to ProRes files but just so people know for RAW clips this setting does nothing for the previous reasons (Resolve handles it).

@Ryan - yes your summary is correct.

@Tom - I can't comment on when the update with Gen 5 support will come but hopefully soon. Until then If you have a P6K Pro with Gen 5 and need to match ProRes Gen 4 from another camera and you're working with a Resolve version that doesn't have Gen 5 RCM/CST support, your only practical option would be to shoot BRAW on at least one of the cameras. If Resolve is part of the workflow I'm not sure why one would pick ProRes honestly since BRAW clips can be smaller if needed and perform just as well on desktop (transcoding is fast too if necessary) but I realise everyone has their reasons for how they work.

@John - your experience sounds odd. If you could share some individual BRAW frames (exportable from the RAW tab) and some screenshots of your settings and the output you see we could try figure out if something is unexpected. R17 went GM today so I would try that first - I'm not sure when you say clip if you mean "luma" or gamut but if it's colour (not surprising with 709 primaries) then definitely try the gamut compression toggle if you haven't.


*The only time a colour science version may matter for RCM/ACES is if the "fit" from sensor response to XYZ is modified but that is rare (at least in our case so far which is why you see no difference in RCM with Gen 4/5).

@Tom - I can't comment on when the update with Gen 5 support will come but hopefully soon. Until then If you have a P6K Pro with Gen 5 and need to match ProRes Gen 4 from another camera and you're working with a Resolve version that doesn't have Gen 5 RCM/CST support, your only practical option would be to shoot BRAW on at least one of the cameras. If Resolve is part of the workflow I'm not sure why one would pick ProRes honestly since BRAW clips can be smaller if needed and perform just as well on desktop (transcoding is fast too if necessary) but I realise everyone has their reasons for how they work.

Thank You! (I have some influence in the decision, but the other two collaborators are using BMD Pocket cameras with Adobe PP. There ought to be a law against it...)

I usually work in Resolve YRGB so I see all the raw options. Good news about Resolve 17!

Hook, since Gen 5 is coming natively to the OG Pocket 6K could that mean that Gen 5 could come natively to the UMP G2 also? Would love to skip the step of having to tell Resolve to put my G2 into Gen 5 color science to take advantage of it. It really is awesome and enhances the G2. But would love for it to be native.

CH, I am with Tim on this and also hope that BMD can be convinced to provide Gen 5 color science in the UMP G2 in upcoming firmware. And yes, that will be an awesome G2 enhancement.

Does anyone know how much DR in terms of percentage does Gen 5 has over Gen 4?

I usually work in Resolve YRGB so I see all the raw options. Good news about Resolve 17!

Wrong page. That's the Camera Raw palette on the Color Page. Also not talking about Raw options, talking about Gen-5 color.

Go to project settings -> Color Management -> Color Science -> DaVinci YRGB Color Managed
Resolve Color Management Preset -> Custom
Use Separate Color Space and Gamma = box checked
Input Color Space/Timeline Color Space/Output Color Space; those are the Color Space Transforms
Click the dropdown box for each, see BMD Gen-1/Gen-3/Gen-4/Film/Video/Wide Gamut/4K/4.6K/6K
Do you see anything anywhere in those drop downs that say "Gen-5?" I don't. I'm on Windows 10, Resolve Studio 17.0.00039 shipping version, not Public Beta.

Go to the Media Page. Right click on one of your BRAW files. Choose Input Color Space. Do you see anything in the long list that says Gen-5? It's not there.

I know how to use Gen-5, no problem. What I'm saying is with Prores (not BRAW), that you don't have a color space transform to match the P6K-Pro color space to the P6K, without you having to match them with the grading tools. CaptainHook has acknowledged this. If your version of Resolve Studio has Gen-5 Color space transforms (CST) that would be useful to know, mine does not. Gen-5 color science *is* found in the Camera Raw color palette of the color page, but you can't use transform Prores there because it's not RAW.

Right now I don't have a P6K-Pro which is said to ship with Gen-5 color. If we were talking about BRAW we would not be having this conversation, but others I collaborate with do shoot in Prores. So if one camera has Gen-5 Prores (P6K-Pro) and another camera has Gen-4 Prores (P6K), then when the Prores footage is injested in Resolve it needs to be interpreted because the gamma curves and color gamuts are not the same, at least they do if you want a color space transform to do the work of matching the cameras.

And BMD seems a bit vague about whether the P6K-Pro also includes Gen-4 science, or whether the firmware update for P6K will lose Gen-4 science.

If all that is understood, please know I am otherwise 100% delighted with Gen-5 color science for my P6K as applied to BRAW in Resolve 17. More than satisfied, best Resolve ever.

But I just want to know. It's my opinion that between the two models, color science between P6K and P6K-Pro should be easily interchangeable with Prores as they it is now with BRAW.

Does anyone know how much DR in terms of percentage does Gen 5 has over Gen 4?

nothing - the changes are in the middle (Simply expressed)

And BMD seems a bit vague about whether the P6K-Pro also includes Gen-4 science, or whether the firmware update for P6K will lose Gen-4 science.

To put it mildly, it seems highly unlikely that Gen4 and Gen5 will exist on the same camera at the same time. In addition to being a real UI complication, BMD likes to nudge us all to move on. [cough]BRAW[cough].

But, also, in that spirit, it seems highly likely that RCM will support Gen5 sooner rather than later.

And BMD seems a bit vague about whether the P6K-Pro also includes Gen-4 science, or whether the firmware update for P6K will lose Gen-4 science.

To put it mildly, it seems highly unlikely that Gen4 and Gen5 will exist on the same camera at the same time. In addition to being a real UI complication, BMD likes to nudge us all to move on. [cough]BRAW[cough].

But, also, in that spirit, it seems highly likely that RCM will support Gen5 sooner rather than later.

If you take a look at the BRAW clips from the Ursa 12K, you still have the option to swap back to GEN 4 in Camera Raw. I imagine once firmware is updated to GEN 5 across the Pocket cams you may still have some ability to go backwards in post if capturing BRAW. Maybe that can be confirmed?

I hope after Camera Update 7.3 is released and DaVinci Resolve Studio 17 is in general release, we can put together a white paper describing best practices for Colour Science Gen 5 and Resolve YRGB, Resolve Colour Managed, and ACES for both delivering to HDR and SDR. I think I have an idea what is required and I’m comfortable with that, but it would be much better to have the best practices for various scenarios defined by BMD. The manuals don’t really seem to discuss best practices for web (computers/mobile devices), television (SDR/HDR), and Cinema deliverables.

It would seem Captain Hook is the best person to provide input or oversee the task. This thread has been extremely helpful, but a white paper seems to me the best place to put this for ease of the readership. Sorry to put a burden on anyone, but BMD is always so good at clarifying for their users what other vendors leave to the anointed and hallowed cognoscenti. In the meantime, like you, I’m writing notes to myself to get a handle on this.

Many of us would absolutely love such a White Paper of video clarifying this workflow. I've been reading these 2 threads and am totally confused. Appreciate all the effort of users such as JoeSouth and the many others as well as BM's Captain Hook, but it seems like this subject is too complex to learn in forum posts. But I surely thank these knowledgeable folks for all their time helping.

If you take a look at the BRAW clips from the Ursa 12K, you still have the option to swap back to GEN 4 in Camera Raw. I imagine once firmware is updated to GEN 5 across the Pocket cams you may still have some ability to go backwards in post if capturing BRAW. Maybe that can be confirmed?

Correct.

If you take a look at the BRAW clips from the Ursa 12K, you still have the option to swap back to GEN 4 in Camera Raw. I imagine once firmware is updated to GEN 5 across the Pocket cams you may still have some ability to go backwards in post if capturing BRAW. Maybe that can be confirmed?

BRAW in Resolve, sure. But in-camera ProRes recording? I doubt it.

Joe gets it. I think some are only looking at this as not a problem because they work singularly, probably don't even use color space transforms. The beauty of CST is that it works with SLog2/3, VLog, all the logs and gammas from the other cams, so when you put them together in a multicam project, the starting point for matching them becomes much easier. So it's just ironic, that of all the cams you can do this with, the ones you can't would be the ones in Gen5 Prores like P6K-Pro, at least not at this time. I'm sure it will change but with the shipping version of Resolve Studio 17.0 now in the field, would think it would have been included before now with the P6K-Pro release just around the corner. All the focus has been on Gen5 BRAW which is great, but if a BMD camera is going to only have Gen5 color science, you'd darn well better be shooting RAW as CaptainHook said, and hope your editor that is using color space transforms to match with the other cameras shooting on the job is himself using Resolve or you could find you and your camera excluded from the project. Okay, that's alarmist sounding, not my intention, but still.

I pretty much had that problem with my ursa mini pro and pocket 4K all having colour science 4 and my mini 4.6 being stuck with science 3, due to media costs I have to use prores on the mini

Although I haven’t had much of a play with colour science 5 (which I love the video mode of btw, I much prefer the colours), I created a lut that matched the cs3 to the cs4 in resolve with some trial and error in the different curves pallettes and some charts.

Overall, I choose Extended Video as a starting point with my grade. I usually customize to bring back Highlights alway the way down.

I love Color Science 5. It's great! I see a difference. Challenge for me has been using Resolve 17 HDR Palette because it's a new learning curve. But I'm getting there.

Hook & Team, please bring CS5 natively to the UMPG2. Would make life a bit easier. But otherwise, keep on with the amazing improvements. I hope to see dual ISO in a UMP12K type of camera in the future.

@John - your experience sounds odd. If you could share some individual BRAW frames (exportable from the RAW tab) and some screenshots of your settings and the output you see we could try figure out if something is unexpected. R17 went GM today so I would try that first - I'm not sure when you say clip if you mean "luma" or gamut but if it's colour (not surprising with 709 primaries) then definitely try the gamut compression toggle if you haven't.

I found the (or at least an) explanation -- it's seen in 16.3, but can't speak to 17. With a raw tab setting of CS5, rec. 709 and BM Design Video, [unclipped] highlights on some shots blow out and are unrecoverable unless the first adjustment is the raw tab exposure control. Go straight to primary wheels, and it's a lost cause. You can't get them back -- as if they're just stuck.

Here's the Parade scope, prior to any raw tab exposure reduction:

clipped.JPG (27.57 KiB) Viewed 3684 times

Reduce the raw tab exposure value, and they come back. But there's no way to recover them elsewhere.

[CaptainHook]

And BMD seems a bit vague about whether the P6K-Pro also includes Gen-4 science, or whether the firmware update for P6K will lose Gen-4 science.

I thought my description of your options above helped clear up the ambiguity but no you can't switch colour science versions on camera so if its Gen 5 on camera, that's it.

Does anyone know how much DR in terms of percentage does Gen 5 has over Gen 4?

nothing - the changes are in the middle (Simply expressed)

The changes allow for more DR above middle grey at all current native ISOs/cameras we have. BMDFilm Gen 5 allows for around ~10.3 stops above middle grey so the DR is limited by each camera at this point. Middle grey is the same at 38.4% so you can check the existing DR charts in the manuals to see what the DR is for the Gen 4 curves as they are optimized for the DR available from each camera at a given ISO.

I created a lut that matched the cs3 to the cs4 in resolve with some trial and error in the different curves pallettes and some charts.

The difficulty with that is that Gen 3 is white balance dependant (essentially two different primaries or gamuts interpolated for the given CCT - one at 6500K and the other 2856K and this approach goes back to the original colour science in BMCC etc), so it requires a different transform for every WB setting. Being practical, making a 'daylight' and 'tungsten' version could be enough to be useful in practice or even just a compromise of the two in a pinch, or if you happen to know you'll always be working with daylight-ish etc.

I found the (or at least an) explanation -- it's seen in 16.3, but can't speak to 17. With a raw tab setting of CS5, rec. 709 and BM Design Video, [unclipped] highlights on some shots blow out and are unrecoverable unless the first adjustment is the raw tab exposure control. Go straight to primary wheels, and it's a lost cause. You can't get them back -- as if they're just stuck.

Can you share a BRAW frame for that scope? What camera is this? It seems like maybe the ISO was set 'too high' for the scene which is why bringing down exposure (or the ISO) would bring it back into range. With 9 stops over middle grey available on the Gen 5 Video curve 3 stops of that is extremely flat as part of the highlight rolloff - it really looks here like the ISO is pushed up really high into the Video curve so that the top 3 stops have been compressed like that, you could prob recover with an aggressive curve but yes ideally just lower the ISO or exposure from the RAW tab. A braw frame would be super helpful to understand for sure though.

[CaptainHook]

I found the (or at least an) explanation -- it's seen in 16.3, but can't speak to 17. With a raw tab setting of CS5, rec. 709 and BM Design Video, [unclipped] highlights on some shots blow out and are unrecoverable unless the first adjustment is the raw tab exposure control. Go straight to primary wheels, and it's a lost cause. You can't get them back -- as if they're just stuck.

Can you share a BRAW frame for that scope? What camera is this? It seems like maybe the ISO was set 'too high' for the scene which is why bringing down exposure (or the ISO) would bring it back into range. With 9 stops over middle grey available on the Gen 5 Video curve 3 stops of that is extremely flat as part of the highlight rolloff - it really looks here like the ISO is pushed up really high into the Video curve so that the top 3 stops have been compressed like that, you could prob recover with an aggressive curve but yes ideally just lower the ISO or exposure from the RAW tab. A braw frame would be super helpful to understand for sure though.

Actually, scratch that. Gen 5 Video was first modified for R17 (BRAW 2.0 Beta 4) - R16.3 Video should be the same between Gen 4 and Gen 5 as it was before we had finished the changes. You should see no difference in BMD Video if you switch between Gen 4 and Gen 5 in that case and I'm not actually sure 16.3 is available on the website anymore as it was an interim release before R17?

Can you share a BRAW frame for that scope? What camera is this? It seems like maybe the ISO was set 'too high' for the scene which is why bringing down exposure (or the ISO) would bring it back into range. With 9 stops over middle grey available on the Gen 5 Video curve 3 stops of that is extremely flat as part of the highlight rolloff - it really looks here like the ISO is pushed up really high into the Video curve so that the top 3 stops have been compressed like that, you could prob recover with an aggressive curve but yes ideally just lower the ISO or exposure from the RAW tab. A braw frame would be super helpful to understand for sure though.

I'll PM you with a dropbox link to the braw frame. It's the BMPCC 4K, daytime, ISO 400 and raw tab 400.

There's definitely a difference between a raw tab setting of Gen4 and Gen5 in this 16.3 beta release. It's always visible on the vectorscope with the rec.709/video setting, and obvious to the eye with BMD Design/BMD Film settings. But note: I see now that the issue above occurs with either Gen 4 and Gen 5. A few versions of 16.3 are currently available on the website; I'm stuck with it for a time because of a pre-Ampere GPU, and current market conditions....

I wouldn't have commented on this matter at all -- given the early version -- if not for seeing a previous remark about hard clipping with BMD video. Hope it's not a waste of time.

And BMD seems a bit vague about whether the P6K-Pro also includes Gen-4 science, or whether the firmware update for P6K will lose Gen-4 science.

I thought my description of your options above helped clear up the ambiguity but no you can't switch colour science versions on camera so if its Gen 5 on camera, that's it.

Yes. Thank you for your help!

[CaptainHook]

I'll PM you with a dropbox link to the braw frame. It's the BMPCC 4K, daytime, ISO 400 and raw tab 400.

There's definitely a difference between a raw tab setting of Gen4 and Gen5 in this 16.3 beta release. It's always visible on the vectorscope with the rec.709/video setting, and obvious to the eye with BMD Design/BMD Film settings. But note: I see now that the issue above occurs with either Gen 4 and Gen 5. A few versions of 16.3 are currently available on the website; I'm stuck with it for a time because of a pre-Ampere GPU, and current market conditions....

I wouldn't have commented on this matter at all -- given the early version -- if not for seeing a previous remark about hard clipping with BMD video. Hope it's not a waste of time.

Thanks for that, here's what I see just turning on BMD Video (and HR) with your file - I'll PM you the full screenshot so you can compare to your output

Gen 5 Video

Gen 4 Video


Just to confirm - you are setting colourspace to Blackmagic and gamma to BMD Video, then change colour science from Gen 4 to Gen 5 and see a difference between the two colour sciences? I wouldn't expect a change until the changes went into 2.0 beta 4 and Resolve 17 early November - which I posted about here: viewtopic.php?f=2&t=125084
(I go over some of the same things discussed here)

16.3 beta and it was done to add support for the 12K and BRAW 2.0 beta as R17 beta was not ready in time - but there were 3-4 subsequent releases of 16.2.x after that with no 12k or beta 2.0 support as it was moved in R17 betas once available. It was my understanding 16.3 was removed from the website and I had a quick look and couldn't see it (but maybe I missed it).

Its likely the issue you are seeing is specific to 16.3 I think.

We're a bit at cross purposes. The scope above was seen with a setting of *either* Gen 4 or Gen 5 (noted after the fact; whatever's happening, it's not unique to a Gen 5 setting), and raw settings of color space rec 709 and Gamma BMD Video.

BMD Design/BMD Film settings produced the expected normal result. Could be this odd result is unique to this orphaned beta.

John, I think it's in your RCM settings from the project settings, color management, color space & transforms,

or

your input color space, input gamma for the file in the media page.

Tom, there's no RCM in use here. Even if it were, input color space can't be set for braw and other raw formats, it's an automatic (and inalterable) when color management is in use.

And the color space/gamma raw tab choices described above wouldn't be possible in RCM 16, which fixes them at BMD Design/BMD Film. Can't speak to 17.

For Hook, just to conclude this wild goose chase, here's what's what:

last.JPG (116.2 KiB) Viewed 3490 times

Note the zero setting for exposure. Reducing it will bring back the highlights, but they're unrecoverable using any other control, like primary wheels. Ten to 1 this is a beta anomaly and a complete waste of time now. But since I put my foot in....

For Hook, just to conclude this wild goose chase, here's what's what:

John, it's not a phenomenon I'm seeing with my own clips in the latest 17 release I downloaded yesterday.

Your questions do reiterate for me the need to clarify the use cases (and what to expect from different combinations) for different color space and gamma choices now available.

Tom, there's no RCM in use here. Even if it were, input color space can't be set for braw and other raw formats, it's an automatic (and inalterable) when color management is in use.

And the color space/gamma raw tab choices described above wouldn't be possible in RCM 16, which fixes them at BMD Design/BMD Film. Can't speak to 17.

Okay, well I was only suggesting that because I was on 16.3 up until last week and saw something like that, hard clipping at 10b 768 on wfm, but I'm unable to reproduce it now on 17.0.00039.

[CaptainHook]

I would have to do a R16.3 beta install again to investigate but I think it's safe to say it seems like an issue in 16.3 so will probably leave it there unless someone finds something similar in >=17.0 GM since the 16.3 and 17.0 betas are no longer on the website.