Final Explanation of Gamma and Color Shift Problems

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

Vit Reiter

  • Posts: 553
  • Joined: Mon Sep 04, 2017 5:36 pm

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 15, 2020 11:15 pm

Dmitry Shijan wrote:In YRGB Color managed project i suggest you don't rely on auto detection and always manually set correct input color space for each file.
I never had a problem with auto detection in DaVinci.

Plus, I don't know what gamma input video has? I'm not talking about rushies from the camera where it's obvious, but about the master video that I get to edit or QC and I need to export it with the same gamma settings as the original. Most such videos are Rec.709 Gamma 2.4. However, if I export to Rec.709 Gamma 2.4, the video loses the transfer characteristic parameter and is unusable in the pipeline. I always have to deliver the video with the transfer characteristic parameter otherwise the client will return it to me. I solve it now in DaVinci by exporting the video to Rec.709 Scene, which will retain the transfer characteristic, but I have to make it looks like the original in unmanaged color spaces environment. It would be better if DaVinci exported properly.
DaVinci Resolve 16.2.3 Studio
Mac Pro 2013, AMD FirePro D700, 64GB RAM, macOS Mojave 10.14.6
iMac 2017, Radeon Pro 575, 24GB RAM, macOS High Sierra 10.13.6

Film Editor, Colorist, DIT, Encoding technician
linkedin.com/in/vít-reiter-film-editor
Offline

Vit Reiter

  • Posts: 553
  • Joined: Mon Sep 04, 2017 5:36 pm

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 15, 2020 11:25 pm

Andrew Kolakowski wrote:Lost my interest in discussion with you.
I respect that, but rather you have nothing to say. How to export Rec.709 Gamma 2.4 with transfer characteristics cannot be said because it is not possible from the current DaVinci.

Even so, thank you for the discussion. Thanks!
DaVinci Resolve 16.2.3 Studio
Mac Pro 2013, AMD FirePro D700, 64GB RAM, macOS Mojave 10.14.6
iMac 2017, Radeon Pro 575, 24GB RAM, macOS High Sierra 10.13.6

Film Editor, Colorist, DIT, Encoding technician
linkedin.com/in/vít-reiter-film-editor
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Mar 16, 2020 10:28 am

Resolve already does it in only and correct way based on current possibilities. There is no other way- not sure what is so hard to understand here :D
Offline

PalmerWoodrow

  • Posts: 193
  • Joined: Tue Apr 09, 2013 10:22 am

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Mar 20, 2020 4:39 am

Here's some info for everyone: QuickTime Player uses incorrect math for gamma. That's why its images are washed out compared to correct displays in other applications. I believe Safari duplicates the incorrect math in order to save face, and FCP probably does too (although FCP 7 showed different gamma in its viewer when paused vs. playing back).

QuickTime has always suffered from profound gamma problems. From one codec to the next, and one application to the next, you never know what you're going to get. Apple realized this (translation: was pilloried for this by the higher-end production community) in the mid-to-late 2000s and doesn't seem to have learned its lesson.

So, when checking gamma, don't use Apple applications.
iMac (Retina 5K, 27-inch, Late 2014)
4 GHz Intel Core i7
32 GB 1600 MHz DDR3
AMD Radeon R9 M295X 4096 MB
Areca Thunderbolt RAID
Offline

Wayne Steven

  • Posts: 2908
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth
  • Warnings: 1

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Mar 20, 2020 6:09 am

Somebody mentioned something like this, that they were a consultant or something hired in.
Last edited by Wayne Steven on Fri Mar 20, 2020 11:44 am, edited 1 time in total.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Mar 20, 2020 9:56 am

PalmerWoodrow wrote:Here's some info for everyone: QuickTime Player uses incorrect math for gamma. That's why its images are washed out compared to correct displays in other applications. I believe Safari duplicates the incorrect math in order to save face, and FCP probably does too (although FCP 7 showed different gamma in its viewer when paused vs. playing back).

QuickTime has always suffered from profound gamma problems. From one codec to the next, and one application to the next, you never know what you're going to get. Apple realized this (translation: was pilloried for this by the higher-end production community) in the mid-to-late 2000s and doesn't seem to have learned its lesson.

So, when checking gamma, don't use Apple applications.


Sorry, but this is somehow "misleading" information.
Apple's color engine (which applies to QT X, Safari etc.) is not using wrong gamma. It use about 1.96 gamma which comes from original Rec.709 spec. For this preset it's correct gamma approximation.
Problem is that files should not be tagged with Rec.709 (1-1-1), but correct standards for todays production specs.
With current, outdated tagging presets correct way is what Resolve and Baselight does, but this is quickly lost in wildness of other apps.

You are spreading more myths than info based on facts (same as never existed/understood QT gammas shift).
Offline
User avatar

waltervolpatto

  • Posts: 7911
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Mar 20, 2020 6:03 pm

Andrew, but now the original rec709 is deprecated by to newer BT1886 that specify a gamma of 2.4 for the display... in a sense QT is wrong...
SuperServer 5039AD-I
C9X299-PGF - DDR4-2400 16x4 GB
i9-7920xCPU 12c 2.90GHz Water cooled
2x 1080ti DeckLink Studio 4K (11.5.1)
W10-1903 - BMR St. 16.2.2.012
nvidia: 441.66 studio
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Mar 20, 2020 9:11 pm

So introduce and set BT.1886 tag and all will be good. Why are you expecting old tag to use new value? It's reserved tag for original Rec.709 spec.
You can't change old preset as this is not they way how you do it with standards.
We don't have Rec.709 updated with new eg. HLG curve do we?
Offline
User avatar

waltervolpatto

  • Posts: 7911
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Mar 21, 2020 1:14 am

Will you suggest a new Bt1886/2.4 tag?

Because it's basically rec709/2.4.
SuperServer 5039AD-I
C9X299-PGF - DDR4-2400 16x4 GB
i9-7920xCPU 12c 2.90GHz Water cooled
2x 1080ti DeckLink Studio 4K (11.5.1)
W10-1903 - BMR St. 16.2.2.012
nvidia: 441.66 studio
Offline

Wayne Steven

  • Posts: 2908
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth
  • Warnings: 1

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Mar 21, 2020 4:29 am

I read that the default now without the gamma for 709 bring specified should be 2.4 because of 1886.

My previous post on doing a common instructions, tutorials, formulas and code base is the way to introduce broad comparability in apps snd players.

You are better off writing to the standards organisation, and each tool developer to resolve this in the way I suggested before, than endlessly throwing your hands in the air. A lot of solutions suggested here are gamut limited.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Mar 21, 2020 3:20 pm

waltervolpatto wrote:Will you suggest a new Bt1886/2.4 tag?

Because it's basically rec709/2.4.


Resolve already tags files as Rec.709 with 2.4 gamma, but this is not preserved/supported by about anything (except Baselight) due to apps not fully respecting QT spec.
It's also specific to MOV headers. h264/5/VP9 have different structure (they don't support custom gamma tag).
We want new tag options for h264/5/VP9, so files can be described properly.

Again- why do you try to call something new with proper existing spec using old one reserved for something already exiting and different?
It's "basically" Rec.709 with 2.4 gamma- is it? Can be Rec.709 2.2 as well (still present a lot), so how do you know which is which?
Whole confusion comes exactly from such an approach.
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Mar 21, 2020 3:22 pm

Wayne Steven wrote:I read that the default now without the gamma for 709 bring specified should be 2.4 because of 1886.


Maybe in Catalina, not in any prior OSX as far as I can tell.
I will check Catalina.
Offline

iNikitaPOD17

  • Posts: 10
  • Joined: Sun Mar 08, 2020 8:09 am
  • Real Name: Nikita Podobedov

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Mar 21, 2020 5:32 pm

Cosmin Hodiș-Mîndraș wrote:It seems that every "QuickTime gamma bug" is experienced by people expecting QuickTime player and YouTube to look like Resolve's color viewer. Those producing for broadcast are always concerned about calibrated Rec709 professional monitors, fed by UltraStudio or DeckLink cards, and care little for computer displays. Fair enough. The thing is, I really want my YouTube uploads to look as close to Resolve as possible, even if some of my productions are intended for broadcast.

So I made a few tests, exporting QuickTime files from Resolve using several timeline color spaces. The result, as shown by macOS finder, are:
  • Rec.709 Gamma 2.4 - Color profile: (1-2-1)
  • Rec.709 Gamma 2.2 - Color profile: (1-4-1)
  • Rec.709 (Scene) - Color profile: HD(1-1-1)
  • sRGB - (1-13-1)
(What 1-1-1 or 1-2-1 mean is explained in the first post of this topic).




Any of those files, once exported from Resolve, looked different in QuickTime. Once uploaded to YouTube, those files looked different as well, low contrast and desaturated. I downloaded them from YouTube and every file had the same color profile: HD (1-1-1). The same happened when I compressed the files using HandBrake, the color profile was changed to HD (1-1-1) as well.

What's quite clear at this point is the fact that YouTube ignores the source color profile, assuming 1-1-1. So, as a consequence, I have to Cosmin Hodiș-Mîndraș.709 (Scene) in Resolve (because it exports 1-1-1 profiles), and color grade for it. But how to set Resolve to look like QuickTime Player? Enabling "Use Mac Display Color Profiles for Viewers" in DaVinci Resolve helped a bit, but not a lot, some shadows were crushed and definitely saturation looked different.

I then thought about creating a display LUT for Resolve, which mimics the QuickTime and YouTube output:
  • I created a 1728x1728px 8-bit HALD image using ImageMagick:
    Code: Select all
    convert hald:12 -depth 8 hald12_8bit.tif
  • I imported the image in Resolve, setting the timeline color space to Rec.709 (Scene) and frame size to 1728x1728px;
  • I rendered the movie using ProRes 4444;
  • I opened the rendered movie in QuickTime, and made a screenshot (quite tricky, because I had to display the movie unscaled, and macOS Catalina shows no options on "Option+Scaled in Display Preferences for the built-in display; luckily I have an external LG UltraFine display that I could put in an unscaled mode - 5120x2880px);
  • I cropped the screenshot and opened it in Lattice:
    Image
  • I resized the LUT to 21:
    Image
    Image
  • ... and I exported a .cube LUT.

After loading the LUT in 3D Color Viewer Lookup Table, QuickTime Player and Resolve viewer screenshots look identical, so I can confidently grade my timeline for YouTube, knowing that what I see is what I get.

But how about shots graded for broadcast, in Rec.709 2.4 timelines? Well, if broadcast is my main delivery and YouTube is second, I reversed the LUT in Lattice for a quick and dirty final "YouTube touch", essentially adding the "missing" saturation and contrast in one node just for YouTube delivery, on top of my main broadcast grade, temporarily switching the project to Rec.709 (Scene).

I'm not saying that this is a viable workflow for anybody. I'm quite sure that it works only on Apple's P3 displays; it's just an experiment, maybe a stupid thing to do, but the results are, for the first time (on my Macs) and at least for me, well, predictable again.

If anyone is interested, here is a link to the LUTs:
https://www.dropbox.com/s/0sux7nydfyk721w/Rec709_Scene_to_Mac.zip?dl=0



Dear colleagues, help, I want to make a LUT as in this manual. I do everything the same way and get to the stage when I do a screen shot in QT and can't figure out what to do in lattice. I get a straight gamma (linear) not as in the screenshot of the author.
Offline

Gerhard Riesenhuber

  • Posts: 361
  • Joined: Fri Aug 24, 2012 8:32 am

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 3:06 pm

In ACES (ODT set to Rec709) a H264 QT export flags with (1-2-1), where 2 is unspecified.

The automator "color profile change" doesn't work anymore under Catalina.
Also the BBC script (https://github.com/bbc/qtff-parameter-editor) is for ProRes. Only way I see is to re-encode with a colormanaged project with timeline set to Rec709.

How else to correct the wrong tags?
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 3:52 pm

Use 16.2 and set in export/advanced setting gamma to Rec.709. This sets 1-1-1 tag (and purely affects tagging itself).
Other way of fixing exiting files is hex editor. I may write up how to do it. Once you learn it will take you 30sec to do.
BBC tool is not for ProRes only. It's for MOV and also for PRoRes private headers. MOV headers are more important as most tools reads them first. You can fix them with it as well for any codec ( ProRes private headers fix is optional).
Offline

Gerhard Riesenhuber

  • Posts: 361
  • Joined: Fri Aug 24, 2012 8:32 am

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 4:03 pm

Andrew Kolakowski wrote:Use 16.2 and set in export/advanced setting gamma to Rec.709. This sets 1-1-1 tag (and purely affects tagging itself).
Other way of fixing exiting files is hex editor. I may write up how to do it. Once you learn it will take you 30sec to do.
BBC tool is not for ProRes only. It's for MOV and also for PRoRes private headers. MOV headers are more important as most tools reads them first. You can fix them with it as well for any codec ( ProRes private headers fix is optional).


Thank you Andrew, that helps!
Offline

Supermachoalpha

  • Posts: 29
  • Joined: Tue Oct 16, 2018 2:05 pm
  • Real Name: Nate Game

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 4:12 pm

I have an imac with Catalina. I noticed i still have a slight color shift when exporting in Resolve 16.2. What steps do I need to do exactly so that my final exported file in Quicktime looks exactly like the Davinci resolve viewer? Thanks in advance for all the research you guys have been doing.
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 4:16 pm

Wayne Steven wrote:I read that the default now without the gamma for 709 bring specified should be 2.4 because of 1886.


You read and I've tried and it's not true at all. Nothing changed in Catalina in regards to tagging engine.
Missing tag is not treated as 2.4 gamma at all. It's same gamma as Rec.709, but different gamut.
Last edited by Andrew Kolakowski on Sun Mar 22, 2020 4:27 pm, edited 2 times in total.
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 4:20 pm

Supermachoalpha wrote:I have an imac with Catalina. I noticed i still have a slight color shift when exporting in Resolve 16.2. What steps do I need to do exactly so that my final exported file in Quicktime looks exactly like the Davinci resolve viewer? Thanks in advance for all the research you guys have been doing.


Turn on "Use Mac Display color profiles for viewers" in settings. Preview should be the same for Rec.709/Rec.709 gamma and Rec.709+2.4 gamma based projects (not going to work for 2.2 gamma though).

I can also see tiny (it needs direct swap between previews to see) gamma difference for 2.4 gamma. Resolve seems to be jumping from identical to 99.9% identical preview depending on the versions.
Offline

Supermachoalpha

  • Posts: 29
  • Joined: Tue Oct 16, 2018 2:05 pm
  • Real Name: Nate Game

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 4:46 pm

Thanks, Andrew! I will give it a try.
Offline
User avatar

Dmitry Shijan

  • Posts: 1470
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 5:06 pm

Andrew Kolakowski wrote:
Supermachoalpha wrote:I have an imac with Catalina. I noticed i still have a slight color shift when exporting in Resolve 16.2. What steps do I need to do exactly so that my final exported file in Quicktime looks exactly like the Davinci resolve viewer? Thanks in advance for all the research you guys have been doing.


Turn on "Use Mac Display color profiles for viewers" in settings. Preview should be the same for Rec.709/Rec.709 gamma and Rec.709+2.4 gamma based projects (not going to work for 2.2 gamma though).

I can also see tiny (it needs direct swap between previews to see) gamma difference for 2.4 gamma. Resolve seems to be jumping from identical to 99.9% identical preview depending on the versions.


As described earlier it is not the same. It is strange but "Use Mac Display color profiles for viewers" In Resolve attempt to emulate macOS color management but result is always slightly different than in native macOS apps. It is very close but not 100% the same.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Wayne Steven

  • Posts: 2908
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth
  • Warnings: 1

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 5:35 pm

Andrew Kolakowski wrote:You read and I've tried and it's not true at all. Nothing changed in Catalina in regards to tagging engine.
Missing tag is not treated as 2.4 gamma at all. It's same gamma as Rec.709, but different gamut.



I only read that it's supposed to, doesn't mean it was implemented there (or in many places).

Dmitry has disappeared, but he was saying gamma was a display choice, rather than what is set.
Last edited by Wayne Steven on Mon Mar 23, 2020 11:08 am, edited 1 time in total.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 5:57 pm

OSX color engine reads tag and when it understands it uses it for conversion to screen profile. No tag (or unknown) means using default one which is some old one (it's somewhere on Apple QT spec sites). Nothing you can choose or have much control over.
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 5:58 pm

Dmitry Shijan wrote:
Andrew Kolakowski wrote:
Supermachoalpha wrote:I have an imac with Catalina. I noticed i still have a slight color shift when exporting in Resolve 16.2. What steps do I need to do exactly so that my final exported file in Quicktime looks exactly like the Davinci resolve viewer? Thanks in advance for all the research you guys have been doing.


Turn on "Use Mac Display color profiles for viewers" in settings. Preview should be the same for Rec.709/Rec.709 gamma and Rec.709+2.4 gamma based projects (not going to work for 2.2 gamma though).

I can also see tiny (it needs direct swap between previews to see) gamma difference for 2.4 gamma. Resolve seems to be jumping from identical to 99.9% identical preview depending on the versions.


As described earlier it is not the same. It is strange but "Use Mac Display color profiles for viewers" In Resolve attempt to emulate macOS color management but result is always slightly different than in native macOS apps. It is very close but not 100% the same.


Depending on Resolve version it's either 100% the same or 99.9%. BM keeps playing with it.
Offline
User avatar

Dmitry Shijan

  • Posts: 1470
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 6:06 pm

Andrew Kolakowski wrote:Depending on Resolve version it's either 100% the same or 99.9%. BM keeps playing with it.

To be honest i didn't check and compare in 16.2 just because i don't use that option at all.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Mar 22, 2020 6:26 pm

16.2 is 99.9% the same. 15.x if I remember well were 100% the same. At last some version was 100% the same. I remember this for sure.
Offline
User avatar

Christopher Dobey

  • Posts: 208
  • Joined: Fri Jul 18, 2014 5:58 pm
  • Location: San Francisco Bay Area, California

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Mar 24, 2020 9:02 pm

It will be the ISO and not Apple that will (or needs to) release a specific BT.1886 NCLC Transfer Function tag right?
BMPCC 6K
Resolve Studio 16.1.2 | macOS 10.15.3
Intel i9 9880HK 8-core | 64GB 2.4GHz DDR4
AMD 5500M 8GB VRAM | AMD WX9100 16GB VRAM eGPU
Decklink Mini Monitor 4K | LG OLED B7A 65" | Micro Panel | Dell UP2718Q
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Mar 24, 2020 9:32 pm

BT.1886 spec exists, but not sure who would be the correct body to add new tags for MOV as a format. Of course in OSX it would be Apple. There has been recently added flagging for HDR metadata in MOV. No idea who controlled/approved it. Here is small hint:
https://github.com/MediaArea/MediaInfoL ... 8cf3ada504
With other codecs h264/5 etc. it's probably down to ITU.
Offline
User avatar

Dmitry Shijan

  • Posts: 1470
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Mar 24, 2020 9:45 pm

It is all about backwards compatibility. Even if in future BT.1886 will became default by Mac, what should people do with all current and legacy and current content graded with gamma2.4 and tagged with Rec709 (1-1-1)? Re encode and re tag all existing videos on planet? It is just too late to change things again... The solution is simple - Apple just need to change the way how to read Rec709 (1-1-1) gamma. It should visually match Rec709 (1-1-1) look to to legacy non color managed gamma look.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Mar 24, 2020 9:51 pm

I doubt any serious body will do such a "fix". REc.709 flagged files can mean few different gammas. There is probably more 2.2 based files with Rec.709 tag than 2.4 based, so it's not a real solution (2.2 would be probably better choice in such a case).
With specs etc this is not the way how you do it.
Old content is "old" problem. It's not video which needs to change, just tag. Providers like Youtube/Vimeo etc. could re-flag all files if they wanted without a huge problem I assume.
Offline
User avatar

Dmitry Shijan

  • Posts: 1470
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Mar 24, 2020 10:09 pm

10 years or more Apple fight against the world by using gamma 1.8 for system color management. When they gave up and move system to gamma 2.2, it took them another 10 years to fix things and match too bright system color management to normal looking Photoshop-like color management. It was all about images with ICC profiles but not about video. So i guess in next 10-20 years we may see similar story with video color management variations.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline
User avatar

Christopher Dobey

  • Posts: 208
  • Joined: Fri Jul 18, 2014 5:58 pm
  • Location: San Francisco Bay Area, California

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Mar 26, 2020 1:47 am

Yep no re-encoding is needed, simply a BT.1886 Transfer function NCLC tag so it can be probably transformed to a P3-D65 sRGB screen (12-13-1) in macOS 10.15.4 [as we saw in the FilmLight video]. All BT.709 (1-1-1) footage can be left as is. If the BT.1886 tag ever did get standardized I'd hope YouTube would announce support for it like they did with HDR tagging in 2016
BMPCC 6K
Resolve Studio 16.1.2 | macOS 10.15.3
Intel i9 9880HK 8-core | 64GB 2.4GHz DDR4
AMD 5500M 8GB VRAM | AMD WX9100 16GB VRAM eGPU
Decklink Mini Monitor 4K | LG OLED B7A 65" | Micro Panel | Dell UP2718Q
Offline

Wayne Steven

  • Posts: 2908
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth
  • Warnings: 1

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Mar 26, 2020 3:21 am

Isn't there any absolute space mapping standard out there where the space could then be transformed remapped to the display. That would solve many issues. On the camera side it cuts out a space to match it's performance in frame, and maps values according to transformation parameters, which should result in smaller files. The post production process redoes this to preserve it's changes while perhaps still preserving the original space and allowing reversibility of changes, and then the deliverable has a transformation to the parameters it needs. The viewer/display then maps it to it's need.

The chain is for efficient preservation at small size.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline
User avatar

waltervolpatto

  • Posts: 7911
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Mar 26, 2020 4:53 am

Wayne Steven wrote:Isn't there any absolute space mapping standard out there where the space could then be transformed remapped to the display. That would solve many issues. On the camera side it cuts out a space to match it's performance in frame, and maps values according to transformation parameters, which should result in smaller files. The post production process redoes this to preserve it's changes while perhaps still preserving the original space and allowing reversibility of changes, and then the deliverable has a transformation to the parameters it needs. The viewer/display then maps it to it's need.

The chain is for efficient preservation at small size.

similar to XYZ for projectors...
SuperServer 5039AD-I
C9X299-PGF - DDR4-2400 16x4 GB
i9-7920xCPU 12c 2.90GHz Water cooled
2x 1080ti DeckLink Studio 4K (11.5.1)
W10-1903 - BMR St. 16.2.2.012
nvidia: 441.66 studio
Offline

macfawlty

  • Posts: 8
  • Joined: Wed Sep 12, 2018 4:38 pm
  • Real Name: David Fielding

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Mar 30, 2020 8:49 pm

Thanks so much for the depth of analysis on this gamma/color shift issue. Seeing a very significant shift between the gamma in my timeline vs. exported ProRes 422HQ was disconcerting to say the least. Most of the analysis went over my head, but these 2 sentences were all I needed to resolve it.

"If looking ahead, the quick solution for this problem - do NOT use gamma 2.4 in DaVinci Resolve Timeline project settings. If you work in YRGB project, always set Timeline gamma to Rec.709 (Rec.709 (Scene)".
Offline

Vit Reiter

  • Posts: 553
  • Joined: Mon Sep 04, 2017 5:36 pm

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Mar 31, 2020 12:01 am

macfawlty wrote:Thanks so much for the depth of analysis on this gamma/color shift issue. Seeing a very significant shift between the gamma in my timeline vs. exported ProRes 422HQ was disconcerting to say the least. Most of the analysis went over my head, but these 2 sentences were all I needed to resolve it.

"If looking ahead, the quick solution for this problem - do NOT use gamma 2.4 in DaVinci Resolve Timeline project settings. If you work in YRGB project, always set Timeline gamma to Rec.709 (Rec.709 (Scene)".
Oh no, this is a absolutely wrong. Is your video for YouTube or watching on computer? Then use gamma 2.4 and set Gamma tag to Rec.709 on Deliver page for unmanaged project. Color Space Tag is the same as project, this is Rec.709 Gamma 2.4. That's right.
DaVinci Resolve 16.2.3 Studio
Mac Pro 2013, AMD FirePro D700, 64GB RAM, macOS Mojave 10.14.6
iMac 2017, Radeon Pro 575, 24GB RAM, macOS High Sierra 10.13.6

Film Editor, Colorist, DIT, Encoding technician
linkedin.com/in/vít-reiter-film-editor
Offline
User avatar

Dmitry Shijan

  • Posts: 1470
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Mar 31, 2020 12:09 am

It depends of Resolve version:

In Resolve 16.2 You can set YRGB project timeline to Rec709 (Scene) and choose "Same as project" render tags. This will not produce gamma shift. (This emulates how Resolve 16.1 worked)

Or you set YRGB project timeline to gamma 2.4 (or any other you like) and choose Rec709 render tags. This will not produce gamma shift. (This emulates how Resolve 16 and earlier worked)
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Vit Reiter

  • Posts: 553
  • Joined: Mon Sep 04, 2017 5:36 pm

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Mar 31, 2020 12:28 am

Dmitry Shijan wrote:It depends of Resolve version:

In Resolve 16.2 You can set YRGB project timeline to Rec709 (Scene) and choose "Same as project" render tags. This will not produce gamma shift. (This emulates how Resolve 16.1 worked)

Or you set YRGB project timeline to gamma 2.4 (or any other you like) and choose Rec709 render tags. This will not produce gamma shift. (This emulates how Resolve 16 and earlier worked)
Dmitry, video processing other than Rec.709 Gamma 2.4 is not a good idea. (with some exceptions)
DaVinci Resolve 16.2.3 Studio
Mac Pro 2013, AMD FirePro D700, 64GB RAM, macOS Mojave 10.14.6
iMac 2017, Radeon Pro 575, 24GB RAM, macOS High Sierra 10.13.6

Film Editor, Colorist, DIT, Encoding technician
linkedin.com/in/vít-reiter-film-editor
Offline
User avatar

Dmitry Shijan

  • Posts: 1470
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Mar 31, 2020 1:52 am

Vit, as described dozens of times - what you see on timeline that you get on render. You can tweak gamma look with CST node or during grading as you like. Set timeline to Gamma 2.4, or Rec709 or in most cases it is BMFilm Log and wide gamut BMD film. Gamma it is just a tonal "look" of your image. You just shape image look with grading tools, Log to Rec709 CST node or LUT as you like and render with Resolve 16.2 with Rec709 metadata tags.

Output CST node with Luminance Mapping gamma is important and it is usually set to Rec709 to match appearance of most common BMD, ARRI and other Log to Rec LUTs.
To preserve "what you see on timeline to that you get on render" on macOS you need to add Rec709 tags in 16.2 render settings.

You need to care about timeline gamma settings only in Resolve 16.1 because it always tags rendered image "same as timeline" without asking user (which is technically seems correct, but in current incorrect reality appears incorrect) and so produce gamma shifts.

Example1. Set YRGB timeline to Gamma 2.4 put video to timeline and render with Rec709 tags
Example2. Set YRGB timeline to BMDFilm Log, put same video to timeline and render with Rec709 tags
Both videos will look exact the same because Timeline gamma/Color space setting in Resolve YRGB non color managed project don't affect look and don't transforms any pixels data.

Timeline gamma in Resolve YRGB non color managed project only acts as metadata or input profile for some tools:
It set input for "Use Mac Display color profiles for viewers" transformation.
It set input for quick right click - set node color space/gamma
It set input for for DCP renders (timeline to XYZ i guess)
It set input for CIE Scopes preview
Maybe something else i didn't noticed yet?

Source video gamma/color space, Timeline gamma/color space, Render gamma/color space, Display gamma/color space, are separate independent things. In color managed system they are just connected (transformed each to other) with application and system color management.

In non color managed OS instead of transform system simply assigns display gamma/color space to source video gamma. So in non color managed system it is better to match color space and gamma between display and source video because display gamma will always affect the look of final video output.

People just keep asking same questions instead of attempt to read and understand info collected in first post. All answers are there.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

gbendinelli

  • Posts: 3
  • Joined: Wed Apr 01, 2020 4:26 am
  • Real Name: Gus Bendinelli

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Apr 01, 2020 4:43 am

I've been following this thread closely as Resolve 16.0 exports played back in Quicktime have a gamma shift that I've never experienced before. Since I started coloring in Resolve years ago on my Macbook Pro, I've been lucky enough to have exports that play back in Quicktime with a perfect match to what I see in the viewers. I know this has not been the case for everybody, but I've come to rely on it as a part of my workflow. Furthermore, while it's easy to say that "it'll never match so don't even bother," I work regularly on music videos and commercials where the label and client are viewing ProRes drafts of my color correction, and it's critical that Quicktime (and Quick Look) playback is accurate. The people watching these drafts are not going to download VLC or another player due to gamma issues—we're lucky if they download the file at all rather than watching it in a browser.

In experimenting with the different methods here, I've found a formula that works well for my purposes:

1. Make sure "Use Mac Display Profile for Viewers" is off (Preferences>System>General).
2. Leave Color Science set to DaVinci YRGB and Timeline Color Space set to the default Rec.709 2.4 Gamma.
3. When exporting, set the Color Space Tag Override to "Rec.709" and the Gamma Tag Override to "sRGB".

In my experience, these settings create a near-perfect match that is much closer than setting the Gamma Tag Override to Rec.709 or setting the whole timeline to Rec.709 (Scene). I suspect that this is because Apple's default Color LCD calibration is based on the sRGB gamma curve, not the Rec.709 gamma curve. This workaround also does not require the use of any output LUTs or additional nodes which is preferable for me.

To be clear I am not advocating this over using calibrated hardware, this is simply a solution that I've found that works well for my needs as somebody who's working end-to-end on a stock Macbook Pro. Is it a band-aid over a larger problem? Of course. But I hope it can help some of you who are struggling in similar situations.
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Apr 01, 2020 5:29 pm

You just repeated what Dmitry said many times :)
You should set tag on export to Rec.709 not sRGB and you will have exact match.

All of this is (as you said) not a real solution, but a way to have Resolve preview matching QT preview. Those files will still look "different" on PC or in TV or on proper calibrate preview.
Last edited by Andrew Kolakowski on Wed Apr 01, 2020 5:30 pm, edited 1 time in total.
Offline
User avatar

Dmitry Shijan

  • Posts: 1470
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Apr 01, 2020 5:29 pm

gbendinelli wrote:1. Make sure "Use Mac Display Profile for Viewers" is off (Preferences>System>General).
2. Leave Color Science set to DaVinci YRGB and Timeline Color Space set to the default Rec.709 2.4 Gamma.
3. When exporting, set the Color Space Tag Override to "Rec.709" and the Gamma Tag Override to "sRGB".

In my experience, these settings create a near-perfect match that is much closer than setting the Gamma Tag Override to Rec.709 or setting the whole timeline to Rec.709 (Scene). I suspect that this is because Apple's default Color LCD calibration is based on the sRGB gamma curve, not the Rec.709 gamma curve. This workaround also does not require the use of any output LUTs or additional nodes which is preferable for me.

To be clear I am not advocating this over using calibrated hardware, this is simply a solution that I've found that works well for my needs as somebody who's working end-to-end on a stock Macbook Pro. Is it a band-aid over a larger problem? Of course. But I hope it can help some of you who are struggling in similar situations.


Interesting idea, i didn't experiment with sRGB gamma for video yet. If you 100% sure that your client will watch video on mac only and only original version that you directly put to his computer or FTP - it could be ok.
But if your client decide to transcode video with Hangbrake or something similar in most cases it will lost sRGB gamma tag and will use Rec709 tag. As a result transcoded video will look different to original on his mac.
If you upload video to Youtube or Vimeo it also will lost sRGB gamma tag and will use Rec709 tag. As a result transcoded video will look different to original on his mac.
Things can change over time and Youtube or Vimeo may start preserve tags during transcoding somehow. Most likely to keep things safe, it will work for new uploads only.

As it was mentioned earlier - we really need some interactive list with info about apps and online services that can read and preserve tags in video during transcoding. We are in very early stage of color management for video and it is hard to predict how things will change in future in other systems and applications.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

gbendinelli

  • Posts: 3
  • Joined: Wed Apr 01, 2020 4:26 am
  • Real Name: Gus Bendinelli

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Apr 01, 2020 6:23 pm

Andrew Kolakowski wrote:You just repeated what Dmitry said many times :)
You should set tag on export to Rec.709 not sRGB and you will have exact match.


The second half of my post above specifically states that in my testing, the Rec.709 tag for gamma does not result in a good match to the Resolve viewers on a stock Macbook Pro setup. The sRGB gamma tag results in a near-perfect match.

Dmitry Shijan wrote:But if your client decide to transcode video with Hangbrake or something similar in most cases it will lost sRGB gamma tag and will use Rec709 tag. As a result transcoded video will look different to original on his mac.
If you upload video to Youtube or Vimeo it also will lost sRGB gamma tag and will use Rec709 tag. As a result transcoded video will look different to original on his mac.


I've tested transcoding to various formats in Handbrake as well as uploading to Vimeo and YouTube, and all exports remain consistent across all players and transcode formats when you use the sRGB gamma tag.
Offline

Andrew Kolakowski

  • Posts: 6574
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Apr 01, 2020 6:38 pm

gbendinelli wrote:
Andrew Kolakowski wrote:You just repeated what Dmitry said many times :)
You should set tag on export to Rec.709 not sRGB and you will have exact match.


The second half of my post above specifically states that in my testing, the Rec.709 tag for gamma does not result in a good match to the Resolve viewers on a stock Macbook Pro setup. The sRGB gamma tag results in a near-perfect match.

How do you deal with color gamut problem if you use non-color managed preview in Resolve (and you don't use any preview LUTs etc)? Or is your screen just Rec.709 gamut not near P3.
Offline

gbendinelli

  • Posts: 3
  • Joined: Wed Apr 01, 2020 4:26 am
  • Real Name: Gus Bendinelli

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Apr 01, 2020 8:29 pm

Andrew Kolakowski wrote:How do you deal with color gamut problem if you use non-color managed preview in Resolve (and you don't use any preview LUTs etc)? Or is your screen just Rec.709 gamut not near P3.


I'm working on a Macbook Pro from 2013, before Apple was pushing for P3, which might be a part of why my setup is working.
Offline

Cosmin Hodiș-Mîndraș

  • Posts: 45
  • Joined: Wed Mar 13, 2013 9:00 pm
  • Location: Cluj-Napoca, Romania

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Apr 02, 2020 7:14 am

iNikitaPOD17 wrote:Dear colleagues, help, I want to make a LUT as in this manual. I do everything the same way and get to the stage when I do a screen shot in QT and can't figure out what to do in lattice. I get a straight gamma (linear) not as in the screenshot of the author.


Hi, sorry for being late with the answer. I have a hunch that I'll have plenty of time for a while... Well.

The screen should be put in an "unscaled mode" (so 1:1 pixels to screen dots), and QuickTime should display an un-scaled image (I used QT in full screen mode and selecting View - Actual size). The idea is to capture the color transformation that macOS apply to the HALD exported video on a pixel-by-pixel basis, so avoid any scaling or resizing.

Finally, open your screenshot in Photoshop or even Preview, crop it precisely (1728x1728px in my case) and export it as 8-bit TIFF. That's it, you can now open the image in Lattice.
iMac 27-inch Retina 2017 (4.2 GHz i7, Radeon Pro 580 8GB, 32GB RAM, 2TB SSD) | DaVinci Resolve Studio 16.1 | UltraStudio 4K Extreme 3 | Resolve Micro Panel | Desktop Video 11.4.1 | Ursa Mini Pro 4.6K v6.1 firmware
Offline

iNikitaPOD17

  • Posts: 10
  • Joined: Sun Mar 08, 2020 8:09 am
  • Real Name: Nikita Podobedov

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Apr 02, 2020 10:01 pm

Cosmin Hodiș-Mîndraș wrote:
iNikitaPOD17 wrote:Dear colleagues, help, I want to make a LUT as in this manual. I do everything the same way and get to the stage when I do a screen shot in QT and can't figure out what to do in lattice. I get a straight gamma (linear) not as in the screenshot of the author.


Hi, sorry for being late with the answer. I have a hunch that I'll have plenty of time for a while... Well.

The screen should be put in an "unscaled mode" (so 1:1 pixels to screen dots), and QuickTime should display an un-scaled image (I used QT in full screen mode and selecting View - Actual size). The idea is to capture the color transformation that macOS apply to the HALD exported video on a pixel-by-pixel basis, so avoid any scaling or resizing.

Finally, open your screenshot in Photoshop or even Preview, crop it precisely (1728x1728px in my case) and export it as 8-bit TIFF. That's it, you can now open the image in Lattice.


Good afternoon, I now only realized that 1: 1 scaling is not possible on the internal screen, and therefore you used the external screen to take a screenshot and crop. Well, when you created the LUT, do you see the exact coincidence of the image in Resolve and QT? Using your LUT, I see small deviations when exporting in both gamma and color (on my screen it is more greenish in QT and in Resolve more violet) I decided to repeat your experience to try to build your own ... Tell me when creating LUT on it end result, icc display profile affects? (maybe this is the difference of deviation?)
Offline

iNikitaPOD17

  • Posts: 10
  • Joined: Sun Mar 08, 2020 8:09 am
  • Real Name: Nikita Podobedov

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Apr 02, 2020 11:14 pm

Cosmin Hodiș-Mîndraș wrote:
iNikitaPOD17 wrote:Dear colleagues, help, I want to make a LUT as in this manual. I do everything the same way and get to the stage when I do a screen shot in QT and can't figure out what to do in lattice. I get a straight gamma (linear) not as in the screenshot of the author.


Hi, sorry for being late with the answer. I have a hunch that I'll have plenty of time for a while... Well.

The screen should be put in an "unscaled mode" (so 1:1 pixels to screen dots), and QuickTime should display an un-scaled image (I used QT in full screen mode and selecting View - Actual size). The idea is to capture the color transformation that macOS apply to the HALD exported video on a pixel-by-pixel basis, so avoid any scaling or resizing.

Finally, open your screenshot in Photoshop or even Preview, crop it precisely (1728x1728px in my case) and export it as 8-bit TIFF. That's it, you can now open the image in Lattice.


I managed to translate the internal monitor to a 1: 1 ratio and crop the screenshot of how you did it. I followed the instructions added to the grate, resized to 21 and exported. But your 3D LUT turned out to be much more accurate than mine. What's the matter? HALD file size I used 512x512...
Attachments
4.jpg
4.jpg (322.66 KiB) Viewed 93087 times
2.jpg
2.jpg (332.5 KiB) Viewed 93087 times
1.jpg
1.jpg (510.97 KiB) Viewed 93087 times
Offline

Cosmin Hodiș-Mîndraș

  • Posts: 45
  • Joined: Wed Mar 13, 2013 9:00 pm
  • Location: Cluj-Napoca, Romania

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Apr 06, 2020 4:57 pm

iNikitaPOD17 wrote: But your 3D LUT turned out to be much more accurate than mine


Well, that's because during LUT development I thought, at some point, that applying a "Mix curves" process in Lattice would be beneficial. Later I reconsidered this, but the screenshots I posted were taken before :)

If you open my LUT in Lattice, it should show similar curve "trips".
iMac 27-inch Retina 2017 (4.2 GHz i7, Radeon Pro 580 8GB, 32GB RAM, 2TB SSD) | DaVinci Resolve Studio 16.1 | UltraStudio 4K Extreme 3 | Resolve Micro Panel | Desktop Video 11.4.1 | Ursa Mini Pro 4.6K v6.1 firmware
Offline

iNikitaPOD17

  • Posts: 10
  • Joined: Sun Mar 08, 2020 8:09 am
  • Real Name: Nikita Podobedov

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Apr 08, 2020 10:02 pm

Cosmin Hodiș-Mîndraș wrote:
iNikitaPOD17 wrote: But your 3D LUT turned out to be much more accurate than mine


Well, that's because during LUT development I thought, at some point, that applying a "Mix curves" process in Lattice would be beneficial. Later I reconsidered this, but the screenshots I posted were taken before :)

If you open my LUT in Lattice, it should show similar curve "trips".


Well, yes the curves are the same, why is there a slight deviation towards the Magenta when exporting?
PreviousNext

Return to DaVinci Resolve

Who is online

Users browsing this forum: No registered users and 96 guests