Final Explanation of Gamma and Color Shift Problems

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 12:36 am

Updated on 12.11.2019

This is my attempt to understand and explain gamma/brightness and color shifts problems between different video formats and players. This problem is rather complicated and consists of multiple factors.
(This investigation is NOT related to legacy outdated Mac OSX versions gamma 1.8 vs Windows gamma 2.2 shift problems in quicktime .mov files. It is NOT related to incorrect setup of Full/Video levels in Project/Render settings which may produce darker/lighter image. It is also NOT related to various types of HDR metadata tags.)


Part 1. Video file metadata tags:

There are 3 main metadata tags located inside video file:

- Color primaries
- Matrix coefficients
- Transfer characteristics

Image

These metadata tags describes color space and gamma for apps that support color managed video output. "Color primaries" and "Matrix coefficients" describes color space settings. "Transfer characteristics" describes gamma setting.
You can find full list of supported tags here https://x265.readthedocs.io/en/default/ ... -colorprim
To see those metadata tags you need some media file inspector app (Invisor, VideoSpec, VideoScan, MediaInfo)

Modern color management for video is not standardized enough between different apps and operating systems. Some apps can read all those tags, some can read some of those tags, and some apps just ignore all those tags.


Part 2. Few basic things to understand:

- DaVinci Resolve Color Space Transform Node or LUT attached to clip or to timeline does not add or affect metadata tags.

- If you use DaVinci Resolve 16.1 and set project to YRGB (non color managed) workflow, it adds metadata tags to rendered video file according to timeline settings (Rec709, Rec2020, PCI-P3, Linear, g2.2, g2.4, g2.6 and so on...). Things may be different in older versions of DaVinci Resolve.

- The amount of supported metadata tags in video files is limited to few common color spaces and gammas, so if you set Timeline to some camera specific Log gamma and wide gamut Color Space, DaVinci Resolve will not add any metadata tags to rendered video file.

- If you use YRGB Color Managed workflow, DaVinci Resolve adds metadata tags according to selected Output Color Space in project settings.

- All macOS system apps (QuicktimeX player, Preview, QuickLook, Safari browser) always read "Color primaries", "Matrix coefficients" and "Transfer characteristics" tags from video file and transforms color space and gamma described in video metadata tags to monitor profile. This gamma color management method is rather exotic and unusual among other apps and video players, so as a result video output on mac always have that slightly brighter gamma look.

- When you check “Use Mac Display Color Profiles for Viewers” in DaVinci Resolve Preferences->System->General, it attempts emulate that specific macOS color management. It reads Timeline ColorSpace and Gamma settings (or "Output Color Space" project settings if you use YRGB Color Managed workflow) and transforms them to monitor profile.

- VLC Player v3 use it's own internal color management processing. It reads only "Color primaries" and "Matrix coefficients" metadata tags. It only transforms color space described in video metadata tags to monitor profile. VLC don't read Rec.709 gamma metadata from video file and don't transforms it in any way.
VLC output is usually 100% match to DaVinci Resolve viewer output if “Use Mac Display Color Profiles for Viewers” turned OFF.

- Older versions of VLC and most other video players, as well as Firefox browser can't read any of video metadata tags and so always output video "as is" without any additional color or gamma transforms.

- As i remember, Windows OS is not system wide color managed for video, so it always relies to video players and apps own internal color management. (Please correct me if i'm wrong)


Part 3. Different types of Monitor ICC profiles:

Monitor ICC profiles may have a lot of different types (XYZ LUT based, LAB based, Curve+Matrix based, Gamma+Matrix based). Different ICC profile types provides different calibration complexity and accuracy, but same time may cause compatibility problems in some apps when it comes to color space and gamma transform from video source to monitor profile.
Original bundled Monitor ICC profiles usually always designed within safe guidelines (Gamma 2.2 or sometimes sRGB and single curve+matrix profile type), so usually they don't cause any problems.
If you decide to calibrate your monitor manually and choose unusual profile type or some custom gamma curve, or use too complicated XYZ LUT based ICC profile type, you may expect problems with strange gamma shifts between different apps. Some monitor calibration software even warns you about possible compatibility problems and suggests to choose simpler but more compatible profile type.

Image Image


Part 4. Missed metadata tags inside video files:

Now we can move to final comparative tests and investigations to understand the core of the gamma/brightness shifts problem.

Software used:
- DaVinci Resolve 16.1
- VLC player 3.0.8
- macOS 10.14.6 (Mojave) QuicktimeX Player
- HandBrake video transcoder 1.2.2

Test setup:
- DaVinci Resolve project settings set to YRGB (non color managed).
- As a source i used DNG RAW footage captured with BMMCC camera.
- I created various combinations of project Timeline gamma settings and CST Node output settings and render each example to ProRes422.
- Next i convert all those files to .mp4 h264 with HandBrake video transcoder to see the difference in metadata tags between ProRes source and .mp4 delivery.
- Next i open all those video files with Invisor media file inspector and made comparative screenshots of metadata tags.
- And finally i open each of those video files in VLC and QuicktimeX players and made comparative screenshots.
- In addition i made few reference screenshots of DaVinci Resolve viewer.

Image

Tags added to ProRes .mov file rendered by DaVinci Resolve 16.1, if Timeline set to Rec.709:
- Color primaries: BT.709
- Matrix coefficients: BT.709
- Transfer characteristics: BT.709

Tags added to ProRes .mov file rendered by DaVinci Resolve 16.1, if Timeline set to gamma 2.4:
- Color primaries: BT.709
- Matrix coefficients: BT.709

Tags added to ProRes .mov file rendered by DaVinci Resolve 16.1, if Timeline set to some unusual camera specific wide gamut color space and Log gamma:
- matrix_coefficients_Original: BT.709

Tags added to h264 .mp4 file rendered by DaVinci Resolve 16.1, if Timeline set to Rec.709:
- Color primaries: BT.709
- Matrix coefficients: BT.709
- Color range: Limited
- Transfer characteristics: BT.709

Tags added to h264 .mp4 file rendered by DaVinci Resolve 16.1, if Timeline set to gamma 2.4:
- Color primaries: BT.709
- Color range: Limited
- Matrix coefficients: BT.709

Tags added to h264 .mp4 file transcoded by Handbrake, YouTube, Vimeo or any other similar app:
- Color primaries: BT.709
- Matrix coefficients: BT.709
- Color range: Limited
- Transfer characteristics: BT.709

And some Rec.2020 tests:

Tags added to ProRes .mov file rendered by DaVinci Resolve 16.1, if Timeline set to Rec.2020:
- Color primaries: BT.2020
- Matrix coefficients: BT.2020 non-constant
- Transfer characteristics: BT.709

Tags added by Handbrake during transcoding Rec.2020 ProRes .mov file to h264 .mp4 file (Handbrake can read metadata tags from source file and preserve them in transcoded .mp4 file):
- Color primaries: BT.2020
- Color range: Limited
- Matrix coefficients: BT.2020 non-constant
- Transfer characteristics: BT.709

Tags added to h264 .mp4 file rendered by DaVinci Resolve 16.1, if Timeline set to Rec.2020 (Wrong Color primaries metadata. Seems software bug found here):
- Color primaries: BT.709
- Matrix coefficients: BT.709
- Color range: Limited
- Transfer characteristics: BT.709

Here are some animated examples. VLC colors looks exact same between all tests and match to DaVinci Resolve 16.1 viewer.
Archive with original images here: https://www.dropbox.com/s/zmmi3xwgwl96k ... s.zip?dl=0

DaVinci Resolve 16.1 viewer:
Image

Timeline set to Rec.709 (VLC looks the same all the time, QuickTime X player looks the same):
Image

Timeline set to gamma 2.4 (VLC looks the same all the time, QuickTime X player produce shifts in gamma):
Image

Timeline set to BMDfilm (VLC looks the same all the time, QuickTime X player produce shifts in color and gamma):
Image


Part 5. Different look in different versions of DaVinci Resolve:
Metadata tags in DaVinci Resolve are added automatically based on selected Timeline settings. DaVinci Resolve 16 and 16.1 adds tags metadata in different way.

Tags added to ProRes .mov file rendered by DaVinci Resolve 16, if Timeline set to gamma 2.4:
- Color primaries: BT.709
- Matrix coefficients: BT.709
- Transfer characteristics: BT.709

Tags added to ProRes .mov file rendered by DaVinci Resolve 16.1, if Timeline set to gamma 2.4:
- Color primaries: BT.709
- Matrix coefficients: BT.709

So before DaVinci Resolve 16.1 both Rec709(Scene) and gamma 2.4 project settings where tagged exact the same: BT.709-BT.709-BT.709. Technically it may looks like mistake to tag g2.4 project with Rec709 metadata, but in real life it allow to avoid problems with gamma shifts between QuickTime player ProRes and mp4 YouTube transcodings and preserve "What You See is What You Get" concept.
But as we already know, color managed QuickTime player on macOS can read gamma tag, and by QuickTime specification decide that empty gamma tag means something like Gamma 2.4. As a result output video in QuickTime now looks darker, but all rest apps in the world just fill that empty gamma tag with BT.709 and so produce gamma shift during transcoding or during preview.

Currently only macOS QuickTime player can read all customized metadata tags in video files. Most other software and hardware video players on planet will ignore any custom metadata tags and will use BT.709-BT.709-BT.709 to playback HD video. I guess that Apple designed those tags long time ago as a part of QuickTime player, and later those tags where partially adopted to mp4 h264/h265 specification to communicate with video players for 4K Rec2020 HDR delivery.


Part 6. The solution:

If looking ahead, the quick solution for this problem - do NOT use gamma 2.4 in DaVinci Resolve Timeline project settings. Always set Timeline gamma to Rec.709 (Rec.709 (Scene)) and adjust final desired gamma look with CST Node or manually with gamma slider wheel whatever you like. This will help to avoid hidden gamma shifts and metadata tags mismatches between editing or transcoding, and will provide cleanest "What You See is What You Get" workflow. HandBrake, Youtube, Vimeo or any other current video transcoder by default expects Rec.709 gamma curve in video file and during transcoding always automatically adds BT.709 Color primaries, Matrix coefficients and Transfer characteristics to converted video file.


Part 7. YRGB vs YRGB Color Managed project settings in DaVinci Resolve

I done a little test and noticed gamma shift in DaVinci Resolve viewer between same project settings:

YRGB (non color managed) project:
- Timeline set to Rec.709 (Scene)
- CST nodes set to: input Rec.709, Output Rec.709

YRGB Color Managed project:
- Input Rec709
- Timeline Rec709
- Output gamma Rec709

After some research it appears that in YRGB Color Managed project Resolve automatically assigns gamma 2.4 input to any ProRes file on timeline. This overridden project input color settings and produced gamma shift in my test.
So if you use simple Rec.709 source file, don't forget to right click on clip and select input Rec.709 (Scene).
If you use Log source from camera, select input according to your Log camera format.
If you use RAW source from camera, select input bypass.

With proper input color space and gamma image preview will look the same between both YRGB and YRGB Color Managed project settings.

Image


Part 8. Gamma shift between macOS QuickTime player and other apps:

Even with proper tagged Rec.709 source you still may see slight gamma shift between macOS and VLC. Color management logic just very different between those apps and there is nothing we can do with it except to adjust gamma during grading to provide the gamma look somewhere in between.

If you want to (almost) match QuickTime player "look" in DaVinci Resolve viewer, Set YRGB (non color managed) Timeline to Rec709(Scene) as described earlier and check “Use Mac Display Color Profiles for Viewers” in DaVinci Resolve Preferences->System->General

If you want to match VLC player "look" in DaVinci Resolve viewer, Set YRGB (non color managed) Timeline to Rec709(Scene) as described earlier and uncheck “Use Mac Display Color Profiles for Viewers” in DaVinci Resolve Preferences->System->General

I think that main troublemakers here are Wide gamut Apple P3 monitors and color managed macOS QuickTime player. Corporations always force people to spend money for something new and partially useless, so step by step hardware moved to wide gamut/HDR displays. But same time video color management is not widely established and not standardized yet. It may take another 10 years until Windows will add system wide color management for video, and until Apple realize that it probably reads and transforms video gamma in wrong way and will fix it. This problem was well explained in details by fumoboy007 in this thread https://github.com/mpv-player/mpv/issues/4248 In short: Apple reads simplified 1.961 gamma instead of precise ITU-R 709 transfer function which is produce system wide gamma shift between color managed QuickTome player and other non color managed video players and apps.

As a quick fix for P3 displays you can use monitoring correction LUTs made by Cosmin Hodiș-Mîndraș. More details in this post viewtopic.php?f=21&t=101253&start=50#p564992
Download LUTs here https://www.dropbox.com/s/0sux7nydfyk72 ... c.zip?dl=0

My personal opinion is that in current real life situation for video monitoring you better get modern monitor with real 100% sRGB coverage. I will produce the most accurate result for video without any additional color management shifts.

P.S. When you test something and for some reason you switch between display ICC profiles in macOS System Preferences, don't forget to restart DaVinci Resolve to see correct result. Other apps with own internal color management like Photoshop, VLC, Firefox also needs to be restarted to output correct result.

As a bonus here is FilmLight QuickTime Colour Management tutorial with some similar things explained:



And also few helpful articles suggested by Marc Wielage:

Grading for Mixed Delivery: Cinema, Home, and Every Screen in Between
https://blog.frame.io/2019/10/14/gradin ... -delivery/

Color Management for Video Editors
https://jonnyelwyn.co.uk/film-and-video ... o-editors/
Last edited by Dmitry Shijan on Tue Nov 12, 2019 11:06 am, edited 17 times in total.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Supermachoalpha

  • Posts: 24
  • Joined: Tue Oct 16, 2018 2:05 pm
  • Real Name: Nate Game

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 2:10 am

I appreciate this explanation, however, none of these steps were required in Resolve 15. I decided to experiment by downgrading back to Resolve 15 on my iMac and the files export just fine with no color shift. They exported the exact same way they looked on the Resolve viewer. I then updated to 16.1 on my MacBook pro 15" 2017 (had DR 15 installed) and now it's exporting with the color shift/desaturated image. It is definitely a Resolve 16.1 issue. As you may have noticed, more people are posting about it now.
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 2:58 am

It seems those massive reports about gamma and color shifts problems in 16.1 are related to projects created in previous versions of DaVinci Resolve.
I just created similar setup inside old project and it looked really strange there. Next i restarted DaVinci Resolve, created new project and all things now looks as it should (VLC output exact match to DaVinci Resolve viewer). Next i open old project again and it magically looked as it should. Maybe new version of DaVinci Resolve needs time to reset something inside, or maybe 16.1 is just a not stable enough yet.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline
User avatar

Marc Wielage

  • Posts: 6180
  • Joined: Fri Oct 18, 2013 2:46 am
  • Location: Hollywood, USA

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 7:12 am

All of these explanations avoid the importance of SMPTE Color Bars at the head of the files to use as a reference. I go a step further and use a few seconds of Bars and a few seconds of Grayscale Ramp prior to slate, and each will tell me immediately if there's a gamma or chroma shift in the export. But you have to check the image on the scopes.

Cullen Kelly has written a terrific piece for Frame.io:

Grading for Mixed Delivery: Cinema, Home, and Every Screen in Between
https://blog.frame.io/2019/10/14/gradin ... -delivery/

and I think it covers the issues and the solutions very well. Understanding color management is also helpful:

Color Management for Video Editors
https://jonnyelwyn.co.uk/film-and-video ... o-editors/
marc wielage, csi • VP/color & workflow • chroma | hollywood
Offline

Wayne Steven

  • Posts: 2400
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 12:52 pm

Thanks Dimitry.
Often people deceive themselves so much they do not understand, even when the truth is explained to them.
Offline
User avatar

Cary Knoop

  • Posts: 1189
  • Joined: Sun Mar 12, 2017 6:35 pm
  • Location: Newark, CA USA

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 2:25 pm

Nice work!
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 7:48 pm

animated image examples moved to first post
Last edited by Dmitry Shijan on Tue Oct 29, 2019 3:14 am, edited 7 times in total.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 8:13 pm

Resolve 16.1 and QTX (and other OSX driven previews) are correct. It's all other apps which don't read custom gamma tag and show wrong preview. With color managed preview you can't have it accurate if gamma tag is not respected or set. It's all quite simple at the end. QTX recognises only few combination of other tags (Rec.2020 etc.), but at least it always respects gamma tag if it's present.
Fact that a player preview is the same as Resolve one is only half of the success. If you then check it in proper chain it may look not good at all.
OSX as well as Windows need up to date color managed engines. Current ones are limited and outdated. Same applies to most services like YouTube, Vimeo etc. They need to read tags and set them properly in final transcoded files as well.
We only need maybe 5 sets of widely recognised (up to date) standards, but unfortunately this seems to be way beyond Apple, Microsoft etc. Quite shocking, but no one cares as there is no money "there" to be made.
Last edited by Andrew Kolakowski on Thu Oct 24, 2019 8:33 pm, edited 1 time in total.
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 8:30 pm

Andrew Kolakowski wrote:Resolve 16.1 and QTX (and other OSX driven previews) are correct. It's all other apps which don't read custom gamma tag and show wrong preview.


Yes, on paper OSX do it right but in reality you can see that reading of gamma tag only makes things worse and produce endless variations of gamma shifts depending of tags existence.
Color management for video is still in very very early stage. There are billions of videos that actually Rec.709 but for some reason don't have gamma tags inside inside. OSX will read them as g2.4.

I think all that color management for video gamma was made by mad scientists from Apple only to support natively system wide HDR content and HDR displays on Mac.

By the way, VLC decide to bypass Rec.709 gamma reading, but it can read HDR gamma from proper tagged Rec.2020 h265 HDR content. Pure math vs real life situation is always different.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Supermachoalpha

  • Posts: 24
  • Joined: Tue Oct 16, 2018 2:05 pm
  • Real Name: Nate Game

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 8:40 pm

Awesome, Dmitry. Did you set the timeline to Rec709 Scene?
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 8:47 pm

Dmitry Shijan wrote:Color management for video is still in very very early stage. There are billions of videos that actually Rec.709 but for some reason don't have gamma tags inside inside. OSX will read them as g2.4.
.


Not the case. Rec.709 tagged files are converted by OSX engine as they would be originally made with 1.96 gamma which is far from any used standard. This is one of the key problems.
This is the whole point- create video to a standard and tag it accordingly. Later convert it to display profile using these info. Any remaining problems will be related to display profile not describing monitor accurately, but this is where calibration steps in for those who wants higher accuracy.
VLC reads HDR metadata only from h265 private headers, not MOV ones even if they do exist now for HDR (and Resolve does set them).
Last edited by Andrew Kolakowski on Thu Oct 24, 2019 9:26 pm, edited 1 time in total.
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 9:11 pm

Supermachoalpha wrote:Awesome, Dmitry. Did you set the timeline to Rec709 Scene?

Yes, Rec709 in my tests means Rec709 (Scene)
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 10:22 pm

The most exiting thing is that Timeline color space and gamma setting in Resolve is sort of "assign ICC profile" in graphic editors. It is not changing or converting actual video source. Even if you set timeline to Rec709, you can keep doing CST transforms to wide color spaces and log gammas (like BMDfilm to REDLog3G10) and than use RED IPP2 LUTs to transform to Rec709 without saturation clipping.

P.S. If someone interested, archive with original test images here: https://www.dropbox.com/s/zmmi3xwgwl96k ... s.zip?dl=0
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 11:56 pm

:arrow: And here is the difference between DaVinci Resolve 16. and 16.1. Is this a bug or a feature?

Tags added to ProRes .mov file rendered by DaVinci Resolve 16, if Timeline set to gamma 2.4:
- Color primaries: BT.709
- Matrix coefficients: BT.709
- Transfer characteristics: BT.709

Tags added to ProRes .mov file rendered by DaVinci Resolve 16.1, if Timeline set to gamma 2.4:
- Color primaries: BT.709
- Matrix coefficients: BT.709
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Wayne Steven

  • Posts: 2400
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Oct 24, 2019 11:59 pm

Why doesn't resolve handle all this is conversion? If you have the mappings of how it looks on screen, you have the information on how it should look transfered to automatically remap.
Often people deceive themselves so much they do not understand, even when the truth is explained to them.
Offline
User avatar

Marc Wielage

  • Posts: 6180
  • Joined: Fri Oct 18, 2013 2:46 am
  • Location: Hollywood, USA

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Oct 25, 2019 2:15 am

Wayne Steven wrote:Why doesn't resolve handle all this is conversion? If you have the mappings of how it looks on screen, you have the information on how it should look transfered to automatically remap.

If VLC gets it right, but QuickTime Player gets it wrong, whose fault is it? It seems to me this is an Apple problem. Read the articles I posted above.

BTW, there are quite a few alternatives to QuickTime Player out there, at least for Mac. One is CinePlay from Digital Rebellion, and we usually arrange for clients to use this to judge our work:

https://www.digitalrebellion.com/cineplay/

There's a reason why having a color managed output and a calibrated monitor are the only way you can believe what you see. Otherwise, there are too many risks that the OS (or a bad display) are affecting the image.

I just had a client the other day whine that he didn't like how a show looked on his computer, so I pulled it up on my laptop and showed him the brightness control and what the PLUGE pulse represented in SMPTE bars. He was appalled to realize that every single person could potentially see a different image, and I sighed and said "welcome to my world."
marc wielage, csi • VP/color & workflow • chroma | hollywood
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Oct 25, 2019 3:01 am

Marc Wielage wrote:If VLC gets it right, but QuickTime Player gets it wrong, whose fault is it? It seems to me this is an Apple problem. Read the articles I posted above.


I used mac from 10.5 and see that almost every new version of mac changes something color management. First they change system wide gamma 1.8 to 2.2 and fix color management problems during many versions in many parts of the system. Next it was a version when system always assigned sRGB profile to untagged images and then transformed them to Monitor profile. In theory it is correct way to deal with untagged images. But in later versions they just start to assign monitor profile to untagged images. This produced different (wrong, non color managed) colors of untagged images. Why they do this? Probably dual stage system wide color transforms on the fly eat too many resources somehow? I don't know. Also i remember that in some earlier versions 2-3 years ago even proper tagged images in mac Preview always looked brighter than in Photoshop. Now they fix it and color management for images looks the same between system color management and Photoshop internal color management.
Same goes to gamma reading for video files. It looks very strange for me. I experiment with switching to different monitor profiles to see if i can find something that will not produce gamma shift between QT player and VLC. And no success yet. QT player is always brighter. Even if i set monitor profile to native Rec709, and expect that Quicktime player will Transform Rec709 to Rec709 without gamma shift, it keeps generate brighter gamma than VLC. I remember long long time ago it was a forum thread on some technical forum (probably avforums or avsforum or something so), where people measure and draw somehow chart about how Mac reads gamma for system wide color management. People came to conclusion there that gamma reading is done in very strange and uncommon way.

Also never use use old "Classic" QuickTime player on modern macOS systems. That thing screws gamma and colors in it's own way. It is not supported anymore, but some people still keep to download it from Apple and install to system :)
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Wayne Steven

  • Posts: 2400
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Oct 25, 2019 7:27 am

Thanks Marc, but I was referring to the resolve side setting up a standardised outputs (name the output, and it does all the settings for that). Whatever a display or Player does with that is up to them.


You know, the world is as bad as I feared when I wanted to set up an alternative to windows and java in the 1980's/1990's, that did everything exactly right so not to stuff up everybody's world, and made life easy. Even Apple is just not good enough. You just can't trust anybody. You look at the aged care home crises over here (most homes here are one star rating compared to America, and we have had a royal Commission into the abuse. People are going in able to look after themselves and land up Dead in 3+ months) people are lazy morons wanting to maximise profits at the cost of quality.
Often people deceive themselves so much they do not understand, even when the truth is explained to them.
Offline

Tom Early

  • Posts: 1254
  • Joined: Wed Jul 17, 2013 11:01 am

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Oct 25, 2019 12:30 pm

Dmitry Shijan wrote::arrow: And here is the difference between DaVinci Resolve 16. and 16.1. Is this a bug or a feature?

Tags added to ProRes .mov file rendered by DaVinci Resolve 16, if Timeline set to gamma 2.4:
- Color primaries: BT.709
- Matrix coefficients: BT.709
- Transfer characteristics: BT.709

Tags added to ProRes .mov file rendered by DaVinci Resolve 16.1, if Timeline set to gamma 2.4:
- Color primaries: BT.709
- Matrix coefficients: BT.709


That's a feature. As Andrew said, Resolve 16.1 is right. And where are you getting these tags from? Because as the Filmlight video you posted states, if you do CMD-I on a video file in the MacOS Finder, it will show you the tags, and what I get from Rec709 Gamma 2.4 timeline outputs is 1-2-1, the '2' being 'unknown' as that document states, but also something that can be manually specified. Importing such a file back into Resolve shows it as Rec709 Gamma 2.4. Importing a file tagged 1-1-1 (any FCPX ProRes export, Premiere too I believe*) shows it as Rec709 (Scene). If you keep it that way in a properly colour managed workflow then it will give you a lot of problems.

*ProRes exports from Avid don't have any NCLC tags on them though
MBP2016, MacOS 10.14.6, 2.9 GHz Intel Core i7, 16 GB 2133 MHz LPDDR3,
Radeon Pro 460 4096 MB, Resolve Studio 16.1.1.005
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Oct 25, 2019 4:08 pm

Marc Wielage wrote:
Wayne Steven wrote:Why doesn't resolve handle all this is conversion? If you have the mappings of how it looks on screen, you have the information on how it should look transfered to automatically remap.

If VLC gets it right, but QuickTime Player gets it wrong, whose fault is it? It seems to me this is an Apple problem. Read the articles I posted above.


VLC is not always correct at all. I grade to Rec.709 with 2.2 you with 2.4 gamma and for VLC files are the same as it doesn't read gamma tag. How can it be correct then for both files?

For common and properly tagged files like Rec.709 with 2.4 gamma, Rec.2020 or P3 with eg. 2.4 or 2.6 gamma QTX is correct. It will properly convert preview to screen profile. Unfortunately not all possible combinations of tagging are supported in OSX color engine.
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Oct 25, 2019 4:19 pm

Tom Early wrote:
Dmitry Shijan wrote::arrow: And here is the difference between DaVinci Resolve 16. and 16.1. Is this a bug or a feature?

Tags added to ProRes .mov file rendered by DaVinci Resolve 16, if Timeline set to gamma 2.4:
- Color primaries: BT.709
- Matrix coefficients: BT.709
- Transfer characteristics: BT.709

Tags added to ProRes .mov file rendered by DaVinci Resolve 16.1, if Timeline set to gamma 2.4:
- Color primaries: BT.709
- Matrix coefficients: BT.709


That's a feature. As Andrew said, Resolve 16.1 is right. And where are you getting these tags from? Because as the Filmlight video you posted states, if you do CMD-I on a video file in the MacOS Finder, it will show you the tags, and what I get from Rec709 Gamma 2.4 timeline outputs is 1-2-1, the '2' being 'unknown' as that document states, but also something that can be manually specified. Importing such a file back into Resolve shows it as Rec709 Gamma 2.4. Importing a file tagged 1-1-1 (any FCPX ProRes export, Premiere too I believe*) shows it as Rec709 (Scene).


1-2-1 is a proper tagging for Rec.709 with gammas like 2.2 or 2.4. What you don't see in Finder info is gamma tag. If you use 1-2-1 (two means unknown Transfer Function) then you should specify gamma tag and set it to your desired value. This is what Resolve 16.1 and Baselight do ( and what all apps should do). With such a flagging you can have eg. Rec.2020 files with 2.4 gamma or P3 files with 2.6. Such an info describes file fully and allows color managing system properly convert it to screen profile.
All is fairly simple, but about none of the video apps care much about properly setting and reading tags, so we have one big mess.
I advise to use tool called JES Extensifier for reading and setting MOV headers. It can be found on the net, but its author has passed away. Latest version is 3.x which is new OSXs friendly (not necessarily latest one).

Rec.709 tagging (1-1-1) is actually quite bad and should be never used as it will cause OSX color engine to treat file as if it was graded with 1.96 gamma. Who grades to 1.96 gamma? It's all explained in Baselight video.
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Oct 25, 2019 4:36 pm

Tom Early wrote:
*ProRes exports from Avid don't have any NCLC tags on them though


Are you sure? If yes then this is great example of what I just mentioned in post above. And we talk about AVID- most pro editor out there.... :D

ProRes should have MOV tags as well as its private ones set and of course both should be the same.
Offline

dobmatt

  • Posts: 14
  • Joined: Sat Oct 12, 2019 12:48 am
  • Real Name: Matthew Dobrski

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Oct 25, 2019 8:54 pm

Marc Wielage wrote:
Wayne Steven wrote:I just had a client the other day whine that he didn't like how a show looked on his computer, so I pulled it up on my laptop and showed him the brightness control and what the PLUGE pulse represented in SMPTE bars. He was appalled to realize that every single person could potentially see a different image, and I sighed and said "welcome to my world."


Once you realize this, all discussion and speculations about the issue are quite senseless actually ...
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Oct 25, 2019 9:05 pm

Screen quality still doesn't explain why same file looks different in eg. 5 different apps (on the same screen). Quality of preview devices is another problem (further in the chain).
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Oct 25, 2019 10:13 pm

One more time. There is no such a standard in modern video world as Rec.709 with gamma 2.2 or 2.4. All video software will always expect to read your final mp4 video files as Rec.709 color space with Rec.709 gamma. Some video players can read Rec.2020 wide gamut color space with Rec.709 gamma. There is also 10 bit h265 codec Rec.2020 with HDR gamma delivery specifications, but this is a very different story.

Metadata tags don't change actual footage source. They are just can explain to Quicktime video player how to do gamma correction on the fly during playback.
VLC is not "correct or not correct", It just always ignores any gamma metadata tag corrections/transformations. it just always show you video gamma as is. All rendered files from my test are mathematically exact same, because they where created with exact same BMDfilm to Rec709 CST node. They look different only during playback in different video players that read (or not read) different metadata tags.

Same goes to Resolve timeline settings. Timeline settings (at least in YRGB in non color managed project) will not change your source video in any way, as well as it will not affect rendered data. Resolve timeline settings is also just sort of metadata tag.

To be honest i have no idea where from comes that default timeline setting gamma 2.4 when you start YRGB project in Resolve. Can someone from developers explain why they put that gamma 2.4 that always screws footage and produce endless problems?

By the way, According this Web browser color management test, https://cameratico.com/tools/web-browse ... ment-test/ it seems mac OS (at least version 10.14.6) finally solve problem with tagged vs untagged sRGB images look in Safari browser and other system apps. Same time In last version of Firefox tagged vs untagged sRGB images are still look different.
So i think we need to wait another 10 years until we can see proper system wide color management for video :)

And where are you getting these tags from?

Read first post in this thread.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline
User avatar

Marc Wielage

  • Posts: 6180
  • Joined: Fri Oct 18, 2013 2:46 am
  • Location: Hollywood, USA

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Oct 26, 2019 5:01 am

dobmatt wrote:Once you realize this, all discussion and speculations about the issue are quite senseless actually ...

That is sadly true and very sobering.

Note this has been a problem for at least 40 years that I know of. Hell, I had clients back in 1985 calling and screaming at me on the phone because what they saw on their office monitor did not look like how it looked in our (calibrated) color-correction room. In some ways, the level of ignorance in the world about things like this has not changed.
marc wielage, csi • VP/color & workflow • chroma | hollywood
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Oct 26, 2019 7:33 am

Dmitry Shijan wrote:One more time. There is no such a standard in modern video world as Rec.709 with gamma 2.2 or 2.4. All video software will always expect to read your final mp4 video files as Rec.709 color space with Rec.709 gamma. Some video players can read Rec.2020 wide gamut color space with Rec.709 gamma. There is also 10 bit h265 codec Rec.2020 with HDR gamma delivery specifications, but this is a very different story.


It's very opposite.
About every professional grade is made to Rec.709 with 2.4 gamma which is defined in BT.1886, but almost no one uses this naming.
Rec.709 original spec does not define gamma for preview devices (only recording) and this is actually source of all this mess. People started using 2.2 gamma and later 2.4. BT.1886 meant to sort it out by defining actual standard. Problem is that software developers and spec for color engines in OSes never got updated.
There is actually no such a thing like Rec.709 gamma for preview device.
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Oct 26, 2019 10:00 am

From Wiki:
While Rec. 709 does not specify the display referred gamma, display gamma is discussed in EBU Tech 3320 and specified in ITU-R BT.1886 as a gamma of 2.4.
In typical production practice the encoding function of image sources is adjusted so that the final picture has the desired aesthetic look, as viewed on a reference monitor with a gamma of 2.4 (per ITU-R BT.1886) in a dim reference viewing environment (per ITU-R BT.2035).


So gamma of 2.4 is recommended for display, but not for source video file. Just recommendation to correct Rec. 709 video with Gamma 2.4 during playback from non color managed video player by calibrating monitor with gamma 2.4 or by correcting is somehow else during playback.
There is also no any info in this specification about color management for video and gamma transformations similar to what Apple did in QuickTime player.

Timeline gamma setting in YRGB (non color managed) project don't change the way how you see images in Resolve. You can change timeline from Rec709 to ACES with linear gamma, and it will look same same before in Resolve image viewer and in any non color managed video player. For actual correction you need to use Gamma wheel or CST Node.

Let's say If your specific workflow delivery require P3 color space with gamma 2.6, so you set Timeline settings to P3/gamma2.6 to provide proper metadata tags, AND you need to setup CST Node in timeline or clip to actually transform source video to P3/gamma 2.6.

- Timeline settings only adds metadata tags to exported file.
- CST Node only transforms actual pixels data.
They both should match if you want preserve "What You See is What You Get"


P.S.
If you use YRGB Color Managed project settings, you just set output color space and gamma, and Resolve will add proper metadata tags based on these settings.

P.P.S.
If you want to preserve that ITU-R BT.1886 appearance specification in final file, do CST Node Source to Rec709, and next add another "reversed" CST Node g2.4 to Rec709. This will create desired gamma correction for dark environment view conditions and same time it will look the same in any color managed and non color managed devices and will not produce unexpected gamma shifts during transcoding to other formats.

Image
Image
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Hendrik Proosa

  • Posts: 755
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Oct 26, 2019 10:09 am

Dmitry Shijan wrote:P.P.S.
If you want to preserve that ITU-R BT.1886 appearance specification in final file, do CST Node Source to Rec709, and next add another CST Node g2.4 to Rec709. This will create desired gamma correction for dark environment view conditions.

Appearance is intended to be the same in different environments, end to end gamma of encoding and display should take care of this. What kind of display calibration would you use this kind correction on and why should data encoded this way be let run wild in the world? It isn't sRGB nor rec709, so what is it?

Edit: wrote something that made little sense to myself before so fixed the wording.
I do stuff.
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Oct 26, 2019 3:50 pm

Dmitry wrote:So gamma of 2.4 is recommended for display, but not for source video file. Just recommendation to correct Rec. 709 video with Gamma 2.4 during playback from non color managed video player by calibrating monitor with gamma 2.4 or by correcting is somehow else during playback.


We are talking export mainly and tagging final files.
How you interpret source it's really up to you. I would always advice to check what Resolve auto detected and correct it if needed. You have full control.
BT.1886 is not any correction for Rec.709, but a proper modern standard for displays which everyone should use for SDR grading. You seems to be bit lost and keep talking about Rec.709 as it would be some master standard. It was never a standard for grading but for recording only. BT.1886 is what you should be talking when discussing displays and previewing for today's deliveries. This is used by all post houses now (actually they switched to pure 2.4 gamma curve some time ago). Now it's all properly described and some new TVs do actually use BT.1886 naming in menus (and math behind it).
Last edited by Andrew Kolakowski on Sat Oct 26, 2019 4:03 pm, edited 2 times in total.
Offline
User avatar

Jean Claude

  • Posts: 2973
  • Joined: Sun Jun 28, 2015 4:41 pm
  • Location: France
  • Warnings: 1

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Oct 26, 2019 3:56 pm

Marc Wielage wrote:.....
There's a reason why having a color managed output and a calibrated monitor are the only way you can believe what you see. Otherwise, there are too many risks that the OS (or a bad display) are affecting the image.
....


+1
Just by following these good advice, it saves you a lot of disadvantages ...
"Saying it is good, but doing it is better! "
Win10-1809 | Resolve Studio V16.1 | Fusion Studio V16.1 | Decklink 4K Extreme 6G | RTX 2080Ti 431.86 NSD driver! |
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Oct 26, 2019 11:05 pm

Andrew, my opinion that when you render video file from Resolve, you formally produce new "recording". It is like "recording" from camera, which records video from sensor to Rec709 file.

When you view video in player, or monitor video during grading you may need preserve that BT.1886 gamma view specification. It is a work of monitor calibration or video player output correction, or work grading tools like CST node with BT.1886 correction.

Rec709 (Scene) as timeline in real life produce 100% stable color and gamma match between Resolve, .mov and .mp4 Handbrake transcodings, VLC, non color managed web browsers. It also looks exact the same when you put any of those rendered transcoded videos (or even tiff image file) back to Resolve.
It also looks same between all macOS color managed apps and browsers.

Gamma 2.4 timeline tag, will look same between non color managed apps, but depending of transcoded format will produce endless gamma shifts in macOS color managed environment.

Gamma 2.4 (or any other selected on Resolve timeline settings in YRGB project) don't affect how image looks in Resolve image viewer. It only adds or removes random metadata tags, that random apps randomly can or can't read.

I really can't get what are you argue for. Are you sure you read first post in this thread?
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Oct 26, 2019 11:25 pm

Exporting file is not recording at all.
Rec.709 has actually almost lost its meaning for recording as well as most of them are LOG based now.
It's just an old standard.
Offline

mpetech

  • Posts: 35
  • Joined: Wed Sep 04, 2013 9:52 pm

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 12:01 am

Andrew Kolakowski wrote:About every professional grade is made to Rec.709 with 2.4 gamma which is defined in BT.1886, but almost no one uses this naming.


Not true. 2.4 gamma in SDR world is fairly recent when you consider the age of color broadcast video. Many TV, monitors, calibration software, etc. in fact still defaults to 2.2, including Sony broadcast monitors.
You walk into any grading room in NYC or LA and I bet you it is 50/50 chance that it is 2.2 or 2.4.
Not to mention 2.4 vs 2.2 is partly subjective for TV and streaming.
Offline

mpetech

  • Posts: 35
  • Joined: Wed Sep 04, 2013 9:52 pm

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 12:05 am

Jean Claude wrote:
Marc Wielage wrote:.....
There's a reason why having a color managed output and a calibrated monitor are the only way you can believe what you see. Otherwise, there are too many risks that the OS (or a bad display) are affecting the image.
....


+1
Just by following these good advice, it saves you a lot of disadvantages ...


A calibrated monitor is only one aspect. You have to still know how it is calibrated and understand your signal chain or software settings.
I have walked in some color sessions with the colorist thinks he is looking at the correct image when in fact it is wrong (wrong gamma, wrong color, etc.). The monitor was calibrated correctly just not the profile they needed for that particular project.
Offline

RikshaDriver

  • Posts: 70
  • Joined: Sun Aug 12, 2018 10:08 am
  • Location: Oz, Gizzard of
  • Real Name: Asim Siddiqui

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 3:37 am

Rec 709 (Scene) is Scene referred and the ITU standard transfer function that your camera will generally encode to (unless using a funky gamma).

If you are recording outside 709 and have a LUT that is transforming to Rec709, it is generally Scene referred, not Display referred, unless explicitly done so.

You’ve set your timeline to Scene to match the converted input (I.e. BMD to 709), but timeline is not the key here... it’s your input settings.

In general terms, make sure your “Input LUT” output matches your timeline setting or your Input matches your camera output.

Also keep in mind any grading you do will be relative to that timeline setting.. any change in gamma will react differently to your grade... that’s why VFX are often done in linear.
Offline
User avatar

Marc Wielage

  • Posts: 6180
  • Joined: Fri Oct 18, 2013 2:46 am
  • Location: Hollywood, USA

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 6:36 am

mpetech wrote:A calibrated monitor is only one aspect. You have to still know how it is calibrated and understand your signal chain or software settings.
I have walked in some color sessions with the colorist thinks he is looking at the correct image when in fact it is wrong (wrong gamma, wrong color, etc.). The monitor was calibrated correctly just not the profile they needed for that particular project.

Your experience is different from mine. I know immediately when the monitor is wrong when I look at test signals and scopes. What I can't tell is when the monitor is right, because only a probe and a skilled technician can do that. Staying on top of this is a standard part of the job.

Funny related story: I once did a test for a major American network here in LA, and told them as I was leaving, "by the way, I think your monitor is dark." They asked me why and how that could be, and I told them I looked at a 100% white test pattern and knew immediately it wasn't at 100 nits. They later checked it and admitted somebody had "bumped" the menu and it was at about 80 nits. You wonder how many weeks or months went by before somebody checked it.

Note this has nothing to do with the o.p.'s question. Did you read the links I posted above?
marc wielage, csi • VP/color & workflow • chroma | hollywood
Offline

Wayne Steven

  • Posts: 2400
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 6:56 am

How do the new auto calibration features go in keeping things consistent?

Is there any test patterns suites where, if you have good vision and can or can't tell the difference on the pattern on controlled lighting, your monitor is obviously out and you should get it checked?
Often people deceive themselves so much they do not understand, even when the truth is explained to them.
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 8:55 am

mpetech wrote:
Andrew Kolakowski wrote:About every professional grade is made to Rec.709 with 2.4 gamma which is defined in BT.1886, but almost no one uses this naming.


Not true. 2.4 gamma in SDR world is fairly recent when you consider the age of color broadcast video. Many TV, monitors, calibration software, etc. in fact still defaults to 2.2, including Sony broadcast monitors.
You walk into any grading room in NYC or LA and I bet you it is 50/50 chance that it is 2.2 or 2.4.
Not to mention 2.4 vs 2.2 is partly subjective for TV and streaming.


Even if it's 50/50 it's still 2.2 gamma not something Rec.709 related as it doesn't define gamma for display.
Last edited by Andrew Kolakowski on Sun Oct 27, 2019 5:01 pm, edited 1 time in total.
Offline
User avatar

Jean Claude

  • Posts: 2973
  • Joined: Sun Jun 28, 2015 4:41 pm
  • Location: France
  • Warnings: 1

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 9:46 am

mpetech wrote:
Jean Claude wrote:
Marc Wielage wrote:.....
There's a reason why having a color managed output and a calibrated monitor are the only way you can believe what you see. Otherwise, there are too many risks that the OS (or a bad display) are affecting the image.
....


+1
Just by following these good advice, it saves you a lot of disadvantages ...


A calibrated monitor is only one aspect. You have to still know how it is calibrated and understand your signal chain or software settings.
I have walked in some color sessions with the colorist thinks he is looking at the correct image when in fact it is wrong (wrong gamma, wrong color, etc.). The monitor was calibrated correctly just not the profile they needed for that particular project.


Hello
Do not worry. I have my Calman Business and a probe and everything is fine. :)
"Saying it is good, but doing it is better! "
Win10-1809 | Resolve Studio V16.1 | Fusion Studio V16.1 | Decklink 4K Extreme 6G | RTX 2080Ti 431.86 NSD driver! |
Offline

Tom Early

  • Posts: 1254
  • Joined: Wed Jul 17, 2013 11:01 am

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 3:45 pm

Some observations:

If I use RCM, and convert from Rec709 Gamma 2.4 input/timeline to P3D65, sRGB, or leave output at Rec709 Gamma 2.4, then in QT Player X, each video looks the same, apart from the shadows (getting darker as you go from P3 then to Rec709 2.4 then to sRGB).

If I have 'Use Mac Display Color Profiles for Viewers' checked in System Preferences, then each of these videos in QTX will mostly what I see in the Resolve viewers, with the shadows being problematic again (this time Resolve is darker).

HOWEVER, if I use RCM, and convert from Rec709 Gamma 2.4 input/timeline to Rec709 Gamma 2.2 or Rec709 (Scene), then neither of the resulting videos will even remotely match each other or the other outputs listed above when played in QTX, despite having correct tagging in the Finder; Rec709 Gamma 2.2 is much lighter, and Rec709 (Scene) is much darker; neither matches the Resolve viewer at all with 'Use Mac Display Color Profiles for Viewers' checked in System Preferences.

In DaVinci YRGB mode, setting my timeline to Rec709 Gamma 2.2 for some reason plays back LIGHTER in QTX than when I export with my timeline set to Rec709 (Scene), when surely it should be darker (since setting my timeline to 2.4 gamma results in the darkest file of the three).

Weird thing is, when doing these tests I seemed to get different results in the Resolve viewers at different times; sometimes what seemed like a very good match between the viewer and the output file at one point would later be clearly different across the whole image later on, despite using the same settings and the same output file. I would sometimes also get identical looking viewers as I switched between different output settings with Mac Display Profiles turned on (and as I changed the setting each time the viewer would refresh in the same way that everything refreshes when changing monitor profiles in MacOS System Preferences, i.e. flickering before resting on the new profile), but other times there would be no refresh and Rec709 Gamma 2.4 would clearly be darker than the other options (sRGB/2.2/Scene/P3), which matched each other still.


It doesn't surprise me that Mac Display Profiles would show everything darker than it should, I mean it's supposed to match an output monitor (allowing for being shown on a non-calibrated display) but I've always noticed that it crushes the shadows. And now, despite having correct tagging, QTX is doing the same to videos I know should be lighter.


- Rolling back to 16.0, if I have RCM set up as Rec709 gamma 2.4 all the way, then the exported file is tagged 1-1-1, BUT if I have Mac Display Profiles enabled then it looks IDENTICAL in QTX to the Resolve viewer. Having said that, if I now change the Output Colour Space to other options then I see a big difference, which is surely not correct.
MBP2016, MacOS 10.14.6, 2.9 GHz Intel Core i7, 16 GB 2133 MHz LPDDR3,
Radeon Pro 460 4096 MB, Resolve Studio 16.1.1.005
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 4:55 pm

There is some bug in Resolve 16.1. It sets gamma tags properly for 2.4 based projects (but not 2.2), but preview is slightly off.
In older versions after you manually set headers QTX and Resolve preview were identical for 2.4 or 2.2 gamma.
Check P3 with 2.6 gamma. This should look the same if I remember well.
At some point Resolve had a hack (actually bad hack) for 2.4 (but not other gammas as you noticed) based projects preview and I think this may left some issue.

It still needs polishing in Resolve but we are closer.
Offline

Tom Early

  • Posts: 1254
  • Joined: Wed Jul 17, 2013 11:01 am

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 9:25 pm

OK here's another one.

I have a video from a couple of years ago, the QT file is tagged 1-1-1.

If I bring this into Resolve and tag the file as Rec709 Gamma 2.4 (because that's how it was originally graded on a calibrated broadcast monitor), and if I connect to my TV via HDMI (it's a few years old but pretty decent*), and use either the standard viewer or Clean Feed viewer in either a YRGB project or RCM set to Rec709 Gamma 2.4, then the video in QTX is IDENTICAL to what I see in Resolve's viewers**. Remember, the QT file is tagged 1-1-1 in the Finder. How does this make sense? :?: :?: :?:

On the other hand, if I now switch the displays so that Resolve's viewers are now on my MBP, then I have to switch the output to P3 to get it anywhere near the QTX view (which is expected), but even then there's a small difference in the shadows.

*I can check against 2 more TVs tomorrow, one much more modern, and will update then

**I checked the output from an UltraStudio Mini Monitor on this same TV and noted how it looked the same as the Clean Feed Viewer display; for good measure I checked a QT file in QTX, again tagged 1-1-1, against the same video on a recent commercial Blu-Ray from a PS4 Pro, and aside from some very minor saturation differences in the blues (which I'll put down to the Blu-Ray encoding/compression for now), they matched too.

So for now, even though it is "wrong", I think I'll stick with 1-1-1 tagging for Rec709 files.
MBP2016, MacOS 10.14.6, 2.9 GHz Intel Core i7, 16 GB 2133 MHz LPDDR3,
Radeon Pro 460 4096 MB, Resolve Studio 16.1.1.005
Offline

Andrew Kolakowski

  • Posts: 5924
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 9:44 pm

Which version of Resolve? If I'm correct 16 betas had BM hack for 2.4 gamma projects and they provided matching preview even if files were tagged as 1-1-1.

I've also seen different behaviour between Mac screens and 'others'. Maybe TVs are not color managed in which case both preview are converted with same 'gamma' which is used by TV. Signal goes to TV as 'untouched' YUV (I assume your TV receives YUV over HDMI, not RGB). There are so many unknown elements here. As I said before- seeing same preview doesn't guarantee it's correct one. Does it look same as over BM card?
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 10:52 pm

Yes, Interesting observation. Here is mine. In Resolve viewer (I switched to Resolve 16 right now and use ProRes Rec709 clip as source video).

YRGB Color Managed proejct with:
input Rec709, Timeline Rec709, Output g2.4

looks EXACT same as

YRGB (non color managed) proejct with:
Timeline set to Rec709
CST nodes set to: input Rec709, Output Rec709

Update.
After some research it appears that in YRGB Color Managed project Resolve automatically assigns gamma 2.4 input to any ProRes file on timeline. This overridden project input color settings and produced gamma shift in my test.
So if you use simple Rec.709 source file, don't forget to right click on clip and select input Rec.709 (Scene).
If you use Log source from camera, select input according to your Log camera format.
If you use RAW source from camera, select input bypass.

Now image preview looks the same between both YRGB and YRGB Color Managed project settings when you set both outputs to Rec709

P.S. When you change display ICC profiles in Mac system preferences, don't forget to reatart Resolve to see corerct result. Other apps with own internal color management like Photoshop, VLC, Firefox also needs to be restarted to output corerct result.
Last edited by Dmitry Shijan on Tue Oct 29, 2019 1:56 pm, edited 2 times in total.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Oct 27, 2019 11:27 pm

So before Resolve 16.1 both Rec709 and g2.4 output project settings where tagged the same: BT.709-BT.709-BT.709. Technically it may looks like mistake to tag g2.4 project with Rec709 metagata, but in real life it allow to avoid problems with gamma shifts between Quicktime player ProRes and mp4 YouTube transcodings and preserve "What You See is What You Get" concept.

And as described here
viewtopic.php?f=21&t=101253&p=562259#p561326 In Resolve 16.1 they start to tag gamma 2.4 as BT.709-BT.709-empty

So now color managed QuickTime player on macOS can read that tag, and deside that empty gamma tag means Gamma 2.4, and so output darker looking image. But all rest apps in the world just fill empty tag field with BT.709 and so produce gamma shift during transcoding or preview.

This is another proof for my previous tests.
Do not mess with unusual metadata tags.
Your video may have any gamma look you create during grading, but until you export file for some very specific delivery (some P3 cinema projector or customized in house VFX forkflow), final exported master file just should always be tagged with BT.709-BT.709-BT.709

Computer video players, hardware BluRay players, software video transcoders, YouTube, Vimeo transcoders, all they expect that your HD video source is tagged as BT.709-BT.709-BT.709
Standard HDTV resolution, used by full HD and HD ready 1080p TV displays such as high-end LCD, plasma and rear projection TVs, and a typical PC resolution (lower than WUXGA); also used for 1125-line video, as defined in SMPTE 274M, ATSC A/53, ITU-R BT.709; https://en.wikipedia.org/wiki/Rec._709


P.S. in perfect world Resolve should provide dealing with metadata tags in more transparent way, like this:

Image
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Oct 28, 2019 1:09 am

I think that main troublemakers here are Wide gamut monitors and color managed macOS QuickTime player. Corporations always force people to spend money for something new and partially useless, so step by step hardware moved to wide gamut/HDR displays. But video color management is not widely established and not standardized yet. It may take another 10 years until until Windows will add system wide color management for video, and until Apple realize that it probably reads and transforms video gamma in wrong way and will fix it.
My personal opinion is that in current real life situation you better get modern monitor with real 100% sRGB coverage. I will produce the most accurate result for video without any additional useless color management.
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Offline

Wayne Steven

  • Posts: 2400
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Oct 28, 2019 2:40 am

Does BM have a step by step guide to all this stuff? A best practices guide and faq built into Resolve would be a bonus. People expect people to go on line too much now, but built in is a lot faster and more convenient. I was planning to air gap my production system in order to preserve commercial privacy.
Often people deceive themselves so much they do not understand, even when the truth is explained to them.
Offline

Supermachoalpha

  • Posts: 24
  • Joined: Tue Oct 16, 2018 2:05 pm
  • Real Name: Nate Game

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Oct 28, 2019 6:29 am

Dmitry, is there a way to change the tags in Resolve? How would I be able to change them to match the same viewing result in Quicktime as the one in the Resolve viewer? By the way, thanks for being so helpful in looking into this situation.
Offline
User avatar

Dmitry Shijan

  • Posts: 964
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Oct 28, 2019 6:42 am

Supermachoalpha wrote:Dmitry, is there a way to change the tags in Resolve? How would I be able to change them to match the same viewing result in Quicktime as the one in the Resolve viewer? By the way, thanks for being so helpful in looking into this situation.


Tags are added automatically by Resolve based on selected Timeline settings.
I have no idea if there is a way to change tags in already rendered mp4 file. Some metadata editor for mp4 files may be useful.

Set YRGB (non color managed) Timeline to Rec709(Scene) as described in the first post viewtopic.php?f=21&t=101253#p560853

If you want to (almost) match QuickTime player "look" in Resolve viewer, check “Use Mac Display Color Profiles for Viewers” in DaVinci Resolve Preferences->System->General

If you want to match VLC player "look" in Resolve viewer, uncheck “Use Mac Display Color Profiles for Viewers” in DaVinci Resolve Preferences->System->General
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/
Next

Return to DaVinci Resolve

Who is online

Users browsing this forum: Majestic-12 [Bot] and 50 guests