Final Explanation of Gamma and Color Shift Problems

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Aug 07, 2020 10:50 pm

So, has the new Apples this week, and new resolve, made any progress here, or in the HDR side?
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline
User avatar

waltervolpatto

  • Posts: 10502
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Aug 08, 2020 5:01 pm

Wayne, Apple should simply adhere to the existing standards certified by SMPTE and BTU.

That is it.

Very simple.
W10-19043.1645- Supermicro MB C9X299-PGF - RAM 128GB CPU i9-10980XE 16c 4.3GHz (Oc) Water cooled
Decklink Studio 4K (12.3)
Resolve 18.5.1 / fusion studio 18
GPU 3090ti drivers 512.59 studio
Offline
User avatar

dariobigi

  • Posts: 566
  • Joined: Thu Aug 23, 2012 4:52 am
  • Location: New York City

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Aug 08, 2020 8:56 pm

waltervolpatto wrote:Wayne, Apple should simply adhere to the existing standards certified by SMPTE and BTU.

That is it.

Very simple.
Amen! But since I’m not religious... +1000


HP z840 - (2) 14 Core 2.6GHz - 64GB RAM - 2 RTX 2080Ti - SSD 8TB RAID - Win 10 Pro 1809 - DR 15.3 Studio - Fusion 9.0.2 Studio - BMD Mini Panel - FSI CM250 - Panny BT300
Dario Bigi, Colorist
http://dariobigi.com
New York
C: 1-646-436-4600

HPz840 - 2x 14 Core 2.6GHz - 64GB RAM
Windows 10 Pro 21H2
2x RTX 2080Ti (SD 472.84)
Resolve Studio 17.4.6
Decklink 4K Extreme (12.2.2)
FSI CM250
Panasonic FZ1000
Mini Panel
Offline
User avatar

Jamie LeJeune

  • Posts: 2012
  • Joined: Tue Feb 26, 2013 4:33 am
  • Location: San Francisco

Re: Final Explanation of Gamma and Color Shift Problems

PostSat Aug 08, 2020 10:59 pm

Brad Hurley wrote:Couldn't you just change your reference monitor to gamma 2.2?
In my case it doesn't make sense to set up for 2.2 because:

1. Most of my work is primarily going to either film festival theaters or PBS Broadcast rather than direct to YouTube so...

2. I have my monitors and my room and my bias lighting all dialed in together to the recommended specs for BT1886 (AKA REC709 at 2.4 gamma)

There is also stuff that goes to Netflix/iTunes/Amazon, but they don't want 2.2 gamma either

If I was only grading for web deliverables, I would use a different setup. But for my work, sticking to the traditional standard is what makes most sense and then using CST + correct NCLC tagging for any outputs destined for the web or mobile devices.
www.cinedocs.com
http://www.imdb.com/name/nm4601572/
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Aug 09, 2020 12:09 am

waltervolpatto wrote:Wayne, Apple should simply adhere to the existing standards certified by SMPTE and BTU.

That is it.

Very simple.


That's part of what I'm saying, but there are more issues out there than Apple.............

Extending ACES, and replacing all other systems with one single coherent system with hard to stuff up algorithm and code samples, maybe a solution. Wouldn't you agree?
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline
User avatar

waltervolpatto

  • Posts: 10502
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Aug 09, 2020 12:53 am

Wayne Steven wrote:
waltervolpatto wrote:Wayne, Apple should simply adhere to the existing standards certified by SMPTE and BTU.

That is it.

Very simple.


That's part of what I'm saying, but there are more issues out there than Apple.............

Extending ACES, and replacing all other systems with one single coherent system with hard to stuff up algorithm and code samples, maybe a solution. Wouldn't you agree?


For Example, in order to keep the quality of signal in a normal distribution pipeline, ACES require too big of a file (very inefficient for transmission/end delivery).

you cannot envision a beautiful a system that at the end cannot deliver the final product to the audience.
W10-19043.1645- Supermicro MB C9X299-PGF - RAM 128GB CPU i9-10980XE 16c 4.3GHz (Oc) Water cooled
Decklink Studio 4K (12.3)
Resolve 18.5.1 / fusion studio 18
GPU 3090ti drivers 512.59 studio
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostSun Aug 09, 2020 6:04 am

With a technical mindset, I'm probably not saying as big a file as you are, but I'm not saying a small, small file. One of the benefits of higher precision, is it comes at relatively little extra when compressed. For instance, 16 bit, clean, compresses a lot better than 8 bit, resulting in a file size not twice as big. What takes up a lot of space is compressing deviations from the approximation. The increase in size of the approximation from extra precision is not the biggest concern. So, here not talking about low compressed work files, but locking down precision in highly compressed distribution files. With my ideas you can cut out a subgamut in the ACES space, and use that, the precision is that it exactly maps back into ACES space if you want to do anything with it, it transform it to a displays custom subspace within ACES, so precision losses are minimal and reprocessing quality maximal. You can even make a non linear stepped space, where only the precise distribution levels are transmitted, and those levels either match the bit depth precision, or have a lookup table and ramping curve schemes when not matching, to remap any differences precisely. For player processing it maps back into ACES space precision. So, you might say, use 4 bits less in the playback distribution file, which is recovered at the receiver. Think of it as a much better alternative to log, where the values vary from a linear distribution, producing error in processing. I said an consumer distribution extension to ACES, not present ACES. Is there something else I'm missing Walter?

I don't see why people have to be against rectifying things, as long as it is the last time, which is where a flexible define your own scientific industrial use version comes in, where you can make a custom version of the new consumer version easily, if anything was accidentally missed or I'll defined. Even if there were aliens, and they came down to the planet, you could define a format to film perfectly for them and us at the same time without a new gamut. In matter of fact, the structures I. Looking to can define a format that every living creature on the planet could see how they normally see, at the same time. In the 1980's I even defined a reflection value into my design, which is pretty useless for normal depiction in hind sight, but snazzy effect to look at objects in a video game that were you and your background in colour. It turns out, there is actually a different display type that can do that.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline
User avatar

waltervolpatto

  • Posts: 10502
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Aug 10, 2020 12:23 am

Get a EXR 16 bit aces, compress it enough to toy pass thru a 30mbit pipe ( a generous one for home entertainment) then try to do anything with it.
I bet it is garbage.

Having a subset, it is technically equivalent to have a smaller color space, like 709 to start with.
Distributing gamma encoded, it means that in the file similar code values have similar impact on the image and therefore less susceptible to compression artifacts.

In a distribution system you must place the burden of the coding/compressing at the beginning of the chain where it is done once and leave the end result to be fast in delivery the final image: if I have to process 60 frames a second, I cannot put a full graphic card in every TV out there.

and think about it: the first thing the TV will do is to get the ACES signal, decode the compression, then deciding how to map the colorimetry to the panel that is NOT ACES... you lose all control over how the image is formed.

Better have a standard
Color to that standard
Delivery that standerd
Have TV that show the content on a panel that adhere to that standard.

It works. It has for more than a century.
W10-19043.1645- Supermicro MB C9X299-PGF - RAM 128GB CPU i9-10980XE 16c 4.3GHz (Oc) Water cooled
Decklink Studio 4K (12.3)
Resolve 18.5.1 / fusion studio 18
GPU 3090ti drivers 512.59 studio
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Aug 10, 2020 2:09 am

waltervolpatto wrote:Better have a standard
Color to that standard
Delivery that standerd
Have TV that show the content on a panel that adhere to that standard.

It works. It has for more than a century.


You are saying, exactly, what I am saying.

The reality, is we have panels that don't map to various standards. A good consumer set can go under in one area, and over in another. The same for cameras too. Unless you do a version of each file for the possibly tens of thousands of panels out there, you need to map in the display to it's performance limits, which is probably not done very well. The proposal is for a more accurate map, by first mapping it into the. widest colour space, and using standardised transform code and algorithms developed by the standards organisation, to transform it to the displays characteristics. But, to go further, cameras don't care about our colour gamut and go beyond it. So, it's hard to preserve the increased performance, let alone whatever we do to it to push it outwards. I'm not talking about preserving ACES, I'm talking about using the colour space etc. Using a preset cutout, such as p3, rec2020/2100, 709 etc, you actually don't really use any extra space, it just maid that into the display processing space for better controlled standardised transform into the panels space, and precision for users to move it around, particularly on proposed super premium 16 bit panel. That's existing old files using the existing standards. However, to make an even better file for distribution, it's raised to 12-16 bits, an existing preset gamut preset can still be used (the files auto saved into a higher precision file, based on the original processed workflow of the file. Now. It is more like you see it when you grade it, and the user and panel looks, can push it around even more again. I have a high end set here, best for it's type I could get here. It is still rather limited. I can still push it around a lot more and it tries to do a good job. It doesn't go as horribly wrong as in the past, but, it still tops out, and steps more than it could. So 8 bit linear to 10 bit is useful, but HDR is brittle. But, preserving the camera's full performance in a cut out, plus whatever we do to it in post, is better again. But, on this size matter, a cutout could use a12 bit file might consume close to a 10bit file in data and map into a 16bit space for display processing on arrival. There is no smaller way to do it, without doing more prediction of levels, which is compression. It is no more than current standard methods and the way forwards to preserving the camera performance and grade, replacing all standards mess, more quality and flexibility for users to adjust, more market opportunities for manufactures to sell high end product to do this. We are still stuck on 10 bit panels, an 8 bit area quality initiative (while 12 bits would be better for TV/user processing).
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Aug 21, 2020 7:36 pm

Some thoughts from FilmLight about another side of this problem.
Suggestion to move everything to pure gamma 2.2 may simplify things but there is a problem with Apple (again). Apple deeply optimized macOS system wide color management to sRGB monitor gamma, so from my subjective tests calibrating monitor to pure gamma 2.2 on macOS produce strange tonal shifts.

Image
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline
User avatar

waltervolpatto

  • Posts: 10502
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Aug 24, 2020 1:46 am

you need to map in the display to it's performance limits, which is probably not done very well


no. you deliver a standard, it is responsibility of the manufacturer to make a display that adhere to the standard.
W10-19043.1645- Supermicro MB C9X299-PGF - RAM 128GB CPU i9-10980XE 16c 4.3GHz (Oc) Water cooled
Decklink Studio 4K (12.3)
Resolve 18.5.1 / fusion studio 18
GPU 3090ti drivers 512.59 studio
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Aug 24, 2020 9:53 am

+1. As I said. You have industry associations, not to sit, but to do. Get them to get together and take a well written submission to a meeting with Apple, to change everything. Stating that membees are trying to assure quality in their products.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline
User avatar

waltervolpatto

  • Posts: 10502
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Aug 24, 2020 10:29 pm

Wayne Steven wrote:+1. As I said. You have industry associations, not to sit, but to do. Get them to get together and take a well written submission to a meeting with Apple, to change everything. Stating that membees are trying to assure quality in their products.

That does not make sense to me: there is a standard. Apple decided to do pretty much what they please.

If a master does not play well on their devices, sorry not sorry, it’s Apple problem.
W10-19043.1645- Supermicro MB C9X299-PGF - RAM 128GB CPU i9-10980XE 16c 4.3GHz (Oc) Water cooled
Decklink Studio 4K (12.3)
Resolve 18.5.1 / fusion studio 18
GPU 3090ti drivers 512.59 studio
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Aug 25, 2020 2:20 am

I said that, to get Apple to change their ways.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Vit Reiter

  • Posts: 952
  • Joined: Mon Sep 04, 2017 5:36 pm

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Aug 25, 2020 9:23 am

waltervolpatto wrote:That does not make sense to me: there is a standard. Apple decided to do pretty much what they please.
If a master does not play well on their devices, sorry not sorry, it’s Apple problem.
:D :D :D
Apple makes computer displays, not displays for video signal reference monitoring. You can make standard video on Apple computers. Monitoring has to be watching on corresponding displays if you need accurate previews.
DaVinci Resolve 18.6.6 Studio (macOS Monterey 12.7.4)
Mac Pro 2013, AMD FirePro D700, 64GB RAM

Film Editor, Colorist, DIT, Datalab technician
linkedin.com/in/vít-reiter-film-editor
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Aug 25, 2020 9:53 am

It is just sstrange to separate things like this in modern world. I agree with FilmLight position from some point of view - there are just millions types of different displays and devices and no one cares anymore if it "professional" display or not.
People just watch some web stuff on smartphones or some online streaming service content on TV or some movie from blurays/dvds/torrents/HDD players, or just use computer and computer large monitor for everything. People collaborate remotely and usually work on same project form different places on different monitors and systems.
Modern $300 "consumer" display may give better quality than legacy $3000(0) "professional". "Consumer" display may be calibrated same as "professional". Consumer display may have same IPS or OLED panel as professional display but just have less options and less rugged body.

I guess image and video world should be unified somehow to one new single standard (but i don't know how). I want to back in time and explain developers in 1990s - don't do things like this. Don't do that multiple Rec601/Rec709/sRGB/g2.2/g2.4/g1.8 (hello Apple) separation. Join together and define one single gamma and color space for ANY image/video related content and for ANY display. Because if you don't - future generations will struggle with endless gamma shifts, and it will be no way back because huge amount of sRGB images and Rec709 videos content will be already generated and it will be too late and impossible to change something globally in future.
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline
User avatar

waltervolpatto

  • Posts: 10502
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Aug 25, 2020 6:18 pm

Dmitry Shijan wrote:It is just sstrange to separate things like this in modern world. I agree with FilmLight position from some point of view - there are just millions types of different displays and devices and no one cares anymore if it "professional" display or not.
People just watch some web stuff on smartphones or some online streaming service content on TV or some movie from blurays/dvds/torrents/HDD players, or just use computer and computer large monitor for everything. People collaborate remotely and usually work on same project form different places on different monitors and systems.
Modern $300 "consumer" display may give better quality than legacy $3000(0) "professional". "Consumer" display may be calibrated same as "professional". Consumer display may have same IPS or OLED panel as professional display but just have less options and less rugged body.

I guess image and video world should be unified somehow to one new single standard (but i don't know how). I want to back in time and explain developers in 1990s - don't do things like this. Don't do that multiple Rec601/Rec709/sRGB/g2.2/g2.4/g1.8 (hello Apple) separation. Join together and define one single gamma and color space for ANY image/video related content and for ANY display. Because if you don't - future generations will struggle with endless gamma shifts, and it will be no way back because huge amount of sRGB images and Rec709 videos content will be already generated and it will be too late and impossible to change something globally in future.


but it is: ITU BT1886. It is defined goddamit.
W10-19043.1645- Supermicro MB C9X299-PGF - RAM 128GB CPU i9-10980XE 16c 4.3GHz (Oc) Water cooled
Decklink Studio 4K (12.3)
Resolve 18.5.1 / fusion studio 18
GPU 3090ti drivers 512.59 studio
Offline
User avatar

dariobigi

  • Posts: 566
  • Joined: Thu Aug 23, 2012 4:52 am
  • Location: New York City

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Aug 25, 2020 7:15 pm

Dmitry Shijan wrote:It is just sstrange to separate things like this in modern world. I agree with FilmLight position from some point of view - there are just millions types of different displays and devices and no one cares anymore if it "professional" display or not.
People just watch some web stuff on smartphones or some online streaming service content on TV or some movie from blurays/dvds/torrents/HDD players, or just use computer and computer large monitor for everything. People collaborate remotely and usually work on same project form different places on different monitors and systems.
Modern $300 "consumer" display may give better quality than legacy $3000(0) "professional". "Consumer" display may be calibrated same as "professional". Consumer display may have same IPS or OLED panel as professional display but just have less options and less rugged body.

I guess image and video world should be unified somehow to one new single standard (but i don't know how). I want to back in time and explain developers in 1990s - don't do things like this. Don't do that multiple Rec601/Rec709/sRGB/g2.2/g2.4/g1.8 (hello Apple) separation. Join together and define one single gamma and color space for ANY image/video related content and for ANY display. Because if you don't - future generations will struggle with endless gamma shifts, and it will be no way back because huge amount of sRGB images and Rec709 videos content will be already generated and it will be too late and impossible to change something globally in future.
If I had a time machine I’d do better things with it. They are ignoring a global standard. They’re A$$#ol€s.


Sent from my iPhone using Tapatalk
Dario Bigi, Colorist
http://dariobigi.com
New York
C: 1-646-436-4600

HPz840 - 2x 14 Core 2.6GHz - 64GB RAM
Windows 10 Pro 21H2
2x RTX 2080Ti (SD 472.84)
Resolve Studio 17.4.6
Decklink 4K Extreme (12.2.2)
FSI CM250
Panasonic FZ1000
Mini Panel
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Aug 26, 2020 10:12 am

They have nothing to follow as there are no approved headers for BT.1886.
In pure OSX environment you can actually have all grading formats behaving fine (although some work needs to be doe as well). It's when you go outside (youtube etc). when everything breaks, so is it Apple's fault?

Where is the header in h264/5 to say file is BT.1886? It's not Apple who needs to introduce it.

Image

Apple can later choose to properly set/read it, but first it has to exist (like others do).
Atm. when you grade file as Rec.709 2.4 gamma you have no way to tell other tools that file has these parameters. If you stick to MOV and pure OSX environment then you can, but we need wider support.
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Aug 26, 2020 4:08 pm

In addition to sRGB vs other image/video gammas especially insane sounds tech explanations like these:
Rec. 601 (aka ITU.601, BT.601, or SMPTE 170M) and Rec. 709 (aka ITU.709 or BT.709). SMPTE 240M (almost the same as Rec. 709)
Why why the hell someone somewhere decide to create something "almost the same as Rec. 709" ? Why they can't provide single naming for these things? (Is it Rec. 601, or ITU.601, or BT.601, or SMPTE 170M? I don't know give me all of them!)
Why industry just can't start use from early days sRGB color space/gamma for both digital image and digital video? How it all came to those insane gamma variations, and sRGB for image vs Rec.709 video variations?
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Aug 26, 2020 7:41 pm

Your questions have answers.
Fact that so many (not really actually) do exist is not the problem here. Problem is with lack of those which are not there and currently used the most :)
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Aug 27, 2020 3:39 am

If it isn't Apple's fault, how come it's Apple people who are having issues with. Even their Movie format and software, there were big issues displaying native files. Generally, I had a better time viewing things untouched bynan Apple product. I know that was years ago, but this is just a continuation of an old trend. The two suggestions I've given, are the answer. A new standard with conformity code and algorithms, every existing file can be reheadered for (or refiled, or recontainerd, for) and Apple conforming.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Aug 27, 2020 11:32 am

Good luck watching today HDR, SDR etc videos untocuhed on mixture of near Rec.709 or near P3 gamut (or anywhere in between) screens. Good luck :D

Can you also explain me please what Apple is not confirming to? Because I don't really know.
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Aug 27, 2020 3:52 pm

Yes, I'd rather watch it not on an Apple product.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

jacobsch

  • Posts: 18
  • Joined: Mon Aug 13, 2018 2:24 pm
  • Real Name: Jacob Schindler

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Sep 21, 2020 5:28 pm

A few months ago, using the Rec 709 (Scene) solution, coupled with Cosmin Hodiș-Mîndraș' iMac LUT featured on page 2 of this thread, I (we) had a simple solution for WYSIWYG grading on the GUI Color Viewer on a 5K iMac. Simple and perfect for my needs and a years-long thorn in my side removed.

Now for some reason that has stopped working. I'm on 16.2.4 and setting up my Color Management tab in the same way: Davinci YRGB / Rec 709 (Scene) / Rec_709_Scene_to_Mac_21 3D Color Viewer LUT. But when I render I'm getting the dreaded washed-out gamma bug as before.

Playing around with things, it looks like the 3D Color Viewer LUT is having no effect on the GUI Viewer. Whatever LUT I choose in that menu, the viewer looks the same... What has changed? How can I get back to using the Rec_709_Scene_to_Mac_21 3D Color Viewer LUT?

Many thanks in advance.
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Sep 21, 2020 6:24 pm

If you only care about how it looks on Mac then simply use Rec.709-A. No LUTs are needed.
Offline

jacobsch

  • Posts: 18
  • Joined: Mon Aug 13, 2018 2:24 pm
  • Real Name: Jacob Schindler

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Sep 21, 2020 6:42 pm

That worked a treat. Just about perfect reproduction, very slight dip in sturation, but 97% there...

I would suggest you make this clearer in the updates on page 1. The vast majority of people grading in Resolve are grading simple SDR deliverables for Vimeo and YouTube and using the Color page Viewer.

Thanks a lot for the help :-)
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Sep 21, 2020 6:46 pm

There should be no drip. It meant to be 100% the same.
Not my post on the 1st page.
This method is only good for Apple environment, so it's not a REAL solution anyway (as there is simply no one atm.).
Offline

jacobsch

  • Posts: 18
  • Joined: Mon Aug 13, 2018 2:24 pm
  • Real Name: Jacob Schindler

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Sep 21, 2020 6:59 pm

Thanks Andrew. Yes I understand it's a hack, but all I need is a hack that works for me. I upload to Vimeo and the vast majority of my audience would be on Apple machines or devices.

See the stills below comparing Resolve GUI with QT player and Vimeo. It's super subtle, but there's a slight loss of sat, her cheeks are a little less warm in QT player.

In passing, what happens when using Rec709-A when someone is on Windows or Android? Will it look completely wrong (i.e. super over saturated for example) or just slightly different?

GUI vs QT
Image

GUI vs Vimeo
Image

Thanks!
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Sep 21, 2020 7:48 pm

Mainly gamma will be different.

Are you sure you have no preview LUTs enabled?

If you see any difference then it's new(old) issue in Resolve. Every update seems to be jumping between perfect match or 99% match :) Been like this for long time. 1 good, one almost good and so on :)
Are you on latest version?
Offline

Tim Cromar

  • Posts: 26
  • Joined: Mon Oct 21, 2013 9:06 am

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Oct 26, 2020 1:28 pm

I have read the threads and watched the video, and understand what should be correct procedure, but talking of uploading to YouTube:

Even if I tag my output files with my pipeline/timeline for an 'iMac > Web' workflow, e.g. I tried 12-13-1 (P3D65, sRGB, rec709), YouTube just transcoded the file straight into 1-1-1 (rec709,rec709(camera), rec709). The gamma and colours were way off my iMac P3 display.

Is the only option now to tag the file 'incorrectly' as 1-1-1 for web-only delivery to streaming sites viewed on Apple devices? :?
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Oct 26, 2020 5:21 pm

For wide color space uploads try to use Rec2020 with Rec709 gamma. Rec2020 with different HDR gammas also probably may be recognized by YouTube (but i didn't test any of these).
Rec2020 was designed to be the universal futureproof wide color space for video delivery. P3 used in Cinema theaters and Apple displays. P3 is not for YouTube.
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Oct 26, 2020 9:36 pm

Tim Cromar wrote:I have read the threads and watched the video, and understand what should be correct procedure, but talking of uploading to YouTube:

Even if I tag my output files with my pipeline/timeline for an 'iMac > Web' workflow, e.g. I tried 12-13-1 (P3D65, sRGB, rec709), YouTube just transcoded the file straight into 1-1-1 (rec709,rec709(camera), rec709). The gamma and colours were way off my iMac P3 display.

Is the only option now to tag the file 'incorrectly' as 1-1-1 for web-only delivery to streaming sites viewed on Apple devices? :?


Not sure what gamuts YouTube properly passes through for SDR- probably only Rec.709, so unless you're doing HDR stick to Rec.709. Not many people outside Macs have screen around P3 gamut anyway.
If you are doing it for yourself forget about crappy YouTube.
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostMon Oct 26, 2020 9:41 pm

Dmitry Shijan wrote: P3 used in Cinema theaters and Apple displays. P3 is not for YouTube.


Not true at all. P3 is not anymore for DCI usage only. It's new Rec.709 for premium delivery. Correct approach is to grade to P3, but then export it inside Rec.2020 (due to Rec.2020 been recognised as a "transmission protocol").
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Oct 27, 2020 4:31 am

Your premium TV's are built towards P3 these days, and some more. My TV goes beyond P3, rather nice when I stretch the content.
Last edited by Wayne Steven on Tue Oct 27, 2020 2:49 pm, edited 2 times in total.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Tim Cromar

  • Posts: 26
  • Joined: Mon Oct 21, 2013 9:06 am

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Oct 27, 2020 9:14 am

Andrew Kolakowski wrote:If you are doing it for yourself forget about crappy YouTube.

Can we can afford to be so dismissive of this platform? Many professionals must now include YouTube in their set of deliverables. We have to look ahead to find a workflow that plays nicely with these new 'standards'.

Working with sRGB (and by extension Apple's Display P3) as a legitimate space is acknowledged in this webinar – though a Resolve solution is less clear.


Right now I still feel unsure of precisely what a colour/gamma consistent 'Grade-on-an-Apple-device > YouTube' workflow should look like in Resolve – that doesn't bail out by using incorrect 1-1-1 NCLC tagging.

*UPDATE*
I uploaded tests to YouTube grading in various colour output colour spaces & gamma. Project set to Rec709-A. Outputs changed in final CST node. Matching tags added in Resolve delivery page:

12-31-1 (P3D65, sRGB2.1, 709)
1-13-1 (709, sRGB2.1, 709)
9-1-9 (2020, 709(1.96), 2020)

Results:

1) Any Apple generated file that was uploaded/tagged as a x-1-x Gamma (1.96) displayed correctly throughout the pipeline: Viewer > QTX > YouTube
2) Once uploaded to YouTube – and downloaded (in YouTube studio as .mp4) reveal ALL are now tagged as 1-1-1.
3) Declaring the explicit correct tags in the delivery page to match output space/gamma results in shifts.
4) Declaring 'Same as project' tags in the delivery page results in a 1-1-1 output – and results in correct render.

Not sure what this means anymore! ;) Unless I conclude that the only way to grade on Apple device > YouTube without gamma issues is to use Timeline to Rec709-A (as has been suggested in first post) and accept 'incorrect' 1-1-1 tagging. Or Grade to proper Rec709/2.4 and just try to explain the shift to clients when uploaded to YouTube and converted from 1-2-1 to 1-1-1.
Offline
User avatar

waltervolpatto

  • Posts: 10502
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Oct 27, 2020 6:31 pm

4) Declaring 'Same as project' tags in the delivery page results in a 1-1-1 output – and results in correct render.


Only if you’re project is setup for 709/2.4, if you have any other setting like AlexaLogC it will be wrong.
W10-19043.1645- Supermicro MB C9X299-PGF - RAM 128GB CPU i9-10980XE 16c 4.3GHz (Oc) Water cooled
Decklink Studio 4K (12.3)
Resolve 18.5.1 / fusion studio 18
GPU 3090ti drivers 512.59 studio
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostTue Oct 27, 2020 8:18 pm

Tim Cromar wrote:
Andrew Kolakowski wrote:If you are doing it for yourself forget about crappy YouTube.

Can we can afford to be so dismissive of this platform? Many professionals must now include YouTube in their set of deliverables. We have to look ahead to find a workflow that plays nicely with these new 'standards'.



If you have such a need and targeting Mac users then grade to Rec.709-A. Not a proper solution, but best you can do atm.

There is no other way until youtube transcodes are improved to take color tags into account.
Vimeo should be better in this aspect as they seem to normalise everything based on the source tagging.
Offline

Tim Cromar

  • Posts: 26
  • Joined: Mon Oct 21, 2013 9:06 am

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Oct 28, 2020 9:48 am

Andrew Kolakowski wrote:If you have such a need and targeting Mac users then grade to Rec.709-A. Not a proper solution, but best you can do atm. … There is no other way until youtube transcodes are improved to take color tags into account.
Vimeo should be better in this aspect as they seem to normalise everything based on the source tagging.


Thanks for all your advice. Yes, grading in Rec709-A is a solution of sorts, but it puts the emphasis on changing the grading setup to a 'non-standard' – while letting YouTube off the hook for being non-compliant to the correct tags.

I've done some more tests today and have found an acceptable compromise for now (for my needs) that results in only a very slight saturation shift when YouTube re-encodes to 1-1-1. I encourage other Mac Display P3 users to test and report success/failures:

***NB I know this is NOT a calibrated end-to-end Rec709 2.4 setup for broadcast! But YouTube is of course, a new broadcast in a way…***

Setup:

iMac Retina 2019, Mojave, DR 16.2.7
Display Profile: Display P3

Resolve:
Mac colour management = Doesn't matter (in this case)
Project Timeline: Colour = P3 D65 Gamma = sRGB
Render tags = Colour & Gamma = Same as project

I then render to ProRes or H265 Main10(bit) for YouTube upload. File is tagged correctly 12-13-1 (P3 D65, sRGB, 709)

All viewers and renders opened in QTX, MPV Player, even VLV match (with almost imperceptible hue shifts).
I feel this is a more satisfying and justifiable workflow as the grading setup itself isn't set to to some non-standard colour space (Rec709-A).

Your mileage may, of course vary! ;)
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Oct 28, 2020 10:12 am

Tim Cromar wrote:
Andrew Kolakowski wrote:If you have such a need and targeting Mac users then grade to Rec.709-A. Not a proper solution, but best you can do atm. … There is no other way until youtube transcodes are improved to take color tags into account.
Vimeo should be better in this aspect as they seem to normalise everything based on the source tagging.


Thanks for all your advice. Yes, grading in Rec709-A is a solution of sorts, but it puts the emphasis on changing the grading setup to a 'non-standard' – while letting YouTube off the hook for being non-compliant to the correct tags.



This is the problem- it's only for Mac users. No other real solution until there is a new widely recognised tag for 2.4 gamma.
You can always grade to 2.4 and at the end before export for Mac have it converted to Rec.709-A gamma.
Offline

Tim Cromar

  • Posts: 26
  • Joined: Mon Oct 21, 2013 9:06 am

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Oct 28, 2020 10:42 am

Andrew Kolakowski wrote:
Tim Cromar wrote:
Andrew Kolakowski wrote:You can always grade to 2.4 and at the end before export for Mac have it converted to Rec.709-A gamma.


You mean tag gamma as Rec.709-A? This results in a gamma-shifted file.

Or convert to Rec.709-A in CST?
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Oct 28, 2020 10:44 am

You have to convert it to REC.709-A, not just tag.

As you said- YouTube will tag about everything (except HDR) as 1-1-1. For such a tag Mac color engine will always use 1.96 based inverse math, so anything originated with different gamma will ALWAYS look wrong.
2.4->converted to 1.96-> reverse 1.96 = correct preview
2.4 (or anything else) -> tagged as 1-1-1 -> always reversed over 1.96 = never correct
Offline

Tim Cromar

  • Posts: 26
  • Joined: Mon Oct 21, 2013 9:06 am

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Oct 28, 2020 10:55 am

Andrew Kolakowski wrote:You have to convert it to REC.709-A, not just tag.


Grading in Rec709 2.4 timeline

w/ CST output set to Color: Rec.709, Gamma: Rec.709-A

Tagged: Color: Rec.709, Gamma: Rec.709-A

Results in gamma shift. Did I miss something? (Use Mac color management either ON or OFF produces varying gamma shifts in this case).
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Oct 28, 2020 11:05 am

Graded 2.4, converted to Rec.709-A, tagged 1-1-1 should work.
Managed preview definitely has to be ON in Resolve.

Just done quick test. Not even used CST, but set Timeline Color Space to Rec.709/2.4 and Output Color space to Rec.709/Rec.709-A gamma. Rendered file looks correct in OSX environment.
Last edited by Andrew Kolakowski on Wed Oct 28, 2020 11:17 am, edited 1 time in total.
Offline

Tim Cromar

  • Posts: 26
  • Joined: Mon Oct 21, 2013 9:06 am

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Oct 28, 2020 11:15 am

Image

As you describe, but not working here. :?
Offline

Andrew Kolakowski

  • Posts: 9209
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Oct 28, 2020 11:22 am

Not sure why.
Just try setting project itself this way.
Attachments
Screenshot 2020-10-28 at 12.21.25.png
Screenshot 2020-10-28 at 12.21.25.png (150.8 KiB) Viewed 13569 times
Offline

Tim Cromar

  • Posts: 26
  • Joined: Mon Oct 21, 2013 9:06 am

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Oct 28, 2020 11:25 am

Yes, I just tried that and confirm the gamma shift is gone. But this is using Davinci Color Managed system. I prefer Davinci YRGB. Frustrating!
Offline
User avatar

waltervolpatto

  • Posts: 10502
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Final Explanation of Gamma and Color Shift Problems

PostWed Oct 28, 2020 4:01 pm

Andrew Kolakowski wrote:Not sure why.
Just try setting project itself this way.


because you are actually doing the transformation at the setting level, if you are in non managed you need to put a ofx
W10-19043.1645- Supermicro MB C9X299-PGF - RAM 128GB CPU i9-10980XE 16c 4.3GHz (Oc) Water cooled
Decklink Studio 4K (12.3)
Resolve 18.5.1 / fusion studio 18
GPU 3090ti drivers 512.59 studio
Offline

Saneuskis

  • Posts: 28
  • Joined: Wed Jul 22, 2020 3:11 am
  • Real Name: Alex Zh

Re: Final Explanation of Gamma and Color Shift Problems

PostThu Dec 03, 2020 7:17 am

Man, you did a good job! At least it gives flexibility for exporting in different formats, without loosing your actual grade! Thank you.


Joakley wrote:
Cosmin Hodiș-Mîndraș wrote:It seems that every "QuickTime gamma bug" is experienced by people expecting QuickTime player and YouTube to look like Resolve's color viewer. Those producing for broadcast are always concerned about calibrated Rec709 professional monitors, fed by UltraStudio or DeckLink cards, and care little for computer displays. Fair enough. The thing is, I really want my YouTube uploads to look as close to Resolve as possible, even if some of my productions are intended for broadcast.

So I made a few tests, exporting QuickTime files from Resolve using several timeline color spaces. The result, as shown by macOS finder, are:
  • Rec.709 Gamma 2.4 - Color profile: (1-2-1)
  • Rec.709 Gamma 2.2 - Color profile: (1-4-1)
  • Rec.709 (Scene) - Color profile: HD(1-1-1)
  • sRGB - (1-13-1)
(What 1-1-1 or 1-2-1 mean is explained in the first post of this topic).

Any of those files, once exported from Resolve, looked different in QuickTime. Once uploaded to YouTube, those files looked different as well, low contrast and desaturated. I downloaded them from YouTube and every file had the same color profile: HD (1-1-1). The same happened when I compressed the files using HandBrake, the color profile was changed to HD (1-1-1) as well.

What's quite clear at this point is the fact that YouTube ignores the source color profile, assuming 1-1-1. So, as a consequence, I have to use Rec.709 (Scene) in Resolve (because it exports 1-1-1 profiles), and color grade for it. But how to set Resolve to look like QuickTime Player? Enabling "Use Mac Display Color Profiles for Viewers" in DaVinci Resolve helped a bit, but not a lot, some shadows were crushed and definitely saturation looked different.

I then thought about creating a display LUT for Resolve, which mimics the QuickTime and YouTube output:
  • I created a 1728x1728px 8-bit HALD image using ImageMagick:
    Code: Select all
    convert hald:12 -depth 8 hald12_8bit.tif
  • I imported the image in Resolve, setting the timeline color space to Rec.709 (Scene) and frame size to 1728x1728px;
  • I rendered the movie using ProRes 4444;
  • I opened the rendered movie in QuickTime, and made a screenshot (quite tricky, because I had to display the movie unscaled, and macOS Catalina shows no options on "Option+Scaled in Display Preferences for the built-in display; luckily I have an external LG UltraFine display that I could put in an unscaled mode - 5120x2880px);
  • I cropped the screenshot and opened it in Lattice:
    Image
  • I resized the LUT to 21:
    Image
    Image
  • ... and I exported a .cube LUT.

After loading the LUT in 3D Color Viewer Lookup Table, QuickTime Player and Resolve viewer screenshots look identical, so I can confidently grade my timeline for YouTube, knowing that what I see is what I get.

But how about shots graded for broadcast, in Rec.709 2.4 timelines? Well, if broadcast is my main delivery and YouTube is second, I reversed the LUT in Lattice for a quick and dirty final "YouTube touch", essentially adding the "missing" saturation and contrast in one node just for YouTube delivery, on top of my main broadcast grade, temporarily switching the project to Rec.709 (Scene).

I'm not saying that this is a viable workflow for anybody. I'm quite sure that it works only on Apple's P3 displays; it's just an experiment, maybe a stupid thing to do, but the results are, for the first time (on my Macs) and at least for me, well, predictable again.

If anyone is interested, here is a link to the LUTs:
https://www.dropbox.com/s/0sux7nydfyk721w/Rec709_Scene_to_Mac.zip?dl=0



Wow.

I’m kinda tracking what you did...as far as creating LUTs to compensate for this shift.

A lot of work that I couldn’t pull off.

Anyway, so as far as using this LUT, you essentially just add it to a final node on your video and render?

You said the image in resolve and the QuickTime video were identical. Before rendering, did the image in resolve viewer look OVER saturated, darker, etc? To adjust for quicktime’s brightening and desaturatimg?
Offline

David Chabashvili

  • Posts: 34
  • Joined: Thu Nov 27, 2014 2:01 pm

Re: Final Explanation of Gamma and Color Shift Problems

PostFri Dec 04, 2020 4:46 pm

Informative thread for sure, but I'm on Windows machine, Resolve 17, and having quite noticeable gamma (and saturation) shifts when rendered file (mp4 or any other for that matter) is played on Macs (old and new). I think I tried all the combinations. Most close I was able to get was to set the Timeline Color Space to REC709 and tag in Advanced Settings as Gamma 2.2. But still not there, images on Macs are darker and saturated. I cant wrap my head what I'm doing wrong. I do not recall having such problem on previous versions. Any suggestions for Windows users?
PreviousNext

Return to DaVinci Resolve

Who is online

Users browsing this forum: Baidu [Spider], Bing [Bot], govind, kfriis, liza.de, pkwest, SteveW, VMFXBV and 231 guests