Apple ProRes 4444 is float or integer

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

Olivier MATHIEU

  • Posts: 94
  • Joined: Thu Aug 04, 2016 1:55 pm
  • Location: Grenoble, FRANCE

Apple ProRes 4444 is float or integer

PostWed Apr 26, 2017 9:26 am

I'm wondering about HDR/RAW caching capabilities (and also deliver) of the Apple ProRes 4x4 XQ.
In the resolve's Manual it says it fit for HDR/RAW caching like "Uncompressed 16bit - Float"
In Apple's Apple ProRes White paper it's only says that Apple ProRes 4x4 is 12bit, but no mention of Float or integer.
I've run some tests caching in Apple ProRes 4x4 is very similar than "Uncompressed 16bit - Float". In my test Caching in Apple ProRes 422 HQ clips a RAW file that caching in the two previous formats don't.

Any solid informations about that would be very appreciated !
Thanks in advance
MacOS 10.14.5
i7 8 cores 3,8 Ghz
RAM 32 Go (2800Mhz)
RADEON VII 16 Go
Resolve Studio 16
GUI 3440 x 1440
Offline

Forest Finbow

  • Posts: 31
  • Joined: Thu Mar 30, 2017 11:51 am
  • Location: Paris, France

Re: Apple ProRes 4444 is float or integer

PostWed Apr 26, 2017 10:05 am

Funny you come up with such a technical question, I've been recently wondering about that stuff too, so I'm in a perfect position to pursue the questioning...

I dug a bit in DR caches and my findings were a little surprising to me.

They go as follows (as read from DVCC cache Binary Large OBjects):

DNxHR is either 8 bit (LB up to HQ) or 16 bit (444 and HQX)
Apple prores is either 10 bits (Proxy up to 422HQ) or 16 bits (444 and XQ)
RGB is 10 bit or 16 bit.

If the BLOBs are correct, this means Resolve does not implement vanilla versions or those codecs but a custom implementation. If it was vanilla, Prores 422 and bellow should be 8 bits, DNxHR HQ should be 10 bits and ProRes 444 should be 12 bits.

My understanding of this has to do with the purpose of Resolve: digital grading.
In the digital imagery realm, 12 bit generally stands for unsigned integers while 16 bits can be either unsigned int or short float. Resolve UI specifically mentions "16 bit float", so my guess is that the 16 bit codecs are using short floats, and I assume they are signed floats, giving the ability to have negative values (the stuff that comes back from the shadows if you lift a crushed grade).

While using a short signed float is technically halving the quantification of the 16 bit linear RAW (unsigned int), I suppose that's probably nowhere near a visible issue dealing with video at max 12 bit displays and will allow you to cache stuff that yields negative values while still being able to recover them later down your grade.
Offline

Olivier MATHIEU

  • Posts: 94
  • Joined: Thu Aug 04, 2016 1:55 pm
  • Location: Grenoble, FRANCE

Re: Apple ProRes 4444 is float or integer

PostWed Apr 26, 2017 10:17 am

Thank you for your reply
Forest Finbow wrote:DNxHR is either 8 bit (LB up to HQ) or 16 bit (444 and HQX)
Apple prores is either 10 bits (Proxy up to 422HQ) or 16 bits (444 and XQ)
RGB is 10 bit or 16 bit.

How did you manage to get those infos ? very interesting indeed

Prores 422 and bellow should be 8 bits

heu 10bit, no ?

I agree that 16bit float render cache format is necessary for good rendering.
So if Apple ProRes 444 and DnxHR 444 are also 16bit float, it diminish the interest of the "uncompressed 16bit float", for me ....
MacOS 10.14.5
i7 8 cores 3,8 Ghz
RAM 32 Go (2800Mhz)
RADEON VII 16 Go
Resolve Studio 16
GUI 3440 x 1440
Offline

Forest Finbow

  • Posts: 31
  • Joined: Thu Mar 30, 2017 11:51 am
  • Location: Paris, France

Re: Apple ProRes 4444 is float or integer

PostWed Apr 26, 2017 7:23 pm

Prores 422 and bellow should be 8 bits

heu 10bit, no ?

Oh my bad, indeed, all 422 flavors of prores can support up to 10 bits), so that explains that part, nevertheless, the Apple ProRes white paper mentions 4444 and 444XQ going all the way up to 12 bits only, as you've mentioned. No 16 bits variants are specified anywhere for rgb values ( alpha channel).

As to how I got those figures, it's a bit of a reverse engineering job. I've first generated different cached medias with all the different settings offered by resolve. I've then located the XML files of my disk database project, tracked down cached medias within those files and fetching the specific attribute that stores cache informations (such as cache codec, size and bitdepth): the FieldsBlob. Then open that string as a HexText in a hexadecimal editor, and locating the corresponding bits. The name of the codec is stored as ANSI numbers (wchars) and the bitdeph as an unsigned int. Hex values give 0x08 for DNxHR up to HQ, 0x0A (which is 10 in base 10) for Prores up to HQ, and 0x10 (which is 16 in base 10) for the 444/HDR variants.
Offline

Olivier MATHIEU

  • Posts: 94
  • Joined: Thu Aug 04, 2016 1:55 pm
  • Location: Grenoble, FRANCE

Re: Apple ProRes 4444 is float or integer

PostWed Apr 26, 2017 7:47 pm

ok Thanks
MacOS 10.14.5
i7 8 cores 3,8 Ghz
RAM 32 Go (2800Mhz)
RADEON VII 16 Go
Resolve Studio 16
GUI 3440 x 1440
Offline

Andrew Kolakowski

  • Posts: 5360
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Apple ProRes 4444 is float or integer

PostWed Apr 26, 2017 8:09 pm

ProRes 10bit (8bit possible also), DNxHR 8/10bit or 12bit.
ProRes444/XQ modes meant to be 12bit.
Offline

Peter Griffo

  • Posts: 3
  • Joined: Fri Oct 04, 2013 7:36 pm

Re: Apple ProRes 4444 is float or integer

PostThu Apr 27, 2017 7:39 am

Andrew Kolakowski wrote:ProRes 10bit (8bit possible also), DNxHR 8/10bit or 12bit.
ProRes444/XQ modes meant to be 12bit.

Prores 4444 ist very often also only 10bit, XQ on the other hand nearly always 12bit.

That said, all integer, no float.

Gesendet von meinem ONEPLUS A3003 mit Tapatalk
Offline

Olivier MATHIEU

  • Posts: 94
  • Joined: Thu Aug 04, 2016 1:55 pm
  • Location: Grenoble, FRANCE

Re: Apple ProRes 4444 is float or integer

PostThu Apr 27, 2017 7:54 am

Peter Griffo wrote:Prores 4444 ist very often also only 10bit, XQ on the other hand nearly always 12bit.

That said, all integer, no float.

Thank you to share your knowledge
Could you elaborate, please
How integer Render format can preserve so high value ?
MacOS 10.14.5
i7 8 cores 3,8 Ghz
RAM 32 Go (2800Mhz)
RADEON VII 16 Go
Resolve Studio 16
GUI 3440 x 1440
Offline

Forest Finbow

  • Posts: 31
  • Joined: Thu Mar 30, 2017 11:51 am
  • Location: Paris, France

Re: Apple ProRes 4444 is float or integer

PostThu Apr 27, 2017 10:55 am

Unsigned 16 bit int values will range from 0 to 65535. Typical 16 bit RAW cameras will store pixel voltage as 16 bits values, thus Yielding 65536 levels of the dynamic range. That's your question answered: an int can store a big value, if you give it a large quantification ;-)

Since as soon as you apply a gamma correction to the signal, there is literately no real use to keep that much quantification. Most codecs will try and fit actual display capabilities, ranging from 8 bits for standard displays up to 12 bits for rare very high quality professional displays.

My understanding of why Resolve uses floats instead of int goes as follow:

Resolve has a really special use for those numbers, it's going to mess arround with them a lot. Internally it's working with signed 32 bit float numbers, each pixel value can range from -3.4^38 to + 3.4^38. No matter what number was stored by the camera, it will (very) comfortably fit in the range. Please note the appearance of negative values, this is a very interesting feature. It is useless to have negative numbers in cameras or delivery codecs, since pure black is coded by zero, and nothing is darker than pure black. Resolve will however use those numbers in calculations to avoid rounding errors (clippings) building up. It of course has the side effect to allow you to have a grade crunch or burn the picture in one node and then retrieve those pixels in an other node down the line.
At the output level however, if a pixel is coded by a value outside the range of the display it will clip. Delivery codecs such as Apple ProRes will actually not even implement signed values, all that they can store are positive integers (ranging from 0 to max quantification)

Caching within resolve is where it gets tricky. If you pick an delivery codec (any standard codec really) you will clip any numbers falling out of the display range in your cache. So for a 10 bit codec, any numbers bellow 0 and above 1023 will get clipped. Caching RAW has this drawback that not only your cached node can't have anything darker than black retrieved later, but more noticeably is that any number above 1023 is also clipped, and that is what most people will see since the RAW footage didn't feature any negative value pixel to start with.

Resolve has addressed the problem by allowing us to store float numbers in the cache. The first "codec" available for that was the 16bit float RGB. They picked 16 bit since 32 bit was a bit of an overkill and twice the size in storage space: the maximum value a camera rush can hand to resolve is 65535, and the max 32 bit float is tens of orders of magnitude bigger than that (3.4^38). They could have picked 16 bit int, since it would match the rushes, but then no negative numbers would be cachable. They could also have picked signed int, but the range then falls down to –32,768 to 32,767 and would introduce clipping once again so they picked float instead.

It turns out, the range of a half float (16 bit float) is -65504 to 65504, at the loss of some precision in the large numbers but much increased precision in the small ones, which is really where we need the most of it (the 0-1024 range).

I've found out that all labeled HDR cache codecs (444s essentially) store their values as 16 bits. So they are clearly not vanilla implementations of those codecs. I've actually tried many methods to read them independently (VLC/FFMPEG etc.) without success. I do not know however if those 16 bits are float or int, I yet have to come up with an idea to test that. However either way, I would concur with the fact that there is no real reason to cache in 16bit RGB float anymore. Even if it is integers that are stored in ProRes/DNxHR 444, with 32000+ levels of quantification I'm absolutely sure that it won't damage the footage in any noticeable way, especially if all you do is temporary caching to increase read performance!
Offline

Olivier MATHIEU

  • Posts: 94
  • Joined: Thu Aug 04, 2016 1:55 pm
  • Location: Grenoble, FRANCE

Re: Apple ProRes 4444 is float or integer

PostThu Apr 27, 2017 11:49 am

good reading you !!

Unsigned 16 bit int values will range from 0 to 65535. Typical 16 bit RAW cameras will store pixel voltage as 16 bits values, thus Yielding 65536 levels of the dynamic range. That's your question answered: an int can store a big value, if you give it a large quantification ;-)

God, i lost myself :D so obvious !
MacOS 10.14.5
i7 8 cores 3,8 Ghz
RAM 32 Go (2800Mhz)
RADEON VII 16 Go
Resolve Studio 16
GUI 3440 x 1440
Offline
User avatar

Cary Knoop

  • Posts: 1004
  • Joined: Sun Mar 12, 2017 6:35 pm
  • Location: Newark, CA USA

Re: Apple ProRes 4444 is float or integer

PostThu Apr 27, 2017 4:33 pm

Forest Finbow explained it well.

In very simple terms, the number of bits in a codec represent the resolution of a fixed range (e.g the number of steps between 0..1), while the floating 32 bit values used in DaVinci represent the resolution of a floating range (e.g the range spans beyond 0..1).

The effect is that data is never clipped!
Offline
User avatar

Jean Claude

  • Posts: 2420
  • Joined: Sun Jun 28, 2015 4:41 pm
  • Location: France

Re: Apple ProRes 4444 is float or integer

PostThu Apr 27, 2017 4:43 pm

Never done the test so it must be tedious: test with a binarize effect: threshold by threshold ... Pure theory ... :oops:
Windows 10 PRO X64 1903 | DaVinci Resolve Studio 16.0.0B.033 | Fusion Studio 16.0.beta 28 build 28 | Decklink Studio 4K 6G | Desktop Video 11.2.00 | X99-A II, i7-6850K | RTX 2080Ti | Nvidia 430.86 NSD
Offline

Olivier MATHIEU

  • Posts: 94
  • Joined: Thu Aug 04, 2016 1:55 pm
  • Location: Grenoble, FRANCE

Re: Apple ProRes 4444 is float or integer

PostThu Apr 27, 2017 4:48 pm

What's confusing me, is what the maximum values represent.
for a 8bit file ou 10 bit, their respective maximum value (255 - 1023) represent the same Y/R/V/B.
➧ Same roof
on other hand, the max 10bit file and the maximum of 16bit file seems not represent the same Y/R/V/B
➧ higher roof

Am I wrong ?
And I can perfectly understand that a 16bit integer format can clip somme values ....

Thanks again for all the explanations.
MacOS 10.14.5
i7 8 cores 3,8 Ghz
RAM 32 Go (2800Mhz)
RADEON VII 16 Go
Resolve Studio 16
GUI 3440 x 1440
Offline
User avatar

Cary Knoop

  • Posts: 1004
  • Joined: Sun Mar 12, 2017 6:35 pm
  • Location: Newark, CA USA

Re: Apple ProRes 4444 is float or integer

PostThu Apr 27, 2017 5:17 pm

Olivier MATHIEU wrote:What's confusing me, is what the maximum values represent.
for a 8bit file ou 10 bit, their respective maximum value (255 - 1023) represent the same Y/R/V/B.
➧ Same roof
on other hand, the max 10bit file and the maximum of 16bit file seems not represent the same Y/R/V/B
➧ higher roof

Am I wrong ?
And I can perfectly understand that a 16bit integer format can clip somme values ....

Thanks again for all the explanations.

I think it is easiest to think of a video having levels between 0 and 1 or 0 to 100 IRE, or black and white (and sometimes levels can even go beyond those if video levels are used, e.g. WTW, BTB).

The bit depth is the resolution, the number of discrete steps between this 0 to 1. A higher bit depth does not guard against clipping, it just has finer steps between black and white.

Floating point helps in that the absolute values literally 'float' beyond 0 to 1, this does guard against clipping.

Imagine three videos:

video A = 8 bit, video levels
video B = 8 bit, data levels
video C = 10 bit, video levels

Resolve would map all of those between 0 and 1. C would just have a higher resolution, more steps to define the luminance and color. Also because video A and C have video levels they could have negative and larger than one values (although BTB is practically not used in cameras). However nothing is clipped because the values are represented as 32 bit floats.
Offline

Olivier MATHIEU

  • Posts: 94
  • Joined: Thu Aug 04, 2016 1:55 pm
  • Location: Grenoble, FRANCE

Re: Apple ProRes 4444 is float or integer

PostThu Apr 27, 2017 6:48 pm

I know all about data level : video or full
By higher roof I didn't mean different data level.

My test
I import F65 Raw and put it in an timeline
RAW_Project_ISO800.png
Exposure : ISO 800
RAW_Project_ISO800.png (37.08 KiB) Viewed 3169 times

In the CameraRAW I push far left the exposure to 160
RAW_clip_ISO160.png
Exposure : ISO 160
RAW_clip_ISO160.png (34.31 KiB) Viewed 3169 times

and I push it far right to 3200
RAW_clip_ISO3200.png
Exposure : ISO 3200
RAW_clip_ISO3200.png (27.38 KiB) Viewed 3169 times

Then in the Color Wheels I drop the Gain to 0.19
RAW_ISO3200_Gain019.png
Drop the Gain to 0.19
RAW_ISO3200_Gain019.png (24.93 KiB) Viewed 3169 times

right-clic on the clip thumbnail ➧ Render cache clip source ➧ On
Cache file format is Apple ProRes 4444 XQ
The parade doesn't move
I change the cache file Format to Apple ProRes 422 HQ
Rendercache_SourceClip_422HQ.png
Rendercache the source clip in Apple ProRes 422 HQ
Rendercache_SourceClip_422HQ.png (6.9 KiB) Viewed 3169 times

The render cache file has clipped a lot of value

As Forest Finbow explained it, it's probably not a vanilla version of the Apple ProRes 4444XQ while all Apple ProRes are integer. It should be labeled differently.
MacOS 10.14.5
i7 8 cores 3,8 Ghz
RAM 32 Go (2800Mhz)
RADEON VII 16 Go
Resolve Studio 16
GUI 3440 x 1440
Offline

Forest Finbow

  • Posts: 31
  • Joined: Thu Mar 30, 2017 11:51 am
  • Location: Paris, France

Re: Apple ProRes 4444 is float or integer

PostThu Apr 27, 2017 11:36 pm

Hey ! I did come up with a test! It's actually a very trivial one to do:

Build a node graph consisting of two nodes.
Have the first one burn out or crunch pixels and then retrieve them in a second node.
Cache the first node, and watch the scope upon playback.

Here are some illustrations with gain cranked all the way up to Resolve maximum (+16):

uncomp-and-16-bit-float.png
Uncompressed 16 bit float looks exactly like disabling the cache : 100% retrieval.
uncomp-and-16-bit-float.png (61.89 KiB) Viewed 3148 times


16-bit-int-(444).png
DNxHR 444 exbibits clipping at insane values (Gain set to 16)
16-bit-int-(444).png (48.44 KiB) Viewed 3148 times


8-bit.png
And DNxHR 8 bits will retain... well just a couple percent really
8-bit.png (13.4 KiB) Viewed 3148 times


I've also done the test with negative values and they yield the similar results, with the exception of 8bit not retrieving anything at all.

So voila!

Unsurprisingly, 422 codecs fall in the traditional 0-100% realm. Quantification will not affect dynamic range, just intermediate precision as they traditionally do and of course, they clip.

444 codecs store ints, not floats. Negative values are retained so they appear to be signed ints but resolve might just be clever enough to shift the bits before and after encoding/decoding.

Now interestingly, while cache metadata indicates quantification to be 16 bits in both codecs, I found out that RGB float can retain way way more dynamic range than DNx. I needed to stack 5 consecutive +16 Gain nodes in order to clip the cache for RGB while DNx clipped with one single node. (Yes, I managed to clip the 16 bits float and no, not the internal 32 bit. It's unbeatable, I'm not going to roll up to 4^38 !!!)

This suggests that DNx is 12 bits after all, despite what's mentioned in the metadata.
If it was 16 bit ints, the range would go from is −32,767 to +32,767, while uncompressed RGB 16 bit floats would store an extra stop of dynamic range going -65 503 to + 65 503. But clearly it is much more than an extra stop.

My conclusions at this point is that 16 bit float still has the highest dynamic range, well above 444 HDR for sure. Given the insane values I had to push the footage however, I'm pretty sure it won't often be that useful. Maybe for very specific workflows that would involve stuff like caching OpenEXR, or caching linear gamma debayered RAW. Anyways, I feel that's a good thing to know about.
Offline

Olivier MATHIEU

  • Posts: 94
  • Joined: Thu Aug 04, 2016 1:55 pm
  • Location: Grenoble, FRANCE

Re: Apple ProRes 4444 is float or integer

PostFri Apr 28, 2017 8:55 am

Forest Finbow wrote:So voila!

Unsurprisingly, 422 codecs fall in the traditional 0-100% realm. Quantification will not affect dynamic range, just intermediate precision as they traditionally do and of course, they clip.

444 codecs store ints, not floats. Negative values are retained so they appear to be signed ints but resolve might just be clever enough to shift the bits before and after encoding/decoding.

That'a one answer I expected. Thanks

Now interestingly, while cache metadata indicates quantification to be 16 bits in both codecs, I found out that RGB float can retain way way more dynamic range than DNx. I needed to stack 5 consecutive +16 Gain nodes in order to clip the cache for RGB while DNx clipped with one single node. (Yes, I managed to clip the 16 bits float and no, not the internal 32 bit. It's unbeatable, I'm not going to roll up to 4^38 !!!)

This suggests that DNx is 12 bits after all, despite what's mentioned in the metadata.
If it was 16 bit ints, the range would go from is −32,767 to +32,767, while uncompressed RGB 16 bit floats would store an extra stop of dynamic range going -65 503 to + 65 503. But clearly it is much more than an extra stop.

My conclusions at this point is that 16 bit float still has the highest dynamic range, well above 444 HDR for sure. Given the insane values I had to push the footage however, I'm pretty sure it won't often be that useful. Maybe for very specific workflows that would involve stuff like caching OpenEXR, or caching linear gamma debayered RAW. Anyways, I feel that's a good thing to know about.

Crystal clear
Have you try with Prores ?
Thanks
MacOS 10.14.5
i7 8 cores 3,8 Ghz
RAM 32 Go (2800Mhz)
RADEON VII 16 Go
Resolve Studio 16
GUI 3440 x 1440
Offline

Andrew Kolakowski

  • Posts: 5360
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Apple ProRes 4444 is float or integer

PostFri Apr 28, 2017 10:54 am

Forest Finbow wrote:Hey ! I did come up with a test! It's actually a very trivial one to do:

Build a node graph consisting of two nodes.
Have the first one burn out or crunch pixels and then retrieve them in a second node.
Cache the first node, and watch the scope upon playback.

Here are some illustrations with gain cranked all the way up to Resolve maximum (+16):

uncomp-and-16-bit-float.png


16-bit-int-(444).png


8-bit.png


I've also done the test with negative values and they yield the similar results, with the exception of 8bit not retrieving anything at all.

So voila!

Unsurprisingly, 422 codecs fall in the traditional 0-100% realm. Quantification will not affect dynamic range, just intermediate precision as they traditionally do and of course, they clip.

444 codecs store ints, not floats. Negative values are retained so they appear to be signed ints but resolve might just be clever enough to shift the bits before and after encoding/decoding.

Now interestingly, while cache metadata indicates quantification to be 16 bits in both codecs, I found out that RGB float can retain way way more dynamic range than DNx. I needed to stack 5 consecutive +16 Gain nodes in order to clip the cache for RGB while DNx clipped with one single node. (Yes, I managed to clip the 16 bits float and no, not the internal 32 bit. It's unbeatable, I'm not going to roll up to 4^38 !!!)

This suggests that DNx is 12 bits after all, despite what's mentioned in the metadata.
If it was 16 bit ints, the range would go from is −32,767 to +32,767, while uncompressed RGB 16 bit floats would store an extra stop of dynamic range going -65 503 to + 65 503. But clearly it is much more than an extra stop.

My conclusions at this point is that 16 bit float still has the highest dynamic range, well above 444 HDR for sure. Given the insane values I had to push the footage however, I'm pretty sure it won't often be that useful. Maybe for very specific workflows that would involve stuff like caching OpenEXR, or caching linear gamma debayered RAW. Anyways, I feel that's a good thing to know about.


I don't think this is really related to ProRes or DNxHR, 422 or 444. ProRes and DNxHR are at best 12bit.
It's more about how Resolve first inserts data into codec and then how it handles data once it comes out of codec (decoded frames). DNxHR HDR (ProRes HDR) is treated in some special way I think, but you will be still somehow "limited" by the fact that codec itself is 12bit at best.
I don't think BM has written special version of ProRes or DNxHR for HDR cache :)
12bit is still will be plenty in really world scenarios- maybe sometimes you can hit limits with HDR projects, but I'm not sure. If you treat this 12bit starting data properly (you can actually store data in some special way, eg. LOG form etc inside codec to better use bits where they are important), do maths in 32bit float then you should be fine.
Offline
User avatar

Jean Claude

  • Posts: 2420
  • Joined: Sun Jun 28, 2015 4:41 pm
  • Location: France

Re: Apple ProRes 4444 is float or integer

PostFri Apr 28, 2017 12:15 pm

I will add:
From what I could read in OFX sources: Blackmagic does not limit between 0.0f and 1.0f.
There is even an effect that is capable of revealing what is above 1.0f (which is completely burnt):

SoftClipPluginV2.1.ofx.bundle. This is written by Paul Dore.

Each channel: R-G-B-A is read in float. No distinction 'a priori' if the source is 8/10/12/16 bits. Prores or other (s)
Windows 10 PRO X64 1903 | DaVinci Resolve Studio 16.0.0B.033 | Fusion Studio 16.0.beta 28 build 28 | Decklink Studio 4K 6G | Desktop Video 11.2.00 | X99-A II, i7-6850K | RTX 2080Ti | Nvidia 430.86 NSD
Offline

Forest Finbow

  • Posts: 31
  • Joined: Thu Mar 30, 2017 11:51 am
  • Location: Paris, France

Re: Apple ProRes 4444 is float or integer

PostFri Apr 28, 2017 6:22 pm

Olivier MATHIEU wrote:Crystal clear
Have you try with Prores ?
Thanks

Well, Yes I have, and I made even more findings here!
I have always been sort of puzzled about the fact that Resolve allowed me to cache in ProRes, given I work in a Windows environnement. It turns out, the ProRes option is not exactly what it looks like under windows, it in fact stores uncompressed RGB, good old integer RGB, exactly like TIFF does. I suspect the option here is just a Dummy that connects to no codec, the dummy just writes out on the hard drive what it receives. While that was quite inconvenient for me to test the ProRes codec performances, it's actually a very interesting piece of information I have now : I can see, on my hard drive the raw numbers sent by resolve to the 444 compressor.
So I've been reverse engineering the DVCC files with a Hex editor, and guess what? They are 16 bits numbers after all, not 12 bits. I don't know what the ProRes library would have done with them if it was present, it might in turn rescale them down to 12, but I witnessed Resolve sending 16 bit integers, and further more I was able to see the bit shift done to represent negative values with an unsigned integer.

As I suspected, the numbers I've read from the proper 16 bit float RGB files are not integers, they were indeed short floats. As others have pointed out, they range from 0 to 1 for the actual signal displayed, so +16 gain brings them to the range 0 to 16, still very far from clipping (somewhere above 65000) and negatives are negatives ;-)

I'm still trying to figure out exactly how the numbers work, in order to calculate exactly the dynamic range available with the different options. I'll update them here as soon as I get something that I fully understand.

So far I found out :

Code: Select all
Level = float = int (ProRes/DNxHR 444)
-1 L  = -2.0f = 5203  // Darker than black (Black value with lift set to -1)
0 %   = 0.0f  = 16383 // Black
50 %  = 0.5f  = 33660 // Mid gray (the line siting at 512 on your waveform)
100%  = 1.0f  = 41355  // White (visual clip in the viewers)
+16G  = 16.0f = 65532 // Hard clip (White value with gain set to +16)


In the ProRes 444 integer output, the black point is shifted up 16384 levels. That seems like a clever idea since we'll typically have more pixels above black level than bellow (remember the range of a 16 bit int is 0-65536) What I can't figure yet is the seemingly arbitrary numbers for positive values. Why would pure white be set at 41355 ? Is that some sort of power law kicking in ? The testing timeline is set to à 2.4 gamma, could that explain those numbers ? Mid gray sitting at 33660 indicates that some sort of power law is at work but which one, is it a Log, a y2.4, I don't know yet...
Last edited by Forest Finbow on Fri Apr 28, 2017 6:37 pm, edited 1 time in total.
Offline

Forest Finbow

  • Posts: 31
  • Joined: Thu Mar 30, 2017 11:51 am
  • Location: Paris, France

Re: Apple ProRes 4444 is float or integer

PostFri Apr 28, 2017 6:37 pm

Andrew Kolakowski wrote:I don't think this is really related to ProRes or DNxHR, 422 or 444. ProRes and DNxHR are at best 12bit.
It's more about how Resolve first inserts data into codec and then how it handles data once it comes out of codec (decoded frames). DNxHR HDR (ProRes HDR) is treated in some special way I think, but you will be still somehow "limited" by the fact that codec itself is 12bit at best.
I don't think BM has written special version of ProRes or DNxHR for HDR cache :)
12bit is still will be plenty in really world scenarios- maybe sometimes you can hit limits with HDR projects, but I'm not sure. If you treat this 12bit starting data properly (you can actually store data in some special way, eg. LOG form etc inside codec to better use bits where they are important), do maths in 32bit float then you should be fine.

I know ProRes and DNxHR are specified as 12 bit max in their respective white papers but at least one third party implementation of those very same codecs did support 16 bit: Miraizon.
This was of course until Apple decided to sue them out of business for allowing ProRes encoding in windows (naughty boys, how dare they offer such a service ;-) !)
So I actually wouldn't be that surprised if Blackmagic did implement a 16bit version for caching HDR, it has been done before.
I agree however that even scaling down to 12 bits is plenty enough for normal HDR workflows. HDR delivery is actually most of the time HDR10, so 10 bits only.
Offline

roger.magnusson

  • Posts: 877
  • Joined: Wed Sep 23, 2015 4:58 pm
  • Location: Sweden

Re: Apple ProRes 4444 is float or integer

PostSat Apr 29, 2017 12:27 am

Forest Finbow wrote:I have always been sort of puzzled about the fact that Resolve allowed me to cache in ProRes, given I work in a Windows environnement.

BMD confirmed that as a bug in Resolve 12.5.1. The misleading "ProRes" option for caching was removed in later versions.
Offline

Andrew Kolakowski

  • Posts: 5360
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Apple ProRes 4444 is float or integer

PostSat Apr 29, 2017 9:48 am

Forest Finbow wrote:I know ProRes and DNxHR are specified as 12 bit max in their respective white papers but at least one third party implementation of those very same codecs did support 16 bit: Miraizon


Don't look what codec can take as input or what pixel format can present on output after decoding. It's important how data is stored when it's compressed. Each encoder/decoder is designed to provide specific pixel formats. For example Cineform support tons of different ones.
Apple's ProRes can take 32bit and output 32bit float, but it doesn't mean data will keep this precision. It will be 12bit at best. Most codesc use 16bit to work with any data above 8bit (with v210 pixel format been special case for 10bit). Only ffmpeg has pixel formats for almost each bit-depth, e.g. gbrp12le, gbrp14le etc. Most pro software use 16bit pixel format b64a for anything above 8bit or float these days.
First time I have heard Miraizon could work at 16bit, so it really preserved this precision. Their code was ffmpeg one with some tweaks. Due to rate control (so it kept Apple bitrate restrictions) it was actually bit worse than original ProRes (2-3dB in PSNR).
I don't think BM can't even take Apple code and extend it- it would be against licensing agreement.
Last edited by Andrew Kolakowski on Sat Apr 29, 2017 10:00 am, edited 1 time in total.
Offline

Forest Finbow

  • Posts: 31
  • Joined: Thu Mar 30, 2017 11:51 am
  • Location: Paris, France

Re: Apple ProRes 4444 is float or integer

PostSat Apr 29, 2017 10:00 am

Ah, that's very interesting thank you very much. I suspected that the codec was able to downsample an input signal but I didn't know it could take such a broad variety of input formats.
Offline

Andrew Kolakowski

  • Posts: 5360
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Apple ProRes 4444 is float or integer

PostSat Apr 29, 2017 10:08 am

This is normal. How many pixel format are supported depends purely on developer and his decision.
Apple's ProRes can natively "take" data in these pixel formats (at least based on wiki):
0 - unknown
1 - '2vuy' (8-bit 4:2:2)
2 - 'v210' (10-bit 4:2:2)
3 - 'v216' (10,12,14,16-bit 4:2:2)
4 - 'r408' (8-bit 4:4:4:4 with alpha)
5 - 'v408' (8-bit 4:4:4:4 with alpha and super black)
6 - 'r4fl' (32-bit floating-point 4:4:4:4)
7 - 0x20 (8-bit RGB)
8 - 'BGRA' (8-bit RGB with alpha)
9 - 'n302' seems to be undocumented
10 - 'b64a' (16-bit ARGB)
11 - 'R10k' (AJA 10-bit RGB)
12 - 'l302' seems to be undocumented
13-15 invalid

You can check what pixel format was sent to encoder with mediainfo. It's not 100% accurate info though as it can happen that header is wrong (when application is badly designed).
For example Resolve sets it to 2 (v210) in case of PoResHQ, but to 14 for ProResXQ, which based on the spec is invalid. AME uses 10 - 'b64a' for ProResXQ.
Application using e.g. QuickTime can "ask" for specific pixel format (that's why you edit Adobe xml rules file), but after this it can convert given pixel format further. Problem is that QT engine on Mac can convert between pixel formats and this is an evil.
For example transcoders (when there is no other processing needed) should work in native pixel formats to be fast and avoid conversion problems. So when you convert from e.g. ProResHQ to DNxHR HQX the best is to take v210 pixel format (which is native for both codecs) and simply pass data straight from ProRes decoder to DNxHR encoder (ffmpeg does it this way).
Resolve won't do it this way due to the way how its engine is designed- everything after decoding is normalised to 32bit float and then passed to encoder. Some support 32bit float on input, if not data is converted by Resolve engine to 1 of the supported by codec pixel formats.
Offline

Andrew Kolakowski

  • Posts: 5360
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Apple ProRes 4444 is float or integer

PostSun Apr 30, 2017 1:04 pm

My recent, more precise test regarding 12bit in ProRes444/XQ.

Starting with 16bit TIFF with gradients at 1024/1032/1040

DNxHR 12bit resolves 12bit (there is some levels shift, but precision seams to be kept).
Cineform 12bit RGB mode resolves 12bit (perfect levels result in AE).
JPEG2000- AME 12bit RGB mode- same as Cineform- perfect levels.
ProRes 444 and XQ- 12bit is sort of kept. 1024 and 1032 levels are flatten (maybe values are on the edges of steps). Looks like there is also some other crap going on. Tried different apps to encode ProResXQ, but results are the same.
Here is how bars look like from ProResXQ (boosted for better visibility):

Image

In the same time decoding same ProRes XQ file with ffmpeg flattens all bars, so it looks like ffmpeg does "convert" to 10bit, or in other words behaves differently than official ProRes decoder.

Any 10bit codec (ProResHQ, DNxHR, Cineform in 10bit mode etc) = flat bars as precision is to low to keep different colors.

text corrected
Last edited by Andrew Kolakowski on Sun Apr 30, 2017 3:46 pm, edited 3 times in total.
Offline

Forest Finbow

  • Posts: 31
  • Joined: Thu Mar 30, 2017 11:51 am
  • Location: Paris, France

Re: Apple ProRes 4444 is float or integer

PostSun Apr 30, 2017 1:09 pm

Brilliant, you're a gold mine of informations I'm digging this (and feel a little ashamed I did not pull that from Wikipedia my self ;-))

From what I could read in the cache files generated by Windows ProRes XQ caching (prior to its removal), they are Plain RGB 16 bits. Litterally triple little endian uint16 streamed one after the other, just like 16 bpc uncompressed TIFF.
If my suspicion is correct that the encoder choice ProResXQ under windows is just a dummy writing off uncompressed media to the hard drive, it actually makes sense that the pixel format would not be 'b64a', since there is no alpha channel written to the disk. That might be the reason Resolve is picking the pixel format to 14 for ProResXQ...
I don't have a mac just now to test this, but you might know, does Resolve feature alpha channel in RGBA (4444) output in the deliver tab ? I've tried here in the windows environnement and could not succeed in having any Alpha in RGBA renders, even picking very explicit codecs such as DPX RGBA or Quicktime Uncompressed ARGB, not to mention of course DNxHR 444...
A separate DPX Alpha render works, so Resolve definitely knows which pixels are opaque in the timeline...
Offline

Andrew Kolakowski

  • Posts: 5360
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Apple ProRes 4444 is float or integer

PostSun Apr 30, 2017 1:15 pm

ProRes444/XQ can have alpha. Alpha channel can be 8bit or 16bit and it's losslessly encoded (sort of zip).
Offline

Forest Finbow

  • Posts: 31
  • Joined: Thu Mar 30, 2017 11:51 am
  • Location: Paris, France

Re: Apple ProRes 4444 is float or integer

PostSun Apr 30, 2017 1:34 pm

Andrew Kolakowski wrote:DNxHR 12bit resolves 12bit (there is some levels shift, but precision seams to be kept).
Cineform 12bit RGB mode resolves 12bit (perfect levels result in AE).
JPEG2000- AME 12bit RGB mode- same as Cineform- perfect levels.
ProRes 444 and XQ- 12bit is not kept. 1024 and 1032 levels are flatten. Looks like there is maybe 11bit precision+there is also some other crap going on. Tried different apps to encode ProResXQ, but results are the same.


Hum this is very interesting. It indicates that Cineform or JPEG2K would actually be better options for high quantification caching (ie 12 bits+ aka HDR). Unfortunately Resolve does not allow for those options in the cache codec settings... Plus I'd like to point out that JPEG2K supports 16 bit per layer, unsigned or signed and even FLOATING POINT (jpf). Should we bring this to the attention of the DR Dev team ? ;-)
Offline

Forest Finbow

  • Posts: 31
  • Joined: Thu Mar 30, 2017 11:51 am
  • Location: Paris, France

Re: Apple ProRes 4444 is float or integer

PostSun Apr 30, 2017 1:37 pm

Andrew Kolakowski wrote:ProRes444/XQ can have alpha. Alpha channel can be 8bit or 16bit and it's losslessly encoded (sort of zip).

Yes in the specs, my question is can Resolve fill that alpha channel with meaningful information ? as I've mentioned I've tried to get resolve to output RGBA in with many different spec compliant options but without success.
Offline

Andrew Kolakowski

  • Posts: 5360
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Apple ProRes 4444 is float or integer

PostSun Apr 30, 2017 1:44 pm

Better not to store it at all :)
Offline

Forest Finbow

  • Posts: 31
  • Joined: Thu Mar 30, 2017 11:51 am
  • Location: Paris, France

Re: Apple ProRes 4444 is float or integer

PostSun Apr 30, 2017 2:58 pm

Yes, I presume that is exactly what resolve is doing here : not storing alpha at all.
The trouble is that RGB 16 bits + no alpha (which turns out to be 'b48r' source pixel format) is not officially supported by ProRes, according to the header info you revealed here.
I guess that's the reason Resolve tags it as '14' in the quicktime header src_pix_fmt and not '10' : 'b64a'
Offline

Andrew Kolakowski

  • Posts: 5360
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Apple ProRes 4444 is float or integer

PostSun Apr 30, 2017 3:10 pm

Possible, but you have to be careful with trusting src_pix_fmt header.
Offline

Olivier MATHIEU

  • Posts: 94
  • Joined: Thu Aug 04, 2016 1:55 pm
  • Location: Grenoble, FRANCE

Re: Apple ProRes 4444 is float or integer

PostMon May 01, 2017 5:48 am

Yes in the specs, my question is can Resolve fill that alpha channel with meaningful information ? as I've mentioned I've tried to get resolve to output RGBA in with many different spec compliant options but without success.

On Mac, inDeliver Page, it works with "individual clips"
but if id doesn't work, you can still output a Key as RGB
capture 2017-05-01 à 07.47.11.png
ink the node'salpha output to the RGB node graph's output
capture 2017-05-01 à 07.47.11.png (23.44 KiB) Viewed 2892 times


But for Render Cache Files, I think there is no alpha :
In the edit page Compound clip haven't got any alpha channel
Resolve's engine seems to be 32 float YRGB but not 32 float YRGBA
MacOS 10.14.5
i7 8 cores 3,8 Ghz
RAM 32 Go (2800Mhz)
RADEON VII 16 Go
Resolve Studio 16
GUI 3440 x 1440
Offline

Olivier MATHIEU

  • Posts: 94
  • Joined: Thu Aug 04, 2016 1:55 pm
  • Location: Grenoble, FRANCE

Re: Apple ProRes 4444 is float or integer

PostMon May 01, 2017 5:53 am

Level = float = int (ProRes/DNxHR 444)
-1 L = -2.0f = 5203 // Darker than black (Black value with lift set to -1)
0 % = 0.0f = 16383 // Black
50 % = 0.5f = 33660 // Mid gray (the line siting at 512 on your waveform)
100% = 1.0f = 41355 // White (visual clip in the viewers)
+16G = 16.0f = 65532 // Hard clip (White value with gain set to +16)

It seems completly right with our test isn't ? :-)

Plus I'd like to point out that JPEG2K supports 16 bit per layer, unsigned or signed and even FLOATING POINT (jpf).

Good to Know.
But Nowadays free Jpeg2K encoder are quite cpu hungry. It kills the purpose of using it as Render Cache File 's Codec, no ?
MacOS 10.14.5
i7 8 cores 3,8 Ghz
RAM 32 Go (2800Mhz)
RADEON VII 16 Go
Resolve Studio 16
GUI 3440 x 1440
Offline

Olivier MATHIEU

  • Posts: 94
  • Joined: Thu Aug 04, 2016 1:55 pm
  • Location: Grenoble, FRANCE

Re: Apple ProRes 4444 is float or integer

PostMon May 01, 2017 6:05 am

Resolve won't do it this way due to the way how its engine is designed- everything after decoding is normalised to 32bit float and then passed to encoder.

I've also noticed that, and think like you, it's less accurate (and may be a waste of power ?!) especialy with the "Media Management....➧ Transcode".
But it make me think of an old unexplained technical stuff in Davinci Resolve. I will open a new topic.
MacOS 10.14.5
i7 8 cores 3,8 Ghz
RAM 32 Go (2800Mhz)
RADEON VII 16 Go
Resolve Studio 16
GUI 3440 x 1440

Return to DaVinci Resolve

Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Igor Riđanović, RCModelReviews and 32 guests