In what bit depth are images processed in the edit page?

In what bit depth are images processed in the edit page?

OK, I am trying to understand the processing nature of resolve edit page. Which I'm not sure how it works. Maybe someone here can help me understand what is happening.

The main issue is the bit depth.

Lets say you drag a Text + fusion comp to a timeline and you open it in fusion and you apply glow tool. Just glow that ships with resolve. You boost the glow node and when you go back to edit page, you will probably expect glow to be around letters, but letters to be opaque, ad instead you will probably see trough them. This has been brought up few times by others in the past.

The issue is that in fusion, processing by default is at 32-bit float and that allows for values above 1.0.

0-1 scale is in fusion by default, 0 = black, and 1 = White.

If you work in integer 8-bit or 16-bit than it can't go above 1.0 value. But if you are in 32-bit float or half float, known as 16-bit float, you can go beyond 1.0. And that is the problem, especially in the alpha channel. More on this later.

For those who don't know a quick explanation.

Color Bit Depths

The term bit depth describes how many colors are available in the color palette used to make up an image. The higher the bit depth, the greater the precision of color in the image, and therefore the greater the color reproduction. The higher precision is most apparent in gradients with subtle changes. Lower bit-depth gradients have noticeable banding artifacts, whereas higher bit-depth images can reproduce more colors, so fewer, if any, banding artifacts occur.

Fusion page by default within DaVinci Resolve always uses 32-bit float bits per channel precision to process images. However, in Fusion Studio you can choose to process images with 8-bit integer, 16-bit integer, 16-bit float, and 32-bit float bits per channel. And technically you can do this in fusion page as well if you add ChangeDepth tool or use settings on node per node basis. But by default processing in fusion page is 32.bit float. Except for masks.

Understanding Integer vs. Float

Generally, 8-bit integer color processing is the lowest bit depth you’ll come across for video formats. 8-bit images come from older or consumer-grade video equipment like mobile phones and camcorders. If you try to perform any significant gamma or color correction on 8-bit images, you can often see more visible banding.

16-bit integer color depth doubles the amount of precision, eliminating problems with banding. Although you can select 16-bit integer processing for an 8-bit clip, it does not reduce banding that already exists in the original file. Still, it can help when adding additional effects to the clip. This sounds like the best solution until you realize that many digital cameras like Blackmagic Design URSA Mini Pro and others record in formats that can capture over-range values with shadow areas below 0.0 and super highlights above 1.0, which are clipped in 16-bit integer.

The 16-bit float color depth sacrifices a small amount of the precision from standard 16-bit integer color depth to allow storage of color values less than 0 and greater than 1.0. 16-bit float, sometimes called half-float, is most often found in the OpenEXR format and contains more than enough dynamic range for most film and HDR television purposes yet requires significantly less memory and processing time than is required for full float, 32-bit images.

.........................

The problem.

When using tool such as glow tool, by default it will apply its glow effect on all channels. Red, Green, Blue and also alpha. And that is the problem, alpha above 1.0 value when seen on the edit page viewer, becomes semi transparent. You can see trough the object you are applying glow effect to. Even if it looks good in fusion page, even in the color page, but not in the edit page.

Here in 32-bit float, glow tool produces RGB and A values in ranges that can go beyond 1.0.

sshot-1622aa.jpg (93 KiB) Viewed 2757 times

This can be additionally confirmed with 3D histogram and when you see it on the edit page, you can actually see trough the letters, because A for Alpha in this case is 1.56 something, and edit page can seemingly only display up to 1.0, effectively clipping the rest, causing transparency issue.

sshot-1623a.jpg (462.89 KiB) Viewed 2757 times

..........................................

The Solution

The solution is really simple. All you have to do is limit Alpha values to be in 0-1 range. And you can do this in various ways. One method is to deactivate Alpha channel processing in the glow node itself, leaving Alpha untouched, usually at 1.0.

You can also add Change Depth Node before the glow and change to integer mode, either 8 or 16 bit. Which will limit values in 0-1 range. But unlike first option, you lose nice neon like glow effects that you can pool off in float range.

So you can also add a Brightness / Contrast node which has option to clip white and black values, keeping them in 0-1 range. And if you want to only limit alpha but keep the RGB you can only activate Alpha channel processing with the Brightness / Contrast node.

sshot-1627a.jpg (463.23 KiB) Viewed 2757 times

If you use third party awesome glow fuses like XGlow and Fast Glow, you probably won't see this problem because they already have alpha deactivate and glow is not applied to alpha, only RGB.

For some reason, Glow node that ships with resolve, by default has RGB and A turned on, causing the issue. If you deactivate A from processing, you can than right click on the node and save new settings as default for every new time you use the tool.

So the transparency is not really an issue. But it brings up a question.

........................

The Question

I was under the impression 32-bit float processing is applied across the resolve pages. But it would see that edit page viewer, is not in float mode processing. But in Integer. Curiously color page, and fusion and even "P" for presentation mode, when you go full screen preview on edit page, is not showing transparency issue, so I imagine its not in integer either.

Is edit page in integer when seeing footage in the viewer when editing? And if so, is it to boost performance since its not about image color manipulation as much as editing footage? And color, fusion and full screen preview are full float bit depth processing?

I would like some clarification on this?

By the way, I didn't use any proxy mode or caching or anything other than full quality for this test. You can test it as well on your machine.

Thank you.

I googled it and got this.....

On the DaVinci Resolve Edit page, the default bit depth is 10-bit, meaning each color channel can store 10 bits of information, allowing for a wider range of colors and smoother gradients compared to lower bit depths like 8-bit; however, you can adjust this depending on your project's needs and the source footage bit depth.
Key points about bit depth in DaVinci Resolve:
Accessing bit depth settings:
You can access the bit depth settings within the project settings, specifically in the "Delivery" tab where you can choose the output bit depth for your final render. (here it should say 'export' instead of render)
Higher bit depth benefits:
Working with a higher bit depth like 10-bit or even 32-bit float (available on the Fusion page) can be beneficial for color grading and post-processing as it provides more precision for color adjustments, minimizing banding artifacts.

I googled it and got this.....

Is that a summery from a chat AI bot of various articles online? Some information could be conflicting and some is related to monitoring. I'll do more tests and see what I can find. Thanks.

It was my understanding that Resolve processing all images in 32-bit Float. But the Viewers would be showing an 8 bit image, unless you have the option for 10 bit checked in Preferences.

I wonder if that's correct...

It was my understanding that Resolve processing all images in 32-bit Float. But the Viewers would be showing an 8 bit image, unless you have the option for 10 bit checked in Preferences.

I wonder if that's correct...

I wonder as well. It would make sense what you say, but I found that if I use automatic color management of resolve for the timeline or project, than the problem is not there. The transparent text after applying glow effect. It is no longer an issue. Than I try to do manual color management by converting text+ node in fusion from linear to rec709, same as timeline. I used CST or Color Space Transform Node in fusion to do that. When I do that, much like automatic color management problem is solved. Fusion I think is meant to be working in linear color space, so tools that are internal generator nodes, like text+, are created in linear space and when sending them to rec709 timeline, problem happens. And when auto color management for timeline or project is turned on, than all the proper conversion happens automatically.

Anyway, back to your comment. I think that is the most logical explanation. Probably actual processing happens in 32-bit float and its the viewer that is 8-bit or 10-bit integer. Also, when I go full screen with the "P" shortcut, transparency is not an issue, so probably only the viewer for editing not full screen is integer.

I'm still trying to figure out what is happening at what point of the process, since additionally until latest updated of 19.1.2, color page viewer also had issues with transparency. Where only some tabs in the color page would not work properly. Curves and Color Wrapper. They would not show transparent pixels, but other tabs in color page would. This was fixed in 19.1.2 release so no longer an issue. But it was one of the things that prompted this inquiry into what is happening.

I also recall that a while back someone was talking about how when working with certain type of footage where aspect ratio of the frame and aspect ratio of pixels is not standard. And how for some reason there is inconsistency between viewer in the edit page and viewer in the color page. One would show it correctly and the other one would not. I forget which one was the problem. But that and this transparency issue was something that I wanted to try to understand.

Also notice how since version 19 of resolve we can activate transparent background with check board pattern for cut and edit page, but not color, fair light and deliver pages. Also I think in full screen preview it shows black instead of checkerboard. This inconsistency is sometimes making things unclear about what exactly is happening behind the scenes. So I was curious. Thanks for the feedback.

if I use automatic color management...then the problem is not there.

Got ya.

That is my own preferred method of working, so that's probably why I never saw this.

if I use automatic color management...then the problem is not there.

Got ya.

That is my own preferred method of working, so that's probably why I never saw this.

Yeah. I didn't try to think of this problem, at least not originally as color management issue when I wrote the post, because I could provoke and remove the problem of transparency by limiting alpha values to go beyond 1.0 value. So I figured it has to be primarily related to bit depth. Which probably it is. However, color management also can be used to solve it by using appropriate color space conversion. But I guess that is another rabbit hole.

If you go from float data to uint, usually values outside normalized range of 0.0-1.0 are clamped, but it depends on what the target colorspace is as integer storage can store whatever you want (within precision limits) depending on the transfer function equation. Log encodings frequently store some negative values to preserve the noise range and their upper limit is obviously much higher than 1.0 diffuse white level in scene-linear relation. Whether edit page works on floats or not… well, if floats have to be output on deliver page (it allows writing exrs with no clamping), edit page MUST pass through float data too. Can’t do that if it converts to some x bits of unsigned first.

Now, to understand why something turns ”transparent” with out of range alpha values, consider the over operation where (per channel) result C = A*alphaA + B*(1.0-alphaA)
If alphaA is bigger than 1.0, B contribution turns negative, creating an impression of B in the result. This isn’t transparency with any stretch of imagination but rather a negative imprint as the contribution depends on the scale of B but balance of channels is inverted.

Manual probably tells all about it but some operations and data transitions in Resolve clamp the range so problem appears and disappears depending on whether alpha channel is clamped or not. Keep in mind that if element is output from Fusion comp it actually gets composited with underlying track not in MediaOut but somewhere down the stream as it passes through the color page first, before it hits the layer blend modes and whatnot in edit. But you can view the Fu output before that too, and this is where the selective clamping can have its effect as it can be purely viewer dependent for example. Resolve is a patchwork of different loosely connected components with little in common between them in logic of operation. It is actually a problem if delivery output is wonky, in other cases it is just annoyance as it fails the WYSIWYG logic creative apps crave for.

Log story short, 32bit float.

PS.

16-bit integer color depth doubles the amount of precision, eliminating problems with banding

16bit vs 8bit has 256-times precision, each single value is now divided into 256 new ones. Precision would double going from 8 to 9 bits. Yes, double precision floating point values etc, it is a misleading name as precision does not double (memory allocation doubles), but in case of single- to double float it increases the ”granularity” by 29 bits (doubles it 29 times compounded) which is A LOT and also increases the range A LOT.

You may be right. there is correlation between color management and bit depth in the tests I've done. So they both probably play a factor. And you are also probably correct about how before edit page sees the image it has to go trough color page.

DaVinci Resolve_19_page3200_image (2).jpg (124.64 KiB) Viewed 2144 times

Fusion page by default within DaVinci Resolve always uses 32-bit float bits per channel precision to process images. However, in Fusion Studio you can choose to process images with 8-bit integer, 16-bit integer, 16-bit float, and 32-bit float bits per channel. And technically you can do this in fusion page as well if you add ChangeDepth tool or use settings on node per node basis. But by default processing in fusion page is 32.bit float. Except for masks.

In Fusion Page, there is a setting: Frame Format > Color Depth
Can someone explain to me what this setting does in this context ?

The manual says:

The color bit depth settings only apply to Fusion Studio. Rendering in DaVinci Resolve always use 32-bit float [...] Generally, these options are ignored by the composition unless a Loader or Creator tool’s Color Depth control is set to Default.

So... can i completely ignore these settings ?
Or are they used when I for example use a background node and the Depth is set to "Default"?

In Fusion Page, there is a setting: Frame Format > Color Depth
Can someone explain to me what this setting does in this context ?

Yeah, from what I can see, in fusion studio, not resolve version of fusion, in the preferences of fusion studio, under frame format section, you could dept up default bit depth for generator nodes, for example background tool/node.

When they added fusion to resolve, it would seem that some of the preferences are there in fusion page as they are in fusion studio, but they are overridden by resolve itself, so some settings are at this point just dead and don't affect anything.

For example if you leave frame format settings in fusion page to 16-bit float. Or something other than 32-bit float, resolve will override it and always work in 32-bit float.

sshot-26.jpg (138.38 KiB) Viewed 1962 times

However, its still possible to change bit depth and thereby processing on the node by node basis. Either in the nodes that have the setting, or by changing it with a "change depth" tool/node.

sshot-27.jpg (69.44 KiB) Viewed 1962 times

sshot-28.jpg (84 KiB) Viewed 1962 times

Similar is for frame number of composition, auto resolution, and playback fps and of course bit depth. Proxy settings also. All those which are controllable from preferences in fusion studio, have been carried over to fusion page, and they are still there as preferences. They haven't been removed. But changing them has no effect, its been overridden by Resolve Edit timeline. This can be confusing mess, since I understand some of the reasons why resolve edit page would have primacy, but it would be nice to have the settings that no longer function either removed from user interface or there should be some kind of warning message.

Otherwise you make changes in fusion page preferences, but they have no effect. I imagine this cases confusion among veteran fusion studio users, who now feel they can't change things they are used to, and there is no clear reason why. And it can be confusion to new users because the preference is there, but has no effect. And than there is the problem of parity and consistency between fusion studio and resolve version of fusion.

The manual says:

The color bit depth settings only apply to Fusion Studio. Rendering in DaVinci Resolve always use 32-bit float [...] Generally, these options are ignored by the composition unless a Loader or Creator tool’s Color Depth control is set to Default.

So... can i completely ignore these settings ? Or are they used when I for example use a background node and the Depth is set to "Default"?

Yes, as I've mentioned in previous section .If for example background node is set to default mode it will be 32 bit depth, and you can double change in the upper right corner of the viewer. But you could check it on node by node basis, if you change the bit depth to something else. Most users who use resolve, probably don't really know any of this, so by default set to default for them is going to be 32 bit float. If you want to or need to change it, its possible, but not from Fusion preferences anymore, only either by changing it in the node that allows it, or using change depth tool for situation where its not in the node itself.

However, as you probably know , when doing merge operation, background input takes primacy over foreground input. So if the background is of certain resolution as well as bit depth it will take primacy and change bit depth downstream. Even if foreground is set to some other bit depth or if you use change depth tool. Background input always wins.

Thank you Kruno for your detailed answer !

I imagine this cases confusion among veteran fusion studio users, who now feel they can't change things they are used to, and there is no clear reason why. And it can be confusion to new users because the preference is there, but has no effect.

I completely agree. BMD should get rid of these remnants or make them work in the new environment. Contradictory statements in the manual also add to the confusion.

[quote="Christoph Schmid"I completely agree. BMD should get rid of these remnants or make them work in the new environment. Contradictory statements in the manual also add to the confusion.[/quote]


Definitely. I really hate they removed proxy menu that was so useful and actually worked prior to Resolve 19 release. I guess as they integrate more of fusion into rest of resolve, the most fusion studio and fusion in resolve start to be differnt applications. This may not be a bad thing per se, if they both do the things that are useful, as long as there is well communicated reasons why something is differnt. We still don't have a lot of open FX tools in fusion studio, even things like 3D keyer or Patch Replacer that are very useful and available in fusion page, are not in fusion studio. So it seems that the migration process is a bit of a mess. And its slow going.

I always wondered why the glow effect in fusion looked so terrible in the edit page.. I did find the same solution by deactivating the alpha channel but never understood why, it was more or less luck. Interesting to read about your findings.

I always wondered why the glow effect in fusion looked so terrible in the edit page.. I did find the same solution by deactivating the alpha channel but never understood why, it was more or less luck. Interesting to read about your findings.

Thanks. Few days back I think I've also try to figure out why when you have clip with transparency, you go to file menu... Excort Current frame as Still... and save it as .png you get black background instead of transparent one until you interpret the clip as premultiplied. But if you go to fusion page and do Save as... and save as .png transparency is honored. As if edit page and color page, process images and video one way in the timeline and another way in the viewer. Still trying to get to the bottom of that one.

Covered more in this thread. :https://forum.blackmagicdesign.com/viewtopic.php?f=21&t=214304&p=1111746&hilit=magic+mask#p1111746