Blackmagic Forum

See this thread...

http://www.bmcuser.com/showthread.php?22311-Oops!-BRAW-isn%92t-Raw-%96-It%92s-a-Y%92CbCr-codec

What's BMD saying?

Ellory, one of the people posting in that thread mentions it could still be RGB values with the offsets for a red and blue rather than the full red and blue values. That seems to make more sense than saying it’s Y’CrCb. After all DaVinci Resolve Colour Managed works in RGB space, no? I did notice that glow or ring but I’ve seen that before with green screens; just thought that it was normal but it sounds like it’s problematic.


Sent from my iPhone using Tapatalk

rick.lang wrote:Ellory, one of the people posting in that thread mentions it could still be RGB values with the offsets for a red and blue rather than the full red and blue values. That seems to make more sense than saying it’s Y’CrCb. After all DaVinci Resolve Colour Managed works in RGB space, no? I did notice that glow or ring but I’ve seen that before with green screens; just thought that it was normal but it sounds like it’s problematic.

Offsets were mentioned as a way to transform values for more efficient compression, it has nothing to do with possible ringing because ringing would need bigger spatial mixup, offsets are 100% reversible (within limits of compression-induced rounding ofcourse). For example if you have three consecutive image elements (lets simplify for clarity) which represent a gradient where R=G=B and each element has different luminosity, one might just save the value of G and for R and B save difference from G. In this case it would be 0, so effectively we have reduced 2/3 of data to zeroes which massively helps compression.

Whether Davinci Color Managed works in RGB or not is totally irrelevant, conversion from Y'CbCr representation to RGB is computationally a non-issue in relation to debayer, gamut transforms and whatnot.

I had similar thought about move to luma-chroma space, because all this compression efficiency must come from somewhere and it seems like one of the first logical places to get it. Frank Glencairn suggests in that thread that ringing comes from wavelet transform, in that case it should be noticeably reduced between different compression rates. Wavelet case should be very easy to test though, compress CDNG file to Cineform raw and see what it looks like.

I'd rather guess, based on prores444 sample from that clip set with exact same ringing (that definitely should not be there for data originating as proper 444), that BM utilizes the same in-camera debayering that is used for Prores and output paths and dumps that data into braw recorder with some additional processing. Might be 100% wrong though, just a thought.

rick.lang wrote:Ellory, one of the people posting in that thread mentions it could still be RGB values with the offsets for a red and blue rather than the full red and blue values. That seems to make more sense than saying it’s Y’CrCb. After all DaVinci Resolve Colour Managed works in RGB space, no? I did notice that glow or ring but I’ve seen that before with green screens; just thought that it was normal but it sounds like it’s problematic.

I'm actually the one who posted that but I was really just kind of thinking out loud and showing my work lol

After thinking about it more, I realized why someone would go with Y'CbCr instead of Green with Red and Blue as offsets and it's because it's more consistent. With my idea, it would work fine a good portion of the time but it's too reliant on green which creates some major pitfalls if there's no green or if someone thing all green. Basically, in both situations the red and blue channels would not only require a bunch of precision but they would actually each need 13-bits instead of 12 in order to get enough contrast with the green channel. In other words a lot of the pixels in a picture of, for example, a magenta flower with leaves would be larger to store than if it was straight RGB values though, like I said in the original post, the desaturated, low-contrast nature of log footage would curb that issue quite a bit.

Your post made me think of something though. It does feel weird that a company that has a NLE known for it's color correction would make a camera and codec that requires going from RGB to Y'CbCr back to RGB and that made me think about what's happening at the sensor level and how the partial debayering fits into all this.

You see I'm trying to think about how to convert Bayer data into Y'CbCr data but the conversion from RGB to Y'CbCr requires combining data from all three RGB channels to Y, Cr, and Cb but since each bayer pixel only has one channel, it would require demosaicing. However, if it just held demosaiced data in the file then what part of the demosaicing work would be left for the decoder to do? The only other thing I could thinkin of is that they would demosaic just to convert to Y'CbCr and then discard pixels in each channel so you essentially have Y'CbCr version of the bayer pattern which would then be re-demosiaced (too many prefixes) by the decoder. That just sounds dumb though.

Previously I had theorized that the partial debayering happening in-camera was actually them doing all the gradient direction analysis on the image in-camera and then saving it as directional map in the file so that the decoder can take those values and complete the interpolation process software-side.

However after I've converted the CDNGs from that test into several different codecs and I do not see that issue in the ones that are RGB but every time I use one that's Y'CbCr, there's that artifacting. So it seems it is in fact Y'CbCr and maybe that extremely dumb sounding re-debayering idea is real.

Hendrik Proosa wrote:Wavelet case should be very easy to test though, compress CDNG file to Cineform raw and see what it looks like.

That was actually one of the codecs I converted it to and I saw the same ringing show up. For what it's worth, that ringing is actually present in the original CDNG file but to a much more subtle degree.

You can see it here.

While fiddling with my braw reader for Nuke I noticed something interesting in one of the test footages. Writing bride has macroblock-like artifacts on her face, especially strong in blue channel, but also visible in others (added images have increased contrast to make it more visible). I'm not a wavelet transform specialist but to my knowledge it can't produce same-sized blocks like DCT transform does. Block width is exactly 16 pixels, in case of dumping every other sample in Y'CbCr representation it would work out to 8x8 element DCT macroblock:


Another illustration, conversion to YCbCr shows that Cb and Cr channels have pretty heavy macroblocking (again, for illustration purposes, contrast is increased), but Y is untouched. IMHO this suggests pretty clearly that this raw has gone through luma-chroma conversion in some stage:


And to clarify the origins of the image, this is the raw float data coming directly from decoder. Decoding is done using BRAW SDK. YCbCr conversion is done in Nuke using Colorspace node.

As we know if transform video to Linear gamma, Expose and WB can be adjusted in ProRes file exact same as in DNG RAW.

So my guess BRAW may be just a wrapper for some kind of ProRes-like video codec which don't have any bayer data at all.
Actual Debayering done in camera similar to ProRes (Partial de-mosaic). And Resolve or BRAW SDK just operates in linear gamma similar to RAW (Reduced de-Mosac)

But this is just a guess, in real life things may be more complicated. Andrew Kolakowski post linc to other theory here viewtopic.php?f=2&t=79208&start=150#p439279

If we have such a big macroblocking ( I assume this is 12:1) than this is possible.
In this case whole BM website and info would be full of crap as they take uncompressed RAW as reference though.

I'm stopping to like fact that whole thing is such a secret. Another RED alike story?
There is no need for big secrets from BM side.

Hendrik Proosa wrote:While fiddling with my braw reader for Nuke I noticed something interesting in one of the test footages. Writing bride has macroblock-like artifacts on her face, especially strong in blue channel, but also visible in others (added image has increased contrast to make it more visible). I'm not a wavelet transform specialist but to my knowledge it can't produce same-sized blocks like DCT transform does. Block width is exactly 16 pixels, in case of dumping every other sample in Y'CbCr representation it would work out to 8x8 element DCT macroblock:

braw_macroblocks.png

Another illustration, conversion to YCbCr shows that Cb and Cr channels have pretty heavy macroblocking, but Y is untouched. IMHO this suggests pretty clearly that this raw has gone through luma-chroma conversion in some stage:

braw_macroblocks_02.png

And to clarify the origins of the image, this is the raw float data coming directly from decoder. Decoding is done using BRAW SDK. YCbCr conversion is done in Nuke using Colorspace node.

What is the BM RAW in this case- 12:1? Typical DCT macroblocking. Wavelet doesn't do such a macroblocks.
What is the zoom level?

Andrew Kolakowski wrote:What is the BM RAW in this case- 12:1? Typical DCT macroblocking.
What is the zoom level?

Bride clips are all 12:1, zoom level was 400% I believe when I took the snaps. Took a peek at greenscreen clips that circulated here, macroblocking in Cb and Cr is there in all compression levels, more compression, more pronounced.

Even with 3:1/Q0 clips?

Andrew Kolakowski wrote:Even with 3:1/Q0 clips?

Yes, it is there. Examples are with increased contrast, just to illustrate. Cb channel, same Colorspace node conversion.
Q0:

3:1

The overall look of braw is pleasing to me, but I don't think this hide and seek play BMD is pulling is a good move. Advertising something as raw when it seems to not be that at all is just bad. Compression is one thing, but doing conversions out from RGB data also is whole another story.

What is even funnier, macroblock prominence in Q0/3:1 seems visually to be on the level of Prores422HQ.

Hendrik Proosa wrote:
What is even funnier, macroblock prominence in Q0/3:1 seems visually to be on the level of Prores422HQ.

Which is not good as it should be way lower at 3:1. ProResHQ is about 6:1.
It should be on level of ProRes XQ. It means codec is very simple.

Fascinating observations looking under the covers. For now, I’m staying with ProRes 444 on the Mini 4.6K primarily to minimize uncontrollable moiré until BRAW becomes clearer. By the time I’ve got the BMPCC4K, I’m hopeful best practices will be known for BRAW. Based on bitrates I really expected Q0 to be as good or better than ProRes XQ and CinemaDNG 3:1.

CinemaDNG 3:1 has been my go to codec for most shoots including weddings, but this time I’ll switch.

rick.lang wrote:Fascinating observations looking under the covers. For now, I’m staying with ProRes 444 on the Mini 4.6K primarily to minimize uncontrollable moiré until BRAW becomes clearer. By the time I’ve got the BMPCC4K, I’m hopeful best practices will be known for BRAW. Based on bitrates I really expected Q0 to be as good or better than ProRes XQ and CinemaDNG 3:1.

CinemaDNG 3:1 has been my go to codec for most shoots including weddings, but this time I’ll switch.

Ditto. That's when I'll repurchase the BMPCC4K (when it is supported there) and for now just stick to my trusty BMPCC and URSA as well.

Dmitry Shijan wrote:So my guess BRAW may be just a wrapper for some kind of ProRes-like video codec which don't have any bayer data at all.
Actual Debayering done in camera similar to ProRes (Partial de-mosaic). And Resolve or BRAW SDK just operates in linear gamma similar to RAW (Reduced de-Mosac)

That wouldn't make any sense.

One of the big advantages of a undebayered image is, that it is only black and white, so the data rate is way lower than a debayered image.
If you want to make a low data rate codec, why would you throw that advantage out of the window, and than counter it with loss of image information by even more compression? So you would ether raise the data rate with Y’CbCr (compared to a undebayered image), or asking for compression losses/artifacts.

That's nothing BM would develop as a new flagship codec, it's totally against their DNA as I know it.
Also we already have that since years, it's called ProRes and DNX - why make an other one?

It is possible to to transform to YCbCr without debayer, adjascent photosites can be used without interpolation if pattern is known and stored. This might be useful for compression, but it has nothing to so with partial debayer, so interesting part is, what does BMD mean with this partiality? Because this seems to be pretty obvious that YCbCr representation is used for compression, compressing sensor photosite RGB data will not produce compression artifacts in chroma only.

The only thing about partial debayering I found is this:

https://books.google.co.uk/books?id=2O5 ... CXoECAIQAQ

BRAW analysis by Daniel Rozsnyó.

Bottom line is, all recording formats are Codecs of one type or another, including DNG Raw. BRaw is no different.
Cheers

DNG RAW is RAW Bayer data (compressed or not), where BM RAW is not so obvious. It's messed around RAW data with pre-processing applied (and users's 0 control over this pre-processing). We can already see it's quite heavily de-noised and bit soft because of it. I rather had RAW data and choose in Resolve what I wan to do with it. Starting not like BM RAW at all- way to much is happening outside of user control to call it RAW format. Suddenly BM found so much processing power in camera that they can do amazing things? Surely Resolve with all GPU power can do way more, so something is not 100% clear here. Sounds like lots of pr and talk about not much at the end.

Andrew Kolakowski wrote: Suddenly BM found so much processing power in camera that they can do amazing things?

The Power was always there - how do you think Prores/DNX was made in camera the whole time?

By suboptimal debayering compared to Resolve one.
If this is just partial (still no one knows what it means) debayering then there is spare processing power I assume. The more we know the less impressive BM RAW becomes. It's not impressive at all anymore actually.

And as someone else mentioned- by providing SDK BM didn't really make BM RAW open source. Opens source is when source code is out there so you can see what is going on and make now changes, etc. Pre-compiled librarries are not really "open source" as far s I understand it. They just allow you easily integrate codec yo your app. If SDK doesn't allow you to get assess to "RAW" (or whatever it's) data then it's same "black box" like RED SDK is.

roger.magnusson wrote:BRAW analysis by Daniel Rozsnyó.

This is interesting, seems to confirm most ideas floating around, like utilizing (pseudo) luminance-chrominance representation and DCT based compression. On the bright side it seems to also confirm that there is, within limits of compression, possible to get back to Bayer pattern data. But it is still unclear if apparent softness comes from compression only or some kind of additional in-camera noise processing.

Andrew Kolakowski wrote:It's messed around RAW data with pre-processing applied (and users's 0 control over this pre-processing).

In the end, what is your goal? Purist raw, or maintaining raw image quality with easier post?

We can already see it's quite heavily de-noised and bit soft because of it. I rather had RAW data and choose in Resolve what I wan to do with it. Starting not like BM RAW at all- way to much is happening outside of user control to call it RAW format.

Since it's currently in beta, now would be an ideal time to pass on that sort of feedback to BMD...

Suddenly BM found so much processing power in camera that they can do amazing things? Surely Resolve with all GPU power can do way more, so something is not 100% clear here. Sounds like lots of pr and talk about not much at the end.

You do remember that BMD uses FPGAs in its cameras, right? So the de-Bayer isn't fixed, it's hardware... but reprogrammable.

BMD's goal is to be budget friendly. If you prefer the expensive solution get a Red, a ThreadRipper, and a Turing. BMD's trying to deliver a workflow that makes economical sense for a production company whose A camera is an Ursa Mini or Pocket 4K.

Shooting my last feature in cDNG would have required nine eight terabyte disks. Using Redcode, we ended up needing only three, since one eight TB disk is big enough for all of our dailies.

It's a win-win here... you get to use braw when you want a nice, smooth worfklow, and you have access to cDNG in case you actually need it, which you probably won't by the time braw is in full release and a bit more mature than just launched.

Main issues- to heavy de-nosing without any user control and examples of big macro-blocking which should not really be there at 3:1 compression level. Lack of proper information for public (eg. detailed white paper) and big claims about open source nature, which are not really true. This is about it. Other than this have not much against BM RAW format itself.

Hendrik Proosa wrote:

roger.magnusson wrote:BRAW analysis by Daniel Rozsnyó.

This is interesting, seems to confirm most ideas floating around, like utilizing (pseudo) luminance-chrominance representation and DCT based compression. On the bright side it seems to also confirm that there is, within limits of compression, possible to get back to Bayer pattern data. But it is still unclear if apparent softness comes from compression only or some kind of additional in-camera noise processing.

I'm 95% sure softness is due to some de-noising. Even simplest codec at 3:1 should not be that soft.

Andrew Kolakowski wrote:Main issues- to heavy de-nosing without any user control and examples of big macro-blocking which should not really be there at 3:1 compression level. Lack of proper information for public (eg. detailed white paper) and big claims about open source nature, which are not really true. This is about it. Other than this have not much against BM RAW format itself.

I don't know why you insist that they claimed it is open source. They never did and there is no hint on their website.

Ok- they call it "open standard"

"Blackmagic RAW is the world’s only truly modern, high performance, professional RAW codec that is open, cross platform and free."

Providing pre-compiled libraries doesn't really count as "open standard" if I understand it properly. It's still unclear if you can even get access to actual "RAW" (or whatever it's) data.

The “open standard” refers to being able to use rhe SDK files to add support to BRaw in non BM NLEs for decoding. It has nothing to do with incoding (recording in camera), so it is not open source. Your in,y access is what the SDK or Resolve allows.
Cheers

How is this different to Arri, RED etc. then?

Hendrik Proosa wrote:

Andrew Kolakowski wrote:What is even funnier, macroblock prominence in Q0/3:1 seems visually to be on the level of Prores422HQ.

In my tests I'm still seeing useful benefits to using BRAW (even at 12:1) over ProResHQ. From test shots of a tree moving in the wind against the sky, I keyed the sky and pushed it to magenta. Below is the zoomed in result comparing the same grade from ProResHQ v BRAW 12;1 in highlight mode.
I see lots of DCT macroblocks in the key pulled from ProRes, but none on the shot from BRAW. I don't have the chops to delve into the possible math BMD used for BRAW, but visually I'd say BRAW is producing better, cleaner image data than ProResHQ.

Very Interesting... Jamie, thanks for the look at the differences.
Cheers

Yes, very interesting. But motion doesn’t matter, both codecs are I-frame only.

Never said the difference had anything to do with motion.
Motion or no motion, stressing the grade revealed compression artifacts in ProResHQ that simply aren’t there in BRAW.

EDIT: Thanks for clarification Uli

I was only referring to trees moving in the wind.
We both know about non-GOP formats. But others might draw the wrong conclusion.

Jamie LeJeune wrote:

Hendrik Proosa wrote:

Andrew Kolakowski wrote:What is even funnier, macroblock prominence in Q0/3:1 seems visually to be on the level of Prores422HQ.

In my tests I'm still seeing useful benefits to using BRAW (even at 12:1) over ProResHQ. From test shots of a tree moving in the wind against the sky, I keyed the sky and pushed it to magenta. Below is the zoomed in result comparing the same grade from ProResHQ v BRAW 12;1 in highlight mode.
I see lots of DCT macroblocks in the key pulled from ProRes, but none on the shot from BRAW. I don't have the chops to delve into the possible math BMD used for BRAW, but visually I'd say BRAW is producing better, cleaner image data than ProResHQ.

Look few posts above. BRAW (at least some modes) should definitely produce better result than ProResHQ.

Just to clarify, my example images have increased contrast for illustration purposes, to show that macroblocks are there on all levels of compression. My comment about blocking being visually similar to Prores422HQ comes from comparing them YCbCr representation. I made the test by encoding cdng footage to different Prores flavors, both with and without additional denoising. Tried denoised cdng because braw is arguably being denoised at least to some extent in camera.

Another example, noisy cdng converted to 422hq on the left and braw Q0 on the right, Cb channel, increased contrast for illustration purposes (cdng debayer in Nuke is awful, don't mind the "interlaced" look):


And the same thing when cdng is denoised:

Hi, I haven't got to read this thread yet, but elsewhere is was wondering if this was the answer as to how BRaw works?

Wayne Steven wrote:I think I have cracked what how BRaw is breaking up processing leading to the results you see in the samples.

You seperate out the luma and chroma from the Bayer pattern. This leaves the whole Bayer pattern as continuous Luma data to be compressed as waveform. Chroma tends to be area based which is easier to compress, so you compress the three colour channels seperately. The maximum discernable colour is yellow with something like 27 grades, so the chroma data can be low bit depth. Both should get more compressible, but maybe the sum of the two isn't as small as DNG. More aggressive compression to compensate will reduce the accuracy which might account for the less sharp detail we see. This is basically a form of Bayered 4:4:4 which might account for the light fringe around colour tiles people are seeing. It is displayable.

On the other end, it basically uses this data to complete debayering to a high standard. But I think the compression process is more aggressive, you may loose quality compared to DNG. But in reality, you get speed at the sacrifice of file size and quality. If you were to match it and DNG in quality would it be much faster, and would it be even larger? Let's just hope it is niose removal causing much of this, and that they can further tune this. Pretty interesting way of doing it.

Well reading the thread and that analysis now, it seems to be like I'm saying. It maps back to Bayer values, but is simply a step in the internal debayering process to go to screen or output. It's not going be Bayer unless it can go backwards.

Uli Plank wrote:Yes, very interesting. But motion doesn’t matter, both codecs are I-frame only.

But intra codecs will macroblock based on the compressibility of the scene. Wavey trees might not get as easy to compress. But this seems to get more an alternative to ProRes than DNG based on detail and macroblocking people are finding. I wonder what waves on the are going to look like.

Robert Niessner wrote:

Andrew Kolakowski wrote:Main issues- to heavy de-nosing without any user control and examples of big macro-blocking which should not really be there at 3:1 compression level. Lack of proper information for public (eg. detailed white paper) and big claims about open source nature, which are not really true. This is about it. Other than this have not much against BM RAW format itself.

I don't know why you insist that they claimed it is open source. They never did and there is no hint on their website.

Open standard usually implies you know the internals, and it is open to use (encode and decode).

Andrew Kolakowski wrote:How is this different to Arri, RED etc. then?

The biggest difference is that BMD is willing to work with other companies to support Braw, rather than limiting it to their own cameras.

Whether or not other camera manufacturers would get on board is another story entirely.

I’m not sure about that. The way I understand them they are ready to support other software to decode it.

Maybe I'm being optimistic, but it WOULD be an improvement all around; people with sub-$10K Sony, Canon, and Panasonic cameras could get a raw codec that isn't limited to a niche NLE, and everyone can edit it without extra investment due to BMD's generosity... until people realize how silly they're being for resisting the $300 that it costs for the studio version, since there are some plugins that would EACH cost several hundred dollars if they weren't coming from Black Magic.

Rakesh Malik wrote:

Andrew Kolakowski wrote:How is this different to Arri, RED etc. then?

The biggest difference is that BMD is willing to work with other companies to support Braw, rather than limiting it to their own cameras.

Whether or not other camera manufacturers would get on board is another story entirely.

You have free access to Arri, RED, Sony (not 100% sure) SDKs and can add decoding into your app exactly same way as for BM. I don't really see a difference here.
If BM wants to "give" encoding for the remanufactures for free then fine, but I'm not sure if that many going to jump into it. Definitely not RED, Arri or Sony. Each of them can do it themselves, they don't really need BM for this. It's financial and strategic decision to give RAW out of camera. There are no technical difficulties- nothing more than in case of other (already debayered) formats, like ProRes etc. One RAW format would simplify things, but I I doubt it will ever happen.

No where has BM stated they are going to share BRaw with other camera manufacturers. Actually the opposite, the SDKs are for decoding only, not incoding, and Mr Lam verified this that BM is only sharing decoding using rhe SDKs they have released.
Cheers

Andrew Kolakowski wrote:You have free access to Arri, RED, Sony (not 100% sure) SDKs and can add decoding into your app exactly same way as for BM. I don't really see a difference here.

The question is whether or not BMD will be willing to share the specs for encoding to other camera manufacturers, which would probably have no affect on BMD's camera sales unless BMD plans on moving up a budget tier which I doubt it will, since BMD seems to prefer to wait until it's able to deliver its newer cameras without raising their prices.

If BM wants to "give" encoding for the remanufactures for free then fine, but I'm not sure if that many going to jump into it. Definitely not RED, Arri or Sony.

Red and Arri, certainly correct. Same for Sony's CineAlta line and Pansonic's Varicam line. However, the sub-$10K Sony, Panasonic, and Canon cameras that do not include internal raw recording, and instead record raw via cDNG on external recorders, are strong candidates, especially in light of Atomos' adoption of ProResRaw.

In other words, any manufacturer that's on board with ProResRaw should be champing at the bit to join the BRaw party.

Each of them can do it themselves, they don't really need BM for this. It's financial and strategic decision to give RAW out of camera. There are no technical difficulties- nothing more than in case of other (already debayered) formats, like ProRes etc. One RAW format would simplify things, but I I doubt it will ever happen.

And yet the cinema cameras priced between $8K and $10K don't have internal raw recording, and most of them are on board with ProResRaw.

Denny Smith wrote:No where has BM stated they are going to share BRaw with other camera manufacturers. Actually the opposite, the SDKs are for decoding only, not incoding, and Mr Lam verified this that BM is only sharing decoding using rhe SDKs they have released.
Cheers

I think Grant indicated in one interview that they may contemplate doing it for other cameras in future.

Where did somebody mention doing it with jpeg? I have contemplated doing that too. Jpeg HDR/high bit depth (obscure codec) got good compression ratio at high bit depth.

Rakesh Malik wrote:The question is whether or not BMD will be willing to share the specs for encoding to other camera manufacturers...

BMD doesn't even share the general technical information about the codec, I think sharing encoder to other manufacturers is wishful thinking at best in current stage. Let BMD produce a whitepaper first about this new "open standard". All the current official info has been saying the same things in different wording without actually saying anything. Partial demosaic means absolutely nothing in itself and it is the only thing they have actually said besides having CBR and VBR modes that has even mild technical hint in it.