Blackmagic Forum

I'm very interested in how braw does this nearly invisible compression other than the softening at 12:1. How, generally, does this work? I understand there's probably some proprietary stuff involved that cannot be shared, but I have always been interested in this stuff!

Thanks and have a good day!

Have you seen this? He basically explains the whole process from the moment the light hits the front of your lens, through to final delivery.

Nick Heydon wrote:Have you seen this? He basically explains the whole process from the moment the light hits the front of your lens, through to final delivery.

Ah, I thought it was just an announcement for .braw but it's a full explanation! Thanks so much!

Doesn't really explain how the compression works though.

smunaut wrote:Doesn't really explain how the compression works though.

You can read more in detail in the patent:
https://patents.google.com/patent/US201 ... n&sort=new

It’s based on DCT. You can look that up as a compression scheme.

It also has a unique implementation. So for example it’s based on prioritising how much of the image is in focus. A long lens shot that is shot wide open with less depth of field will compress higher than a wide angle T16 shot.

JB

John Brawley wrote:It’s based on DCT.

Ah yes, DigiBeta.

RED asserts a patent on "compressed" RAW. Only RED and 4 Sony cameras (F5/F55/F65/Venice) record compressed RAW (settled litigation).

BRAW is compressed, so is BRAW really RAW?

Noel Sterrett wrote:

John Brawley wrote:It’s based on DCT.

Ah yes, DigiBeta.

RED asserts a patent on "compressed" RAW. Only RED and 4 Sony cameras (F5/F55/F65/Venice) record compressed RAW (settled litigation).

BRAW is compressed, so is BRAW really RAW?

Some say no, but for me if it works like raw, meaning all the parameters that can be adjusted. its raw for me.


Ricardo Marty

Noel Sterrett wrote:

John Brawley wrote:It’s based on DCT.

Ah yes, DigiBeta.

RED asserts a patent on "compressed" RAW. Only RED and 4 Sony cameras (F5/F55/F65/Venice) record compressed RAW (settled litigation).

BRAW is compressed, so is BRAW really RAW?

I think the answer here depends on what you think RAW means.

For me being RAW means...

1. Can I set a white point / tint in post by Kelvin.
2. Can I set an ISO / EI in post by Number.
3. Can the shot be reprocessed or demosiaced

BRAW is 2.5 of those three to me and is functionally acting like RAW.

It’s more efficient than ProRes 444 in terms of playback, has smarter metadata (LUTs etc) and functionally grades like RAW.

There’s a movement to this weird purist version of RAW but in reality that means you’d argue that an 8 bit uncompressed RAW DNG from a sigma FP is more “RAW” than a 12 bit BRAW file. I know which one I think gives a better result, is easier to actually work with and has less post issues.

Uncompressed DNG is a dead end and frankly as resolution goes up it’s even more dead end.

Yes, DCT is also what Digibeta, HDCAM, HDCAM SR but also ProRes, ProRes RAW, all the DV, DVCAM, DVCPRO formats is based on. The very latest version of REDs own REDCODE for the Komodo is also DCT based

JB

Noel Sterrett wrote:

John Brawley wrote:It’s based on DCT.

Ah yes, DigiBeta.

RED asserts a patent on "compressed" RAW. Only RED and 4 Sony cameras (F5/F55/F65/Venice) record compressed RAW (settled litigation).

BRAW is compressed, so is BRAW really RAW?

As long as i get the advantages of RAW-
Change White balance
Change ISO
Allows for heavy colour grading

then thats RAW enough for me. I really don't care about the ones and zeros

if you look at the BMD patent it basically looks like the debayer (or a good part of it) is done before coding / compressions. Convert sensor data to YCbCr 4:2:0 and then encode that.

BRAW is not technically RAW data but it is of a very high quality, enough to be able to change RAW controls in post prod (ISO, White Balance etc.). I think maybe ProRes 4444 recorded with a log curve could theoretically be of higher quality, depending on how it is implemented inside the camera.

smunaut wrote:Convert sensor data to YCbCr 4:2:0 and then encode that

Don't say that you'll make some people angry here

Noel Sterrett wrote:RED asserts a patent on "compressed" RAW. Only RED and 4 Sony cameras (F5/F55/F65/Venice) record compressed RAW (settled litigation).

What about Canon Cinema RAW Light? C200, C300 MKIII and C500 MKII I think.

A trade for the RF mount?

antoine wrote:BRAW is not technically RAW data but it is of a very high quality, enough to be able to change RAW controls in post prod (ISO, White Balance etc.). I think maybe ProRes 4444 recorded with a log curve could theoretically be of higher quality, depending on how it is implemented inside the camera.

ProRes 444 is very very good and I’ve shot that codec more than any other in my career. It’s really good.

But it renders slower than BRAW on my machine. I get higher FPS out of BRAW on Mac hardware.

It also will suck at higher than 4k resolution.

BRAW also allows other things like the in-camera scaling on the 12k.

Think about the 12k. How would you get a 12k 444 ProRes file using existing media. (Do the numbers)

Or even 8k.

JB

smunaut wrote:if you look at the BMD patent it basically looks like the debayer (or a good part of it) is done before coding / compressions. Convert sensor data to YCbCr 4:2:0 and then encode that.

Do you read in patent that it down sampling color to 4:2:0 or is your opinion?


Inviato dal mio iPhone utilizzando Tapatalk

It's written "as is" in the patent : "YCbCr420 format data" figure 3.

But it's not really "downsampling". Because with Bayer pattern you don't have any more color information to begin with ... so you haven't thrown any information away, you just haven't done a full debayer to 4:4:4.

smunaut wrote:It's written "as is" in the patent : "YCbCr420 format data" figure 3.

But it's not really "downsampling". Because with Bayer pattern you don't have any more color information to begin with ... so you haven't thrown any information away, you just haven't done a full debayer to 4:4:4.

thanks, i not read patent and i think i not have enough knowledge to understand correctly it

smunaut wrote:"YCbCr420 format data"

YCbCr is an efficient video color space (eg., DigiBeta). No doubt "Blackmagic RAW" is as well. But marketing it as "RAW" is quite a stretch.

Marketing anything that's compressed as RAW is stretching it.
Even log conversion is throwing away some information.

4:4:4 sampling refers to 12/12 pixels.

4:2:0 refers to 6/12 pixels.

For the same Cmos contains only 4.

4:2:0 sampling is sufficent to capture uncompressed raw and have enough room left over to have 24 audio channels muxed in. Unfortunately that is illegal.

Braw is necessarily not raw so the luma channel is debayered and sampled, 4:0:0. Now there are only 2 samples left but there are also only 2 real colors. What if Braw references the original raw value and not the debayered value? Now we have a format that satisfies Red's condition that the chroma resolution is necessarily lower than the luma and haven't given up much of anything especially if the luma is 12 bit.

Good Luck

A win for Howie.

JB

John Brawley wrote:
ProRes 444 is very very good and I’ve shot that codec more than any other in my career. It’s really good.

But it renders slower than BRAW on my machine. I get higher FPS out of BRAW on Mac hardware.

It also will suck at higher than 4k resolution.

BRAW also allows other things like the in-camera scaling on the 12k.

Think about the 12k. How would you get a 12k 444 ProRes file using existing media. (Do the numbers)

Or even 8k.

JB

I agree, workflows are also very important and we can see the people at Blackmagic planned everything to make it smooth everywhere, that's really cool

Noel Sterrett wrote:

smunaut wrote:"YCbCr420 format data"

YCbCr is an efficient video color space (eg., DigiBeta). No doubt "Blackmagic RAW" is as well. But marketing it as "RAW" is quite a stretch.

I've skimmed the patent and I think there's a confusion about "YCbCr420 format data" because of some missed context.

If I understand everything correctly the idea is to convert raw sensor data to Y, Cb and Cr images which allows to treat it as YCbCr420 format data without any data loss because of configuration of sensor photosites. Section [0112] of the patent is talking about it and FIG 10 is an example for sensors with Bayer filter. So "YCbCr420 format data" here is just a convenient container which allows to use existing data formats to transfer raw sensor data. And it shouldn't be confused with conventional meaning for YCbCr color space and especially 4:2:0 chroma subsampling.

Now some potential data loss can occur when raw sensor data is converted to Y, Cb, Cr images. And especially when that raw sensor data in "YCbCr420 format" is compressed - see for example sections [0040], [0041] of the patent for some recommendations about compression. But at the same time image sensors are not some kind of "perfect" devices. And I guess there is some kind of averaging / smoothing of the sensor data happens in any case. At the end of the day raw sensor data with some data loss can regain more image information than the same final image with the same data loss.

antoine wrote:If I understand everything correctly the idea is to convert raw sensor data to Y, Cb and Cr images which allows to treat it as YCbCr420 format data without any data loss because of configuration of sensor photosites.

If that were true, you could convert YCbCr420 back the the identical raw data. I do not believe that is the case.

In a Bayer array, the two green pixels are in different locations, and that is not unimportant.

I never said there was any loss ...

Actually I even said explicitly there was not : """But it's not really "downsampling". Because with Bayer pattern you don't have any more color information to begin with ... so you haven't thrown any information away"""

Before the compression (which is obviously lossy), the whole transforms are linear and reversible, so you could even get the original bayer data back if you wanted to.

Noel Sterrett wrote:If that were true, you could convert YCbCr420 back the the identical raw data. I do not believe that is the case.

Yeah you can ... it's just linear algebra, the transform is reversible before you introduce the quantization/compression.

Four values in four different locations are transformed into three values in one location. It is not reversible.

Noel Sterrett wrote:Four values in four different locations are transformed into three values in one location. It is not reversible.

?!? That's not how a bayer pattern works. You only get 1 value per pixel. Your "4k sensor" has ~ 4M green pixels, 2M red pixels and 2M blue pixels.

For a 4096x2160 bayer sensor you start with a single 4096x2160 image ( 8M values ) that in their described filter gets converted into :
- 4096x2160 Y luma image
- 2048x1080 Cb chrome image
- 2048x1080 Cr chrome image

or aboute 12M distinct values ....

smunaut wrote:That's not how a bayer pattern works. You only get 1 value per pixel.

One value per pixel after de-mosaic, but 4 values per mosaic, GRBG, RGGB, etc.. All are in different locations.

Once you de-mosaic, you can't go back to the original.

BRAW is not necessarily bad. It has many worthwhile features, it's just not RAW.

No ...

That's not how the resolution of camera sensors works. When they tell you it's 4096x2160, they're counting the sub pixels. A mosaic of RGGB is 4 pixels.

Read this for instance ( random google hit but looks right ) :
https://www.rocketstock.com/blog/4k-cam ... really-4k/

smunaut wrote:A mosaic of RGGB is 4 pixels.

That's what I said - "4 values per mosaic".

Once you transform from RGGB (4 values) to YCrCb (3 values) , you can't get the same 4 values back. GG is not the same GG.

Noel Sterrett wrote:

smunaut wrote:A mosaic of RGGB is 4 pixels.

That's what I said - "4 values per mosaic".

Once you transform from RGGB (4 values) to YCrCb (3 values) , you can't get the same 4 values back. GG is not the same GG.

You're completely missing the fact that you're not transforming it into 3 values ...
Each mosaic gets converted to 4 Y, and 1 Cr and 1 Cb ... hence 6 values ...

Demonstrate, with the appropriate mathematics, a transform from RGGB to YCrCb and back that produces identical RGGB values.

Noel Sterrett wrote:Demonstrate, with the appropriate mathematics, a transform from RGGB to YCrCb and back that produces identical RGGB values.

RGGB is converted to YYYYCrCb - please check FIG 10 in the patent for details. And that conversion is the main idea of the patent. Sections [0031] and [0032] can be a starting point for the math used. And section [0113] is a starting point for the math needed to reconstruct the raw sensor data.

RGGB -> YCrCb is not reversible. I have not heard anything from BMD to suggest that it is.

BRAW is great, it's just not RAW.

Who is doing true, unaltered RAW these days? IIRC, the Thomson Viper did it, about 20 years ago.

Oh, and BTW, more than 90% of production on Arri Alexas is done in ProRes.

Uli Plank wrote:Who is doing true, unaltered RAW these days? IIRC, the Thomson Viper did it, about 20 years ago.

Oh, and BTW, more than 90% of production on Arri Alexas is done in ProRes.

Viper wasn't RAW if I recall, more LOG. Google Viper Filmstream. It was more designed to work with TAPE.

Dalsa was the first to do 4K raw. (yeah it wasn't RED)

From memory they were doing 16 bit linear 4K uncompressed back in the early 2000's....

I think the bigger question here is what IS RAW as far as Noel is concerned? And why would it matter?

And we're now way beyond the OP.

JB

Noel Sterrett wrote:RGGB -> YCrCb is not reversible. I have not heard anything from BMD to suggest that it is.

BRAW is great, it's just not RAW.

It seems you're completely ignoring what Sylvain, me and the patent are saying.

Just for the sake of completeness. It's not RGGB -> YCrCb. No. No. Not at all.

It's RGGB -> YYYYCrCb which is perfectly reversible. I've attached the FIG 10 from the patent:



And there are quotes of sections [0113] and [0114] from the patent - note "This process allows reconstruction of the sensor raw image 102":

[0113]
The method may further include transforming the YCbCr 420 image to a different image format for further image processing. This can include de-coding the YCbCr image data and convert it back into RGB image format. For the illustrative embodiment, the RGB values of each pixel can be generated using the following calculations:

R(i,j)=kR×CR(i,j)+Y(i,j) for {i=1, . . . ,N; j=1, . . . ,M}
G(i,j)=Y(i,j)−kR/2×CR(i,j)−kB/2×CB(i,j) for {i=1, . . . ,N; j=1, . . . ,M}
B(i,j)=kB×CB(i,j)+Y(i,j) for {i=1, . . . ,N; j=1, . . . ,M}

[0114]
where (i,j) indicates the pixel location. This process allows reconstruction of the sensor raw image 102.

"Demoacing" is mentioned 44 times in the patent:

"[ 0005 ] A process known as “ demosaicing ” is used to generate full color information for each photosite. In this process , the missing color information of a photosite is determined from neighboring photosites by an interpolation process."

The secret sauce in demosaicing is the interpolation process. Which photosites are included in the interpretation, and what are their weights?

Figure 5(a) shows 9. Nine photosites, then, are used to generate one value. The problem is that you can't de-interpolate those original nine values from the interpolated value you have.

BRAW does not just store RAW in a more convenient format. If it did, there could be a patent issue.

To my knowledge, no one representing BMD has suggested original RAW data can be resurrected from a recorded BRAW file. If it could, there could be a patent issue.

BRAW is great, it's just not compressed RAW. If it was, there could be a patent issue.

Uncompressed RAW, on the other hand, has no known patent issue. Which is likely the reason it was chosen by Sigma for the fp. It is unfortunate that BMD dropped CinemaDNG from their cameras.

One can't cherry pick quotes from the patent doc as it describes conventional, raw, Braw, and alternate Braw encoding implementations all within the same document.

I'm about 99% sure that Fig 10 is Braw. At the lowest level of processing Braw uses 2D Gaussian filter kernels, there's no way to reconstruct original RGB values because they were never recorded to begin with, rather approximated from a plurality of neighbor pixels.

Good Luck

Howard Roll wrote:I'm about 99% sure that Fig 10 is Braw. At the lowest level of processing Braw uses 2D Gaussian filter kernels, there's no way to reconstruct original RGB values because they were never recorded to begin with, rather approximated from a plurality of neighbor pixels.

Which is where I think BMD's "partial de-mosaic" comes from.

JB

Noel Sterrett wrote:"Demoacing" is mentioned 44 times in the patent:

"[ 0005 ] A process known as “ demosaicing ” is used to generate full color information for each photosite. In this process , the missing color information of a photosite is determined from neighboring photosites by an interpolation process."

This section is from "Description of the Related Art" chapter.

Noel Sterrett wrote:The secret sauce in demosaicing is the interpolation process. Which photosites are included in the interpretation, and what are their weights?

Figure 5(a) shows 9. Nine photosites, then, are used to generate one value. The problem is that you can't de-interpolate those original nine values from the interpolated value you have.

BRAW does not just store RAW in a more convenient format. If it did, there could be a patent issue.

To my knowledge, no one representing BMD has suggested original RAW data can be resurrected from a recorded BRAW file. If it could, there could be a patent issue.

BRAW is great, it's just not compressed RAW. If it was, there could be a patent issue.

Uncompressed RAW, on the other hand, has no known patent issue. Which is likely the reason it was chosen by Sigma for the fp. It is unfortunate that BMD dropped CinemaDNG from their cameras.

The result of the classical demosaicing process is an image with final RGB values for every pixel. As I understand it they do some kind of averaging of values for pixels of the same color. It requires some knowledge about the sensor and helps to reduce noise level. Maybe to avoid any patent issues that averaging process is called "demosaicing" in the patent. And I guess it's called "partial de-mosaic" in all other places. But the final result of the method described is to preserve colors for all pixels of the sensor (RGGB for Bayer filter). So at the end it has all the raw data from the sensor but with some kind of averaging / smoothing applied.

Noel Sterrett wrote:I don't care about 8K or 12K, 4K is plenty. I do care about color.

I think one of the ideas about 12K sensor is to get more accurate colors for 4K images. With multiple sensor photosites per image pixel "partial de-mosaic" in BRAW makes perfect sense. Imagine a 4K camera with the 12K sensor (BMPCC 4kPRO?). Partial de-mosaic can process 12K data from the sensor and get 4K BRAW at the end because you don't really want 12K raw data for a 4K camera. Also "what is raw sensor data?" question becomes even more philosophical in this case.

hhdima wrote:I think one of the ideas about 12K sensor is to get more accurate colors for 4K images. With multiple sensor photosites per image pixel "partial de-mosaic" in BRAW makes perfect sense. Imagine a 4K camera with the 12K sensor (BMPCC 4kPRO?). Partial de-mosaic can process 12K data from the sensor and get 4K BRAW at the end because you don't really want 12K raw data for a 4K camera. Also "what is raw sensor data?" question becomes even more philosophical in this case.

Downresing, i.e., compressing, from 12K to 4K can preserve the RAW nature of the sensor. But it is no longer really RAW sensor data. The Sigma fp does a 6K to 4K downres. But it's still RAW, just compressed.

I want to get as close to the sensor as possible, camera manufacturers want to keep me as distant as possible. Luckily for me, there is at least one "uncompressed" RAW option.

I moved the Sigma BRAW / DNG discussion to here:

viewtopic.php?f=2&t=138074

Noel Sterrett wrote:
To my knowledge, no one representing BMD has suggested original RAW data can be resurrected from a recorded BRAW file. If it could, there could be a patent issue.

If you ask some people (who know a lot about RAW and processing) then they say (and done it) you can recover RAW data. If it's not 100% RAW data it's eg. 98% which is easily good enough in practice.

Not sure where this obsession about RAW as "RAW data from sensor" comes from. It doesn't have to be 100% untouched sensor data to get all the advantage of it. There are so many other aspects which are as (if not more) important.

BRAW SDK doesn't provide "official" access to RAW data, opposite to ProResRAW or Cineform RAW.
Funny that whole internet was screaming that ProRes RAW is not real RAW (because they had no ISO etc. controls), where in reality it's areal RAW (you have access to RAW pixels in the decoder) and if anything BRAW is not (by pure fact that I have no access to RAW pixels and can't use eg. my own processing). It still doesn't change fact that both formats have good and bad points and overall represent nothing more than 20 years old Cineform RAW.

Invert this! Just kidding, the sacrifice of braw is resolution not color.

Good Luck

I assume you are talking about 12K one, not older models.

No that's the R/B Braw color channel filter for Bayer, for the 12K each color has a discrete filter.

Good Luck