Ursa Mini 4.6k RAW lossless VS. 3:1

Hey everybody,

In preparation for my 4.6k arrival, I've been curious about raw compression. I understand 3:1 is lossy and I'm discarding two thirds of the original sensor data, and lossless CinemaDNG retains this data conversely. The tradeoff in data retention being much smaller file sizes, roughly 50% from what I can gather:

(correct me if I'm wrong here)
4608 x 2592 CinemaDNG RAW 3:1 - 144 MB/s (24FPS)
4608 x 2592 Lossless CinemaDNG RAW - 310 MB/s (24FPS) (found this value from a reply in another thread)

My question is this: What type of data is being discarded and how is this determined? Two thirds sounds like an awful lot; and phrases like 'visually lossless' and 'you're probably not going to notice a difference' seem rather vague. Granted, I don't plan on projecting my footage onto the moon YET but if there's a discernible difference I'm curious as to what that might be. I enjoy manipulating my BMPC raw footage with Resolve and would hate to run into a situation where I'm getting artifacting or some other issue all because I didn't want to buy more CFast cards.

Anyway, if anyone has technical or anecdotal insight into this it would be greatly appreciated.

Thanks!

To put compression into perspective:
Downloaded X.264 MKV HD movie ~ 250:1
HDV = 127:1
AVCHD = 133:1
XDCAM EX = 38:1
XDCAM HD = 27:1
XAVC-4k = 27:1

ProRes 422 = 8:1
ProRes 422 HQ = 5.4:1
DNxHD 220 = 6:1

MP3 256kBit = 5.86:1

Good question. I actually also wonder if there's a real quality difference between RAW 3:1 and Prores 444, other than the obvious aspects like Color Temperature and ISO not being burned in to the RAW file.
Cheers

Eric, difficult to answer your question at this time. On.y the beta testers have cameras and they can't release any variants of raw files yet.


Sent from my iPad using Tapatalk

Uncompressed 12bit RAW 4608 x 2592 @ 24fps = 429.6 MB/s
Lossless compressed ~ 330.5 MB/s (compressibility depends on footage content)
Lossy 3:1 compressed = 143.2 MB/s

Debayered RGB16bit uncompressed = 1.72 GB/s
Debayered RGB8bit uncompressed = 860 MB/s

BluRay = 38.7 MB/s

So 3:1 sounds like a lot but isn't.
We don't know the compression technology behind BMDs lossy 3:1 but most likely it is similar to REDcode.
That means it will use a mild wavelet compression which should work a little bit like a noise reduction filter when the image is restored from its compressed state. I would say at 3:1 it will be very hard to see any artifacts at all, especially if you have no originally uncompressed sample to compare.
At 8:1 you should see some smearing of details in fine detailed textures. That was the ratio of the first RED ONE and it shot very nice images.

As Robert so well explained, 3:1 is considered very mild compression in the scheme of things...

As well as the ratio you should know that the compression SCHEME is also as important as the ratio. It's tricky to compare a wavelet based 3:1 (say Redcode) to a DCT based one (like cDNG or ProRes)

In the real world, I mostly use 3:1 because I get the advantages of RAW and 16 bit linear files at a data rate that isn't that far from ProRes 4444.....

Unless the show is already only shooting ProRes.....

JB

We don't know the compression technology behind BMDs lossy 3:1 but most likely it is similar to REDcode.

I believe it's DCT. Like ProRes.

JB

Thanks guys!

A distinction between information and data is necessary. Data represents information. What lossless compression does is reorganize the representation of information and cram ALL the original information into less data (hence, lossless compression).

With 3:1 compression you don't really discard 2/3 of the information, because there are lossless entropy coding stages involved in the process. You reduce entropy in the lossy stage, and then do lossless passes on the result. If you are looking for numbers, with BM cameras discarding 1/2 is more like it, although this is a simplification. And with other cameras you discard even less to get to 3:1, since BM images have way more entropy than pretty much any other camera.

We don't know the compression technology behind BMDs lossy 3:1 but most likely it is similar to REDcode.

I believe it's DCT. Like ProRes.

JB

It is DCT, but it is not what one would call a "proper" DCT. There are significant differences compared to a typical DCT implementation. BM cameras essentially use it to exercise constant rate control. It is a peculiar way of utilizing it.

Thanks Mihail for some more in depth information. I was going to contact you (as a user of slimRAW) about this but you were faster than me

Maybe this helps too:

Aren't there CinemaDNG re-compression applications out now? Would it be possible to do a comparison of 3:1 vs. lossless?

[Edit] Found it! SlimRaw.

I thought I'd update this thread after owning the 4.6k for a moment and say this: Depending on what you're shooting and how you plan to treat your footage in post, there's a clear difference between lossless and 3:1 (shocking). It's 'visually' equal when graded in a very straight-forward way. But when stretched(!), the grain characteristics change significantly. This is where I've noticed a difference to my eye. I guess I was originally curious where the compression would show up, and as Tom Majerski mentioned in an another thread, RAW compression can be witnessed in the shift of grain characteristics. The lossless grain from this camera looks amazing!!! I need to buy more hard drives... Anyway, it's been a blast testing this thing

Question for you Eric (as my camera is still yet to be shipped), you say the footage feels different between RAW and 3:1, how would you compare both to ProRes 422 and 444?

Hey Sebastian,

From a bitrate perspective Prores 422 is 147Mbps VS Prores 444 which is 330Mbps. A considerable difference. I personally don't have any experience shooting prores (I don't need to nail my WB and prefer the flexibility in post, which is acceptable for me but unacceptable for many productions). You're definitely better off shooting the highest bitrate possible depending on your shoot and post workflow. Hopefully that helps!

Good question. I actually also wonder if there's a real quality difference between RAW 3:1 and Prores 444, other than the obvious aspects like Color Temperature and ISO not being burned in to the RAW file.
Cheers

Prores 444 is about the same data rate as lossless raw so why bother.

Prores 444 is about the same data rate as lossless raw so why bother.

My computer has a hell of a hard time decoding RAW (6fps) and adding nodes to it (2fps), but is fine with ProRes of any flavours and is happy with nodes added.

ProRes 444 is about the same rate as raw 4:1.


Sent from my iPad using Tapatalk

I was more wondering on visual IQ rather than bit rates.
If 4:1 is better visually than PR444, then I'm obviously going to shoot 4:1.

I might have to wait till I get my camera to do a comprehensive test...

ProRes 444 is about the same rate as raw 4:1.

You're right, I was thinking of PR 444 XQ which is 250 MB/s compared to 165MB/s for PR 444.

Also, I think, calling the DCT compressed cdng (3:1, 4:1, etc) 'raw' is a bit misleading. The term compressed bayer format seems more fitting.

Also, I just realised that the specs say you can't use prores if you want to record at the full 4.6k resolution. Makes comparisons more difficult.

I hear you! There is no raw in terms of directly reading sensor data as that would be two 11bit streams from the dual gain sensors that BMD uses. So the 22bits becomes 16bits linear becomes 12bits log, but we call it raw anyway just to say it refers to the luminance of a photosite versus all other codecs that refer directly or indirectly to the interpreted colour of a pixel.

In this forum, we have gone from the BMCC with its uncompressed raw to the URSA with losslessly compressed raw (mathematically compressed but unpacks to uncompressed raw matching the original) to raw 3:1 and raw 4:1 visually lossless (lossy compressed raw that may be difficult to discern the loss of detail but does a reasonably good job retaining colour information).

We just need to pick our poison as we balance the need for fidelity with the real world constraints of the ability of media to record what we want and the limitations of budget and storage capacity to work with our data.

I'm sure we would all love to shoot either lossless raw or ProRes 444 XQ Film, but for most acquisition cost and post budgets are the limiting factors. Especially feature documentary shooters with 100:1 shooting ratios at one extreme versus a 15 second beauty commercial with VFX at the other. The documentarian will likely shoot ProRes LT HD if they were using BMD cameras at all. The commercial shoot could be either lossless raw open gate or ProRes 444 XQ UHD if their media can record it.

Test for the quality required but do the math for what fits the budget. Applies to professionals and enthusiasts alike.


Sent from my iPad using Tapatalk

Also, I just realised that the specs say you can't use prores if you want to record at the full 4.6k resolution. Makes comparisons more difficult.

You are correct ProRes maxes out on the camera at 3840x2160 but it is a downscale of the full sensor. At the end of the day the full 4.6k does not resolve the 4.6k resolution but is there to debayer down to a full UHD or DCI. If you need DCI your gonna have to shoot one of the RAW flavors. If you need UHD I really think ProRes is not really a compromise at all. IMO