BRAW is actually Macro-Blocking RAW

The place for questions about shooting with Blackmagic Cameras.
  • Author
  • Message
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostThu Sep 26, 2019 11:58 am

Can you copy that and clip 1/4 second in a camera and post some illustrative frames? Maybe you still have access to a braw camera?
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

John Paines

  • Posts: 5830
  • Joined: Tue Jul 28, 2015 4:04 pm

Re: BRAW is actually Macro-Blocking RAW

PostThu Sep 26, 2019 12:58 pm

levisdavis wrote:The BRAW file is 6GB. ( Don't have space to post the file; unless someone might be able to assist. )


Put it on a timeline, trim the clip to a section where it occurs -- 5 seconds? 10 seconds? -- and use the Media Manager to copy the clip. The output file will be that segment of braw only, unrecompressed, with a .braw extension.

You can also export a single braw frame from the raw tab of the Color page.
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostThu Sep 26, 2019 3:11 pm

Hi guys, I've just posted a thread asking about functional specification design tools that output code and circuit designs, and a faster alternative to FPGA.

Any engineers here who might be able to help?

viewtopic.php?f=14&t=99599
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

levisdavis

  • Posts: 170
  • Joined: Mon Apr 22, 2013 11:43 pm
  • Location: Phoenix, AZ

Re: BRAW is actually Macro-Blocking RAW

PostThu Sep 26, 2019 11:59 pm

https://fil.email/2JMHwsbQ

Download includes...
1. Original file.
2. Sidecar file.
3. DR 16.0 Project file for a means to look at the grade; if you want to see it exactly from my point of view.

Active for 7 days.
Levi Davis
Professional Imagery. Simple.
azcamera.biz
Offline
User avatar

Jack Fairley

  • Posts: 1863
  • Joined: Mon Oct 03, 2016 7:58 pm
  • Location: Los Angeles

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 1:43 am

It looks horrible when I import your project, but the issue isn't with the media itself, which looks fine in the BRAW player or in a blank project.
Ryzen 5800X3D
32GB DDR4-3600
RTX 3090
DeckLink 4K Extreme 12G
Resolve Studio 17.4.1
Windows 11 Pro 21H2
Offline

levisdavis

  • Posts: 170
  • Joined: Mon Apr 22, 2013 11:43 pm
  • Location: Phoenix, AZ

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 4:49 am

Mind sharing a picture. Looks "horrible" is non-productive.
Levi Davis
Professional Imagery. Simple.
azcamera.biz
Offline

levisdavis

  • Posts: 170
  • Joined: Mon Apr 22, 2013 11:43 pm
  • Location: Phoenix, AZ

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 4:50 am

Meanwhile, the B-RAW looks horrible next to a GH5.
Levi Davis
Professional Imagery. Simple.
azcamera.biz
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 4:59 am

So used RCM to convert to rec709, adjusted exposure, made sure optimization was turned off, and made timeline resolution the same as the file. THen I added an exaggerated S curve.

There is what looks like a layer of film that is in fixed position and it does not look like macroblocking to me. It is really only noticeable to me when scrubbing quickly through the timeline. Is this what you are talking about?

Edit
Used splitter. The artifact or whatever it is appears to be in the green channel. Not an expert but this is the only defect I am seeing personally. I haven't seen this in any of my footage but also haven't looked for it like I did here. Also, it is only apparent when pushing the values hard. I cannot see the artifacts in a standard rec709 conversion at all.

Another Edit...
At ISO400, native ISO, the values are all smashed at the top of the waveform monitor with clipping in the clouds. This is probably why, at Q5, clawing back the values is causing artifacts.

Did you try a second take exposing around the middle instead of riding the highlights like this? If not, that is unfortunate, because it could likely have provided a clue as to why these artifacts are appearing.

I have used the GH series including the GH5 and in my experience if you smashed the values of your shot all at the top with a GH5, it would not be so easily manipulated as this footage is here.
Last edited by Dune00z on Fri Sep 27, 2019 5:22 am, edited 1 time in total.
Offline

levisdavis

  • Posts: 170
  • Joined: Mon Apr 22, 2013 11:43 pm
  • Location: Phoenix, AZ

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 5:07 am

Correct. Scrubbing is an excellent method to illustrate B-RAW recording method!

Can easily grade GH5 10-bit 4:2:2 150mb/s 2 - 3 stops further without any issue of "spatial noise reduction" / "macro blocking" / "stagnate noise-floor" that becomes a textured layer unto itself.

This is something being baked into the image. Doesn't matter if it's Q0 or Q5. Just hoping to bring about an awareness.
Levi Davis
Professional Imagery. Simple.
azcamera.biz
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 5:25 am

During scrubbing and playback BRAW usually switches to lower quality/lower resolution mode. So it is not a best way to inspect footage at pixel level.
I also checked that levisdavis's sample and processed file with different workflows and different timeline settings, and i can see those macro blocks over blue sky if i increase contrast and saturation.
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 5:43 am

Dmitry Shijan wrote:During scrubbing and playback BRAW usually switches to lower quality/lower resolution mode. So it is not a best way to inspect footage at pixel level.
I also checked that levisdavis's sample and processed file with different workflows and different timeline settings, and i can see those macro blocks over blue sky if i increase contrast and saturation.


I turned off optimization and kept debayering at full res so it should not be if the tools work as they are supposed to when you turn this off? Regardless the same issue can be seen in a render with the wacky grade and reflects exactly what I see when scrubbing. Green channel shows the artifacts clearly which reflects exactly what is seen when scrubbing.

I did several different kinds of grades and it only appears when doing something wacky.

I set project to RCM rec2020 g2.4 timeline colorspace to rec709 g2.4 output, set ISO to 400, pulled exposure back FOUR stops, even checked on highlight recovery for the clipped highlights, and added simple contrast and saturation and there is no apparent artifacts (aside from normal noise) of any kind that I can see, which is kind of shocking really in my opinion considering how exposed to the right the shot is.
Last edited by Dune00z on Fri Sep 27, 2019 5:44 am, edited 1 time in total.
Offline
User avatar

Uli Plank

  • Posts: 21809
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 5:43 am

The Lum Vs Sat curve on node 4 looks pretty extreme to me…
Now that the cat #19 is out of the bag, test it as much as you can and use the subforum.

Studio 18.6.6, MacOS 13.6.6, 2017 iMac, 32 GB, Radeon Pro 580
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 5:56 am

Yes, so Braw is not as high-speed.

Levi. Saying the comment about the gh5, is a bit inflammatory.

The original macro blocking effect posted was to do with a white dress footage, I believe might have been a BM sample clip. So, it might be to do with handling extreme exposure, which is something of concern to me if I want to recover as many stops as possible past the official stops. However, a new micro might possible have a newer generation of sensor with better exposure range.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 6:52 am

And one of these might be what the original macro blocking thread was talking about:

https://downloads.blackmagicdesign.com/ ... ancing.zip
https://downloads.blackmagicdesign.com/ ... rtrait.zip


Is it me, or are they trying to expose these 6k/4k comparison images differently to make the 6k look better?

Image

Image

I notice in a lot of samples, the 6k looks a little better.

But in the Wyoming 4k clip in the gallery, the person doing it knows his stuff. He gets latitude out of it, compared to some of the original clips. The Life from the ocean 4k clip is pretty good to from r for doco.

https://www.blackmagicdesign.com/au/pro ... ra/gallery

(Frankly, I liked Ricks version of models at night better too).
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 11:13 am

I graded those wedding shots and I did not see whatever problem people are talking about but I don't intentionally break every shot I grade looking to generate artifacts so at the time I may just have missed the boat. There is noise that cleans up with a normal nr pass. The pocket 4k shots of models are noisy at iso1000 and are shot in prores, not braw. The camera is noisy when underexposed but cleans up with a simple nr pass. I don't see this burger shot with downloadable files so it's anyone's guess aside from digging in a thread how they were done. The two images look very different and my guess this is due to either a filter it different lens on the top image or simply different grades. The pocket 6k and 4k should look nearly identical with the resolution edge to the 6k.


Wayne Steven wrote:And one of these might be what the original macro blocking thread was talking about:

https://downloads.blackmagicdesign.com/ ... ancing.zip
https://downloads.blackmagicdesign.com/ ... rtrait.zip


Is it me, or are they trying to expose these 6k/4k comparison images differently to make the 6k look better?

Image

Image

I notice in a lot of samples, the 6k looks a little better.

But in the Wyoming 4k clip in the gallery, the person doing it knows his stuff. He gets latitude out of it, compared to some of the original clips. The Life from the ocean 4k clip is pretty good to from r for doco.

https://www.blackmagicdesign.com/au/pro ... ra/gallery

(Frankly, I liked Ricks version of models at night better too).
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 12:34 pm

Thanks for that Duane.

That should have been graded the same, or presented an original basic braw/raw look. White people aren't normally that white in real life, or coke cans that orange looking. :) Changing the sensor chip or colour filter can drastically change colour response. So, I am interested if these are both Sony tech, and which versions of the tech, and same colour filter.

I spent a very long time looking for that sample thread, and a couple other threads that show some stuff, but couldn't find them. One of the points, is don't matter how much you try you shouldn't get those artifacts in a top mode.

I think my problem is that linked samples in the threads were taken down. Very frustrating.
Last edited by Wayne Steven on Fri Sep 27, 2019 2:30 pm, edited 1 time in total.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 1:26 pm

I worked in vfx long enough to know that any codec regardless of whether it's raw or not can be broken. Highly compressed raw will look great if used right, bad if broken intentionally, just like any codec I've ever used.
Offline

Bunk Timmer

  • Posts: 182
  • Joined: Sat Oct 15, 2016 8:14 pm

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 1:56 pm

levisdavis wrote:Correct. Scrubbing is an excellent method to illustrate B-RAW recording method!
I am looking at the footage on a Apple cinema HD display from 2002 and I wonder if the reason I can see problem you describe is that I have a piss poor old screen that is not able to handle the 16 bit footage that is thrown at it ...and in a attempt to display it nonetheless dithers the footage at those places that are beyond the reach of the screen ...as it looks an awful lot like dithering.

Makes you wonder if the 10 bit of the GH5 is really 10 bit, doesn't it?
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostFri Sep 27, 2019 2:25 pm

Dune00z wrote:I worked in vfx long enough to know that any codec regardless of whether it's raw or not can be broken. Highly compressed raw will look great if used right, bad if broken intentionally, just like any codec I've ever used.


I'm saying a top mode should be lossless, and never break. Forget all the lossy and visual lossless rubbish. So bayer interpolation then becomes the main weak point. Not talking about maxing out the lossless image though, that is a limitation of the data.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

levisdavis

  • Posts: 170
  • Joined: Mon Apr 22, 2013 11:43 pm
  • Location: Phoenix, AZ

Re: BRAW is actually Macro-Blocking RAW

PostSat Sep 28, 2019 4:57 pm

... Wondering why a sky with a clean piece of glass is breaking a codec. Is it a latitude issue given that the camera is native ISO 100 - 1000? Does ISO 100 and ISO 1000 naturally produce extra stress on the codec?

The clip was recorded on the BMPCC4K at ISO 100 in BRAW Contstant Quality Q5. Graded as an ISO 100 clip in Resolve. I've experienced the same codec stress in other Constant Quality and Constant Bitrate... again recording with ISO 100.

In terms of stressing the codec in this clip, there is 2-stops of additional contrast being applied to the image. This is a luma-curve adjustment. Perhaps the stress persists in that the luma-curve is placed upon an ISO 100 value? Perhaps recording this scene with an in-camera ISO value at 500 would eliminate the stress?

Respectfully, the camera that I've worked with for the past 2 years handles 4-stops of exposure adjustment: shadows/midtone/highlight. Exposing-to-the-right, results in a grade with strong cloud/sky contrast. Additionally, a 10-minute scene can easily be speed-ramped to 60-seconds without any form of Macro-Blocking / Spatial-Noise Reduction / Black Shade Compensation creating a layer of texture unto itself.

That said, is there any way to narrow in the results of what BRAW is recording to eliminate the stress? If only 9-stops of latitude is needed, why not make the 9-stops the best 9-stops possible? If 13-stops are spread thin, than every stop there-under is also thin. That's sort of why I'm wondering about the ISO 500 value. I'm looking to grade in contrast in the midtones. Should I therefore be exposing my shot for a mid-ISO value so that way there is more room for grading mid-tones? ... Creating a thought process for shooting for the grade rather than shooting for an ISO value... "Want to grade for highlights, choose ISO 100. Want to grade for shadows, choose ISO 1000."
Levi Davis
Professional Imagery. Simple.
azcamera.biz
Offline
User avatar

Robert Niessner

  • Posts: 5026
  • Joined: Thu Feb 21, 2013 9:51 am
  • Location: Graz, Austria

Re: BRAW is actually Macro-Blocking RAW

PostSat Sep 28, 2019 5:31 pm

What we are seeing here in the sky is definitely not macroblocking, because that looks totally different, like smoothed out 16x16 blocks.

You can see how that would look like in those Sony FS5 sky samples:
https://www.eoshd.com/2016/01/sony-fs5- ... edges-bug/

I'd say what we are seeing here in the sky of the braw sample is the result of BMDs noise reduction algorithm.
Saying "Thx for help!" is not a crime.
--------------------------------
Robert Niessner
LAUFBILDkommission
Graz / Austria
--------------------------------
Blackmagic Camera Blog (German):
http://laufbildkommission.wordpress.com

Read the blog in English via Google Translate:
http://tinyurl.com/pjf6a3m
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostSat Sep 28, 2019 5:45 pm

You over exposed the image significantly including clipping highlights, which exceeds what would be considered exposing to the right even since highlights were clipped.

This is not how I would really recommend shooting in general but if this is what works for you normally then maybe don't do it with this camera.

ISO 100 curve would provide the least allotment of the DR to the highlights, and everything is shot as highlights, so it might be a better idea to expose for and grade in the native ISO instead?

Regardless

I set clip ISO to native ISO (400), pulled exposure back by 4 stops, and graded normally in RCM including heavy contrast adjustments and experienced zero artifacts neither in monitor while grading nor in renders. Whatever artifacts should be there via NR in the footage was invisible to me and as Robert and I said, it doesn't look like macroblocking anyways.
Offline
User avatar

Jamie LeJeune

  • Posts: 2027
  • Joined: Tue Feb 26, 2013 4:33 am
  • Location: San Francisco

Re: BRAW is actually Macro-Blocking RAW

PostSat Sep 28, 2019 6:06 pm

I've reached the same conclusion as Robert and Duane — Beyond having been shot massively overexposed, I can’t find anything inherently wrong with the file that fits the claims of macroblocking. I pushed the raw tab settings to an absolutely ludicrous degree (note the saturation pushed all the way to 2.00 in the screenshot below, which on any normal shot with people in it would make their faces look like Umpa Lumpas from Willy Wonka's factory!) and was unable to reveal any problem stemming from the codec.

Here’s a still of the raw tab settings used (no color nodes were applied, just raw tab settings):
Screen Shot 2019-09-28 at 10.39.46 AM.png
Screen Shot 2019-09-28 at 10.39.46 AM.png (60.41 KiB) Viewed 23909 times


Here's a screenshot just as a reference to see the extreme color and contrast resulting from those raw tab settings (and since it's a JPEG, it will of course have JPEG macroblocks baked into it, so it's not a reference BRAW performance):
Screen Shot 2019-09-28 at 10.51.55 AM.png
Screen Shot 2019-09-28 at 10.51.55 AM.png (998.72 KiB) Viewed 23909 times


To see the results in full quality, here’s a link to a ProRes4444 file rendered out from Resolve at the UHD source resolution of the file with only those raw settings applied (so no scaling was involved to muddy the waters):
https://drive.google.com/file/d/174LSwhUsklL1Vz5P-k4ASkmQvfGb9T9h/view?usp=sharing

I can’t find an ounce of macroblocking in the rendered shot even though it's been pushed way too far. The codec is just fine and not causing an issue in this shot.

My guess here is that OP applied a bad LUT or was wrenching on the curves, either of which can break any image no matter how well exposed and how perfect the codec.
It’s standard operator error. It doesn’t reveal any inherent problem with BRAW.

levisdavis wrote:"Want to grade for highlights, choose ISO 100. Want to grade for shadows, choose ISO 1000."
In fact, it's the reverse. ISO100 puts more stops below middle grey. ISO1000 puts more stops above middle grey. There's a dynamic chart in the camera manual that illustrates it. The digital cinema cameras from ARRI and Red have a similar relationship between ISO and middle grey.
In contrast, still cameras that also have a video mode — like the GH5 mentioned, or the Sony A7s or the Canon 5d models — do not work this way, so many users of those cameras who then move on to their first cinema camera are often confused by the difference and have to take some time to learn how to adjust for it.
www.cinedocs.com
http://www.imdb.com/name/nm4601572/
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostSat Sep 28, 2019 8:42 pm

Jamie is right, in order to see the artifacts I had to make an S Curve that would have broken anything... Wrenching is a good word for it. The curve I made was highly exaggerated and I kept pulling it until the artifacts could be seen. This was the only way I could really see the issue mentioned. In every other test grade I did playing with the file, the artifacts were invisible.
Offline

levisdavis

  • Posts: 170
  • Joined: Mon Apr 22, 2013 11:43 pm
  • Location: Phoenix, AZ

Re: BRAW is actually Macro-Blocking RAW

PostSat Sep 28, 2019 8:58 pm

Are we referring to artifacts or spatial noise reduction? ( The image is within the histogram when shot. )

Thank you for following up by the way! I'll download the file this evening!

I'm wrenching less with the BMPCC4K than the GH5, by the way as the codec is revealing spatial noise.
Levi Davis
Professional Imagery. Simple.
azcamera.biz
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostSun Sep 29, 2019 12:44 am

I see no issues with the codec at all unless intentionally breaking the it with curves I and no one else would ever use in a grade.

You clipped the highlights so there is no way you metered "correctly" for exposure as in keeping everything within range of the sensor. I recommend using false color instead of the histogram for exposure.

The gh5 codec that you hold so highly also breaks when you do so intentionally as you have done with the pocket 4k braw. Record with WB set to 3200k in daylight with both cameras then bring WB back to where it's supposed to be in post. Gh5 will look like crap and the braw footage won't.

As we have said, perfectly clean and artifact free renders are possible while doing exactly what you want with contrast when you do so in a different way. I pulled the footage back 4 stops and greatly interested contrast as well as saturation with zero artifacts.

To me this is all user error on your part and less to do with a defect in the codec.

levisdavis wrote:Are we referring to artifacts or spatial noise reduction? ( The image is within the histogram when shot. )

Thank you for following up by the way! I'll download the file this evening!

I'm wrenching less with the BMPCC4K than the GH5, by the way as the codec is revealing spatial noise.
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostSun Sep 29, 2019 2:10 am

"Less to do" or none to do Duane?
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

levisdavis

  • Posts: 170
  • Joined: Mon Apr 22, 2013 11:43 pm
  • Location: Phoenix, AZ

Re: BRAW is actually Macro-Blocking RAW

PostSun Sep 29, 2019 5:16 am

Downloaded the ProRes 444... Thanks for sharing!

If you have the time, take a look at the same B-RAW recording, 13:30:25:13 and up till 13:33:37:11, notice the lower right hand side / right side of the frame. There is a layer of spatial noise reduction that creates texture on top of the image. You might Scrub in Resolve or you might render and Scrub a ProRes 444. The spatial noise texture will remain in the output file; the texture is prominent when scrubbing or speeding up the clip, 400 percent for example.

( Update: I shot a few 1-minute clips this evening on a V1 BMPCC4K with the latest firmware: Q0, Q5, and ProRes 422. These recording were captured with a SanDisk Extreme Pro CFAST 2 256GB card. I'm looking over the files now. )
Attachments
Spatial Noise_Stop.JPG
Spatial Noise_Stop.JPG (154.17 KiB) Viewed 23796 times
Spatial Noise_Start.JPG
Spatial Noise_Start.JPG (162.96 KiB) Viewed 23796 times
Levi Davis
Professional Imagery. Simple.
azcamera.biz
Offline

levisdavis

  • Posts: 170
  • Joined: Mon Apr 22, 2013 11:43 pm
  • Location: Phoenix, AZ

Re: BRAW is actually Macro-Blocking RAW

PostSun Sep 29, 2019 5:35 am

Looked over the three recorded files from this evening that were captured with a second BMPCC4K body... I've said my peace with this post.

... Let's move to other things like creativity while we still can! Thanks for taking the time to look over the notes and for being a part of the community. It's an honor!
Levi Davis
Professional Imagery. Simple.
azcamera.biz
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostSun Sep 29, 2019 10:46 am

Wayne Steven wrote:"Less to do" or none to do Duane?


I can't say none because whatever is there is there. But grading the shot in the way done here makes little sense and serves the purpose to expose an issue that wouldn't be visible in any shot that's exposed and graded properly... This not really an issue IMO but I suppose it could be if you shoot and grade as the op does.

ISO 100 for example is not the proper curve for the shot given here and the grade is also nonsensical. Over exposing highlights intentionally at ISO 100 and crushing all the values of the shot at the top just to pull it all down with curves in post doesn't make sense... It's 4 stops over exposed with clipped highlights. The first sensible approach would have been to pull back exposure during shooting and the second to use native ISO or a higher ISO to allocate more DR to the highlights in the ISO curve. When I switched to ISO 400 and exposure to -4 I could grade easily with no issues visible.

The original claim is that the codec macroblocks, which it doesn't.

Whether or not spatial nr applied is an issue in general with the codec is inconclusive from this thread and the footage given. I can say anecdotally that spatial nr has never once been an issue for me and I've been shooting the camera since release, mostly in 8:1, but I also don't create the conditions that are shown in this thread (over exposing by 4 stops, making drastic curves, pumping a lot of saturation, speeding up clouds by 400 percent).

This isn't a dig at the op, I'm simply being realistic here. Shooting and grading in the way done in the ops shot just doesn't make a lot of sense to me and even with it I got clean renders when making my own grades in post. Whatever issues are with the codec don't seen to be a problem in my view, whatever issues that are there.
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostSun Sep 29, 2019 2:42 pm

Thanks Duane.

When I first heard about it, I thought it would suite portraying the distorted vision of a sick man (maybe dying of thirst) looking into bright clouds.

I'm glad that's over. But I've seen people discussing looking into footage of a wedding and there were lots of squares ( :) ).

The quicker every Braw issue disappears the quicker we can forget DNG. I wish they would announce the refinements they have done to it, as there is a pot of old noe dated discussion about it, some of which might be fixed.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostSun Sep 29, 2019 7:02 pm

I just don't think you can tell anything by someone else's render or grade unless you know exactly how it's done. I've had macroblocking and banding in renders from shots that were perfectly graded with no issues whatsoever. The finishing codec and then of course streaming compression makes things look bad and all of a sudden there's a problem with the camera or grade... When it's just inherent in the way video is delivered most of the time. I've had perfectly clean footage get compressed to death die to streaming compression and noisy footage smoothed out by the same compression.

Even the most expensive productions on the best cameras with excellent grades get "ruined" when it's delivered because the delivery codec can't handle it's greatness.




Wayne Steven wrote:Thanks Duane.

When I first heard about it, I thought it would suite portraying the distorted vision of a sick man (maybe dying of thirst) looking into bright clouds.

I'm glad that's over. But I've seen people discussing looking into footage of a wedding and there were lots of squares ( :) ).

The quicker every Braw issue disappears the quicker we can forget DNG. I wish they would announce the refinements they have done to it, as there is a pot of old noe dated discussion about it, some of which might be fixed.
Last edited by Dune00z on Mon Sep 30, 2019 12:10 am, edited 1 time in total.
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostSun Sep 29, 2019 8:28 pm

You were talking about renders being ruined with macro blocking during render initially or after?

I can say that sure, pushing around low accuracy footage without enough real levels, is going produce banding, particularly if you are sacrificing a number of stops to get clean low end footage, but everything else is virtually a codec/recording issue you have to handle. CDNG had issues, but recorded at lossless what you should see is how the Bayer sensor without olpf really looks, what happens prior braw, real stuff that has to be handled, which is a recording issue from acquisition. Braw apparently does not get it completely right either.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 12:17 am

Sorry I'm not great at typing with my phone and I miss some stuff with Swype. I am talking about how an 8 bit codec will cause banding or blocking regardless of how good the footage is captured and graded, and 8 bit is the majority of finishing codecs that get uploaded. Then of course, streaming compression does it's own share of destruction.

You can upload 10 bit however still the streaming compression can kill it anyways.




Wayne Steven wrote:You were talking about renders being ruined with macro blocking during render initially or after?

I can say that sure, pushing around low accuracy footage without enough real levels, is going produce banding, particularly if you are sacrificing a number of stops to get clean low end footage, but everything else is virtually a codec/recording issue you have to handle. CDNG had issues, but recorded at lossless what you should see is how the Bayer sensor without olpf really looks, what happens prior braw, real stuff that has to be handled, which is a recording issue from acquisition. Braw apparently does not get it completely right either.
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 3:53 am

OK, I thought you meant on the ingress end. The interesting thing is that I wonder if YouTube tries to avoid that? But, I read somebody recently saying 4k on YouTube is so much better quality than fullhd, that he upscales his fullhd footage to 4k then submits it to youtube (which likely is better again, as the upscaled version's extra data probably doesn't impact compression as much, leaving more data rate to the original detail). Handy idea.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 5:41 am

Back to provided sample, blocking pattern is due extreme overexposed shoot. As you can see at native ISO400 it use only 1/3 of total tonal range and blue channel is even less. To bring it back to normal point it \needs lower expose at few stops and add crazy Contrast value of 1.5. As a result not enough bits stress a codec a lot and probably combined with noise reduction produces blocky artifacts.
It is really nice to see samples like this to understand the usable real life limits.

The main lesson - you should never expose and shoot things in the way like this with digital sensors.

Image
Image
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline

John Griffin

  • Posts: 1340
  • Joined: Sun Aug 23, 2015 4:18 pm

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 7:13 am

Dmitry Shijan wrote:Back to provided sample, blocking pattern is due extreme overexposed shoot. As you can see at native ISO400 it use only 1/3 of total tonal range and blue channel is even less. To bring it back to normal point it \needs lower expose at few stops and add crazy Contrast value of 1.5. As a result not enough bits stress a codec a lot and probably combined with noise reduction produces blocky artifacts.
It is really nice to see samples like this to understand the usable real life limits.

The main lesson - you should never expose and shoot things in the way like this with digital sensors.

Image
Image

Wouldn't be a problem with a true 'RAW' codec......
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 8:30 am

John Griffin wrote:Wouldn't be a problem with a true 'RAW' codec......


To know it for sure we need exact same side by side test with compressed DNG.
Probably it all about bit depth. It wouldn't be a problem with true 16 or 32 bit image source. But 12 bit source have its own limits.
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 8:47 am

I thought that braw was packaged in 12bit log which unpacks to 16bit like arri does?

Regardless I've seen raw artifacting in a number if different raw when overexposed this far and stretched like this. It may be that braw has less room at Q5 (highly compressed) than 3:1 or uncompressed raw but this would likely be a limitation based on compression and not braw as a whole. Redcode falls apart at +4 overexposure with less compression. Doesn't make it bad, you just don't shoot it that way.

If your idea is to shoot and grade in this manner normally and expect perfect results then clearly braw q5 is not for this purpose. This is the only thing that can be understood from this IMO without any further tests at different compressions, exposures, and codecs to compare.

Dmitry Shijan wrote:
John Griffin wrote:Wouldn't be a problem with a true 'RAW' codec......


To know it for sure we need exact same side by side test with compressed DNG.
Probably it all about bit depth. It wouldn't be a problem with true 16 or 32 bit image source. But 12 bit source have its own limits.
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 8:52 am

Hmm, mm.

Yes, bit depth, processing and compression!
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 9:31 am

Dune00z wrote:I thought that braw was packaged in 12bit log which unpacks to 16bit like arri does?


It is still only 12 bit source data captured from sensor, even if you unpack (upscale) it to 16bit and processed in Resolve with 32 bit precision.

8-bit can represent 256 shades of grey.
12-bit can represent 4096 shades of grey.
16-bit can represent 65536 shades of grey.

Also it may depend of sensor technology. Older BM cameras with Fairchild Imaging sensors dual gain technology capture 2 x 11-bit data and combine it to single 12 bit output. This helps to extend dynamic range, makes noise structure less "digital" and also may help to provide more additional depth to color in extreme near clipped values.
Sony sensors in Pocket4K and 6K works in more traditional mode, and have only hard switch to capture in high gain mode or low gain mode.
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline

John Griffin

  • Posts: 1340
  • Joined: Sun Aug 23, 2015 4:18 pm

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 11:42 am

Dmitry Shijan wrote:
John Griffin wrote:Wouldn't be a problem with a true 'RAW' codec......


To know it for sure we need exact same side by side test with compressed DNG.
Probably it all about bit depth. It wouldn't be a problem with true 16 or 32 bit image source. But 12 bit source have its own limits.

Is DNG a true RAW codec though?
Offline

Dune00z

  • Posts: 356
  • Joined: Thu Oct 18, 2018 3:40 pm
  • Real Name: Duane Eues

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 12:44 pm

I understand what bit depth means.

What I'm saying is that IF bmd is doing the same 12 bit log encoding that Arri does, and this has been said by the community, it should be 16 bit linear packed as 12 bit log and unpacked as 16 bit with supposedly no difference in practical use between it and recorded 16 bit linear, it's just smaller data files.

If bmd's 12 bit log encoding doesn't do this then it isn't the same as Arri's at all and would explain why some people are disappointed when pulling values to an extreme degree.

Considering how I don't think that the sensor is 16 bit in the pocket cameras, that would mean there is no way it does what Arri does with 12 bit log encoding at all, and you shouldn't be trying to stretch values all crazy in post.

Dmitry Shijan wrote:
Dune00z wrote:I thought that braw was packaged in 12bit log which unpacks to 16bit like arri does?


It is still only 12 bit source data captured from sensor, even if you unpack (upscale) it to 16bit and processed in Resolve with 32 bit precision.

8-bit can represent 256 shades of grey.
12-bit can represent 4096 shades of grey.
16-bit can represent 65536 shades of grey.

Also it may depend of sensor technology. Older BM cameras with Fairchild Imaging sensors dual gain technology capture 2 x 11-bit data and combine it to single 12 bit output. This helps to extend dynamic range, makes noise structure less "digital" and also may help to provide more additional depth to color in extreme near clipped values.
Sony sensors in Pocket4K and 6K works in more traditional mode, and have only hard switch to capture in high gain mode or low gain mode.
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 1:15 pm

Arri use sensor designed and manufactured by ON Semiconductor, it can capture 2x14 bit data. https://www.arri.com/en/learn-help/tech ... ev-sensors
Fairchild Imaging is different company (but formally a part of ON Semiconductor https://eu.mouser.com/manufacturer/fair ... conductor/ ) their sensors can capture 2x11 bit. https://www.fairchildimaging.com/products/scmos-sensors
Those are two dIfferent sensors produced by different companies but dual gain capture concept is similar between them.
Higher bit depth needs more processing power, more cooling and so larger battery.
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline
User avatar

Jamie LeJeune

  • Posts: 2027
  • Joined: Tue Feb 26, 2013 4:33 am
  • Location: San Francisco

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 5:32 pm

Dmitry Shijan wrote:The main lesson - you should never expose and shoot things in the way like this with digital sensors.
100% agree.

Despite the extreme overexposure, I was able to yield an artifact free result, even with an extreme contrast pull, by sticking to simple adjustments in the raw tab. See the post above where I included a screenshot of the settings and a link to the rendered result in UHD ProRes4444.
I'd say BRAW is an impressive codec that it can manage to not break under those conditions.
www.cinedocs.com
http://www.imdb.com/name/nm4601572/
Offline

John Brawley

  • Posts: 4308
  • Joined: Tue Aug 21, 2012 7:57 am
  • Location: Los Angeles California

Re: BRAW is actually Macro-Blocking RAW

PostMon Sep 30, 2019 8:51 pm

Dmitry Shijan wrote:
Dune00z wrote:I thought that braw was packaged in 12bit log which unpacks to 16bit like arri does?


It is still only 12 bit source data captured from sensor, even if you unpack (upscale) it to 16bit and processed in Resolve with 32 bit precision.



Depends but certainly on the dual gain cameras, they are greater than 16 bit before the hit the file encoding part. 16 bit lin with a straight mathematical transform to 12 bit Log. The sensor itself of course is different, and the way they are summed is different. This was also the same in DNG, so I don't think you can say it's somehow different with BRAW.

I think you can't get something for nothing. It's compressed. You can't expect it to not have some limitations. There are other issues with DNG that people seem to have forgotten as well.

For example, this is a similar shooting situation with blue skies in DNG.

viewtopic.php?f=2&t=69405&p=388194

DNG compression artefacts.

JB
John Brawley ACS
Cinematographer
Currently - Los Angeles
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: BRAW is actually Macro-Blocking RAW

PostTue Oct 01, 2019 12:53 am

I thought braw was 100% not log, as the consumer level of Sony websites were often 12 bits or less (sometimes 14 bits). It's one of the hidden 8sdyes I have with the pocket. If you did have 16 bit into 12 bit log, that 12 bits still has levels with some large gaps in between levels, rather then 16 bit accuracy, that may stuff things around a little when pushed (but, yes, log is great anyway). I only wish the pockets did that.

Cdng should be much less compressed at the given datarate, which if at a lossless, or near lossless rate, represents a truer representation of how the sensor sees the image to work with, which Braw doesn't have. It's that part "A" grade is missing. Then you can sort out the worts the sensor actually sees to the best you can, it is not forgotten. Braw itself, simply needs a mode of good temporal noise removal user option, lossless compression, no spatial noise and detail reducing effects. Just one mode, which Zraw seems to be trying to do with the noise reduction. Using jpeg xs, or free cineform raw for it, will decrease file sizes. As it is likely such a mode is going be less than 2:1, given the non bayer overhead, but good temporal might put it above 2:1. Doing this with a real Bayer codec is going be likely 3:1-4:1 with good temporal noise removal. Thanks, Bayer inventor in the 1970's.
Last edited by Wayne Steven on Tue Oct 01, 2019 1:49 am, edited 2 times in total.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: BRAW is actually Macro-Blocking RAW

PostTue Oct 01, 2019 12:55 am

John Brawley wrote:For example, this is a similar shooting situation with blue skies in DNG.

viewtopic.php?f=2&t=69405&p=388194
DNG compression artefacts.


Yes, seems like similar compression blocks on that overexposed 4.6K camera DNG sample. So it is a codec only related limitation.

Image
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline

visalapol

  • Posts: 39
  • Joined: Sun Sep 02, 2018 1:37 pm
  • Real Name: Visalapol Nutavej

Re: BRAW is actually Macro-Blocking RAW

PostTue Oct 01, 2019 3:11 am

My limit experience, I'm run my basic `Grab still` did not see micro block or any problem.
Attachments
2.jpg
2.jpg (439.59 KiB) Viewed 23424 times
Offline

Bunk Timmer

  • Posts: 182
  • Joined: Sat Oct 15, 2016 8:14 pm

Re: BRAW is actually Macro-Blocking RAW

PostTue Oct 01, 2019 11:36 am

Jamie LeJeune wrote:
Dmitry Shijan wrote:
The main lesson - you should never expose and shoot things in the way like this with digital sensors.

100% agree.
I disagree.
It’s RAW after all. The only thing you should take care of is to make sure nothing gets clipped. Beyond that you can do whatever you want in post. See attached pic.

BRAW_settings.jpg
BRAW_settings.jpg (161.97 KiB) Viewed 23384 times

First one is the setting from Jamie. The other three are complete different settings. Same result, no person will be able to tell the difference without scopes.
If there is a reason for the “macro-blocking” I expect it to be the BRAW compression and nothing else. The OP mentioned Q5, so apperently 5:1 isn’t good enough to go extreme on sky footage.

Yes you can go less extreme than the OP and you will see no artifacts what so ever …however I can go way more extreme with my BMCC 2.5K cDNG footage and no "artifacts" will show.
PreviousNext

Return to Cinematography

Who is online

Users browsing this forum: No registered users and 134 guests