Any Firmware Update to Hide FPN in BMMCC?

Micro Cinema Camera was released 5 years ago, and it is still unique and affordable tool. From early days it had problem with Fixed Pattern Noise in deep shadows. That problem looks strange because original BMPCC with same sensor never had FPN problem. There where some rumors that FPN was caused by different cooling system or due different sensor generation with less noise in shadows. But as i remember it was never confirmed or denied officially.
This tiny camera with dual gain sensor technology can see really deep into shadows and overall noise structure looks very film-like. Same time those ugly digital FPN lines limit half of camera potential. In some special situations FPN may be visible even without huge Exposure/ISO increase.
There where some workarounds to apply dark frame FPN mask but they add useless complexity to workflow and don't fully works in reality. When FPN removed from darker shadows, it became more visible in lighter parts of the image, and vice versa. Seems it is just impossible to make universal FPN mask remover Resolve.

After all this time i still have a huge request to developers - is it possible to release some firmware update that will 100% hide FPN in BMMCC? Some manual custom sensor calibration option, or black shading, or better FPN auto-calibration on startup, or some other tweak?
You fixed FPN in last firmware for URSA 4.6K, isn't it mean that you can easy control and fine tune FPN with firmware?

Here are some examples to illustrate camera abilities and problems:

As you may know, there is a huge difference when you apply Noise Reduction before or after Exposure/ISO adjustment. If you apply Noise Reduction after Exposure/ISO adjustment, final result will always look cleaner, but same time FPN became more visible because it is now less hidden under the noise structure.

RAW ISO 800 -> BMDFilm to Rec709


RAW ISO 800, Exposure +5 -> TNR -> BMDFilm to Rec709


RAW ISO 800 -> TNR -> BMDFilm to Linear -> Increase Gain -> Linear to BMDFilm -> BMDFilm to Rec709


Here is another example when FPN became too visible in some situations even if you don't apply Luma Noise Reduction:

Similar low light examples from Original Pocket Cinema Camera (probably more grain but no any visible FPN):

And another example with FPN fixed in Resolve. Looks nice, but it is complicated, not too reliable and not too universal method:
- Require large amount of manual adjustments
- Require manual isolating highlights of FPN reference frame.
- Too dependent of image brightness
- Hot pixels from FPN reference frame produce additional pixel dots artifacts and require additional manual isolating

So why don't make some better in-camera calibration option to get maximum usable data from sensor?

It would be great to get an update for the BMMCC, especially since it's still being sold by BMD, but it's possible that any firmware update would need to remove the compressed raw format (CDNG 3:1) due to the RED patent issue; they might even have to remove CDNG altogether like they did in the BMPCC 4K although that's less clear to me. If the camera doesn't have the processing power for BRAW, that would mean you'd only have ProRes. A lot to give up in return for hiding FPN.

That said, if they could bring BRAW to the BMMCC and fix the FPN problem they could keep the Micro for sale and there might be a small but viable market among those who prefer the look of that sensor and don't need 4K. But it seems likely there will be a new Micro camera from BMD at some point, maybe this year; it's the only format they haven't refreshed in the last few years.

Dmitry Shijan, i admire a lot your knowledge and your workflow.
i tried to remove fpn but in other way without big success with dark framing removing tech.

if i understand correctly from your node graph:

you mix picture with
same picture desaturated to have luma of FPN
move to linear space
manage gain to had right "mask" of FPN
invert to apply the opposite fpn effects
opacity to find the right effects to remove FPN

with layer you probably mix in screen or similar blending mode
move again from linear to log

but i miss what you do with selection do you isolate effects on shadow? on dark / light part of picture?

later when you mix again restart you common workflow (that i just start to learn in other posts).
let me know
i'm very interest in deep of that, i had to deal with ump and bmpc4k shooting often, and i'm not feared from haerd work.

It would be great to get an update for the BMMCC, especially since it's still being sold by BMD, but it's possible that any firmware update would need to remove the compressed raw format (CDNG 3:1) due to the RED patent issue; they might even have to remove CDNG altogether like they did in the BMPCC 4K although that's less clear to me. If the camera doesn't have the processing power for BRAW, that would mean you'd only have ProRes. A lot to give up in return for hiding FPN.

That said, if they could bring BRAW to the BMMCC and fix the FPN problem they could keep the Micro for sale and there might be a small but viable market among those who prefer the look of that sensor and don't need 4K. But it seems likely there will be a new Micro camera from BMD at some point, maybe this year; it's the only format they haven't refreshed in the last few years.

Are you sure BRAW will fix FPN? I guess it only fix Cross Hatching problem and produce less noise due different debayer method, but not fix FPN by itself.

Not sure if BRAW for BMMCC is a good idea at all. BRAW is OK for downsampled 4K or 6K sources, but it is too soft at 1:1 sensor size. No raw pixels sharpness control in BRAW means problems for people who shoot with or without OLPF filters. Small color space with red channel clipping, halo edges artifacts, no real RGB source.
Also i guess that RED patent is not related to HD only cameras. It always says that cinema camera should be "2K and higher resolutions"
For my opinion 3:1 compressed DNG for BMMCC is OK, and 1 BM per frame is small enough.

Also i guess that RED patent is not related to HD only cameras. It always says that cinema camera should be "2K and higher resolutions"

Ah, I didn't know that! That explains why BMD didn't have to issue a firmware update to remove CDNG 3:1 from the Micro.

Dmitry Shijan, i admire a lot your knowledge and your workflow.
i tried to remove fpn but in other way without big success with dark framing removing tech.

if i understand correctly from your node graph:

you mix picture with
same picture desaturated to have luma of FPN
move to linear space
manage gain to had right "mask" of FPN
invert to apply the opposite fpn effects
opacity to find the right effects to remove FPN

with layer you probably mix in screen or similar blending mode
move again from linear to log

but i miss what you do with selection do you isolate effects on shadow? on dark / light part of picture?

later when you mix again restart you common workflow (that i just start to learn in other posts).
let me know
i'm very interest in deep of that, i had to deal with ump and bmpc4k shooting often, and i'm not feared from haerd work.

Removing FPN maually by subtracted Dark Frame in Resolve is a real pain. It may look like a solution but in reality it will never give desired result, because when you remove FPN from one tonal range you usually add FPN and hot pixels artifacts patterns to another tonal range of the image. Isolation of FPN only area with qualifier also produce additional artifacts.
FPN should be removed somehow in-camera on RAW level.

It was a FPN workflow thread for older BM cameras, but it end with nothing viewtopic.php?f=2&t=19422

Some info about stacking technology.
https://petapixel.com/2013/05/29/a-look ... -blending/
https://patdavid.net/2013/05/noise-remo ... ian_6.html
https://blog.keepcoding.ch/?p=2324

Here is my quick workflow that anyway don't works for me. Maybe someone improve or suggest something in future:

1. Shoot Dark frame:
BMMCC FPN is not in pure black area, so shooting dark frame with closed cap will NOT work for it. You need to shoot underexposed defocused grey card. Use some full frame telephoto lens and some best iris setting that will not produce vignetting.
You need to shoot 5-10 different exposure examples from shadows area. And later select one that contain "better" FPN vs background ratio in your workflow.
UPDATE: Instead of externally lighted grey card you may use small area of large computer monitor screen. It produces very uniform surface. Set monitor brightness to 100% to avoid flickering. Open solid grey color object in graphic or text editor and control intensity with grey fill slider. Make sure that monitor surface have no dust, spots or any other defects.

2. Prepare Dark Frame(s):
DNG Sharpness setting of Dark frame should match Sharpness setting of future DNG video that you work with. So you may generate few variations of Dark Frame with different sharpness.
Desaturation in RAW and Desaturation in Secondary node produce slightly different FPN "look". Desaturation in RAW seems works better.
You may also experiment with increase Exposure of Dark Frame DNG source.
Convert RAW to TIF in Resolve and export in native BMDFilm color space and gamma.
You need near 500 frames and combine them into single frame by stacking with blend mode Mean (In many photography stacking articles Median blend mode is suggested, but for FPN Median produce less accurate result). You can do it in Photoshop or any other image editor that have an option for layers stacking.

Some people use 5 frames TNR in Resolve instead, but 5 averaged frames it is not enough data for clean FPN frame.

Some additional tips and quotes about Mean vs Median:
https://helpx.adobe.com/photoshop/using ... tacks.html

"I would like to see a comparison of median averaged vs. mean averaged noise in photos. Technically mean average should reduce noise by a factor of square root of # of averages, whereas median average is prone to artifacts (which can be a positive thing in the case of the moving object).

median doesn't completely eliminate noise at all, depending on the noise distribution it can be worse than the mean: mean(1.01 0.95 1.03 1.02) = 1.005, while median(1.01 0.95 1.03 1.02) = 1.015

Using mean is a little like a single long exposure and includes a bit of info from each of the photos in the stack. Using median removes the effect that any outliers play.

for example with the following value 10, 11, 12, 13, 14 100
if using median, the result would be 12.5 (rounded to 13) - the 100 simply 'falls off' the end.
if using mean, the result would be 26.6 (rounded to 27). The 100 is included in the 'averaged' and still creates a hotspot.

if the 100 were replaced with 256, again the median is 12.5 (13), but the mean is 52.7 (53) - an even more significant hotspot.

As the number of images in the stack increases, this outlier becomes less of an issue, but other things like (eg movement, or the sun moving) can creep in."

Here is how final Dark Frame looks with Exposure increased to +5


As you can see there are endless amount of options and variations. Not too convenient. Not too precise. Too many guesses. And we even not start to apply FPN fix in Resolve yet .

3. Next you need to apply Dark Frame In Resolve to fix FPN:
Right-click on different Dark Frames and Add to Media Pool as Matte.
Go to node graph, right-click on any node and Add Matte.
Connect Matte node with green colored in-out as image, instead of blue colored alpha channel in-out.
Right-click on Matte node and change its source to one of your imported Dark Frames. Select the one that better removes FPN (you need to compare and guess).
Add invert node.
Add Layer Mixer Node, set blending Mode to Overlay.
Fine tune Dark Frame Exposure with linear gain node (you need to compare and guess).
Isolate Highlights, so Dark Frame will not affect Highlights and will not add FPN lines to Highlights (you need to compare and guess again).

From some point of view blending nodes in linear gamma may looks better, but in reality it just arrange light in slightly different way and also produce some non uniformity.
Here is example without transforming nodes to linear gamma. And with additional Qualifier node (Spots selection) to remove hot white dots on Dark Frame (sort of isolated bright dots of Dark Frame subtract itself. Another variable guess option)

But i repeat again - this workflow is near useless in real life because it is too complicated and not precise. FPN calibration should be a part of camera firmware and should be done directly inside RAW data.

Thank Dmitry, I will try. I know, I hope I not need and I know that is not simple or clean, but you have reached me a lot in the last year about developing to enlarge dr and now a lot about tu to recover fpn. If Blackmagic Design develop a better way (like red) to remove it in hardware, I will be happy.
In the mean time I can experiment with your technique. Thanks


Inviato dal mio iPhone utilizzando Tapatalk

Here if also some FPN related info from Wiki:

Fixed-pattern noise https://en.wikipedia.org/wiki/Fixed-pattern_noise
Correlated double sampling https://en.wikipedia.org/wiki/Correlate ... e_sampling
Dark-frame subtraction https://en.wikipedia.org/wiki/Dark-frame_subtraction
Bias frame https://en.wikipedia.org/wiki/Bias_frame

I think this question require deep tech knowlege in sensor development. So we can only guess how those things actually works in camera...
Some users in earlier threads found that FPN in 4.6K cameras is very temperature dependent:
viewtopic.php?f=2&t=61221
viewtopic.php?t=55468

Also how about Fourier Transform Filtering (FFT) to Remove Fixed Pattern Noise in camera? Or maybe add this technology as a Plug-in in Resolve? This magical method was mentioned in 4.6K FPN discussion thread. I done some search and found some info in other forum discussions, but i can't fully understand how it works yet. Seems it is some plugin in ImageJ processing program.

https://www.dpreview.com/forums/thread/4110569
https://www.dpreview.com/forums/thread/3806560
https://photo.stackexchange.com/questio ... hoto-paper

Fourier Transform Filtering (FFT)
seems many interesting, but we must understand how it work on motion picture.
The strange things is that no one developer of live streaming tv thought to study how to remove this kind of problems, from old cameras to studio mixer, from old shooting to newer and more.

Seems there is a FFT3D VapourSynth filter in Hybrid, but not sure if it works as FPN remover. https://avisynth.org.ru/fft3dfilter/fft3dfilter.html
"FFT3DFilter uses Fast Fourier Transform method for image processing in frequency domain. It is based on some advanced mathematical algorithmes of optimal filtration. It works not locally, but makes some delocalized (block) processing. In 3D mode, it results in effect similar to partial motion compensation. This filter can reduce noise without visible quality loss and artefactes, even with quite strong settings. It can greatly improve compression and reduce encoded file size.
Also it has option of limited sharpening without both noise amplifying and oversharpening (haloing)."

If back to basic FPN setup, we can see same effect as in unlucky firmware update for Ursa with FPN lines drifted to highlights area.
And it was a main problem from early days of this workflow. We remove FPN from shadows but same time we add FPN to highlights. Qualifier masks is not a scientific solution. It is just a fake sloppy partial fix that only produce additional artifacts.
I done some further tests and see that FPN mask applied in Linear gamma seems more accurate way to go. It produce less hot pixel dots compare to FPN mask applied in Log gamma.
So i was thinking maybe it is just something wrong with Dark Frame? Maybe we see this gamma shift in FPN because we export and process gamma corrected BMDFilm TIFFs inside gamma corrected stacking app? Maybe we need to export Dark Frame to TIFFs or EXRs in 32 bit float with linear gamma? Process it with Mean or Median stacking in same Linear gamma, and save it to single Dark Frame also in Linear gamma?
And here is the tech problem - we need some really powerful stacking app that can do things in 32 bit float with Linear gamma. Photoshop can't do these operations in 32 bit. PhotoLine works well in 32 bit float, but it can't do layers stacking.
UPDATE: I try to apply maximum TNR, keep image in Linear gamma and export Dark Frame as 32 bit EXR from Resolve, and it make no difference. Linear same as Log Dark Frame source produce same FPN stripes in bright areas.

uhm... from your test what i can understand is that FPN is quite removable only on some shot, where you have constant luminosity (dark or light), but if you have both (shadow and highlights) when you remove from one you add to another.

I think we need an "ai intelligent" tool that blend dark frame in different mode due pixel luminosity, to adapt to different pattern effects to different luma value.
could be scriptable? from fusion section.

I'm not a developer, but if some cameras can do that in realtime (what i understood from red's comments), must be an algorithm that allow us to do that if before to shoot camera can store a dark frames to remove later.

an idea could be that braw add additional sidecar folder with dark frame folder to manage this kind of post in automatic way later from resolve.
i know that redcode store with raw a lots of data, and often early shooting with more noise and problems later can be developed better.

Schermata 2020-06-14 alle 15.15.14.png (706.8 KiB) Viewed 6883 times

experiment with your tech i prefer to use neat video for denoise and i observed that on my shoot luma is quite clear, like you can see in this picture, but cr and cb had more pronuonced FPN, may be different debayering strategy could fight better the FPN and solve problem, or different application of dark frame for single channel could be more effective (or give us more color strain).

Here is quick example of Dark Frame exposure levels i used in my tests to fix FPN. As you can see it is not ultra deep black level. To make it work i usually need to increase Exposure to +4 or +5:



And to compare here is frame shoot with closed cap. This frame also consists FPN, but it is hidden so deep in the dark that you never can see it in reality. And if you increase Exposure to +5, you start to see a lot of horizontal dynamic sensor noise artifacts that makes impossible to combine it into clean working Dark Frame:

By the way, for test purposes and quick experiments there is no need to create Dark Frame based on 500 frames stacking in external apps. You may just use 3-4 TNR nodes with maximum settings in Resolve and export Dark Frame in Log or Linear gamma as EXR or TIF. Overall it works OK, but it is not ultra clean because it is based only on 5 frames.
Also instead of externally lighted grey card you may use small area of defocused large computer monitor screen. It produces very uniform surface. Set monitor brightness to 100% to avoid flickering. Open solid grey color object in graphic or text editor and control intensity with grey fill slider. Make sure that monitor surface have no dust, spots or any other defects.

Here is example of setup for quick Dark Frame export in Linear gamma and with Exposure increased to +5:


And here is how it works with EXR Dark frame in Linear gamma. Bright parts removed with Qualifier selection. But i repeat - This is not perfect solution. Even with Qualifier selection Dark Frame introduce additional artifacts:

I love the micro camera.

It really was the original Z cam.

I doubt that they are going to invest any moire engineering time for an "old" product like this.

I'm surprised it hasn't been updated to a new camera to be honest. I think though the pocket cameras have been a smash hit and that's been soaking up the engineering resources.

JB

Yes, it is. BMD states before the Picket came out, that they were unable to source a suitable UHD/4K S16 sized sensor with the same IQ as the HD sensor in the Micro Cinema Camera, to use going forward. Suitable MFT size sensors Were available, and the Pocket 4K is the results. I would also liked to have seen a larger “Micro Cinema” using the Pocket 4K sensor, would have been grand.
Cheers

Here is idea:
If it is too complicated or impossible to add advanced FPN calibration to camera firmware, maybe at least add this option to camera utility somehow?
- Connect camera to USB.
- Open camera utility.
- Calibrate sensor FPN during 10-20 minutes for maximum quality.

Here are few more real life FPN examples from BMMCC:
Original ISO 800:


Exposure increase +3

If red can do in camera, and store x data’s calibration should be possible, but..
My question is
where to store that calibration data’s?

In past I remember that many Blackmagic Design cameras from Bmpc4k to um4k,4.6k. Ursa Mini Pro when units are too fpn sensible are sended back to Blackmagic Design that “calibrate” and problem is sensible reduced.
May be, but only Blackmagic Design know if we user can do and what.

May be a risk procedure?
May they need some special tool?
May be that should insert in software sensible cs info that they want to avoid to share to avoid reverse engeneering?

I’m happy also if it will be a payment software, like I bought usb base to correct back focus of some lenses I’m ready to buy many tool to work better with my cameras.


Inviato dal mio iPhone utilizzando Tapatalk

Here is interesting test shared by thebrothersthre3 at other forum. Original files here https://drive.google.com/drive/folders/ ... sp=sharing
Maybe Pocket6k/4k sensors have no FPN because back illuminated sensors don't allow 100% deep black where FPN problem begins? Or is it some hardware FPN correction provided by Sony specially for their sensors?

Ursa 4.6k under exposed (zoomed in to show noise pattern better)


Pocket 6k under exposed (zoomed in to show noise pattern better)

And then look how much information there is out the window.

What happens when you compare them "top" down ?

JB

Yes, Ursa dynamic range is way wider. Still downloading original files....

Here are examples processed with more consistent workflow directly from downloaded sources.
Both cameras just Scaled to 1080p timeline as is.




This is not 100% honest test because Ursa source is DNG and from file metadata it show Software: 4.8. It is some legacy firmware version?
FPN in Ursa became visible here on very early stage. Noise structure itself is very fine due dual gain output.
Pocket6K in motion produce sort of horizontal dynamic pattern which is also impossible clean with noise reduction. Noise structure consists of large particles, typical to basic digital sensors.

As for dynamic range: