Can Resolve remove GOP flicker?

Hi

I have a drone that often produces video with a nasty flickering effect, I'm informed this is common with DJI drones and it's a result of a terrible codec. Every 8 frames the brightness and sharpness jumps. The way it works it records a complete frame and then it just records changes for the next 7 but the codec is terrible and the bandwidth is too small so at the next full frame the brightness and sharpness jump, it's really distracting and I'd really would love to get rid of this. Any suggestions on how I could use Resolve to remove this? I'm running Studio 14b7 on W10

Cheers

Chris

I have generally used TNR noise reduction and keyframes to tame the drone footage I'm given. It ain't great, but it's OK in small doses. If I have to frame-by-frame something to get rid of an exposure bump, at least I know it'll be pretty close.

The flicker-reduction OFX modes only seem to work for me when there's not a great deal of movement in the picture, plus it depends on which part of the screen you're analyzing, so that would be a possible solution if all else fails.

Cheap 8-bit cameras shooting H.264 video are a drag no matter what you do.

Thanks.

I've tried TNR but have no idea what I'm doing. I'll keep playing but what I've seen is often only half the frame flickers. I will keep at it. I've tried the timelapse deflicker plugin without success. I guess you'd be horrified to find I'm shooting log with this little 8 bit cellphone camera sensor! Does seem to increase its dynamic range though

Cheers

Chris

This is called key frames pumping effect and it mens (as you said) that encoder is crap. It puts way to much bits into I frames causing them to be way better quality than other P,B frames. You can try filters which are made for removing light flickering problems. It's not easy to fix and you would probably need open source tools and help of dom9 forum guys to make it "fully" go away.

Or try BorisFX BCC Flicker Fixer

Problem is that it's not just brightness increase, but actual sharpness and details. You would have to make those "better" frames actually worse quality (by bur etc) to match others in order to "fix it".

You would have to make those "better" frames actually worse quality (by bur etc) to match others in order to "fix it".

this would be indeed an interesting use case for applying machine learning algorithms to improve the visual quality of the bad frames, but unfortunately i'm not not aware of any existing tool of this kind usable by ordinary mortals, which would work in this particular case.

Yes, but easier would be to make good ones worse Way less of them also.

If Chris could give a link to a source clip to do a test ?

If Chris could give a link to a source clip to do a test ?

yes -- and if it's a really horible distorted but nevertheless important clip, he could even initiate a recovery competition at kaggle as a last resort.

@Martin Schitter

No one has already set up a recovery competition at kaggle for this case? You should do it. You seem to like challenges.

Thanks everyone. I've already deleted the footage with the worst flicker but I'm sure I can find some more. I'll post a link later today

OK so I've uploaded a sample clip with the most extreme example of flicker, check it out:

P.S. did someone move my thread to a different forum? Weird, it's in beta feedback which I wouldn't have expected. Or am I having a senior moment again?

OK so I've uploaded a sample clip with the most extreme example of flicker, check it out

yes -- that's very helpful to get a first impression of your issue, but a youtube video isn't very helpful to analyze the underlaying technical details and recovery strategies, because the videos gets recompressed by you youtube a second time and nobody of us is able to isolate the original i-frames afterwards or distinguish, which artifacts are camera related and what's just a result of youtubes processing, etc.

it would be more suitable, if you could make the unmodified clip out of the camera downloadable somewhere.

I was afraid of that. I ditched Dropbox ages ago when I decided they were evil. Still, they are useful so I created a new account so I could share, hope this works:

https://www.dropbox.com/s/cta4dqjp0oa55 ... r.mov?dl=0

I was afraid of that. I ditched Dropbox ages ago when I decided they were evil. Still, they are useful so I created a new account so I could share, hope this works

yes -- they are indeed evil! -- but let's ignore it for one opportunistic moment...

i downloaded your clip, and it looks at least much better than the youtube counterpart. but the reported issue is clearly visible. it's indeed a very well demonstration for insufficient long-GOP handling resp. recording bandwidth. but what's very interesting, is the fact, that this compression artifacts are not only caused by fast motion over highly structured surfaces, but rather caused or at least amplified by lots of sensor noise. if you compare the amount of artifacts in the shadows and well illuminated parts of the lawn, this becomes quite obvious. sure, they are also noticeable in the bright parts of the image, but the real troubles are IMHO more noise resp. sensor size related. unfortunately it's nearly impossible to restore this kind of distortions in post by usual tools.

I was afraid of that. I ditched Dropbox ages ago when I decided they were evil. Still, they are useful so I created a new account so I could share, hope this works

yes -- they are indeed evil! -- but let's ignore it for one opportunistic moment...

i downloaded your clip, and it looks at least much better than the youtube counterpart. but the reported issue is clearly visible. it's indeed a very well demonstration for insufficient long-GOP handling resp. recording bandwidth. but what's very interesting, is the fact, that this compression artifacts are not only caused by fast motion over highly structured surfaces, but rather caused or at least amplified by lots of sensor noise. if you compare the amount of artifacts in the shadows and well illuminated parts of the lawn, this becomes quite obvious. sure, they are also noticeable in the bright parts of the image, but the real troubles are IMHO more noise resp. sensor size related. unfortunately it's nearly impossible to restore this kind of distortions in post by usual tools.

Thanks for taking a look!

I'm wondering because this was shot on auto and late in the afternoon if the ISO was bumped up by the camera causing lots of noise and this is its attempt to clean it up? More recently I have been shooting fully manual, maybe if I limit the ISO it might help

just a suggestion to work around this kind of issues, you could perhaps use this little command, to extract only the I-frames (=the good ones) of the clip into a new clip with 1/8 of the framerate.

Code: Select all

ffmpeg -i 'original.mov' -vf "select=eq(pict_type\,I)" -c:v prores -r 3000/1001 result.mov


this may not be very useful, if there are lots of moving objects in your shot, but it could open creative alternatives and workarounds in some other cases.

Sorry to ask a side question on this, but is the “GOP flicker” the reason I see mini stutters in clips like these?

https://www.dezeen.com/2017/07/21/video ... buildings/

All this time I thought it was my PC being unable to play back footage smoothly via Chrome.

I just compare:
- neatvideo
- BCC deflicker
- BMD Timelap Deflicker

And for now the best is BMD Timelap Deflicker

I have to check because it is surprising (I did not expect to participate in this test ....)

Personally, I think the big issue of this (drone source) clip is the aliasing of the sensor: just look at the bridge or horizontal or ~ 42.06 vertical poles.

In short: drones: sensor / shutter pb : I avoid or better settings / capabilities.

I'll be back when I'm done.

I just compare:
- neatvideo
- BCC deflicker
- BMD Timelap Deflicker

And for now the best is BMD Timelap Deflicker

I have to check because it is surprising (I did not expect to participate in this test ....)

Personally, I think the big issue of this (drone source) clip is the aliasing of the sensor: just look at the bridge or horizontal or ~ 42.06 vertical poles.

In short: drones: sensor / shutter pb : I avoid or better settings / capabilities.

I'll be back when I'm done.

Thanks mate

Can I ask what settings you used for timelapse deflicker? I couldn't make it work for me when I tried

I just compare:
- neatvideo
- BCC deflicker
- BMD Timelap Deflicker
And for now the best is BMD Timelap Deflicker

i don't think, deflicker filters will help in this particular case. they are just correcting flickering changes in brightness, but will not improve anything related to compression artifacts and loss of spatial resolution.

temporal denoisers could indeed gain some improvments in this case, but i think, they will more work like andrew proposed it earlier: making everthing a little bit more worse to hide the changes.

the only technical strategy, which would look really useful in a case like this are convolutional neural networkes trained with the least distorted i-frames and reconstructing the other frames by utilizing generative adversarial networks resp. auto-encoders. that's a quite complex process, but it's practical application an efficiency was demonstrated in a lot of recent research papers concerning compression artifact removal for JPEG stills (eg. https://arxiv.org/abs/1605.00366 ).

machine learning techniques are already involved in some upscaling, deinterlace and motion compensating denoise video filters, but i'm not aware of any ready made tool dedicated to compression artifact removal up to now.

...I'm wondering because this was shot on auto and late in the afternoon if the ISO was bumped up by the camera causing lots of noise and this is its attempt to clean it up? More recently I have been shooting fully manual, maybe if I limit the ISO it might help

I shoot a bit with the DJI Zenmuse x3 and x4s cameras (Osmo) but always on manual shutter and with a heavy ND fitted for outdoor work. (ND 16 min) With a slow manual shutter and auto ISO around 100, I can shoot outdoors 100% reliably without any highlight clipping.

Manual shutter @ 1/50th for 25fps and sometimes with ISO set to AUTO (hovers around 100) so I have not seen this issue. Note the X3 sensor seems to be native 4K so in my tests, it gave poor performance in HD modes.

PS: Shoot 4K/UHD Log with ND fitted (x3/x4) then transcode to ProRes HD 444 and you will be sweet... (DNxHD is OK but ProRes seems to 'scale' better when performing a simultaneous transcode from UHD to HD)

...I'm wondering because this was shot on auto and late in the afternoon if the ISO was bumped up by the camera causing lots of noise and this is its attempt to clean it up? More recently I have been shooting fully manual, maybe if I limit the ISO it might help

I shoot a bit with the DJI Zenmuse x3 and x4s cameras (Osmo) but always on manual shutter and with a heavy ND fitted for outdoor work. (ND 16 min) With a slow manual shutter and auto ISO around 100, I can shoot outdoors 100% reliably without any highlight clipping.

Manual shutter @ 1/50th for 25fps and sometimes with ISO set to AUTO (hovers around 100) so I have not seen this issue. Note the X3 sensor seems to be native 4K so in my tests, it gave poor performance in HD modes.

PS: Shoot 4K/UHD Log with ND fitted (x3/x4) then transcode to ProRes HD 444 and you will be sweet... (DNxHD is OK but ProRes seems to 'scale' better when performing a simultaneous transcode from UHD to HD)

Thanks!

I now have a set of ND filters and I'm comfortable enough with flying it to think about manual settings so I'll give that a go.

The 4K 'log' setting on the Zenmuse x3 at least, seems to reduce the internal codec's colour processing requirements so with the limited bit-rate available, it might have less work to do. The only time I've seen any dreadful 8bit 'banding' with this camera is when using certain LUT options included with a Resolve installation (especially the DJI LUTs) so I don't recommend them when grading log images from these cameras.

yes -- given some advices, how to avoid this kind of issues while recording drone flights, is probably the most useful help.

most attention should be directed on avoiding to much motion! that's something, where long-gop codecs will always be limited to some degree and need some care. i have never seen critical situtation concerning this aspect in tripod shots or usual camera movements, but this drone examples demonstrate very well, what may happen, if you stress this kind of codecs beyond their capabilities resp. against their design principles. and it's always wise to work in harmony with the actual used tools, and not against them. so, first of all: avoid to much motion and change between frames on this kind of cameras and record mode!

The source file shows a data rate of 60Mb/s which is very low for a 4K DCI format.
Looking at the file in detail reveals a lot of encoder problems such as smeared frames in the faster areas of motion. The grass detail is very poor in places.
If you have any control over the data rate in the drone, I would increase it to improve future shots.
There is not a lot that can be done to fix encoder issues like this except fairly drastic noise reduction but that may create other visible issues.

The source file shows a data rate of 60Mb/s which is very low for a 4K DCI format.
Looking at the file in detail reveals a lot of encoder problems such as smeared frames in the faster areas of motion. The grass detail is very poor in places.
If you have any control over the data rate in the drone, I would increase it to improve future shots.
There is not a lot that can be done to fix encoder issues like this except fairly drastic noise reduction but that may create other visible issues.

The drone has a hard limit of 60 Mbps so nothing I can do there. Subsequently I have learned that if the sharpness is turned down (to try and avoid the oversharpened video look) then aggressive NR kicks in which toasts fine detail. Not sure what I had it set to in this video but probably -1. I use either 0 or +1 now but further complicating matters is it seems to change with each firmware update. I'm now on my fifth firmware update since I have had it so it's really unclear where we stand now

The source file shows a data rate of 60Mb/s which is very low for a 4K DCI format.

This has been driving me up the wall since 4K started popping up more in consumer technologies. IMO 1080 content is already bit starved on most streaming services, but now we can watch 4K bit starved content too!

I have a DJI Mavic "Pro" and have had this issue too. There is many good advice in this thread, but one thing you should try, given the 60Mb/s is to avoid the DLog setting.
I'm not sure about other DJI cameras, but the Mavic processor isn't very good with DLog.
Instead try a picture profile with a little more contrast and saturation. It will help with the flicker issue.

Try to get a decent image in camera that needs as little post production as possible. If you turn down contrast and saturation, you will need to boost those in post, and this will only exagerate the noise print.

Other things mentioned here as well, use ND filters to get your shutter speed down, and don't turn down your sharpness... as you have noticed, it kicks in NR that turns detail into mudd.
Some say that you even need to turn it up to +1 for it to be completely off.
Turning the DJI sharpness to +1 might also give you a little play to use more extreme noise reduction settings in Davinci.

I use both temporal and spatial NR inside Davinci and while I still notice a faint amount of flicker in my footage (because once you see it, you can never unsee it), I have never had anyone remark that they see flicker in my drone footage.

Each situation is different, but the best thing to do with these cheap cameras is to try to get your raw footage to look as good as possible and not rely on doing everything in post.

I have a DJI Mavic "Pro" and have had this issue too. There is many good advice in this thread, but one thing you should try, given the 60Mb/s is to avoid the DLog setting.
I'm not sure about other DJI cameras, but the Mavic processor isn't very good with DLog.
Instead try a picture profile with a little more contrast and saturation. It will help with the flicker issue.

Try to get a decent image in camera that needs as little post production as possible. If you turn down contrast and saturation, you will need to boost those in post, and this will only exagerate the noise print.

Other things mentioned here as well, use ND filters to get your shutter speed down, and don't turn down your sharpness... as you have noticed, it kicks in NR that turns detail into mudd.
Some say that you even need to turn it up to +1 for it to be completely off.
Turning the DJI sharpness to +1 might also give you a little play to use more extreme noise reduction settings in Davinci.

I use both temporal and spatial NR inside Davinci and while I still notice a faint amount of flicker in my footage (because once you see it, you can never unsee it), I have never had anyone remark that they see flicker in my drone footage.

Each situation is different, but the best thing to do with these cheap cameras is to try to get your raw footage to look as good as possible and not rely on doing everything in post.

Thanks heaps for that. Do you also think my problem was exacerbated by high ISO?

The drone has a hard limit of 60 Mbps so nothing I can do there. Subsequently I have learned that if the sharpness is turned down (to try and avoid the oversharpened video look) then aggressive NR kicks in which toasts fine detail.

yes -- it's really hard to find optimal settings for systems working close to unavoidable technical limits.
but you will often find advices on the net, which do not reflect technical reasons and requirements of the utilized compression technique in an adequate way. often people try to reduce the internal NR of cameras that much, that resulting irregular changes between successive frames can not be handled by long-gop codecs anymore in a sufficient way. in some cases you may even get really nice pictures on static scenes by such settings, but the image quality will break immediately when additional motion comes into play, etc.

it's really important to understand at least the basics about video compression resp. consider the interrelationship between a couple of fundamental causal factors and carefully watch out for behavior like in your demonstration video (significant quality differences between i-frames and the frames in between), if you want to fine tune this kind of preferences in a more reasonable way. if you don't want to spend much efforts on this kind of optimization, it's often better to use the factory defaults instead of questionable advices, which do not take all possible interfering factors into account and will inevitably lead to very insufficient results in some situations.

... Do you also think my problem was exacerbated by high ISO?

Probably as a high ISO is most likely applying Gain which can = Noise. I'm not sure which DJI camera you have on your Drone but the Zenmuse x3, x4s and x5 cameras I think, come with the various DJI Inspire drones. The Zenmuse x3 is very basic with a 1 2/3" sensor, the x4 is 1" and the x5 is MFT so for the entry level Zenmuse x3, an ISO of 100-200 should be all you need filming in normal daylight.

I'm from a broadcast background where video camera sensitivities are measured in Gain, not ISO. If your camera has a Native ISO of say, 100 with Gain at Zero dB and it is fitted with a lens of about f/2.8, it will be fine in daylight so long as you have a real iris to adjust. Many basic DJI cameras with fixed wide angle lenses don't have a real iris so they rely on a combination of Shutter and ISO to control Exposure. You need to find out or estimate the native ISO of your camera when no Gain is applied, (eg 800) lock it there, then set your shutter speed to 180 degrees (eg: 1/50th @ 25fps, 100th at 50fps, etc) and maybe use a variable ND filter to keep it there.

As I mentioned earlier, even the basic DJI X3 camera can shoot clean, noise free outdoor Log UHD at 25fps, shutter locked to 1/50th, ND 16 applied and ISO set to Auto. IMO unrealistically high shutter speeds completely ruin 90% of all DJI/GoPro videos posted to the popular streaming services.

The drone has a hard limit of 60 Mbps so nothing I can do there. Subsequently I have learned that if the sharpness is turned down (to try and avoid the oversharpened video look) then aggressive NR kicks in which toasts fine detail.

yes -- it's really hard to find optimal settings for systems working close to unavoidable technical limits.
but you will often find advices on the net, which do not reflect technical reasons and requirements of a given compression technique in an adequate way. often people try to reduce the internal NR of cameras that much, that resulting irregular changes between successive frames can not be handled by long-gop codecs anymore in a sufficient way. in some cases you may even get really nice pictures on static scenes by such settings, but the image quality will break immediately when additional motion comes into play, etc.

it's really important to understand at least the basics about video compression resp. consider the interrelationship between a couple of fundamental causal factors and carefully watch out for behavior like in your demonstration video (significant quality differences between i-frames and the frames in between), if you want to fine tune this kind of preferences in a more reasonable way. if you don't want to spend much efforts on this kind of optimization, it's often better to use the factory defaults instead of questionable advices, which do not take all possible interfering factors into account and will inevitably lead to very insufficient results in some situations.

Thanks, and yes questionable advices are everywhere, especially with this problem as everyone thinks they cracked the problem. I've already gone back to factory color and sharpness and that looks good. Next I will try turning d-LOG off and lock my ISO as low as I can plus ND filters for shutter speed

... Do you also think my problem was exacerbated by high ISO?

Probably as a high ISO is most likely applying Gain which can = Noise. I'm not sure which DJI camera you have on your Drone but the Zenmuse x3, x4s and x5 cameras I think, come with the various DJI Inspire drones. The Zenmuse x3 is very basic with a 1 2/3" sensor, the x4 is 1" and the x5 is MFT so for the entry level Zenmuse x3, an ISO of 100-200 should be all you need filming in normal daylight.

I'm from a broadcast background where video camera sensitivities are measured in Gain, not ISO. If your camera has a Native ISO of say, 100 with Gain at Zero dB and it is fitted with a lens of about f/2.8, it will be fine in daylight so long as you have a real iris to adjust. Many basic DJI cameras with fixed wide angle lenses don't have a real iris so they rely on a combination of Shutter and ISO to control Exposure. You need to find out or estimate the native ISO of your camera when no Gain is applied, (eg 800) lock it there, then set your shutter speed to 180 degrees (eg: 1/50th @ 25fps, 100th at 50fps, etc) and maybe use a variable ND filter to keep it there.

Hi

I have the Mavic and now have the factory ND filter set to try to control exposure. I now have zebras turned on and I have some limited control over shutter and ISO. In the sample i supplied it was late evening so the ISO was probably sky high. I'm learning from this thread that my lack of familiarity with the exposure controls is as much of an issue as a faulty codec so that gives me great hope and explains why the problem seemed to be intermittent

On any of these cameras which record internally to 8bit 4:2:0 long GOP codecs of marginal bitrate, low shutter speeds and high ND filters are your best friend. It's a pity that otherwise appealing videos are often totally ruined without them.

GOP flicker, as in Trump's Grand Old Party (Republicans)?

Hi Suhaib,

Do you have a solution?

Ok peoples

I went out today with the drone since it was a lovely winter's day. Shot some at 4K with ISO locked at 100 and an ND16 on it. The result had a noticeable flicker in it but to my absolute amazement when I played around with the settings in TNR, voila! it completely removed the flicker! Astonishing! You'd never know it ever had flicker! Smooth as a baby's bum! I turned it back off in case I was dreaming and sure enough the original had distinct flicker. I'm just over the moon now!

Thanks for everyone's help on this!

Hi Chris

Have you done a test again with the V14B8. As your clip is in MP4 and there have been improvements with the decoding of compressed files, maybe it's much better. I have just created a project 59.94 / TL 59.94, etc ..., import "bike reveal. flickr.mov" in a it remains a (small) bit of the flicker but if you activate the smart cache (in DNXHR 444 HDR, that is enough and play), Even without anything: it's not that ugly.

What is important is the shooting : OK. Now to try to retrieve clips not re feasable: Marc's advice in postprod remain the best.

Hi Chris

Have you done a test again with the V14B8. As your clip is in MP4 and there have been improvements with the decoding of compressed files, maybe it's much better. I have just created a project 59.94 / TL 59.94, etc ..., import "bike reveal. flickr.mov" in a it remains a (small) bit of the flicker but if you activate the smart cache (in DNXHR 444 HDR, that is enough and play), Even without anything: it's not that ugly.

What is important is the shooting : OK. Now to try to retrieve clips not re feasable: Marc's advice in postprod remain the best.

No I haven't. I thought 7 was latest!

I'll try and thanks to you and everyone else