Critical Bug DaVinci Resolve 19.0.1 with Neat Video Plugin

Hello,

I've stumbled onto a serious bug that makes Neat for Resolve quite unusable for me. So I use Sony A7SIII 10Bit 4:2:2 H.264 4K MP4 50p footage on a 25p timeline and in Davinci Wide Gamut/Intermediate color space.

I add Neat as the first node before the rest of the grade. The issue also occurs when adding it instead as an effect on the clip in the edit page. Additionally it happens on all configurations (CPU, GPU, CPU & GPU) In the latest Resolve (19.0.1) after I select the noise pattern and apply the effect on the node, it does two things wrong. First of all almost all times, the frame changes slightly (like it picks one after or one before the selected frame. This is evident when disabling and re-enabling the node, there is slight motion, which is a problem.

The main issue however is that if I leave it to render without doing anything to the UI sometimes it renders most of the times fine, however, if something happens to the UI such as I move a dial or auto-save, or even scrolling on the timeline (in the color page) at that moment what renders is a slight jump (like it skips frames). This also happens if I leave the playhead in the middle of the clip and leave it to render. It renders from the playhead onwards, and finishes with the beggining of the clip up until the play head (if it's the only clip in the timeline). However at the place where the playhead is, it jump cuts a few (or even one) frame, creating a stuttering effect.

Response from Neat Video Team:

"
Dear Nikolaos,

thank you for the message.

> I've stumbled onto a serious bug that makes Neat for Resolve quite unusable
> for me. So I use Sony A7SIII 10Bit 4:2:2 H.264 4K MP4 50p footage on a 25p
> timeline ...

Say no more. This is a known bug of Resolve itself and it is described in this page:
https://www.neatvideo.com/support/known-issues/resolve

Quote:
Incorrect order of frames with retimed clips (Resolve 17 and possibly other versions)
Resolve serves to Neat Video incorrect frames or frames in an incorrect order when (1) the source clip is retimed in the project or (2) the frame rates of the clip and timeline are different. This then shows up in the output video as a stutter or partially reversed motion.

A possible workaround: go to the Edit page, select the clip, right-click to open the context menu and select "Render in Place".

We asked the developers of Resolve to address that, but as you can see, the issue remains unfixed since Resolve 17.

Thank you,
Vlad

Support, Neat Video team, ABSoft
mailto:support@neatvideo.com https://www.neatvideo.com"

Why hasn't the Resolve team, resolved this yet???

Two years ago, I worked with FCP engineering and Neat Video on the original manifestation of this problem, which affected both FCP and Resolve. On FCP it was not isolated to Neat Video but also happened with some other temporal effects if the clip was retimed or rate conformed. It was then isolated to M1 Max and M1 Ultra, apparently starting with Monterey 12.3.

I'm just an end user, but I formerly worked in software development and was concerned that frame order errors could be encoded into our archives.

My focus was then mostly on FCP, and it was fixed there within a few weeks. I thought it was also fixed in Resolve around that time. Since then I've used Resolve considerably on M1 Ultra, including Neat Video, and I haven't seen the problem.

M1 Max & M1 Ultra have multiple parallel media engines, which could imply that problem was related to synchronization. You have an M2 Max, so your reported Resolve/Neat Video behavior would be consistent with some manifestation of the earlier problem.

Yesterday I re-checked the original scenarios on FCP 10.8.1, MacOS Sequoia 15.0 on M1 Ultra, and it didn't happen. I could not reproduce it on Resolve 19.01, but I'll examine it more closely today.

If you are going to try it and resolve, make sure that it’s 50p footage on a 25p timeline. Doesn’t happen if the footage matches the timeline.

I can reproduce it using a 1080p/29.97 clip on a 29.97 TL with 25% retiming, Neat Video 5.6.5, MacOS Sequoia 15.0, Resolve Studio 19.01 on an M1 Ultra Mac Studio. The symptoms are similar to the problem in 2022: infrequent, random frame order errors on clips with Neat Video if they are retimed OR rate conformed. I haven't tested M1 Max yet, but it's reasonable to expect it might happen there.

Here is an example I just reproduced. This is 1080p/29.97 ProRes 422, using a ProRes 422 original. It's all-intraframe so there's no possibility of Long GOP decoding/encoding issues. It's retimed to 25% speed (required to reproduce problem), so it's expected that every frame will be repeated 3 times. You can see the mis-order problem around frame 7. At the clip end (00:03:10), there some anomalies with repeated frames: https://www.dropbox.com/scl/fo/bb8paubu ... 7sk5t&dl=0

Your M2 Max case involves MacOS 14.4.1, so does not apparently require Sequoia 15.0.

I'm trying to reproduce it without Neat Video by using various temporal effects inc'l TSNR, so far without success. The 2022 problem was readily reproduced with Neat Video but also happened (at least on FCP) with various temporal effects.

The problem can happen when either rendering to cache or rendering to the output file. If rendered to cache, the incorrect frame order is stable and can be examined in playback using JKL commands. However if exported to either ProRes or H.264, it will be re-encoded from the source files unless you enable the Deliver page Advanced Settings "Bypass re-encode when possible", and "Use render cached images." If re-encoded during export, the visible problem in the cache often won't appear in the output file.

The Resolve 17 manifestation mentioned in Neat Video's response was the 2022 case. With FCP, that was either fixed or worked around via an application-layer fix in FCP, and did not require a MacOS or Neat Video update. Unless Neat Video has re-examined this on the latest versions, it might not be exactly the same.

My current focus is reproducing it without Neat Video. In your case, a possible workaround might be using Resolve's Temporal/Spatial Noise Reduction. If you find a scenario where it happens without Neat Video, please contact me. But even the possibility of incorrect frame order is gravely serious; that is a form of data corruption.

Great job reproducing the issue. The issue is exaggerated with some sort of UI interaction - though not necessary. Frames are heavily reordered especially when auto-save or some other part of the UI is altered during render. Again this is not necessary to reproduce the issue. Unfortunately the only alternative that comes close to Neat is the new Resolve UltraNR, but that in combination with a 2 frame Temporal NR bogs the system down to a halt. Really inefficient compared to Neat.

Hopefully Blackmagic will give this priority soon.

I have emailed Blackmagic's tech support - for anyone running into this issue please also send a report yourself - and attach this relevant thread.

Here is the link to this thread for quick reference:
https://forum.blackmagicdesign.com/viewtopic.php?f=21&t=208737

Thanks. The info about auto-save or UI actions during render is useful.

I'm narrowing it down. Resizing the UI doesn't affect it after all. I created a test file. 4k 50p ProRes 422, placed on a 25p timeline. It has a timecode burned in. Retiming for the project and the clip is set to linear.
It's a problem with the rendering engine.


I import the test clip and apply the neat video latest version (i have created a grey square at the bottom center to sample some noise as well)..
Then I slow down the file to 50%
If smart render is off, it's more accurate - but not always. Sometimes 1 frame off. However, rendering it out to a file is still problematic even with Render Cache set to off.
If smart render is on, it has a 2 frame offset and starts at 00:00:00:01 instead of :00.

To test how it messes up I add REC and Source Timecode burn-in on top.

If I don't retime the file (so 50p at 100% realtime in a 25p timeline) the issue doesn't show up. All timecodes align. However this is a ProRes file. If I use my footage (A7SIII 10-bit 422 MP4 H.264 4Kp50) at 100% realtime on a 25p timeline, I still need to run some tests, however the issue is there. So I'm guessing MPEG encoding further messes it up.

If render cache is off, file is at 50% linear retime and I scrub and see the file timecode and compare it to the source timecode, it is usually off by 1-2 frames. If I reset something on the clip (such as the transform section) it refreshes to the normal timecode value. However if I then turn render cache to smart, let it render and refresh the playhead by moving 1 frame back and then 1 forward again, the rendered version is 2 frames off and it does not properly skip 1 frame when I preview it frame by frame. The data-burn in shows the timecode correctly, but the clip's burned in timecode is always off.

Here is the link to my test file for anyone wanting to try this out:
https://kronik.wetransfer.com/downloads ... 848/e273cd

I've tried numerous things to make it happen without Neat Video; so far I cannot. This includes many UI actions and auto backup during render to cache, combining retiming and rate conforming, adding numerous temporal effects, using four stacked layers each scaled to 1/4 of the screen, etc.

On the 2022 problem, that would easily make it happen (with FCP). I recall it also happened on Resolve, but I can't remember the details. In 2022 the alleged cause was the host process (FCP or Resolve) serving "incorrectly-ordered frames" to the Neat Video plugin.

Today, even four instances of Neat Video 5.6.5 on different layers in FCP will not make it happen. That's on the same M1 Ultra machine and OS and same test parameters as used when it happens with Resolve 19.01. But Neat Video on FCP uses a different API -- FxPlug4 vs the OpenFX API on Resolve.

The OpenFX documentation actually states that it's the plugin's responsibility to be prepared for non-sequential, i.e., out-of-order frames. The plugin can request sequential rendering from the host, but this cannot be guaranteed, and the plugin must handle non-sequential frames. From this standpoint, the host providing non-sequential frames is not a bug but possibly an optimization or platform characteristic.

https://openfx.readthedocs.io/en/main/R ... al-effects

"Host applications, due to their architectures, may or may not be able to guarantee that a plugin can be rendered strictly in-order. Node-based applications typically have much more difficulty in guaranteeing such behaviour... to indicate whether a host supports such behavior we have a property, kOfxImageEffectInstancePropSequentialRender...For hosts, this property takes three values…

0, which indicates that the host can never guarantee sequential rendering,
1, which indicates that the host can guarantee sequential rendering for plugins that request it,
2, which indicates that the host can sometimes perform sequential rendering."

It's conceivable that Resolve 19 has more multithreaded concurrency, or on the Max/Ultra can sometimes provide non-sequential frames (maybe due to parallel media engines). However, it appears from the OpenFX docs that it's the plugin's responsibility to handle that.

That said, the 2022 fix for this on FCP was in FCP 10.6.4, not in Neat Video or MacOS. However in that case it affected other FCP built-in temporal effects, not just Neat Video.

If Neat Video thinks Blackmagic hasn't fixed this since Resolve 17, that seems unlikely. That implies the problem has been here for two years, and nobody has noticed and reported it. The recent reports (and you being on MacOS 14.4.1 Sonoma, not Sequoia) imply it began with Resolve 19, involves interaction with Neat Video on the Max/Ultra CPUs, and is not unique to Sequoia.

I will set up a tidy replication scenario and discuss this with Neat Video.

I've tried numerous things to make it happen without Neat Video; so far I cannot. This includes many UI actions and auto backup during render to cache, combining retiming and rate conforming, adding numerous temporal effects, using four stacked layers each scaled to 1/4 of the screen, etc.

I have a workaround for you.

1) don't make Neat Video the first node in the graph. I think there's a lot of problems with that approach, because basically you're "stomping" on the signal before any major contrast changes are applied.

2) don't use Neat Video in projects where run speeds are changing from the actual clip speed. Neat (and other temporal-based processors) need to know exactly where the next frames are going to come, and if the frames don't come at a specific time and in a specific order, it's going to screw up.

3) my workaround is: use no NR at all in the initial pass. Render it out in a mezzanine format like ProRes 444 or DHxHR 444. Now, create a new timeline that only has a single node in it for Neat Video. Drop the flattened file in the timeline, and cut it up to change the intensity of Neat Video based on the nature of the material: crank it up when there's a lot of noise, turn it down when the noise level is minimal. Split the clips to allow changing NR levels on scene cuts.

You'll find this will absolutely work. Try it and see.

Hi Marc,

Regarding not adding Neat as the first in chain, this does not fix the issue. Whether it's added at the beginning, middle or end of the chain, or even as an effect on the clip in the edit page.

I do however need to add it before the added (artificial) grain. And efficiency and space wise this is far from ideal. The only "workaround" is to use Davinci's inferior noise reduction on "retimed" clips or keep rendering Neat until no break is visually seen on the Render Cache, and export using the render cache files. So your suggested workarounds don't really address my issue. The whole point is to mobilize Blackmagic to take a look at this and fix this bug. It's a frame corruption issue, and Neat cannot do anything on their part (per their reply). If I have time, I will create a video of the issue and post it here. Resolve has a corruption issue when Neat is used, both when using Render Cache, or leaving it off and exporting directly to a video file. It fails to create a proper sequence of frames when Neat is on anywhere in the chain and linear retiming.

Thank you for your suggestions though.

...my workaround is: use no NR at all in the initial pass. Render it out in a mezzanine format like ProRes 444 or DHxHR 444. Now, create a new timeline that only has a single node in it for Neat Video....

Thanks, Mark. Anything from an industry professional with your experience bears close examination. We think this issue is bug specific to Max/Ultra, and it's unclear how long a fix might take -- esp. if there's a disagreement between Blackmagic and Neat Video about who is responsible.

Your advice about adding Neat Video to a rendered output file could be an actual workaround since the bug involves retimed or rate-conformed material. Those changes would be "baked in" by that point, so it might avoid the problem. If specific effects require NR first (like film grain), maybe those could be deferred to this stage. Provided the material is not further retimed or rate conformed the frame order bug likely won't happen.

Your suggestions seem two-pronged: A workaround for the bug and possibly a good idea anyway. It is an additional step, but I have sometimes been forced into that for pure performance reasons. It would be funny if the bug was fixed and we kept using your "workaround" for other reasons.

It seems to be limited to material out of recent Sony cameras.
Just last week I had to use NV on a few clips out of an Arri Alexa LF, a Sony Venice (OCN files), and a UMP 4.6K. All of them went through fine.
So, my suggestion would be to transcode your sources to a high quality mezzanine codec for now.
The problem might be connected to the general issue with Sony files on M3 Max machines, which seems to be cured by MacOS Sequoia (15.0). While I'd be very reluctant to go for such a young version of an OS, it might be an option if you get a lot of such footage in.

Unfortunately, the problem still occurs with Apple ProRes footage. It is not sony exclusive. Nor is it fixed to the M3 max, but happening on my M2 Max. I highly doubt an os upgrade fixes this, and I am definitely not willing to install it since it breaks a lot of other functionality that I need.

Nikos, yes, it happens with a pure ProRes 422 workflow - input files, render format and export format.

The key items are (Nikos please correct if needed):

- Actually tested: M1 Ultra, M2 Max. It is likely it happens on the other Max/Ultra versions, although not yet tested.
- Neat Video (in my tests 5-frame temporal setting makes it more likely)
- Timeline clip must be either retimed or rate conformed
- Resolve 9.01 (possibly 9.0 also)
- Not dependent on Sequoia 15.0 -- it also happens on Sonoma
- It appears as an out-of-order frame if rendered to cache (ProRes 422 in my case)
- It only happens periodically. You may have to delete cache and try again to see it
- It may not persist if exporting, unless you use "Bypass reencode" and "Use cached images"
- I think it can happen despite the cache state, but that needs verification

Max has one H264/H265 decode engine, two H264/H265 encode engines and two ProRes encode/decode engines. Ultra has two H264/H265 decode engines, four H264/H265 encode engines, and four ProRes encode/decode engines. All other M-series have one encode and one decode engine. Problem is isolated to these chips, which may imply a concurrency issue with the parallel engines causing periodic out-of-sequence frames, which Neat Video is not handling properly.

The OpenFX docs say plugins must handle out-of-sequence frames, so that is a point of contention since Neat Video claims only in-sequence frames should be delivered to their plugin. This is the core issue that must be resolved.

Your advice about adding Neat Video to a rendered output file could be an actual workaround since the bug involves retimed or rate-conformed material. Those changes would be "baked in" by that point, so it might avoid the problem. If specific effects require NR first (like film grain), maybe those could be deferred to this stage. Provided the material is not further retimed or rate conformed the frame order bug likely won't happen.

Your suggestions seem two-pronged: A workaround for the bug and possibly a good idea anyway. It is an additional step, but I have sometimes been forced into that for pure performance reasons. It would be funny if the bug was fixed and we kept using your "workaround" for other reasons.

Yeah, in many cases, we're so crunched for time, there's almost no ability to have a discussion about the matter. In post (especially in LA), usually there's people screaming, "I don't care WHY it's screwing up! Just get it DONE!" So rendering it twice will solve the problem.

We sometimes cheat by using two machines at once, so the instant the first render is done, we launch Resolve on the second system, open up the rendered file on a networked drive, apply Neat (and come up with enough settings to work on any changes), and start rendering -- slowly -- with Neat. And then we continue working on other things on the first system.

Slo-mo shots, speed ramps, 24-frame film-based material in a 29.97 project, and Interlaced video are all a challenge for noise reduction processes, because scene cuts are affected, and interpolated pictures can get smeared across a scene boundary. One of the biggest "torture tests" I know of for noise reduction is somebody in a pitch-black room carrying a candle. You'll see NR artifacts and frame smearing on cuts, when a piece of the flame continues across the next frame. Fire is also a tough challenge for compression engines: I see artifacts around fire all the time in House of the Dragon, and that's a $20 million-per-episode TV show for MAX. So these are known issues that technicians try to solve every day.

I do however need to add it before the added (artificial) grain. And efficiency and space wise this is far from ideal. The only "workaround" is to use Davinci's inferior noise reduction on "retimed" clips or keep rendering Neat until no break is visually seen on the Render Cache, and export using the render cache files. So your suggested workarounds don't really address my issue. The whole point is to mobilize Blackmagic to take a look at this and fix this bug. It's a frame corruption issue, and Neat cannot do anything on their part (per their reply). If I have time, I will create a video of the issue and post it here. Resolve has a corruption issue when Neat is used, both when using Render Cache, or leaving it off and exporting directly to a video file. It fails to create a proper sequence of frames when Neat is on anywhere in the chain and linear retiming.

Having a combination of variable frame-rates, AND noise reduction, AND added grain sounds like a recipe for a trainwreck. There's lots and lots of situations where this is going to crash and burn badly. I would re-think your workflow and find a better way to deal with this.

If it were me, I would do four things:

1) render the whole thing out to a single-speed flattened file in a high-quality codec (like 444 or even EXR)

2) noise-reduce the whole thing on a scene-by-scene / shot-by-shot basis to make it completely consistent

3) render our the NR version as a 444 flattened file

4) do one final grain-added pass for the "deliverable" file (and that can be just one setting that kicks in from first frame of picture, unless you need to layer titles or something like that).

Don't laugh: we were doing this as far back as 2015, when we first started using Neat. As I said in the message above, it helps when you have multiple machines. You can truly run Resolve in parallel processors (even with render farms) if you have dual machines and two licenses. It's not easy, it's not cheap, but it will get the job done. I would not attempt to do everything in a single pass an expect it to be done efficiently, without any glitches, and without any motion problems.

I thought it may be related to the system version or our hardware, since I have not yet seen such a problem here. But since we are using NV in a very similar way as Marc explained, it might be just that.

I do however need to add it before the added (artificial) grain. And efficiency and space wise this is far from ideal. The only "workaround" is to use Davinci's inferior noise reduction on "retimed" clips or keep rendering Neat until no break is visually seen on the Render Cache, and export using the render cache files. So your suggested workarounds don't really address my issue. The whole point is to mobilize Blackmagic to take a look at this and fix this bug. It's a frame corruption issue, and Neat cannot do anything on their part (per their reply). If I have time, I will create a video of the issue and post it here. Resolve has a corruption issue when Neat is used, both when using Render Cache, or leaving it off and exporting directly to a video file. It fails to create a proper sequence of frames when Neat is on anywhere in the chain and linear retiming.

Having a combination of variable frame-rates, AND noise reduction, AND added grain sounds like a recipe for a trainwreck. There's lots and lots of situations where this is going to crash and burn badly. I would re-think your workflow and find a better way to deal with this.

If it were me, I would do four things:

1) render the whole thing out to a single-speed flattened file in a high-quality codec (like 444 or even EXR)

2) noise-reduce the whole thing on a scene-by-scene / shot-by-shot basis to make it completely consistent

3) render our the NR version as a 444 flattened file

4) do one final grain-added pass for the "deliverable" file (and that can be just one setting that kicks in from first frame of picture, unless you need to layer titles or something like that).

Don't laugh: we were doing this as far back as 2015, when we first started using Neat. As I said in the message above, it helps when you have multiple machines. You can truly run Resolve in parallel processors (even with render farms) if you have dual machines and two licenses. It's not easy, it's not cheap, but it will get the job done. I would not attempt to do everything in a single pass an expect it to be done efficiently, without any glitches, and without any motion problems.

It's literally happening on 50p footage played in 100% speed on a 25p timeline as well as at 50%. So it is either a skipped frame or no frame skipping at all. There is no complex retiming.

Also if you download my test file and see for yourself, it's clearly a 2 frame or 1 frame offset render error. It calculates the frames correctly when unrendered and shifts frames by a specific amount once rendered. There is no complex math. If FCP can do it, so can Davinci.

We gave you workarounds. Fixing is up to BM.

...It's literally happening on 50p footage played in 100% speed on a 25p timeline as well as at 50%. So it is either a skipped frame or no frame skipping at all. There is no complex retiming.

Also if you download my test file and see for yourself, it's clearly a 2 frame or 1 frame offset render error. It calculates the frames correctly when unrendered and shifts frames by a specific amount once rendered...

I'm still studying this. BTW for some reason I couldn't reproduce it with your 50p footage in a 25p timeline; also tried various retimings. I don't know why -- there's obviously a problem. Can you provide a .drp file from your test project?

Due to system-to-system uncertainties, the 2022 problem required an immense amount of testing before I filed it with Apple. That problem would sometimes seem to fade in and out, which is typical of a multithreaded race condition. This problem feels similar.

It currently seems isolated to Neat Video, but there is another possible frame sequence error reported on an M1 Ultra not involving that: viewtopic.php?f=21&t=208477&p=1083015#p1083248

It's literally happening on 50p footage played in 100% speed on a 25p timeline as well as at 50%. So it is either a skipped frame or no frame skipping at all. There is no complex retiming.

OK, let me put it this way: what if you remove Neat Video entirely from the workflow. Render your 50p files to 25p and export them as a 25p ProRes 444 file, with all the color corrections and everything baked in.

Now: take the 25p file into a brand new 25p timeline and apply Neat Video to the 25p file in a 25p session. Report back and tell us if it works.

I've tried numerous things to make it happen without Neat Video; so far I cannot. This includes many UI actions and auto backup during render to cache, combining retiming and rate conforming, adding numerous temporal effects, using four stacked layers each scaled to 1/4 of the screen, etc.

I have a workaround for you.

1) don't make Neat Video the first node in the graph. I think there's a lot of problems with that approach, because basically you're "stomping" on the signal before any major contrast changes are applied.

2) don't use Neat Video in projects where run speeds are changing from the actual clip speed. Neat (and other temporal-based processors) need to know exactly where the next frames are going to come, and if the frames don't come at a specific time and in a specific order, it's going to screw up.

3) my workaround is: use no NR at all in the initial pass. Render it out in a mezzanine format like ProRes 444 or DHxHR 444. Now, create a new timeline that only has a single node in it for Neat Video. Drop the flattened file in the timeline, and cut it up to change the intensity of Neat Video based on the nature of the material: crank it up when there's a lot of noise, turn it down when the noise level is minimal. Split the clips to allow changing NR levels on scene cuts.

You'll find this will absolutely work. Try it and see.

I'm glad to see that I'm not the only one experiencing issues with Neat Video. This is a very straightforward workaround Marc, thanks for sharing.

I'm having this issue in DVR 19.1.3 and Neat Video 6.0 - only in the color tab and on the timeline level.

I'm having this issue in DVR 19.1.3 and Neat Video 6.0 - only in the color tab and on the timeline level.

The Neat Video "Known Issues" on their website says: "Beginning with Resolve 19.1.3 and Neat Video 6.0.1, this issue is resolved using an internal workaround implemented in both host application and plug-in. Please update Resolve and Neat Video to use this."

If you are not on 6.0.1, I'd suggest upgrading to that or a later version. Change history of fixes and improvements:

https://www.neatvideo.com/features/vers ... s=wrs29927