Invalidate Render Cache if source clips are modified on disk

One of the more painful gotchas I noticed with Resolve is that if you Render Cache anything, Resolve has no idea if the clip(s) that were originally cached have been modified on disk (or gone offline altogether), and will not invalidate the cache for those clips if they were.

So if for example, you've cached a composite of an animated super over video footage, and then the mogfx artist later overwrites the animated super file on disk with a revised version, the next time you open the Resolve timeline, the previously cached version with the old graphic is still visible, and the original Render Cache remains in place.

I would expect Resolve to detect that the source clip has changed on disk, and invalidate the cache for any part of the timeline where that clip appears.

It's even more disconcerting that even if I take the clip offline in Resolve (modified or not), it still shows up in the viewer:



So in order to get the most current state of the timeline, I have to manually invalidate the cache by deleting the cache for those selected clips.

I can see the advantages of a caching strategy like this, but I personally don't like it, because to me, the term "Cache" always implies something temporary, and something that can change or be invalidated any at moments notice. In most other apps that use a caching scheme, the cached frames are invalidated as soon as the user changes a parameter that would affect the appearance of the frame, or if the source media itself has changed on disk. This to me is the correct behavior, as it always ensures that you are seeing the actual state of the clip(s) as they would be if you output them without any caching enabled.

If the current behavior is the preferred behavior for some users, I would suggest offering a "pre-render" system, much like how After Effects "Comp Proxies" work, where you can actually associate a composition or clip with another pre-rendered file. This would make it a "static" cache of that composite/clip, and only invalidated if the user specifically detaches that pre-render.

Following up with a cross-referenced post regarding another instance (Edit page keyframing) where I've found where caches are not invalidating themselves when they should:

viewtopic.php?f=32&t=101829

Also, a related issue where Resolve caches offline clips (including the "Media Offline' placeholder slug!):

viewtopic.php?f=21&t=57843

While I agree that it can be improved, how do you want it to work?
Every time you access the timeline the machine has to scan all of the frames used and check for the time stamp and re-cache if a time stamp is different?

I personally want a second file, so I can present in the timeline a before/after but I understand that every workflow is different.

While I agree that it can be improved, how do you want it to work?
Every time you access the timeline the machine has to scan all of the frames used and check for the time stamp and re-cache if a time stamp is different?

I honestly don't know how other apps like Premiere and After Effects manage to do this, under the hood. But suffice to say they do manage to do it.

My usual workflow in Premiere is to drop in FPO graphics or renders from AE, and then have the mogfx artist overwrite the same file with updated renders, if the client tweaks things. This way, I don't have to keep re-importing "R2, R3, et. al." render files in my online project, and then next time I open the project, if one of those render files is updated on disk, Premiere will automatically invalidate the rendered caches for that clip, which ensures that I am always seeing the correct media at all times, even if the clip needs rendering/caching in order to play back in realtime.

Whether it's a performance issue for Resolve to have scan for filesystem changes in realtime is for BMD engineering to figure out, I suppose. But for what it's worth, on macOS, there is the FSEvents API, which allows the operating system to notify an application if specific files or foldershave been changed on the filesystem, instead of having the application have to constantly poll the filesystem for changes, which is most definitely a very resource intensive operation. I'm not sure if this is how Adobe handles this (and I'm not even sure FSEvents includes files on network shares?), but somehow they do handle it, so it would be nice if Resolve could do the same, because right now, the way it handles (or rather, doesn't handle) cached clips like this leaves a LOT of room for error when delivering render intensive projects.

Premiere will automatically invalidate the rendered caches for that clip

You may be looking at the result of Dynamic Link there, a feature which is neither available nor needed in Resolve as Fusion is built right in.

Premiere will automatically invalidate the rendered caches for that clip

You may be looking at the result of Dynamic Link there, a feature which is neither available nor needed in Resolve as Fusion is built right in.

Jim, how does this "not needed" help or relate to what original poster requests? Fusion inside Resolve does not invalidate cache for Resolve timeline.

Cache invalidation can be done simpler or more elaborate, easiest is to keep a note on what the modification time of source was and use this as an indicator. Caches in most cases are based on hashing together all relevant data from file, processing and whatnot context and when this hash changes, cache is invalid.

Jim, how does this "not needed" help or relate to what original poster requests?

It adds both understanding, and a suggestion for a new work flow.

1. If I'm correct about the Dynamic Link thing, then what the OP is seeing in Premiere Pro would not occur when using any media other than a DL'd AE comp. He'd see the same behavior there as he sees in Resolve. That suggests that what he wants isn't common, and may not be as easy to implement for reasons laid out by Walter.

2. With Fusion built right in, that can be used in place of AE for these updated graphics, allowing for a smoother work flow.

Premiere will automatically invalidate the rendered caches for that clip

You may be looking at the result of Dynamic Link there, a feature which is neither available nor needed in Resolve as Fusion is built right in.

What on Earth are you talking about, Jim?

I’m not discussing Dynamic Ljnk at all, I’m discussing automatic cache invalidation for modified clips. What does Dynamic Link have anything to do with this?

I know I’m a broken record in saying this to you, but please try to learn how to read and understand what posters are actually saying, and not extrapolate what you think they are saying.



Sent from my iPhone using Tapatalk Pro

1. If I'm correct about the Dynamic Link thing, then what the OP is seeing in Premiere Pro would not occur when using any media other than a DL'd AE comp.

No, you are not correct. You should perhaps try the workflow yourself before commenting authoritatively on it.

As I mentioned several times, my example workflow scenario was where I had a rendered AE mogfx element (prores4x+) placed in a Premiere timeline. This clip needs to be rendered over the video clip it was super’ed over in order to play back in real-time. And when the mogfx artist overwrites the render file with an updated version, the new version shows up in the timeline as expected, but now needs to be re-rendered again, because either the modification time, file size, checksum, (or whatever other mechanism Premiere uses to detect that a clip has been modified on disk) has changed

This is a workflow I have used hundreds of times before, and in fact, I rarely if ever use Dynamic Link because I simply dont trust it anymore, after experiencing numerous project stability issues as well as the fact that I don’t like having major external dependencies in my final timelines like a live AE comp, which requires additional levels of project management to ensure that everything archives properly at the end of a project. So no, this has nothing to do at all with Dynamic Link, and your conjecture that it might be ignores the fact that After Effects behaves the same way, as far as cache invalidation goes, even without Dynamic Link.

2. With Fusion built right in, that can be used in place of AE for these updated graphics, allowing for a smoother work flow.

Yet again, Jim, you’re intentionally muddying the discussion with unsolicited “workarounds” to the original feature request. This is a feature request sub-forum, not a general support forum.

Are you suggesting that all of my After Effects mogfx artists (as well as freelance artists) switch to Fusion for doing the mogfx, simply to avoid this caching issue? If so, then this is a truly delusional “workaround” that belies your understanding of the actual feature request being suggested.

Also, my specific example was just that—an example...and one of many examples I could bring up where automatic cache invalidation is important. If my example did not mention mogfx, then your “workaround” with Fusion would not apply, because it doesn’t truly address the issue being discussed.

Please stay on topic.









Sent from my iPhone using Tapatalk Pro

Just following up with another observation about this problem.

I just noticed that if you switch the Proxy Mode to Half or Quarter Res, then the modified source clip appears in the viewer as expected, however the actual Render Cache does not invalidate (blue line persists). Switching back to "None" as the Proxy Mode still shows the previous--now invalid--results of how the clip was cached before the underlying source media was modified on disk.

This is a huge problem.

An argument could probably be made by some people that the render cache should be preserved, no matter what happens to the source media on disk. However, I strongly disagree with this, because I'm of the opinion that in practice, a "cache" is/should be inherently transient, whereas a "pre-render" option would be a more sensible solution for people who want a more permanent solution, as suggested in this thread.

I've been working on a feature length film with someone on the other side of the country, and we're using Resolve in conjunction with Resilio Sync to update rendered clips, so that the director can view the colorgrade on his TV at home, without requiring that we constantly render and upload new versions of the film every time revisions are made. This workflow work perfectly, except for this one major flaw regarding cache invalidation. Obviously they can just delete and regenerate the caches on their end, but it sucks to have to regenerate all the caches in the film, when only a few shots need a regenerated cache. And I dont want to have to have them deal with the hassle of manually deleting caches for selected clips, which is a pain to do on a 2 hour long, 900-shot timeline.