Some Caching Questions

Hi. I've read the manual, but I still have some questions about Caching and media storage locations.

What is the difference between the cache location designated by the first media storage location in the DaVinci System preferences and the ones designated under the project Master Settings? Does the System preference merely define the default working folders for new projects, or is there something special about it? Does project level cache file location *override* the system level one, is it the other way around, or do they work in tandem (perhaps they’re not the same thing?)

I assume Optimized media always gets stored next to the Render Cache. Say I want use the Optimized media as proxies for working on a laptop on the road and leave the full footage at work. Is this as easy as directing the Project setting cache file location to an external drive, and moving the drive between the laptop and the desktop? Won’t the render cache files lose sync and cause problems?

If I open a project on a machine that didn't create it (via postGRESQL database server), I get a bunch of errors that ask me to update gallery/capture paths because each machine has different scratch drives, with different names. However, if I create a still, exit the project, then re-open the project on the same machine, the still persists. Where does DaVinci keep gallery stills when it can't find the designated volume?

If I *were* to name each machine's scratch drive the same thing, I assume Resolve would load and save cache files to the scratch drive of whichever machine opened the project. Of course, the separate drives would never be in parity, will this cause playback/render problems, or is Resolve smart enough to do something like tag the rendered frames with a MAC address? What about setting all our machines' paths to a single NAS loacation? We've got 10GBe, but is that asking for trouble?

What about Remote Rendering with “use optimized media” and “use render cached images” enabled, how does that work? When I send a job to another machine via Remote Rendering, what designates the location of Cache files to which the render *client* looks? Does each machine look to find a cache at it’s own System Preference Master? Does it look wherever the project tells it to look? Or, does it look wherever the job-submitting machine’s preferences are set?

Very confused.

Thanks.

So many excellent questions! I may be able to answer some, but I'm still on 12.5 most of the time. Perhaps some things have changed.

...or do they work in tandem (perhaps they’re not the same thing?)

The preference setting is the where you set the root of the volume to be used for caches. You can not set a different volume in the project settings caches field. But you can set a subdirectory in this field. For example, you can set each project to have it's own cache path, but they will be on the same volume.

Where does DaVinci keep gallery stills when it can't find the designated volume?

No idea, but this is very important. The same is true of caches. They are saved somewhere. I just did a search for *.dvcc files and I can't find them.

...Resolve smart enough to do something like tag the rendered frames with a MAC address?

I don't there is any attempt to index and manage the caches. There is (at least in V12.5) a brute force mechanism that invalidates caches and forces other seats of Resolve to re-render when something was changed. Or this may be a bug, but the effective outcome is that it's impossible for Resolve to connect to an older cache when the timeline was changed.

This is a lesser of two evils. You lose all of your caches, but at least, you can't find yourself in a situation where the timeline changes but the cache does not.

What about setting all our machines' paths to a single NAS loacation? We've got 10GBe, but is that asking for trouble?

I can't address your bandwidth requirements. They largely depend on the traffic volume, but because of the above invalidation mechanism, I'm unable to work this out, at least not between Windows and Mac. This however works for picture galleries.

Code: Select all

Where does DaVinci keep gallery stills when it can't find the designated volume?

I have a followup. In my case, on a Mac the preference top listed media storage is a SAN volume. The cache path in Project Settings is a Windows path, using a drive letter "E:\" which makes no sense to a Mac.

What happens then is that the caches are saved on the SAN volume from the preferences, but Resolve creates an "E" directory on that volume and saves caches inside. I suspect that's where the .Gallery directrory would end up too had I set it up the same way.

I assume Optimized media always gets stored next to the Render Cache. Say I want use the Optimized media as proxies for working on a laptop on the road and leave the full footage at work. Is this as easy as directing the Project setting cache file location to an external drive, and moving the drive between the laptop and the desktop? Won’t the render cache files lose sync and cause problems?

Just following up on this bit. I played around with this a little, didn't get as far as looking into what actually happens to the render cache, because the optimized media doesn't contain sound. This seems strange to me because I was using Prores LT for my optimized media codec. But I suppose the prores isn't wrapped with quicktime and doesn't share whatever container it uses with sound? Where is sound stored? Does DaVinci just stream it from the original media? Seems like that would waste resources, but maybe not?

The preference setting is the where you set the root of the volume to be used for caches. You can not set a different volume in the project settings caches field. But you can set a subdirectory in this field. For example, you can set each project to have it's own cache path, but they will be on the same volume.

I appreciate your help looking into the matter, Igor, but I don't think this is true.

In the project I currently have open, my scratch drive (Davinci Resolve -> Preferences -> Media Storage Locations -> 1st slot) is set to /Volumes/RAID. The project I'm working on has both cache file location and gallery still location (in Project Settings -> Master Settings -> Working folders) set to /Volumes/LOCAL_SSD/Users/MYUSER/Movies. When I create a new gallery still, it shows up in my local movies folder, not on the RAID. The file it creates is a .dpx, and a new file gets created with a matching timestamp every time I create a new gallery still.

I *believe* that the Preferences -> 1st Media storage location setting is just the default path that each new project inherits in their Project Settings -> Working folders field(s?). And I *believe* that the Project Settings override the System Preference. This is my working theory, at least. HOWEVER, If this is true, then I don't see how collaborative workflow machines (working on the same project) can make use of fast, locally attached RAIDs for scratch disks.

Say you've got three machine; A, B, and C collaborating on project X. Each machine has access to shared network storage N. Each machine also has it's own, locally attached RAID; Ra, Rb, and Rc. If Machine A sets its working folder(s) to Ra in the Project settings, then B and C are also set to Ra, to which they don't have access. If B, upon seeing the missing cache error at startup, rectifies the situation by changing the Project working folder to Rb, then A and C will now be set to Rb, which is wrong. Is the solution to this problem path mapping?

Path mapping isn't really explained very well in the user manual, either. It seems like it basically tells the computer "if this first path is invalid, then try this second path." It seems mostly designed for mixed platform situations where one system uses backslashes for path separators a and another uses forward, or something to that effect. But say in Project X, I have A set to Ra, plus a pathmap to Rb. Machine B should get an invalid path error for Ra, and switch gracefully to Rb. This sounds fine and dandy, except what about Machine C? How do you set a third alternative path map? What if you had a whole classroom full of machines instead of just three?

Or, maybe it's better to set the Project working folders to N, because each machine would have access. But this raises questions of cache invalidation. Does the collaborative workflow bin locking system prevent the cache from becoming corrupt? Also, caching over a network will never be as fast as caching to a locally attached RAID, but is it fast enough to make practical sense? If so, are the RAIDs just superfluous in an office environment where users will usually be working on the same project at the same time?

Also, as I said, I'm just *assuming* that the Project Settings override the System Settings. One thing that's confusing about this is that if the Project Settings take precedence, then why is the path mapping done in the System Settings?

More to the point, if the project working folder overrides the system default as the scratch drive location, then does the Path Map work to substitute a path for an invalid project working folder?