Blackmagic Forum

So I've been running through a few tests and found some strange results when dealing with File Caching on my Loaders.

I am getting a bit annoyed with how slow the file caching plays back, its seems nearly as slow as the exr or Quicktime that the loader is reading. It is caching to Raw, so I tried adding a new loader (not cached) in place of the old one that directly reads the Raw File cached sequence and it is magnitudes faster. I've messed around with various modes but I can't understand why the original file cached loader is not running anywhere near as fast as the direct raw file loader, surely it should just read the same raw files?

Are there any Fuse's out there that can convert the file to raw and read that instead (Like file cache is supposed to do!)

I also have not been able to get the Network render for file cache to work, I can network render my comps (Though Quicktime loaders seem very buggy).

Running Windows 10, Caching to raided SSD's, Footage on Raided HDD's. GTX 1080, Xeon 2697 v4. 64GB

Any help appreciated.

OK, I have Found problem with Network Rendering for Cache files, my cache drive only had read share permissions over the network, not write, changed this and problem has been fixed.

Why are you caching your loaders? Fusion does that for you.

Sent from my GT-N5110 using Tapatalk

Vito La Manna wrote:Why are you caching your loaders? Fusion does that for you.

There's different layers of cache, though. You can cache the images to RAM, the images to files, the files to files, etc..

Loader Cache allows you to cache a file across multiple loaders and comps.

I have been looking at trying to improve the performance of Fusion while I am comping.

I am working with 4K renders and when they have a number of elements in there the system Ram (64GB) gets used up pretty quickly by Fusion. I am trying to reduce processing bottlenecks manually by using file caching (and forcing RAM caching) and also trying to avoid issues with Quicktime files by having them file cached to raw files which improve speed and eliminate slow QT processing (which it does).

I also have a very quick PCIe SSD that I am utilising as a file Cache to improve performance of my raided HDD drives where the source is stored.

From my testing I have noticed that RAM cache priority is applied from the viewed node backwards along the flow which means changing any node back in the flow that is no longer cached requires that chain to be reprocessed back down the flow. File Caching can eliminate some of this re-processing. Its not always practicle but carefully used File Caching should be able speed up my workflow (and also final renders) dramatically. Also using network render and pre-render for the file caching may also speed things up. Proxy and ROI are other tools that I am using as well to improve performance.

When you comp a lot, 20% performance here and there can make a big difference. How much easier and satisfying the job would be if everything was realtime.

Just to clarify, are you using Loader Cache or are you disk caching the Loaders?

Disk caching the loaders.

Kel Philm wrote:Disk caching the loaders.

Yeah, don't do that. Try the Loader Cache. As long as you're using a fast file format, it will be much faster.

Would a Loader Cache be faster than a File Cache (.raw) stored on the same drive? I assume .raw is Fusions optimal file format given it uses this for Caching, also my tests showed that .raw is much faster than any other format when reading into memory (EXR is pretty good though).

I've checked the loader cache and it appears to just copy the file over to the folder you assign it to, not sure how that is useful unless you specify a faster drive than the source or am I missing something. My tests did not yield any great results with the Loader Cache.

I don't know whats going on under the hood though and my tests are hardly scientific so happy to learn.

Correct, it's just copying the files to a faster drive. But the disk cache has to read the files in the LD, create images from the data, create new files, put the image data into those files, then read those files again and create new images. If you're trying to just get faster reads, then yes, converting to .raw or .dpx or something fast like that will help, but you shouldn't be doing that in the disk cache but as a separate render. There isn't really a need to keep the LD interactive, as the input isn't likely to ever change.

Thanks Chad,

This seems to be the crux of my original issue. I assumed once I had pre rendered the loader as a file cache it would only bother reading the cache file when needed, not also reading the original file (that as you stated usually never changes) and processing it each time, which to be honest seems a bit pointless. Would locking the cache force this perhaps?

I will log this with BM as I think its implementation needs to be looked at, any large comp with a number of QT loaders slows to a crawl because of decoding when this can be avoided with a pre-processing trade off.

It feels like Fusion's Loader Cacheing is incredibly slow because it checks if the files still exists on the network. This check itself takes the same amount of time than just reading the file on a congested network.

How can this "check" be switched off or time delayed?

i like the idea of the caching, it sounds simple and effective, but it feels kinda buggy/slow on my pc. i started a topic a while ago: viewtopic.php?f=22&t=63561

Thanks Rick,

But I'm trying to stop Fusion from constantly checking if the local files are the same as on the server. When I unplug my LAN cable fusion is really fast when "play cache when original files can't be located" is selected. As soon as I plug my LAN cable in it grids to a halt.

Did anyone report a bug about this issue?