General caching software for performance improvement?

Has anyone used general caching, e.g. AMD's StoreMI 2, Intel RST, PrimoCache, VeloSSD, etc. for improving video editing (etc.) performance, specifically to cache the source material?

I'm a Resolve Studio newbie, so I'm not fully versed on Resolve file and cache handling. And I'm setting up a new rig, trying to decide how much to spend/where on drives & caching (see sig. for my system #3 specs). My main focus is speed of scrubbing through my source to find the bits I want to use.

I'm wondering if this would be a good general solution:
1) dedicate 1 NVMe to DaVinci's optimized & rendering cache
2) dedicate *another* NVMe to a general *automatic* cache for the *source* files, rather than manually copying source material to it for editing (and later having to delete it when done).

This is assuming the source is stored on slow/spinning hard drives (in my case RAID6 for redundancy, but could work for HDD, TB, and USB)

Since they're block-level caches, it should only cache what is actually used most (i.e. not entire clips), and I wouldn't have to manually track what/where/when/etc. And it would benefit ALL of my work, not just video editing.

For those not familiar, there are 2 types, "file-level" and "block-level" caching:
1) FILE-based caching, is the same as manually copying whole source files onto a fast (eg NVMe) drive - even if you only use 5% of it.
2) BLOCK-level caching: only copy the 512B (or 4kB) blocks that are accessed repeatedly onto the fast/NVMe drive (done automatically)

So if I repeatedly use (ie read/play/use on the timeline) a 1-minute section of a 60-minute clip, ONLY that 1 minute of video will get copied into the fast cache, leaving far more room for other material in the cache. It's automatic, and the usual threshold is about 3 reads of the same block causes it to be copied to fast cache. If it's not used for a while, AND more room is needed to cache other blocks, the oldest/least-used blocks will be cleaned out (but still on the hard drive - I would only use the cache in read-only mode for safety, but it is possible to use for write caching as well). If you never fill the cache, the data will never be aged out (so it would act just like manually copying it to the cache).

Since this is automatic, you don't have to remember to copy your source there, or remove it when done. If you switch to another project, as you work on the new project, the new source material will be cached automatically. Or if you work intensively on one section of your timeline with vfx, etc., all of that would get cached ... until you move to another section, automatically bump the old stuff out for the new (all material always safe on the HDD, too). This would let your smaller NVMe act as if it's infinite in size, with a small delay for cache loading from HDD when you move to work on a new section.

Obviously, this would be HUGE for those using slow drives to store their source (single hard drives, USB, TB, etc.), and working directly from there. Also, those with a LOT of source material where they only use a fraction of it in projects, and have limited fast cache storage for their source.

BUT, this is SOURCE material, in whatever form (codec) it is. So if you optimize all of your source media into the Resolve cache anyway, you won't be accessing the (slow) source more than once - which means the *source* wouldn't get copied into the "general" cache (and it would be a waste).

Or am I way off the mark, and Resolve does something like this automatically anyway? (BTW most of my source is in 1080p mp4 h.264 and h.265, with some 4k coming into the mix going forward).

Would I be better off just getting a bigger Resolve cache drive? (i.e. single 2TB NVMe dedicated to Resolve cache, rather than 1TB NVMe for Resolve cache + 1TB NVMe for "general" cache for source)

Or, perhaps I get 1 big NVMe *general* cache drive to front my 30+TB RAID6, and set Resolve to just use the RAID6 for caching, with the general block-level cache fronting the HDDs. The system will cache *those* blocks for Resolve automatically as needed (i.e. Resolve will write it's cache output to HDD, but *perform* as if it has a dedicated NVMe cache - and the cache will be managed automatically between projects and all my other work - so each project will appear to have an "infinite" NVMe cache).

Thoughts?

PS, I don't think it would work with material that is changing on the source - but you shouldn't be changing the *source* files anyway, right? Test this thoroughly if you do change the source files themselves ...

I'm leaning towards the last option: one big (2TB?) NVMe drive as system cache, and using PrimoCache to front my RAID6 HDD store. So it's as much "automated tiered storage" as "cache".

I'll set my Resolve caches to point to the HDD RAID6 drive, and they'll ALL benefit from the PrimoCache caching automatically.

How it will work:
) Resolve writes (optimized media, vfx interim renders, etc.) to the "DR cache drive", thinking it's writing to the HDD volume; but it's actually written to the NVMe cache via PrimoCache, so it's fast

) PrimoCache then copies it to the HDD in the background; so even if the NVMe cache has to make room for new caching, it will still be on the HDD, and never have to be re-rendered unless changed - think about that

) It will also stay in the NVMe cache - so performance will be the same as just setting DR to cache directly to NVMe. (note: PrimoCache also supports a RAM cache, so if your system is up to rendering that fast, it will take it at RAM speeds, then move it to NVME)

) If I touch a section of video that isn't already in the cache, when DR reads those frames (from HDD), PrimoCache will load only those frames into cache (NOT the entire file), saving space on the NVMe

) On subsequent accesses, PrimoCache will serve it up immediately from NVMe - DR doesn't do or know any different, it's just FAST

) If I read/scrub/edit enough video and the cache is full, the oldest frames will be dumped from cache (but still on HDD), and replaced with the new. Again, DR doesn't do or know any different, it just works. And my cache goes much further, since I'm not storing ENTIRE source files in the cache, only the frames touched.

) If/when I access vfx or source frames that were pushed out of the cache, rather than having to re-render those frames from scratch, it just has to load them from the HDD - so final renders will go quicker on large projects

Yes, there are access patterns where this will not be the best solution - it is ideally suited to where you tend to focus on subsets of your material for a time, then move to a different section, and you have limited budget for lots of fast NVMe cache. But you'll never have to manually manage cache again, NOR re-render if you overflow your cache. Now a small NVMe could appear "infinite" (just with 2 different performance levels LOL).

Open to thoughts/ideas/suggestions/good war stories!


NOTES:
Example of PrimoCache in use & performance at LinusTechTips

PrimoCache lets you set up multiple caches, so each can have a different size limit, read/write policy, backing store, RAM cache, etc. Please RTM if you are considering this scenario.

If you use "Defer-Write" with a Level-1 RAM cache, it will also reduce wear on your SSD's, since it will collect writes until they're flushed to the SSD or HDD. Note: only use this for temp data (i.e. render cache, opt. media, etc., NOT source), as it is lost if the system freezes or you lose power before it flushes the contents to SSD/HDD.