Optimized media and how it's used

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

Andrew Prince

  • Posts: 18
  • Joined: Mon Nov 02, 2015 12:39 am

Optimized media and how it's used

PostTue May 05, 2020 4:22 pm

Hi all,

Just a quick question....

I have a multicam project, all DNxHD. All my media files are on a RAID 0 array. I still cannot get enough throughput on that array to edit in real time, so I've generated optimized media for all my MXF clips. The optimized media is on an M2 drive that's fast as you like.

When I'm in Resolve, doing absolutely nothing except admiring my timeline, I can see in Task Manager that the RAID array is being hammered. When I look in Resource Monitor, I can see that Resolve is actually hammering the original media on the RAID array. I definitely ticked the "use optimized media" option in the playback menu.

So my really quick question is.... why? Why does Resolve need to even look at my original media drive? It doesn't contain any media that it should be using right?

In the screen shot below, you can see drive F: (RAID array), drive C: (system) and drive D: (M2 with optimised media).

Resolve 15.3.1 RAID being hammered.png
Resolve 15.3.1 RAID being hammered.png (307.25 KiB) Viewed 495 times
Kind regards, Andrew
www.ampromedia.co.uk
www.carillonvideo.co.uk
www.mama-genesis.co.uk
Offline

Ellory Yu

  • Posts: 1272
  • Joined: Wed Jul 30, 2014 5:25 pm
  • Warnings: 1

Re: Optimized media and how it's used

PostTue May 05, 2020 4:55 pm

A few suggestions that may help.

1) In your Project Master settings, I would set the Optimized Media Format and Render cache format to the lightest possible codec like DNxHR SQ or DNxHR LB, and check Enable background caching. I would also check all auto caching of transition, etc. and you can try also setting the optimize resolution to "Half".

2) Next, Generate Optimized Media for all the clips you want optimize. You can do all of them and that is up to you. Every time you change something in #1, you'll have to delete optimize media and do step #2 again.

3) You already said that you have checked "Use Optimized Media if Available". So you're good here.

4) For Render Cache, set it to User or Smart mode.
MSI Raider X99 MB / Intel i7 5280 6 core CPU / 2 x AMD R9 390X 8Gb GPU / 64Gb 3200 RAM / Decklink 4K Mini Monitor / 240Gb SSD OS Drive / 2Tb RAID 0 SSD Cache/Scratch Drive / 14Tb 7200rpm HDD Data Drive / Win 10 Pro 64bit / Resolve 16.2.1
Offline

Andrew Prince

  • Posts: 18
  • Joined: Mon Nov 02, 2015 12:39 am

Re: Optimized media and how it's used

PostTue May 05, 2020 5:38 pm

Thanks for your reply. :-)

I've done all those things prior to my post above. I also had smart caching set up. I'm just wondering why it's still hammering the RAID array. However, after thirty minutes or so, it stopped. So I think maybe it was indexing the files or something? It's still accessing those original files when I play the timeline, but the hard drive access isn't high. It's just spiking ever five seconds or so.

I think in future I'll dial back on the bitrate of DNxHD to start with. That'll help with throughput on my drives. I guess my RAID array performance just isn't as great as I thought it might be. The system is using the motherboard RAID controller and they're not exactly the best implimentation of RAID I've come across.

Thanks for your help. :-)
Kind regards, Andrew
www.ampromedia.co.uk
www.carillonvideo.co.uk
www.mama-genesis.co.uk
Offline

Jim Simon

  • Posts: 7361
  • Joined: Fri Dec 23, 2016 1:47 am

Re: Optimized media and how it's used

PostTue May 05, 2020 6:47 pm

You're not playing media in a timeline in the posted image. Do that, then look at Task Manager.
Offline

Andrew Prince

  • Posts: 18
  • Joined: Mon Nov 02, 2015 12:39 am

Re: Optimized media and how it's used

PostTue May 05, 2020 10:37 pm

Jim Simon wrote:You're not playing media in a timeline in the posted image. Do that, then look at Task Manager.


Correct. My point here is, I don't have to actually do anything in Resolve to hammer the drive that should not be hammered. If I play the timeline, it's just as bad.
Kind regards, Andrew
www.ampromedia.co.uk
www.carillonvideo.co.uk
www.mama-genesis.co.uk
Offline

Andrew Kolakowski

  • Posts: 6512
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Optimized media and how it's used

PostTue May 05, 2020 10:44 pm

Have all your files optimised media been already generated? If you have many long files it can take hours, eg. whole night.
Offline

Andrew Prince

  • Posts: 18
  • Joined: Mon Nov 02, 2015 12:39 am

Re: Optimized media and how it's used

PostTue May 05, 2020 10:54 pm

Andrew Kolakowski wrote:Have all your files optimised media been already generated? If you have many long files it can take hours, eg. whole night.


Yes, I think so. I followed the recommendation in the reference manual to set up a smart bin to show me all the files that were optimised. There were none that showed as not optimised.
Kind regards, Andrew
www.ampromedia.co.uk
www.carillonvideo.co.uk
www.mama-genesis.co.uk
Offline

Andrew Kolakowski

  • Posts: 6512
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Optimized media and how it's used

PostWed May 06, 2020 8:56 am

And it's still doing it?
If dos then this is some problem between Resolve and RAID. Update drivers for RAID.
So activity is 100% and moment you close Resolve it goes down?
Offline

Andrew Prince

  • Posts: 18
  • Joined: Mon Nov 02, 2015 12:39 am

Re: Optimized media and how it's used

PostWed May 06, 2020 12:29 pm

Andrew Kolakowski wrote:And it's still doing it?


Yes. It did it for a good thirty minutes and then stopped. And on other occasions, if it's doing it, then it stops as soon as I exit Resolve.

Note that I'm running Resolve 15.3.1 and all my drivers are up to date.

I believe that it's indexing files and that's why it goes away after thirty minutes or so.
Kind regards, Andrew
www.ampromedia.co.uk
www.carillonvideo.co.uk
www.mama-genesis.co.uk
Offline

Jim Simon

  • Posts: 7361
  • Joined: Fri Dec 23, 2016 1:47 am

Re: Optimized media and how it's used

PostWed May 06, 2020 12:31 pm

Audio waveforms do get generated from the original media, even when Optimized Media is attached.

Once the activity stops, it doesn't return?
Offline

Andrew Prince

  • Posts: 18
  • Joined: Mon Nov 02, 2015 12:39 am

Re: Optimized media and how it's used

PostWed May 06, 2020 12:40 pm

Jim Simon wrote:Audio waveforms do get generated from the original media, even when Optimized Media is attached.


Ah right. That is a good answer! The optimised media is coming from the M2 drive, but the original MXF files will still get accessed to get the waveforms or maybe audio. That works for me.

Good thinking! :-)
Kind regards, Andrew
www.ampromedia.co.uk
www.carillonvideo.co.uk
www.mama-genesis.co.uk
Offline

Andrew Prince

  • Posts: 18
  • Joined: Mon Nov 02, 2015 12:39 am

Re: Optimized media and how it's used

PostWed May 06, 2020 12:41 pm

Jim Simon wrote:Once the activity stops, it doesn't return?


And yes... if the drive is being hammered without me doing anything, then it eventually stops and doesn't return. However, the RAID array DOES get access when I play a timeline again, but not heavily.
Kind regards, Andrew
www.ampromedia.co.uk
www.carillonvideo.co.uk
www.mama-genesis.co.uk
Offline

Andrew Prince

  • Posts: 18
  • Joined: Mon Nov 02, 2015 12:39 am

Re: Optimized media and how it's used

PostSat May 23, 2020 11:33 am

Okay, this problem came back when I upgraded to Resolve 16.2.2.011. I believe this is the latest version.

To better describe the issue, I've taken a screen capture video, as below.

The system specs are:

Intel i7-5820 @ 3.3GHz
16 Gb DDR4 RAM
Nvidia GTX 780 Ti
500 Gb SSD system drive (C)
500 Gb M2 cache drive (D)
3 x 3 Gb SATA3 media drives in RAID 0 (F)

I also installed a BMD MiniMonitor PCI card which drives a second monitor, but for this thread, the MiniMonitor is not configured in Resolve. I only tend to use it when grading.

Again, all the original media is on the media drive (F). All the optimised media is on the cache drive (D). It looks to me like Resolve is heavily utilising the original media, not the optimised media.

Anyone any ideas? Thanks in advance. :-)

Attachments
Resolve Disk Usage.png
Resolve Disk Usage.png (454 KiB) Viewed 83 times
Kind regards, Andrew
www.ampromedia.co.uk
www.carillonvideo.co.uk
www.mama-genesis.co.uk
Offline

Andrew Prince

  • Posts: 18
  • Joined: Mon Nov 02, 2015 12:39 am

Re: Optimized media and how it's used

PostSun May 24, 2020 3:45 am

Interesting.... I was doing some more testing with my project, Task Manager and Resource Monitor. The devil is in the detail. It seems there are two issues:

1. Resolve isn't using the optimised media.

I have since deleted it and will generate it again, just in case it lost it. But then I noticed:

2. Resource Monitor clearly shows that when Resolve is playing my timeline, the original media drive is being hammered. However, it appears that it's the SYSTEM process that is doing said hammering. That's weird right?

So, my next test is, I'm switching off "allow files on this drive to have contents indexed in addition to file properties" on my RAID / original media drive. My thinking is that Windows 10 is trying to get its head around those big MXF files on the RAID array, whenever Resolve is trying to use those files or maybe is pulling the audio from those files?

I'm not sure if Resolve uses the original media audio or audio it creates in the optimised media files, but that could perhaps explain the SYSTEM overhead on the original media files?
Kind regards, Andrew
www.ampromedia.co.uk
www.carillonvideo.co.uk
www.mama-genesis.co.uk
Offline

Andrew Prince

  • Posts: 18
  • Joined: Mon Nov 02, 2015 12:39 am

Re: Optimized media and how it's used

PostSun May 24, 2020 4:01 pm

Seriously, nobody has any idea what could be causing this issue?

Okay, I have some more info for you. I finally nailed it to something small:

1. I load the project.

The project has brand new optimised media and I even defragmented the RAID array, despite the fact it didn't need it. The timeline plays SLOW. Far too slow. It's a multicam edit timeline. Not sure if I mentioned this before? Four cameras. Resolve just cannot keep up, but this is clearly because the RAID array is being hammered (mostly by the SYSTEM process, it has to be said). That is to say, the original media on the RAID array is being hammered but the optimised media on the cache drive (D) is not being touched. Task Manager shows zero disk access, while playing the timeline, on the cache drive.

2. Untick "Use Optimised Media if Available".
3. Tick "Use Optimised Media if Available".
4. Play the multicam timeline.

It now plays fine and I can see that the disk access on the cache drive (where the optimised media lives) is bubbling away.

So... It's a bug in Resolve 16.2.2.011? Have I finally solved it?
Kind regards, Andrew
www.ampromedia.co.uk
www.carillonvideo.co.uk
www.mama-genesis.co.uk

Return to DaVinci Resolve

Who is online

Users browsing this forum: Brett Simms, Dermot Shane and 72 guests