Need advice for drive configuration

Do you have questions about Desktop Video, Converters, Routers and Monitoring?
  • Author
  • Message
Offline

Patrick Abde

  • Posts: 117
  • Joined: Thu Feb 26, 2015 12:56 am

Need advice for drive configuration

PostSun Aug 01, 2021 12:34 pm

I recently got myself a prebuilt HP gaming desktop. I wanna set it up for resolve the best way possible. It's a little limited though, I only have 1x m.2 slot available, and 2x sata ports.
At the moment, there's only a 256gb NVMe installed in the m.2 slot.
I want to upgrade, but which is this the right way to go about it?

#1
NVMe ssd- OS/software + cache/scratch
Sata ssd- project files
Sata hdd- storage

#2
NVMe ssd- OS/software
Sata ssd- project files + cache/scratch
Sata hdd- storage

#3
NVMe ssd- OS/software
Sata ssd- cache/scratch
Sata ssd- project files
+External hdd - storage

#4
NVMe ssd- cache/scratch
Sata ssd- OS/software
Sata ssd- project files
+External hdd - storage

Im completely lost. I've tried searching for solutions but I'm getting mixed info.
Appreciate any help i can get!
Offline
User avatar

Uli Plank

  • Posts: 21107
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: Need advice for drive configuration

PostSun Aug 01, 2021 5:46 pm

Which kind of media will you use? Which GPU is in that laptop? What will be your delivery codec?
Depending on these factors your setup of storage can have a lot or very little influence on the overall performance of the system.
Maybe AI can help you. Or make you obsolete.

Studio 18.6.5, MacOS 13.6.5
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G, iMac 2017, eGPU
Offline

Ellory Yu

  • Posts: 3929
  • Joined: Wed Jul 30, 2014 5:25 pm

Re: Need advice for drive configuration

PostMon Aug 02, 2021 12:43 am

My setup is OP’s #3 but with a Raid 0 SSD for cache and Raid for my HDD.
URSA Mini Pro 4.6K G2, Blackmagic Design Pocket Cinema Camera 6K, Panasonic GH5
PC Workstation Core I7 64Gb, 2 x AMD R9 390X 8Gb, Blackmagic Design DeckLink 4K Mini Monitor, Windows 10 Pro 64-bit, Resolve Studio 18, BM Micro Panel & Speed Editor
Offline

Patrick Abde

  • Posts: 117
  • Joined: Thu Feb 26, 2015 12:56 am

Re: Need advice for drive configuration

PostMon Aug 02, 2021 2:16 am

Uli Plank wrote:Which kind of media will you use? Which GPU is in that laptop? What will be your delivery codec?
Depending on these factors your setup of storage can have a lot or very little influence on the overall performance of the system.


As for now, it's mainly 4k, h264 (gh5) footage. I deliver both in mp4's and DNxHR's. I typically have around 750GB of "raw"-footage for each project.

System:
NVIDIA GeForce RTX 3060 12 GB
AMD Ryzen™ 5 5600G
16 GB RAM, 256 GB M.2 NVMe SSD

Full specs here:
https://support.hp.com/se-sv/document/c07636467

Image

I guess i'll probably upgrade the RAM too eventually, but for now, I just need to get the drive configuration sorted so I can get going with the edits. The computer is brand new. I haven't had any time with it yet, apart from the initial windows setup, so I have no clue of how it handles any of my stuff yet. Eager to try it out though!
Offline

Patrick Abde

  • Posts: 117
  • Joined: Thu Feb 26, 2015 12:56 am

Re: Need advice for drive configuration

PostMon Aug 02, 2021 2:34 am

Ellory Yu wrote:My setup is OP’s #3 but with a Raid 0 SSD for cache and Raid for my HDD.


Okay. Having one of the SATA ssd's dedicated for project files would leave me with just one free sata slot for cache. I'm not too familiar with raids and the different raid types yet, but I would need more SATA slots to run any of those, right?

Or do you mean that I should go with option #3 anyway? Even without the raid thing?
Offline

Ellory Yu

  • Posts: 3929
  • Joined: Wed Jul 30, 2014 5:25 pm

Re: Need advice for drive configuration

PostMon Aug 02, 2021 6:35 am

Patrick Abde wrote:
Ellory Yu wrote:My setup is OP’s #3 but with a Raid 0 SSD for cache and Raid for my HDD.


Okay. Having one of the SATA ssd's dedicated for project files would leave me with just one free sata slot for cache. I'm not too familiar with raids and the different raid types yet, but I would need more SATA slots to run any of those, right?

Or do you mean that I should go with option #3 anyway? Even without the raid thing?


For me, I did option 3 and it does well for my workflow and projects. You have to decide if that’s what will work for you. I strongly suggest the RAID but surely you can do without.

I will assume when you are talking about project files, you mean the Resolve local database and exported DRPs. If that is the case, I would up the NVMe to a 1 or 2TB M.2 drive and leave the database local on that drive. Da Vinci Project files are small but frequently accessed so good to put it on a fast drive like NVMe. I am assuming also you are not doing Posgres DB. That would be a different set up. Then you can use both SATA to have 2 x 2TB ssd drives set as RAID 0. You can use that for you cache, about 4TB - which will be plenty to optimize media’s for better playbacks. For working projects, I keep them on my RAID 10 HDD, and render them out there.
I keep a project structure where each project has a Folder, and each folder has sub folders for the project assets:CLIP\VIDEO, CLIP\AUDIO, EDIT (contains versions of exported DRP files), RENDER, FX, MISC…
I have also removable drive bays because some clients bring SSD and HDD but this is another topic and configuration. BMD sells a multi dock but I use just a 4 bay removable enclosure and it is a lot cheaper and works for me. I also used an M-disc drive to archive DRAs which for 90% of the project I do, I can compress the DRA and write it onto a 100gb M-disc. Others have their methods like using LTOs but they are pretty expensive for my use.

Just remember that when you are setting up to have a performant system, how the drives are setup can be your weakest link too.
Last edited by Ellory Yu on Tue Aug 03, 2021 6:57 pm, edited 1 time in total.
URSA Mini Pro 4.6K G2, Blackmagic Design Pocket Cinema Camera 6K, Panasonic GH5
PC Workstation Core I7 64Gb, 2 x AMD R9 390X 8Gb, Blackmagic Design DeckLink 4K Mini Monitor, Windows 10 Pro 64-bit, Resolve Studio 18, BM Micro Panel & Speed Editor
Offline

Patrick Abde

  • Posts: 117
  • Joined: Thu Feb 26, 2015 12:56 am

Re: Need advice for drive configuration

PostMon Aug 02, 2021 2:44 pm

Ellory Yu wrote:
Patrick Abde wrote:
Ellory Yu wrote:My setup is OP’s #3 but with a Raid 0 SSD for cache and Raid for my HDD.


Okay. Having one of the SATA ssd's dedicated for project files would leave me with just one free sata slot for cache. I'm not too familiar with raids and the different raid types yet, but I would need more SATA slots to run any of those, right?

Or do you mean that I should go with option #3 anyway? Even without the raid thing?


For me, I did option 3 and it does well for my workflow and projects. You have to decide if that’s what will work for you. I strongly suggest the RAID but surely you can do without.

I will assume when you are talking about project files, you mean the Resolve local database and exported DRPs. If that is the case, I would up the NVMe to a 1 or 2TB M.2 drive and leave the database local on that drive. Da Vinci Project files are small but frequently accessed so good to put it on a fast drive like NVMe. I am assuming also you are not doing Posgres DB. That would be a different set up. Then you can use both SATA to have 2 x 2TB ssd drives set as RAID 0. You can use that for you cache, about 4TB - which will be plenty to optimize media’s for better playbacks. For working projects, I keep them on my RAID 10 HDD, and render them out there.
I keep a project structure where each project has a Folder, and each folder has sub folders for the project assets:CLIP\VIDEO, CLIP\AUDIO, EDIT (contains versions of exported DRP files), RENDER, FX, MISC…
I have also removable drive bays because some clients bring SSD and HDD but this is another topic and configuration. BMD sells a multi dock but I use just a 4 bay removable enclosure and it is a lot cheaper and works for me. I also used an M-disc drive to archive DRAs which for 90% of the project I do, I can compress the DRA and write it onto a 100gb M-disc. Others have their methods like using LROs but they are pretty expensive for my use.

Just remember that when you are setting up to have a performant system, how the drives are setup can be your weakest link too.


Oh, I realize now that I may have been a bit unclear. What I meant by "Project Files" was: All the source media for the ongoing project/projects. Those would only live on that SSD untill I finished the project and then be moved over to a much larger disk (possibly externally, and I guess in some sort of raid config).

At least that's how I interpered the hardware recommendations from reading https://www.pugetsystems.com/recommended/Recommended-Systems-for-DaVinci-Resolve-187/Hardware-Recommendations#storage
They use the wording "Project Files", which I assumed was all the media files(raw footage/audio etc..), but now I'm unsure, is that not what they mean?

Anyway, with that in mind, my idea was to;
start a new project by creating a new folder inside the "Project Files"-drive and naming it for example: "Ben & Jerry's wedding", and then neatly fill it up with all media needed.

Option #3 for example, would look something like this:
Screenshot_20210802-163555__01.jpg
Screenshot_20210802-163555__01.jpg (131.52 KiB) Viewed 2558 times


Whenever I'm done with all my deliveries and everything, I'd export a .drp file and place it inside the "Ben & Jerry's wedding"-folder.
I'll then copy that whole folder over to my Storage drive (external in this case), to sit together with all the other old projects. All of that storage would also be backed up somewhere.
When all is done, I'd delete the whole Ben & Jerry folder from the "Project Files"-drive. And also, i guess, everything that's been generated to the "Cache/Scratch"-drive, to free up space for the next project/projects in line.

Btw, one other thing I'm unsure of is, where do I render my finished video files to? Should i render them directly into the Ben & Jerry folder like I illustrated, or to the scratch disk and then move it over when its done? Or something else? Is one way faster than the other?

I don't know, perhaps I'm completely off with everything?? :lol:
Btw, if you're still here, thanks for reading! I'm very sorry for the massive post :roll:
Offline
User avatar

Uli Plank

  • Posts: 21107
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: Need advice for drive configuration

PostMon Aug 02, 2021 4:12 pm

1. Project files are pretty small, they are in the data-base and that should be on a drive that is fast and permanently connected to the machine. Of course, you'll want to make backups of these regularly. These will be the .drp files and can be used to reconstruct your work in case of a full crash.

2. Cache, proxies and/or optimized media should be on a very fast drive that is not identical to the system drive. They can be re-generated at any time if things go wrong.

3. Your media files would not need very high throughput if you are working in H.264/265, but they should be on a drive with very good access times. There should be a backup, or rather two, of any of these on physically separated media.

As a minimum configuration, your system, programs and data-base could be on one internal drive. The files for number 2 should be on a separate internal drive. Finally, your media files can be on an external drive with a fast connection. A RAID is not needed as long as you don't have files which need fast throughput, like intermediate (converted) clips with high resolution and frame rate.
Maybe AI can help you. Or make you obsolete.

Studio 18.6.5, MacOS 13.6.5
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G, iMac 2017, eGPU
Offline

Patrick Abde

  • Posts: 117
  • Joined: Thu Feb 26, 2015 12:56 am

Re: Need advice for drive configuration

PostMon Aug 02, 2021 5:46 pm

Uli Plank wrote:1. Project files are pretty small, they are in the data-base and that should be on a drive that is fast and permanently connected to the machine. Of course, you'll want to make backups of these regularly. These will be the .drp files and can be used to reconstruct your work in case of a full crash.

2. Cache, proxies and/or optimized media should be on a very fast drive that is not identical to the system drive. They can be re-generated at any time if things go wrong.

3. Your media files would not need very high throughput if you are working in H.264/265, but they should be on a drive with very good access times. There should be a backup, or rather two, of any of these on physically separated media.

As a minimum configuration, your system, programs and data-base could be on one internal drive. The files for number 2 should be on a separate internal drive. Finally, your media files can be on an external drive with a fast connection. A RAID is not needed as long as you don't have files which need fast throughput, like intermediate (converted) clips with high resolution and frame rate.


Okay. I'm a little confused. Are you suggesting #4? Or perhaps a fifth alternative? Let's say:

#5
NVMe ssd- cache/scratch
Sata ssd- OS/software
+External HDD with fast connection ?

Fastest connection I have would be USB 3.2 Gen 3. Is that good enough? This, option #5 leaves us with a free sata port though. Shouldn't i take advantage of it?

Btw, this is taken from http://www.pugetsysteams.com

Screenshot_20210802-191936__01.jpg
Screenshot_20210802-191936__01.jpg (447.62 KiB) Viewed 2534 times


Are they really talking about database and drp's when they mention the second drive? Would that really need its own drive?

Also this:
Screenshot_20210802-230957__01.jpg
Screenshot_20210802-230957__01.jpg (160.19 KiB) Viewed 2508 times

I feel like I'm getting mixed info still and it's very unclear to me what I should do.
Last edited by Patrick Abde on Mon Aug 02, 2021 10:42 pm, edited 1 time in total.
Offline
User avatar

Uli Plank

  • Posts: 21107
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: Need advice for drive configuration

PostMon Aug 02, 2021 10:00 pm

Is your footage high bitrate RAW footage? That's what Puget Systems are referring in the text.
Maybe AI can help you. Or make you obsolete.

Studio 18.6.5, MacOS 13.6.5
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G, iMac 2017, eGPU
Offline

Patrick Abde

  • Posts: 117
  • Joined: Thu Feb 26, 2015 12:56 am

Re: Need advice for drive configuration

PostMon Aug 02, 2021 10:27 pm

Uli Plank wrote:Is your footage high bitrate RAW footage? That's what Puget Systems are referring in the text.


No. It's not raw. I work mostly with h.264, so far.

But I'm still reading it as "secondary drive: preferably at least SSD, for media files and cache. (May benefit a lot from an NVMe if the media on it is high bitrate raw footage)"

And I read the third, optional one as "preferably at least SSD, for separating the cache from the secondary drive"

And the fourth one as "external cheap and large enough storage to store all your ****"

Am i reading it wrong?
Offline

Ellory Yu

  • Posts: 3929
  • Joined: Wed Jul 30, 2014 5:25 pm

Re: Need advice for drive configuration

PostTue Aug 03, 2021 6:11 am

Patrick, Uli and I are pretty much saying the same thing. Bottom line (1) always put your cache/scratch in a separate and fast drives; put OS and app on the same drive is fine and again, a fast drive only for that purpose. Project files are Resolve files stored either on a local database or a Postgres database. For your implementation I think the local database is fine. This local database can be on the same drive as the Resolve app, which is the default. You can put all your media on a SDD or HDD, but it should be it’s own drive. Often times, like in my workflow, these media’s are in large drives that is set up as RAID. I can also output the rendered picture to that drive.

So in your varying option, IMO, you can go with #3 or #4, or some variation of #3 & #4. I forgot to mention this before but on my setup the sata 1 (OS) and sata 2 (cache) are on sata card 1 and sata 3 (media drive), 4,5,6 (removable bay 0,1,2), 7 (optical M-disc). This ensures my OS, app, database, and cache are on one I/O bus and the others are on another I/O bus for performance reasons.

I hope this is clearer and from that of Uli as well. Good luck.
Last edited by Ellory Yu on Tue Aug 03, 2021 6:59 pm, edited 1 time in total.
URSA Mini Pro 4.6K G2, Blackmagic Design Pocket Cinema Camera 6K, Panasonic GH5
PC Workstation Core I7 64Gb, 2 x AMD R9 390X 8Gb, Blackmagic Design DeckLink 4K Mini Monitor, Windows 10 Pro 64-bit, Resolve Studio 18, BM Micro Panel & Speed Editor
Offline
User avatar

Uli Plank

  • Posts: 21107
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: Need advice for drive configuration

PostTue Aug 03, 2021 6:18 am

No, you are not reading it wrong, but it clearly says that an NVMe for footage is not needed if you have low bitrate, as in your case. Even a RAID is not needed for such footage, but can be desirable for intermediate formats.

Access times are important, but these are pretty good on SSD too, lightyears ahead of spinning platters. And, right, a separate drive for cache is a good thing. For that, an NVMe is desirable, since you could cache in a high bitrate format. But it can be much smaller than your media drive if you don't work with many projects at the same time.

The rest is a question of your way to organize things, which is an individual decision, not a technical must.
Maybe AI can help you. Or make you obsolete.

Studio 18.6.5, MacOS 13.6.5
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G, iMac 2017, eGPU
Offline

Patrick Abde

  • Posts: 117
  • Joined: Thu Feb 26, 2015 12:56 am

Re: Need advice for drive configuration

PostTue Aug 03, 2021 9:31 am

Alright then, things are clearer now! I appreciate you guys taking your time to elaborate, I really do!

I think this is what I'll do now for now:
Keep the original 256GB NVMe as my primary drive for os/software. Then get a new 2TB SSD as secondary drive for both the media and the cache (for now).

I'll try finishing just one whole project like that to evaluate and get an idea of how large of a drive I'd need for all the cache/optimized media etc.
I'll later then add a third drive for that (aka scratch disk).

My project media are usually between 500 and 700GB but I picked a 2TB drive so that I can fit two big projects, or even maybe three at a time. I usually don't work like that, but just in case, and also for headroom, other media files, potential camera upgrade etc.

It would then end up looking something like this:
1. Internal 256GB m.2 NVMe - OS/Software
2. Internal 2TB SATA SSD - Project Media
3. internal 1TB(?) SATA SSD - Cache
4. Some sort of external storage + backup solution.

So option #3!

What do you think, do you approve? Yes to the dress? :D
Offline
User avatar

Uli Plank

  • Posts: 21107
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: Need advice for drive configuration

PostTue Aug 03, 2021 10:33 am

Sounds fine to me.
Maybe AI can help you. Or make you obsolete.

Studio 18.6.5, MacOS 13.6.5
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G, iMac 2017, eGPU
Offline

Ellory Yu

  • Posts: 3929
  • Joined: Wed Jul 30, 2014 5:25 pm

Re: Need advice for drive configuration

PostTue Aug 03, 2021 7:03 pm

Patrick Abde wrote:Alright then, things are clearer now! I appreciate you guys taking your time to elaborate, I really do!

I think this is what I'll do now for now:
Keep the original 256GB NVMe as my primary drive for os/software. Then get a new 2TB SSD as secondary drive for both the media and the cache (for now).

I'll try finishing just one whole project like that to evaluate and get an idea of how large of a drive I'd need for all the cache/optimized media etc.
I'll later then add a third drive for that (aka scratch disk).

My project media are usually between 500 and 700GB but I picked a 2TB drive so that I can fit two big projects, or even maybe three at a time. I usually don't work like that, but just in case, and also for headroom, other media files, potential camera upgrade etc.

It would then end up looking something like this:
1. Internal 256GB m.2 NVMe - OS/Software
2. Internal 2TB SATA SSD - Project Media
3. internal 1TB(?) SATA SSD - Cache
4. Some sort of external storage + backup solution.

So option #3!

What do you think, do you approve? Yes to the dress? :D


Sounds like a good plan.
URSA Mini Pro 4.6K G2, Blackmagic Design Pocket Cinema Camera 6K, Panasonic GH5
PC Workstation Core I7 64Gb, 2 x AMD R9 390X 8Gb, Blackmagic Design DeckLink 4K Mini Monitor, Windows 10 Pro 64-bit, Resolve Studio 18, BM Micro Panel & Speed Editor

Return to Post Production

Who is online

Users browsing this forum: No registered users and 17 guests