Best Hard Drive For Raw Footage

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

Matt Thela

  • Posts: 97
  • Joined: Fri Aug 25, 2017 4:36 pm

Best Hard Drive For Raw Footage

PostThu Jun 13, 2019 9:47 pm

Hi I'm looking to buy a new Hard Drive that will contain the footage that I am working on for my current project. (I already have ones to store all my raw projects and one to export the finished clips to.)

Each project is a mixture of CDNG and Braw Clips, so some of the files are around 500 mb/second


What Max Data Transfer Rate Drives should I be looking for? Is 500-700 mb/second ok? or should I go for the gbs/second range?

I'm having trouble with my Playback in Resolve stuttering on my clips. The cpu, Ram and GPU are all below 50% when this happens but my current drive is always around 100% when this the slow playback happens!


I currently have Thunderbolt 3 connectivity.
Offline

Michael_Andreas

  • Posts: 1672
  • Joined: Sat Jan 05, 2019 9:40 pm
  • Real Name: Michael Andreas

Re: Best Hard Drive For Raw Footage

PostThu Jun 13, 2019 9:57 pm

With a spinning hard drive, you will encounter delays when the hard drive has to step the head to acquire a new clip. Recommend you look into getting an SSD for holding the media files, and then archive to hard drive when the project is complete. You also should look at the use of proxy files.
_________________________________________________
DR Studio 17.4.1 Win10Pro 21H1/19043.1320 - i7-6700K@4GHz, 32GB RAM
RTX 2070 8GB, "Studio" driver 472.39
OS,Library: 1TB SSD - Project: 1TB SSD - Cache: 1TB NVMe
Offline
User avatar

Marc Wielage

  • Posts: 11052
  • Joined: Fri Oct 18, 2013 2:46 am
  • Location: Hollywood, USA

Re: Best Hard Drive For Raw Footage

PostThu Jun 13, 2019 10:04 pm

If you're just editing and viewing, a Proxy workflow (or Transcodes) can work just fine. Just be sure that the new file names are identical to the camera source file names.

If you need to access the original source files, fast RAID drives (like a RAID5 that can do 1GB/s) is highly recommended. I'm a big fan of the G-Speed Shuttles, but if you need shared storage for a large office, then you'd need something like QNAP or Lumaforge:

http://www.accusys.com
https://www.avid.com/products/avid-nexis-pro
http://www.facilis.com
http://www.lumaforge.com
https://opendrives.com
http://promax.com/shared-storage/
https://www.qnap.com/en-us/
http://www.studionetworksolutions.com/
https://www.synology.com/en-us
marc wielage, csi • VP/color & workflow • chroma | hollywood
Offline

MishaEngel

  • Posts: 1432
  • Joined: Wed Aug 29, 2018 12:18 am
  • Real Name: Misha Engel

Re: Best Hard Drive For Raw Footage

PostThu Jun 13, 2019 10:52 pm

These QNAP are flexible:

https://www.qnap.com/en-us/product/?conditions=2-8

Go atleast for RAID 6 (Raid 5 is not secure anymore).
Offline

Matt Thela

  • Posts: 97
  • Joined: Fri Aug 25, 2017 4:36 pm

Re: Best Hard Drive For Raw Footage

PostThu Jun 13, 2019 11:24 pm

Thanks for the help guys!


I just realized I have 3 500 mb/second 1 TB SSD drives that I no longer am using to regularly. Would I able to make them into a Raid 0 or other Raid array?
Offline

MishaEngel

  • Posts: 1432
  • Joined: Wed Aug 29, 2018 12:18 am
  • Real Name: Misha Engel

Re: Best Hard Drive For Raw Footage

PostFri Jun 14, 2019 12:00 am

Matt Thela wrote:Thanks for the help guys!


I just realized I have 3 500 mb/second 1 TB SSD drives that I no longer am using to regularly. Would I able to make them into a Raid 0 or other Raid array?


Raid 0 is good as scratch drive, when you loose some data it's just some.
Raid 6 is for storage of important data (originals, final renders etc...)

Read this for the possibilities https://www.qnap.com/en-us/product/tvs-1282t3

It's not cheap but it works.
Offline
User avatar

Marc Wielage

  • Posts: 11052
  • Joined: Fri Oct 18, 2013 2:46 am
  • Location: Hollywood, USA

Re: Best Hard Drive For Raw Footage

PostFri Jun 14, 2019 1:29 am

RAID0's work fine for source drives in final color, but I wouldn't rely on them as my only storage. If a RAID0 crashes, it's pretty much gone. But a RAID0 -- particularly an SSD -- is unbelievably fast.
marc wielage, csi • VP/color & workflow • chroma | hollywood
Offline
User avatar

waltervolpatto

  • Posts: 10536
  • Joined: Thu Feb 07, 2013 5:07 pm
  • Location: 1146 North Las Palmas Ave. Hollywood, California 90038 USA

Re: Best Hard Drive For Raw Footage

PostFri Jun 14, 2019 1:58 pm

Get 4-5 ssd and build a raid 5/6, you have storage and speed.

Me personally I found for dirt cheap a SAS enclosure and I put 8x8tb spinning disk inside: 45 TB of space at reasonable speed (above 2k uncompressed)
W10-19043.1645- Supermicro MB C9X299-PGF - RAM 128GB CPU i9-10980XE 16c 4.3GHz (Oc) Water cooled
Decklink Studio 4K (12.3)
Resolve 18.5.1 / fusion studio 18
GPU 3090ti drivers 512.59 studio
Offline

Matt Thela

  • Posts: 97
  • Joined: Fri Aug 25, 2017 4:36 pm

Re: Best Hard Drive For Raw Footage

PostFri Jun 14, 2019 5:02 pm

Thanks for all the help guys!


This morning I turned my 3 SSD drives into a Raid 0 on windows and tried that.

At first Resolve was running smoothly but after playing through a few clips adjusting the speed of a clip, the playback started to lag again.


I used Blackmagic's Drive Speed Tool and its showing a Read and Write speed of this Raid 0 as about 500 mb/second (Which is the speed of each individual three drives)

Shouldn't there be a read and write speed boost after combining the three together? I read somewhere that a Raid 0 can double the speed of what one of the individual drives would do. (I was expecting more than a GB/second speed)
Offline

MishaEngel

  • Posts: 1432
  • Joined: Wed Aug 29, 2018 12:18 am
  • Real Name: Misha Engel

Re: Best Hard Drive For Raw Footage

PostFri Jun 14, 2019 7:28 pm

Matt Thela wrote:Thanks for all the help guys!


This morning I turned my 3 SSD drives into a Raid 0 on windows and tried that.

At first Resolve was running smoothly but after playing through a few clips adjusting the speed of a clip, the playback started to lag again.


I used Blackmagic's Drive Speed Tool and its showing a Read and Write speed of this Raid 0 as about 500 mb/second (Which is the speed of each individual three drives)

Shouldn't there be a read and write speed boost after combining the three together? I read somewhere that a Raid 0 can double the speed of what one of the individual drives would do. (I was expecting more than a GB/second speed)


500 mb/s is really slow, only about 62.5 MByte/s.
Offline

Michael_Andreas

  • Posts: 1672
  • Joined: Sat Jan 05, 2019 9:40 pm
  • Real Name: Michael Andreas

Re: Best Hard Drive For Raw Footage

PostFri Jun 14, 2019 8:32 pm

Let's be careful here whether we're talking bits or bytes here. I found an SSD rated at 500 MBps, upper case "B" is bytes with an interface speed of 6 Gbps, lower case "b" is bits. For rough order of magnitude, I divide bits/sec by 10 to get Bytes/sec assuming there will be some over head. So 6 Gbps ~ 600 MBps, which is not much more than 500 MBps. To achieve maximum speed from a RAID0 would require a controller that buffers the data from the RAID stripes and provides it a full speed.

What is the actual data rate of your files? How many tracks are you running? Have you looked into proxies?
_________________________________________________
DR Studio 17.4.1 Win10Pro 21H1/19043.1320 - i7-6700K@4GHz, 32GB RAM
RTX 2070 8GB, "Studio" driver 472.39
OS,Library: 1TB SSD - Project: 1TB SSD - Cache: 1TB NVMe
Offline

Matt Thela

  • Posts: 97
  • Joined: Fri Aug 25, 2017 4:36 pm

Re: Best Hard Drive For Raw Footage

PostFri Jun 14, 2019 9:29 pm

Thanks for the replies!

All three SSDs are Samsung 850 Pros (1 Tb each)

Each are rated at 550 MB/second. But 6Gb/second (I didn’t know the lowercase was a different speed rate)

I’m currently editing 4.6K clips shot at 29 FPS

What type of Raid controller do you reccomend installing for these Drives?


Also sorry for the stupid question, but is there a way to make proxy files directly in Resolve? I’ve never used them before.
Offline

Michael_Andreas

  • Posts: 1672
  • Joined: Sat Jan 05, 2019 9:40 pm
  • Real Name: Michael Andreas

Re: Best Hard Drive For Raw Footage

PostFri Jun 14, 2019 10:08 pm

Avail yourself of the documentation accessible under the Help pulldown menu in DR, the Reference Manual and the training manuals. Work through the Definitive Guide and Advanced Editing.

One thing you should do is under the DaVinci Resolve pulldown menu, Preferences, Media Storage make sure that the directories there point to one of your SSDs rather than your C: drive if that's a spinning hard drive. Proxy media would be created in these locations.
_________________________________________________
DR Studio 17.4.1 Win10Pro 21H1/19043.1320 - i7-6700K@4GHz, 32GB RAM
RTX 2070 8GB, "Studio" driver 472.39
OS,Library: 1TB SSD - Project: 1TB SSD - Cache: 1TB NVMe
Offline

Ole Kristiansen

  • Posts: 284
  • Joined: Tue Nov 28, 2017 8:37 pm

Re: Best Hard Drive For Raw Footage

PostSat Jun 15, 2019 8:39 am

Matt Thela wrote:Thanks for the replies!

All three SSDs are Samsung 850 Pros (1 Tb each)

Each are rated at 550 MB/second. But 6Gb/second (I didn’t know the lowercase was a different speed rate)

I’m currently editing 4.6K clips shot at 29 FPS

What type of Raid controller do you reccomend installing for these Drives?


Also sorry for the stupid question, but is there a way to make proxy files directly in Resolve? I’ve never used them before.


Hi

I have 3 pieces of Samsung 850 EVO 500GB SSD set up for software Raid 0 in Windows 10 Pro -
here you can see the speed measured with Blackmagic Disk Speed ​​Test! My C drive is a
Samsung 970 EVO NVMe M.2 pci x4 - speed is measured with CrystalDiskMark !

"Also sorry for the stupid question, but is there a way to make proxy files directly in Resolve? I’ve never used them before."

At one point I made this short video on how to transcode into Media Management in Davinci Resolve!

Davinci Resolve Transcode

Attachments
samsungnvme.jpg
samsungnvme.jpg (411.67 KiB) Viewed 5807 times
Offline

Andrew Kolakowski

  • Posts: 9212
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: Best Hard Drive For Raw Footage

PostSat Jun 15, 2019 10:52 am

Michael_Andreas wrote:...To achieve maximum speed from a RAID0 would require a controller that buffers the data from the RAID stripes and provides it a full speed.

What is the actual data rate of your files? How many tracks are you running? Have you looked into proxies?


This is not really correct.
If you put 3 of those drives in RAID0 you will have 1.5GB/sec. Each drive will do its 500MB/sec and in this case SATA6 (6Gb/sec) connection is not a limiting factor (there is no need for any buffering). What you don't want to do is to connect eg. NVMe drive (which can d eg. 2.5GB/sec) over some "case" which is eg. SATA6. Then you massively limit speed of your drive. If you have motherboard with many SATA6 connectors you can connect SSDs and make software RAID0. Speed will scale linearly with number of drives up to some point (eg. 4GB/sec). Those SATA6 ports will be connected to some PCI-E bus and depending on motherboard you can hit limit earlier or later.
You can also get RAID controller and connect your drives there.
Most good RAID controllers are PCI-E x8 (so around 8 GB/sec). If you want RAID5/6 then speed is limited due to needed calculations (eg. R=6GB/sec, W=4GB/sec). For higher speed you need more RAID controllers or NVME controller, which is still bit new and expensive (some motherboards allow you to do NVME RAID0 but typically for limited number of drives). Enterprise all flash boxes can do eg. 18GB/sec aggregated speed, but those are very expensive solutions.

To answer original question: all depends on budget. RAID type is up to you. If you keep original source backed up I would do simple RAID0. Get many HDD drives (5+) in RAID, or few SSD (eg. 2+) or single NVME drive if you do small projects. You don't need to over-complicate things. cDNG or BRAW does not need crazy speeds.
Last edited by Andrew Kolakowski on Sat Jun 15, 2019 1:54 pm, edited 1 time in total.
Offline

Jim Simon

  • Posts: 30311
  • Joined: Fri Dec 23, 2016 1:47 am

Re: Best Hard Drive For Raw Footage

PostSat Jun 15, 2019 1:52 pm

I'm using a 12TB Seagate as my Media drive. No problems even with multicam projects in cDNG.
My Biases:

You NEED training.
You NEED a desktop.
You NEED a calibrated (non-computer) display.
Offline

Matt Thela

  • Posts: 97
  • Joined: Fri Aug 25, 2017 4:36 pm

Re: Best Hard Drive For Raw Footage

PostSat Jun 15, 2019 5:08 pm

Ole Kristiansen

I have 3 pieces of Samsung 850 EVO 500GB SSD set up for software Raid 0 in Windows 10 Pro -
here you can see the speed measured with Blackmagic Disk Speed ​​Test! My C drive is a
Samsung 970 EVO NVMe M.2 pci x4 - speed is measured with CrystalDiskMark !


That’s awesome! What kind of raid software do you use for your software Raid 0? I used the one that came with Windows but only got 500MB/second speed.
Offline

Ole Kristiansen

  • Posts: 284
  • Joined: Tue Nov 28, 2017 8:37 pm

Re: Best Hard Drive For Raw Footage

PostSat Jun 15, 2019 5:50 pm

Hi Matt

I just create Raid 0 in windows 10 Pro like this !


Return to DaVinci Resolve

Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Robert Niessner, W3C [Validator], Yahoo [Bot] and 210 guests