Need guidance on storage solutions for video editing

Do you have questions about Desktop Video, Converters, Routers and Monitoring?
  • Author
  • Message
Offline

Abdelrahman Magdy

  • Posts: 249
  • Joined: Fri Jan 23, 2015 10:08 pm

Need guidance on storage solutions for video editing

PostSun Mar 04, 2018 4:15 pm

I work as a freelance video editor and I am now getting bigger projects which the I/O speeds of my internal and external USB hard disks are not able to handle very well, so I am looking to invest in a media storage solution that I can use for editing, but I am not sure which route to go.

There is the DAS option, but I am not sure if that would be enough as I work on PC and only have USB3 connections. Should I invest in a Thunderbolt PCI-E card. Would it improve the speed or not?

There is also the NAS option. Would be faster than a storage attached directly?

I have also read about SAN, but it seems like something that is more suitable for a bigger facility, with multiple users accessing the media. Is that actually the case?

Is there any other options I am missing?
System specs:
Resolve Studio version: 15.0.0.086
Fusion Studio version: 9.0.2
OS: CentOS 7
CPU: Intel i7-4790k 4GHz Quad-Core
RAM: 32GB
GPU: NVIDIA GeForce 1080 Ti 11GB
Offline

Glenn Sakatch

  • Posts: 262
  • Joined: Sat Apr 13, 2013 5:36 pm

Re: Need guidance on storage solutions for video editing

PostSun Mar 04, 2018 9:05 pm

I have a promise Pegasus thunderbolt 2 connected to my z820. Has worked great for years..but not all pcs can add a thunderbolt card.

I also. Just added a qnap 1282 into the mix, connected to the pic via a 10 g card. Been using it. For a month or so, and it is wonderful. Also have a 10g connection to an iMac, and it is a great solution for a small shared storage setup
Offline

EvanAnthony

  • Posts: 115
  • Joined: Sat Apr 11, 2015 6:19 pm

Re: Need guidance on storage solutions for video editing

PostMon Mar 05, 2018 10:04 pm

I also have promise Pegasus R6 thunderbolt 2 connected to my z840. Works great I also have a G-Raid. TB2 also works great.

How fast are you looking to go?
This is a solid price for USB3 and about 350-400Mbps
https://www.videoguys.com/shop/featured ... -16tb.html
Offline

Abdelrahman Magdy

  • Posts: 249
  • Joined: Fri Jan 23, 2015 10:08 pm

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 12:44 am

Thanks guys for your help. I am looking for the fastest I can afford.. don't have a budget at the moment to be honest, but I wanted to weigh the options

Problem is I am still trying to figure out if my motherboard will take a thunderbolt expansion card or not, but I can't find any information at the moment

In reality, is thunderbolt much faster than USB3. From what I read, TB2 is supposed to be about double the speed of USB3.0, but do you guys actually see that much difference between them.

Sent from my SM-G920F using Tapatalk
System specs:
Resolve Studio version: 15.0.0.086
Fusion Studio version: 9.0.2
OS: CentOS 7
CPU: Intel i7-4790k 4GHz Quad-Core
RAM: 32GB
GPU: NVIDIA GeForce 1080 Ti 11GB
Offline

Glenn Sakatch

  • Posts: 262
  • Joined: Sat Apr 13, 2013 5:36 pm

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 1:02 am

If you are talking about the HP thunderbolt card, it is designed to only work in certain models. It has a specific slot it goes in and a header cable that attaches to a specific spot on the motherboard. If you have found a generic thunderbolt card, I would be very suspicious.

Yes the thunderbolt connection is definitely faster. It all depends on the type of media you are working with as well. If you are transcoding to DNX then you don't need as fast of a drive. If you want to work with camera raw formats, then your whole system will probably need some work.
Offline

Abdelrahman Magdy

  • Posts: 249
  • Joined: Fri Jan 23, 2015 10:08 pm

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 1:53 am

I am talking about something like this.. an expansion card that ASUS makes for some of its motherboards

https://www.asus.com/us/Motherboard-Accessory/ThunderboltEX-3/specifications/

Regarding the media I use, most of the time I transcode to something like DNX. However, sometimes I use EXRs from 3D applications or from Fusion, and the hard disk struggles playing EXRs sometimes.

However, the one thing I was doing recently that made me notice the problem even more was a multicam edit, where we had more than 20 different angles, and the disk couldn't even play the media with all of these angles activated. I basically had to disable all of them and start working on a couple of these takes, finish them, and then, in another pass, work on the next ones, etc.

All of these were transcoded from Premiere into Cineform low-quality proxies, and the playback quality was set to the lowest, and still Premiere couldn't play them because of the hard disk.
System specs:
Resolve Studio version: 15.0.0.086
Fusion Studio version: 9.0.2
OS: CentOS 7
CPU: Intel i7-4790k 4GHz Quad-Core
RAM: 32GB
GPU: NVIDIA GeForce 1080 Ti 11GB
Offline

Glenn Sakatch

  • Posts: 262
  • Joined: Sat Apr 13, 2013 5:36 pm

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 2:42 am

Exrs and 20 streams at once is probably going to give you issues on most systems.
Offline

Abdelrahman Magdy

  • Posts: 249
  • Joined: Fri Jan 23, 2015 10:08 pm

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 4:24 am

Glenn Sakatch wrote:Exrs and 20 streams at once is probably going to give you issues on most systems.


Just to be clear.. it wasn't 20 streams of EXRs. That would have probably killed my computer :lol:

So would a solution that uses 10 g card be faster than USB3?

Also, I have read somewhere that with NAS, it is better to connect to the router's ethernet, instead of directly to the computer? Is that true or is it better to connect to the computer?
System specs:
Resolve Studio version: 15.0.0.086
Fusion Studio version: 9.0.2
OS: CentOS 7
CPU: Intel i7-4790k 4GHz Quad-Core
RAM: 32GB
GPU: NVIDIA GeForce 1080 Ti 11GB
Offline

Glenn Sakatch

  • Posts: 262
  • Joined: Sat Apr 13, 2013 5:36 pm

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 3:58 pm

No, i understand what you meant by EXRs and multicam, my point is 20 streams of multicam is going to push a lot of systems to the brink.

USB 3 is theoretical at what 5g? but you probably won't ever hit it. The drive inside the enclosure is going to limit that speed way before you hit 5g. A 10 g connector, with a properly built raid...8 drives or so, should be able to hit 10g, or very close to it. The key is not just the connector, but the drive setup that is using that connector. Single drives will probably never reach the speed of a well built raid solution of 8 drives.

My Qnap gives me a speed test result of 10g, to both my imac (through 10g external device) and the Z820.

In the end, to get a fast storage system, you will have to spend a bit of money, but it is nothing like it used to be. My entire Qnap setup was probably a grand or so more than my Pegasus, but the performance difference is quite noticeable.
Offline

Dan Sherman

  • Posts: 444
  • Joined: Fri Jul 01, 2016 11:07 pm
  • Warnings: 1

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 5:09 pm

Imo, if you have the technical ability, go with a custom NAS, as they offer far more performance per dollar and flexibility than any of the consumer grade NAS or DAS systems.
Offline

Abdelrahman Magdy

  • Posts: 249
  • Joined: Fri Jan 23, 2015 10:08 pm

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 5:55 pm

@Glenn
Thank you very much. I will definitely be working off a RAID to get that speed. I understand that I will need to put a bit of money to get something that works properly. I just didn't want it to be too crazy :D


@Dan
I wouldn't be that much against building my own NAS. I have looked into some guides before on how to do it. But of course I would prefer something that is already made if the difference in performance is not massive as it will make things easier of course.

If I go with custom NAS, any specific recommendations regarding the software to use. The only one I have looked into was FreeNAS.
System specs:
Resolve Studio version: 15.0.0.086
Fusion Studio version: 9.0.2
OS: CentOS 7
CPU: Intel i7-4790k 4GHz Quad-Core
RAM: 32GB
GPU: NVIDIA GeForce 1080 Ti 11GB
Offline

Abdelrahman Magdy

  • Posts: 249
  • Joined: Fri Jan 23, 2015 10:08 pm

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 6:07 pm

Okay.. now I have a new issue :(

I have looked into the 10g cards, and I won't be able to use any of them as I ran out of PCIe slots. The only slots available are PCIe x1 which won't work for the 10g cards. And, AFAIK, these cards have to use PCIe.

Are there any other options for me to use a NAS, or should I just go with a USB3 RAID, which means I am getting less speed?
System specs:
Resolve Studio version: 15.0.0.086
Fusion Studio version: 9.0.2
OS: CentOS 7
CPU: Intel i7-4790k 4GHz Quad-Core
RAM: 32GB
GPU: NVIDIA GeForce 1080 Ti 11GB
Offline

Andrew Kolakowski

  • Posts: 4331
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 6:26 pm

USB3 in reality is about 400MB/sec.
USB 3.1 Gen2 is about 1000MB/sec. which is about the same as 10gbit ethernet.
Then you have TB2 (1.5GBytes/sec) and TB3, which should go up to above 3GBytes/sec.
Offline

Dan Sherman

  • Posts: 444
  • Joined: Fri Jul 01, 2016 11:07 pm
  • Warnings: 1

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 7:30 pm

Abdelrahman Magdy wrote:Okay.. now I have a new issue :(

I have looked into the 10g cards, and I won't be able to use any of them as I ran out of PCIe slots. The only slots available are PCIe x1 which won't work for the 10g cards. And, AFAIK, these cards have to use PCIe.

Are there any other options for me to use a NAS, or should I just go with a USB3 RAID, which means I am getting less speed?


With this in mind, you would probably be better off looking into upgrading/replacing your current machine imo.
Offline

Abdelrahman Magdy

  • Posts: 249
  • Joined: Fri Jan 23, 2015 10:08 pm

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 7:32 pm

Dan Sherman wrote:
Abdelrahman Magdy wrote:Okay.. now I have a new issue :(

I have looked into the 10g cards, and I won't be able to use any of them as I ran out of PCIe slots. The only slots available are PCIe x1 which won't work for the 10g cards. And, AFAIK, these cards have to use PCIe.

Are there any other options for me to use a NAS, or should I just go with a USB3 RAID, which means I am getting less speed?


With this in mind, you would probably be better off looking into upgrading/replacing your current machine imo.
Ouch... this is what um trying to avoid atm, as I don't have enough budget to upgrade to something decent.

Any ways around this?

Sent from my SM-G920F using Tapatalk
System specs:
Resolve Studio version: 15.0.0.086
Fusion Studio version: 9.0.2
OS: CentOS 7
CPU: Intel i7-4790k 4GHz Quad-Core
RAM: 32GB
GPU: NVIDIA GeForce 1080 Ti 11GB
Offline

Dan Sherman

  • Posts: 444
  • Joined: Fri Jul 01, 2016 11:07 pm
  • Warnings: 1

Re: Need guidance on storage solutions for video editing

PostTue Mar 06, 2018 7:45 pm

Abdelrahman Magdy wrote:
Dan Sherman wrote:
Abdelrahman Magdy wrote:Okay.. now I have a new issue :(

I have looked into the 10g cards, and I won't be able to use any of them as I ran out of PCIe slots. The only slots available are PCIe x1 which won't work for the 10g cards. And, AFAIK, these cards have to use PCIe.

Are there any other options for me to use a NAS, or should I just go with a USB3 RAID, which means I am getting less speed?


With this in mind, you would probably be better off looking into upgrading/replacing your current machine imo.
Ouch... this is what um trying to avoid atm, as I don't have enough budget to upgrade to something decent.

Any ways around this?

Sent from my SM-G920F using Tapatalk


If your case and motherboard supports it, internal disks raid.

Return to Post Production

Who is online

Users browsing this forum: Google [Bot] and 5 guests