Blackmagic Forum

I've got a HP ProLiant system that's got a group of 4TB drives in a raid array for my main media storage and exports. That comprises about 20TB of storage. Then I've got a group of drives in a RAID array for my cache / scratch. Plus two drives that are run as the system drives for a Windows Server 2016 environment.

I'm wondering what you guys might recommend on the best configuration to use this as a network storage array. Currently / for a while I've had it configured where the 20TB array is setup as a network share via SMB that I can access from multiple devices and map the media to the network share. Then I have the cache disk setup as a iSCSI and generate optimized media to that.

However this hasn't been performing quite as I'd like and I'm wondering how you guys have your network based projects setup? Currently it's just me accessing it so I'm not worried about other users writing to it at the same time. Although it would be nice to set this up as best as possible for future growth.

I'm editing on a MacBook Pro 2017

Get yourself some TB3 RAID box, but I think your CPU/GPU will be the main bottleneck.

That would be fine if I wanted to go invest thousands in a thunderbolt array only for my system. But I've already got server hardware with drives and as mentioned in the future id like to be able to share projects with others. So I'm looking for a network solution.

What specific performance issues are you having? What flavor of RAID? How are your devices networked? 1Gb? 10Gb?

Dropped frames / stuttering playback

Currently the workflow is copy footage to storage array on network

In davinci navigate to media drive and select clips to be in project

then optimize media

However I'm getting poor playback on even 1080 footage.

It's a gigabit network and I'll have to check the raid configurations to make sure

Are you sure it's storage issue?
What if you cache media to Mac internal drive?
1gbit will be bit to slow for intermediate 4K files.

Andrew Kolakowski wrote:Are you sure it's storage issue?
What if you cache media to Mac internal drive?
1gbit will be bit to slow for intermediate 4K files.

The MacBook Pro only has a 512GB SSD. Optimally I'd like to completely have a network based work flow so that in the future it'll be easier to share media and projects.

Edrick Smith wrote:Dropped frames / stuttering playback

Currently the workflow is copy footage to storage array on network

In davinci navigate to media drive and select clips to be in project

then optimize media

However I'm getting poor playback on even 1080 footage.

It's a gigabit network and I'll have to check the raid configurations to make sure

What format are you using for optimized media? What does your resource usage look like during playback on the Mac?

Edrick Smith wrote:

Andrew Kolakowski wrote:Are you sure it's storage issue?
What if you cache media to Mac internal drive?
1gbit will be bit to slow for intermediate 4K files.

The MacBook Pro only has a 512GB SSD. Optimally I'd like to completely have a network based work flow so that in the future it'll be easier to share media and projects.

Your Mac has limited power and this maybe the main problem, not actual storage speed.
First make sure your bottleneck is not caused by CPU/GPU by trying some small test from internal SSD (this has plenty speed).

Andrew Kolakowski wrote:

Edrick Smith wrote:

Andrew Kolakowski wrote:Are you sure it's storage issue?
What if you cache media to Mac internal drive?
1gbit will be bit to slow for intermediate 4K files.

The MacBook Pro only has a 512GB SSD. Optimally I'd like to completely have a network based work flow so that in the future it'll be easier to share media and projects.

Your Mac has limited power and this maybe the main problem, not actual storage speed.
First make sure your bottleneck is not caused by CPU/GPU by trying some small test from internal SSD (this has plenty speed).

It's the top of the line practically 2017 15" Macbook Pro I would certainly hope it can handle 1080 video.


Sent from my iPhone using Tapatalk

Most likely it should, but it's not so obvious, depending what you do and what type of source use.
Don't hope- if you have project which doesn't work well with your storage, try the same from internal drive. Quite simple test, no need to debate

If you want everything to be shared then you don't have that many choices. You can (depending on your RAID box) go to 10gbit using TB3 to 10gbit adapter.

i have a server (64Tb) that holds cam orig / vfx assorted media
and a DAS (32Tb or 16Tb depending on machine) for caches
this works well for my use case

This is the activity monitor while streaming the content in DaVinci from the media share. This is 1080 content.

So I ran the Black Magic Disk Speed Test

I seem to be capping out at 13MB/s on Read and 38 MB/s on write on the iSCSI connection and not much better on the SMB media share, that coming in at around 18MB/s on Read and 38MB/s on write.

I've attached the report from the server

That's poor. Good gigabit should give you at least 70MB/sec up to 110.

Are you on one of the latest OSX?
Check this:

https://dpron.com/os-x-10-11-5-slow-smb/

I tested the speed with black magics disk software locally on the Windows 2016 server and I'm getting 56.6MB/s write and 340.5MB/s read on my media storage array and 518MB/s Write and 816MB/s read on my cache array.

I am running the latest version of OS X. I'm going to test the speed test on a Windows Client.

From a windows client I'm also getting relatively poor speeds. of 32 MB/s write and 92.7MB/s read. Although the write speeds being that they maxed out at 56.6MB/s for write locally would make a bit of sense with the overhead of the network protocol. So it does seem to be Mac OS that has horrible speeds.

However it seems to be true with both SMB and iSCSI attached storage. Locally on the storage array I'm getting 518/816MB/s Write/Read. However over the network via a gigabit link I'm only getting 37.8 Write and 10.7MB read.

For S&G's I connected an ethernet cable direct between the MacBook Pro and the server via the 4 port NIC it has installed. Assigned the laptop and the server their own IPs to establish a direct link and I still get horrendous performance. So it seems to be something with the Windows Server and the network sharing via SMB and iSCSI?

Yes, you have some issue with SMB compatibility.
Have you tried this tweak from the link?

Andrew Kolakowski wrote:Yes, you have some issue with SMB compatibility.
Have you tried this tweak from the link?

The issue occurs both with SMB and iSCSI as far as I know they use two different protocols?

Edrick Smith wrote:The issue occurs both with SMB and iSCSI as far as I know they use two different protocols?

They are indeed totally different. My guess is that there's a network problem somewhere, whether it's on the server NIC, the client NIC, or something in between.

Jack Fairley wrote:

Edrick Smith wrote:The issue occurs both with SMB and iSCSI as far as I know they use two different protocols?

They are indeed totally different. My guess is that there's a network problem somewhere, whether it's on the server NIC, the client NIC, or something in between.

I thought originally perhaps cabling or network switch but I isolated that and just directly attached the systems together with a different network cable. Client side I've tried two systems one mac and one PC. Server side the system has two on board nics and a 4 port card and same results


Sent from my iPhone using Tapatalk

Edrick Smith wrote:

Andrew Kolakowski wrote:Yes, you have some issue with SMB compatibility.
Have you tried this tweak from the link?

The issue occurs both with SMB and iSCSI as far as I know they use two different protocols?

Yes, but both still can be affected in similar way. We are dealing with Apple which does things not mentioning a word about it. For Apple there are no pro and home users- they do what they want and don't care much about any lost compatibility etc (specially when we talk about fairy old protocols).

Judging by this link you may have exactly the same problem as it describes.

How old is that HP box? It may not be worth to waste a time on it.
Some 50$ home routers give better speed on network shares.