2019 Mac Pro rendering YouTube 4k h264 video only in real-ti

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

mheartsj

  • Posts: 9
  • Joined: Thu Jan 02, 2020 9:29 pm
  • Location: San Jose, CA
  • Real Name: Mike Ho

2019 Mac Pro rendering YouTube 4k h264 video only in real-ti

PostThu Jan 02, 2020 9:52 pm

Hi,

I'm not a professional by any means, so I could use some help! I've searched through the forums and read through the manual and yet I can't figure out how to speed up the performance of Davinci Resolve 16 Studio.

Project
I have a fairly simple project with a multi-cam video split between a Canon XF705 HEVC/MXF video 4K@23.976fps, an iPhone X Filmic Pro 4K@24fps (HEVC), and two audio streams (Tascam WAV). For the multi-cam, I just have the project set that way, but I haven't tried to transition between cameras yet, so it's only rendering the XF705 stream. I also have some slides as 4K PNG files edited in on its own video stream, set to fade in/out at 24frame crossfades. Each video is about 1h 7m long. I have the Render Cache set to ProRes 422 HQ and the settings to use both GPUs on Metal.

Machine
I have a 2019 Mac Pro with 12-Core CPU, 96GB RAM and one Radeon Pro Vega II Duo card. I also have a Drobo 5D3 running 3x10TB 7200 RPM disks over Thunderbolt 3 on its own controller (nothing else is connected), where my media is stored.

Problem
The problem I'm having is that while it's much faster than my old setup, it's still rendering the videos at near real-time. I've tried encoding it in the YouTube 4K default, 4K h264 master, 4K h265 master, 4K GoPro YUV 10-bit versions. They're all still completing at around real-time, give or take a minute. What also surprises me is that looking at iStat Menus, I see my CPU running all cores at around 40-50%, but my GPU is running at around 5-10%. The bottleneck is not the Drobo either since the dashboard is showing it writing at around 15MB/s where I've seen it write as much as 300+ MB/s (Thunderbolt 3 has 40Gbps or 5GBps bandwidth).

Is there any way to make the rendering faster? I would like to think that if I utilized the full resources of the system, I could render this in a matter of minutes. Another data point is that while I'm editing, it can playback at 24fps, but when I add a slide and some cross-fade or stop and start, it will stutter, drop to 9fps and then build its way back up. All this without GPU usage spiking and CPU still at around 40%. Any thoughts? I'd like to maximize what I paid for! Thanks in advance!
Offline

mheartsj

  • Posts: 9
  • Joined: Thu Jan 02, 2020 9:29 pm
  • Location: San Jose, CA
  • Real Name: Mike Ho

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostTue Jan 07, 2020 4:51 am

Took a while to get my post approved... Anyone have any thoughts on how I would configure my setup or project to render faster? Thanks!
Offline

Jim Simon

  • Posts: 34455
  • Joined: Fri Dec 23, 2016 1:47 am

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostTue Jan 07, 2020 5:30 pm

mheartsj wrote:Is there any way to make the rendering faster?


Better hardware.
My Biases:

You NEED training.
You NEED a desktop.
You NEED a calibrated (non-computer) display.
Offline
User avatar

rick.lang

  • Posts: 18122
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

2019 Mac Pro rendering YouTube 4k h264 video only in real-ti

PostTue Jan 07, 2020 7:36 pm

Where are you rendering? Where is the render cache? If you are using the Drobo for your source media, and your cache, and optimized media, and your render output: you are thrashing on the Drobo hard disks.

Not sure what internal SSD storage you have, but that would help things move along. Recommend you acquire one of those blazing fast PCIe3 NVMe SSDs in RAID 0.

You need some of those things to be stored in different places and make sure only one purpose is served by the Drobo spinning hard disks.

Edit
You don’t need the maximum cost NVMe SSD RAID to see a good improvement. At some point the 12-core CPU may limit your throughput once you improve your overall data throughput. It’s a balancing act.
Rick Lang
Offline

mheartsj

  • Posts: 9
  • Joined: Thu Jan 02, 2020 9:29 pm
  • Location: San Jose, CA
  • Real Name: Mike Ho

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostTue Jan 07, 2020 8:37 pm

Jim Simon wrote:
mheartsj wrote:Is there any way to make the rendering faster?


Better hardware.


I appreciate the Mac vs Windows/Linux sarcasm in this post, but unfortunately, it's not really productive or beneficial. :(
Offline

mheartsj

  • Posts: 9
  • Joined: Thu Jan 02, 2020 9:29 pm
  • Location: San Jose, CA
  • Real Name: Mike Ho

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostTue Jan 07, 2020 8:42 pm

rick.lang wrote:Where are you rendering? Where is the render cache? If you are using the Drobo for your source media, and your cache, and optimized media, and your render output: you are thrashing on the Drobo hard disks.

Not sure what internal SSD storage you have, but that would help things move along. Recommend you acquire one of those blazing fast PCIe3 NVMe SSDs in RAID 0.

You need some of those things to be stored in different places and make sure only one purpose is served by the Drobo spinning hard disks.

Edit
You don’t need the maximum cost NVMe SSD RAID to see a good improvement. At some point the 12-core CPU may limit your throughput once you improve your overall data throughput. It’s a balancing act.


Thanks, Rick! You're right that I'm rendering from my Drobo with my source and output there. My render cache is on the internal SSD. I'll try copying my source material to the SSD and then update the references on the project. I'll report back when I get a chance to do so and see if there's a difference.

What made me wonder if this is useful is that the Drobo can clearly handle higher throughput than what's there, so I'm not sure what's slowing it down. It'd be interesting to understand what's being maxed out when there's no indication that anything is. CPU is at 50%, GPU is at 10%, disk bandwidth is at 0.2%, main memory is at around 20%, and GPU memory is around 5%.
Offline
User avatar

Uli Plank

  • Posts: 25124
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostTue Jan 07, 2020 10:42 pm

The Drobo is not particularly fast when sending data back and forth.
My disaster protection: export a .drp file to a physically separated storage regularly.

Studio 19.1.3
MacOS 13.7.4, 2017 iMac, 32 GB, Radeon Pro 580 + eGPU
MacBook M1 Pro, 16 GPU cores, 32 GB RAM, MacOS 14.7.2
SE, USM G3
Offline
User avatar

rick.lang

  • Posts: 18122
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 08, 2020 1:19 am

My guess is the spinning disk read/write heads are slowing everything down. If the heads are spending all their time seeking the addresses for conflicting operations, the delay of moving the heads is very significant. That’s a form of thrashing with heads going back and forth to two or more addresses instead of doing one task if continuously reading OR writing.

Are your internal SSDs large enough to more than handle moving something to them as you suggest. I don’t think you mentioned the size of your internal SSDs. The default size on the Mac Pro is only large enough to store apps and the OS. I hope you have lots of room as you don’t want to come close to filling those either.
Rick Lang
Offline

mheartsj

  • Posts: 9
  • Joined: Thu Jan 02, 2020 9:29 pm
  • Location: San Jose, CA
  • Real Name: Mike Ho

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 08, 2020 1:33 am

Thanks, Rick :)

I opted for 2TB of space. It's sufficient so far and I should have more than enough room to edit these videos. I'll transfer them back after each one. If not, I have two 2TB external SSD drives but they're over USB 3.2, so I'm not sure if the 20Gbps is fast enough.
Offline
User avatar

rick.lang

  • Posts: 18122
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 08, 2020 1:51 am

Good you have the 2TB option. My iMac 2015 had the maximum 1TB option at that time and it’s really insufficient.

Suitability for video work:
My ‘negativity’ about Apple components is based on my woeful experiences with iMac 2009 and iMac 2015. No longer in favour of supporting Steve Jobs fetish about computers being ‘thin’ as the ultimate design goal. Spinning disks are bad inless you can RAID at least 6 of them. Fusion drives are garbage, the little SSD they have is just lipstick on a pig and fast processors are brought to their knees by systems better suited to fry eggs than work on video.
Rick Lang
Offline

Jim Simon

  • Posts: 34455
  • Joined: Fri Dec 23, 2016 1:47 am

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 08, 2020 2:21 am

mheartsj wrote:I appreciate the Mac vs Windows/Linux sarcasm in this post


There actually was none. If you're using software at this level, it's assumed you're good enough with computers to have the system properly set up. So if you want your system to go faster, you need faster hardware. That applies to most any software, on any OS.
My Biases:

You NEED training.
You NEED a desktop.
You NEED a calibrated (non-computer) display.
Offline

mheartsj

  • Posts: 9
  • Joined: Thu Jan 02, 2020 9:29 pm
  • Location: San Jose, CA
  • Real Name: Mike Ho

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 08, 2020 2:24 am

Jim Simon wrote:
mheartsj wrote:I appreciate the Mac vs Windows/Linux sarcasm in this post


There actually was none. If you're using software at this level, it's assumed you're good enough with computers to have the system properly set up. So if you want your system to go faster, you need faster hardware. That applies to most any software, on any OS.


Which part of the specs do I need to upgrade? From reading this forum, 12 cores, 96GB RAM, and the fastest GPU on the market should be plenty fast enough. And I'm utilizing less than 10% of the available resources. So I don't think it's a hardware problem. It's either the software configuration or Rick's suggestion, which I'm going to test.
Offline
User avatar

Donnell Henry

  • Posts: 1122
  • Joined: Wed Dec 25, 2013 9:04 pm
  • Location: Brooklyn ny

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 08, 2020 3:01 am

It’s rick suggestion. Your external hard drive is the bottleneck. Test it on the internal ssd like you stated and report back
GODS CREATE
Offline

Rohit Gupta

Blackmagic Design

  • Posts: 1671
  • Joined: Wed Aug 22, 2012 5:00 am

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 08, 2020 3:23 am

mheartsj wrote:Hi,

I'm not a professional by any means, so I could use some help! I've searched through the forums and read through the manual and yet I can't figure out how to speed up the performance of Davinci Resolve 16 Studio.

Project
I have a fairly simple project with a multi-cam video split between a Canon XF705 HEVC/MXF video 4K@23.976fps, an iPhone X Filmic Pro 4K@24fps (HEVC), and two audio streams (Tascam WAV). For the multi-cam, I just have the project set that way, but I haven't tried to transition between cameras yet, so it's only rendering the XF705 stream. I also have some slides as 4K PNG files edited in on its own video stream, set to fade in/out at 24frame crossfades. Each video is about 1h 7m long. I have the Render Cache set to ProRes 422 HQ and the settings to use both GPUs on Metal.

Machine
I have a 2019 Mac Pro with 12-Core CPU, 96GB RAM and one Radeon Pro Vega II Duo card. I also have a Drobo 5D3 running 3x10TB 7200 RPM disks over Thunderbolt 3 on its own controller (nothing else is connected), where my media is stored.

Problem
The problem I'm having is that while it's much faster than my old setup, it's still rendering the videos at near real-time. I've tried encoding it in the YouTube 4K default, 4K h264 master, 4K h265 master, 4K GoPro YUV 10-bit versions. They're all still completing at around real-time, give or take a minute. What also surprises me is that looking at iStat Menus, I see my CPU running all cores at around 40-50%, but my GPU is running at around 5-10%. The bottleneck is not the Drobo either since the dashboard is showing it writing at around 15MB/s where I've seen it write as much as 300+ MB/s (Thunderbolt 3 has 40Gbps or 5GBps bandwidth).

Is there any way to make the rendering faster? I would like to think that if I utilized the full resources of the system, I could render this in a matter of minutes. Another data point is that while I'm editing, it can playback at 24fps, but when I add a slide and some cross-fade or stop and start, it will stutter, drop to 9fps and then build its way back up. All this without GPU usage spiking and CPU still at around 40%. Any thoughts? I'd like to maximize what I paid for! Thanks in advance!


I suspect one of your sources is a variant of H264 or H265 which is not hardware accelerated, and is probably the bottleneck. I assume you are rendering to H265/8bit which is HW accelerated. To find the bottleneck, you can make get rid of the sources all but 1 at a time, so find what is slowing you down. Also, you can try rendering to ProRes as a test which should be plenty fast.
Rohit Gupta

DaVinci Resolve Software Development
Blackmagic Design
Offline
User avatar

rick.lang

  • Posts: 18122
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

2019 Mac Pro rendering YouTube 4k h264 video only in real-ti

PostWed Jan 08, 2020 3:26 am

Good point Rohit, and that might explain why the CPU usage is fairly high, but the CPUs are far from maxed out so it’s possibly not the only factor.
Rick Lang
Offline

Rohit Gupta

Blackmagic Design

  • Posts: 1671
  • Joined: Wed Aug 22, 2012 5:00 am

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 08, 2020 3:28 am

rick.lang wrote:Good point Rohit, and that might explain why the CPU usage is fairly high, but the CPUs are far from maxed out so it’s possibly not the only factor.


Some of the temporal compressed codecs like h264/h265 don't scale very well across cores, so it's expected.
Rohit Gupta

DaVinci Resolve Software Development
Blackmagic Design
Offline
User avatar

Paul Draper

  • Posts: 201
  • Joined: Thu Apr 27, 2017 8:33 pm

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 08, 2020 10:05 pm

mheartsj wrote:Is there any way to make the rendering faster? I would like to think that if I utilized the full resources of the system, I could render this in a matter of minutes.
FWIW: in my experience, the only thing that makes that difference is to generate optimised media (from H.264 to say, ProRes or similar). For reference, I'm generally using H.264 4k 50p media from DJI Osmo Pocket; Insta360 One X, and possibly the same from a Lumix GH5 (if 50p) or 25p All-i codec. All 4k media is edited on a 1080p /25fps timeline (generally my highest res destination).

A bit of a drag because this can be time-consuming, however, once 'optimise media' is done, the Deliver timeline flies though with more like the speed you're expecting. Also especially useful with multiple renders, I/Os in the queue.

A couple of caveats: for whatever reason, Resolve annoyingly loses this information for optimised media and cache files, eg, if a copy and paste to a fresh master timeline is made; if there is a save of the project to a new version, and so on. PITA (especially re. the user cache). Anyways, last thing I always do now before rendering out is to select all media on the timeline and 'optimise media', go have a coffee break. Then do your export work without fuss and with speed. Hope that helps.

Edit,
PS: Oh, I have found it necessary to have optimised media and cache on a different, dedicated 'fast cache' drive that is different /separate to where the original media is located - that preference is identified in Resolve's preferences for the primary disk (ie, where the cache and optimised media is written, NOT the primary media disk, in your case, the Dobro). Many use something like a multi M.2 RAID 0 PCIe carrier; eg for the Mac, the Sonnet M.2 4x4, Amfeltec, Highpoint. For more about that, see this MacRumors forum thread.

Whichever, I suggest to use a different faster drive for optimised media & cache.
MacPro 7,1 16 core, 192GB, MacOS 15.1, Vega II Duo. Antelope Pure 2 & Orion 32+, UA Apollo x8. RAID-4 Thunderbay 6, RAID-0 Sonnet M.2 4x4. MiniMon 4k, Dell U3415W & BenQ SW2700PT. Nuendo, Wavelab, Resolve Studio 19
Offline
User avatar

Uli Plank

  • Posts: 25124
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 08, 2020 10:40 pm

Because of the frequent problems with optimized media, I tend to do a transcode with the Media Management function into a light-weight format and work with that. Before render, I conform back to the originals.
My disaster protection: export a .drp file to a physically separated storage regularly.

Studio 19.1.3
MacOS 13.7.4, 2017 iMac, 32 GB, Radeon Pro 580 + eGPU
MacBook M1 Pro, 16 GPU cores, 32 GB RAM, MacOS 14.7.2
SE, USM G3
Offline

mheartsj

  • Posts: 9
  • Joined: Thu Jan 02, 2020 9:29 pm
  • Location: San Jose, CA
  • Real Name: Mike Ho

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostThu Jan 09, 2020 7:31 am

So I've done some tests and here are the results. For simplicity, I kept the source disk and the target disk the same. I'm rendering a video with a 4k source to a 4k destination that is 1:07:18 in 24fps. Interspersed are a few 4k PNG slides with 24 frame crossfades. The audio track is in WAV format and I'm rendering it to whatever default is for each preset.

Transcoding raw video from Canon HEVC to ProRes 422 HQ: 1:28:26

Disk: SSD | Source: Canon HEVC
ProRes master: 1:17:58
YouTube 4k: 1:05:21
h264 master: 1:06:14
h265 master: 1:05:41

Disk: SSD | Source: ProRes 422 HQ
CPU: 25% | GPU: 10% | Disk: 4.4MB/s write, 194MB/s read
YT 4k: 27:35
ProRes 422 HQ: 20:44
h264 master: 27:59
h265 master: 35:19

Disk: Drobo | Source: Canon HEVC
YT 4k: 1:06:32
h264 master: 1:05:57
h265 master: 1:06:14
GoPro CineForm YUV 10-bit: 1:12:18

Disk: Drobo | Source: ProRes 422 HQ
YT 4k: 36:31
ProRes: 1:06:16
h264 master: 39:42

I also noticed that peak FPS is 76 (ProRes to ProRes on SSD). And it hits around 80% CPU but GPU is still around 10%. Is there any way to use more resources?

I'm seeing several trends here:
  1. The biggest bottleneck is the original HEVC source from the XF705. So it's the decoding of the source that's the bottleneck (despite it not really consuming resources like GPU/CPU). I need to convert to ProRes first.
  2. If I'm rendering to h264/h265 (YT preset uses h264), I can do it on the Drobo and not have much of a performance hit (~10 min deficit)
  3. If I want to render to ProRes, then I should do it on an SSD (21min vs 1h6min)
Offline
User avatar

Uli Plank

  • Posts: 25124
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostThu Jan 09, 2020 7:36 am

Possibly the HEVC from the Canon is not supported by hardware decoding.
My disaster protection: export a .drp file to a physically separated storage regularly.

Studio 19.1.3
MacOS 13.7.4, 2017 iMac, 32 GB, Radeon Pro 580 + eGPU
MacBook M1 Pro, 16 GPU cores, 32 GB RAM, MacOS 14.7.2
SE, USM G3
Offline

Jonathan Happ

  • Posts: 79
  • Joined: Tue Sep 20, 2016 4:50 pm

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostThu Jan 09, 2020 1:19 pm

mheartsj wrote:I also noticed that peak FPS is 76 (ProRes to ProRes on SSD). And it hits around 80% CPU but GPU is still around 10%. Is there any way to use more resources?


I think that the GPU is not engaged in encoding ProRes. You will benefit from the GPU, when you de and encode H264/H265 and for certain effects. - so far as I know - For better ProRes encoding Apple introduced this ProRes Afterburner card... Not sure, if Resolve would use that.
Mac OS 10.13.6
Mac Pro Late 2013
32 GB RAM
3,7 GHz Quad-Core Intel Xeon E5

Win10
Supermicro Workstation
2x Intel Xeon 2,1GHz 12C
96GB RAM
512GB SSD System
1TB SSD Cache
Nvidia GeForce RTX 2080 Ti(2x)
DeckLink 4K Extreme 12G
Internal Raid 8x 4TB SATA
Offline

Andrew Kolakowski

  • Posts: 9391
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostThu Jan 09, 2020 2:30 pm

80% of CPU on many cores system is already a good usage. Saying this ProRes to ProRes should go faster than 70fps by my guess.
I had 100fps+ going from UHD ProRes to DNxHR on 10 cores i9 overclocked to 4GHz.

Or maybe not if your CPU is staying at base 3.3GHz clock. High clock makes huge difference (basically linear with clock increase).
Offline
User avatar

rick.lang

  • Posts: 18122
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

2019 Mac Pro rendering YouTube 4k h264 video only in real-ti

PostThu Jan 09, 2020 4:53 pm

Jonathan Happ wrote:
mheartsj wrote:I also noticed that peak FPS is 76 (ProRes to ProRes on SSD). And it hits around 80% CPU but GPU is still around 10%. Is there any way to use more resources?


I think that the GPU is not engaged in encoding ProRes. You will benefit from the GPU, when you de and encode H264/H265 and for certain effects. - so far as I know - For better ProRes encoding Apple introduced this ProRes Afterburner card... Not sure, if Resolve would use that.


I believe BMD will be making announcements regarding the Afterburner card in the next few months. Might be tied to Resolve 17 but that’s speculative on my part. I don’t know what is under the hood in that card. We know Apple is using it now for ProRes and ProRes raw. Is it possible another vendor could load their own firmware to the card and use the card for BRAW and ProRes?
Rick Lang
Offline

mheartsj

  • Posts: 9
  • Joined: Thu Jan 02, 2020 9:29 pm
  • Location: San Jose, CA
  • Real Name: Mike Ho

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostThu Jan 09, 2020 5:19 pm

So not having the GPU running during rendering baffles me and I'm wondering what set of sources and codecs I need to get the GPU to kick in. Anyone from BMD have thoughts?

What is so interesting is that Resolve is advertised on the Radeon Pro Vega II page: https://www.amd.com/en/graphics/worksta ... ro-vega-ii
How do I take advantage? Complex edits, color correction, and visual effects? I don't have any of those (yet), so maybe that's what I'm missing.

I really want to thank the community for their input and insight into all of this. I hope everyone understands that I'm just looking to get the best out of the new hardware I bought. For simple videos like mine, you apparently don't need to buy the Radeon Pro Vega II. And if I knew that, I could've saved myself $5,000.

I hope there's future value though, in that there can be some hardware-acceleration during rendering for YT/h264/h265 or for decoding Canon XF-HEVC/HEVC/h265.
Offline

Andrew Kolakowski

  • Posts: 9391
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostFri Jan 10, 2020 3:36 pm

Hardware decoding capabilities are controlled by hardware vendors mainly. Others (like BM) just use what is provided. Anything 4:2:2 is not hardware accelerated (nor 10bit h264) and nothing suggests it's going to change any time soon. Nvidia, Intel, AMD - none of them promised support for 4:2:2 decoding as far as I know.
Another thing- some hardware acceleration is good, but GPU power is not unlimited. When you create more complex projects and start relying on GPU for many things then you will get into situation when GPU is at 100% and CPU at 10%. H264/5 decoding is slightly different as this is handled by totally separate GPU unit which is responsible just for this task.
Offline

Andrew Kolakowski

  • Posts: 9391
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostFri Jan 10, 2020 3:39 pm

rick.lang wrote:
Jonathan Happ wrote:
mheartsj wrote:I also noticed that peak FPS is 76 (ProRes to ProRes on SSD). And it hits around 80% CPU but GPU is still around 10%. Is there any way to use more resources?


Is it possible another vendor could load their own firmware to the card and use the card for BRAW and ProRes?


This is probably impossible. Apple would have to release some public code for this. Atm. this is more like RED Rocket- fixed code controlled by manufacturer.
Offline

Rohit Gupta

Blackmagic Design

  • Posts: 1671
  • Joined: Wed Aug 22, 2012 5:00 am

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostFri Jan 10, 2020 4:44 pm

rick.lang wrote:
Jonathan Happ wrote:
mheartsj wrote:I also noticed that peak FPS is 76 (ProRes to ProRes on SSD). And it hits around 80% CPU but GPU is still around 10%. Is there any way to use more resources?


I think that the GPU is not engaged in encoding ProRes. You will benefit from the GPU, when you de and encode H264/H265 and for certain effects. - so far as I know - For better ProRes encoding Apple introduced this ProRes Afterburner card... Not sure, if Resolve would use that.


I believe BMD will be making announcements regarding the Afterburner card in the next few months. Might be tied to Resolve 17 but that’s speculative on my part. I don’t know what is under the hood in that card. We know Apple is using it now for ProRes and ProRes raw. Is it possible another vendor could load their own firmware to the card and use the card for BRAW and ProRes?


Resolve has always supported the Afterburner card since launch. It’s used to access ProRes decode. This basically offloads the decompression, so CPU cores are not used at all. You can use this to get 8K playback even on a 8-core system, for example.
Rohit Gupta

DaVinci Resolve Software Development
Blackmagic Design
Offline

Andrew Kolakowski

  • Posts: 9391
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostFri Jan 10, 2020 9:27 pm

This is because it's handled on OSX system level. I assume BM (or others) has no access or any ability to do any change to Apple's card features (only Apple can do it).
Last edited by Andrew Kolakowski on Sat Jan 11, 2020 9:51 am, edited 1 time in total.
Offline
User avatar

Frank Glencairn

  • Posts: 1921
  • Joined: Wed Aug 22, 2012 7:07 am
  • Location: Germany

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostSat Jan 11, 2020 9:25 am

mheartsj wrote:I'm seeing several trends here:
  1. The biggest bottleneck is the original HEVC source from the XF705. So it's the decoding of the source that's the bottleneck (despite it not really consuming resources like GPU/CPU). I need to convert to ProRes first.
  2. If I'm rendering to h264/h265 (YT preset uses h264), I can do it on the Drobo and not have much of a performance hit (~10 min deficit)
  3. If I want to render to ProRes, then I should do it on an SSD (21min vs 1h6min)


You are using a codec that doesn't scale well across the threads and hardware (CPU) that doesn't accelerate that codec - what you see is pretty much expected, given your hardware and the codec.

Since you bought the wrong hardware for your needs (codecs), or using the wrong codecs for your hardware, you can ether return the hardware and buy something that actually works for you, use better codecs, or spend your time to transcode your material to a better codec before you start working.
https://sites.google.com/view/frankglencairn/home
Offline

Andrew Kolakowski

  • Posts: 9391
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostSat Jan 11, 2020 10:07 am

mheartsj wrote:....
Is there any way to make the rendering faster? I would like to think that if I utilized the full resources of the system, I could render this in a matter of minutes. Another data point is that while I'm editing, it can playback at 24fps, but when I add a slide and some cross-fade or stop and start, it will stutter, drop to 9fps and then build its way back up. All this without GPU usage spiking and CPU still at around 40%. Any thoughts? I'd like to maximize what I paid for! Thanks in advance!


For such a projects you need custom machine eg. 10-16 cores at as fast as possible clock (pass 4GHz).
I have 10 cores i9 at 4GHz and 36 cores at 2.6GHz Supermicro. For single h264/5 encode i9 is faster. Even if load is fairly low Xeons don't happily go to turbo clocks.
Only for multiple simultaneous encodes Xeons start to be way faster. Nothing really can be done about it.
When you pass 8-16 cores (depending on app and nature of the task) things don't scale very well with number of cores. Higher clock gives basically linear increase all the time.
Offline

mheartsj

  • Posts: 9
  • Joined: Thu Jan 02, 2020 9:29 pm
  • Location: San Jose, CA
  • Real Name: Mike Ho

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostThu Jan 16, 2020 3:51 am

Just to close it out the thread based on rendering on Mac Pro, I used the Blackmagic Design Disk Speed Test software to run some numbers to give some perspective. I let it run about 10-15 times to have the numbers stabilize before I took a screenshot.

DiskSpeedTest SSD.png
BMD Disk Speed Test - Mac Pro SSD (2x1TB)
DiskSpeedTest SSD.png (224.21 KiB) Viewed 5919 times

DiskSpeedTest 5D3.png
BMD Disk Speed Test - Drobo 5D3 via Thunderbolt 3 (all 7200RPM disks)
DiskSpeedTest 5D3.png (224.01 KiB) Viewed 5919 times

DiskSpeedTest Drobo.png
BMD Disk Speed Test - Drobo 5D via Thunderbolt 2 (all 7200RPM disks)
DiskSpeedTest Drobo.png (225.06 KiB) Viewed 5919 times
Offline
User avatar

Marc Wielage

  • Posts: 12715
  • Joined: Fri Oct 18, 2013 2:46 am
  • Location: Hollywood, USA

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostThu Jan 16, 2020 5:26 am

mheartsj wrote:So I've done some tests and here are the results. For simplicity, I kept the source disk and the target disk the same.

Are you saying the Source drive and the Target drive are the same physical disk? If so, I think this is unwise.

I think you're better off having a boot drive (which you could use for Project files), a Source drive (just for playback), and a Render drive (for final delivery). As an alternate, you might have a second external drive as a Cache/Gallery still drive, as long as it stays connected. Rendering to the same drive as the source puts it through a lot of potential stress in terms of head access. I'll do it -- reluctantly -- if it's a really short file and I'm in a huge hurry. Anything else goes to an external drive.

Note that YouTube always re-encodes anything uploaded, so you may want to experiment with using different formats to see if it makes a visual difference.
Certified DaVinci Resolve Color Trainer • AdvancedColorTraining.com
Offline
User avatar

jasonvp

  • Posts: 76
  • Joined: Fri Oct 25, 2019 7:53 pm
  • Location: Northern VA
  • Real Name: Jason Van Patten

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostThu Jan 16, 2020 10:07 am

The challenge you're running into is that the h.264/h.265 hardware encoder on the Vega II is a generation or two old. The GPU itself is fairly fast for rendering. But remember that rendering does not equal encoding/exporting. So if your timeline has things on it that the GPU can do as far as rendering goes, it should tear through those pretty quickly. But the hardware encoder can only manage 4K/60 real time. That's its limit and there's no way you're going to make it faster. Moving source media around won't help; moving scratch space around won't help, etc, etc.

The upcoming W5700X card will have a brand new hardware encoder on it. As a renderer, it'll probably be slower than the card you have now. But when it comes to h.264/h.265 output, it'll beat the Vega II pretty handily.
System: Mac Pro (2019)
CPU: 3.2GHz 16-core Xeon
GPU: AMD Radeon Pro Vega II
RAM: 96GB
Storage: 1TB sys drive, 8TB NVMe RAID01, 4TB SSD RAID0
Offline

sulekh

  • Posts: 5
  • Joined: Tue Oct 01, 2013 11:39 pm

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostTue Jan 12, 2021 11:08 pm

I think I have a related issue that I am currently trying to troubleshoot with Blackmagic Support. I had experienced a serious performance drop with Resolve 17 on my fully maxed out 2013 Mac Pro. I was forced to upgrade my machine. I bought 2019 16 Core Mac Pro, 96Gb Ram, 8TB storage, 2x AMD Pro Vega II Duo and Afterburner. I was using Resolve 17 and was told by support to roll back to 16.2.8 release version to investigate.

On this project I was working with Red 8K FF Anamorphic footage. I was transcoding it to 1080 DNxHD. I got 15fps. I was using the internal disk(3000Mb/s) as source and external all SSD RAID(1800Mb/s) as destination. As you can see it is primarily being processed by CPU while the GPU is not being engaged if at all.
I had to use 2013 Mac Pro with Red RocketX to make the day at 30fps. But I can't use Red Rocket X with Catalina/ Big Sur as it is not supported. Red had made an announcement about Metal but I don't see it making any material difference to the performance. I have tried Metal and Open CL.

I have a similarly underwhelming performance with Mini LF UHD Arri Raw to DNxHD where I got 48fps. I used to get similar performance from 2013 Mac till something was tweaked within Resolve and the performance dropped to the extent that I ordered this new machine. I will update the thread with where we land on this.

if we conclude that my system specs are only capable of delivering what I am getting now, I will have to send the system back. This ain't worth 27K.

Please chime in if you have a similar machine with better performance. Do I need to order 28 Core? Add NVME RAID. Sine the Blackmagic / Resolve hardware configuration is unavailable for 2019 Mac Pro, I am shooting in the dark here. Shine the light if you know better. Thanks,
Attachments
Screenshot 2021-01-11 at 10.40.08 AM.png
GPU usage
Screenshot 2021-01-11 at 10.40.08 AM.png (106.67 KiB) Viewed 4878 times
Screenshot 2021-01-11 at 10.40.18 AM.png
CPU usage
Screenshot 2021-01-11 at 10.40.18 AM.png (143.94 KiB) Viewed 4878 times
Offline

Andrew Kolakowski

  • Posts: 9391
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 13, 2021 12:09 am

sulekh wrote:On this project I was working with Red 8K FF Anamorphic footage. I was transcoding it to 1080 DNxHD. I got 15fps. I was using the internal disk(3000Mb/s) as source and external all SSD RAID(1800Mb/s) as destination. As you can see it is primarily being processed by CPU while the GPU is not being engaged if at all.
I had to use 2013 Mac Pro with Red RocketX to make the day at 30fps. But I can't use Red Rocket X with Catalina/ Big Sur as it is not supported. Red had made an announcement about Metal but I don't see it making any material difference to the performance. I have tried Metal and Open CL.


For 8K RED you most likely need Nvidia. RED new SDK has new decoder/debayer which can properly use power of modern Nvidia cards. As far as I'm aware this is not working on AMD as it's CUDa based (not OpenCL).

In your setup you waste all CPU on decoding JPEG2000 8K which is crazy computing intensive. CPU will be your bottleneck as first you need to decode RED's Jpeg2000 in order to go further (debayer it). It all fall on CPU decoding.

There is very simple solution for your case of 8K Red source to just HD output though.
Simply set RED decoding to 1/4 resolution in REDRAW settings (and don't force highest quality in render settings). This won't give you much quality loss (maybe tiny on resolution), but should speed conversion a lot. There is absolutely no need to decode and debayer your 8K source at full resolution if you later scale it to HD, so use this and then you should gain few x times in speed (at least in theory).
Of course if you want to render final 8K/4K master then you can't escape from full quality decoding and then have to wait or buy new machine :D
Offline
User avatar

rick.lang

  • Posts: 18122
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 13, 2021 1:40 am

Don’t forget to get those internal
SSDs working. When you have spinning hard disks it’s imperative to not have inputs and outputs and cache on the same drive. But with sufficient internal SSDs, they seem to cope better.

I’m very pleased with my 16-core, 48GB, 8TB internal SSD, and single Radeon Pro Vega II. Your machine is better but use the internal resources as much as possible assuming all your system settings are correct.
Rick Lang
Offline
User avatar

Uli Plank

  • Posts: 25124
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 13, 2021 2:05 am

I think what Rohit wrote hits the main problem. I'm afraid the Mac mini M1 would be faster when decoding HEVC in that specific format, since it has hardware decoding. Unfortunately, Nvidia cards are not supported by Apple ay more.
Maybe a Mac mini for transcoding only would be an option if you have a lot of that material coming in.
My disaster protection: export a .drp file to a physically separated storage regularly.

Studio 19.1.3
MacOS 13.7.4, 2017 iMac, 32 GB, Radeon Pro 580 + eGPU
MacBook M1 Pro, 16 GPU cores, 32 GB RAM, MacOS 14.7.2
SE, USM G3
Offline

Andrew Kolakowski

  • Posts: 9391
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 13, 2021 10:49 am

sulekh wrote:On this project I was working with Red 8K FF Anamorphic footage. I was transcoding it to 1080 DNxHD. I got 15fps. I was using the internal disk(3000Mb/s) as source and external all SSD RAID(1800Mb/s) as destination. As you can see it is primarily being processed by CPU while the GPU is not being engaged if at all.
I had to use 2013 Mac Pro with Red RocketX to make the day at 30fps. But I can't use Red Rocket X with Catalina/ Big Sur as it is not supported. Red had made an announcement about Metal but I don't see it making any material difference to the performance. I have tried Metal and Open CL.


I just tested on my 4 core i7 MacbookPro and I have 12fps on 8K RED to DNxHD render (look it's about as good as your system) :lol: with 1/4 decoding hack.
Without it's 2fps.
There is no visible quality difference at 1:1 pixels. Use clever settings and then you should be looking at 60fps+ render.
Offline
User avatar

Uli Plank

  • Posts: 25124
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 13, 2021 11:22 am

Red is the big exception since it's codec is very efficient when skipping pixels.
My disaster protection: export a .drp file to a physically separated storage regularly.

Studio 19.1.3
MacOS 13.7.4, 2017 iMac, 32 GB, Radeon Pro 580 + eGPU
MacBook M1 Pro, 16 GPU cores, 32 GB RAM, MacOS 14.7.2
SE, USM G3
Offline

Andrew Kolakowski

  • Posts: 9391
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 13, 2021 11:27 am

It's Jpeg2000 so behaves like any other wavelet based codec (eg. Cineform). Not really an exception. It just has this nature, so why not use it. It can be very useful option.
Offline
User avatar

Uli Plank

  • Posts: 25124
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 13, 2021 11:29 am

But why don't I get the same gains with Cineform?
My disaster protection: export a .drp file to a physically separated storage regularly.

Studio 19.1.3
MacOS 13.7.4, 2017 iMac, 32 GB, Radeon Pro 580 + eGPU
MacBook M1 Pro, 16 GPU cores, 32 GB RAM, MacOS 14.7.2
SE, USM G3
Offline

Andrew Kolakowski

  • Posts: 9391
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostWed Jan 13, 2021 1:09 pm

Cineform gains are only for preview (controlled by Timeline Proxy Mode setting). For exports Resolve always uses full quality decode and there is no control over it.
Offline

sulekh

  • Posts: 5
  • Joined: Tue Oct 01, 2013 11:39 pm

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostFri Jan 15, 2021 2:36 pm

After changing the preference in decode from debayer to decompression and debayer for r3d, the frame rates improved. Metal decompression was implemented in Resolve in 16.2.1.

Thanks Dwaine Maggart for pointing this out.

8K FF Ana to DNxHD (Half res Good) went from 15fps to 78fps
8k FF to DNxHD(Half Res Good) went from 32fps to 88fps

GPU usage has shot up to 15X and CPU usage has gone down 4X. I would still hope that Red would upgrade the Red Rocket X firmware to work with Big Sur for RRX since it allows Full Res debayer.
Offline

Andrew Kolakowski

  • Posts: 9391
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostFri Jan 15, 2021 4:00 pm

Why would you decode/debayer 8K at full resolution if you're going to just HD? You're gaining NOTHING, except loose a lot of time. I've checked and half res and quarter show no visible difference, so if you're creating some HD files for editing etc. you may as well use 1/4 decoding and have transcoding speeds way above 100fps.
Or are you talking about some final renders for 4K/8K?

I'm not sure if RED maintains Rocket card at all anymore?
Offline
User avatar

Uli Plank

  • Posts: 25124
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostFri Jan 15, 2021 4:15 pm

Those overpriced cards are long gone, Red supports CUDA now.

But don't worry, it's pretty fast under Metal too.
My disaster protection: export a .drp file to a physically separated storage regularly.

Studio 19.1.3
MacOS 13.7.4, 2017 iMac, 32 GB, Radeon Pro 580 + eGPU
MacBook M1 Pro, 16 GPU cores, 32 GB RAM, MacOS 14.7.2
SE, USM G3
Offline

sulekh

  • Posts: 5
  • Joined: Tue Oct 01, 2013 11:39 pm

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostSat Jan 16, 2021 10:54 pm

Thank you Andrew. I will test quarter res debayer. A majority of my work in the last year has been on Alexa Mini LF in 4.5K Open Gate and Sony Venice 6K FF. It was a breeze to to handle 3x DXL2 cameras in 8K FF on set yesterday. I wish I could wait for Apple's own chip for Mac Pro but summer in LA is brutal on 2013 Mac Pro hence the upgrade. I feel like a new Mac Pro with Apple chip may at least be 2-3 years away.
Offline
User avatar

Uli Plank

  • Posts: 25124
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostSun Jan 17, 2021 3:51 am

I'd expect 2 years. But some new top-of-the-line iMacs might be pretty impressive too.
My disaster protection: export a .drp file to a physically separated storage regularly.

Studio 19.1.3
MacOS 13.7.4, 2017 iMac, 32 GB, Radeon Pro 580 + eGPU
MacBook M1 Pro, 16 GPU cores, 32 GB RAM, MacOS 14.7.2
SE, USM G3
Offline

Andrew Kolakowski

  • Posts: 9391
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostSun Jan 17, 2021 11:11 am

sulekh wrote:Thank you Andrew. I will test quarter res debayer. A majority of my work in the last year has been on Alexa Mini LF in 4.5K Open Gate and Sony Venice 6K FF. It was a breeze to to handle 3x DXL2 cameras in 8K FF on set yesterday. I wish I could wait for Apple's own chip for Mac Pro but summer in LA is brutal on 2013 Mac Pro hence the upgrade. I feel like a new Mac Pro with Apple chip may at least be 2-3 years away.


You hardware will cope, just need to be clever about it :)
Offline
User avatar

roger.magnusson

  • Posts: 3802
  • Joined: Wed Sep 23, 2015 4:58 pm

Re: 2019 Mac Pro rendering YouTube 4k h264 video only in rea

PostSun Jan 17, 2021 12:39 pm

The latest rumor is that there will be a new Intel Mac Pro, as well as a smaller Apple Silicon Mac Pro next year. https://www.macrumors.com/2021/01/15/apple-two-new-mac-pro-models/

Image

Of course this isn't confirmed.

Return to DaVinci Resolve

Who is online

Users browsing this forum: dirk-pel, Fully_Loaded34, KrunoSmithy, Mads Johansen, qsipher, Robert John and 272 guests