DNxHD doesn't like AMD Threadripper

I did the following

* officially installed the officially CentOS 7.3 supported platform with an older GTX 650 (as 1080 ti has issues with CentOS 7.3)

* Installed the Decklink (boy DKMS not included with CentOS 7.3 !? weird) support software

* Installed 14.1.1 (free as to not waste time registering my studio License)

* captured short video in mediaexpress

* loaded video clip onto timeline

* added a "render job" using DNxHD 444 (and DNxHD or DNxHR crashes)

* clicked start render and program segment faults as seen in image:

* created this new thread requesting official support for (guessing as I am a developer as well) fixing whatever optimization flag doesn't work on AMD CPUs)

--Doug (dx9s)

I found the "Select the help menu and then select 'Create diagnostic log on desktop'" in another thread.

Attached is the log

Additionally DNxHD doesn't work period under AMD Threadripper.

Cannot import or export DNxHD on an AMD 1950X. Can only export uncompressed video or horrible MP4 quality or whatever that fairly poor GoPro CineForm codec. DNxHD is fairly decent (when it works under Intel CPUs) and only option as ProRes export is disabled/blocked. Apple makes it expensive in attempt to push customers to their 'average' hardware platforms.

I tried a few random things on internet like disabled SMT (16c/16t), disabling cores down to 4c/4t. Didn't make any difference.

I figured out how to install a moderately old but new enough version of FFMPEG 2.6.8 under CentOS 7.3 and command line transcoded the source AVI (captured from mediaexpress) file into DNxHD compressed MOV and a ProRes compressed MOV.

ProRes will import onto timeline. But even browsing to the folder with the DNxHD MOV file inside the "media" view inside Resolve causes it to immediately crash. Had to move to a separate folder as ProRes *WAS* in same folder as DNxHD MOV file.

All three files open fine (once the required gstreamer support library is installed) from totem (gnome's default media player).

All three files playback (import into and export out of Resolve) fine on my Intel laptop, but I can't hear anything because it's a laptop and Decklink cards require PCIe desktop and Resolve only supports Decklink for audio under Linux.

I still wonder why Resolve is linked against pulseaudio on linux:

Code: Select all

[dx@x399 ~]$ cd /opt/resolve/bin/
[dx@x399 bin]$ ldd resolve | grep pulse
   libpulse.so.0 => /lib64/libpulse.so.0 (0x00007f97982bc000)
   libpulsecommon-10.0.so => /usr/lib64/pulseaudio/libpulsecommon-10.0.so (0x00007f9794556000)

Perhaps the developers PLAN on adding pulseaudio support but is isn't stable at the moment. -- heck if they only added pulseaudio support into the paid version, that would be good.

--Doug (dx9s)

Cineform at highest quality setting is better than best DNxHD/R mode, so not sure what are you talking about.

Doug,

I wouldn't use threadripper on Linux yet, the kernel doesn't have the optimizations for Ryzen.

I'm running a 1950x on Windows 10 and it's fine. My 3930k system with a 1080ti worked fine on Centos 7.3/7.4.

Any reason why you can't use windows? Unless you have the orange dongle, it's pointless to run a threadripper based Linux system for Resolve.

This machine was recently built to perform several tasks, of which running Resolve is one of them. Windows presents additional problems that is easy to avoid (don't use windows).

I don't understand the argument about the kernel. If you are saying that CentOS (7.3 and one could argue 7.4) isn't optimized for current hardware, that is true -- but I have to use it to demonstrate the issue in order to hopefully get support. The argument that you should use older hardware on Linux is a confusing one as well. I discovered this problem on a current ("unofficial") Ubuntu install on same machine, but in an attempt to try to avoid finger pointing I installed the official CentOS and still getting some wagging fingers!

I haven't worked enough with (GoPro) Cineform codec to make good observations other than it significantly slower (like seconds to minutes, or easily 60:1 in speed). Having to (abandon) work around a codec issue when DNxHD should work -- this should not be an option.

--Doug (dx9s)

Cineform is not slow either- it should be about as fast as DNxHD in Resolve (if not faster).
Well- you have hardware/OS+Resolve issues, so any judgment on your machine is misleading.

Try it on Windows for free. DNX works on my Windows 10 Pro 1950x system.

Did you reach out to Blackmagic's support team before purchasing a system far-off from their config guide?

The 4.10 kernel got the ball rolling with Ryzen. CentOS 7.4 uses the 3.10 Linux kernel (iirc) and Red Hat backports lots of things to it. The point being is CentOS/RHEL is slow to add cutting edge features since their mantra is rock solid support/performance/uptime. So yes, older hardware is better in this case since it's supported. You can add another repo and update to 4.10+, but who knows if BMD would "support" it.

Windows 10 is more than fine and DNxHD/DNxHR will work FLAWLESSLY on it. Why don't you like Windows?

Cineform is not slow either- it should be about as fast as DNxHD in Resolve (if not faster).
Well- you have hardware/OS+Resolve issues, so any judgment on your machine is misleading.

I have too have experienced Cineform renders going way slower than ProRes or DNX in 12.5 and 14.

On Mac, PC or Linux?

On my simple Mac ProRes is fastest (25fps) then Cineform (around 19fps) and DNxHD at about 17fps.

I don't want this to become a rant but in re-reading this I added this PREFACE to warn those easily triggered in a easily triggered society!

I had for a short while an X299 Intel 7820X that wasn't stable so it went back...

BUT when it was running, DNxHD/R exports was way faster than real time. The disk IO seemed to be the limiting factor.

The hardware was (went through 3 ASUS and 1 Gigabyte motherboards) 7820X with 128GB of RAM, GTX 1080Ti, NVMe and SATA SSDs (plus a spinning drive for archiving). Just had too many issues outside of Resolve (stemming from things way more complicated for this topic).

Resolve was flawless on the Ubuntu machine (actually got one project done in just a few days). The other needs / applications had issues on X299 with buggy hardware, just couldn't reliably use USB3 hardware on it that works fine on other machines (including the replacement X399).

Now I have a X399 1950X (originally 1920X but it would not recognize all the RAM) with 128GB of RAM, same video card, *HAD* Ubuntu (temporarily have CentOS 7.3 to humor support) with same storage and everything except the DNxHD codec will work inside Resolve. The other non-Resolve tasks work flawlessly (and insanely fast barely using the CPU). USB3 couldn't be happier!

In so doing, notice huge improvement in NVMe IOs, was READ peaks of just under 2GB/s X299, where now close to 4GB/s X399 / not quite double for same NVMe M.2, don't recall exact numbers AT-THIS-MOMENT -- PCIe lanes on Threadripper are more plentiful and dedicated paths to CPU, making NVMe or USB 3 Gen1/2 operations more reliable. For those wanting to save time and want the fastest local storage, NVMe RAID0 for close to 12GB/s (that is GigaBYTES) is possible.

This isn't a question of "why are you doing that" as really it should be focused on "hey Support, you have an issue here, probably want to look into this as it will be common place hardware starting now" -- I just happen to be the first person to find this problem, that should be able to be fixed. It will take some time for the software developers to convince the upper end management that they actually need to spend $3000+ (I am well over $4000 now) on a new hardware platform to Q/A+Developers against.

I am not sure why I have to justify the hardware here! Or choice of OS. Windows 10, you don't own it and you can't control it. You don't have any choice of when Microsoft believes it should be update (and I could go on, and that is with Win10 Enterprise, I feel sorry for those with consumer versions). W10 doesn't support a few of my use cases and Dual booting is a lame workaround for running one application. Not trying to pick on anybody here, I just have found life outside of Windows, Mac OS X Could be option if they actually made decent hardware -- but Apple will still probably screw up the next rev for the Mac Pro like they did the trash can.

For work (at work, not personal hardware) I do use Windows 10 inside a virtual machine (happens to be inside QEMU/KVM) for the few things I *MUST* run to get paid (Microsoft SQL related primarily, and the silly ticket system which only runs on windows, leaving the Mac folks to have similar work around). I manage more Ubuntu Servers than people have fingers and thusly am I Ubuntu guy for desktop as it makes managing the servers easier. I don't use windows for anything personally and rarely for work. I am not your average Linux user. I am a real use case.

--Doug (dx9s)

On Mac, PC or Linux?

On my simple Mac ProRes is fastest (25fps) then Cineform (around 19fps) and DNxHD at about 17fps.

I would paid money to add ProRes onto Linux.. Say Blackmagic can work out a (up to, hopefully less) "$49" Prores upgrade for Linux.. I would pay it (enable Prores export into Resolve Studio).. I like the codec. And I have a feeling it would be insanely fast on current hardware for Linux.

It's already insanely fast under FFMPEG (of which Blackmagic has been told to stay away from / probably to keep Apple happy).

The only option is the Advanced Panel (which is $30K or something like that) which "unlocks" ProRes for Linux. I've researched!

--Doug (dx9s)

I don't want this to become a rant

It already is, moreover you have now opened two topics on this.

I would suggest you log a bug in case you have not done so already and patiently wait for a response.

BUT when it was running, DNxHD/R exports was way faster than real time. The disk IO seemed to be the limiting factor.

When it comes to pure encoding those codecs can do e.g. 1000fps for HD on 10 cores+ machines.
They will be about always limited by "host application".

It already is, moreover you have now opened two topics on this.

I would suggest you log a bug in case you have not done so already and patiently wait for a response.

The other thread was inquire into others with Threadripper soliciting input and luckily discovered windows 10 seems to work. I say luckily because I've opened a ticket with them and BMD has not "qualified" Threadripper CPUs/Systems for even Windows use. I didn't see any qualified list, but trying to search for one -- I am sure one exists someplace.

They plan to test for more recent hardware (for Windows and Linux) soon but they sound busy so it will probably take some time. I responded with some ideas (suggested CentOS 7.4 as the GCC/buildchain should be newer and might produce binaries that don't cause Segment Faults) ..

Unfortunately Intel and AMD CPUs are not 100% binary compatible (very very close). One issue I know about is 0x66 opcodes behave different on Intel depending if it is in 32-bit or 64-bit mode. AMD wrote the book on 64-bit x86 instructions (literally, Intel licensed it from AMD). See https://patchwork.kernel.org/patch/5821421/ for an example of a patch to fix the 0x66 op-code difference.

I suspect something like this is in the DNxHD code as compiled under an older GCC compiler (note: CentOS 7.3 is kind of old) and is a binary that contains some op-codes that are not cross CPU compatible. The 0x66 issue produces a similar Segment Fault issue. *worse* case, they might end up having to make two installs using two different builds (one targeted for Intel 64-bit CPUs, one for AMD). I really hope that isn't necessary and an updated GCC toolchain can fix it. (one binary build for both CPUs)

--Doug (dx9s)

Peter Chamberlain recently mentioned on this forum he was in the midst of revising the hardware guide to include Threadripper, so hopefully the light is coming.

I had to use Google to search for the document as it wasn't obvious from the Blackmagic website:

http://documents.blackmagicdesign.com/D ... _Guide.pdf

For those wanting to find the build guide. I wish I knew this document before spending a lot of time and money.

Now I need to decided to try to return what I can and recoup that cost, or wait (could easily be several years) for the developers to make any progress. Each day that passes is costing me money in recoverable costs. I estimate I will have lost well over $1000 in non-refundable parts alone.

--Doug

Really?

https://www.blackmagicdesign.com/suppor ... and-fusion

it could not be more "visible"

Sorry, but you can blame only yourself. Spending a lot of money without proper research may end up exactly like your case.

Yes. Now that I know what to look for, it is pretty simple.

Adding a reference to the PDF file (i.e. Linux_Installation_Instructions.pdf) within the downloaded installer would be nice.

--Doug (dx9s)

I would paid money to add ProRes onto Linux.. Say Blackmagic can work out a (up to, hopefully less) "$49" Prores upgrade for Linux.. I would pay it (enable Prores export into Resolve Studio).. I like the codec. And I have a feeling it would be insanely fast on current hardware for Linux.

You can render ProRes with Advanced Resolve for Linux. But you need to buy the $29,995 panels in order to do so.

BMD is not in charge of ProRes licensing -- that's Apple. They make the decision who can use it and how much it will cost.

Is this still the case with AMD Threadripper and DNxHD???

Is this still the case with AMD Threadripper and DNxHD???

On Linux, yes. Threadripper + CentOS 7.4 + Resolve studio 14.3 still has the dnxhr incompatibility.

Smooth as butter in Windows, especially with raided nvme boot drives.

Is there an update for when the DNxHD codec might work on AMD processors under Linux? Still not working in v15 b5.