NVLink, R3d full decode and what GPU?

Hi

1st of all I'd like to thank BM for the powerhouse of a tool that is Resolve studio... feels petty (excuse the pun;-) every questioning or moaning about software that does so much for so little.

But I would love a little more clarity over a few things?

1) I notice that NVlink is now supported? Does this mean tools like TNR will be more efficient when working on a dual GPU system with NVLink? At the moment TNR can bring a render to its knees... and doesn't scale well over multi GPU systems? If not why is NVlink inc and what is the benefit? (now or the future)

2) I have been testing a multi GPU system with the aim of upgrading my system asap (I currently have a HP Z8 2x 14core Xeon 2.6ghz with P2000 for GUI and Titan XP for compute.. this was based on a Recommend from BM resolve 15 config guide)

I tested 2x2080ti today to check if r3d Decoding and debayer on GPU scales with multi GPU systems and it seems to do it well.. ! Great! Will NVLink help also? now or in the future?

... and this question is not for anyone who get frustrated by others buying something they wouldn't... RE r3d files Is there any other benefit to be had from the RTX A6000 or RTX A5000 gpu's over the Geforce equivalents..? (I already understand the cost per Cuda core, warrantee, drivers, fan, power consumption, and reduced memory chip number for same Ram size arguments/points)

At the moment I can get 17.5 fps out (no grade just full premium 16bit 8k to 4k timeline decode) of my CPU/GPU in debayer only mode. This become 23fps if I engage decode only on GPU with R3ds BUT my Titan Xp gets maxed out and utilisation is 99% so there is nothing left over for grading!

I am thinking I will upgrade my GPU to a RTX A5000 as it should just about give me RT and leave me about 15%+ of the GPU available for light grading. This machine sometimes does editorial files with a light grade not just fin programe grading and this is why I don't proxy anything and speed is essential.

(if I go heavy with the grade I can switch back to CPU assistance for the de-waveleting and then I will be at 17.5 fps but with loads of GPU available)

In this senario I feel I can then add another RTX A5000 and be RT or better for everything I personally do regaularly apart from keyed or qualified TNR..

3) Anyone with direct experience of the RTX A5000 (not Quadro RTX 5000)? I realise they just shipped in very very limited numbers so this is not that likely. If not anyone with direct experience of the RTX A6000??

And is there any downside in having a separate GUI card that doesn't do any Resolve compute work (FUSION usages aside)

ALSO PLEASE could BM please do the Config guides for Resolve again??

thank you

Michael

https://www.pugetsystems.com/labs/artic ... 5372943847 See the Disqus at the bottom

Hi Dario

Thank you for your post. I probably should have referred to the Puget blog as I had read as much as I could on the Puget site before posting.

.. the discussion attached to the link you posted is informative but the issue is that in testing GPUs Puget only turn on the debayer portion of the GPU acceleration of Red files. The computationally heavier 'de-wavelet' part is not engaged so they they are def not taxing the GPU enough as the CPU is the limiter. (I suspect Puget keep it this way so their tests can sit in a historical context and full R3d decode is a very recent thing within resolve)

If they re-did all their A6000 tests with a FULL R3d GPU decode engaged everything would be super relevant for me and my needs .. ... Their 32 core Core CPU would sit near idle and we would really see what the A6000 can do...

I got a near doubling of FPS today using 2 GPUs over one going from 8k Red footage to a DnxHR file on 4k timeline

Screenshot 2021-05-10 at 23.09.48.png (139.5 KiB) Viewed 1361 times

as you see above from Puget when testing multi Gpu there is no difference if the CPU can't feed the GPU fast enough.. and this is what happens if the GPU decode option is set to debayer only.

thanks again

Michael

If I'm correct RED processing is actually scaling quite well with number of GPUs.
Using CPU to decode RED (do they show CPU load?) can massively limit speed of the whole system (specially with such a fast GPUs). JPEG2000 decoding on CPU has its limits (+ you also encode DNxHR on CPU, so system balancing is badly optimised for Puget test).

If I'm correct RED processing is actually scaling quite well with number of GPUs.
Using CPU to decode RED (do they show CPU load?) can massively limit speed of the whole system (specially with such a fast GPUs). JPEG2000 decoding on CPU has its limits (+ you also encode DNxHR on CPU, so system balancing is badly optimised for Puget test).

Yes it does seems to scale very well.. I have 2 small concerns. 1st is that a single A5000 is not enough and then if I buy 2 the 2nd concern I have is that there are 3 players in this game. Nvidia (who helped Red put wavelet decoding onto GPU). Red who then create the SDK that all Red decoding needs to use.. And BM who need to implement the SDK within their Resolve context. My worry is things might change and if I am going to buy 2 x RTX a5000 cards I want things not to break. For example I tried some very old r3ds I have and they can't be decoded on GPU.

I had a nicely balanced system for 6k or less Dragon r3ds now I want this system upgraded for r3ds of up to 8k!

It all may break regardless of number of GPUs in your system, so this is always a risk.
Those specific implementations are always on the edge and as you pointed can break at any time (new GPU, RED not updating SDK etc.).
Unfortunately JPEG2000 decoding on CPU is relatively slower than on GPU, so even crazy number of cores may not help. Although I think 64 cores system should have no problem going to eg 60fps on 8K.
All could be way simpler if RED was based on eg. Cineform For eg. 20% efficiency loss you would gain 10x decoding speed and no one would need so much those crazy GPUs/CPUs anymore RED realised it already and KOMODO is based on DCT based way faster (compared to Jpeg2000) codec. Maybe their next 'big' camera will drop old REDCODE as well.

1 hint - remember that RED should scale well with partial resolution decoding, so unless you really need full decoding then setting 1/2 or 1/4 decoding on 8K sources makes things going sooo much faster. For final exports - you have to pay the price of 8K resolution

Maybe you are right and Red move away from wavelets but I tested Komodo against our Gemini and Monstro and DCT is fast but for like for like quality at any given resolution data rates are considerably higher...

Once greatest strength is often ones greatest weakness ..

I second that. Being wavelet, R3D is very good at scaling by choosing partial decoding. Unless you desperately need full resolution while grading or editing, only go full res on final renders.

1) I notice that NVlink is now supported? Does this mean tools like TNR will be more efficient when working on a dual GPU system with NVLink? At the moment TNR can bring a render to its knees... and doesn't scale well over multi GPU systems? If not why is NVlink inc and what is the benefit? (now or the future)

NVLink support is bogus bait. I asked support about it, answer was that they don’t know what it does in Resolve besides not crashing (what a feature!). If you think you could just ask here and get an answer from any of the BMD staff members, just look at this thread: viewtopic.php?f=21&t=135984

Hi Hendrik

Ok I have reread the thread and thanks as I hadn't really full taken it on board.. So no memory pooling I had hoped it was all in service of allowing things like TNR and advanced SNR to work better across multiple GPUs.

Hopefully BM is just managing our expectations while they work out what is possible and will surprise us with some super feature later in the year...

thank you

Michael

1) I notice that NVlink is now supported? Does this mean tools like TNR will be more efficient when working on a dual GPU system with NVLink? At the moment TNR can bring a render to its knees... and doesn't scale well over multi GPU systems? If not why is NVlink inc and what is the benefit? (now or the future)

NVLink support is bogus bait. I asked support about it, answer was that they don’t know what it does in Resolve besides not crashing (what a feature!). If you think you could just ask here and get an answer from any of the BMD staff members, just look at this thread: viewtopic.php?f=21&t=135984

Maybe you are right and Red move away from wavelets but I tested Komodo against our Gemini and Monstro and DCT is fast but for like for like quality at any given resolution data rates are considerably higher...

Once greatest strength is often ones greatest weakness ..

Well- you may use wavelets as well, just simpler approach. I think Cineform is probably 20% worse than Jpeg2000 but decoding speed is probably 10x bigger.
I don't think good DCT codec should be much worse.
Unfortunately due to RED encryption we can't measure it.

Don't forget that RED made so many bold claims which we now know are pure crap (including one which say REDCODE is not Jpeg2000 based, but something mysterious ).
I assume you've seen videos about RED from JinniTech.

note.

Pulled the Titan XP and bought a RTX A5000 and it works very well! about 50% faster for what I needed it for..

It even oddly seems to help CPU utilisation in some situations!

am considering pulling my GUI GPU and adding a 2nd A5000 as r3D decoding works well across 2 identical cards... will try linking them when/if I do!