a dilemma: stick with a single Titan Xp, or add another one?

When reading the newest Configuration Guide, one thing surprised me considerably. On Page 42, in the "GPU selection" part, it's said that both OFX and NR nodes will only use a single GPU:

Config Guide.JPG (76.4 KiB) Viewed 2576 times

On the other hand, in many articles including this one https://www.pugetsystems.com/labs/artic ... n-Xp-1060/, it's shown that with the "torture" type of color grading (differing from lighter usage scenarios mainly by including some OFX and TNR node(s)), adding a second GPU (in this case, a Titan Xp) can actually increase performance by up to some 30% (in the case of the 16-core i9 CPU which I'm going to upgrade to).

GPU scaling.jpg (58.43 KiB) Viewed 2576 times

As I was about to also add another Titan Xp to my machine while changing the mobo to support the i9 CPU - I'd appreciate it if our BMD forum members commented on which of those contradicting info is more accurate; thanks!

Piotr

[Peter Chamberlain]

Hi, thanks for pointing out the inconsistently in the doc.. its looks like old text in error. I'll get that reviewed. OpenFX plugins that are not made for multiple GPUs will use a single but the other items are all multiple GPU processes.

Thanks Peter - and how about the NR?

Piotr

[Rohit Gupta]

NR also uses multiple GPUs.

Thanks guys

My last questions are perhaps a little too hardware-oriented, but who knows - perhaps you do know the answers, as well? Here I go::

- we know all GPUs should be the same model for Resolve to scale best; now: is there any reason for Resolve to treat the Titan Xp Collectors Edition as a different model that the regular Titan Xp? I don't seem to be able to find the Collectors Edition (of which I currently have one) any more, anywhere...

- while upgrading from my current X99 mobo with as much as 64GB of 2,133 MHz CL-14 RAM, I could make the switch much less expensive if I kept say 48GB of this memory, and sell my current mobo with the remaining 16GB (this way I'd avoid paying the premium for 48GB of say the 4,200 MHz DDR4 RAM sticks). BUT: how much would the overall Resolve performance suffer with the i9-7960X CPU interacting with XMP@2,400 RAM? Does it make sense to spend so much on almost entirely new platform and an expensive CPU/GPU while skimping on the RAM? On the other hand, the RAM can always be upgraded in the future..And considering the fact Fusion needs a lot of RAM (like 64GB minimm with 128GB recommended) - this probably would be the most sensible way of carrying out this massive upgrade. My question is though: which kind of Resolve's modus operandi is the most affected by slower memory?

Piotr

Piotr, we ran a bunch of tests with different « heavy » 4K timelines and we pretty much get double the performance with our second Titan Xp. Well worth the upgrade. Real-time 4K noise reduction at “better” settings + OFX plug-ins (such as FilmConvert) + blurs etc. So glad we upgraded. Makes real-world grading of 4K footage in 4K timelines feels like it’s mini DV!

The collectors edition is purely cosmetic. It’s the exact same card.

Piotr, we ran a bunch of tests with different « heavy » 4K timelines and we pretty much get double the performance with our second Titan Xp. Well worth the upgrade. Real-time 4K noise reduction at “better” settings + OFX plug-ins (such as FilmConvert) + blurs etc. So glad we upgraded. Makes real-word grading of 4k footage in 4K timelines feel like it’s mini DV!

Hmm...With your information, Francois, and looking at the Titan Xp's scalability for various CPUs - it almost looks like adding another Titan could as well be the only upgrade I need at the moment as with 8-core CPU the gains are highest... And most certainly, the most effective one I should definitely start with, perhaps waiting a bit till the 16-core i9 gets cheaper. In the meantime, my i7-5960X would have more time to serve its lifetime duty - considering it's only 2 years old at the moment; selling away "old hardware" will always mean financial loss but after say another 1-2 years, I'd certainly care much less than now. Plus - at plain timeline playback (like with cutting in the Edit page, with all grading turned off), it's still plenty fast enough giving me full 25/50 fps in my UHD/25p/50p projects respectively - and this is when CPU is taxed much heavier than the GPU... Sure - my exports would be much faster with a 16-core, but hey! - rendering out is the most trivial task of all currently offered by Resolve, and I always use this time for doing something else (when exporting short programs), and those longer one can be done overnight.

Thanks for that, mate! Have just ordered my second Titan Xp

Piotr

it should be taken into consideration, if actual gain by an additional card was archived on a more professional dual CPU mainboard or on a simple desktop workstation utilizing only one processor?

The "Torture" test GPU scalabillity table I posted in my OP is all about single CPU systems, with different core numbers. But of course in real world, what you mentioned is important (if only for the PCIe 3 Gen. lanes @x16 speed availability for the second/third GPU); that said even my somewhat old X99 mobo I does have enough lanes (with 40-lane CPUs) to run up to two GPUs at full speed.

Piotr

Have just added the second Titan Xp to my system, still with just 8 core CPU... Now I can play an UHD@50p, XAVC-I clip real time in spite of 4-frame TNR node, plus a film grain OFX node and one with usual CC... Impressive!

Piotr

Glad it worked out Piotr! Adding a second Xp makes a HUGE difference for UHD timelines. Better indeed to spend the money on that than on more CPU cores (when we can't afford to upgrade everything!).

And Martin, our 2012 12-core Mac Pro does indeed have two 6-core Xeon CPU's. Not sure if the gains would be as big with single CPU systems. But I would think if there's enough lanes for both GPU's to run at full speed then it doesn't make a difference.

Haha - it's not all so sweet after all, Francois. Yesterday I was after the very first launch of Resolve after the second Titan Xp addition, and the CPU/GPU taxation looked pretty balanced (which frankly was a surprise to me; after all I had doubled the GPU computing capacity while leaving that of the CPU untouched). When I reported about a success, I was in a hurry and didn't notice my cache drive in that particular project's Properties had been the same volume that the media seat on (a RAID 0 volume E:\), so - for example during caching - the system had to read from drive E:\, write my DNxHR HDR cache files to the same drive E:\, and then read them from this very same drive E:\ again for the final playback. Obviously - even with my E:\ volume being no slouch (4x 3GB WD Black 7200 HDDs) - with simultaneous reading from-writing to-reading from again, it clearly became a bottleneck; hence the CPU taxation was more or less balanced with that of the GPU (similar to what I had seen with just a single Titan Xp)...

However today, I re-directed caching to the by far the fastest drive in my system (the M.2 NVMe SSD disk) - and the real situation revealed before me. The two GPUs have such a high computing power, that to keep up the pace, the CPU gets overloaded - yet it's still too slow to balance with the GPU. And since it's overclocked to 4.2 GHz, the CPU temperature went up above 95 C - thank God I was monitoring them and immediately stopped the caching or I could fry my CPU due to the high rate it exchanged data with my double-Titan XP GPU). At the same time, each of the GPUs were only taxed* at much less than half of what I saw with only a single Titan Xp in my system)!!!

So obviously, with two Titan Xps on board, it's now my 8-core CPU which is the bottleneck - and a serious one, too... For the time being, I reduced my CPU's O.C. considerably down to 3.8 GHz (counter-productive, I know - but at least it doesn't overheat). Thus the ultimate conclusion is that - having said "A" by adding the second Titan XP - I just have to say "B", and replace my current 8-core CPU with the 16-core 7960X. And considering I also need the right motherboard and RAM to accommodate it, I still have to spend more than I did on my second Titan Xp

But at least it now really looks very promising; after I upgrade the CPU, my system will not only get balanced again (in terms of the CPU vs. GPU computing capabilities), but it will also become a seriously fast machine for Resolve . I only wish all this didn't cost so much

Cheers
Piotr

PS. * - a warning to all those who assess their GPU taxation by the percentage number as summarized by Windows 10 Task Manager: I have no idea where this number is taken from, but when it says e.g. 40%, please do yourself a favor and switch monitoring of one of the graphs from say "3D" to "Compute_0" - it may as well show that the GPU is laboring at 98% !!!

To François (or anyone running 2 identical GPUs under Windows 10)

Just wanted to ask you for a favor: could you please check something for me? When under full stress (e.g. Resolve caching some seriously graded clips - inc. TNR & OFX), please open Windows Task Manager and compare the graphs for each of the two GPUs. I have just noticed that the graphs named "Copy" tend to be quite different (by up to 15%) between my 2 Titan Xp cards; I have no idea what the "Copy" activity is all about - but looking at the "Compute_0" graphs, they tend to always be at almost exactly the same levels (when laboring during caching, usually at around 98%).

Since the balance of CPU and each of the GPUs has now become my obsession (and probably will stay that way - until I upgrade my CPU from 8 to 16 core, at which point I will hopefully stop watching all those graphs at all, haha) - I now wonder if all is OK with how my 2 Titan Xp's are working (one being a regular Xp, the other - the "Collectors Edition" one). Thanks in advance

Piotr

Anyone, please?

Perhaps I should be more precise: when testing my 2x Titan Xp capabilities, I run the "candle" test (is it the right name for the benchmark project with a short clip with a cup being put on a table, and various grade versions consisting of different numbers of TNR and Blur nodes)? Anyway: the good news is that all the grades run strong at the solid green 24 fps, including the most demanding one (66 Blur nodes). And this very version of said test's grading shows my 1st Titan Xp GPU laboring at full 100%, while the 2nd one doesn't even reach 85% (in their respective "Compute_0" graphs of Windows Task Manager's performance tab; the CPU only is loaded at some 30%).

Is this difference normal? YES - the Resolve UI is running on the 1st GPU; could this be the sole reason for the 15% load difference?

Piotr

PS. Frankly speaking, I'm more and more hesitant whether to spend another fortune on a new X299 motherboard, faster DDR4 RAM and the 16-core i9 CPU; yes - as I posted earlier in this thread - with 2x Titan Xp GPU, my current 8-core i7 is definitely the weakest link. But is it really a bottleneck, in my practical usage scenarios? Well - the thing is that the CPU really gets over-loaded only when fed with lots of data from both the Titans - but this only happens during rendering or caching. With rendering out from the Export page I can happily wait several more minutes - and as to caching, which with just a single Titan Xp was still required quite frequently, well - after I added the second Titan Xp to my system, I almost never need it to such an extent that I can't work without it! Even with TNR and a couple of OFX nodes, I can play my projects back fluently enough (even if not with the solid green dot fps). I only wish my x99 mobo had one more PCIe slot - I wouldn't need to use those 3M PCIe Risers, and keep the precious Decklink outside the PC case!

Double Titan Xp rules, my Friends