Feature Request - Danci Resolve Benchmarking

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

Robert Jones

  • Posts: 113
  • Joined: Mon Mar 14, 2016 4:29 pm

Feature Request - Danci Resolve Benchmarking

PostThu Jun 21, 2018 4:24 pm

Dear Blackmagic Design,

I was sitting here and watching cinebench and other benchmarking videos on Youtube, and it hit me. It would be so important to us the users if we had a Davinci Resolve Benchmark software.

Could you possibily make this for us? This would help us to have comparitive information and also see where bottle necks mey be in our systems (Ram, Graphics GPU, CPU, Harddisk). As you alsready have a harddisk benchmarking software it would be really beneficial to us to be able to have a Davinci Benchmark as how there is a Cinebench Benchmark.

Side note, it will also help to market Davinci resolve, as how you see Cinebench all over the place. It could be a defacto video editing / color grading benchmarking software. As I have not seen any benchmarking software as a editing tester.

Kind regards,
Robert
Mojave 10.14.3
Resolve 16. Beta 5 (Standard)
Radeon RX 580 8192 MB
Mac 5.1 2,66 GHz Quad-Core Intel Xeon, 24 GB 1066 MHz DDR3
Caldigit Pro HD, 8TB
Offline
User avatar

Jean Claude

  • Posts: 2542
  • Joined: Sun Jun 28, 2015 4:41 pm
  • Location: France

Re: Feature Request - Danci Resolve Benchmarking

PostThu Jun 21, 2018 5:49 pm

Hi,

Maybe try (old and obsolete but why not (I did it a long time ago) :
https://www.liftgammagain.com/forum/ind ... mark.3718/
:)

Do not forget the "Blackmagic Disk Speed Test" :)
"Saying it is good, but doing it is better! "
Offline

Robert Jones

  • Posts: 113
  • Joined: Mon Mar 14, 2016 4:29 pm

Re: Feature Request - Danci Resolve Benchmarking

PostSat Jun 23, 2018 9:43 am

Jean Claude wrote:Hi,

Maybe try (old and obsolete but why not (I did it a long time ago) :
https://www.liftgammagain.com/forum/ind ... mark.3718/
:)

Do not forget the "Blackmagic Disk Speed Test" :)


Thanks for the link. I saw this some time ago. can't recall when though, but yes this is a sort of workaround for now. A benchmarking tool for resolve would be perfect.
Mojave 10.14.3
Resolve 16. Beta 5 (Standard)
Radeon RX 580 8192 MB
Mac 5.1 2,66 GHz Quad-Core Intel Xeon, 24 GB 1066 MHz DDR3
Caldigit Pro HD, 8TB
Offline
User avatar

Micha Clazing

  • Posts: 220
  • Joined: Sat Apr 01, 2017 3:26 pm

Re: Feature Request - Danci Resolve Benchmarking

PostSat Jun 23, 2018 12:31 pm

This would actually be a great idea, not just to increase exposure to Resolve, but also for the BMD Resolve team to track performance improvements/regressions across Resolve versions and betas and on different hardware (Intel vs Ryzen, CUDA vs OpenCL/Metal).
Offline
User avatar

Marc Wielage

  • Posts: 5689
  • Joined: Fri Oct 18, 2013 2:46 am
  • Location: Hollywood, USA

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 24, 2018 3:20 am

One problem with creating a standardized benchmark is that a lot depends on camera source material, frame-rate, aspect ratio, as well as the number of nodes, the number of GPU-intensive nodes (like OFX and NR), and so on. So an 8K Red file with 10 nodes per shot would yield completely different speeds than a 4K CinemaDNG file with 25 nodes per shot, and then a ProRes HD file with 4 blur nodes, 2 NR nodes, and 40 color nodes would be different still. Which is more stressful for the computer? It depends.
marc wielage, csi • VP/color & workflow • chroma | hollywood
Offline

Hendrik Proosa

  • Posts: 602
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 24, 2018 9:16 am

Marc Wielage wrote:One problem with creating a standardized benchmark is that a lot depends on camera source material, frame-rate, aspect ratio, as well as the number of nodes, the number of GPU-intensive nodes (like OFX and NR), and so on. So an 8K Red file with 10 nodes per shot would yield completely different speeds than a 4K CinemaDNG file with 25 nodes per shot, and then a ProRes HD file with 4 blur nodes, 2 NR nodes, and 40 color nodes would be different still. Which is more stressful for the computer? It depends.

This is why it is a standardized benchmark - to remove this kind of variables and provide as pure performance evaluation as possible. Disk speeds should be eliminated completely by using a RAM disk or in-memory files because disk speeds are better measured with other tools. Standard benchmark should include different scenarios to stress different aspects, for example temporal operations, GPU processing, CPU processing, transcoding and so on. It does not matter if your 8K with X nodes yields something else than 2K with Y nodes because you compare 8K benchmark results with 8K, 2K with 2K and so on This is the whole point of benchmarking, to provide a clearly defined methodology for tests, whatever the tests actually are.
I do stuff.
Offline

Robert Jones

  • Posts: 113
  • Joined: Mon Mar 14, 2016 4:29 pm

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 24, 2018 11:08 am

Hendrik Proosa wrote:
Marc Wielage wrote:One problem with creating a standardized benchmark is that a lot depends on camera source material, frame-rate, aspect ratio, as well as the number of nodes, the number of GPU-intensive nodes (like OFX and NR), and so on. So an 8K Red file with 10 nodes per shot would yield completely different speeds than a 4K CinemaDNG file with 25 nodes per shot, and then a ProRes HD file with 4 blur nodes, 2 NR nodes, and 40 color nodes would be different still. Which is more stressful for the computer? It depends.

This is why it is a standardized benchmark - to remove this kind of variables and provide as pure performance evaluation as possible. Disk speeds should be eliminated completely by using a RAM disk or in-memory files because disk speeds are better measured with other tools. Standard benchmark should include different scenarios to stress different aspects, for example temporal operations, GPU processing, CPU processing, transcoding and so on. It does not matter if your 8K with X nodes yields something else than 2K with Y nodes because you compare 8K benchmark results with 8K, 2K with 2K and so on This is the whole point of benchmarking, to provide a clearly defined methodology for tests, whatever the tests actually are.



Thank you Hendrik, for such a clear response. I couldn't have said it better. There are so many ways that engineers could sit and figure out how to make this possible. And where footage is concerned. Back magic could easily have source files of 720, 1080, 2K, 4K & 8K, that everyone can use to do the video related test for download. Which then creates a "standardised" piece of footage. Doesn't need to be long, just enough to get a clear preformance metering, based on what the engineers know will be good enough.

There is nothing that is impossible, just have to speak / find the right person who believes the impossible must be possible some how. Heck resolve wasn't suppose to be possible, but yet here it is.

Greeting,
Robert
Mojave 10.14.3
Resolve 16. Beta 5 (Standard)
Radeon RX 580 8192 MB
Mac 5.1 2,66 GHz Quad-Core Intel Xeon, 24 GB 1066 MHz DDR3
Caldigit Pro HD, 8TB
Offline

Robert Jones

  • Posts: 113
  • Joined: Mon Mar 14, 2016 4:29 pm

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 24, 2018 11:15 am

Dear Blackmagic Design,

The bench marking software could come with snippets of different resolutions packed into it. And that played in loop internally and used for an internal timeline with varying transitions etc. There are so many ways this can be done, but this is something that engineers excel at, problem solving.

We simply need a way to test our systems. This can go a long way with beta testing, and also problem solving for users of your software such as myself when trying to figure out where the bottleneck could be.

Kind regards,
Robert
Mojave 10.14.3
Resolve 16. Beta 5 (Standard)
Radeon RX 580 8192 MB
Mac 5.1 2,66 GHz Quad-Core Intel Xeon, 24 GB 1066 MHz DDR3
Caldigit Pro HD, 8TB
Offline

Tero Ahlfors

  • Posts: 434
  • Joined: Fri Apr 03, 2015 3:02 pm

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 24, 2018 1:25 pm

You can get all kinds of test footage from camera manufacturers so if you know what you're getting you can test yourself. There is no single solution for all workflows.
Offline

Hendrik Proosa

  • Posts: 602
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 24, 2018 4:15 pm

Tero Ahlfors wrote:You can get all kinds of test footage from camera manufacturers so if you know what you're getting you can test yourself. There is no single solution for all workflows.

Benchmarking is not workflow optimization, it is performance measuring. Cinebench does not tell you if your workflow is valid or if you can even get something out of the door at all, it tells you how your system compares relative to other testers.
I do stuff.
Offline

Tero Ahlfors

  • Posts: 434
  • Joined: Fri Apr 03, 2015 3:02 pm

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 24, 2018 4:23 pm

Why not use Cinebench or whatever benchmark software instead then?
Offline

Hendrik Proosa

  • Posts: 602
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 24, 2018 4:28 pm

Tero Ahlfors wrote:Why not use Cinebench or whatever benchmark software instead then?

Well, maybe because OP asked for a way to benchmark Resolve, not general OpenGL or CPU performance? Resolve has a very distinct image processing engine with its ups and downs and general benchmarking does not have 1:1 correlation with what is happening in Resolve, far from it.
I do stuff.
Offline

Robert Jones

  • Posts: 113
  • Joined: Mon Mar 14, 2016 4:29 pm

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 24, 2018 6:19 pm

Hendrik Proosa wrote:
Tero Ahlfors wrote:Why not use Cinebench or whatever benchmark software instead then?

Well, maybe because OP asked for a way to benchmark Resolve, not general OpenGL or CPU performance? Resolve has a very distinct image processing engine with its ups and downs and general benchmarking does not have 1:1 correlation with what is happening in Resolve, far from it.


I'm so glad that certain people are realizing what I am talking about. It really has me a bit shocked that this hasn't been done as of yet for any editing application, to my knowledge. Everyone uses there own footage when trying to test resolve on YouTube or make comparisons to other editing application. Which do to the non-standardized method, isn't really applicable to no one but themselves. So many variables makes the comparison not comparable.

So we need a standardized method / benchmark equivalent to Cinebench, but for resolve, and who knows it can eventually become a industry standard, as we do need this as an industry as a whole.

Kind regards,
Robert
Mojave 10.14.3
Resolve 16. Beta 5 (Standard)
Radeon RX 580 8192 MB
Mac 5.1 2,66 GHz Quad-Core Intel Xeon, 24 GB 1066 MHz DDR3
Caldigit Pro HD, 8TB
Offline
User avatar

Antoine Grasset

  • Posts: 149
  • Joined: Sat Mar 14, 2015 7:05 pm
  • Location: Paris, France

Re: Feature Request - Danci Resolve Benchmarking

PostTue Jun 26, 2018 12:56 am

+1 Seems like a good feature idea.

It makes me think of the denoiser Neat Video, which has an internal tool that will test different GPU/CPU combos. It ouputs the speed for each variant and selects the best one for you, according to what the software needs.

Resolve could have an internal tool that benchmark your system from inside Resolve.
It could output disks speed, CPU score, RAM score, GPU score (if several GPU available, testing different combinations).
Offline

Robert Jones

  • Posts: 113
  • Joined: Mon Mar 14, 2016 4:29 pm

Re: Feature Request - Danci Resolve Benchmarking

PostWed Jun 27, 2018 11:16 am

Antoine Grasset wrote:+1 Seems like a good feature idea.

It makes me think of the denoiser Neat Video, which has an internal tool that will test different GPU/CPU combos. It ouputs the speed for each variant and selects the best one for you, according to what the software needs.

Resolve could have an internal tool that benchmark your system from inside Resolve.
It could output disks speed, CPU score, RAM score, GPU score (if several GPU available, testing different combinations).


+1
This also sound like a good option also to have.
Mojave 10.14.3
Resolve 16. Beta 5 (Standard)
Radeon RX 580 8192 MB
Mac 5.1 2,66 GHz Quad-Core Intel Xeon, 24 GB 1066 MHz DDR3
Caldigit Pro HD, 8TB
Offline

Zdenek Podsednik

  • Posts: 14
  • Joined: Tue Jul 26, 2016 9:45 pm

Re: Feature Request - Danci Resolve Benchmarking

PostWed Aug 15, 2018 9:38 pm

+1
Davinci Resolve Studio 15.3; Win10
Dell XPS8700 Bios A11, i7-4790 CPU @ 3.60GHz; RAM24GB; Radeon RX 480 by Asus w/8GB GDDR5
Offline

Andrew Kolakowski

  • Posts: 5411
  • Joined: Tue Sep 11, 2012 10:20 am

Re: Feature Request - Danci Resolve Benchmarking

PostWed Aug 15, 2018 10:23 pm

Scratch has such an option to measure different system components related to its performance.
It checks things like: how fast given source can be read from disk, decoded, how fast textures travel in GPU, CPU to GPU etc. Very useful.
Last edited by Andrew Kolakowski on Wed Aug 15, 2018 10:43 pm, edited 1 time in total.
Offline

Mark Grgurev

  • Posts: 224
  • Joined: Fri Nov 03, 2017 7:22 am

Re: Feature Request - Danci Resolve Benchmarking

PostWed Aug 15, 2018 10:26 pm

This sounds like an excellent idea!
Offline

johndaughtridge

  • Posts: 6
  • Joined: Wed Dec 05, 2018 12:49 am
  • Real Name: John Daughtridge

Re: Feature Request - Danci Resolve Benchmarking

PostSat Jun 15, 2019 11:33 am

I agree it would be fantastic to publish benchmarks and I'm wondering if this ever picked up steam? I benchmark resolve using my own test suite on my youtube channel but a standard across the software would be ideal.


This is my current performance benchmarking playlist. If anyone has ideas for aspects to test please let me know!
Thanks John
Offline

Carsten Sellberg

  • Posts: 805
  • Joined: Fri Jun 16, 2017 9:13 am

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 16, 2019 3:50 am

Robert Jones wrote: I couldn't have said it better. There are so many ways that engineers could sit and figure out how to make this possible. And where footage is concerned. Back magic could easily have source files of 720, 1080, 2K, 4K & 8K, that everyone can use to do the video related test for download. Which then creates a "standardised" piece of footage. Doesn't need to be long, just enough to get a clear preformance metering, based on what the engineers know will be good enough.

There is nothing that is impossible, just have to speak / find the right person who believes the impossible must be possible some how. Heck resolve wasn't suppose to be possible, but yet here it is.

Greeting,
Robert


Hi.

8 months ago I read this about some benchmarks of Intel's new Core i9-9900K CPU:

https://www.tomshardware.com/news/intel ... 37908.html

The conclusion was "never trust vendor's benchmarks". So I will like to suggest you to take initiative to create a Resolve User Group, with the purpose to create the wanted benchmarks.
If you ask BMD, am I sure you can have a sub forum here to coordinate and save save your work. But I will recommend you, to get real independence and do it somewhere else.

I think you can get a long way by creating different Resolve scripts. But as a see bench marking, is the difficult job to explain, your result to the users. May be all the explaining, can be done in a User Group?

Regards Carsten.
URSA Mini 4.6K
Offline

Michael_Andreas

  • Posts: 127
  • Joined: Sat Jan 05, 2019 9:40 pm
  • Real Name: Michael Andreas

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 16, 2019 4:26 am

What would be useful for benchmarking is to have a collection of camera clips of various levels of resolution, codec types and complexity: HD, FHD, QHD, UHD, 4K, 8K etc. Also a collection of projects with different levels of processing: simple transcode, noise reduction, color correction.

Then people with different systems could bench mark render times. Also, people who are having issues could see if the problem can be replicated with the benchmark camera clips and that might make problem solving on the forum easier.
_________________________________________________
DaVinci Resolve Studio 15.3.1.003 running on Win10 Pro, i7-6700K@4GHz,16GB RAM, NVIDIA GeForce GTX1070 8GB
Offline
User avatar

Dmitry Shijan

  • Posts: 607
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: Feature Request - Danci Resolve Benchmarking

PostSun Jun 16, 2019 1:52 pm

Nice idea! i like benchmarks too
All my custom made accessories for BMMCC/BMMSC now available here https://lavky.com/radioproektor/

Return to DaVinci Resolve

Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Max Besser and 48 guests