Page 1 of 1

Exporting for grading?

PostPosted: Tue Nov 27, 2018 5:05 am
by Rod Duarte
Hi there!

So, being new to Resolve, I edited one video on it and by the end of the editing process, it was SO slow that I couldn't bring myself to wait for the rendering before I could color grade, it got me so frustrated that I skipped grading altogether.

But in order to avoid this in the future, I'd like to figure out a way to prevent it.

So the first thing I'm thinking, is to actually publish the movie to a lossless format once I finish edits, and then import the whole movie into a new project and grade that way.

It'll make some things harder because if it's a video with shots in different environments and different lighting, it'll be more difficult to grade separately as opposed to doing it while I still had them in clips format, right?
But, if my machine is suffering to do it, then this would be a viable workflow path, right?

Thanks in advance!

Re: Exporting for grading?

PostPosted: Tue Nov 27, 2018 8:29 am
by peterjackson
Did you try optimized media, proxy mode, user cache?

Sent from my SM-N950F using Tapatalk

Re: Exporting for grading?

PostPosted: Tue Nov 27, 2018 9:53 am
by Marc Wielage
Rod Duarte wrote:So, being new to Resolve, I edited one video on it and by the end of the editing process, it was SO slow that I couldn't bring myself to wait for the rendering before I could color grade, it got me so frustrated that I skipped grading altogether.

The D500s will not help much on your machine. Even with the bigger D700s, the Trashcan Macs labor under a stressful load.

What specific kind of camera files are you using? For editing, you could always use optimized media to lighten the load, and then at the very end, switch back to the original media for color. If the files are massive (particularly 8K Raw), you will find you're going to need a bigger machine with faster CPUs and more powerful GPUs. But I have done entire features on even smaller Trashcan Macs and limped by, just debayering to lower resolutions, cutting the project up into smaller reels, using very fast drives, and being careful about not overloading my available system resources.

Re: Exporting for grading?

PostPosted: Tue Nov 27, 2018 5:19 pm
by Rod Duarte
All of the footage is from a GoPro HERO 5.

So it's in H.264, right?

The original footage is in 2.7K 60fps, but my projects are set to 1080p.

Re: Exporting for grading?

PostPosted: Tue Nov 27, 2018 5:38 pm
by Rod Duarte
peterjackson wrote:Did you try optimized media, proxy mode, user cache?

Sent from my SM-N950F using Tapatalk


Yes, no and yes.

Those two take FOR-EVER.
And at one point the red bar just stopped turning blue. But THAT I think was because of disk space. I solved that issue. Let's see what happens on the next video.

Re: Exporting for grading?

PostPosted: Tue Nov 27, 2018 7:25 pm
by Jim Simon
You're working with difficult media on an underpowered system. Things will take longer.

Re: Exporting for grading?

PostPosted: Tue Nov 27, 2018 7:50 pm
by Rod Duarte
Jim Simon wrote:You're working with difficult media on an underpowered system. Things will take longer.


See, that's not helpful at all.

I should convert the source media to another format before bringing them into Resolve then, yes?
Which format?

Re: Exporting for grading?

PostPosted: Tue Nov 27, 2018 8:00 pm
by Rakesh Malik
Rod Duarte wrote:
Jim Simon wrote:You're working with difficult media on an underpowered system. Things will take longer.


See, that's not helpful at all.

I should convert the source media to another format before bringing them into Resolve then, yes?
Which format?


Your best bet is probably ProRes.

Re: Exporting for grading?

PostPosted: Wed Nov 28, 2018 12:53 am
by Rod Duarte
Rakesh Malik wrote:
Rod Duarte wrote:
Jim Simon wrote:You're working with difficult media on an underpowered system. Things will take longer.


See, that's not helpful at all.

I should convert the source media to another format before bringing them into Resolve then, yes?
Which format?


Your best bet is probably ProRes.


Thank you. Will try that. Now, Resolve can do it in the "media" tab, right?
Or should I use something else to "ingest"?

Re: Exporting for grading?

PostPosted: Wed Nov 28, 2018 2:03 am
by Uli Plank
You can put all your media into a timeline, even doing a rough cut, and then go to the Deliver page. Switch it to individual clips and the desired format (like ProRes or ProRes LT) and let Resolve render.

Work with those media until your creative part is done and replace it with original media before final render.

Optimized Media would be an alternative, but I found that not as reliable at times.

And do some reading, Resolve is a powerful and complex software. The keywords here would be offline/online workflow.

Re: Exporting for grading?

PostPosted: Wed Nov 28, 2018 3:31 am
by Marc Wielage
Jim Simon wrote:You're working with difficult media on an underpowered system. Things will take longer.

Jim has provided an absolutely correct answer. Figuring out an efficient workflow is a hugely important task for post.

My advice would be to convert all the H.264 files prior to editing and post to a simpler, low-overhead format like ProRes 422 (as Uli advises above). You won't lose anything and they'll run much more smoothly on your Mac.

Re: Exporting for grading?

PostPosted: Wed Nov 28, 2018 6:48 am
by John Griffin
I'd persevere with Optimised Media + Render Cache and avoid transcoding to another lossy codec like Pro Res. I've experienced visible image quality loss going from H.264 to ProRes 4.2.2. Transcoding to a lossless codec would be preferable in terms of preserving the data in the original files but you may find you will need a lot of storage and it will take a long time. If you are new to Resolve then the initial setup and optimisation can be a bit daunting but it will reap benefits when optimised. For example having a separate SSD just for Optimised media / Render Cache. From your system it looks like the GPU is the weak link and for Resolve this is what it uses for most of the 'heavy lifting' so far be it from me to advise another NLE on a BM forum you may be better off using one less reliant on your workstation's weak area.

Re: Exporting for grading?

PostPosted: Wed Nov 28, 2018 8:42 am
by Sam Steti
Hey,
My opinion : transcode any gopro footage to ProRes 422 or 422 HQ before working on these clips in Resolve (or export - from a resolve TL - any h264 -> PR and then get rid of initial clips)...

Doing it from another software allows you to use Resolve in the meantime, but Resolve does it ok (individual clips, beware the naming).
BTW, it looks like Motion takes full advantage of dual GPUs now (I can see it when I monitor it), maybe try it to convert ... Or even Compressor, which in the end (that's to say 10 years after it competitors :D ) has a 64 bit version now

Re: Exporting for grading?

PostPosted: Wed Nov 28, 2018 1:26 pm
by Jim Simon
Rod Duarte wrote:See, that's not helpful at all.

I should...


I think my analysis suggests a couple options.

1. A better system. (I recommend building your own. You'll get more performance for your money.)

2. Less difficult media. This can be proxies, transcodes, or even a different camera. The new Pocket 4K records ProRes, which is very easy to edit. External recorders also offer ProRes and DNx recording options.

3. Both of the above.

Re: Exporting for grading?

PostPosted: Wed Nov 28, 2018 6:44 pm
by Rakesh Malik
Rod Duarte wrote:Thank you. Will try that. Now, Resolve can do it in the "media" tab, right?
Or should I use something else to "ingest"?


The simplest way is probably to use ProRes as your optimized media option, and generate optimized media for anything you decide to put in your timeline.

A bit less simple is probably just to import all of your footage into Resolve and make a stringout timeline for each bin and render them out as individual clips, then edit and color grade those.

Re: Exporting for grading?

PostPosted: Thu Nov 29, 2018 4:33 am
by Marc Wielage
John Griffin wrote:I'd persevere with Optimised Media + Render Cache and avoid transcoding to another lossy codec like Pro Res.

It's 8-bit H.264 -- the image has already been stomped on to the point that transcoding to ProRes 422 (or HQ) won't do any harm, provided the levels are good.

This is basically like taking very dirty water in a cracked bottle, and I'm telling you to put it in a bigger, cleaner bottle. The bigger, cleaner bottle (422) doesn't really make the water any less dirty, but it is easier to drink from.

When you get down to it, Optimized Media is basically transcoding in the background -- the only advantage is that it's automatic. The disadvantage is you have no control over where the media is and how it's named, plus there are risks in having the media disconnect. If you transcode the media manually, you can back it up, you can move it around, you can easily manually reconnect, and you can control the codec very carefully.

Re: Exporting for grading?

PostPosted: Thu Nov 29, 2018 7:05 am
by John Griffin
Marc Wielage wrote:
John Griffin wrote:I'd persevere with Optimised Media + Render Cache and avoid transcoding to another lossy codec like Pro Res.

It's 8-bit H.264 -- the image has already been stomped on to the point that transcoding to ProRes 422 (or HQ) won't do any harm, provided the levels are good.

This is basically like taking very dirty water in a cracked bottle, and I'm telling you to put it in a bigger, cleaner bottle. The bigger, cleaner bottle (422) doesn't really make the water any less dirty, but it is easier to drink from.

When you get down to it, Optimized Media is basically transcoding in the background -- the only advantage is that it's automatic. The disadvantage is you have no control over where the media is and how it's named, plus there are risks in having the media disconnect. If you transcode the media manually, you can back it up, you can move it around, you can easily manually reconnect, and you can control the codec very carefully.

I've seen increased banding in skies doing this very thing which seems counter intuitive but that's what happened. I had to use an uncompressed codec or a very high flavour of ProRes 4.4.4 to avoid it

Re: Exporting for grading?

PostPosted: Fri Nov 30, 2018 6:57 pm
by Rod Duarte
This is all great feedback. I'll be working on a new video next week and I shall test the suggestions and come back with the findings to keep this going. You guys are great!

Re: Exporting for grading?

PostPosted: Sun Dec 02, 2018 5:34 am
by Rod Duarte
So this is interesting.

In this video here, the guy says the paid Studio version handles H.264 better.

Which would mean that would solve my problems.

Is this true?


Re: Exporting for grading?

PostPosted: Sun Dec 02, 2018 10:09 am
by Andy Mees
Rod Duarte wrote:In this video here, the guy says the paid Studio version handles H.264 better.


Yep, have seen that reported many times ... no idea how well it would impact your workflow Rod, but I will say this. If you are using Resolve routinely and/or professionally, it's just plain good karma to throw Blackmagic Design the small amount of money that they charge for the Studio version ... and as that will unlock some good stuff along the way then its a win win.

Re: Exporting for grading?

PostPosted: Sun Dec 02, 2018 10:43 am
by Jean Claude
Rod Duarte wrote:So this is interesting.
In this video here, the guy says the paid Studio version handles H.264 better.
Which would mean that would solve my problems.
Is this true?


It is true that the Studio version offers a lot of features compared to the FREE version but you must read this first :

DaVinci Resolve 15 Configuration Guide : 09 Apr 2018
https://documents.blackmagicdesign.com/ ... _Guide.pdf

I think that many want to use this software because there is a FREE version (at first) but does not have the necessary hardware to use it in a pleasant way and comes here complaining that it does not work.

After, the best friend you will have is the manual and after to train and test and test :)

(maybe ask a friend to lend a dongle for a test on your current machine to do a benchmark test?)

Re: Exporting for grading?

PostPosted: Sun Dec 02, 2018 5:14 pm
by Rod Duarte
You guys make a LOT of sense.

I'm always happy to pay for software I use, so I had every intention of buying this one anyway.
But, it's great that it has a fully functional free version, so I can see if it's really for me before I spend the money.

What I really don't want to see happening, is paying for it and for it to NOT solve my problem.
One of the reasons I'm trying to ditch Premiere is because Premiere is so poorly optimized, in any platform...

Besides my mac, I also have a PC:
https://uk.pcpartpicker.com/list/H4CBq4

Its hardware is better than the mac in some ways, especially the video card (1080Ti).
But it's got a 6-core processor and 32gb RAM. The mac has 12 and 64 respectively (But the mac is still on DDR3 memory, don't know how much of a difference that makes for this).

I use the PC for gaming and streaming, and the mac for the offline productivity stuff. Pro Tools, Adobe suite etc.
Would Resolve run better on the PC, though?
Thoughts?

Re: Exporting for grading?

PostPosted: Mon Dec 03, 2018 12:12 am
by Andy Mees
Rod Duarte wrote:Its hardware is better than the mac in some ways, especially the video card (1080Ti).
But it's got a 6-core processor and 32gb RAM. The mac has 12 and 64 respectively...
Would Resolve run better on the PC, though?
Thoughts?


Sounds like a situation where having access to pretty much fully functional free version is ideal... just install and test side by side with your exact same current project, workflow and media . Proof is in the pudding.

FWIW my work PC is a 4 core with 24GB RAM and a 1060. I wouldn't wish it on anyone, but I only work in HD and get by pretty much fine. That said, I'm not cutting with 100% H264 sources.

Re: Exporting for grading?

PostPosted: Tue Dec 04, 2018 8:18 pm
by Rod Duarte
Interesting video on this:



Particularly relevant to this discussion is the first thing he says: Free version DOES work with H.264 well even in 4K.

But, then he says it can only work with one GPU. THIS is a big deal for me, right? Since even if I have older GPUs in this mac, it'd still be a leap forward if it could use both of them...

Re: Exporting for grading?

PostPosted: Tue Dec 04, 2018 9:02 pm
by waltervolpatto
for the "I'm not going to pay for a full license before/ because............".

How much your time is worth? The full version will have tools that will speed up things, convert that into money and you will see which cup full up first.

Re: Exporting for grading?

PostPosted: Wed Dec 05, 2018 2:43 am
by Sulo Kokki
People can value $300 very differently, depending on their circumstances, yet still use Resolve.

I do agree that anyone who uses Resolve on a regular basis (job / hobby) should get the Studio version once they settle in. It is indeed a bargain, considering what the price was last decade :lol:

Re: Exporting for grading?

PostPosted: Wed Dec 05, 2018 5:33 pm
by JPOwens
Rod Duarte wrote:hardware is better than the mac in some ways, especially the video card (1080Ti).
But it's got a 6-core processor and 32gb RAM. The mac has 12 and 64 respectively (But the mac is still on DDR3 memory, don't know how much of a difference that makes for this).


Resolve shares its image processing between CPU and GPU on an input/process/output hierarchy. Your H264 woes really stem from the decode being reliant on the CPU and media buss throughput. An optimized system can handle UHD H264... more matched GPUs in an intelligent array will get you into real-time performance, which was daVinci's strength up until the application was ported to commodity platforms. It used to require an SGI Crimson for the User Interface. The rest was in a small refrigerator stuffed with cards and wires. You used to have to buy another card for another Power Window. Marc can furnish the asking price. I never bought a 2K.

Anyway, the current Studio license sells for less than one of the balls on the Renaissance control panel, so why anyone would not invest in its development is in a mindset alien to my reality.

jPo, CSI

Re: Exporting for grading?

PostPosted: Wed Dec 05, 2018 6:35 pm
by Jason Conrad
I've recently done some very un-scientific benchmarking on hardware we have lying around the office, just for the basic operations that I do on nearly all the video I process. I've got the older BlackMagic egpu and 2 AMD WX9100s that I tested with the original 15" MBP with touchbar. I also tested one of the new mac minis, and the 12 core trashcan with D700s. I found that the extra GPUs weren't very helpful for transcoding/decoding R3D or H.264, though they do affect encoding h.264 and h.265 8bit HEVC.

For encoding Prores, Prores LT was significantly speedier than 422 or 4x4 (I know people are thinking, "duh," but seeing the numbers for myself was helpful). When you go to grade, you can just switch off the optimized media, grade the original h.264, and switch back to O.M. for playback as long as you're OK with not seeing your grade play back, which I know is a dealbreaker for a lot of people.

Oh, and Apple came out with 64-bit Compressor after I'd been compiling this data for a week, so I did a few tests with it, and it does seem to accelerate h.264 encoding, but I have't had time to put it though its paces on all the hardware combinations.

I looked a little bit at the difference between OpenCL and Metal, and didn't see much for what I was testing.

I also didn't see as much of a penalty for using "unmatched" GPUs as I was expecting, although my sample size of matched GPUs is low. The two 9100's offer the only fair comparison to anything else. The D700's can run as a pair, but the trashcans can't use eGPUs.

On my hardware, it's fastest to generate optimized media (Prores LT) on the trashcans, and then work with it on the laptop, and export H.264 or HEVC 8-bit from there. 10-bit HEVC, by the way, takes *EONS,* so until I get new hardware, HDR is out of the question :cry: .

Here's a link to my numbers in case anyone's curious. It's still a work in progress. I just used an in house clip to benchmark. I'd love to know if anyone knows of a more standardized test.

https://docs.google.com/spreadsheets/d/ ... sp=sharing

Re: Exporting for grading?

PostPosted: Mon Dec 10, 2018 10:27 pm
by Rod Duarte
Ok so I finally tried it again.
This time, I have 4 different takes of me playing drums, one of vocals and a couple for backing vocals.

Since I recorded the 4 takes with one camera and changing its position each time, I can't do the nice clap sync, unless I clapped to a click track exactly on the same beat everytime, which I could also not do because I was running the studio myself, so I had to set up like a 20 bar count off and then run into the live room, hit record on the camera and put on the headphones.

So to sync, I usually just import all the takes into the timeline, and sync all of them up using, say, a snare drum hit.
Then I drag in the finished audio recording and mute everything from the camera, sync that and I'm off to the races for the editing step.
But usually this is a painful process, because now I have the four video tracks going, and whenever I want to use one take, and it's not the one "on top", I have to cut a hole in all the takes that are above the one I want to use.\
So I thought, maybe this time I'll multicam it. Maybe Resolve is so awesome that it will even sync it for me if I select the "audio" option.
So I let it do that, and I THINK it worked, but I can't know for sure, since when I try to PLAY it back to finally do the multicam cuts... it absolutely chokes.

I should mention that this time, instead of 2.7k video, I went with 4K and flat color instead of GoPro color, because I wanted to grade this one in a specific way.

But I can't get it to even play the footage in realtime. There's no way I'll be able to edit this.

So I tried "Generate Optimized Files".
Nope.

Went back to Premiere and tried it over there.
So then, it failed to sync the takes for one thing. I fixed that manually but nope. Premiere is also not able to play it smoothly.

The next step is to try and convert the footage to ProRes (back on the mac).
But before I do that I wanted to make sure that will work, and if not, that I have more steps to attempt so here we are.
If I convert it all to ProRes, and it doesn't work, what else can I do?
Also I wanted to ask, right now I have all the media on an external hard drive that's connected to the mac via USB 3.0, and it's an older Porsche drive but it's 7200rpm.
So I ran a little test using BlackMagic's disk testing app. From what I see it's really not going to work if I keep the files on there so I should move them to my mac's internal SSD, yes?

Anything else I could do if that also doesn't work?
I'll attach the test results just for convenience as well.
Thanks guys!

DiskSpeedTest.png
DiskSpeedTest.png (225.62 KiB) Viewed 4059 times

Re: Exporting for grading?

PostPosted: Mon Dec 10, 2018 10:38 pm
by Jack Fairley
A single spinning disk is way too slow for 4K multicam. Try putting them on the SSD.

Re: Exporting for grading?

PostPosted: Mon Dec 10, 2018 11:46 pm
by Rod Duarte
Ok so I just did that. When I generated the multicam sequence, it happened SO MUCH FASTER that I was pumped, thought it was gonna work...

But nope. I still can't play the 4 clips in Multicam view in proper speed so I can cut.

So I tried going into Media Management, and chose to Transcode everything to ProRes 422 HQ.
Nope.

So, Generate Optimized Media. It's happening right now. Let's see.

Re: Exporting for grading?

PostPosted: Tue Dec 11, 2018 12:00 am
by Rod Duarte
Nope. Still can't do it.

What now?

Re: Exporting for grading?

PostPosted: Tue Dec 11, 2018 2:01 am
by Andy Mees
Rod Duarte wrote:Nope. Still can't do it.

What now?


Did you generate optimised media at a lower resolution processor friendly codec (to aid playback)? And is the optimised media on the SSD or elsewhere?

Re: Exporting for grading?

PostPosted: Tue Dec 11, 2018 2:56 am
by Rod Duarte
Yes, and yes.

Multicam just won't work.

I managed to edit the video using the good old manual cutting method.
In 4K and everything... just multicam seems to be too much.
I guess FOUR 4K streams at once is a bit much.

What really annoys me, though, is that no matter what I'm doing in Resolve, if I look at the Activity Monitor, the CPU and GPU are never even close to full 100% load.
I don't understand. If the system isn't managing and the software is struggling, why doesn't it just use more processing power? Judging but the fact it's around 50% idle, I'm pretty sure it would do everything smoothly if it took advantage of the ENTIRE CPU and GPU... Ugh!

Re: Exporting for grading?

PostPosted: Tue Dec 11, 2018 6:54 am
by John Griffin
Rod Duarte wrote:
What really annoys me, though, is that no matter what I'm doing in Resolve, if I look at the Activity Monitor, the CPU and GPU are never even close to full 100% load.

This would suggest there is a bottleneck in the data transfer from the source media and / or to the media drive where the processed data is being written i.e the CPU/GPU can't get the data fast enough to run at maximum capacity. FWIW I have my source media on a fast RAID 0 drive and I write my optimised media and Render Cache to a separate SSD ( not the OS drive). I wouldn't think having source media and optimised media /render Cache on the same drive would be optimal.

Re: Exporting for grading?

PostPosted: Tue Dec 11, 2018 7:17 am
by Uli Plank
Hardware decoding of H.264 or H.265 is not shown in the activity monitoring.

Re: Exporting for grading?

PostPosted: Tue Dec 11, 2018 1:44 pm
by Rod Duarte
Uli Plank wrote:Hardware decoding of H.264 or H.265 is not shown in the activity monitoring.


This is good to know!
But at this point everything is in ProRes 422 HQ...

So yeah, it's also all on the SSD now but it IS the boot drive.
I'll see if I can get my hands on an external SSD.
Hmmm but the fastest I can get for an interface on this mac is Thunderbolt 2, right?
Is that enough?

In any case, here's the video, for input, critique, what have you!

Re: Exporting for grading?

PostPosted: Tue Dec 11, 2018 11:25 pm
by Jason Conrad
Rod Duarte wrote:ah, it's also all on the SSD now but it IS the boot drive.


The internal SATA connection to your SSD is faster than TB2. The risk is that your boot drive will run out of space, because Resolve will be generating a lot of large files in the background without warning you when you get low or run out. Operating systems hate running out of space.