Resolve 16 - Object removal

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

Hendrik Proosa

  • Posts: 1250
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Resolve 16 - Object removal

PostTue Apr 09, 2019 9:15 am

This is an interesting one but current demo videos don't showcase the "neural" or AI part of removal. Could someone with studio licence test the object removal effect on a few shots where the neural engine would actually make a difference?

Good candidate shot would:
Have positional camera movement, not just nodal pan AND
have focus object cross different image planes (legs on closeup ground, body over distant bg for example).
Or have focus object cover an area which is not revealed during the shot.

Although AE-s content aware fill is kind of crappy, it is able to crunch through such shots (albeit miserably). I'm interested to see what Resolve can put up against it but can't try it myself currently.
I do stuff.
Offline
User avatar

Piotr Wozniacki

  • Posts: 1178
  • Joined: Tue Aug 02, 2016 12:17 pm
  • Location: Poland

Re: Resolve 16 - Object removal

PostTue Apr 09, 2019 9:27 am

Sorry I cannot help yet, but in the meantime - could you please share the links of the " current demo videos" you mention?

Thanks
Piotr
AMD TR 2990WX CPU | MSI X399 CARBON AC | 64GB RAM@XMP2933 | 2x RTX 2080Ti GPU | 4x 3TB WD Black RAID0 media drive | 3x 1TB NVMe RAID0 cache drive | SSD SATA system drive | AX1600i PSU | Decklink 12G Extreme | Samsung UHD reference monitor (calibrated)
Offline

berzerker

  • Posts: 8
  • Joined: Mon Apr 08, 2019 11:51 pm
  • Real Name: luke kenny

Re: Resolve 16 - Object removal

PostTue Apr 09, 2019 9:35 am

i have tried it on a few shot and can't get it to work well at all
OSX 10.15.7 - Mini Mac 2018 - 32 gig ram 2667 Mhz DD4 - AMD RADEON RX 5700 XT - Davinci Studio 16
Offline

Hendrik Proosa

  • Posts: 1250
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Resolve 16 - Object removal

PostTue Apr 09, 2019 9:51 am

NAB demo had the baseball scene (starts at 1:17:30):


What's new in Resolve 16 had the runners shot (starts at 15:36):


Crappy AE castle demo which uses actual content aware fill can be seen here:
I do stuff.
Offline

Hendrik Proosa

  • Posts: 1250
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Resolve 16 - Object removal

PostWed Apr 10, 2019 12:56 pm

Has anyone given object removal a good time? I don't own a studio licence and can't properly run it on my win7 workstation so I'm scraping the door here :D All kind of cleanup works are somthing I do quite frequently so I'm always interested in new tools.

What is still totally unclear to me is whether it can actually fill an area that is not revealed during shot, none of the examples I have found since release showcase it.
I do stuff.
Offline

Sam Steti

  • Posts: 1568
  • Joined: Tue Jun 17, 2014 7:29 am
  • Location: France

Re: Resolve 16 - Object removal

PostWed Apr 10, 2019 4:40 pm

Hey,
What I can tell by experience is it won't make miracles, but if the direct surroundings of the stuff to remove are good, it can be impressive.
But I know I could do better - for sure - in difficult cases with Mocha Pro : not only because I know it very well, but also because it allows to go deeply into the cleanplate made for "inserting" or "removing" (I could elaborate if needed), where Adobe - or Resolve now - don't.
It's a kind of one strike chance, you win or lose, but if you lose you cannot easily fix it, a bit like the magic of the "shot match this clip" between nodes. It's ok or not...
But for sure, if areas around are cool, it may produce amazing results
Legacy MacPro 8core Xeons, 32 Go ram, 2 x gtx 980 ti, 1SSD for system, 2 SSDs raid0 for footage and caches, OSX ElCap and High Sierra, Resolve 15.3 Studio
Offline

Dermot Shane

  • Posts: 2264
  • Joined: Tue Nov 11, 2014 6:48 pm
  • Location: Vancouver, Canada

Re: Resolve 16 - Object removal

PostWed Apr 10, 2019 6:55 pm

i had a few shots yesterday that i sent to a machine with MochaPro, it also has my test version of 16.0b1 on it.. i tried in v16 and the resualts were useable in this scene, but the clean plate was not accurate enough for other shots

insanely fast tho, it took under 5 min to do four shots, totaling 30 seconds of screen time, on my oldest & slowest machine: z800 / 2x 5680's /96g / 1x 1080

end game for fast & dirty fixes it's amazing, for detailed surfaces, moveing cloth and skin it's a case by case wither it will hold up or you need to roundtrip to Mocha

at least it's dead fast to find out if it's a hero or a zero, so worth the try before exporting
Offline

Hendrik Proosa

  • Posts: 1250
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 8:53 am

After a bit of fiddling got it running on my laptop and while non-studio version has object removal effect watermarked, it is testable enough. And to answer one of my own questions, yes, it can fill an area with content gathered from surrounding pixels without it being revealed during shot. Got the gpu full error very quickly though, so practical usability seems to be so-so (can't do work if it doesn't actually work, however good the features are on paper).

I think it is a good thing to give a shot at first and when it doesn't work, go down the good old methods road. From comp perspective, a combination of good tracking and semiautomatic cleanplate generation would be more practical I think. Not sure object removal is able to use already made tracking data (slamming it on prestabilized shot is an option but annoying) and whether it is possible to get the tracking/vector data (if it exists at all) out from OR itself.
I do stuff.
Offline

Sam Steti

  • Posts: 1568
  • Joined: Tue Jun 17, 2014 7:29 am
  • Location: France

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 9:38 am

Hi there,

Yes, it looks like both of you agree with me. In good conditions, it's super easy and fast, in more tricky cases, one may still have a stop inside Resolve for some other techniques, like trying the patch effect, or maybe track the area to be replaced and apply an offset to reproduce the slide of the content which is next into the tracked PW...
If too tricky and a lot of moves and parallax, then of course Mocha. As I wrote, I think this latter will keep leading as long as it allows to have more controls on the cleanplates...
Legacy MacPro 8core Xeons, 32 Go ram, 2 x gtx 980 ti, 1SSD for system, 2 SSDs raid0 for footage and caches, OSX ElCap and High Sierra, Resolve 15.3 Studio
Offline

Vit Reiter

  • Posts: 617
  • Joined: Mon Sep 04, 2017 5:36 pm

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 11:46 am

How can I add External Clean Plate, please?
DaVinci Resolve 16.2.7 Studio
Mac Pro 2013, AMD FirePro D700, 64GB RAM, macOS Mojave 10.14.6
iMac 2017, Radeon Pro 575, 24GB RAM, macOS High Sierra 10.13.6

Film Editor, Colorist, DIT, Encoding technician
linkedin.com/in/vít-reiter-film-editor
Offline

Trensharo

  • Posts: 395
  • Joined: Mon Jul 23, 2018 1:20 pm
  • Warnings: 1
  • Real Name: Nate Doucette

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 12:53 pm

Hendrik Proosa wrote:After a bit of fiddling got it running on my laptop and while non-studio version has object removal effect watermarked, it is testable enough. And to answer one of my own questions, yes, it can fill an area with content gathered from surrounding pixels without it being revealed during shot. Got the gpu full error very quickly though, so practical usability seems to be so-so (can't do work if it doesn't actually work, however good the features are on paper).

I think it is a good thing to give a shot at first and when it doesn't work, go down the good old methods road. From comp perspective, a combination of good tracking and semiautomatic cleanplate generation would be more practical I think. Not sure object removal is able to use already made tracking data (slamming it on prestabilized shot is an option but annoying) and whether it is possible to get the tracking/vector data (if it exists at all) out from OR itself.

I'm interesting in knowing how this compares with Adobe's Content Aware Fill in After Effects, specifically as it pertains to the VRAM full error messages. Does After Effects have that bottleneck?

What GPU are you using (or how much VRAM do you have)?
Offline

Sam Steti

  • Posts: 1568
  • Joined: Tue Jun 17, 2014 7:29 am
  • Location: France

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 1:17 pm

Vit Reiter wrote:How can I add External Clean Plate, please?

AFAIK you cannot in Resolve. But just to be clear again : i didn't install v16 so far and didn't try the object removal, I'm just very experienced in Mocha and also guess that OR in Resolve probably works like Adobe's Content Aware Fill.
Until someone denies it after real usage, I guess none of Resolve and Adobe tools includes "real" external cleanplates support, see ?
Legacy MacPro 8core Xeons, 32 Go ram, 2 x gtx 980 ti, 1SSD for system, 2 SSDs raid0 for footage and caches, OSX ElCap and High Sierra, Resolve 15.3 Studio
Offline
User avatar

Marc Wielage

  • Posts: 6996
  • Joined: Fri Oct 18, 2013 2:46 am
  • Location: Hollywood, USA

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 1:34 pm

I think this is more of a "concept" feature that will be more refined in the future. What I'd like is something that will (say) automatically generate a matte around an object, and then let the user go in and really tweak it after the fact. The features like Object Removal and Face Refinement are terrific when they work, but when they don't, there's nothing you can do to salvage it. Manual "hooks" in the process would give us the ability to take it the rest of the way.

My hope is that that will eventually come, but it's a bit up in the air.
marc wielage, csi • VP/color & workflow • chroma | hollywood
Offline

Hendrik Proosa

  • Posts: 1250
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 1:47 pm

Sam Steti wrote:
Vit Reiter wrote:How can I add External Clean Plate, please?

Until someone denies it after real usage, I guess none of Resolve and Adobe tools includes "real" external cleanplates support, see ?

AE content aware can use manually painted reference frames, see the last quarter of this clip:


Trensharo wrote:I'm interesting in knowing how this compares with Adobe's Content Aware Fill in After Effects, specifically as it pertains to the VRAM full error messages. Does After Effects have that bottleneck?

What GPU are you using (or how much VRAM do you have)?

I haven't tried the AE one yet. I have 4 gigs of VRAM, nothing too joyful but 4gigs can fit quite a lot of 32bit float image data even at 4K res so I blame bad memory management :D
I do stuff.
Offline

Annaël Beauchemin

  • Posts: 45
  • Joined: Thu Nov 24, 2016 7:12 am

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 1:58 pm

Sam Steti wrote:
Vit Reiter wrote:How can I add External Clean Plate, please?

AFAIK you cannot in Resolve. But just to be clear again : i didn't install v16 so far and didn't try the object removal, I'm just very experienced in Mocha and also guess that OR in Resolve probably works like Adobe's Content Aware Fill.
Until someone denies it after real usage, I guess none of Resolve and Adobe tools includes "real" external cleanplates support, see ?


In the youtube video of the new features presentation, it looks like external plates is a possibility.

Here's a screenshot I grabbed:

Image

I use the Remove function of Mocha Pro alot, so I'll be interested to see how AE/Resolve solutions compare, but I also have my doubts. Tracking precision is really important for this kind of process and Mocha's tracker is just better than Resolve's and AE's.

For some of the easy shots, it might work a treat, so that's more than welcome to have this feature in Resolve.

Furthermore, Mocha Pro 2019.5 OFX plugin is said to be compatible with Resolve, so we will be actually able to use it directly in Resolve as an OFX. So much good news!
Offline

Vit Reiter

  • Posts: 617
  • Joined: Mon Sep 04, 2017 5:36 pm

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 2:48 pm

Annaël Beauchemin wrote:In the youtube video of the new features presentation, it looks like external plates is a possibility.

Here's a screenshot I grabbed:

Image
Thank you for trying, Annaël, but this screenshot doesn't show how to use an external clean plate :(
Marc has apparently truth - external clean plate is concept yet.

Let's wait for it.
No problem for me, I have mocha Pro. I am only curious ;)
DaVinci Resolve 16.2.7 Studio
Mac Pro 2013, AMD FirePro D700, 64GB RAM, macOS Mojave 10.14.6
iMac 2017, Radeon Pro 575, 24GB RAM, macOS High Sierra 10.13.6

Film Editor, Colorist, DIT, Encoding technician
linkedin.com/in/vít-reiter-film-editor
Offline

Sam Steti

  • Posts: 1568
  • Joined: Tue Jun 17, 2014 7:29 am
  • Location: France

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 4:19 pm

True Hendrik, this last part of your video shows "ref frames" in AE works like "cleanplates" in Mocha Pro.

I also agree with Annaël here
For some of the easy shots, it might work a treat, so that's more than welcome to have this feature in Resolve. Furthermore, Mocha Pro 2019.5 OFX plugin is said to be compatible with Resolve, so we will be actually able to use it directly in Resolve as an OFX.
.

I suspect it won't be so easy to have great perf of Mocha inside Resolve, but for sure I will try.

This said, this is now very personal but I confess it doesn't bother me at all to gather specific jobs for Mocha Pro standalone, even if it's wasted time compared to an ofx allowing to remain in Resolve.
BUT, some might share my opinion when i tell you i prefer super stable and great performance solutions over sloppy OFXes, so time is not wasted for me when you opt for a rock solid workflow... For example, so far I'm still one of those who make Fusion Connect to find the standalone, make the job and refresh in Resolve than ReFusion. I'll change soon probably but this is no problem for me.
So when I chat with Mary at (ex Imagineer S.) BorisFX, I can read that they're asked for numerous questions about issues of Mocha's integration in Premiere.
If it's the same, I'll keep the standalone but of course, Mocha inside Resolve is super welcome ! [/personal POV]
Legacy MacPro 8core Xeons, 32 Go ram, 2 x gtx 980 ti, 1SSD for system, 2 SSDs raid0 for footage and caches, OSX ElCap and High Sierra, Resolve 15.3 Studio
Offline

Hendrik Proosa

  • Posts: 1250
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 4:22 pm

It is possible to choose "External" as clean plate source but as help file does not cover object removal at all, I have no idea how that is actually supposed to be used.
I do stuff.
Offline

xunile

  • Posts: 1572
  • Joined: Mon Apr 23, 2018 5:21 am
  • Real Name: Eric Eisenmann

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 6:13 pm

Hendrik Proosa wrote:It is possible to choose "External" as clean plate source but as help file does not cover object removal at all, I have no idea how that is actually supposed to be used.


The manual included with Resolve 16 beta is the Resolve 15.3 manual. They haven't updated it yet for 16.
Win 10 Home
Intel Core I7-7700HQ 32 GB 1 TB NVME SSD 1 TB SATA SSD
GTX-1060-6GB
Resolve 16.2.4
Offline

Hendrik Proosa

  • Posts: 1250
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 7:21 pm

xunile wrote:
Hendrik Proosa wrote:It is possible to choose "External" as clean plate source but as help file does not cover object removal at all, I have no idea how that is actually supposed to be used.


The manual included with Resolve 16 beta is the Resolve 15.3 manual. They haven't updated it yet for 16.

Kind of defeats the purpose of beta, doesn't it? If one does not know how something should (or shouldn't work) you can't decide if it is a feature, a bug or just not implemented. No public bug tracker by deliberate choice (it has been made pretty clear by BMD reps here in forum) is not helping either.
I do stuff.
Offline

Theodor Groeneboom

  • Posts: 273
  • Joined: Fri Jun 26, 2015 12:51 pm

Re: Resolve 16 - Object removal

PostThu Apr 11, 2019 7:26 pm

Marc Wielage wrote:I think this is more of a "concept" feature that will be more refined in the future. What I'd like is something that will (say) automatically generate a matte around an object, and then let the user go in and really tweak it after the fact. The features like Object Removal and Face Refinement are terrific when they work, but when they don't, there's nothing you can do to salvage it. Manual "hooks" in the process would give us the ability to take it the rest of the way.

My hope is that that will eventually come, but it's a bit up in the air.


So they should have built it in Fusion instead. The context for this type of work is much more appropriate there.
Offline

Dermot Shane

  • Posts: 2264
  • Joined: Tue Nov 11, 2014 6:48 pm
  • Location: Vancouver, Canada

Re: Resolve 16 - Object removal

PostFri Apr 12, 2019 2:46 am

there's a one page explanation of how to use it in the "v16 new features" document on BMD's support site
Offline

Hendrik Proosa

  • Posts: 1250
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Resolve 16 - Object removal

PostFri Apr 12, 2019 5:22 am

Dermot Shane wrote:there's a one page explanation of how to use it in the "v16 new features" document on BMD's support site

And it says nothing about how to use the external cleanplate or the status of that feature.
I do stuff.
Offline
User avatar

Jean Claude

  • Posts: 2973
  • Joined: Sun Jun 28, 2015 4:41 pm
  • Location: France

Re: Resolve 16 - Object removal

PostFri Apr 12, 2019 5:44 pm

Hello Hendrik,

(here a very quick test)
Try :
Node 1 - some LGG
Node 2 - PowerWindows + Tracker + (maybe play with Node key ?) and add Node Outside
Node 3 (outside) : OFX Removal :

Blend mode : Adaptive Mode
Click on Scene Analysis
Next
Clean Plate => External
Click on Build Clean Plate

Warning : It is an OFX that requires huge GPU resources (incredible. Maybe TENSOR CORE for AI)

1_Disable removal OFX.jpg

2_Enable removal OFX.jpg

:)
"Saying it is good, but doing it is better! "
Win10-1809 | Resolve Studio V16.1 | Fusion Studio V16.1 | Decklink 4K Extreme 6G | RTX 2080Ti 431.86 NSD driver! |
Offline

Hendrik Proosa

  • Posts: 1250
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Resolve 16 - Object removal

PostFri Apr 12, 2019 5:53 pm

Where is the external clean plate stored and in what format?
I do stuff.
Offline

Oli Koos

  • Posts: 208
  • Joined: Thu Apr 30, 2015 10:29 am

Re: Resolve 16 - Object removal

PostFri Apr 12, 2019 6:14 pm

Jean Claude wrote:Hello Hendrik,

(here a very quick test)
...

Hi Jean Claude, could you please share a picture of your node graph? thanks
Last edited by Oli Koos on Fri Apr 12, 2019 7:55 pm, edited 1 time in total.
Offline

John Spirou

  • Posts: 49
  • Joined: Sun Sep 17, 2017 3:51 pm

Re: Resolve 16 - Object removal

PostFri Apr 12, 2019 6:37 pm

External means, take the information from outside the alpha channel...not external file or image.

Sent from my SM-T830 using Tapatalk
Offline

Hendrik Proosa

  • Posts: 1250
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Resolve 16 - Object removal

PostFri Apr 12, 2019 8:58 pm

John Spirou wrote:External means, take the information from outside the alpha channel...not external file or image.

Well, in any other software, external cleanplate means external file that you can manually prep. Where do you get that it means "take from outside alpha"? I can get cleanplate produced from surrounding pixels using Internal setting too.
I do stuff.
Offline
User avatar

robozb

  • Posts: 351
  • Joined: Wed Apr 17, 2019 6:48 am
  • Location: Hungary
  • Real Name: Roboz Bela Tamas

Re: Resolve 16 - Object removal

PostTue Jul 02, 2019 1:28 pm

My experience is the result is really ugly with Object Removal in Studio Resolve16b4, for me it was good for nothing, especially not for production environment. Furthermore this plug-in kills the app, hangs, freeze, etc, completely useless.

What is in AE that is much more better!

i7-8700K 24GB DDR4 GTX 1060 3GB
Attachments
3.JPG
3.JPG (26.69 KiB) Viewed 3346 times
Offline

Jim Simon

  • Posts: 11014
  • Joined: Fri Dec 23, 2016 1:47 am
  • Warnings: 1

Re: Resolve 16 - Object removal

PostTue Jul 02, 2019 1:44 pm

Hendrik Proosa wrote:demo videos don't showcase the "neural" or AI part of removal.


I'm curious how you arrived at that conclusion?
Offline

Hendrik Proosa

  • Posts: 1250
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Resolve 16 - Object removal

PostTue Jul 02, 2019 2:20 pm

Jim Simon wrote:
Hendrik Proosa wrote:demo videos don't showcase the "neural" or AI part of removal.

I'm curious how you arrived at that conclusion?

From watching the demo videos that were available at the time. Removing stuff from areas where backing is revealed in different time in the same shot does not need any kind of neural or ai to do, Mocha remove module has been doing it for ages.

Now, it can remove stuff from static frames also and fill it in with surrounding area details, so it does have the neural part. Just those demos didn't showcase that one bit.
I do stuff.
Offline

Dermot Shane

  • Posts: 2264
  • Joined: Tue Nov 11, 2014 6:48 pm
  • Location: Vancouver, Canada

Re: Resolve 16 - Object removal

PostTue Jul 02, 2019 3:45 pm

robozb wrote:My experience is the result is really ugly with Object Removal in Studio Resolve16b4, for me it was good for nothing, especially not for production environment. Furthermore this plug-in kills the app, hangs, freeze, etc, completely useless.

What is in AE that is much more better!

i7-8700K 24GB DDR4 GTX 1060 3GB


not an AE user so no idea how that compares.. but a 3g GPU is not going to get very anyone far in Resolve
Offline

Oli Koos

  • Posts: 208
  • Joined: Thu Apr 30, 2015 10:29 am

Re: Resolve 16 - Object removal

PostFri Jul 05, 2019 6:27 am

Has object removal been improved over all the betas? I have yet to test the latest one but did not get the impression that it is on par with AFX
Offline
User avatar

Jean Claude

  • Posts: 2973
  • Joined: Sun Jun 28, 2015 4:41 pm
  • Location: France

Re: Resolve 16 - Object removal

PostFri Jul 05, 2019 5:17 pm

Oli Koos wrote:Has object removal been improved over all the betas? I have yet to test the latest one but did not get the impression that it is on par with AFX


Hello,
It still works as well since Beta 001 :)
"Saying it is good, but doing it is better! "
Win10-1809 | Resolve Studio V16.1 | Fusion Studio V16.1 | Decklink 4K Extreme 6G | RTX 2080Ti 431.86 NSD driver! |

Return to DaVinci Resolve

Who is online

Users browsing this forum: jasonpm and 89 guests