Resolve 16 - Object removal

This is an interesting one but current demo videos don't showcase the "neural" or AI part of removal. Could someone with studio licence test the object removal effect on a few shots where the neural engine would actually make a difference?

Good candidate shot would:
Have positional camera movement, not just nodal pan AND
have focus object cross different image planes (legs on closeup ground, body over distant bg for example).
Or have focus object cover an area which is not revealed during the shot.

Although AE-s content aware fill is kind of crappy, it is able to crunch through such shots (albeit miserably). I'm interested to see what Resolve can put up against it but can't try it myself currently.

Sorry I cannot help yet, but in the meantime - could you please share the links of the " current demo videos" you mention?

Thanks
Piotr

i have tried it on a few shot and can't get it to work well at all

NAB demo had the baseball scene (starts at 1:17:30):

What's new in Resolve 16 had the runners shot (starts at 15:36):

Crappy AE castle demo which uses actual content aware fill can be seen here:

Has anyone given object removal a good time? I don't own a studio licence and can't properly run it on my win7 workstation so I'm scraping the door here All kind of cleanup works are somthing I do quite frequently so I'm always interested in new tools.

What is still totally unclear to me is whether it can actually fill an area that is not revealed during shot, none of the examples I have found since release showcase it.

Hey,
What I can tell by experience is it won't make miracles, but if the direct surroundings of the stuff to remove are good, it can be impressive.
But I know I could do better - for sure - in difficult cases with Mocha Pro : not only because I know it very well, but also because it allows to go deeply into the cleanplate made for "inserting" or "removing" (I could elaborate if needed), where Adobe - or Resolve now - don't.
It's a kind of one strike chance, you win or lose, but if you lose you cannot easily fix it, a bit like the magic of the "shot match this clip" between nodes. It's ok or not...
But for sure, if areas around are cool, it may produce amazing results

i had a few shots yesterday that i sent to a machine with MochaPro, it also has my test version of 16.0b1 on it.. i tried in v16 and the resualts were useable in this scene, but the clean plate was not accurate enough for other shots

insanely fast tho, it took under 5 min to do four shots, totaling 30 seconds of screen time, on my oldest & slowest machine: z800 / 2x 5680's /96g / 1x 1080

end game for fast & dirty fixes it's amazing, for detailed surfaces, moveing cloth and skin it's a case by case wither it will hold up or you need to roundtrip to Mocha

at least it's dead fast to find out if it's a hero or a zero, so worth the try before exporting

After a bit of fiddling got it running on my laptop and while non-studio version has object removal effect watermarked, it is testable enough. And to answer one of my own questions, yes, it can fill an area with content gathered from surrounding pixels without it being revealed during shot. Got the gpu full error very quickly though, so practical usability seems to be so-so (can't do work if it doesn't actually work, however good the features are on paper).

I think it is a good thing to give a shot at first and when it doesn't work, go down the good old methods road. From comp perspective, a combination of good tracking and semiautomatic cleanplate generation would be more practical I think. Not sure object removal is able to use already made tracking data (slamming it on prestabilized shot is an option but annoying) and whether it is possible to get the tracking/vector data (if it exists at all) out from OR itself.

Hi there,

Yes, it looks like both of you agree with me. In good conditions, it's super easy and fast, in more tricky cases, one may still have a stop inside Resolve for some other techniques, like trying the patch effect, or maybe track the area to be replaced and apply an offset to reproduce the slide of the content which is next into the tracked PW...
If too tricky and a lot of moves and parallax, then of course Mocha. As I wrote, I think this latter will keep leading as long as it allows to have more controls on the cleanplates...

How can I add External Clean Plate, please?

After a bit of fiddling got it running on my laptop and while non-studio version has object removal effect watermarked, it is testable enough. And to answer one of my own questions, yes, it can fill an area with content gathered from surrounding pixels without it being revealed during shot. Got the gpu full error very quickly though, so practical usability seems to be so-so (can't do work if it doesn't actually work, however good the features are on paper).

I think it is a good thing to give a shot at first and when it doesn't work, go down the good old methods road. From comp perspective, a combination of good tracking and semiautomatic cleanplate generation would be more practical I think. Not sure object removal is able to use already made tracking data (slamming it on prestabilized shot is an option but annoying) and whether it is possible to get the tracking/vector data (if it exists at all) out from OR itself.

I'm interesting in knowing how this compares with Adobe's Content Aware Fill in After Effects, specifically as it pertains to the VRAM full error messages. Does After Effects have that bottleneck?

What GPU are you using (or how much VRAM do you have)?

How can I add External Clean Plate, please?

AFAIK you cannot in Resolve. But just to be clear again : i didn't install v16 so far and didn't try the object removal, I'm just very experienced in Mocha and also guess that OR in Resolve probably works like Adobe's Content Aware Fill.
Until someone denies it after real usage, I guess none of Resolve and Adobe tools includes "real" external cleanplates support, see ?

I think this is more of a "concept" feature that will be more refined in the future. What I'd like is something that will (say) automatically generate a matte around an object, and then let the user go in and really tweak it after the fact. The features like Object Removal and Face Refinement are terrific when they work, but when they don't, there's nothing you can do to salvage it. Manual "hooks" in the process would give us the ability to take it the rest of the way.

My hope is that that will eventually come, but it's a bit up in the air.

How can I add External Clean Plate, please?

Until someone denies it after real usage, I guess none of Resolve and Adobe tools includes "real" external cleanplates support, see ?

AE content aware can use manually painted reference frames, see the last quarter of this clip:

I'm interesting in knowing how this compares with Adobe's Content Aware Fill in After Effects, specifically as it pertains to the VRAM full error messages. Does After Effects have that bottleneck?

What GPU are you using (or how much VRAM do you have)?

I haven't tried the AE one yet. I have 4 gigs of VRAM, nothing too joyful but 4gigs can fit quite a lot of 32bit float image data even at 4K res so I blame bad memory management

How can I add External Clean Plate, please?

AFAIK you cannot in Resolve. But just to be clear again : i didn't install v16 so far and didn't try the object removal, I'm just very experienced in Mocha and also guess that OR in Resolve probably works like Adobe's Content Aware Fill.
Until someone denies it after real usage, I guess none of Resolve and Adobe tools includes "real" external cleanplates support, see ?

In the youtube video of the new features presentation, it looks like external plates is a possibility.

Here's a screenshot I grabbed:



I use the Remove function of Mocha Pro alot, so I'll be interested to see how AE/Resolve solutions compare, but I also have my doubts. Tracking precision is really important for this kind of process and Mocha's tracker is just better than Resolve's and AE's.

For some of the easy shots, it might work a treat, so that's more than welcome to have this feature in Resolve.

Furthermore, Mocha Pro 2019.5 OFX plugin is said to be compatible with Resolve, so we will be actually able to use it directly in Resolve as an OFX. So much good news!

In the youtube video of the new features presentation, it looks like external plates is a possibility.

Here's a screenshot I grabbed:

Thank you for trying, Annaël, but this screenshot doesn't show how to use an external clean plate
Marc has apparently truth - external clean plate is concept yet.

Let's wait for it.
No problem for me, I have mocha Pro. I am only curious

True Hendrik, this last part of your video shows "ref frames" in AE works like "cleanplates" in Mocha Pro.

I also agree with Annaël here

For some of the easy shots, it might work a treat, so that's more than welcome to have this feature in Resolve. Furthermore, Mocha Pro 2019.5 OFX plugin is said to be compatible with Resolve, so we will be actually able to use it directly in Resolve as an OFX.

.

I suspect it won't be so easy to have great perf of Mocha inside Resolve, but for sure I will try.

This said, this is now very personal but I confess it doesn't bother me at all to gather specific jobs for Mocha Pro standalone, even if it's wasted time compared to an ofx allowing to remain in Resolve.
BUT, some might share my opinion when i tell you i prefer super stable and great performance solutions over sloppy OFXes, so time is not wasted for me when you opt for a rock solid workflow... For example, so far I'm still one of those who make Fusion Connect to find the standalone, make the job and refresh in Resolve than ReFusion. I'll change soon probably but this is no problem for me.
So when I chat with Mary at (ex Imagineer S.) BorisFX, I can read that they're asked for numerous questions about issues of Mocha's integration in Premiere.
If it's the same, I'll keep the standalone but of course, Mocha inside Resolve is super welcome ! [/personal POV]

It is possible to choose "External" as clean plate source but as help file does not cover object removal at all, I have no idea how that is actually supposed to be used.

It is possible to choose "External" as clean plate source but as help file does not cover object removal at all, I have no idea how that is actually supposed to be used.

The manual included with Resolve 16 beta is the Resolve 15.3 manual. They haven't updated it yet for 16.

It is possible to choose "External" as clean plate source but as help file does not cover object removal at all, I have no idea how that is actually supposed to be used.

The manual included with Resolve 16 beta is the Resolve 15.3 manual. They haven't updated it yet for 16.

Kind of defeats the purpose of beta, doesn't it? If one does not know how something should (or shouldn't work) you can't decide if it is a feature, a bug or just not implemented. No public bug tracker by deliberate choice (it has been made pretty clear by BMD reps here in forum) is not helping either.

I think this is more of a "concept" feature that will be more refined in the future. What I'd like is something that will (say) automatically generate a matte around an object, and then let the user go in and really tweak it after the fact. The features like Object Removal and Face Refinement are terrific when they work, but when they don't, there's nothing you can do to salvage it. Manual "hooks" in the process would give us the ability to take it the rest of the way.

My hope is that that will eventually come, but it's a bit up in the air.

So they should have built it in Fusion instead. The context for this type of work is much more appropriate there.

there's a one page explanation of how to use it in the "v16 new features" document on BMD's support site

there's a one page explanation of how to use it in the "v16 new features" document on BMD's support site

And it says nothing about how to use the external cleanplate or the status of that feature.

Hello Hendrik,

(here a very quick test)
Try :
Node 1 - some LGG
Node 2 - PowerWindows + Tracker + (maybe play with Node key ?) and add Node Outside
Node 3 (outside) : OFX Removal :

Blend mode : Adaptive Mode
Click on Scene Analysis
Next
Clean Plate => External
Click on Build Clean Plate

Warning : It is an OFX that requires huge GPU resources (incredible. Maybe TENSOR CORE for AI)

1_Disable removal OFX.jpg

2_Enable removal OFX.jpg

Where is the external clean plate stored and in what format?

Hello Hendrik,

(here a very quick test)
...

Hi Jean Claude, could you please share a picture of your node graph? thanks

External means, take the information from outside the alpha channel...not external file or image.

Sent from my SM-T830 using Tapatalk

External means, take the information from outside the alpha channel...not external file or image.

Well, in any other software, external cleanplate means external file that you can manually prep. Where do you get that it means "take from outside alpha"? I can get cleanplate produced from surrounding pixels using Internal setting too.

My experience is the result is really ugly with Object Removal in Studio Resolve16b4, for me it was good for nothing, especially not for production environment. Furthermore this plug-in kills the app, hangs, freeze, etc, completely useless.

What is in AE that is much more better!

i7-8700K 24GB DDR4 GTX 1060 3GB

demo videos don't showcase the "neural" or AI part of removal.

I'm curious how you arrived at that conclusion?

demo videos don't showcase the "neural" or AI part of removal.

I'm curious how you arrived at that conclusion?

From watching the demo videos that were available at the time. Removing stuff from areas where backing is revealed in different time in the same shot does not need any kind of neural or ai to do, Mocha remove module has been doing it for ages.

Now, it can remove stuff from static frames also and fill it in with surrounding area details, so it does have the neural part. Just those demos didn't showcase that one bit.

My experience is the result is really ugly with Object Removal in Studio Resolve16b4, for me it was good for nothing, especially not for production environment. Furthermore this plug-in kills the app, hangs, freeze, etc, completely useless.

What is in AE that is much more better!

i7-8700K 24GB DDR4 GTX 1060 3GB

not an AE user so no idea how that compares.. but a 3g GPU is not going to get very anyone far in Resolve

Has object removal been improved over all the betas? I have yet to test the latest one but did not get the impression that it is on par with AFX

Has object removal been improved over all the betas? I have yet to test the latest one but did not get the impression that it is on par with AFX

Hello,
It still works as well since Beta 001