Removal of Background Footage without Green Screen

Hey! Alrighty, So I have some footage in Davinci Resolve that needs to have the background footage taken out.

Here's my problem:
No green screen footage.
Person that I need to take from the footage FROM the background walks from out of the screen into the screen.
ANOTHER problem is that when the person is walking into the screen, they're farther away and then walk up and then off the screen at a much closer distance than when the walked in.

I'm kind of curious of how I get it so specific as to how to get this person specifically. I've been trying doing that magic mask-ing and stuff, but for some reason, I can't get it from frame to frame (somewhat like you can do with Adobe After Effects, but I've tried and Davinci keeps screwing up and freezing no matter what I do. [AND IT'S NOT MY COMPUTER])

I need to take this person that's walking in from a farther distance and then walking out from a closer distance, and then put them ontop of another background/shot.
I do understand about masking and such, however, I'm just wondering how to do this big issue of getting the person just by themselves.

Whenever I try to do this magic mask-ing, I only get so much and not specifically the person themselves. I wouldn't even mind going and doing the outlines of the person and then once the base of them is done, going in and doing the fine lines (like you can do in After Effects), however, I am not understanding how.

Does anybody have any specific video links to anywhere, ideas, or tips/tutorials they can share??

I'm really have issues with this one.
Please help.
THANKS!

[Peter Chamberlain]

Moved to Resolve forum.

Can u detail. extensively. what OS and hardware and resolve version u have?

How long is the clip in question?
How many frames is the person in shot?

It sounds like a complicated rotoscoping job.. maybe in Fusion but with the questions above answered someone might be able to help.

That looks like a serious vfx job

I have the most up to date Davinci Resolve (the one that you receive from the BlackMagic purchase).

The clip is at least 10 seconds long I believe.

For about the main clip, the person is in shot. He walks in from the right from a farther away shot, and then walks around in a bit of a circular pattern; then he is approaching the camera (or moreso moving forward) as he circles back to the same side (right).

It kind of is.
Is there some tutorial of how to work with Fusion on how to remove it?

If so, can you post the links to either videos or even tutorial documents so I can learn/figure it out??

Look up tutorials on rotoscoping. It's tedious and there is definitely a skill involved and that comes with practice. There are automated tools that can be used with varying degrees of success. It all depends on the material you're working with. Probably the most advanced one is Mocha.

The only program I have access to is Davinci and After Effects. (Granted I DO have Fusion and such since I have the paid version of Davinci)
I have tried looking up tutorials and the problem is that the subject I'm trying to take out and rotoscope changes the size since he's walking from off screen from a farther distance and then comes closer and then walks off screen once again.
That is the problem; I'm trying to make sure that the size change works with all of my changes and stuff and then trying to make sure everything works.
I'm so confused.


"Look up tutorials on rotoscoping. It's tedious and there is definitely a skill involved and that comes with practice. There are automated tools that can be used with varying degrees of success. It all depends on the material you're working with. Probably the most advanced one is Mocha."

if you have the budget -

https://runwayml.com/

I have had very surprising results with magic mask. Certainly not perfect but way better than you would expect!

Hi!
There are some video tutorials on Youtube. Search in there for, as example "resolve remove background". When presented there it seems sort of easy, but we all know that it probably is more to it than that.

The only program I have access to is Davinci and After Effects. (Granted I DO have Fusion and such since I have the paid version of Davinci)
I have tried looking up tutorials and the problem is that the subject I'm trying to take out and rotoscope changes the size since he's walking from off screen from a farther distance and then comes closer and then walks off screen once again.
That is the problem; I'm trying to make sure that the size change works with all of my changes and stuff and then trying to make sure everything works.
I'm so confused.


"Look up tutorials on rotoscoping. It's tedious and there is definitely a skill involved and that comes with practice. There are automated tools that can be used with varying degrees of success. It all depends on the material you're working with. Probably the most advanced one is Mocha."

This video should give you a flavour of how to approach this, but as she says in the video its a lot of tedious work rotoscoping each frame

I love Daria's videos, they're really concise and well explained. Shame she's never had a chance to update her own videos for more recent versions of Resolve, but I understand she's now working at BMD so she does official documentation now.

I'm not sure if Color would be the best place for this, though. At the time Daria made that video, Fusion was not a part of Resolve. Fusion provides a lot more power and flexibility for tasks like rotoscoping.

Victoria, we might be able to help more specifically if you could show as some screenshots of your source footage? Or even link to an uploaded video of it? (Eg on YouTube, Vimeo, or on a site like Google Drive or Dropbox). Or is it blocked by NDA?

Or if After Effects is doing a better job for you with its roto AI thing, go ahead and use AE for that task? Use whatever tools work best for a given shot.

You could use the roto tools in AE to create a greyscale matte and then render that out for use in Resolve, either in Fusion or Color. Both Color and Fusion can easily do masking based on an external matte.

Most Resolve and Fusion roto tutorials I have seen suffer from lack of actual roto principles. Whatever you do, do at least this: break your object into separate simpler shapes that don't change shape or change it little. Do not, and I repeat, do not try to roto articulate shape-changing objects (like a human) in one piece or large pieces, it is a mess hard to dig out. The less shapes change the less keyframes you need and the less you have to mess with changing the separate shape points itself instead of simply transforming shape as a whole. And second advice, only roto actual objects (the so-called positive space). This means that don't try to roto holes between objects, roto the objects and holes are what is left. Combine shapes using blend modes.

Here's one video tutorial for Fusion where you can see all the many individual shapes that Hendrik describes.

This video doesn't go into how to make all those shapes, as the video that does that is part of a paid Fusion course (40 hours of material for about $45 I think). This video handles 'temporary roto shapes', ie shapes that only exist at certain points in time. But in the video you can see the dozens of individual shapes he made to roto out a woman walking from back of shot to front - kind of similar to what you described actually, Victoria.

Here's a screenshot showing how many masks he made, and the names for them which give you an idea of how many shapes he needed and what areas they cover:


As mentioned, to see him making all the shapes you'd need to buy the course. I bought it and can recommend it for anyone who is new to Fusion and wants to learn it in more depth. It's good value for money IMHO.

Alright guys!
Some of you have sent me some links to videos.
I've watched them and I am a little confused since I'm having issues. Maybe you can help me here.

Obviously every keyframe is slightly different than the last; not all of them are drastically different, however, almost all of them have a slightly different movement. (Maybe a twitch of the arm, so a point has to be moved, and then moved again in the next keyframe.)

Am I able to do this with the one shape or am I basically going to be doing a shape for each keyframe?
I've been trying to mess with it, and if I change on thing in one keyframe, it changes the entire shape.
Am I doing something wrong, or am I just having to do a shape for each and every keyframe?

Thanks so much guys! this is so confusing for me, since I'm used to After Effects, but for some reason, whenever I've tried to rotoscope with it, it freezes up and then doesn't allow me to work on my project. I've contacted Adobe about this, and they are not helping me at all.

So, since I have Davinci, I feel it's time I learn how to do it on here.

You can not change the number of points for a shape during animation. Create your shapes on a frame where the part of object some shape represents is in its ”medium” position (not too distorted relative to rest of the movement) and work forward and backward from there. If some part gets occluded or moves out of frame you can change the opacity of that shape to zero or move it out of frame so it is out of the way.

How many keyframes you need depends on the movement. Try to find movement extremes and place keyframes there. For example if arm swings back and forth, it has two extremes and lowest passing position. First place keyframes there and then fill in the rest of needed. If movement is very erratic it is possible you need to move and modify shape for every frame but walking person shouldn’t be this erratic.

And again, DO NOT try to create one big shape for whole person (I somehow get the feeling you tried to do that). It will not work. Create lots of small shapes that represent different body parts that don’t change shape much and animate these.

Roto is a fine art and needs practice, so keep to it and it will get easier and will make sense in the end.

Here's an example from a book I'm reading ("Digital Compositing for Film and Video" by Steve Wright) for how one might roto a boy's face:


The book says:

"Figure 2.37 illustrates the wrong approach, with a single large shape for the entire face. As the head turns and the lips and jaw move with speech it will take an enormous amount of point twiddling to keep the spline on target. Figure 2.38 shows the right approach with separate shapes (colored for illustration purposes) for each key body part. Try to identify body parts that will move as a rigid unit such as the forehead and jaw in this example. The upper and lower lips are flexible and will need continuous shape changes so they are separate. Further, even though the upper lip, for example, will flex and change shape, by having it as a separate shape you can “rough it in” with a transformation then go back and dial in the individual control points as needed."

I did try smaller objects and stuff.

Like I did a torse part and then he had to move even just a little bit from the initial point. Soooo I'm wondering if these keyframes are going to be able to BE adjusted during each keyframe.

I experimented with the bigger object of just the person (to just experiment) and then went to the piece by piece (like everyone is saying to do) and it isn't working. Like the keyframes then change to what I had changed (ALL of the keyframes that I had worked on beforehand) so it's like anytime I try to change the keyframes or anything, it doesn't work.

Sounds like you didn't enable shape keying, look into this video for example, at around 6:30 and forward:

He mentioned another video that he would make that included tracking, which would be VERY helpful since I'm using a person (and obviously separate objects since I'd have to break the person into different parts), instead of just using this part of the video.
I'm just confused since almost every video I've seen for tutorials don't mention anything like this shape keying.

Is there a video that has tracking INCLUDED with the shape keying at all??


"Sounds like you didn't enable shape keying, look into this video for example, at around 6:30 and forward:

"

I'm afraid you'd be better off recording with a green screen. If that's not possible and you are regularly doing this as a paid job, have a long, hard look at Silhouette by BorisFX.

Silhouette will not make roto any easier if basics are not well engrained at first. Neither does mocha. They both just remove some of the grunt work.

I know that I'd be better off recording with a green screen, however, sometimes, you don't have that option (which sucks ESPECIALLY with how confusing this is.)

This is the screenshot of the bit of what I need.
Now, you see the path he was walking from. He starts off screen on the right. Starts to walk around and then back again, but he is closer to the camera when he walks off.
THAT is my problem.
I need to just take him out of the background; sorry, take the background off.

Now, I was sent the information with the videos to do to help with the keyframing and such, however, I do not know how to have it track it and everything from frame to frame.
This is my problem.

"I'm afraid you'd be better off recording with a green screen. If that's not possible and you are regularly doing this as a paid job, have a long, hard look at Silhouette by BorisFX."

That would be a very tedious job with such a lighting situation and this background. It's a challenge even for someone with experience and Mocha or Silhouette. I would have told them it'll be very expensive or never perfect.
What you could try is how far you get with Magic Mask. It's impressive for what it does, but it's not really intended for isolating a person, rather for selective grading. It'll never be perfect, but maybe it's OK for the client if he/she doesn't want to spend heaps of money.

Not sure what to suggest, Victoria. Rotoscoping is a painstaking process that you need to take the time to learn and practice. There's no 'magic wand' to do it quickly in a few minutes. Or at least, there are magic tools like AE Rotobrush and Resolve's Magic Mask, but if you've already tried that and it's not working, then you need to go back to the manual, complex process.

If you want to learn all about Fusion, including a couple of hours of detailed rotoscope tutorials, I can recommend VFXStudy's 30 hour "Compositing with DaVinci Resolve and Fusion" course. It's 58 euros.

From the course description:
It’s not about loading and saving – Set up a linear workflow, understand the domain of definition, color bit depth, gamma, gamut, and more.
It’s not about using the Polygon mask – Extract an actor by covering the human anatomy with multiple roto-shapes and animate them for consistent edge detail.
It’s not about the Delta Keyer – Build advanced keying workflows with multiple preprocessing techniques, screen correction, adaptive despill, custom luma keyers, and more.
It’s not just about tools – learn compositing in depth!

Point two of that being your exact requirement right now.

Here's a link to his course.

I'm not saying that I don't want to go through the long process of doing the work of rotoscoping and everything on Resolve, it's moreso the fact that in the video that I had posted above in a response to someone, the person had mentioned how to do the tracking along with the keyframing so I can change it when it tracks it. So I'm curious how I can do that.

If someone knows how to do it, that's great.

If there's no other way, that's fine with me.
I'm just nervous since I'll have to go frame by frame, which I have nothing wrong with it, I'm just trying to make sure that if I can do magic mask or anything that can do tracking, if it can have me "fix" the keyframe if it fits in one frame, but then the person moves, and then it doesn't.

What I've been trying to ask if there's any way to have that magic mask with the ability to move the "mask" or whatever for the rotoscoping frame by frame, without it affecting the rest of the already set "masks" (or whatever) that are already set since they did the tracking.

I've tried using AE and for some reason, this entire part was not working. The program kept freezing.
It's not my computer and Adobe isn't helping me with anything. They keep trying to help me, and then it doesn't work at all.
I've put hours into working on AE, but then it freezes up and it basically says it's loading, but then it doesn't at all.
It's happened before and one time I had to have it on for a full week, because it was so slow (which is understandable) but it gradually froze which I thought was just because it was slow. But nope.

So I'm with Davinci for this project and I'm just needing help.

I can honestly care less of how much work it's going to take; I'm just curious if there's a way to take the information that is used when you try to track an object, and then be able to change the "mask" (or whatever) frame by frame so that the information can be changed during each frame. Somewhat like how AE does.

I'm late to this thread, so maybe I'm missing some info here. But there still seems to be a bit of confusion to what is what.

The word keyframe can mean a couple of things in this context. Every change you make in your polyline mask will result in - technically - a keyframe in your splineview/timeline but as far as the art of rotoscoping (and animation) is concerned a 'keyframe' is a point in time where the position of whatever it is you want to animate is at a 'key' (meaning 'important') position. i.e. if you have a windshield wiper that goes left to right, the most left position is a keyframe and the most right position is a keyframe.
But when you ask your software to interpolate these keyframes it will slide the mask from left to right without taking the rotation of the windshield wipers into account, so you need to correct for that by adding additional (and this is where the potential confusion starts) keyframes.

But to get back to - I think - your question (partially) you could for example track the face of the person you need to roto out with a tracker and then connect the center of your polyline to the result of that tracker. This will make the mask move correctly for 80% of the time (and will take care of some subliminal movements that you'll most likely miss when going fully manual) so you need to adapt the shape of your polyline less often. But it will not be the same AE does.

What I've been trying to ask if there's any way to have that magic mask with the ability to move the "mask" or whatever for the rotoscoping frame by frame, without it affecting the rest of the already set "masks" (or whatever) that are already set since they did the tracking.

No, with respect to the Color Page's Magic Mask tool, it doesn't produce any user accessible animated spline/polygon/mask shapes that you can thereafter manually refine ... you could only manually augment the overall key produced by using it, by combining it's key with additional keys from manually accessible masks eg by using a Key Mixer node.

Hey! Alrighty, So I have some footage in Davinci Resolve that needs to have the background footage taken out. Here's my problem: No green screen footage. Person that I need to take from the footage FROM the background walks from out of the screen into the screen. ANOTHER problem is that when the person is walking into the screen, they're farther away and then walk up and then off the screen at a much closer distance than when the walked in.

My observation: some things are better done by really experienced people using more complex tools. If I were you, I would hire an experienced VFX person to do this instead. Here's some people with which we've consulted to do rotoscope masks:

http://rotomaker.com
(213) 308-6500

https://www.tracevfx.com/
(212) 380-3047

https://www.studio8fx.com/
(888) 399-0409

Most of them will let you send them a low-res preview and they can give you a bid on what it will cost. If you want to go through a course and practice, of course you can do that... but that'll take weeks. Hire somebody, it's done in a few days. It might cost less than you think.

If the camera is locked off, you may be able to create some sort of difference matte by using the first frame where the person is not in the frame to find the difference between all subsequent frames that have the person in the frame. I created an effect like this for Lightworks but I’m not sure how exactly this would be done in Resolve.

With so much nature around, if there's any movement from wind you can forget difference matting.

Maybe this will work

Sorry,
Whenever I'm saying terms, I'm probably not saying the right words that are technical (which I've never really been a good person to remember the TECHNICAL terms, sorry)

I THINK some of you are kinda getting what I'm asking about.

I'm basically asking (which I'm kind of thinking some of you are explaining it to me, but I'm still so confused).

I need to track the person's movements. I understand I can do bit by bit (pieces as to what part of the person I am going to do).

I am ALSO asking if there's any way to have a bit of a tracker that will help me get the ROUGH idea of the outline of the mask (or rotoscope WHATEVER THE WAY IS), and then I can fine tune it.

In regards to the latest video that was sent with the dog, the problem with that, is that the dog is kind of moving slowly and not ENOUGH to have the difference.
I've tried using this thing, however, whenever I've tried to fine-tune it all, I had to draw lots of "red lines" (I honestly don't know the terms, but I'm sure everyone understands what I'm meaning by this. AGAIN, I'm sorry, I'm not a technical person. I just understand visuals)
With the fine tuning of the person/object, I obviously had to use the "blue lines" to select the object. Then to fine-tune it, use the "red lines". But whenever the slight movements, I had to use the "red lines" to fine tune AGAIN, but for some reason, this then sticks for the entire rest of the clip.

What am I missing?

AGAIN, I'm sorry that I'm not understanding if some of you are explaining what I'm looking for. I'm not a technical person so it's difficult to remember terms and such (in my explanation of "red lines" and "blue lines".)
Please forgive me.

Reshoot the scene or hire a VFX artist to do it for you.

You may also want to look at this one too. Tracking is not straightforward. You may need to do it in very short sections to get closer keyframes with the corrected mask.

Perhaps a combination of separate color key for shirt, face, legs + roto where keying won't work? Also boost saturation to make features more keyable?

Victoria, are you able to share the whole video clip? Ideally uploaded to a file sharing site like Google Drive, or failing that YouTube, or Vimeo?

You might look into this: https://github.com/PeterL1n/RobustVideoMatting

Heres another one for you Victoria:

Reshoot the scene or hire a VFX artist to do it for you.

This is actually a pretty good idea. I've worked on projects where they carried around a portable green screen and just placed it behind the actor(s) as they walked, and it's amazing how effective it can be.

Rotoscoping is hard, hard work, and flawlessly cutting somebody out of one place and into another... that requires some real skill, plus you need some expert "light wrap" tools to make it look like the person is being illuminated in that area. This is harder than you think: look at the $150M Marvel movie "Guardians of the Galaxy 2," and there's some awful VFX composites in some of those flying scenes with Kurt Russell. Random surfaces like hair are particularly rough. I suspect this was a time/money thing where it's the best they could do with what they had, but still, you see if it can't be done on that level, it's going to be hard for modest-budget filmmakers to pull it off.

Magic Mask can work under ideal conditions, but it's very processor intensive and tough to get going in a lot of situations. I think it does at least show that work is going on in this area, and automatic rotoscoping is not that far away. But I suspect it'll always need some human judgement and intervention.

Reshoot is good idea only if it is cheaper and/or faster depending on what the priority is. In a lot cases it doesn’t really tick neither boxes.

If it is something that needs to be done correctly I’d look at outsourcing the roto work as already suggested by others.

If it's not a commercial job, try a few of the suggested approaches and remember these famous words: "Experience is what you get, when you didn’t get what you wanted."