Removing Footprints from Snow in a Descending Drone Shot

Hi folks! I'm an experienced editor but new to Resolve and Fusion; relatively new to compositing. I've got a nearly finished music video with a good bit of VFX and the one shot I've got that I thought would be the most straightforward is anything but. It is, of course, the one shot that I tackled with a — albeit necessary — "fix it in post" approach, and it is, of course, the only vfx shot in the video that is intended NOT to be noticed. Any guidance would be appreciated as I have spent days trying to figure it out after watching countless tutorials and poring through the manuals and guides with no success thus far. I'm using the studio versions of Resolve and Fusion 18.

The shot is of two characters charging from hundreds of feet away across a snow covered frozen lake before clashing into each other (whereupon they engage in a musical duel on the moon. It's going to be awesome.) To get the shot I had them start at the end, inches from each other with the drone camera several few feet over their heads, and then had them run backwards away from each other as the drone climbs to a few hundred feet in the air, keeping them towards the edge of the frame. When played back in reverse, well you get the idea.

What I need to do is 1: remove/replace the footprints that were made by them walking out to their marks and 2: remove the footprints they make as they run backwards, but have them appear behind them as it plays back in reverse, to complete the illusion. If I can just get the first problem solved I expect I'll be able to solve problem two by extension.

My primary issue here almost certainly stems from my as-of-yet woefully incomplete grasp of the core principles in how to connect the various nodes. In particular I am still rather perplexed by why certain nodes must have multiple outputs and to which inputs they must connect in order to complete an effect. Bt sometimes they don't! Woefully incomplete, I know. I'll get so close to workable solution but, for instance, the foreground input of a merge won't track with the background, or it will track but it wont overlay the background how I might have expected.

Here is a non-exhaustive but nevertheless representative list of the different ways I've tried to skin this cat. I began with the obvious: TimeStretcher to create a clean plate + Paint clone brush, and although that sort of works, the extreme change over time from near the ground to very high up means continuously expanding the regions of footprints being replaced and having to append the Planar Tracker multiple times. Everything goes out of sync. So I tried keeping the same flow general flow but instead of the clean plate I used the Patch Replacer (alpha channel) in the Fusion page of Resolve with long narrow polygon masks. That works well to account for the change over time but I could not get the Patch Replacer source and target to accept tracking data — even with a single point tracker instead of planar. I saw on these forums that I was not the only one, so I manually (and tediously) keyframed them but then encountered the problem of not being able to get multiple polygonal masks to work together; the foreground mask would disrupt the background. Having felt I learned something, however, I reattempted the Clean Plate/Paint node approach and discovered that I likewise could not get multiple polygons to work together. I then tried to create a Paint Group to keep all my brush strokes to a single node and connect that to the Planar Tracker. That seems promising but I'm completely lost on how to connect the group to the tracker. Expressions? Maybe, probably, but how? Connecting to unsteady tracker won't work (I don't think) because of the degree of rotation and scale. Or not? I just don't know.

I expect that any one of these strategies could or should work but they eventually come up short once I reach the delineation of my inexperience and conceptual deficiencies. Or I may be going about the whole thing absolutely wrong from the jump and ought to be utilizing some tool the existence of which I am still completely unaware. I'm diligent and conscientious but I simply don't know enough to even know what I don't know. I really need personalized guidance; I hope someone would be kind enough to provide some. Many thanks!

Last frame.jpg (87.99 KiB) Viewed 592 times

Mid Frame.jpg (88.35 KiB) Viewed 592 times

First Frame.jpg (94.37 KiB) Viewed 592 times

There are no one size fits all solutions for shots like these.

My first question is: does the reversing of the shot work? I've never actually run backwards, but I'd think that there's a chance that it looks fake no matter what you do.

But that's an artistic choice only you can make. As for fixing the shot, is there any chance you can show us the shot in motion? Is it a realtime (albeit reversed) shot or are you speedramping it as well?

Without seeing the actual motion it's hard to guide you on your approach. A lot of the things you mention sound like they *could* work, if applied correctly. But the big change of scale is what makes it difficult to work with planar trackers - as you've noticed - . It may not even be a case of appending planar trackers but layering multiple ones on top of each other. But it's a white plane with hardly anything trackable in there. If you can get a good 3D track of the shot, that would get you started much quicker.

But again: there are so many ways to approach it and it all depends on the actual shot, so if you can share that, that would be awesome. If it's under NDA you can also DM me to share the shot, so it can stay confidential.

This kind of big moves are complicated to do because the scale of area you work on changes. I would try to get a reasonable 3D camera track out of it, then project the image onto card and paint out the footprints there. Then render the patches out through the cam again, merge over footage and mask then as necessary (people, shadow areas etc). With good enough camera you get rid of the scaling and can do local paint by cloning etc. If necessary you can always project onto different planar geometries, for example divide the shot into three sections (closeup, medium and wide as drone ascends), and then dissolve from one set of patches to another to transition between the sections.

In general, what you want is to get rid of all the footsteps, then recreate the path that they need to leave when moving forward (in your time-reversed shot). And then reveal that path through a mask as they move forward, because that path is not created naturally behind them due to reversed time.

It doesn't make much difference whether you work on original clip or time-reversed one, as logic is simple: footprints paths have to be behind the characters in the final shot, whichever way it runs. I would leave any nonlinear time-remappings to the end, after comp, because then you can change the remapping if necessary. If you applied it first, then changing it would mess up your comp.

There are no one size fits all solutions for shots like these.

My first question is: does the reversing of the shot work? I've never actually run backwards, but I'd think that there's a chance that it looks fake no matter what you do.

But that's an artistic choice only you can make. As for fixing the shot, is there any chance you can show us the shot in motion? Is it a realtime (albeit reversed) shot or are you speedramping it as well?

Without seeing the actual motion it's hard to guide you on your approach. A lot of the things you mention sound like they *could* work, if applied correctly. But the big change of scale is what makes it difficult to work with planar trackers - as you've noticed - . It may not even be a case of appending planar trackers but layering multiple ones on top of each other. But it's a white plane with hardly anything trackable in there. If you can get a good 3D track of the shot, that would get you started much quicker.

But again: there are so many ways to approach it and it all depends on the actual shot, so if you can share that, that would be awesome. If it's under NDA you can also DM me to share the shot, so it can stay confidential.

Surprisingly it does look legit in reverse. The fact that it is directly overhead helps immensely in that regard. It is speedramped; much faster at first and slowing down a bit when they come into more detail. That makes it look kind of fake, but only in that it is obviously sped up. That isn't a problem really since it is a very comedic music video, and besides, it's 23 seconds long but the part of the song where it goes is only about half that. I was hoping to achieve something reminiscent of an anime fight scene — thematically if not visually. No, it's the footprints that give away the game.

I actually haven't had too much trouble getting good tracking data. Quite impressive actually, the footprints themselves are enough until about three quarters of the way through, and juicing up the contrast, gamma, lift, etc., seemed to help, but by the point that the footprints become too indistinct for the planar tracker to observe it becomes somewhat of a moot point.

No NDA but I've actually solved the first half, which I will explain in my reply to Hendrik.

This kind of big moves are complicated to do because the scale of area you work on changes. I would try to get a reasonable 3D camera track out of it, then project the image onto card and paint out the footprints there. Then render the patches out through the cam again, merge over footage and mask then as necessary (people, shadow areas etc). With good enough camera you get rid of the scaling and can do local paint by cloning etc. If necessary you can always project onto different planar geometries, for example divide the shot into three sections (closeup, medium and wide as drone ascends), and then dissolve from one set of patches to another to transition between the sections.

In general, what you want is to get rid of all the footsteps, then recreate the path that they need to leave when moving forward (in your time-reversed shot). And then reveal that path through a mask as they move forward, because that path is not created naturally behind them due to reversed time.

It doesn't make much difference whether you work on original clip or time-reversed one, as logic is simple: footprints paths have to be behind the characters in the final shot, whichever way it runs. I would leave any nonlinear time-remappings to the end, after comp, because then you can change the remapping if necessary. If you applied it first, then changing it would mess up your comp.

I've kept the original timing thus far, but it would have probably been worthwhile to at reverse the shot first because it gets really annoying hitting L when I mean to hit J, and vice versa, because instinctually I expect playing forward mean's they will be running forward, haha! Also, speeding it up to its nominal speed would shorten the clip and save some time on my increasingly outdated iMac.

At any rate, working in 3d space is coming for my next project already in the can but I am not ready for that kind of complexity yet (not is my computer. Hopefully an upgrade will be in the cards soon, I just need the first half of my settlement with Celsius to come available...

But I did find something that did work, at least for the first part of the problem. It turns out that following the
guidance provided in Chapter 81 of the Resolve 18 Reference Manual exactly as described works. I started with the last section of the chapter, Painting a Clean Plate, but encountered problems. I went online and on youtube and that sent me off running down blind alleys and cul-de-sacs. Returning to the manual and trying the previous example actually worked: MediaIn—Planar Tracker [steady]—Merge (background input); Background [transparency 0]—Paint—Merge (foreground input)—Planar Tracker [invert steady tyransform] — MediaOut. It worked. I removed all the footprints except those they make as they run backwards and exported it.

I then put that new clip in the timeline and di the same thing to paint out the remaining footprints. Obviouslyt my subjects end up with paint strokes all over them, but on top of that I placed another of the new clip in which I followed the same template in Fusion but eliminated the Background node and replaced the Merge and Paint nodes with Matte Control and Polygon, respectively. I then animated the polygon to elongate over the footprints as they make them. I did that twice for each subject, using unique tracking data for each and connected them with Channel Booleans.

I haven't figured out how to put the foot prints back in behind them as they run, but I'm fine with none, at least for now. I am SO ready to move on to the next part where they're dueling on the moon. Any quick suggestions, however, to point me in the right direction with putting the footprints back in behind them would be great, because I literally sat there for about twenty minutes just thinking (which I hate) and could not come up with how I might go about it without likely resorting to randomly connecting nodes again. No more of that!

Hopefully this helps someone else who someday, as many threads have for me. I hate it when someone asks a question similar to one I have on some forum (about anything, not just video stuff) and then they come back with "figured it out! Was just doing X wrong," or "I didn't check Y," or "Z was broken, but thanks for all the suggestions!" It's like, Yeah, but what about X, Y, or Z? IWhat did you do because that might be what I need, dammit! Anyway, I'm sure I'll be back with more questions soon, especially when I start with the 3D tools. Man, that looks daunting. Can't wait!

Hey

TLDR for me I confess, but I guess that if you can wipe out footprints all along the clip, the rest should work afterwards...
Now, my question which is why I write here : any chance to see the clip in motion ? or at least a striking part ?

So far, I recognize something I would put some Mocha among the Fusion comp, but how and for what technical purpose depends on the motion actually.