Replace Sky Tracking

Top right corner of the frame has collapsed.

Screenshot 2024-04-19 at 18.41.47.png (576.17 KiB) Viewed 1644 times

I have been playing with the Replace Sky effect. I have a clip of a car driving down a long straight road, from the passenger perspective. There are hills on both sides. The sky was very dull and I wish to replace this. A straight replacement works fine, other than it does not move and therefore looks wrong. Instead I thought of tracking it. Track Original Sky does not work well at all. The track foreground works very well, apart from the fact that after a few seconds, the top right corner of they frame starts to collapse in. What am I missing? Is there a way around this?

See if this help any.

Thanks. I had already watched that. I have followed this. What I can’t work out is why when track foreground is selected does the sky start distorting and then collapsing.

I am doing this on a Mac desktop with Blackmagic GPU.

I have a clip of a car driving down a long straight road, from the passenger perspective. There are hills on both sides. The sky was very dull and I wish to replace this. A straight replacement works fine, other than it does not move and therefore looks wrong.

I'd say the problem there is that you're trying to track something in 3-dimensional space, which is beyond the limit of what this tool can do (to me). You can do simple stuff, like a pan from left to right with people walking on a beach. For a car driving down a road... that's a whole thing.

I can refer you to books and VFX artists who can provide clues as to the mystery of tracking composites in 3-dimensional space (like rapidly moving forward), but it's not something I'd expect to do automatically. The level of difficulty on something like this is considerable.

If I'm confronted by something like this, I'd get a large sky plate from one of the VFX stock footage vendors out there and try to do a manual track as best I could. If the car is bouncing around and the camera is handheld, I'd call a VFX artist, send them the shot, and ask for an estimate. You could go through a Fusion course and probably learn it yourself, given enough time -- a terrific resource for Fusion training is Bernd Klimm's VFX Study:

https://vfxstudy.com

Thank you. I thought this may be the case. I will probably just try some simple colour correction instead.

It just shows that although these are great tools, they do have limits.

You can use SynthEyes, which now belongs to BorisFX. Not cheap, but excellent. I think there's a time-limited demo.
It'll generate a 3D camera track from your footage, which then can be used with a high resolution sky template in Fusion. SynthEyes can even export a scene right into Fusion (or Blender, or about 20 other formats).

Fusion has camera tracker built in, for someone who is new to tracking, it would be first thing to try.

Worth a try, but I'd doubt it'll hold here. It can work well with human-made environments, but repetitive natural shapes are very tricky.
I recently had to track a drone shot over very regular plant structures and SynthEyes got it right, while Fusion, AEFX and C4D failed.

Solver doesn’t know anything about what the content of the shot is and doesn’t care one bit. Problem is in the autospawned feature tracks. Clean those up and problem disappears. Ofcourse when autotrack can’t produce usable feature tracks then there is a problem. Then dedicated matchmovers come to resque.

Thanks for the pointers. I am very new to Resolve Studio and there is a lot to take in but I am learning fast! Version 19 does seem to have some great new features. It really is a great product for a modest cost.

Yeah, thanks, I know about that, I didn't want to make things too complicated without knowing if the OP will go further.
BTW, Jason, if you don't have a high-res sky still, you can generate relatively credible ones with fractal noise generators, scaled stronger horizontally and tilted towards the horizon. A bit of fading to lower contrast at the distance and most viewers won't care.

Thanks for your help. For the moment I am just going to have a bluer sky fading from the horizon upwar4dfs. This seems to work. Thie issue is trying to get the clouds working. I have got many more things to learn before jumping into this! I had just assumed that the replace sky effect would do everything for me. Cleverbut it does have its limits.

Hi Jason, I’m getting the same, just can’t remove those black parts after tracking. Did you find a work around? Cheers, Chris