Jump to: Board index » General » Fusion

Repairing a set of 15 contiguous frames

Learn about 3D compositing, animation, broadcast design and VFX workflows.
  • Author
  • Message
Offline

Umberto Uderzo

  • Posts: 133
  • Joined: Fri Mar 13, 2015 12:19 am

Repairing a set of 15 contiguous frames

PostMon Feb 12, 2018 3:19 pm

I'm trying to repair a set of 15 contiguous frames from a drone footage.
The drone is dollying backwards and at a certain point there is some little shaking due to wind and gimbal being not able to stabilize it.
I initially tought about the Repair Frame tool but really it didn't live up to the expectations, maybe because it is designed to address single frames issue?
Should i concentrare on the Tween tool trying to completely rebuild the 15 frames sequence by interpolating the extremes?

Thanks!
Offline

Sander de Regt

  • Posts: 719
  • Joined: Thu Nov 13, 2014 10:09 pm

Re: Repairing a set of 15 contiguous frames

PostMon Feb 12, 2018 11:10 pm

Do you mean 'repair' or do you want to stabilize the shaking frames?
Sander de Regt

ShadowMaker SdR
The Netherlands
Offline

Umberto Uderzo

  • Posts: 133
  • Joined: Fri Mar 13, 2015 12:19 am

Re: Repairing a set of 15 contiguous frames

PostTue Feb 13, 2018 6:40 am

Thank you Sander for your reply.

I mean repair. The footage is already stabilized but due to wind conditions in this frame interval the electronic image stabilization is visible and the frame, while stabilized, shows blurred edges. I'm speaking of about 3 or 4 pixels horizontal blurring which is visible while watching the total sequence.
I guess it's impossible to fix each single frame per-se, so I'm wondering to leverage some repairing technique that takes avail of surrounding frames.
But I couldn't manage to get something good from an entire 15 consequent blurred frames sequence.
Offline

alan bovine

  • Posts: 66
  • Joined: Mon Aug 07, 2017 6:20 pm

Re: Repairing a set of 15 contiguous frames

PostTue Feb 13, 2018 7:53 am

Thats because you're asking for too much. You're asking Fusion to come up with 15 brand new frames in a sequence; which is a huge task to get right.

No (currently) software can do this automatically with good, predictable results.

A more predictable outcome that is more or less guaranteed to work better; requires that you bite the bullet and do 3d tracking*, geometry reprojection and a bit of roto.


* 3d tracking will be an issue if you're doing image stabilization in the camera/sensor; as its skews and distorts the geometries of the scene by defaullt.
Fusion video tutorials : https://www.youtube.com/channel/UCTCeDas53OEcWcRujkQiwLg/videos?view_as=subscriber
Fusion Tools : https://github.com/statixVFX/stx_tools
Nuke 2 Fusion nodes : https://github.com/statixVFX/nuke2fusion
Offline

Umberto Uderzo

  • Posts: 133
  • Joined: Fri Mar 13, 2015 12:19 am

Re: Repairing a set of 15 contiguous frames

PostTue Feb 13, 2018 8:10 am

This is the reason i was wondering about interpolating the entire frame sequence (it's half a second maybe it would not be noticed by the audience), but i currently could not manage to sort it (yet)
Offline

Sander de Regt

  • Posts: 719
  • Joined: Thu Nov 13, 2014 10:09 pm

Re: Repairing a set of 15 contiguous frames

PostTue Feb 13, 2018 9:21 am

Any chance you could actually show the piece of footage to us? This is pretty theoretical at this point.
Depending on the speed of flight and the amount of perspective you could even 'slowmo' between the two good frames, but like Alan says: 15 frames is *a lot* to come up with from scratch.
It sounds like you're seeing the result of an overactive stabilizer, that stabilizes so much that the original motion blur is visible but 'in place'. Which will look weird.
Photoshop now does 'automagically' remove camera shake, maybe you could batch these images through there. Or even add some shake back into the orginal shot so it won't be as stable, but will look more natural.
Sander de Regt

ShadowMaker SdR
The Netherlands
Offline

Hendrik Proosa

  • Posts: 174
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Repairing a set of 15 contiguous frames

PostTue Feb 13, 2018 9:45 am

Actual footage would be helpful here as Sander said. The number of frames to interpolate is not that relevant, the amount of movement between last good frames and the type of movement (mostly rotational or also movement in space) is what mostly dictates the result. If overall movement is slow, 15 frames can be regenerated nicely using optical flow. But if image changes a lot parallax wise, it will get complicated and will need tracking and scene reconstruction with projections and roto.
I do stuff.
Offline

Umberto Uderzo

  • Posts: 133
  • Joined: Fri Mar 13, 2015 12:19 am

Re: Repairing a set of 15 contiguous frames

PostTue Feb 13, 2018 9:56 am

Too bad i don't have access to the footage right now.
When in studio i'll first try a "slomo" of 15 frames to see if it can address my issue.
It should work as it's a backward dolly, i just need to check if the interpolation will work with surrounding tree branches.
In case of failure i'll post a link to the video segment for hints.
Thanks!
P.S. It's just a self training, i don't have urgency to get things done.
Offline

Umberto Uderzo

  • Posts: 133
  • Joined: Fri Mar 13, 2015 12:19 am

Re: Repairing a set of 15 contiguous frames

PostWed Feb 14, 2018 2:49 pm

Ok, interpolation is not an option because of a lot of artifacts are introduced in the middle of the interpolation.
May work with a couple of frames but no more.
I found a decent fix for the sequence in the following way:
I add a repair node, then feed it as foreground to a merge node. The background is the original footage.
As the 15 frames sequence doesn't show the shaking continuously but it's on groups of 1 to 3 frames, leaving a decent frame from time to time, i activate the repair node only when necessary by adding keyframes to the blend parameter of the merge node.
Not very speedy to work with but looks like it's working.
Offline
User avatar

Marc van Vliet

  • Posts: 42
  • Joined: Thu Nov 13, 2014 1:42 pm

Re: Repairing a set of 15 contiguous frames

PostWed Feb 14, 2018 4:33 pm

Another approach that might work in certain situations is to track the motion of the shot, then freeze the last ok frame, and place that on a static Imageplane with the tracked camera, then the camera moves backwards and when rendered, that (sharp) last frame is overlayed onto the original. You could even take the first ok frame and do a blend to an Imageplane with that over the 15 frames between those. Position that frame somewhere further back along the camera track so that it is fullframe when we arrive at it's original timecode...
Offline

Chad Capeland

  • Posts: 1547
  • Joined: Mon Nov 10, 2014 9:40 pm

Re: Repairing a set of 15 contiguous frames

PostWed Feb 14, 2018 4:49 pm

Agree that's the approach I would take. Track the bad section and stabilize, track the good section and project the bad footage to the good camera move. Assuming the cameras can be tracked and the footage isn't otherwise messed up by motion blur or rolling shutter. If the artifacts are bad, use a tasteful lens flare or flock of birds to cover.
Chad Capeland
Indicated, LLC
www.floweffects.com

Return to Fusion

Who is online

Users browsing this forum: Stefan Kubicek and 10 guests