Optical flow and frames deformation

Would it be possible to track the optical flow for a sequence, then use the optical flow vectors to deform other sequence frames, or better different pictures?

Yes.

Got it. Sorry for the dumb question.

The optical flow nodes compute guestimated trajectories for pixels, its literally a vector map. Shuffle/ChannelBoolean the vectors out of the layer so you can use them on another image. For instance the vector distort, displace and vectorBlur notes can input RGBA vectors (vectorDistort and vectorBlur can also use the channel layer name "Vector")

You can also feed the vectors to a particle system, so you can drive particles with velocities from the image....

Beauty of fusion, just rewire stuff until something magically happens

Thanks Alan,

I was trying to mimick a Superresolution algorithm by tracing the optical flow for each frame then, frame by frame, deforming the previous and the next frame to match the current frame and merging everything toghether at 2x resolution then rescale to original resolution.
Ideally, should get a slight increment in definition but the vector map i'm getting out of the optical flow looks pretty unstable.

There is a tool called "Repair frame" that works in a similar fashion but looks way more stable in tracking.
Don't know what i'm doing bad.

You're on the right track, I've built a tool that does this, but its not meant for "super resolution" but rather for doing simple denoising. It does an average or a median of 3 frames, I use it a lot for denoising indirect illumination passes in CG. But the theory is the same, you push/distort the previous and next frame into looking like the current frame.

Check out my stx_vectorDenoise tool: https://github.com/statixVFX/stx_tools

Thank you Alan,
i'm glad to know i'm on the right track, i'll give a read at your page asap!
Cheers!

@alan bovine: How do you feed the vectors to a particle system, so you can drive particles with velocities from the image?
Can you enlight me!

Thanks,
Nico

Ok, I found a way to do it.
For those who are interested:
1. Assign a optical flow node to your video footage
2.Copy aux node to move the vectors to color
3. After your particle emitter use a pCustomForce node and pipe the output of the copy aux node to the image1 input of pCustomForce.
4. use a expression in ForceX like: "getg1d(px,py)" -> which means read the green channel of image1 input at position x, y.
5. ForceY: "getr1d(px,py)" - here I use the red channel.