Page 1 of 1

Difference keyer howto?

PostPosted: Tue Oct 24, 2017 7:00 am
by Umberto Uderzo
I'm trying to use the difference keyer.

The shot is as follows: no camera movement (on tripod, no pan/tilt and so on). A nearly steady landscape (some branches with a little wind floating, but mainly it's a still) and a guy entering the scene from the side and placing in the middle of the frame.

So... i tought it were and easy step to key with the difference keyer using as a clean plate the first frame of the sequence (where the guy is not visible). But that's not really so easy. I expected some holes in the matte on the guy shape due to some color matching with the clean plate but i didn't expect the matte to be so inaccurate on the landscape itself.

Is it possible that the micro color waviness due to the sensor over time can produce so noticeable defects?
I double checked with a tracker and i can see there is no movement at all of the camera...

Thanks!

Re: Difference keyer howto?

PostPosted: Tue Oct 24, 2017 7:32 am
by Hendrik Proosa
Overall landscape is irrelevant, you only need the close proximity of the subject to be ok. Clear everything else with a garbage mask. That said, difference matte is rather fragile, all kind of micro movement and noise jumps right out and if the bg and fg are similar, you might get holes.

Take the mask, clean it up with garbage and core mattes and fix broken areas with roto.

Re: Difference keyer howto?

PostPosted: Tue Oct 24, 2017 12:13 pm
by Umberto Uderzo
Well, yes the difference keyer is really sensible to subtle changes.
Tried also to simulate it with a Channel booleans and the results are very similar.
The problem is that i come out with a lot of noise around the guy shape so... i don't know if it's better to directly roto manually. Tried also changing color space and it's going slightly better but no way a good result.

Mumble.

Re: Difference keyer howto?

PostPosted: Tue Oct 24, 2017 12:21 pm
by David_Cox
If noise is the root of the problem (not sure whether it is), try blurring both your plate and foreground shots by an equal amount before the difference keyer. It might help a little.
DC

Re: Difference keyer howto?

PostPosted: Tue Oct 24, 2017 2:35 pm
by Sam Steti
Umberto, it must be me but... what are you trying to do specifically ? :)

Re: Difference keyer howto?

PostPosted: Tue Oct 24, 2017 4:28 pm
by Umberto Uderzo
I attach a screenshot (zoom in as the picture is full res):

Viewers show a 100% crop of the picture. The top left is the first frame, the top right is the frame i'm trying to key, the bottom left is the output of the difference keyer and the bottom right is the best i could achieve until not without manual rotoscoping (a cascade of rank nodes to try to clear some random spots)

Trying to perform a difference between the first frame and the current frame with a Channel boolean node shows some foliage movement so i bet this is the reason i don't get a nice output...

Re: Difference keyer howto?

PostPosted: Tue Oct 24, 2017 4:45 pm
by David_Cox
To be honest, I think you have done relatively well. Difference keying is never the ideal solution it promises to be on paper. The leaves in your case will be moving enough to cause a difference, and this will be exacerbated compression in the recording format. A small leaf moving could change the compression profile for a significant area.

DC

Re: Difference keyer howto?

PostPosted: Tue Oct 24, 2017 5:07 pm
by Umberto Uderzo
Thank you for the reply David.
I got it about the leaves movement... i think i'll refine with some roto painting down the pipe.
About the recording format... i'd hope that the Canon 5D3 RAW files were enough free from compression artifacts. Maybe it's the sensor that doesn't spit out enough color quality? 14 bits doesn't mean much if the sensor can't hold some color stability from frame to frame... ?

Re: Difference keyer howto?

PostPosted: Tue Oct 24, 2017 5:16 pm
by David_Cox
I was looking at the bottom left image and mistakenly thought I saw some blockiness in the key due to compression. But looking again, I think the key is just breaking up naturally between the green shades of the shirt and the similar shades of the leaves in the clean plate. The blocky appearance is just an illusion of the transparent checker-board probably.

To be honest, I have yet to do a comp job on any software using difference keying where that process gives anything more than a starting point. Make your adjustments to get the edges as good as you can. Filling in the holes in the middle is easier than roto-ing edges :-)

DC

Re: Difference keyer howto?

PostPosted: Tue Oct 24, 2017 6:30 pm
by Sam Steti
Hi there,

Half roto should be the way to go : I mean, for example for the left side of the clip, if you use a garbage matte whose right side is the guy's shoulder roto and the the whole rest (at his left) is covered by the polyline, the job's done on this part; then you add another polyline piped into the first one and do the same on the right side, a great % of the entire job's done...

Re: Difference keyer howto?

PostPosted: Wed Oct 25, 2017 9:37 am
by Hendrik Proosa
Seems like one of the worst cases for difference keying: guy in green shirt against green nature background. Only camera movement and strong wind is missing to make it "perfect".

I don't know what quality level you are after, but this shot just screams for full roto if you need it to be better than somewhat okay.

Re: Difference keyer howto?

PostPosted: Wed Oct 25, 2017 4:08 pm
by Umberto Uderzo
Thank Sam and Hendrik for your replies.

Well, yes this shot would require a full manual roto, i just wanted to know how far i could push the difference keyer before giving up. I tried also with manual rotoscoping and i must admit that (it must be me) i find the Fusion way a little clunky (but i'm not a professional roto artist, so it must be me).
What i find difficult is the need to track some details, tie the shape to the trackers and a lot of this "low level" stuff that simply Mocha Pro doesn't require due of its nature.

Ok ok, i can do the roto in Mocha but i want to learn Fusion so i guess there must be some functionality that i didn't yet find in the manual or by googling that would simplify the workflow.
For example, one cool thing in Mocha is that before starting to refine the roto shape you can alter the bounding box without adding spline vertex keyframes. This let you to make the shape more or less following the subject and when you start refining the shape details you are not annoyed by all those "bounding box" keyframes to break your shape interpolation.

Is there something similar in Fusion? I couldn't find any "bounding box" keyframing level. Should i simulate it with a transform node?

Re: Difference keyer howto?

PostPosted: Wed Oct 25, 2017 4:25 pm
by Sam Steti
Umberto, you do the roto the same way in Fusion... Don't have a lot of time so far, but you could go to your goal quicker than you may think, without Mocha (that I know very well btw).
You need the guy alone, don't you ?

First forget about this difference keyer : try working with a Primatte one, and use garbage mattes, inverted and/or tracked if needed.
Then add a matte control (anyway)
If you try different keyer piped from the loader, join them in a matte control to, with combine : alpha; operation : minimum.

Re: Difference keyer howto?

PostPosted: Fri Oct 27, 2017 1:11 pm
by Umberto Uderzo
Well Sam...

I really cannot wrap my mind around your hint. Tried with Primatte but the result is really awful as the clear background noise counter acts the clear foreground noise because of the color similarities. Moreover, i couldn't find multiple primatte configurations that merged give me something better than a single primatte alone. So far the difference keyer looks like the one that produced the best result.

Obviously i'm lacking some fundamentals on how to leverage on Primatte efficiently.

So I choosed to switch to manual roto, and to use Mocha as i really don't find a roto technique in Fusion that gives me the speed and precision on motion tracking that Mocha gives. I suspect i'm missing something about rotoing in Fusion but if i didn't watch all rotoscoping tutorials in Fusion i nearly did it :oops:

Re: Difference keyer howto?

PostPosted: Fri Nov 10, 2017 5:46 pm
by Umberto Uderzo
Well... ok at last i managed to finish (or nearly finish) this self-learning shot.
This is my very first "serious" compositing, and i'm partially satisfied while some defects are noticeable :D


Re: Difference keyer howto?

PostPosted: Fri Nov 10, 2017 5:55 pm
by David_Cox
Nice work - well done :-)
DC

Re: Difference keyer howto?

PostPosted: Sat Nov 11, 2017 4:42 pm
by Aurore de Blois
i realize you have completed your shot already, but just looking at your screengrab [it was a bit small though so maybe i am wrong] the 'cleanplate' needs to match the background of your p0late exactly.
it can't be 'close' or not even close.
what i see at left does not match the bg in the plate at right. looks like your cleanplate is on a different frame or perhaps it was created [?] and just wasn't lined up?

Re: Difference keyer howto?

PostPosted: Sun Nov 12, 2017 12:07 am
by Umberto Uderzo
Thanks for your comment David and Aurore.

@Aurore: Well the shot was an intermediate version, i realized a newer one with a more elaborated particle motion. Beside that, i'd like to explain how the compositing is structured: the original video footage in the final compositing is the front rock with the guy, the background is only a still photo (and badly cloned to make it wider). I tried to match photo and video colors with the CC hystogram auto match function. The photo is projected on to a spheric surface. The initial footage were shot on a tripod so the camera movement is achieved using only the Fusion 3D Space and projections (and several defects are visible due to projection on simple basic meshes). That said, i guess you are referring to the background photo?
I enjoy critics as they help me to grow in technique.

Thank you!
Umberto