Manually altering planar tracker in Fusion

Hi there,

I'm sure this is possible and I've really search all over, and read the manual too but I can't work out how to manually edit tracking data in Fusion.

I'm using the Fusion tab in Resolve 15.
I've got my clip, added a Planar Tracking node.
I've marked out my tracking area and I've got the auto tracking working just fine but there are points where the auto tracking gets a bit off. So I go into the shape where it's gone off track, and pull it about a bit to make it fit again. But now if I go backwards in time these changes have been applied to all previous frames.

From my reading I've gathered that I think I need to create keyframes for each time I make a change to the shape. But I've tried loads of things, and I've searched all over but I can't work out how this is done.

Can someone please help?

Many thanks,
Ben

Wherever you use planar tracking (mocha, fu, nuke etc) the principle is the same: shape does not represent the tracking data. Shape provides the area that tracker follows, but underlying track itself is not the shape. Instead, shape is attached to underlying track and moves because of it. Now, if you change the shape, it does not change the track itself one bit. If you don't keyframe the shape and reshape it, this change propagates to all frames and will be off in other frames.

To my knowledge you can't change the actual tracking data of planar tracker in planartracking node itself.

Hi Hendrik,

Thanks for the reply. I think this has clarified things a bit for me. I'm still struggling to do what I want though. I wonder if I could explain it could you point me in the direction I should be going.

I have a person in the shot that is moving around in front of a fairly complicated background.

I would like to remove the background. So I was trying to use the planar tracker to track her movement and then use the "Mask" output of the planar tracker to mask the original footage to remove the background.

But as the track is not perfect, I would like to manually alter the tracker shape (and the mask it produces) to fix it where the track gets lost.

Am I going about this the right way?

Thanks in advance.
B

I have spent weeks - off and on - trying to solve this. (Resolve 16.1.1 fusion)
The planar tracker often works the best of all the trackers - non-wobbly/jittery etc.
But seems to have the least options to modify the tracking data.

The color page window tracker is a breeze to keyframe/alter/adjust and works great.
The fusion tracker can be modified by hand and/or converting the spline points to x/y data.

The fusion planar tracker?!??
Wish I had a clue after so many hours of trying.
Basically seems nothing I do works.
Moving nor keyframing the tracking window doesn't change it.
Setting new keyframes doesn't change it.
The keyframes in the spline editor are locked......
and even unlocking them (right click a spline keyframe - select set key - move it - nothing)

So... if I have a basic scene where the tracking subject goes in/out/in of frame:
1. I stop the track
2. Click the cancel track data to end
3. Keyframe the blend of the planar tracker node to 0.
4. Create a new tracker node to track when subject begins to re-enter frame.

Or - really - switch to the basic fusion tracker and tweak/hope for non-jitters.

I have read the manual, searched for many hours on the interwebs and utubes.

Would be great if BM could make fusion trackers work more like the color page window tracker.
Or at least more user-friendly/editable.

Anyone with a clue that can help?

At the moment, the Planar Tracker is a black box. We can't modify any of the data it produces.

You can reduce jitter using the regular Tracker node by tracking more than once with slightly different patterns. The result will be calculated from the average of the active trackers. I usually go once forward and once backward at least to remove statistical noise. If you're only tracking one feature like this, turn off Rotation and Scaling. Having two slightly jittery trackers very close together will cause problems.

You can commit the planar tracker to a plain transform node after the tracking is complete. Then you can edit the curves of the transform like you'd edit any other.

I also do that because the image quality produced by the planar tracker's algorithm is not great. The regular transform node offers several good options to choose from.

Is that new in a more recent build than I have (my installation of v16 hasn't been updated in a good long time)? Because the only thing I see is the Create Planar Transform button, which makes the PlanarTransform tool, which has only the single, useless spline that locks up the entire matrix in a single un-editable value.

Bryan, I don't think it's a newer build. I think you guys are talking about different things.
Unless you only use a planar tracker set to position/rotation/scale, the results can't be reproduced in a normal tracker as far as I know, but maybe Igor can show us something we don't know yet.

I was partially incorrect. At the bottom of the Planar Tracker there is a button that creates a planar transform with baked in curves. Not a regular Xf transfom. This planar transform has a displacement curve. I don't know if there is a way to convert that curve into something more user friendly.

Darn; I thought you knew something I didn't!

Darn; I thought you knew something I didn't!

That would be hardly unlikely!

Here is a kludgy idea. Track with the planar tracker, create planar transform when done.

Use the planar transform to unsteady a tracking target. The target can be a black screen with a couple of hash marks.

Use the point tracker to track the unsteadied target and you end up with editable XY curves.

Pretty nifty idea.
That's one thing great about Resolve/Fusion,
there are often 5 different ways to do many things.

I find using 'append' to the basic fusion tracker the easiest for problem tracks.

track8.jpg (27.85 KiB) Viewed 14434 times

The way I deal with incomplete or partially successful planar tracks in Fusion is to add a Grid Warp after the planar transformation. A Grid Warp set to 1x1 can be treated as a plane and then manually adjusted on frames that gave me issues in the Planar Tracker (or to create additional frames beyond what the tracker was able to capture).

By setting the Grid Warp source positions to a rough bounding box around my source Planar Tracker shape the subsequent manual adjustments can be made to very closely match the Planar Tracker output itself. So on any jittering frames I'll delete the offending Planar Tracker keyframe and adjust the shape myself by keyframing the Grid Warp, etc.

To make the manual adjustments easier I pipe in a still frame created by a TimeSpeed node (set to the Planar Tracker source frame) to create a difference matte for comparison against my Grid Warp animation.

Keyframing the grid this way against a reference image basically *is* planar tracking, just done by hand, so you can accomplish whatever you need with it.

Maybe of interest to you:
I found a way how to manipulate the planar tracker manually as a workaround for not being able to alter its path directly (in contrast to Tracker or Cloud tracker):

The way planar tracker currently works is that it requires an object to track no matter what - otherwise it does not record a path. Therefore a workaround is to provide an according object artificially to the Planar Tracker, which it can thentrack.

This can be done e.g. by adding a paint node after the background node and before the Planar Tracker node and by painting for example a white bubble per frame with the desired movement you want to record. The planar tracker recognizes this bubble and starts to track it (as long as the white bubble is within the your tracking polygon).

After you have recorded the path of the bubble for the frames where you wanted to do the manual adjustments, you can delete the Paint node again (and e.g. resume the tracking from there onwards on the regular material with the same tracker by continuing your tracking).

--------------
PS: If the setting "Hybrid Point/Area" doesn't work for you for the above technique, try "Point" (for the "Tracker" setting). Noteworthy you can mix both methods within one tracker on different frames.

PS: By the way as a side remark: The white bubble can be painted on top of the original object (e.g. if it is occluded for a couple of frames), which would probably be the easiest and most convenient way to do it. But interestingly it does not even need to be:

That is because the tracker seems to primarily record RELATIVE movements. So if you prefer you could even paint the bubble somewhere else. For this you only need to move the tracking polygon manually over to the white bubble and resume tracking next frame.

If the tracking doesn't start because you are on an untracked frame (an according message pops up), you can simply set a key frame in the inspector (in Version 16 next to where it says "Right-click here for Track spline").

If the tracking doesn't continue at some point the distance between your white bubbles might be too far from frame to frame, so that the tracker doesn't recognize it as one coherent object. In this case make the distance smaller (in other cases changing the polygon might help).

Finally if you want to check if the overall tracking path works for you, you can shift the polygon back to the original posititon and see if the tracking path (represented by the (relative) polygon's movements) does what you want.

I have a weird behavior of the fusion tracker. I have chosen a planar tracker, and from inspector I have chosen a "Hybrid Point/Area" tracker. I have created a polygon to track within.

There is a button on a toolbar up from the viewer with an [x] and hint 'delete point' but it is grayed out and does NOT work. For unknown reason I could not make it work to delete points which are obviously outside the good area. See the "1" and "2" on the below image.

fusion planar tracker 01.png (888.4 KiB) Viewed 9644 times

When I do the tracking sometimes I can see the polygon how it is changing between frames when I see it goes haywire I can stop and correct it slightly by modyfing it's spline points. I completely do not understand why it sometimes is NOT visible and why sometimes it IS visible and I can modify it.

What is more weird: sometimes it disappears suddenly from frame to frame and I cannot manipulate it anymore. I can still see the points mesh but the possibility to manipulate polygon totally disapears. Why? Does anybody have a clue wtf is going on?

fusion planar tracker 02.png (835.84 KiB) Viewed 9644 times

Shortly after this the tracker goes crazy and starts tracking some complete nonsense. And I cannot correct it anymore:

fusion planar tracker 03.png (297.35 KiB) Viewed 9644 times

Even when I press 'delete till end' or when I am trying to do things which should make sense: I cannot recover from such situation.

I have a feeling that the tracker in Color page is magic and works always like a charm. The trackers in fusion look like made by Corel company: totally buggy and working unexpectadly.

@damago1 Do you see any improvement since last year mid Jan? I read this topic as I wanted to put a sign on the road surface. The tracker works for a while then move the sign a few points upper. Not sure what to do I tried to track by color and with different options.
I'm not an expert in this area at all. However if I were you I would give a try to track red, green or blue instead of the default "LUMA".
I think the tracker cannot find change on the picture in the marked area, however found movement other part of the screen, hence tried to follow that one instead of stop following.
In my case when it starts to follow outside of the area then the position of the sign on the road also changes which makes the whole sign unrealistic.

for the planar tracker I think it would be better to track only parts, like the head, of the snow men.

For the streetsign tracking, a 3D camera tracking might work better than planar trackers.