Improve 3d camera tracker results with green screen markers?

Hey there,

currently, I am trying to track a green screen with slightly brighter green markers on it in Davinci Resolve Fusion 17 Studio with the 3D Camera Tracker (iMac Pro 3 GHz 10 core, 64 gb RAM, Radeon Pro Vega 64 16 gb).

However, on some shots the result of the tracking process is just a wiggle motion. I am working with footage of my BMPCC 6k in Blackmagic RAW 12:1. In order to get at least any results I have to put a LUT in front of the tracker. However, this isn't enough to get decent results: I've also tried an Unsharp Mask and Color Correction by adjusting brightness and saturation to make the tracking markers more visible (I am tracking in the green area).
Nevertheless, the result is still a wiggling image. I assume that the noise of the footage is the problem.

Do you have any tips to improve tracking results, especially with green screen markers? Maybe Noise Reduction in front of the tracker?

This would be really helpful, I am on the edge of exasperation.

First thing to check for any 3d track is that your 2d feature tracks are correct. No random wandering tracks and no big wobble. Noise wobble tends to cancel itself out from different feature tracks because its direction is random.

If feature tracks are good, then check that their layout in space is good enough to actually get a 3d solve. They should cover different depths and surfaces, for example it is impossible to get a 3d solve from just the planar back wall of gs studio, you need floor too. If you only have features on single plane it can be solved only as a nodal pan unless you have survey data (which you can't use in Fusion anyway afaik).

If feature layout is also good, then solve simply goes bonkers.

Can you show your footage and what the produced "wiggle motion" looks like?

Thanks for your quick answer.

The 2d feature tracks are good. I can understand what you mean with the feature tracks being in depth in the 3d space. However, I am only tracking a green screen with green markers on it, so there is no depth in it. In best case I get 30 - 50 trackers and solve around 0.4 - 0.7. Yet even with such a proper solve the image attached to it is wobbleing. But why? When I look at every single tracker there is no big wobbleing at all!?

Sorry, I can't show you the footage as this is for commercial use for a client. I hope you can understand my descriptions.

Hey others, feel free to join our discussion. Thank you!

If there is no depth, it can only be solved as a nodal pan (rotations only). Does your created camera's position wobble in XYZ coords, how far are the solved features from camera and at which depth do you place your card/image? If it wobbles, but solver itself thinks solve error is low, it is probably due to camera position wobble and your card is shifting because of this. Imagine for example that solved features are very far away, but your card is 1m from cam. If you move your cam and compensate with rotation, you can perfectly align image with feature locations because due to their distance, there is very little parallax shift. But relative to closeby card this positional shift has huge impact. This kind of problem can only be solved by either moving the card to same distance as solved features or constraining camera transform to rotations only. I don't know much about Fusions cam tracker unfortunately so can't instruct how to set it up there.

Bildschirmfoto 2021-07-15 um 16.45.52.jpg (858.18 KiB) Viewed 1962 times

Yes, it is wobbleing in XYZ cords (probably). In the attached screenshot you can see how far it is away from the camera. What makes me puzzle is why are the tracker points in different depths, although they should actually be almost in the same line. The card is placed behind the keyed footage. Yes the camera is wobbleing, not the card itself.

(The image and the footage is deliberately blurred out)

Since there is no parallax, small errors in calculations can shift features to different depths. Afterall, only thing solver is working from are paths in screen space and constraints, and in this case there aren’t any constraints in place.

Looking into Fusion ref manual, it suggests to ”Skip the Camera Tracker node and find another solution” in case of nodal pans For pans, you could use regular tracker or planar track the free gs area.

Hendrik's right: Fusion's CameraTracker cannot solve a shot without parallax. It doesn't have any tools for constraining the trackers to a plane, nor for setting the camera to rotate only. For markers on a flat greenscreen, I usually find the standard Tracker node is the way to go, as Planar Tracker relies on there being some texture in the surface to get a good result.

A dedicated 3d tracker like Syntheyes or PFTrack could probably give you a solve, but Fusion almost certainly won't.

I can vouch for Syntheyes. Been using it for a couple of months now (previously used for 3D/nodal tracking whatever applications that I used at that point like AE, Fusion, C4D or blender).
Syntheyes is just incredibly fast to use. It's not the most easy in terms of UI, but the main thing you'll do, tracking, is very comfortable and quick. And the export into Fusion is super simple.

Price/value for Syntheyes is also incredible. Especially because you actually own the software. You'll get 1 year of free updates, after that you'll need to buy a (cheaper) maintenance license, but you'll keep the version you currently own. I know a lot of people still using Syntheyes 2019 without issues.
If spending money is not an option then I would use blender. I personally haven't figured out how to get solid tracks, but the Blender crowd swears that it's one of the best trackers out there and I've seen many examples of great tracks done with it.

"What makes me puzzle is why are the tracker points in different depths, although they should actually be almost in the same line".

The first thing I would try is deleting all the points that cannot possibly be tracking anything real.