Jump to: Board index » General » Fusion

Camera Tracking to FBX Exporting Weirdness

Learn about 3D compositing, animation, broadcast design and VFX workflows.
  • Author
  • Message
Offline

DBCreative

  • Posts: 29
  • Joined: Mon Jan 17, 2022 10:16 pm
  • Real Name: Danny Burns

Camera Tracking to FBX Exporting Weirdness

PostTue Aug 02, 2022 2:57 am

I hope to better understand what's happening here, so if anyone knows, please advise.

I have some test footage shot on an iPhone 12 Pro Max with the standard lens. Tracked this footage in Fusion, and entered the exact sensor size and focal length into the solve panel. Fusion ignores the focal length and determines its own focal length, with is wrong. Everything looks ok and placed shapes seem to lock in and no sliding is visible.

Then I use an FBX Export node to get that track data into Blender. In Blender other than the placed aligned shapes being tiny, and the camera being upside down and backwards, it looks OK. BUT entering the focal length from the camera in Fusion into the camera in Blender causes a mismatch in the aligned shapes and they now slide around as though they are not tracked properly. If I then enter the correct focal length into the Blender camera, the image seems to align itself properly and 8 out of 10 times, the tracking now works correctly. In the other 2 it's still misaligned and unfixable.

For comparison, I can track the same footage in After Effects, bring into Blender via the AE2Blend add-on and everything lines up properly. Focal lengths of the cameras match, no sliding of elements, etc. I had been under the impression that Fusion's camera tracker was considerably better than that of AE, but this doesn't seem to be the case. Can anyone help me understand what I may be doing wrong?
Offline

Hendrik Proosa

  • Posts: 3015
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Camera Tracking to FBX Exporting Weirdness

PostTue Aug 02, 2022 7:32 am

Focal length always plays together with filmback size, together they form specific angle of view which is actually solved. Most probably you have mismatched filmback sizes between Fusion and Fbx export result in Blender. The ”correct” value you have is probably 35mm filmback equivalent, but in Fu cam tracker filmback is something else. Check what it is, changing the filmback in Blender should at least in theory produce same result as changing focal length. Why both of these don’t come through from fbx export, don’t know.

Goodness of camera trackers mostly comes from additional features and helper functions, not the barebones solver. Solver part is probably more or less the same these days. But how easy or hard it is to produce manual tracks or how accurate are the auto,atic feature tracks will greatly affect the solver result. As do all the soft and hard constraints and default assumptions.
I do stuff.
Offline

Sander de Regt

  • Posts: 3500
  • Joined: Thu Nov 13, 2014 10:09 pm

Re: Camera Tracking to FBX Exporting Weirdness

PostTue Aug 02, 2022 8:54 am

My viewpoint is: if it matches and it doesn't slide then don't worry about it. The only thing that matters is the angle. That’s how they matchmoved miniatures in the past. If it looks okay it is okay.
Sander de Regt

ShadowMaker SdR
The Netherlands
Offline

DBCreative

  • Posts: 29
  • Joined: Mon Jan 17, 2022 10:16 pm
  • Real Name: Danny Burns

Re: Camera Tracking to FBX Exporting Weirdness

PostTue Aug 02, 2022 3:04 pm

Sander de Regt wrote:My viewpoint is: if it matches and it doesn't slide then don't worry about it. The only thing that matters is the angle. That’s how they matchmoved miniatures in the past. If it looks okay it is okay.

It looks good in Fusion. In Blender, it falls apart. This is only happening with Fusion. If I track the same clip in Blender, with the same camera data, the track is perfect. If I track in AE and bring that track data into Blender, the track is perfect. Something is a miss when tracking and exporting in Fusion. Another user I consulted with, found the same problem, and believes it may revolve around the FBX Export node. More research is needed.
Offline

Sander de Regt

  • Posts: 3500
  • Joined: Thu Nov 13, 2014 10:09 pm

Re: Camera Tracking to FBX Exporting Weirdness

PostTue Aug 02, 2022 3:08 pm

How big is the difference? One thing that trips me up from time to time is that Blender and Fusion have a one frame difference in dealing with their timing. So Fusion will start at frame 0 and Blender at frame 1 which results in a match that's almost perfect, but not in a way that's really hard to tell.
I'm not sure if that might be the case here?
Sander de Regt

ShadowMaker SdR
The Netherlands
Offline

DBCreative

  • Posts: 29
  • Joined: Mon Jan 17, 2022 10:16 pm
  • Real Name: Danny Burns

Re: Camera Tracking to FBX Exporting Weirdness

PostTue Aug 02, 2022 5:11 pm

Sander de Regt wrote:How big is the difference? One thing that trips me up from time to time is that Blender and Fusion have a one frame difference in dealing with their timing. So Fusion will start at frame 0 and Blender at frame 1 which results in a match that's almost perfect, but not in a way that's really hard to tell.
I'm not sure if that might be the case here?


The difference is huge. When you see the tracked footage in Blender with the ground plane from Fusion, it's 100% unusable. The ground plan is all over the place, the focal length is off considerably, etc...

The keyframe issue is a well known one and even when adjusted for, it makes no difference in the right direction. It will however make it worse.
Offline
User avatar

Bryan Ray

  • Posts: 2478
  • Joined: Mon Nov 28, 2016 5:32 am
  • Location: Los Angeles, CA, USA

Re: Camera Tracking to FBX Exporting Weirdness

PostTue Aug 02, 2022 5:57 pm

There are three things to be aware of that often wreck the workflow:

First, Fusion measures film back in inches rather than mm for some reason. Second, it bases the angle of view on the vertical measure, but almost all other software measures horizontal. Third, every version of the FBX file format has problems, but none of them have the same problems. I've had the best luck with the 2014 version of FBX.

All that said, I never track with Fusion if I intend to use the results elsewhere. CameraTracker is just too limited. I only use it for very minor and easy stuff, and even then half the time I throw that work away and grab a more full-featured tracker instead.
Bryan Ray
http://www.bryanray.name
http://www.sidefx.com
Offline

DBCreative

  • Posts: 29
  • Joined: Mon Jan 17, 2022 10:16 pm
  • Real Name: Danny Burns

Re: Camera Tracking to FBX Exporting Weirdness

PostTue Aug 02, 2022 9:03 pm

Bryan Ray wrote:There are three things to be aware of that often wreck the workflow:

First, Fusion measures film back in inches rather than mm for some reason. Second, it bases the angle of view on the vertical measure, but almost all other software measures horizontal. Third, every version of the FBX file format has problems, but none of them have the same problems. I've had the best luck with the 2014 version of FBX.

All that said, I never track with Fusion if I intend to use the results elsewhere. CameraTracker is just too limited. I only use it for very minor and easy stuff, and even then half the time I throw that work away and grab a more full-featured tracker instead.


Thanks for this info, Byran, this is really helpful. In truth, I was looking to camera track in Fusion because it's one of 3 apps I currently have that does camera tracking. I find I get consistently better results from AE and Blender, but Blender is the slowest of the 3, and I'm looking to get away from AE this year. I use a Mac for a lot of my work, so whatever software I get, I'd like it to work on my Mac and my PC. If Fusion isn't the right choice, so be it, I was hoping though...

Return to Fusion

Who is online

Users browsing this forum: KrunoSmithy and 34 guests