3D Tracking Fusion / Houdini

Hello everyone,

First message on this forum, so I hope I'm in the right place.
Thanks in advance for those who will take the time to read me.

I'm working on Houdini, and I want to add explosion, destruction, whatever..To real footages.
I found (only a few numbers of) Fusion <--> Houdini tutorials.
But at the end, I figured how to
- Track the footage
- Export my camera (.fbx)
- Import in Houdini
- Add my FX (a lot of translate/rotate/scale adjustments to see my 3D work on the camera frustrum but... I'm happy! for now...)

But... :
1) No matter which 3D object/FX, when i hit play in Houdini, the object seems to slightly "slide" on the footage at few moments

2) The object/FX jittered a bit. So.. I guessed that "smooth" the camera curve tracked in fusion could help. And.. I think it helped. But not sure if it was the right way/solution.

3) Regarding the scale (when you have to select two points and set it) what is the unit? Maybe it could help too..

Bonus question (maybe "noob" one...) :

4) Not sure to understand how to place a "Ground plane" in fusion (to place my 3D Object/FX later) when it comes to things like mountain for example ? If I want to place multiple explosion in mountains with different angle, I can't have only one ground place for all. What's the right way to proceed? Multiple ground place / Multiple export / Multiple FX layered ?

In general, if you have the patience and the time, feel free to give me any advice/workflow for 3D tracking. I'm looking forward to hear a proper way to achieve that.
The goal for me is to improve my demo reel with Real footage + Compositing VS total 3D.
I'm in Houdini but I think my issues will be the same for any other packages (Blender, Maya, etc...)

Ps : It's probably not the place, but if you need any help or information about houdini, do not hesitate

Thank you !

I'm using Maya, but it's the same workflow, so here's some advices.

1) No matter which 3D object/FX, when i hit play in Houdini, the object seems to slightly "slide" on the footage at few moments

What's your average solver error in the tracker? Also, sometimes the sliding problem appears to be more noticeable if the comped 3d object it's not at the right height, and if you have not a good reference point, setting that right it's a very tedious trial and error process.

The Fusion camera tracker is missing the ability to export a 'ready-to-use' point cloud, and it's crucial to have a good points reference for comping 3d stuff on footage which is not just a flat plane. You can export coordinates, but it's a convoluted process to convert those in nulls or other objects in the main DCC.

Luckily for us, the guys at WeSuckLess have developed a godsend plugin full of script for this kind of stuff: go to the forum and download and install the Reactor plugin for Fusion and DR. Once installed you'll find a script to export a point cloud as nulls in different formats.

2) The object/FX jittered a bit. So.. I guessed that "smooth" the camera curve tracked in fusion could help. And.. I think it helped. But not sure if it was the right way/solution.

Not sure about this one, it could be different things, like a bad tracking, and/or it could depend on some problems related to the footage, like camera lens distortion* (especially at the corners edge of the screen), motion blur judder, etc.
Hard to say without seeing the video.

*the correct pipeline for vfx should include the lens distortion data to export the undistort plate to work in 3d, and then comping using the same data to distord the 3d files on the original distorted plate:

https://vfxcamdb.com/wp-content/uploads/2013/11/ldw.png

3) Regarding the scale (when you have to select two points and set it) what is the unit? Maybe it could help too..

It's not in the manual so I'm not 100% sure, but according the the units in Maya, it's in cm. For example, if I export a 100x100 scale 3d cube from the fusion comp in fbx, it's a 100% match 100x100cm cube in Maya, so if you know exactly the distance between to points from the tracker, just put that in cm, or use a cube or other 3d objects (like humans) to scale the scene accordingly.

In the end, to use real world scale units like me, the scene must be scaled by thousands, because it's always minuscule.

4) Not sure to understand how to place a "Ground plane" in fusion (to place my 3D Object/FX later) when it comes to things like mountain for example ? If I want to place multiple explosion in mountains with different angle, I can't have only one ground place for all. What's the right way to proceed? Multiple ground place / Multiple export / Multiple FX layered ?

The ground plane is what it is: a reference ground plane. For this kind of stuff the point cloud it's the thing you want to use, or you can use any 3D objects like cubes as reference in the scene to use in Houdini later to place your stuff.
Hope it helps

edit: many typos

Think of ground plane setup as specifying the scene coordinate system, because this is what it is. Solved scene can have its coordinate system set up in only one way: it has an origin somewhere and its axes are aligned in specific way. It is usually done through ground plane just because in most scenes ground is more or less aligned with the logical XY (or XZ depending on software) plane, and it means that if some object is placed at zero height it lands either on the ground or close to it. Nothing prevents you from aligning your solved scene upside down or at random angle, but it simply makes it more confusing to navigate.

So, setting ground plane only determines how you want your solved scene aligned, nothing more. Where to place your actual objects is not directly related to it at all, you must find the locations for your elements from point cloud or using some other method, as Mr.Cane wrote.

Fusion's 3d units are abstract. They can represent whatever you need them to: inches, cm, meters, lightyears, chains, cubits.

There may be an assumed unit that is used to approximate depth of field, but I don't know if anybody's ever measured it.

Hello everyone!

A huge thank you to all of you! Really appreciate!

What's your average solver error in the tracker? Also, sometimes the sliding problem appears to be more noticeable if the comped 3d object it's not at the right height, and if you have not a good reference point, setting that right it's a very tedious trial and error process.

At this time, my average solver error was ~0.2.
Bu after a lot of trial and error (and a good look at the documentation), I finally succeed to add more tracking points, and the object do not slide anymore

Luckily for us, the guys at WeSuckLess have developed a godsend plugin full of script for this kind of stuff: go to the forum and download and install the Reactor plugin for Fusion and DR. Once installed you'll find a script to export a point cloud as nulls in different formats.

What a cool plugin! I didn't know! Thank you!
I installed Reactor, but I was not able to find the right plugin to export pointcloud to null unfortunately.
However, the method which consist of adding a 3D object as you said works perfectly, and gave me a good reference point to build my 3D object/FX.

https://vfxcamdb.com/wp-content/uploads/2013/11/ldw.png

I didn't know that! it makes a lot of sense, and I will try to reproduce this pipeline.

In the end, to use real world scale units like me, the scene must be scaled by thousands, because it's always minuscule.

Yeah I figured that too.
1 in Fusion = 1m in Houdini works fine. But yeah, seems always minuscule!

Thank you Mr. Cane, Hendrik and Bryan! A lot of good informations to help me in the process!
Really appreciate your time

Fusion's 3d units are abstract. They can represent whatever you need them to: inches, cm, meters, lightyears, chains, cubits.

I guess the confusion for many people comes from the camera tracker because it has a specific scale slider with a 'set from selection' button, and the prompt asks to Enter the real world distance between two points.

So, of course one could expect to use real world units, as I did myself the first time I used the tracker.
I think that the question Asiphow was asking can be reformulated more like 'what's the scale factor to match the X-software default scene when I export data from Fusion?'

To elaborate more on question n.3:

the choice of which units to assume inside of fusion depends of the destination software in which the data will be imported, or the chosen unit will be pointless. When you export data from the tracker outside Fusion, and if you are referencing real world units, you can't just blindly choose which one to use, otherwise there will be no match.

I use Maya and if I export a 1x1x1 cube from Fusion, it will be a 1x1x1cm cube in Maya: so if I'm exporting data for Maya, my scale factor is 1:1, because the default units are centimeters. Therefore I can choose cm as my arbitrary unit for Fusion.

For Houdini, 3ds Max, Blender or whatever, it's the same, just find the scale factor using a simple cube or a plane and then do the math inside of Fusion or in the destination sw.