Page 1 of 1

How to: place 3D model in video?

PostPosted: Fri Jan 18, 2019 7:45 pm
by TonyLewis
I have a complete 3D model in Fusion, and a separate video that I wish to combine.

The video both pans and zooms in, and of course I want the model to look like it is part of the original video, so the scale and orientation of the model must transform as the video/camera perspective changes.

In my (limited) research, it would appear that I would use a 3d image plane tool, but is there a tool/workflow combination that would 'automatically' match the perceived camera motion path (as it pans and zooms) with the model, in lieu of manually trying to adjust the 3D model scale and rotation via keyframes?

Thanks.

Re: How to: place 3D model in video?

PostPosted: Fri Jan 18, 2019 7:52 pm
by Sander de Regt
Are you using Fusion Studio, Free Fusion or Fusion inside of Resolve (Studio?)
If you have the studio versions you could try to do a 3D camera track to match move the model in 3D space to your original footage so that your Renderer3D will output an image that matches your footage.
Since you have the actual model in 3D space, there is no need to go the 3D image plane route, because that would effectively give you a facsimile of your model.

Depending on what kind of camera/move the shot contains it's difficult to provide you with a catch all solution. Any chance you can share the shot with us so we can help you better?

Re: How to: place 3D model in video?

PostPosted: Fri Jan 18, 2019 9:27 pm
by TonyLewis
Unfortunately, I posted prior to finding the best solution would indeed be the 3D camera tracker.

Also unfortunately I am using the free version of Fusion 9, and you are correct that the 3D camera tracker is only available in the Studio version.

So I will have to either decide to buy the Studio version (which would not be a bad thing, particularly if it gets me the Studio version of Resolve with the same dongle), do the shot in a different program, or abandon the shot.

Thanks for your input.

Re: How to: place 3D model in video?

PostPosted: Fri Jan 18, 2019 9:35 pm
by Sander de Regt
Or you can show us the shot and maybe - if it's only panning and zooming - there are other ways to achieve this. People used to do this before camera tracking in Fusion existed and even though it was more work, the results can still be worthwhile.

Re: How to: place 3D model in video?

PostPosted: Sun Jan 20, 2019 2:41 pm
by TonyLewis
Problem solved.

Fortunately, the background video does a nice linear zoom with no rotation or panning. And the zoom center was where I wanted to place the 3D model. All that I had to do was place the video on a 3D image plane behind my 3D model, then keyframe/animate the 3D model to scale it to match the video and run them both through a 3D merge node. But if the video did have significant rotation or panning, then I would have needed the Studio version's 3D camera tracker (which I probably will eventually buy).

Re: How to: place 3D model in video?

PostPosted: Sun Jan 20, 2019 2:51 pm
by Sander de Regt
Glad you got it worked out. It still sounds like a bit of a workaround to be honest.
When panning/zooming the perspective of your shot doesn't change, so you can get by with normal 2D tracks in 90% of the cases. It still might be worthwhile to share your shot, so we can help you improve your workflow.

Re: How to: place 3D model in video?

PostPosted: Thu Feb 07, 2019 7:18 pm
by TonyLewis
I apparently don't have the rights to upload the video, I will check into it.

Once again, I have the original video on a 2D image plane behind the 3D model of the vehicle and the camera is shooting towards both. There is a directional light above the 3D model to match the lighting of the video. I was lucky enough to find the "sweet spot" to place the model such that I only had to animate the scale of the model using the spline editor. Basically the spline is an exponential curve for scaling. No need for offset of the 3D model.

I need to smooth out the scale spline as the scaling effect is not as smooth as it could be, but at least the basic technique is there, without using a tracker. A nice touch is the background video appearing through the windows of the 3D model.

If the original video did not do a linear zoom, or if there was rotation or shifting of the view, then I would have definitely preferred to use the trackers in the Studio version of Fusion. But for this video, it only took a few minutes here and there to accomplish the shot.