Jump to: Board index » General » Fusion

How to: place 3D model in video?

Learn about 3D compositing, animation, broadcast design and VFX workflows.
  • Author
  • Message
Offline
User avatar

TonyLewis

  • Posts: 48
  • Joined: Mon Sep 24, 2018 11:10 pm
  • Location: Raleigh, NC, USA
  • Real Name: Tony Lewis

How to: place 3D model in video?

PostFri Jan 18, 2019 7:45 pm

I have a complete 3D model in Fusion, and a separate video that I wish to combine.

The video both pans and zooms in, and of course I want the model to look like it is part of the original video, so the scale and orientation of the model must transform as the video/camera perspective changes.

In my (limited) research, it would appear that I would use a 3d image plane tool, but is there a tool/workflow combination that would 'automatically' match the perceived camera motion path (as it pans and zooms) with the model, in lieu of manually trying to adjust the 3D model scale and rotation via keyframes?

Thanks.
I7 @3.6mhz, 32G RAM, W7100 GPU, Intensity Pro 4K capture and display card
Resolve Studio, Fusion Studio, After Effects
Windows 10 Pro x64
Offline

Sander de Regt

  • Posts: 3500
  • Joined: Thu Nov 13, 2014 10:09 pm

Re: How to: place 3D model in video?

PostFri Jan 18, 2019 7:52 pm

Are you using Fusion Studio, Free Fusion or Fusion inside of Resolve (Studio?)
If you have the studio versions you could try to do a 3D camera track to match move the model in 3D space to your original footage so that your Renderer3D will output an image that matches your footage.
Since you have the actual model in 3D space, there is no need to go the 3D image plane route, because that would effectively give you a facsimile of your model.

Depending on what kind of camera/move the shot contains it's difficult to provide you with a catch all solution. Any chance you can share the shot with us so we can help you better?
Sander de Regt

ShadowMaker SdR
The Netherlands
Offline
User avatar

TonyLewis

  • Posts: 48
  • Joined: Mon Sep 24, 2018 11:10 pm
  • Location: Raleigh, NC, USA
  • Real Name: Tony Lewis

Re: How to: place 3D model in video?

PostFri Jan 18, 2019 9:27 pm

Unfortunately, I posted prior to finding the best solution would indeed be the 3D camera tracker.

Also unfortunately I am using the free version of Fusion 9, and you are correct that the 3D camera tracker is only available in the Studio version.

So I will have to either decide to buy the Studio version (which would not be a bad thing, particularly if it gets me the Studio version of Resolve with the same dongle), do the shot in a different program, or abandon the shot.

Thanks for your input.
I7 @3.6mhz, 32G RAM, W7100 GPU, Intensity Pro 4K capture and display card
Resolve Studio, Fusion Studio, After Effects
Windows 10 Pro x64
Offline

Sander de Regt

  • Posts: 3500
  • Joined: Thu Nov 13, 2014 10:09 pm

Re: How to: place 3D model in video?

PostFri Jan 18, 2019 9:35 pm

Or you can show us the shot and maybe - if it's only panning and zooming - there are other ways to achieve this. People used to do this before camera tracking in Fusion existed and even though it was more work, the results can still be worthwhile.
Sander de Regt

ShadowMaker SdR
The Netherlands
Offline
User avatar

TonyLewis

  • Posts: 48
  • Joined: Mon Sep 24, 2018 11:10 pm
  • Location: Raleigh, NC, USA
  • Real Name: Tony Lewis

Re: How to: place 3D model in video?

PostSun Jan 20, 2019 2:41 pm

Problem solved.

Fortunately, the background video does a nice linear zoom with no rotation or panning. And the zoom center was where I wanted to place the 3D model. All that I had to do was place the video on a 3D image plane behind my 3D model, then keyframe/animate the 3D model to scale it to match the video and run them both through a 3D merge node. But if the video did have significant rotation or panning, then I would have needed the Studio version's 3D camera tracker (which I probably will eventually buy).
I7 @3.6mhz, 32G RAM, W7100 GPU, Intensity Pro 4K capture and display card
Resolve Studio, Fusion Studio, After Effects
Windows 10 Pro x64
Offline

Sander de Regt

  • Posts: 3500
  • Joined: Thu Nov 13, 2014 10:09 pm

Re: How to: place 3D model in video?

PostSun Jan 20, 2019 2:51 pm

Glad you got it worked out. It still sounds like a bit of a workaround to be honest.
When panning/zooming the perspective of your shot doesn't change, so you can get by with normal 2D tracks in 90% of the cases. It still might be worthwhile to share your shot, so we can help you improve your workflow.
Sander de Regt

ShadowMaker SdR
The Netherlands
Offline
User avatar

TonyLewis

  • Posts: 48
  • Joined: Mon Sep 24, 2018 11:10 pm
  • Location: Raleigh, NC, USA
  • Real Name: Tony Lewis

Re: How to: place 3D model in video?

PostThu Feb 07, 2019 7:18 pm

I apparently don't have the rights to upload the video, I will check into it.

Once again, I have the original video on a 2D image plane behind the 3D model of the vehicle and the camera is shooting towards both. There is a directional light above the 3D model to match the lighting of the video. I was lucky enough to find the "sweet spot" to place the model such that I only had to animate the scale of the model using the spline editor. Basically the spline is an exponential curve for scaling. No need for offset of the 3D model.

I need to smooth out the scale spline as the scaling effect is not as smooth as it could be, but at least the basic technique is there, without using a tracker. A nice touch is the background video appearing through the windows of the 3D model.

If the original video did not do a linear zoom, or if there was rotation or shifting of the view, then I would have definitely preferred to use the trackers in the Studio version of Fusion. But for this video, it only took a few minutes here and there to accomplish the shot.
I7 @3.6mhz, 32G RAM, W7100 GPU, Intensity Pro 4K capture and display card
Resolve Studio, Fusion Studio, After Effects
Windows 10 Pro x64

Return to Fusion

Who is online

Users browsing this forum: No registered users and 45 guests