Moving Camera, Object: How to add object to 3D environment?

Hi,

I am new to VFX and I know I have entered a very complex world. I have spent past week endlessly trying to do a test case to confirm I can film a moving subject with a moving camera, rotobrush out the background, then insert the subject in a 3D environment such as Unreal Engine. Ultimately I then want to film myself playing guitar and add to 3D environments.

As I understand this requires solving the camera movement, exporting that solve appropriately (scale, axes, origin, ground plane). ALso involves rotobrushing out the background then using resulting image on a projection plate attached to the camera so that camera is always looking at the subject as during the actual shot.

I am getting close to a decent result but have a ways to go, as my result has the subject sliding too much in the 3D environment.

Ok, so here is the shot I am using for an experiment:

Here is what tracked camera looks like in Blender:

Now if I import this in Unreal Engine as FBX I get following result after attaching my image sequence to a projected image plate:

It is as if the camera FOV/perspective isn't quite right in the solve, or is this a different problem being observed?

Notice how at certain moments the cat is sliding across the ground plane. And compared to original video, here the last few frames the cat is disappearing below ground plane (I know this isn't 3D cat - what is happening is image plate is angled such that cat is getting cropped off at end).

Clearly I am not doing this quite the way as needed. So I am hoping someone can help explain to me the steps that I need to follow to get a more stable result, such that the cat in the 3D environment stays stationary on the ground plane. I can't find any tutorials that hit on all these aspects. And I know here people are more geared towards fusion but if someone can explain some sort of workflow that leads to a better result, even explained using different software, then I can adapt to my tools.

Who knows - if it is interesting exercise to someone maybe you can use my example video and achieve some similar but very accurate result and then explain how you did it!

I am demo'ing Syntheyes, and if that is better than Blender for my use case I will likely start using it, and looking through PDF I get impression that the program requires pro level experience... I can't figure out what aspects of the program to use. Camera only, or camera plus object tracking, constraints, object meshes, etc, etc, etc. Overwhelming.

Also, anyone know a site to find a tutor where I could just have someone walk me through all these processes?

thanks

You're Blender track appears to be quite stable. One thing I'd definitely check out first is whether or not your framerate and/or starting frame are identical between Blender and UE. If it's one frame off (which can happen quite often, Fusion defaults its first frame at 0, Blender at 1, UE I don't know.

As for putting objects in 3D space. This is a link to a video I just did on exactly this subject.

But if you're new to VFX I'd really really consider starting out with some easier shots first.
It looks like you're trying to emulate the Ian Hubert approach with normalized camera moves etc to put moving objects in 3D space. That's a pretty tricky thing to pull off even for experienced users.
I'd definitely start by putting locked off people (you playing your guitar) in 3D space first and once you've got that worked out, work your way up to more difficult things.

As Sander wrote, it is probably something simple from this list:
- mismatching sequence start frame
- mismatching frame rate
- mismatching resolution
- wrong interpretation of solved camera fov in UE
- placement of objects in 3D not properly aligned with solved scene

Hi, Blender shows start 0, end 803. Also frame rate in UE matches 25 which is rate of the original video. And UE is showing frames 0-803 for animation.

Will watch your video.

thanks

Hi, regarding interpretation of FOV, I am wondering how to check that? It is showing 43.78 horizontal FOV, and focal length 21.5 (according to my camera it was 21 for this shot). These seem in agreement given my camera.

I shot with GH5, a MFT camera, not sure what that implies for specifying correct lens info.

thanks