Jump to: Board index » General » Fusion

Relighting Live Footage

Learn about 3D compositing, animation, broadcast design and VFX workflows.
  • Author
  • Message
Offline
User avatar

AkhiRoney

  • Posts: 3
  • Joined: Fri Nov 05, 2021 7:27 am
  • Real Name: Isaac Roney

Relighting Live Footage

PostFri Nov 05, 2021 11:31 pm

I'm kind of newb to Fusion and node compositing in general, but I stumbled upon this video where the guy used nuke's depth generator on live footage which made the relight process seem fairly easy from there. Wondering if there is a way to recreate this sort of workflow in Fusion? Or just if anyone has tips and experience on relighting live action in fusion. Thanks!
Offline
User avatar

Pieter Van Houte

  • Posts: 631
  • Joined: Wed Nov 05, 2014 1:04 am

Re: Relighting Live Footage

PostSat Nov 06, 2021 6:50 pm

A start would be a link to that video you stumbled upon... most Nuke workflows can be translated to Fusion.
Support We Suck Less on Patreon -> https://www.patreon.com/wesuckless

https://www.steakunderwater.com/wesuckless
Offline
User avatar

AkhiRoney

  • Posts: 3
  • Joined: Fri Nov 05, 2021 7:27 am
  • Real Name: Isaac Roney

Re: Relighting Live Footage

PostSun Nov 07, 2021 8:36 am

It keeps saying I can't add URL's :lol: it's the first video when you search "relighting real footage" on youtube. Video is by Compositing Academy.
Offline

Frank Feijen

  • Posts: 167
  • Joined: Tue Dec 13, 2016 10:04 am
  • Location: Vilvoorde, Belgium

Re: Relighting Live Footage

PostMon Nov 08, 2021 8:08 am

this one?
Offline

Sander de Regt

  • Posts: 3497
  • Joined: Thu Nov 13, 2014 10:09 pm

Re: Relighting Live Footage

PostMon Nov 08, 2021 1:18 pm

AkhiRoney wrote:It keeps saying I can't add URL's :lol: it's the first video when you search "relighting real footage" on youtube. Video is by Compositing Academy.

You have to have a minimum amount of posts on the forum before you can post URLs.
You can post chopped up urls though, just put a couple of spaces in there and it will work, until you can post real ones.
Sander de Regt

ShadowMaker SdR
The Netherlands
Offline
User avatar

Bryan Ray

  • Posts: 2478
  • Joined: Mon Nov 28, 2016 5:32 am
  • Location: Los Angeles, CA, USA

Re: Relighting Live Footage

PostMon Nov 08, 2021 3:10 pm

We unfortunately don't have any equivalent of the Depth Generator node in Fusion. If you can get your depth in another fashion, then everything else there can be done with ease. That position mask tool is equivalent to Fusion's VolumeMask. We also have a Shader node that can do some of the relighting in 2d space without even needing the camera, or you can use Displace3D and the world position vectors to smash a high-res plane into the shape of the 3d scene and light that however you like. And, of course, the fog bit was just a color corrector. Fusion does also have some actual fog tools that interact with either depth or world position.

As regards the Depth Generator, I'd love to see something like that in Fusion. It would be very handy in a lot of situations. But be aware that it does require trackable footage. It won't work for every shot. Great for drone work, though. It's possible Syntheyes could produce geo that would make it possible to create the depth, normals and position buffers in Fusion. PFTrack can make a passable depth channel, but I've not explored the idea anywhere else.
Bryan Ray
http://www.bryanray.name
http://www.sidefx.com
Offline

Sander de Regt

  • Posts: 3497
  • Joined: Thu Nov 13, 2014 10:09 pm

Re: Relighting Live Footage

PostMon Nov 08, 2021 4:15 pm

I wonder if meshroom like software could deal with the pointclouds of Fusion to create stand-in geo.
They would have nowhere as many points as photogrammetry usually has to work with, but maybe.
Or you could use the images of the shot in photogrammetry software and use the exported geo in a cameramatched 3D world in Fusion.
Sander de Regt

ShadowMaker SdR
The Netherlands
Offline
User avatar

AkhiRoney

  • Posts: 3
  • Joined: Fri Nov 05, 2021 7:27 am
  • Real Name: Isaac Roney

Re: Relighting Live Footage

PostTue Nov 09, 2021 9:35 pm

Bryan Ray wrote:We also have a Shader node that can do some of the relighting in 2d space without even needing the camera,

Sounds interesting. I'll have to play around with that. I've only seen the shader node used for 3d objects and renders that already have normal and position passes.
Bryan Ray wrote:or you can use Displace3D and the world position vectors to smash a high-res plane into the shape of the 3d scene and light that however you like. And, of course, the fog bit was just a color corrector. Fusion does also have some actual fog tools that interact with either depth or world position.

I guess my original question should have been, how would you create a world position map from 2d video?
Sander de Regt wrote:I wonder if meshroom like software could deal with the pointclouds of Fusion to create stand-in geo.

That reminds me of this video https ://youtu. be/rYs5LH6gzvU at the 4:30 mark he starts creating geometry from the point cloud.
Offline

Travis Schmiesing

  • Posts: 133
  • Joined: Wed Oct 14, 2015 4:01 am

Re: Relighting Live Footage

PostThu Dec 02, 2021 8:16 pm

Bryan Ray wrote:We unfortunately don't have any equivalent of the Depth Generator node in Fusion. If you can get your depth in another fashion, then everything else there can be done with ease. That position mask tool is equivalent to Fusion's VolumeMask. We also have a Shader node that can do some of the relighting in 2d space without even needing the camera, or you can use Displace3D and the world position vectors to smash a high-res plane into the shape of the 3d scene and light that however you like. And, of course, the fog bit was just a color corrector. Fusion does also have some actual fog tools that interact with either depth or world position.

As regards the Depth Generator, I'd love to see something like that in Fusion. It would be very handy in a lot of situations. But be aware that it does require trackable footage. It won't work for every shot. Great for drone work, though. It's possible Syntheyes could produce geo that would make it possible to create the depth, normals and position buffers in Fusion. PFTrack can make a passable depth channel, but I've not explored the idea anywhere else.


RunwayML's auto Depth Map generator might work. I havn't tried it, but scroll down a bit and you can see an animated gif of what it claims to be capable of.

https://runwayml.com/video-editing/
Offline
User avatar

Bryan Ray

  • Posts: 2478
  • Joined: Mon Nov 28, 2016 5:32 am
  • Location: Los Angeles, CA, USA

Re: Relighting Live Footage

PostThu Dec 02, 2021 9:19 pm

:o

I wish I'd seen that idea about using the CleanPlate to fix a sparse position pass ages ago! Would have been super useful on a project a couple of years ago.

So yeah, that video's pretty much what you're looking for to get the position pass. Maybe there's a little something extra that could be done with Create Bump Map to get smaller details than the point cloud for normals. And obviously CameraTracker isn't going to be able to do anything about something that's moving in the shot. For that, you'd need a more robust tracking and matchmove solution.
Bryan Ray
http://www.bryanray.name
http://www.sidefx.com

Return to Fusion

Who is online

Users browsing this forum: No registered users and 26 guests