Page 1 of 1

Compositing two clips together using Z pass

PostPosted: Tue Jun 18, 2019 3:04 pm
by TACarp
I have two Multi-Layer OpenEXR image sequences from blender. Both of them have a beauty and depth(z) pass. I have DaVinci resolve free with built in fusion. I would like to merge the two image sequences together using the depth pass.
Below is the same frame with all four layers to give you an idea.
Image
The character is supposed to be inside the building. The front wall is supposed to be in front of him and the back wall is supposed to be behind him. How do I do this using the depth pass?

Re: Compositing two clips together using Z pass

PostPosted: Tue Jun 18, 2019 4:23 pm
by Sander de Regt
In Fusion's merge tool is an option to use depth merge somewhere in the settings. It's definitely worth checking this out, although you'll also need at least a coverage pass to get passable (sorry) results.

Re: Compositing two clips together using Z pass

PostPosted: Tue Jun 18, 2019 6:39 pm
by Chad Capeland
Ideally Coverage and BgRGB.

Re: Compositing two clips together using Z pass

PostPosted: Wed Jun 19, 2019 12:24 am
by TACarp
I found the setting for depth merge with the merge node. I seem to be lost on how to wire the node setup. It seems for depth merge with a single node at least four inputs would be required (image 1 beauty (or similar), image 1 depth, image 2 beauty (or similar) and image 2 depth). The merge node only has three inputs (bottom image, top image, and the mask.) Here is my not working node setup so this forum can better guide me.
Image
Also the two replies above make it sound like fusion isn't very good at this process. Is this true? I have also notices some banding on the depth layers. Will this be a problem?

Re: Compositing two clips together using Z pass

PostPosted: Wed Jun 19, 2019 1:04 am
by Chad Capeland
You'll need to add the depth to the Z channel of the Merge's input images with something like Bol. This is a case where Resolve isn't the ideal platform because it doesn't handle multichannel images well. But depending on your needs, this might not be a dealbreaker.

You might see banding in your depth images because your display may only be showing 8 or fewer bits of range, but as long as your image is 16 bit or float, you should still be fine.

A pure depth-based composite isn't physically sound (but neither is an alpha composite, either). Obvious issues are support for partial transparency. But it does offer some advantages and depending on your production need it can get the job done.

Re: Compositing two clips together using Z pass

PostPosted: Thu Jun 20, 2019 1:58 am
by TACarp
It would help me to see a working node configuration. Do I combine with the bol node both depth passes, both beauty pass, or each beauty pass with its accompanying depth pass. The bol and merge settings in the inspector might also help me. Sorry if I am being needy.

You mention issues with depth compositing, which I understand such as semi-transparency and volumetric shading. Are there other problems? Are these just davinci resolve problems? Alpha won’t work as reliably for me as depth. It speeds up my turn-around time to render my backgrounds a few frames in blender cycles and render my character animation in blender eevee. Compositing is important to my work. Alpha doesn’t work much of the time as sometimes set elements are in front of characters. Are there other methods are there besides depth and alpha to get this result?

Re: Compositing two clips together using Z pass

PostPosted: Thu Jun 20, 2019 3:12 pm
by Bryan Ray
Here's a quick example. The RGB and Z nodes are Loaders that hold the respective images.

Code: Select all
{
   Tools = ordered() {
      Z_1 = Loader {
         Clips = {
         },
         NameSet = true,
         Inputs = {
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
         },
         ViewInfo = OperatorInfo { Pos = { 220, -49.5 } },
      },
      RGB_1 = Loader {
         Clips = {
         },
         NameSet = true,
         Inputs = {
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
         },
         ViewInfo = OperatorInfo { Pos = { 220, 16.5 } },
      },
      ChannelBooleans1_1 = ChannelBoolean {
         Inputs = {
            ToRed = Input { Value = 4, },
            ToGreen = Input { Value = 4, },
            ToBlue = Input { Value = 4, },
            ToAlpha = Input { Value = 4, },
            EnableExtraChannels = Input { Value = 1, },
            ToZBuffer = Input { Value = 0, },
            Background = Input {
               SourceOp = "RGB_1",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "Z_1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 330, 16.5 } },
      },
      Z = Loader {
         Clips = {
         },
         NameSet = true,
         Inputs = {
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
         },
         ViewInfo = OperatorInfo { Pos = { 220, 115.5 } },
      },
      LumaKeyer1 = LumaKeyer {
         CtrlWZoom = false,
         Inputs = {
            Channel = Input { Value = 7, },
            High = Input { Value = 349.999994784594, },
            Low = Input { Value = 129.999998062849, },
            Invert = Input { Value = 1, },
            Input = Input {
               SourceOp = "ChannelBooleans1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 605, 82.5 } },
      },
      ChannelBooleans1 = ChannelBoolean {
         Inputs = {
            ToRed = Input { Value = 4, },
            ToGreen = Input { Value = 4, },
            ToBlue = Input { Value = 4, },
            ToAlpha = Input { Value = 4, },
            EnableExtraChannels = Input { Value = 1, },
            ToZBuffer = Input { Value = 0, },
            Background = Input {
               SourceOp = "RGB",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "Z",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 330, 181.5 } },
      },
      Merge1 = Merge {
         Inputs = {
            Background = Input {
               SourceOp = "ChannelBooleans1",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "ChannelBooleans1_1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 495, 181.5 } },
      },
      RGB = Loader {
         Clips = {
         },
         NameSet = true,
         Inputs = {
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
         },
         ViewInfo = OperatorInfo { Pos = { 220, 181.5 } },
      },
      Note1 = Note {
         Inputs = {
            Comments = Input { Value = "This assumes that your depth map represents Z in its RGB channels. The assignment of Red to Z happens in the Aux Channels tab.\n\nRGBA are set to \"Do Nothing\", which will simply send those channels from the Background input to the Output unchanged. You could, alternately, set them to \"Red BG\" etc to make more explicit what you want to happen, but the end result would be the same.", }
         },
         ViewInfo = StickyNoteInfo {
            Pos = { 275, 214.5 },
            Flags = {
               Expanded = true
            },
            Size = { 196, 337.3 }
         },
      },
      ChannelBooleans2 = ChannelBoolean {
         Inputs = {
            Operation = Input { Value = 3, },
            Background = Input {
               SourceOp = "Blur2",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "LumaKeyer1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 715, 82.5 } },
      },
      Blur2 = Blur {
         Inputs = {
            XBlurSize = Input { Value = 0.3, },
            Input = Input {
               SourceOp = "ErodeDilate1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 715, 16.5 } },
      },
      Filter1 = Filter {
         Inputs = {
            FilterType = Input { Value = 3, },
            Input = Input {
               SourceOp = "ChannelBooleans1_1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 715, -49.5 } },
      },
      ErodeDilate1 = ErodeDilate {
         Inputs = {
            XAmount = Input { Value = -0.000137, },
            Input = Input {
               SourceOp = "Filter1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 715, -16.5 } },
      },
      Blur1 = Blur {
         Inputs = {
            Input = Input {
               SourceOp = "Merge1",
               Source = "Output",
            },
            EffectMask = Input {
               SourceOp = "ChannelBooleans2",
               Source = "Output",
            }
         },
         ViewInfo = OperatorInfo { Pos = { 715, 181.5 } },
      },
      Note2 = Note {
         Inputs = {
            Comments = Input { Value = "Sobel is the actual name of Photoshop's Edge Detect. A little Erode to make the edge thinner and Blur to soften it. These controls will need to be adjusted depending on the relative size of the object to be integrated.\n\nAnd or Multiply in the Boolean. The logic is to only show the edge detect where the inverted Depth Key is solid.\n\nYou'd use the Near/Far Plane controls in the Luma Keyer to select the occluding foreground geometry.\n\nUsing this edge matte as a mask on the blur, you can soften the integration between character and background. It's not as good as an anti-aliased alpha, but it's probably better than depth alone.", }
         },
         ViewInfo = StickyNoteInfo {
            Pos = { 825, -49.5 },
            Flags = {
               Expanded = true
            },
            Size = { 205, 495.3 }
         },
      }
   }
}


The major issue with a depth merge is that a depth map is not anti-aliased. Each pixel must represent an actual distance from the camera, so having intermediate colors on the edges is non-sensical—you don't want the edges of your character to get pushed halfway to the max depth. As a result, it's difficult to get a nicely-blended composite with a depth merge.

A more typical workflow in a situation like this would be to render multiple planes of your action. In this case, your layers would probably be the building minus its front wall, the character, and the building's front wall. Then you'd use the standard Merge to combine them.

You could try to do a little manual edge blending by using a Sobel filter on your character's alpha (Filter tool), and try to isolate the front wall with a LumaKey off the depth in order to use that wall to subtract from the character's edge. That would become a mask to do some edge treatment. It's simple enough to set it up, so I included an example of that in the nodes I shared above.