Merge node equivalent to planar tracker FG only output?

Is there a way to get a merge node to only output the foreground input, but after changes made by the node? In particular, I'm hoping there's a way that I can use the merge node to change the foreground input using the blend modes with the background, but then have that image without the background to use elsewhere.

There is no option in a Merge to output the FG only.

I'd say there are three general 'features' of a Merge node:
- Transforming a foreground
- Adjusting opacity of that foreground
- Compositing that foreground over a background (blend modes)

With regard to blend modes, I don't think the question makes sense. Blend modes are mathematical calculations for creating a new image from the pixels of two input images. Therefore by definition the output is a composite of both images. Removing the background from a Merge would just give you the original foreground, and vice versa.

So the only place I can see this making sense is with regard to the Merge's transform controls, and Blend slider.

To get a "FG only" output of those, you could do this:


First you do your transformation in a Transform node, and the result of that can then be piped out separately if desired. Therefore no transformation would be done in MergeComposite at the end.

Then, for the Blend (opacity) slider, we do that in a separate Merge which is over an Alpha=0 background. MergeOpacity only has the Blend slider operated, and because it's merging over a transparent background, the result is that we just change the FG opacity without compositing it over anything that would change the FG image in any way.

As the Transform and Opacity operations are done independently, their output can be routed elsewhere. The output of MergeOpacity would be something like a "FG only" option, including the opacity and transformation changes, without the background.

As you can see, if we didn't want or need a separate Transform output, we could lose the Transform node and just do both the transformation and the opacity in one Merge, and output both from there prior to doing the actual composite.

So here's an alternative method, which uses fewer nodes because it doesn't provide a separate transform-only output. You could use two Merge nodes, with one being an Instance (linked copy) of the other. One you merge over an Alpha=0 background, and the output of that is your 'FG only'. The other is your actual composite:


Whatever you do in MergeComposite2 will be automatically mirrored in Instance_MergeComposite2, but because the instance is merged over a Background with RGBA = 0,0,0,0, only the foreground is affected in that case.

This does depend on the Blend mode used. For example if you wanted to use Multiply in MergeComposite2, the image coming from the Instance_MergeComposite2 would be pure black, because multiplying any pixel value by 0 will give 0.

Therefore I would de-instance Blend Mode and set Instance_MergeComposite2 to Normal blend mode. With that done, the Instance_MergeComposite2 would always be the FG only after transformation and opacity adjustments, regardless of the blend mode used in MergeComposite2.

If you felt this 'FG only' output was something you needed a lot, you could make a Macro of either of the above two methods, which could then be deployed as a single node and could have multiple outputs, one being the normal, full Merge output, the other being the "FG only" (meaning after transform and opacity, before composite).

There may well be other, perhaps more elegant ways to achieve what you want, so if you give a real life example of what you're trying to achieve, maybe we can help more specifically.

Node code for method one:

Code: Select all

{
   Tools = ordered() {
      TextForeground = TextPlus {
         NameSet = true,
         Inputs = {
            GlobalOut = Input { Value = 100, },
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
            StyledText = Input { Value = "fusion", },
            Font = Input { Value = "Open Sans", },
            Style = Input { Value = "Bold", },
            VerticalJustificationNew = Input { Value = 3, },
            HorizontalJustificationNew = Input { Value = 3, },
            ManualFontKerningPlacement = Input {
               Value = StyledText {
                  Array = {
                  },
                  Value = ""
               },
            },
         },
         ViewInfo = OperatorInfo { Pos = { 1103.5, 469.276 } },
      },
      Transform8 = Transform {
         Inputs = {
            Input = Input {
               SourceOp = "TextForeground",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 1103.5, 514.762 } },
      },
      BGAlpha0 = Background {
         NameSet = true,
         Inputs = {
            GlobalOut = Input { Value = 100, },
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
            TopLeftAlpha = Input { Value = 0, },
            TopRightGreen = Input { Value = 1, },
            BottomLeftBlue = Input { Value = 1, },
            BottomRightRed = Input { Value = 1, },
            BottomRightGreen = Input { Value = 1, },
         },
         ViewInfo = OperatorInfo { Pos = { 984.579, 564.203 } },
      },
      MergeOpacity = Merge {
         NameSet = true,
         Inputs = {
            Background = Input {
               SourceOp = "BGAlpha0",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "Transform8",
               Source = "Output",
            },
            PerformDepthMerge = Input { Value = 0, },
         },
         ViewInfo = OperatorInfo { Pos = { 1103.5, 564.203 } },
      },
      PipeRouter2_2 = PipeRouter {
         Inputs = {
            Input = Input {
               SourceOp = "MergeOpacity",
               Source = "Output",
            },
         },
         ViewInfo = PipeRouterInfo { Pos = { 1225.55, 595.186 } },
      },
      MergeComposite = Merge {
         NameSet = true,
         Inputs = {
            Background = Input {
               SourceOp = "Background",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "MergeOpacity",
               Source = "Output",
            },
            PerformDepthMerge = Input { Value = 0, },
            ReferenceSize = Input { Value = 1, },
         },
         ViewInfo = OperatorInfo { Pos = { 1103.5, 611.008 } },
      },
      Background = Background {
         NameSet = true,
         Inputs = {
            GlobalOut = Input { Value = 100, },
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
            Type = Input { Value = FuID { "Corner" }, },
            TopLeftRed = Input { Value = 1, },
            TopRightGreen = Input { Value = 1, },
            BottomLeftBlue = Input { Value = 1, },
            BottomRightRed = Input { Value = 1, },
            BottomRightGreen = Input { Value = 1, },
         },
         ViewInfo = OperatorInfo { Pos = { 980.449, 611.008 } },
      },
      PipeRouter2_1 = PipeRouter {
         CtrlWZoom = false,
         Inputs = {
            Input = Input {
               SourceOp = "Transform8",
               Source = "Output",
            },
         },
         ViewInfo = PipeRouterInfo { Pos = { 1231.35, 547.723 } },
      }
   }
}

Node code for method two:

Code: Select all

{
   Tools = ordered() {
      TextForeground_2 = TextPlus {
         NameSet = true,
         Inputs = {
            GlobalOut = Input { Value = 100, },
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
            StyledText = Input { Value = "fusion", },
            Font = Input { Value = "Open Sans", },
            Style = Input { Value = "Bold", },
            VerticalJustificationNew = Input { Value = 3, },
            HorizontalJustificationNew = Input { Value = 3, },
            ManualFontKerningPlacement = Input {
               Value = StyledText {
                  Array = {
                  },
                  Value = ""
               },
            },
         },
         ViewInfo = OperatorInfo { Pos = { 573.647, 542.449 } },
      },
      Instance_MergeComposite2 = Merge {
         NameSet = true,
         SourceOp = "MergeComposite2",
         Inputs = {
            SettingsNest = Input { },
            Background = Input {
               SourceOp = "BGAlpha0",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "TextForeground_2",
               Source = "Output",
            },
            Merge = Input { },
            ApplyMode = Input { },
            Channels = Input { },
            ReferenceSize = Input { Value = 1, },
            CommentsNest = Input { },
            FrameRenderScriptNest = Input { },
            StartRenderScripts = Input { },
            EndRenderScripts = Input { },
            EffectMask = Input { }
         },
         ViewInfo = OperatorInfo { Pos = { 667.915, 585.957 } },
      },
      MergeComposite2 = Merge {
         CtrlWZoom = false,
         NameSet = true,
         Inputs = {
            Background = Input {
               SourceOp = "Background21",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "TextForeground_2",
               Source = "Output",
            },
            ApplyMode = Input { Value = FuID { "Multiply" }, },
            FilterMethod = Input { Value = 3, },
            PerformDepthMerge = Input { Value = 0, },
            ReferenceSize = Input { Value = 1, },
         },
         ViewInfo = OperatorInfo { Pos = { 573.647, 630.784 } },
      },
      Background21 = Background {
         NameSet = true,
         Inputs = {
            GlobalOut = Input { Value = 100, },
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
            Type = Input { Value = FuID { "Corner" }, },
            TopLeftRed = Input { Value = 1, },
            TopRightGreen = Input { Value = 1, },
            BottomLeftBlue = Input { Value = 1, },
            BottomRightRed = Input { Value = 1, },
            BottomRightGreen = Input { Value = 1, },
         },
         ViewInfo = OperatorInfo { Pos = { 453.492, 630.784 } },
      },
      PipeRouter2 = PipeRouter {
         Inputs = {
            Input = Input {
               SourceOp = "Instance_MergeComposite2",
               Source = "Output",
            },
         },
         ViewInfo = PipeRouterInfo { Pos = { 725.201, 630.784 } },
      },
      BGAlpha0 = Background {
         NameSet = true,
         Inputs = {
            GlobalOut = Input { Value = 100, },
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
            TopLeftAlpha = Input { Value = 0, },
            TopRightGreen = Input { Value = 1, },
            BottomLeftBlue = Input { Value = 1, },
            BottomRightRed = Input { Value = 1, },
            BottomRightGreen = Input { Value = 1, },
         },
         ViewInfo = OperatorInfo { Pos = { 786.617, 585.957 } },
      }
   }
}

Having written all that, I realised that there can be blend modes in which the output from the two example methods I gave is not the same as doing the opacity within a single Merge.

In particular, Colour Dodge blend mode will definitely give a different result and I would say should not be used with the methods given. Another is Difference.

So be aware that these methods may not work if you intend to do a Colour Dodge or Difference blend mode, I guess because the order of operations is changed: if we change the FG opacity before the Merge, it appears to give a different result compared to changing the opacity within the compositing Merge.

In fact several blend modes including Normal give a very fractionally different result - not noticeable to the naked eye, but visible when inspecting the colour values. The differences were tiny, like 0.0005, but it's still a difference. So that's something to bear in mind.

I think I figured out the solution to my use case, partly with your help, and also with experimentation. My goal was to be able to put one image over another two separate times, and then mix them how I want with a mask before finally putting that over the original background. I think you'll like what I came up with, and it works regardless of blend mode. (Since you said dodge and difference didn't work in your version, I used them in mine just to show that it works.)

First, I'm starting with these two images:

Originals

FGOnlyMerge01.png (722.29 KiB) Viewed 2345 times

The idea is that I want to merge image one over image two using different blend modes, and have control over how I combine the results as though they were one composite to begin with. For example, lets say I like the results of dodge for the parts in shadow, but prefer difference for the normally lit areas. This node structure uses the original foreground as a mask to remove the background that was essential to the changes done in the merge:

FG only merge

FGOnlyMerge2.png (963.83 KiB) Viewed 2345 times

This allows me to have control over the "foreground only" version of the merge, so I can mask it however I want before combining it with the other blend mode, before finally putting it over the original background:

Mask and merge again

FGOnlyMerge3.png (864.05 KiB) Viewed 2345 times

Seems to work exactly how I wanted it to, but it definitely has a lot of nodes and routing. Any idea if this might lead to performance issues if implemented into a bigger node tree?

I'm curious about the differences related to opacity though. In this example, I would have done opacity with the blend control on the last merge node - where the combined images are placed over the background for the final time (Merge4 in my screenshots). I figured this would essentially be the same as using the blend control earlier on. Isn't that the difference between "gain" and "blend"? Gain happens before the blend math, then blend is essentially the opacity of the result over the original background? (I assumed this because the blend slider in the settings section is tied with the one in the blend mode area.)

I didn't read the whole thread, so I don't have context. However, the Alpha Gain control is a multiplier on the Foreground alpha. If your (simplified) Over is this:

FG + bg * (1-A)

Then adding the Gain control makes it this:

FG + bg * (1-A*Gain)

Where A is the foreground's Alpha.

So turning Gain down to 0 turns it the Merge into a simple Add:

FG + bg * (1-A*0) = FG + bg * (1-0) = FG + bg

For more information is overly exhaustive detail, I have an article on the topic:
http://www.bryanray.name/wordpress/blac ... thematics/

Cody, I'll reply to your reply later tonight.

I just wanted to reply now to say wow, Bryan, what a great resource! I'd read all your book last year but hadn't realised you'd added extra material since then. That's really useful, thank you.

In a similar vain, and maybe of interest to Cody and anyone else finding this thread, I can highly recommend Simon Ubsdell's "The Basics Of Compositing Theory", a five-part YouTube tutorial series. The first two parts are basically the video version of what Bryan's written above, with later parts dealing with the alpha channel and brightness. He planned a sixth part discussing gamma, but sadly hasn't managed to get to that yet.

It's all demonstrated in Fusion, using the powerful Custom Tool (as Bryan also used in his written guide). As a newbie myself I can highly recommended it for anyone learning compositing, Fusion, or both:

Brian, when I said "gain" I was referring to the gain slider on blend modes other than "normal" - not sure if that makes a difference to your response. The manual doesn't seem to say anything about "gain" on its own, so maybe it's just short for alpha gain and they are the same?

Ah, I'd have to do some testing on that. It's possible that the slider does different things in different modes. Some of the contrast modes do some very strange things, mathematically speaking.

I think you'll like what I came up with, and it works regardless of blend mode.

Right OK. That's nothing like the Planar Tracker's FG only mode, but I see what you're trying to do now.

Yes, using the FG image as the mask seems fine. If you were doing a Normal or Screen merge there's some further control available, such as "Atop" and "In" which can limit the merge to only the areas defined by the alpha. Unfortunately that's not possible with the other blend modes. (To use Atop and the others you'd need to swap the inputs, eg make your text the 'background' and the background the 'foreground', then Atop or In would only display background in the areas covered by the foreground alpha)

FYI there's another option for cutting stuff out, rather than Merging over Alpha=0 BG. You can also use a MatteControl. Pass the image into the MC's yellow input, and the mask (or the image to use as a mask) into the green input. Then set the MC to "Combine Alpha" and "Post Multiply". That has the advantage of not requiring the Background node.

Here's an example using the MatteControl to do the cutting out, rather than a Merge + BG:


Extending that idea, we could try cutting out the area of the BG prior to doing the merge:


In that second example, we first cut out a FG-sized hole in the BG, then Merge that in the two different ways, before combining those merges. This is the fewest nodes of the methods presented so far. Be aware that this can subtly change the final image, again dependent on blend mode, versus cutting out after the merge. It shouldn't affect the bulk of the image, but in soft-edge situations the edge pixels may process differently depending if you cut out before or after the merge. Try both ways and see what you like.

I don't think you're likely to have any major performance problems with any of the methods, they're all simple operations. Any performance differences between the various methods described are unlikely to make a meaningful difference unless you're rendering tens of minutes of footage from this comp. Assuming you can get the same result out of all of them, use whichever method makes most sense to you, or gives you greatest flexibility, or is most understandable at a glance.

Just remember to try and avoid using MediaIn nodes for still images, which it looks like you are still doing. That can significantly reduce performance in Resolve.

Regarding the Gain slider: in the Normal and Screen blend mode there's Alpha Gain, and any other mode has Gain. As you say, Gain is not explained in the manual, at least not in the Merge node reference section.

My understanding was that Gain was a multiplier against foreground RGB (just like Alpha Gain is a multiplier against FG alpha), prior to the composite operation. There's a few other nodes that have Gain sliders and composite operations, for example the Duplicate node. The manual for Duplicate describes Gain as: "The Gain RGB controls multiply the values of the image channel linearly. All pixels are multiplied by the same factor, but the effect is larger on bright pixels and smaller on dark pixels. Black pixels are not changed since multiplying any number times 0 always equals 0.".

I would guess the Gain in Merge is the same deal, but haven't specifically analysed it.

In some blend modes, eg Color Dodge, Blend and Gain appear identical, ie 0.5 Gain + 1.0 Blend is the same as 1.0 Gain + 0.5 Blend. But in Difference for example they give quite different results.

I've not investigated this any further, however I'm not sure if it's relevant for what you're doing here. I mentioned doing opacity before the merge because I thought you wanted an equivalent of the Planar Tracker FG Only, ie a separation of the Merge's transformation and opacity controls from the actual composite operation. But that's not what you're doing.

Just remember to try and avoid using MediaIn nodes for still images, which it looks like you are still doing. That can significantly reduce performance in Resolve.

Good shout. I keep forgetting that.

Thank you both for your help!