Jump to: Board index » General » Fusion

Complicated "Modify with"/expression question

Learn about 3D compositing, animation, broadcast design and VFX workflows.
  • Author
  • Message
Offline

TCP786

  • Posts: 456
  • Joined: Thu Sep 16, 2021 7:05 am
  • Real Name: Cody Predum

Complicated "Modify with"/expression question

PostWed Oct 13, 2021 9:02 am

I have a somewhat complicated use case for the "modify with" feature, and I need to understand how it works a little better in order to figure out how to solve my issue. Here's what I'm trying to do:

I'm using the ImagePlane3D node to transform the perspective of a 2D image for a composite. As is expected, the X and Y translation parameters affect the perspective of the image rather than just moving the image on screen without otherwise changing it. For example, take an image rotated 45 degrees on the Y axis. The previously parallel lines of the top and bottom of the image now converge on the right side of the screen, with one angled down and the other angled up. Then, by moving it up, it is now above the 3D camera, making both lines angled downward. See Figure 1:
Modifier question 04.png
Figure 1
Modifier question 04.png (177.04 KiB) Viewed 1397 times

But in my case, sometimes I want to achieve that perspective shift while keeping the image in the same part of the screen. This isn't difficult; you just place a regular transform after the Render3D node that follows the ImagePlane3D node and compensate, as shown in Figure 2:
Modifier question 02.png
Figure 2
Modifier question 02.png (181.97 KiB) Viewed 1397 times

Here's where my modifier/expressions question actually comes in. I would love to be able to set up some kind of modifiers that allow me to link the ImagePlane3D node's translation parameters to the downstream Transform node's translation parameters in an inverse way, such that I could change the Y value in ImagePlane3D to apply the perspective change without changing the on-screen position of the image's pivot point.

I know this probably requires quite a bit of non-trivial math once I start factoring more parameters into the equation, but for now I just want to figure out how to properly set up modifiers/expressions.

And two more quick questions about the finer details. First, there must be a way to write more complicated expressions somewhere other than the tiny single-line text field that comes up when you assign a parameter to an expression, right? Second, I am noticing that the X and Y components of the Transform node's Center appear to be manipulated together as a "Point" with an X component and a Y component (a Vector2 I'm guessing?). Is there a way to do the math for X and Y separately as floats, and then assign the results to the appropriate part of the Point type variable?

Thanks for any advice anyone happens to have on this (looking at you, Brian).
Offline

Hendrik Proosa

  • Posts: 3015
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Complicated "Modify with"/expression question

PostWed Oct 13, 2021 10:46 am

I can't help you with any details, but just my two cents in general. To get this kind of relationship you need to know the transformation matrix of imageplane and projection matrix of the renderer. If your pivot is at scene origin for example, applying the transform matrix and projection matrix to it will produce final position in screen space, which can be used to calculate the screen space offset from original position (with no transform matrix). Whether these matrices are available from imageplane3d and renderer, I don't know unfortunately.
I do stuff.
Offline
User avatar

Bryan Ray

  • Posts: 2478
  • Joined: Mon Nov 28, 2016 5:32 am
  • Location: Los Angeles, CA, USA

Re: Complicated "Modify with"/expression question

PostWed Oct 13, 2021 5:45 pm

Iiiinteresting...

There's a useful tool for converting 3d coordinates into screen space coordinates: Locator3D. Put this node somewhere downstream of your Camera3D, and it will provide an auxiliary output that is its own location in screen space. If you link the Locator's 3D position to the object you want to manage, you can then Connect the Center control of your Transform node to it. Tick the Invert Transform switch, and you'll get a counter-translation for your 3d move.

Code: Select all
{
   Tools = ordered() {
      Camera3D1_9 = Camera3D {
         CustomData = {
            Settings = {
               [1] = {
                  Tools = ordered() {
                     Camera3D1_8 = Camera3D {
                        Inputs = {
                           ApertureW = Input { Value = 0.831496062992126 },
                           AoV = Input { Value = 19.2642683071402 },
                           ["Stereo.Mode"] = Input { Value = FuID { "OffAxis" } },
                           ["SurfacePlaneInputs.ObjectID.ObjectID"] = Input { Value = 1 },
                           ApertureH = Input { Value = 0.467716535433071 },
                           FilmGate = Input { Value = FuID { "BMD_URSA_4K_16x9" } },
                           ["MtlStdInputs.MaterialID"] = Input { Value = 1 }
                        },
                        Name = "Camera3D1_8",
                        CtrlWZoom = false,
                        ViewInfo = OperatorInfo { Pos = { -2750, -1897.5 } },
                        CustomData = {
                        }
                     }
                  }
               },
               [6] = {
                  Tools = ordered() {
                     Camera3D1_8 = Camera3D {
                        Inputs = {
                           FLength = Input { Value = 49.0568334445481 },
                           ["Transform3DOp.Translate.Z"] = Input { Expression = "self.ImageInput.Metadata.Translate.Z" },
                           ImagePlaneEnabled = Input { Value = 0 },
                           AoV = Input {
                              Value = 19.2642683071402,
                              Expression = "self.ImageInput.Metadata.RSCameraFOV or self.ImageInput.Metadata['rs/camera/fov']"
                           },
                           AovType = Input { Value = 1 },
                           ["Transform3DOp.Rotate.Y"] = Input { Expression = "self.ImageInput.Metadata.Rotate.Y" },
                           ApertureH = Input { Value = 0.9 },
                           FilmGate = Input { Value = FuID { "HD" } },
                           ["Transform3DOp.Rotate.X"] = Input { Expression = "self.ImageInput.Metadata.Rotate.X" },
                           ["Transform3DOp.Translate.X"] = Input { Expression = "self.ImageInput.Metadata.Translate.X" },
                           PlaneOfFocus = Input { Expression = "self.ImageInput.Metadata.RSCameraDOFFocusDistance or self.ImageInput.Metadata['rs/camera/DOFFocusDistance']" },
                           FilmBack = Input { Value = 1 },
                           ["Transform3DOp.Rotate.RotOrder"] = Input { Value = FuID { "ZXY" } },
                           ["MtlStdInputs.MaterialID"] = Input { Value = 1 },
                           ["Stereo.Mode"] = Input { Value = FuID { "OffAxis" } },
                           ["SurfacePlaneInputs.ObjectID.ObjectID"] = Input { Value = 1 },
                           ["Transform3DOp.Translate.Y"] = Input { Expression = "self.ImageInput.Metadata.Translate.Y" },
                           ApertureW = Input { Value = 1.6 },
                           ["Transform3DOp.Rotate.Z"] = Input { Expression = "self.ImageInput.Metadata.Rotate.Z" }
                        },
                        CtrlWZoom = false,
                        ViewInfo = OperatorInfo { Pos = { -385, -214.5 } },
                        CustomData = {
                        }
                     }
                  }
               }
            }
         },
         Inputs = {
            ["Transform3DOp.Translate.Z"] = Input { Value = 7.93569424565983, },
            AoV = Input { Value = 27.8, },
            FLength = Input { Value = 24.0024263658569, },
            ["Stereo.Mode"] = Input { Value = FuID { "OffAxis" }, },
            FilmGate = Input { Value = FuID { "BMD_URSA_4K_16x9" }, },
            ApertureW = Input { Value = 0.831496062992126, },
            ApertureH = Input { Value = 0.467716535433071, },
            ["SurfacePlaneInputs.ObjectID.ObjectID"] = Input { Value = 1, },
            ["MtlStdInputs.MaterialID"] = Input { Value = 1, },
         },
         ViewInfo = OperatorInfo { Pos = { 0, -280.5 } },
      },
      Shape3D1 = Shape3D {
         CtrlWZoom = false,
         Inputs = {
            ["Transform3DOp.Translate.X"] = Input { Value = -2.446, },
            ["Transform3DOp.Rotate.Y"] = Input { Value = 90, },
            ["MtlStdInputs.MaterialID"] = Input { Value = 4, },
            ["SurfacePlaneInputs.ObjectID.ObjectID"] = Input { Value = 4, }
         },
         ViewInfo = OperatorInfo { Pos = { 0, -247.5 } },
      },
      Locator3D2 = Locator3D {
         Inputs = {
            SceneInput = Input {
               SourceOp = "Camera3D1_9",
               Source = "Output",
            },
            ["Transform3DOp.Translate.X"] = Input {
               Value = -2.446,
               Expression = "Shape3D1.Transform3DOp.Translate.X",
            },
            ["Transform3DOp.Translate.Y"] = Input { Expression = "Shape3D1.Transform3DOp.Translate.Y", },
            ["Transform3DOp.Translate.Z"] = Input { Expression = "Shape3D1.Transform3DOp.Translate.Z", },
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            UseFrameFormatSettings = Input { Value = 1, },
         },
         ViewInfo = OperatorInfo { Pos = { 110, -280.5 } },
      },
      Renderer3D2 = Renderer3D {
         CustomData = {
            ToolVersion = 2,
         },
         Inputs = {
            GlobalIn = Input { Value = 1001, },
            GlobalOut = Input { Value = 1100, },
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
            SceneInput = Input {
               SourceOp = "Merge3D3",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 220, -247.5 } },
      },
      Transform3 = Transform {
         Inputs = {
            Center = Input {
               SourceOp = "Locator3D2",
               Source = "Position",
            },
            InvertTransform = Input { Value = 1, },
            Input = Input {
               SourceOp = "Renderer3D2",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 330, -247.5 } },
      },
      Merge3D3 = Merge3D {
         Inputs = {
            SceneInput1 = Input {
               SourceOp = "Shape3D1",
               Source = "Output",
            },
            SceneInput2 = Input {
               SourceOp = "Locator3D2",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 110, -247.5 } },
      }
   }
}



Regarding the limitations of the Simple Expressions entry box: If I'm writing something particularly complex, I'll often open a programmer's text editor and write it there with sensible line breaks and syntax highlighting, then reformat it into the single-line expression and paste it into the expression field. There's also the option of the Expression Modifier, which gives you access to some sliders and Point controls and a bigger text field for writing the expressions.

As for the Point datatype, yes, it's a vector2. To create one, use the Point() function. For example:
Point(Transform1.Center.X, Transform2.Center.Y)

For a live example of expressions on Point controls (as well as some other advanced techniques), check out the FlexiTrack macro in Reactor. I believe I submitted it as a group, so you can open it up to see how it works.
Bryan Ray
http://www.bryanray.name
http://www.sidefx.com
Offline

TCP786

  • Posts: 456
  • Joined: Thu Sep 16, 2021 7:05 am
  • Real Name: Cody Predum

Re: Complicated "Modify with"/expression question

PostWed Oct 13, 2021 11:47 pm

Interesting. Brian, is the Locator3D node essentially doing what Hendrik is describing? Also, I am not using a Camera3D node. I am starting with an image file in a Loader node, sending it through a Bitmap node and a Background node (so I can change the color within Fusion), then going to the Image3D node followed immediately by the Render3D node and the regular Transform node for any screen-space related adjustments. After that it goes to the rest of the comp related stuff. Here's a basic example of my node tree:
nodes.png
nodes.png (30.61 KiB) Viewed 1252 times

Can I still use the Locator3D node after the ImagePlane3D or Render3D nodes?

Also, my experimentation landed me on this expression for now:
Code: Select all
Point(0.5, 0.5 - ((ImagePlane3D1.Transform3DOp.Translate.Y)/1.7))

I can now slide the Translate Y value around and essentially get the behavior I'm after. I eventually want to be able to manipulate X and Z as well without moving the image's position on screen. That division by 1.7 is just a number I found by experimenting that's almost perfectly accurate (I'm curious if the math might work out to square root of 3 for some reason, but haven't checked yet). I was right that I could just use a constant if I left everything else unchanged, but I think it will have to become a variable based on some trigonometry with the X, Y, and Z values in order to get all 3 of them working together.

Anyway, after the "corrective" Transform node that re-centers the image, I can add another transform node (not included in the above node tree screenshot) to actually put it where I want it in screen space. If I can get it working with X, Y, and Z, and hopefully also the pivot X, Y, and Z as well, I'll have an extremely robust tool to manipulate a 3D transform in a totally bonkers and completely backwards way! (Yet somehow I think it'll be easier for me to use in some of the more confusion circumstances I've run into.)

On the topic of getting it working with X, Y, and Z, does ImagePlane3D expose the distance from the 3D-space camera to the image pivot point? If the camera is (0,0,0), this would be the magnitude of the vector3 position of the image. I can always manually write out The Holy Scripture of Praise to Our Lord and Savior Pythagoras (For He is Always and At Once the Opposite, the Adjacent, and the Holy Hypotenuse), but it'd be nice if I could just grab that number from a variable instead. At least Father Pythagoras has shown us how to find our solutions without resorting to Sin. (Ok, I'll stop with these obtuse jokes. ...starting now.)

I also think that if the ImagePlane3D pivot isn't centered, I might also have to account for its distance from the center of whatever plane is part of the camera's frustrum and includes the ImagePlane3D pivot point, but I'll worry about that once I get all of the Translate parameters working.

Thanks for any additional help.
Offline
User avatar

Bryan Ray

  • Posts: 2478
  • Joined: Mon Nov 28, 2016 5:32 am
  • Location: Los Angeles, CA, USA

Re: Complicated "Modify with"/expression question

PostThu Oct 14, 2021 12:20 am

TCP786 wrote:Interesting. Bryan, is the Locator3D node essentially doing what Hendrik is describing?


I believe so, but I didn't read his message for mathematical comprehension because I already knew the answer. :lol: Also, I'm really bad at matrices.

ImagePlane3D's default location will be at the origin, so the camera's can't be. I have no idea what the properties of the default camera are if you don't actually have one in the scene. I suppose it wouldn't be terribly difficult to find out, though…

It appears as though the default camera is at Z = 5 units, and it has a horizontal angle-of-view of 19.26427 degrees (equivalent to a 35mm lens on the Blackmagic URSA 4k camera).

It appears as though if you give the Locator any scene input (from your card upstream, for instance), that it will recognize that default camera and act exactly as it did in my sample scene. It can't take the Renderer3D node as an input because that only outputs a 2d raster, and Locator requires a 3D Scene. It doesn't necessarily need to go into the Renderer, but it doesn't hurt anything if it does, as by default it's not renderable.
Bryan Ray
http://www.bryanray.name
http://www.sidefx.com
Offline

TCP786

  • Posts: 456
  • Joined: Thu Sep 16, 2021 7:05 am
  • Real Name: Cody Predum

Re: Complicated "Modify with"/expression question

PostThu Oct 14, 2021 3:35 am

Bryan Ray wrote:ImagePlane3D's default location will be at the origin, so the camera's can't be.
Well that's easy; it's just the magnitude of the camera's position vector instead then. So based on this:
Bryan Ray wrote:It appears as though the default camera is at Z = 5 units
The camera's position is (0,0,5), then?
Offline
User avatar

Bryan Ray

  • Posts: 2478
  • Joined: Mon Nov 28, 2016 5:32 am
  • Location: Los Angeles, CA, USA

Re: Complicated "Modify with"/expression question

PostThu Oct 14, 2021 3:59 pm

Yes, exactly.

It's possible that it's somewhere else with a different AoV, but I didn't feel like setting up a more elaborate scene to try to then solve, so I made the assumption that it uses the same AoV as the Camera3D node's default.
Bryan Ray
http://www.bryanray.name
http://www.sidefx.com

Return to Fusion

Who is online

Users browsing this forum: No registered users and 33 guests