Use a Z-Depth Pass for depth of field

Hello, I rendered a Z-depth pass in EXR format with 3DS Max. I want to use this as a depth of field. I found some tutorials but I can't understand why it should be so complicated. In After Effects I use an extractor to get the grayscale visible and then it's easy to use it as DoF. Is there such an easy way to do this in DR or DRS?

Yes, you bring the EXR files to Fusion and extract all auxiliary channels. You can even pass them to the color page.

Thank you . This works only for DRS or also for DR ?
Because I'm only getting my DRS next week.

The fusion page should have all you need in both free and studio versions. Use CopyAux to extract Z-Depth and if you want to play around with DoF, there is a good macro out there to try if you like DepthDefocus:

https://www.steakunderwater.com/VFXPedi ... escription

You can bring in EXR files into Fusion and extract Z-Depth information as mentioned. Easiest way is to use CopyAux node It Copies aux (auxiliary) channels, like Z-depth pass, including motion vectors, into RGBA more efficiently than Channel Booleans. It is a node that takes aux channels and copies them to color channels as well as normalizing them, so its ready to be used as a grayscale image in for example red channel if you want to pass it on to tools that can use it for Depth of Field Effect.

You can find more in reference manual.



"The Copy Aux node is used to shuffle channels between visible channels and auxiliary data channels in a single 2D image. Typically, these auxiliary channels are rendered from 3D applications. Auxiliary channels supported in the Copy Aux node include background color, z-depth, texture coordinates, coverage, object ID, material ID, normals, vectors, back vectors, and world position.

The Copy Aux node is mostly a convenience node, as the copying can also be accomplished with more effort using a Channel Booleans node. Where Channel Booleans deals with individual channels, Copy Aux deals with channel groups. By default, the Copy Aux node automatically promotes the depth of its output to match the depth of the aux channel.

Copy Aux also supports static normalization ranges. The advantage of static normalization versus the dynamic normalization that Fusion’s viewers do is that colors remain constant over time. For example, if you are viewing Z or WorldPos values for a ball, you see a smooth gradient from white
to black. Now imagine that some other 3D object is introduced into the background at a certain time. Dynamic normalization turns the ball almost completely white while the background object
is now the new black. Dynamic normalization also causes flicker problems while viewing vector/ disparity channels, which can make it difficult to compare the aux channels of two frames at different times visually

Basic Node Setup

The Copy Aux node, like many 2D image-processing nodes, receives a 2D image like a Loader node or the MediaIn1 shown below. The output continues the node tree by connecting to another 2D image- processing node or a Merge node.

sshot-4039.jpg (14.15 KiB) Viewed 1463 times

Thank you for the detailed answer Smithy.
Unfortunately I can't get it to work. I think that the nodes are right behind each other, i.e. Loader/Copyauxial/Brightness/Mediaout?
But as you can see, the nodes are red, so nothing gets there.
The EXR file (Z-depth) has a depth of 32, so I think I have to set Out Color Depth to Force float 32 and the Aux Channel to Z ?
Something only comes out when I set the Aux Channel to Color.
Then I at least see a kind of alpha mask, but that can't be right.

Screenshot 2024-05-25 221237.jpg (6.23 KiB) Viewed 1366 times

Thank you for the detailed answer Smithy.

No problem.

Screenshot 2024-05-25 221237.jpg

Hmm that's strange. Would it be possible to provide exr, one frame or something similar you exported in the same way for me to take a look?

Sure , unfortunately, an image is 7MB and I can't upload it here. Are there other ways to upload data here?

Sure , unfortunately, an image is 7MB and I can't upload it here. Are there other ways to upload data here?

Maybe you don't have to. I didn't want to generate a 3D scene in Fusion so I found a EXR with Z depth embedded online to illustrate.

First good idea is to check if the EXR has Z channel by choosing it from the menu. And than choosing normalize range to see it.

sshot-4044.jpg (179.04 KiB) Viewed 1347 times

Once the existence of it is confirm, you can proceed to CopyAux and get the same results in Red Channel. Here are the options to use. Make sure Detect Range to do what normalize button does.

sshot-4045.jpg (131.82 KiB) Viewed 1347 times

As you can see, we get the same thing, only now in Red channel. This particular shallow depth of field example so its barely noticeable, and you can than use brightness contrast as you have to make it even more visible if you so choose. Usually this is not needed. CopyAux node does it. This is unusual example I found.

sshot-4046.jpg (128.72 KiB) Viewed 1347 times

Let me know if it works or not, maybe there is something else a problem. But it should be straightforward. You can also extract Z depth from world pass and world pass from Z depth using a node with that name.

mhh, you're right, the picture doesn't have a Z channel. I don't understand, if I render it out as a Z pass in 3ds max I can use it in After Effects as a grayscale gradient for DoF. So I have to figure out what's wrong with 3ds max?

Screenshot 2024-05-25 233649.jpg (150.43 KiB) Viewed 1335 times

So under the settings in 3ds max Arnold, there is no explicit Z channel. I had output s as rgba. I'm now trying out all the possibilities with a picture, if there is a Z channel somewhere.

Screenshot 2024-05-25 235633.jpg (34 KiB) Viewed 1323 times

mhh, you're right, the picture doesn't have a Z channel. I don't understand, if I render it out as a Z pass in 3ds max I can use it in After Effects as a grayscale gradient for DoF. So I have to figure out what's wrong with 3ds max?

I don't know 3D studio max settings so I can't help you with that, but are you by any chance just exporting a grayscale RGB image that represents Z channel from 3D studio max? Because that makes what I showed redundant, you can use it as it is for 2D DOF effect. Its already ready to go. Or perhaps 3D Studio Max allows that to be done. Either way...

If you want to use it as a Z-Chanel in Aux channels than you can use CopyAux and apply reverse operation. The Z to World Pos node is used to either generate a World Position Pass from a Z channel and a 3D Camera or a Z channel from a World Position Pass and a 3D Camera.

Creating a World Position Pass from Z-depth can be useful when your 3D application is not capable of creating a WPP. It can also be used when a 3D-tracking software outputs a per-pixel Z-depth together with the 3D Camera. Thus, the Volume Mask and Volume Fog nodes could be applied to real-world scenes. The quality of the resulting WPP depends mainly on the quality of the incoming Z channel.

Basically, you have every option covered. I though you were exported EXR with Z-Depth in the Aux Channel. And you wanted to extract it to RGB, for example, Red Channel. If you already have that when you export from 3D studio Max than depending of what you need, you can either use it as is, or convert it to Z-Depth in Aux channel using copyAux node, or as I've mentioned make into world pass. It just depends on what you are starting with, and what you want to do. Unless off course there is some problem with 3D studio max export. You will have to investigate that on your own.

This is all I could find in the user guide.

https://help.autodesk.com/view/ARNOL/EN ... h_aov_html

I have Cinema4D and Arnold, but I haven't used it in a while. If it doesn't work, I can try it myself tomorrow, probably.

Channels/channelsets are the AOVs (Arbitrary Output Variables) in that screenshot. Data selection specifies data type. Z depth is single channel so data type is float.

I set it to float in 3ds max. Does that mean that you don't necessarily need a Z channel since the information is in the RGBs?
I hope the nodes in Fusion are correct.
In any case, if I just turn on the red channel, you can at least see that the information is there. It's actually the same as in After Effects, the EXR has to be normalized so that the grayscale gradient becomes visible. But I've already played around with all the buttons and there's no progression.

Screenshot 2024-05-26 161449.jpg (34.5 KiB) Viewed 1132 times

Screenshot 2024-05-26 162207.jpg (127.9 KiB) Viewed 1132 times

I set it to float in 3ds max. Does that mean that you don't necessarily need a Z channel since the information is in the RGBs?"

Well, I'm not sure what you want the end result to be and what your workflow is, but here are some common scenarios.

Ideally you want to render depth of filed effect in your 3D application or with Fusion if you are using its 3D rendering engine. This gives accurate, high quality results, but the big downside is that it significantly increases render times since its so computationally expensive. Especially if you need to make quick changes its a problem.

That is why many try to extract Z-Depth information to use is it as a way to blur the scene in 2D in the compositing application. I understood, originally, that is what you wanted to do. And you were doing that in After Effects and now wanted in Fusion.

When you approach it that way, z-Depth is just a grayscale image where luminance values inform the lens blur filter where to blur more and where to blur less. Hence we often need to extract Z-Depth channel from multichannel EXR auxiliary channel, or in Nuke they call them AOV's in order to create a grayscale image which is often for connivance sake put in the red channel of the RGB file. Since all channels in RGB read the same luminescence values, Red channel will work as good as Green or blue, but since its first on the list, people typically use red channel.

If you are able to somehow export z-Channel as grayscale RGB out of 3D application, than you don't need to do anything other than feed it into your lens blur filter of choice. Since most of them expect such a grayscale image.

If for workflow reasons, or because you have to or its convenient, multi channel EXR can also be used to extract the very same information as I've described before.

I hope the nodes in Fusion are correct. In any case, if I just turn on the red channel, you can at least see that the information is there. It's actually the same as in After Effects, the EXR has to be normalized so that the grayscale gradient becomes visible. But I've already played around with all the buttons and there's no progression.

Yes, Copy Aux can easily extract z-depth from auxiliary channel of EXR, put it in the RED channel as grayscale image and normalize it if that is what you need. If its not working, either you are doing something in the 3D application or in Fusion. But it should work if all is done correctly.

Sorry, I didn't make it clear what I wanted to achieve. It's like you said Smithy, I don't necessarily want to have the DoF calculated in 3DS Max because it's expensive and because then you no longer have any freedom. With a Z-Pass I still have the freedom to change the focus in post-production.
The workflow with 3ds max Vray is the simplest. Vray renders the Z-pass directly as a grayscale, which then only needs to be placed in AE with a lens effect and that's it. With 3DS Max and Arnold it's a bit more complicated because the Z-pass is only white and has to be normalized with an extractor (you have to play around with the highs and lows until you can see the grayscale result). And that's exactly what I'm trying to do now in DR. The rendered pass from 3ds max Arnold is completely white, so I first have to normalize it (I hope that's the right term "normalize") to make it visible as a grayscale before I take any further steps. And I fail at that, the image remains white, or you can only see an alpha, but there is no grayscale gradient. So I don't know what I have to connect where. CopyAux , ChannelBool , Brightness , none of this brings a grayscale gradient into the image.
Since I am not yet very familiar with DR and English is not my native language, I can imagine that I cannot express myself clearly enough, please excuse me.

It’s not white, z depth is not color data. Depth values express distance from camera and can be arbitrarily large. Any value greater than 1.0 is displayed white when viewed only because display can’t show these values. If you gain down the viewer you will see them if there is actual data.

Depending on z depth operator the type of depth input can vary. If it takes direct depth you don’t nees to normalize anything. That is necessary only if depth filter can’t use direct depth.

It is a bit unclear whether you have color + z in same exr or are you getting two exrs, one for color, another for z?

I assume that Arnold only saves the depth information in the EXE for the Z-Pass; I have my own passes for beauty, reflection, etc. As I have already tried to explain, I know that deep information should be hidden in the white EXE image, it just needs to be made visible. I just can't handle that. But I also don't understand why something so essential can't be solved more simply. I can do it in AE too.

mhh I just wanted to insert an image under 2MB, but EXR is not allowed (

mhh I just wanted to insert an image under 2MB, but EXR is not allowed (

Upload to some file sharing site and post a link. Add space somewhere inbetween the url path if encessary, forum doesn't allow posting some urls for security reasons.

Sorry, I didn't make it clear what I wanted to achieve. It's like you said Smithy, I don't necessarily want to have the DoF calculated in 3DS Max because it's expensive and because then you no longer have any freedom. With a Z-Pass I still have the freedom to change the focus in post-production.

OK.

The workflow with 3ds max Vray is the simplest. Vray renders the Z-pass directly as a grayscale, which then only needs to be placed in AE with a lens effect and that's it. With 3DS Max and Arnold it's a bit more complicated because the Z-pass is only white and has to be normalized with an extractor (you have to play around with the highs and lows until you can see the grayscale result). And that's exactly what I'm trying to do now in DR.

Ah, ok than I was correct. Yes. CopyAux is node made for that to do it all automatically so you don't have to fiddle with numbers or normalize it or anything.

The rendered pass from 3ds max Arnold is completely white, so I first have to normalize it (I hope that's the right term "normalize") to make it visible as a grayscale before I take any further steps. And I fail at that, the image remains white, or you can only see an alpha, but there is no grayscale gradient. So I don't know what I have to connect where. CopyAux , ChannelBool , Brightness , none of this brings a grayscale gradient into the image.

Unless you are missing Z-Depth usable values or Z-Depth channel altogether, CopyAux will simplify the process as I've illustrated.

Since I am not yet very familiar with DR and English is not my native language, I can imagine that I cannot express myself clearly enough, please excuse me.

Everything is fine. No worries.

If you have the information already in the EXR it should be pretty straight forward in Fusion as I've demonstrated. If you don't have any usable data it must be a problem on the Arnold export side of things. Is DOF activated in camera in your 3D application?

If I do DoF in Arnold, then I no longer need a Z-Pass. As already discussed above, it works well, but it is expensive and you no longer have the freedom to change the focus.
If you say CopyAux should do it automatically and normalize the EXR, then it should work. It doesn't do that, but it works in AE. I think I'll get my studio version tomorrow, I think I read that DR has a problem with the normalization

mhh, you're right, the picture doesn't have a Z channel. I don't understand, if I render it out as a Z pass in 3ds max I can use it in After Effects as a grayscale gradient for DoF. So I have to figure out what's wrong with 3ds max?

Screenshot 2024-05-25 233649.jpg

If the exr works in AE, you have to figure out what you're doing wrong in Fusion
Any way for you to post one frame on dropbox or something, so someone can help out?

edit:
crazy thought, but after importing the exr, did you check if the z pass is turned ON, in the "Format" tab in the Property manager or whatever it is called in Fusion.

So...
- import exr, check in the PM on the right/Format tab if the Z checkbox is on and set to 'viewlayer.Z.Z'.
- Add a 'CopyAux' and follow the instructions by Kruno at 10:07. See your Z pass show up.

I installed DRS today, but it's the same thing. To check, here's a screenshot of the file in AE. As you can see, it works.

Screenshot 2024-05-28 at 18.39.17.jpg (538.95 KiB) Viewed 626 times

I have uploaded an EXR (only one frame) to Google drive. It would be great if someone could try it and see if it works for them.
I already said that these are my beginnings in DRS and Fusion is still strange for me. So here's a question that might be ridiculous. Should the EXR be created in Fusion with a loader node, or can it also be done from the Media Pool via D&D as a Media node ?
I've tried both of course, but there's no difference, I can't get this grayscale gradient.

https://drive.google.com/file/d/1-ccN-w ... sp=sharing

I installed DRS today, but it's the same thing. To check, here's a screenshot of the file in AE. As you can see, it works. I have uploaded an EXR (only one frame) to Google drive. It would be great if someone could try it and see if it works for them.

I already said that these are my beginnings in DRS and Fusion is still strange for me. So here's a question that might be ridiculous. Should the EXR be created in Fusion with a loader node, or can it also be done from the Media Pool via D&D as a Media node ? I've tried both of course, but there's no difference, I can't get this grayscale gradient.

OK, so here is what I've found.

First of all, as far as I can see there is RGB and Alpha channels but no Z-Depth Channel. That is something related to export from 3D studio Max, not sure what.

Second its really bizarre numbers I've not seen before, but Black point was set to 220 and White point at 1. Which is way off the normal range I'm used to seeing where even with Z-Depth its not bellow -11 for me. In fact your image was so far outside of normal range, that copy Aux could not even detect the range automatically. But you can set it up manually.

In Resolve all I see is white, and some kind of Alpha channel information, but nothing usable. Than I opened it in Chaos Player (A professional image sequence player designed for fast and smooth playback) to double check and at first same problem. Than I lowered exposure slider by whopping nine stops or so, and I was able to find the missing gradient.

sshot-4037.jpg (100.54 KiB) Viewed 610 times

Than I went back to Fusion and as you can see from readout there is indeed some info but its at 220, something I'm not used to seeing.

sshot-4043.jpg (132.04 KiB) Viewed 610 times

Once I had the numbers of 220 or around about, and I could also see in your screenshot from Extractor in AF that its the same numbers, than I could manually enter them in CopyAux node to get the gradient.

You have to manually enter the numbers as you can see in my screenshot.

sshot-4047.jpg (84.14 KiB) Viewed 610 times

Why are numbers so off the beaten path I am not sure. Why is there no normalized gradient exported as RGB or Z-Depth channel, I don't know. But there you have it. At least you can now get the gradient you wanted from Fusion tools as well. I would look into 3D studio max and export options some more and check forums, is there some kind of bug, glitch, or setting or what that could be changed. If not, than at least you can use the method here.

P.S.
This works in free and studio version of fusion as I've mentioned, the mystery is in the file itself. You can use loader or media pool to bring in EXRs. There should be no issues.

If values are outside 0.0-1.0 range they are direct depth in scene units. Perfectly normal and most useful representation of depth there is. Any depth defocus filter worth their weight can work with direct depth.

First of all, thank you very much for the effort you made to figure it out, Smithy.

Now I can see a grayscale gradient for myself too.

The scaling is strangely completely different (0-1000000) , if I use Detect Range.

Screenshot 2024-05-29 153323.jpg (100.09 KiB) Viewed 427 times

And I had to invert it. Strange that it works differently for everyone.

I put a brightness node in between so it can be fine-tuned better.

Hendrik, you think this is normal, is this the right workflow to do it?

I'm going to do some tutorials on which lens effect I connect this node to create the DoF.

Many thanks to everyone involved

First of all, thank you very much for the effort you made to figure it out, Smithy.

Now I can see a grayscale gradient for myself too.

The scaling is strangely completely different (0-1000000) , if I use Detect Range.

Many thanks to everyone involved

No problem. Interesting challenge. Good luck with your future projects.

The scaling is strangely completely different (0-1000000) , if I use Detect Range.

This is because max values in your file are very large. It probably represents world sphere (areas where nothing was hit by rays) in Arnold, because setting depth to 0.0 in those areas would make very little sense and mess up defocus.

Hendrik, you think this is normal, is this the right workflow to do it?
I'm going to do some tutorials on which lens effect I connect this node to create the DoF.

Workflow depends on which defocus operator you are actually going to use. If it can use direct depth you don't need to normalize anything, it takes depth values as they are.