NVidia hw encoder corruption when using magic mask

When I try to use the HW encoder to produce either h264 or h265, if the source uses magic mask, then the output is corrupted / garbage.

In the output file, all is good in the title, then as soon as it switches to the clip with magic mask, garbage.
Output to DNX / GrassValley / ... work just fine.

Davinci Resolve Studio 17.1 running on linux with RTX2070 Super

I can't reproduce this in Studio 17.1 for Windows using an 8GB 1070. The render comes out fine.

Could be specific to Linux, your media, your GPU driver, your precise Delivery settings...

Mmm ... I'll try to make a minimal reproducing project. I'm using the magic mask to create an alpha output to isolate a subject to overlay it over another clip. That might matter.
Could also have an interaction with hw decode of h265 source clips.

ATM machine is busy rendering so can't test things.

Drivers are 460.39.
Delivery settings were defaults.

That might matter.

It could. Alpha wasn't part of my test.

Turns out it has nothing to do with the magic mask ... still trying to narrow it down.

ATM I need a fusion comp clip (just background + png logo) followed by two h264 video overlaid atop each other (crop and resized so part of them are "off screen") and when switching from fusion comp title (which renders fine) to the two overlaid video it bugs. But if I remove the fusion comp, then it works fine.

Just to get the work done, you might try skipping Fusion and adding the Background and PNG to the timeline directly.

Well the actual composition in the main project is more complex, this was just to reproduce the bug in a minimal config.

For the actual project I got it rendered in DNX and then used ffmpeg to convert to h264.

But I can reproduce the bug without too much trouble so I'd like to narrow it down to get it fixed because not being able to render to h264 or h265 is a major pain :/ (And being on linux, the nvidia hw encoder is the only option, no software encoder available)

Got it.

But that might lead us back to the OS or driver as a root cause here.