Page 1 of 1

Have ugly artifacts when trying to retime clip with alpha

PostPosted: Wed Jan 15, 2020 10:37 am
by crantisz
Have ugly artifacts when trying to retime (decrease speed) clip with alpha. It has black and white dots around image:
Peek 2020-01-15 13-29.gif
Peek 2020-01-15 13-29.gif (778.83 KiB) Viewed 3831 times

Both frame blend and Optical flow have this issue.

With resize filter Bilinear loks better, but alpha channel jumps from frame to frame (ignores frame blend and Optical flow setting)
Peek 2020-01-15 13-35.gif
Peek 2020-01-15 13-35.gif (852.95 KiB) Viewed 3831 times

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Tue Sep 22, 2020 9:33 pm
by Dennis Fassbaender
I have the same issue using motionblur on a PNG-Image-Sequence with alpha-channel.
Is there any workaround to solve this?

Image

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Wed Sep 23, 2020 4:43 pm
by Jim Simon
This kind of processing is done on the GPU, so...list out those specs. Hardware, driver.

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Wed Sep 23, 2020 5:55 pm
by TheBloke
Almost certain this is not any hardware issue. It relates to the alpha channel and pre-multiplication.

Here's an example on the Color page with a PNG image sequence, set to Alpha = Premultiplied and then motion blur applied. The motion blur hides it a bit, but there's white artifacts all over the rim of the fireball:

Image

Here's the same clip, now inside Fusion via MediaIn node, and still with Clip Attributes -> Alpha = Premultiplied:
Image

And here it is in Fusion via MediaIn, with Clip Attributes -> Alpha = Straight:
Image

Again in Fusion, this time loaded by a Loader node (bypassing the Media Pool and Clip Attributes):
Image

The result here is identical to MediaIn with Clip Attributes = Straight.

Again in Fusion and Loader node, set to Post-Multiply By Alpha:
Image

This has been multiplied without any of those weird artifacts.

Finally, again in Fusion with Loader node, but this time passed to an Alpha Divide node:
Image

Image

It's not exactly the same, but it's quite similar to the MediaIn result with Alpha = Pre-Multiplied. Making me think that Resolve is dividing by the alpha when set to Alpha = Pre-Multiplied? Maybe that's what pre-multiplied in Clip Attributes is meaning? I thought it meant it wouldn't multiply by the alpha again, because this had already happened.

In the example VFX clips I'm using (a fireball) I'm pretty sure it is already pre-multiplied, however setting that value in Resolve gives these weird artifacts. I don't know if this is a bug, or if I'm just misunderstanding what this option does. The manual doesn't explain what Resolve will do with the Pre-Multiplied vs Straight setting, it just says you can choose "the type of alpha (Premultiplied or Straight)".

Anyway, assuming the OP has Alpha = Pre-Multiplied set, the issue can likely be resolved simply by setting Alpha to Straight on the clip(s) in question.

TLDR: Try setting the Clip Attributes to Alpha = Straight and see if that fixes the Color page glitches? If not, can you upload one PNG frame that exhibits this issue in Color?

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Sat Sep 26, 2020 2:04 pm
by Dennis Fassbaender
No matter what i try - for me its not cleat what is the correct workflow in Resolve when it comes to use Clips with alphachannel + motion blur.

Here is a screencast me trying to do something (recorded on an ultrawidescreen,sorry):
https://www.dropbox.com/s/vtgwpknya818i ... a.mp4?dl=0

And here is a 100frames apples-image-sequence if you would like to check my PNG´s and try yourself:
https://www.dropbox.com/s/hgbqpjmxzua34 ... a.zip?dl=1

Let me know what you think about this.

Using: GTX1080ti
Driver: NVIDIA / 29.01.2020 / 26.21.14.4219
OS: Windows 10 64BIT / 32GB RAM

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Sat Sep 26, 2020 2:16 pm
by TheBloke
I can't re-create the problem using just Motion Blur. But I see you've got some other Color effects on there. What are the other corrections you've applied in node 01?

Also, you're using an external matte? In your screen capture, what is the second image input coming from? The one you've connected to the alpha input of Corrector 1?
Image

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 8:32 am
by Dennis Fassbaender
Thank you for looking into this!

This was from testing.

Here is the original node-setup:
Image

And here are some interesting results from different settings for "Retime And Scaling":
Image
Image
Image
Image
Image
Image

Nearest + Bilinear seems to be the only working solution - at the lowest level of quality.
And as this also happens when re-timing clips with alpha channel, "nearest" will not be my choice.

When nobody else can re-produce this - does that mean i have a hardware defect?

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 9:29 am
by TheBloke
It's not a GPU issue. I see the same now. It appears to be a combination of scaling + Motion Blur (and maybe other Color effects too).

Are you placing this sequence on a UHD timeline, or some timeline bigger than 1080p such that it gets auto scaled? Or are you scaling it yourself using the timeline scaling settings?

When I place it on a 1080p timeline at 1x zoom, I don't see artefacts. If I set zoom to 2x I do then see artefacts when Motion Blur is applied. Likewise if I place it on a UHD timeline (meaning the 1080p sequence is being scaled 2x), I do see artefacts once Motion Blur is applied:
Image

That's with Resize Filter = Sharper.

However, I can make them go away by using the Super Scale system. In Clip Attributes, set Super Scale to 2x:
Image

It now looks like this when scaled 2x and Motion Blur applied:
Image

So, that may be the simple solution (or workaround): use Super Scale.

Now, as to why this happens in the first place, I am not sure. Is it an inevitable result of scaling up an image with an alpha? No, surely it isn't. If I move over into Fusion using a Loader node, then scale the image sequence 2x using a Transform node, I see no problems at all - including with Motion Blur added:
Image

I don't know what's going on exactly. It feels to me like there's some problem with Color's alpha processing when scaling is applied, because I don't understand why those terrible artefacts occur with the combination of scaling and Motion Blur (and I suspect other Color effects would also trigger it). But I can't say for certain as it's beyond my experience.

Anyway, at least in my testing, using Super Scale fixes it. So that may be enough to get you going.

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 9:49 am
by TheBloke
Here's an interesting thing, which is further evidence that the problem - or misconfiguration/our misunderstanding - relates to sizing that happens before Color:

1. Put the 1080p PNG sequence on a UHD timeline.

2. On the clip on the timeline, in the Inspector set Retime & Scaling -> Scaling: to Crop. This will display it at 1080p resolution, ie not scaled up.

3. Go to Color, apply the Motion Blur.

4. Currently there are no artefacts, because no scaling is occurring
Image

5. On Color, go to Input Sizing and set Zoom to 2.0. The artefacts re-appear, as expected - this is exactly like scaling on the Timeline
Image

6. Set Input Sizing -> Zoom to 1.0 again. Now change to Output Sizing and set Zoom to 2.0. Boom, no artefacts!
Image

So first of all, this gives us a second solution/workaround: you could not use Super Scale, and instead do the upscaling using Colour -> Output Sizing.

Secondly, why does this work? It suggests to me that when scaling is applied to footage with an alpha channel before it reaches Color, artefacts occur when effects alter the alpha channel, ie Motion Blur. However, when Color is able to apply those effects first, then scale up the result, no problems occur.

Is this a bug? I really don't know. If it is a bug it seems like a serious one, that I'd think someone would have noticed and talked about already. So I'm still wondering if we're doing something wrong here, or misunderstanding something. Or maybe this source image sequence is unusual. Or maybe it's related to PNG. I'm not sure.

More testing is probably required, unless a Color page expert can chip in.

In the meantime you should have two solutions / workarounds:
1. Enable Super Scale in Clip Attributes

or

2. Don't enable Super Scale, and instead don't scale on the Timeline (Rescale = Crop) and do your scaling using Color -> Output Sizing, which is applied after your Color page effects have manipulated the clip.

EDIT: I've just realised that Output Scaling applies to every clip on the timeline. So this is only a workaround if every clip can be scaled the same way. If you need to scale different clips differently, or some not at all, using Output Sizing probably won't be usable as a workaround for the alpha issue.

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 10:53 am
by Hendrik Proosa
I think it is a problem with double unpremult somewhere in processing chain, I can produce similar ugly artifacts when doing double unpremult in Nuke. Scaling filter has little effect on this.
double_unpremult.png
double_unpremult.png (45.14 KiB) Viewed 3541 times


Expected result is this:
mblur_expected.png
mblur_expected.png (40.44 KiB) Viewed 3541 times


There is nothing wrong with the png sequence itself.

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 11:06 am
by TheBloke
Yeah I think you're right. Maybe the alpha divide (I think that's what an 'unpremult' is?) is happening both during scaling, and again by the Color effect(s), instead of only once?

Maybe the issue doesn't occur with Super Size because it uses different code, which is processing the alpha correctly.

If that is the issue and it is a bug, I'm just surprised that no-one else is apparently seeing this. So perhaps there's another contributing factor.

I found another workaround on my system: I have BorisFX Continuum installed, which includes a PreMult OFX. Adding that in the Color chain also the fixes the issue, and works with the 2x scaled image, and doesn't require Super Scale:
Image

This made me think I could just do it in Fusion, before the Color page sees the clip:
Image

But this didn't work at all; in fact it made the issue much worse:
Image

So I don't quite understand what's happening in the image pipeline here.

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 11:10 am
by Hendrik Proosa
Alpha handling is a proper mess in Resolve, people have been seeing its different effects for ages, but there is no acknowledgement of this from BMD side. I think your suggestion two unpremults during scaling and during color again could be it.

Alpha divide is the same as unpremultiplication, Fusion has a history of using its own inventions for widely used namings.

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 12:58 pm
by Dennis Fassbaender
Fantastic ideas! Thank you :)
However - i´m still using a FHD timeline.
And the PNG-Sequence will be zoomed-OUT to fit better into the footage.

Sometimes i´m using 4k-vfx-footage in a full-hd timeline. So, a workaround for this situation would be required (for me).

1. This means, SUPERSCALE does not help - as it only optimize upscaling.
2. Using output from the color-page is a great idea, too. BUT - i seems to work on the entire timeline and not on one clip only.

To try it your self just use a FULL-HD timeline and set the PNG´s zoom to 0,8 as example.

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 1:08 pm
by TheBloke
OK, here's what you do:

1. On Edit, reset the Transform Zoom controls to 1x.

2. Go to the Fusion page on the clip.

3. Add a single Transform node between MediaIn and MediaOut, and set its Size to 0.8. The nodes will look like this:
Image

4. Go to Color and do your Motion Blur and whatever other effects. You should not see any artefacts, because you are no longer using Timeline scaling or Input scaling:
Image

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 1:17 pm
by Dennis Fassbaender
I would like to try this workaround!
But i don´t know how to get this PNG´s into fusion.
This is how my timeline looks:

Image

Also when left-clicking and select "new fusion clip" - on the fusion clip the upper clip from my timeline will be visible.

However- maybe this is where the resizing issue comes from:
Image

I used "node-resizing" here - and as you can see: The alpha channel is ignored from resizing.
Looks like a bug. Or is there a way to resize alpha, too?

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 1:19 pm
by TheBloke
OK, go back to where you started - undo anything you've done eg making a Fusion Clip.

All you do is right-click on ApfelFilegen_...png in the Timeline, then choose "Open In Fusion page".
Image

You don't want a Fusion Clip or a Fusion Composition. You want what's called a "single-clip composition", which I call "going into Fusion on the clip". It's a composition placed directly on the clip, which sees that clip via MediaIn1.

If your clip was on the top layer of the timeline, you could simply put your playhead over the clip and then click the Fusion page icon at the bottom. However as it's on a lower layer, you need to right-click and choose "Open In Fusion Page".

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 1:41 pm
by Dennis Fassbaender
This workaround does work!
Thank you for helping me and explaining this!

Should i post a new topic named "alpha channel will not be resized (bug)?
Looks like this could be the bug...

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 1:50 pm
by TheBloke
I think I've also got a method working that uses Node sizing, and doesn't use Fusion:

Image

> Right-click on the node layout and choose "Add Alpha Output"

> In the Corrector node, set Motion Blur, and Node Sizing Zoom = 0.5 (or whatever)

> Add another Corrector node, and drag the source input (green on the left) into both its green and blue inputs.

> In the second Corrector node, also set Node Sizing = 0.5

> From the second Corrector node, drag the blue output into the blue Alpha output on the right-hand side of the flow.

Image

Why does this work? Well, the first Corrector is resizing the image to 0.5 and applying the Motion Blur. However it is not resizing the alpha channel. We can't pass the alpha channel into the first Corrector, because that would cause it to be used as a key, and then it doesn't seem to apply the Motion Blur correctly.

So we create a second Corrector node whose only purpose is to take in the Alpha, resize it to 0.5, and then pass it to the Alpha Output so Edit sees it.

At least I think I'm understanding that process correctly. Certainly the output seems to work, and is comparable to the Fusion method.

Personally I'd use the Fusion method as it's less hassle, but this is an alternative.

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 2:02 pm
by TheBloke
Dennis Fassbaender wrote:Should i post a new topic named "alpha channel will not be resized (bug)?
Looks like this could be the bug...
Well that's not the bug, the alpha channel is being resized - otherwise the image would look totally weird.

The problem is something to do with the alpha handling of the Timeline and Input sizing, at least as relates to the Color page, like Hendrik and I discussed. Possibly there is an extra alpha division step occurring that should not be happening.

Re: Have ugly artifacts when trying to retime clip with alph

PostPosted: Mon Sep 28, 2020 3:23 pm
by Dennis Fassbaender
Thats great!
Thank you so much for helping here :)