Page 1 of 1

Deconvolution sharpening

PostPosted: Sat Feb 17, 2018 9:39 pm
by Umberto Uderzo
Is there on the market any plugin that can be used in Fusion for deconvolution sharpening?

Re: Deconvolution sharpening

PostPosted: Sun Feb 18, 2018 3:02 am
by Bryan Ray
Nothing I'm aware of. I did some poking around to see what's out there. I figure if anyone had one, it would be either Neat Video or Re:Vision, but neither of them do. Neat's developers said back in 2003 that they were looking into it but that it would probably warrant an entirely new product rather than being a feature in Neat. There weren't any other mentions of the topic on their forums since then, other than a few people putting it on wish lists.

The Foundry's Furnace plug-in has a deconvolution tool that handles both motion blur and simple out of focus images, but they stopped selling plug-ins for any host program other than their own in an attempt to scuttle competing compositors. If you happen to have a license for Furnace 4 OFX lying around, it might work.

Re: Deconvolution sharpening

PostPosted: Mon Feb 19, 2018 8:52 am
by Umberto Uderzo
That's sad, indeed.

I was experimenting on some DJI Spark footage, as i wanted to eliminate the original footage sharpening (which is not user configurable for this quadcopter model) and i seen that applying a simple unsharp mask over the already sharpened footage, then subtracting this from the original footage gives me a good difference image that can eliminate on a good degree the original artefacts and give a more neutral image which, hopefully, could be worked with a more capable sharpening algorithm.

If i could find one, obviously :)

P.S. by the way, i couldn't understand why i need to apply the difference image with a 50% blend to avoid an over blurring. I see that the result it quite nice with 50% blur but don't understand the reason. I tried also with a simple ellipse built into Fusion then artificially sharpened and processed to eliminate the sharpen and i came to the same consideration, so it's not due to bad guessing of the footage sharpen parameters (which i see is a 1.0 radius and 1.0 gain)

Re: Deconvolution sharpening

PostPosted: Mon Feb 19, 2018 4:52 pm
by Bryan Ray
Unsharp masking is going to find all of the details in the image, with brightness in the mask corresponding to the level of contrast between two given pixels. After the sharpen, those original details are still there, so running a second unsharp mask will find them again, and the mask will be even stronger. By simply subtracting it, you're not only reducing the sharpen but also the details that originally contributed to it. By using a 50% blend, you're essentially reducing the gain on the mask by 50%, limiting its power to change those original details.

I hope that makes some sense.

You could try Neat Video's sharpen. It won't help with motion blur, obviously, but it does at least do its sharpening in the frequency domain, so it will probably be better than the less sophisticated tools currently available in Fusion. I think they have a demo version with limited resolution and watermarking so you could at least see if it meets your needs before making a purchase.

Re: Deconvolution sharpening

PostPosted: Mon Feb 19, 2018 5:13 pm
by Umberto Uderzo
I think this makes sense... this means that it's not possible to eliminate only the unsharp mask effect completely, but only at 50%?

I must have been be fooled by my test case:

Code: Select all
{
   Tools = ordered() {
      ChannelBooleans2 = ChannelBoolean {
         CtrlWZoom = false,
         Inputs = {
            Blend = Input { Value = 0.5, },
            Operation = Input { Value = 1, },
            Background = Input {
               SourceOp = "PipeRouter1",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "ChannelBooleans1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 1045, 313.5 } },
      },
      ChannelBooleans1 = ChannelBoolean {
         Inputs = {
            Operation = Input { Value = 2, },
            Background = Input {
               SourceOp = "PipeRouter1",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "UnsharpMask2",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 935, 247.5 } },
      },
      UnsharpMask2 = UnsharpMask {
         Inputs = {
            Input = Input {
               SourceOp = "PipeRouter1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 825, 181.5 } },
      },
      Note2 = Note {
         Inputs = {
            Comments = Input { Value = "This is the \"desharpening\" procedure. Another unsharp mask is added (+1) over the original one (+1), then the result is subtracted (-2) from the original image. I expect the original footage unsharpened but that's not...", }
         },
         ViewInfo = StickyNoteInfo {
            Pos = { 660, 82.5 },
            Flags = {
               Expanded = true
            },
            Size = { 484, 76.3 }
         },
      },
      PipeRouter1 = PipeRouter {
         Inputs = {
            Input = Input {
               SourceOp = "UnsharpMask1",
               Source = "Output",
            },
         },
         ViewInfo = PipeRouterInfo { Pos = { 660, 247.5 } },
      },
      UnsharpMask1 = UnsharpMask {
         Inputs = {
            Input = Input {
               SourceOp = "Merge1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 385, 247.5 } },
      },
      Note1 = Note {
         Inputs = {
            Comments = Input { Value = "This is the input footage, in float32 format (already sharpened)", }
         },
         ViewInfo = StickyNoteInfo {
            Pos = { 165, 106.5 },
            Flags = {
               Expanded = true
            },
            Size = { 320, 57.3 }
         },
      },
      Merge1 = Merge {
         Inputs = {
            Background = Input {
               SourceOp = "Background2",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "Background1",
               Source = "Output",
            },
            PerformDepthMerge = Input { Value = 0, },
         },
         ViewInfo = OperatorInfo { Pos = { 275, 247.5 } },
      },
      Background2 = Background {
         Inputs = {
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            Depth = Input { Value = 4, },
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
            TopLeftRed = Input { Value = 0.501960784313725, },
            TopLeftGreen = Input { Value = 0.501960784313725, },
            TopLeftBlue = Input { Value = 0.501960784313725, },
         },
         ViewInfo = OperatorInfo { Pos = { 165, 247.5 } },
      },
      Background1 = Background {
         Inputs = {
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            Depth = Input { Value = 4, },
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
            TopLeftRed = Input { Value = 0.784313725490196, },
            TopLeftGreen = Input { Value = 0.784313725490196, },
            TopLeftBlue = Input { Value = 0.784313725490196, },
            EffectMask = Input {
               SourceOp = "Ellipse1",
               Source = "Mask",
            }
         },
         ViewInfo = OperatorInfo { Pos = { 165, 214.5 } },
      },
      Ellipse1 = EllipseMask {
         Inputs = {
            MaskWidth = Input { Value = 1920, },
            MaskHeight = Input { Value = 1080, },
            PixelAspect = Input { Value = { 1, 1 }, },
            ClippingMode = Input { Value = FuID { "None" }, },
         },
         ViewInfo = OperatorInfo { Pos = { 165, 181.5 } },
      }
   }
}


In which i see a fully "desharpened" output, but on real footage i still see a little residual halo.

Thanks for Nead Video hint, i'll give it a go!

Re: Deconvolution sharpening

PostPosted: Tue Feb 20, 2018 5:39 am
by Bryan Ray
Once any image filtration has occurred, it cannot be 100% undone. There will always be some degradation. You might get slightly better results from building the unsharp mask yourself instead of relying on the tool:

Code: Select all
{
   Tools = ordered() {
      PipeRouter2 = PipeRouter {
         Inputs = {
            Input = Input {
               SourceOp = "UnsharpMask1",
               Source = "Output",
            },
         },
         ViewInfo = PipeRouterInfo { Pos = { -214.502, 69.9991 } },
      },
      Blur1 = Blur {
         Inputs = {
            XBlurSize = Input { Value = 2.17021276595745, },
            Input = Input {
               SourceOp = "PipeRouter2",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { -35.9933, 31.7478 } },
      },
      ChannelBooleans3 = ChannelBoolean {
         Inputs = {
            Operation = Input { Value = 2, },
            Background = Input {
               SourceOp = "PipeRouter2",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "Blur1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { -34.4948, 65.2499 } },
      },
      Note3 = Note {
         Inputs = {
            Comments = Input { Value = "This creates the detail matte—step 1 of an unsharp mask procedure. Usually this information is added to the original image, but we instead subtract it in an attempt to undo the sharpening.", }
         },
         ViewInfo = StickyNoteInfo {
            Pos = { -63.7483, 96.9998 },
            Flags = {
               Expanded = true
            },
            Size = { 144.999, 122.299 }
         },
      },
      ChannelBooleans4 = ChannelBoolean {
         CtrlWZoom = false,
         Inputs = {
            Blend = Input { Value = 0.5, },
            Operation = Input { Value = 2, },
            ToAlpha = Input { Value = 4, },
            Background = Input {
               SourceOp = "PipeRouter2",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "ChannelBooleans3",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { 144.007, 123.25 } },
      },
      Note1 = Note {
         Inputs = {
            Comments = Input { Value = "This is the input footage, in float32 format (already sharpened)", }
         },
         ViewInfo = StickyNoteInfo {
            Pos = { -690, -160 },
            Flags = {
               Expanded = true
            },
            Size = { 320, 57.3 }
         },
      },
      Ellipse1 = EllipseMask {
         Inputs = {
            MaskWidth = Input { Value = 1920, },
            MaskHeight = Input { Value = 1080, },
            PixelAspect = Input { Value = { 1, 1 }, },
            ClippingMode = Input { Value = FuID { "None" }, },
         },
         ViewInfo = OperatorInfo { Pos = { -690, -85 } },
      },
      Background1 = Background {
         Inputs = {
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            Depth = Input { Value = 4, },
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
            TopLeftRed = Input { Value = 0.784313725490196, },
            TopLeftGreen = Input { Value = 0.784313725490196, },
            TopLeftBlue = Input { Value = 0.784313725490196, },
            Gradient = Input {
               Value = Gradient {
                  Colors = {
                     [0] = { 0, 0, 0, 1 },
                     [1] = { 1, 1, 1, 1 }
                  }
               },
            },
            EffectMask = Input {
               SourceOp = "Ellipse1",
               Source = "Mask",
            }
         },
         ViewInfo = OperatorInfo { Pos = { -690, -52 } },
      },
      Merge1 = Merge {
         Inputs = {
            Background = Input {
               SourceOp = "Background2",
               Source = "Output",
            },
            Foreground = Input {
               SourceOp = "Background1",
               Source = "Output",
            },
            PerformDepthMerge = Input { Value = 0, },
         },
         ViewInfo = OperatorInfo { Pos = { -580, -19 } },
      },
      Background2 = Background {
         Inputs = {
            Width = Input { Value = 1920, },
            Height = Input { Value = 1080, },
            Depth = Input { Value = 4, },
            ["Gamut.SLogVersion"] = Input { Value = FuID { "SLog2" }, },
            TopLeftRed = Input { Value = 0.501960784313725, },
            TopLeftGreen = Input { Value = 0.501960784313725, },
            TopLeftBlue = Input { Value = 0.501960784313725, },
            Gradient = Input {
               Value = Gradient {
                  Colors = {
                     [0] = { 0, 0, 0, 1 },
                     [1] = { 1, 1, 1, 1 }
                  }
               },
            },
         },
         ViewInfo = OperatorInfo { Pos = { -690, -19 } },
      },
      UnsharpMask1 = UnsharpMask {
         Inputs = {
            Input = Input {
               SourceOp = "Merge1",
               Source = "Output",
            },
         },
         ViewInfo = OperatorInfo { Pos = { -470, -19 } },
      }
   },
   ActiveTool = "ChannelBooleans4"
}



As you can see, a blurred version of the image is subtracted from the original, leaving only detail that is higher in frequency than the blur radius. And, of course, it's important to perform the operation in floating point (as you have done) in order to make use of the negative values—in an integer image, they'd be clamped to 0.

Re: Deconvolution sharpening

PostPosted: Tue Feb 20, 2018 10:06 am
by Umberto Uderzo
Thank you Bryan, very informative!

Looking at the Wikipedia page on Unsharp Masking:

https://en.wikipedia.org/wiki/Unsharp_masking

I see that the blur is scaled before being applied to the original image.
Is it this the reason that leads to a 50% blur on the final node for a proper result?

Re: Deconvolution sharpening

PostPosted: Tue Feb 20, 2018 5:30 pm
by Bryan Ray
… Kind of. As I said, the reason for the 50% application is because the sharpening filter is being applied twice. The mask's strength is relative to the slope of the gradient between two pixels' values. The first round of sharpening, done by the camera, increases contrast in the edges, therefore increasing that slope. The second round of sharpening, which you're performing, is using that exaggerated slope to build its mask—it's twice as dense as it was during the original sharpen operation. You therefore apply it with half as much strength in order to return to zero. Or as much zero as you can get in these circumstances.

The scaling performed on the blurred image is equivalent to the Gain slider in the USM node. I think. You might be able to get the same results from reducing Gain on your second unsharp mask by 50% instead of using the Blend on the Subtract—it should amount to the same result. There may be some hidden information in the node that I'm not aware of, though, so it may or may not work the same way mathematically.

Re: Deconvolution sharpening

PostPosted: Tue Feb 20, 2018 9:11 pm
by Umberto Uderzo
Now this makes sense to me!
Thank you Bryan for your time in explanations, much appreciated!

Re: Deconvolution sharpening

PostPosted: Wed Feb 21, 2018 2:34 am
by Bryan Ray
Great; glad it helped!