Jump to: Board index » General » Fusion

Convolve

Learn about 3D compositing, animation, broadcast design and VFX workflows.
  • Author
  • Message
Offline
User avatar

Unai Martínez

  • Posts: 9
  • Joined: Thu Apr 13, 2017 10:39 pm
  • Location: London

Convolve

PostThu Apr 13, 2017 11:10 pm

Hi! New here, as well as with Fusion :)

Coming from Nuke, there’s really only one thing I miss: being able to convolve an image with another, eg. to mimic more complex bokeh, or to apply the captured artefacts of a real lens on a CG image.

Sure the Defocus tool lets you tweak the lens model to be bladed and fade from the centre etc. but it still looks very artificial; and the Custom Filter only lets you convolve by kernels of up to 7 × 7 pixels, which is really tiny.

So, is there a tool that I’m missing, or a 3rd party Fuse or plugin for this?

Thanks!
Fake dinosaurs in real cities… and such.
Offline
User avatar

Bryan Ray

  • Posts: 2478
  • Joined: Mon Nov 28, 2016 5:32 am
  • Location: Los Angeles, CA, USA

Re: Convolve

PostFri Apr 14, 2017 6:40 am

I don't remember having seen one, but a convolve is simple enough that it would make a good first project for someone interested in learning how to make Fuses.

Fusion's Custom Filter has all kinds of problems, anyway. You can type in decimals, but it just rounds them off to integers. If you switch to the Common Controls tab and back to the main tab, it will reset to a 3x3 matrix. And, as you pointed out, it's limited to a very small kernel.
Bryan Ray
http://www.bryanray.name
http://www.sidefx.com
Offline
User avatar

Unai Martínez

  • Posts: 9
  • Joined: Thu Apr 13, 2017 10:39 pm
  • Location: London

Re: Convolve

PostFri Apr 14, 2017 8:07 am

Thanks Bryan. I wanted to do an FFT Fuse since it would be valuable for both convolutions and lens diffraction patterns. But there are two things putting me off:
  1. Lack of structured documentation. I started with the User, Tool and Scripting manuals in my installation, and had to jump over to a clone of the VFXPedia before BMD ditched it, but this one refers to eyeonScript instead of fusionScript. I know they’re the same in principle, but for example the manuals refer to some “optimised” functions like Blur() that are not documented anywhere else. Is this because they were added after VFXPedia, or because VFXPedia’s Fuse Tool Script Reference isn’t complete? Is there an official updated reference?
  2. If you look around it seems no-one has done it. This might be because it’s so simple it wasn’t worth posting, or because it’s so complex no-one’s managed to do it, and frankly, seeing the code of some more complex OpenCL Fuses scares me a bit.

EDIT: VFXPedia does explain everything I’ve stumbled upon so far.
Fake dinosaurs in real cities… and such.
Offline
User avatar

Unai Martínez

  • Posts: 9
  • Joined: Thu Apr 13, 2017 10:39 pm
  • Location: London

Re: Convolve

PostThu Apr 20, 2017 11:31 pm

Nevermind, I made it 8-)

Code: Select all
-- Convolve v1.1 by Unai Martínez Barredo - unaimb.com
FuRegisterClass('Convolve', CT_Tool, {
  REGS_Category = 'Blur',
  REGS_OpIconString = 'Cnv',
  REGS_OpDescription = 'OpenCL Convolve Fuse',
  REG_OpNoMask = true,
  REG_NoBlendCtrls = true,
  REG_NoObjMatCtrls = true,
  REG_NoMotionBlurCtrls = true
})
function Create()
  Input = self:AddInput('Input', 'Input', {
    LINKID_DataType = 'Image',
    LINK_Main = 1
  })
  Filter = self:AddInput('Filter', 'Filter', {
    LINKID_DataType = 'Image',
    LINK_Main = 2
  })
  Output = self:AddOutput('Output', 'Output', {
    LINKID_DataType = 'Image',
    LINK_Main = 1
  })
  Normal = self:AddInput('Normalise', 'Normalise', {
    LINKID_DataType = 'Number',
    INP_Integer = true,
    INPID_InputControl = 'CheckboxControl',
    INP_Default = 1
  })
end
local mgr
local clsource = [[
kernel void normFind(FuReadImage_t inpImg, FuWriteImage_t outMul,
                      int2 inpSiz){
  int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
  if(inpPos.x == 0 && inpPos.y == 0){
    float4 gdiv;
    for(int y = 0; y < inpSiz.y; y ++){
      for(int x = 0; x < inpSiz.x; x ++){
        int2 cPos = (int2)(x, y);
        gdiv += FuReadImagef(inpImg, cPos, inpSiz);
      }
    }
    FuWriteImagef(outMul, (int2)(0, 0), (int2)(1, 1), 1 / gdiv);
  }
}
kernel void normMult(FuReadImage_t inpImg, FuWriteImage_t outImg,
                     FuReadImage_t mulImg, int2 inpSiz){
  int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
  float4 mul = FuReadImagef(mulImg, (int2)(0, 0), (int2)(1, 1));
  FuWriteImagef(outImg, inpPos, inpSiz,
                FuReadImagef(inpImg, inpPos, inpSiz) * mul);
}
kernel void convolve(FuReadImage_t inpImg, FuReadImage_t filImg,
                     FuWriteImage_t outImg, int2 inpSiz, int2 filSiz){
  int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
  float4 outVal;
  for(int y = 0; y < filSiz.y; y ++){
    for(int x = 0; x < filSiz.x; x ++){
      int2 filPos = (int2)(x, y);
      float4 inpVal = FuReadImageCf(inpImg, inpPos - filPos + filSiz / 2,
                                    inpSiz);
      float4 filVal = FuReadImagef(filImg, filPos, filSiz);
      outVal = mad(inpVal, filVal, outVal);
    }
  }
  FuWriteImagef(outImg, inpPos, inpSiz, outVal);
}
]]
function OnAddToFlow()
  mgr = OCLManager()
  if mgr then
    prog = mgr:BuildCachedProgram('Convolve', debug.getinfo(1).source:sub(2),
                                  clsource)
  end
end
function OnRemoveFromFlow()
   prog = nil
   mgr = nil
end
function Process(req)
  local filter = Filter:GetValue(req)
  local filcl
  if prog then
    filcl = prog:CreateImage(filter, 'read')
  end
  local success = true
  local nrmcl
  local out
  if Normal:GetValue(req).Value > 0.5 then
    success = false
    local mul = Image({
      IMG_Width = 1,
      IMG_Height = 1,
      IMG_Depth = 8
    })
    local mulcl
    if prog and mul then
      mulcl = prog:CreateImage(mul, 'readwrite')
      if filcl and mulcl then
        local kernel = prog:CreateKernel('normFind')
        if kernel then
          prog:SetArg(kernel, 0, filcl)
          prog:SetArg(kernel, 1, mulcl)
          prog:SetArgInt(kernel, 2, filter.Width, filter.Height)
          success = prog:RunKernel(kernel)
        end
      end
    end
    if success then
      success = false
      local nrm = Image({IMG_Like = filter})
      if nrm then
        nrmcl = prog:CreateImage(nrm, 'readwrite')
        if nrmcl then
          local kernel = prog:CreateKernel('normMult')
          if kernel then
            prog:SetArg(kernel, 0, filcl)
            prog:SetArg(kernel, 1, nrmcl)
            prog:SetArg(kernel, 2, mulcl)
            prog:SetArgInt(kernel, 3, filter.Width, filter.Height)
            success = prog:RunKernel(kernel)
            if mulcl then
              mulcl:ReleaseCLObject()
            end
          end
        end
      end
    end
    if success then
      filcl = nrmcl
    end
  end
  if success then
    success = false
    local input = Input:GetValue(req)
    out = Image({IMG_Like = input})
    if out then
      local incl = prog:CreateImage(input, 'read')
      local outcl = prog:CreateImage(out, 'write')
      if filcl and incl and outcl then
        local kernel = prog:CreateKernel('convolve')
        if kernel then
          prog:SetArg(kernel, 0, incl)
          prog:SetArg(kernel, 1, filcl)
          prog:SetArg(kernel, 2, outcl)
          prog:SetArgInt(kernel, 3, input.Width, input.Height)
          prog:SetArgInt(kernel, 4, filter.Width, filter.Height)
          success = prog:RunKernel(kernel)
          if success then
            success = prog:Download(outcl, out)
          end
          if incl then
            incl:ReleaseCLObject()
          end
          if filcl then
            filcl:ReleaseCLObject()
          end
          if nrmcl then
            nrmcl:ReleaseCLObject()
          end
          if outcl then
            outcl:ReleaseCLObject()
          end
        end
      end
    end
  end
  if prog and not success then
    OnRemoveFromFlow()
    collectgarbage()
    OnAddToFlow()
  end
  Output:Set(req, out)
end
Fake dinosaurs in real cities… and such.
Offline
User avatar

Bryan Ray

  • Posts: 2478
  • Joined: Mon Nov 28, 2016 5:32 am
  • Location: Los Angeles, CA, USA

Re: Convolve

PostFri Apr 21, 2017 12:10 am

Very nice! You certainly learned OpenCL faster than I did! I haven't yet been successful in that arena.
Bryan Ray
http://www.bryanray.name
http://www.sidefx.com
Offline
User avatar

Bryan Ray

  • Posts: 2478
  • Joined: Mon Nov 28, 2016 5:32 am
  • Location: Los Angeles, CA, USA

Re: Convolve

PostFri Apr 21, 2017 1:00 am

Wow. That crashed my video driver! (And Fusion, too, of course.)
Bryan Ray
http://www.bryanray.name
http://www.sidefx.com
Offline
User avatar

Unai Martínez

  • Posts: 9
  • Joined: Thu Apr 13, 2017 10:39 pm
  • Location: London

Re: Convolve

PostFri Apr 21, 2017 8:14 am

Thanks for the kind words, and sorry I crashed your GPU.

…On that note, I’d appreciate any tips for optimising the OCL kernels :P
Attachments
Screen Shot 2017-04-21 at 09.22.18.png
Proof it works :)
Screen Shot 2017-04-21 at 09.22.18.png (725.61 KiB) Viewed 6157 times
Fake dinosaurs in real cities… and such.
Offline
User avatar

michael vorberg

  • Posts: 943
  • Joined: Wed Nov 12, 2014 8:47 pm
  • Location: stuttgart, germany

Re: Convolve

PostSat Apr 22, 2017 7:55 pm

great work!
Offline
User avatar

Bryan Ray

  • Posts: 2478
  • Joined: Mon Nov 28, 2016 5:32 am
  • Location: Los Angeles, CA, USA

Re: Convolve

PostSat Apr 22, 2017 8:56 pm

Previous test was in Fusion 7.7.1, where it crashed. Testing in Fusion 8.2.1b6 gives me interesting, but incorrect results:

Untitled.jpg
Untitled.jpg (226.55 KiB) Viewed 6100 times


It's definitely close to operational—changing the filter image changes the shape of the blooms.

I'm on Windows 7 with a Titan X.

I'm looking forward to the next time I have downtime at work so I can tear the fuse apart to see what makes it tick.
Bryan Ray
http://www.bryanray.name
http://www.sidefx.com
Offline
User avatar

Unai Martínez

  • Posts: 9
  • Joined: Thu Apr 13, 2017 10:39 pm
  • Location: London

Re: Convolve

PostSun Apr 23, 2017 10:29 am

That’s very interesting! In one of the iterations I tried using CreateBuffer instead of CreateImage and the output was jammed in a similar way.

Michael, I’m interested in knowing if it worked for you.
Fake dinosaurs in real cities… and such.
Offline
User avatar

Unai Martínez

  • Posts: 9
  • Joined: Thu Apr 13, 2017 10:39 pm
  • Location: London

Re: Convolve

PostTue Sep 05, 2017 4:35 pm

Hey!

So I was trying to get this working on Fusion 9, and it was erroring until I hand picked the GPU in the OpenCL settings (choosing Select in the Device checkboxes). I decided giving the CPU a go too, and to my surprise, it showed something similar to yours, Bryan.

I’ll try and fix that :)
Fake dinosaurs in real cities… and such.
Offline
User avatar

Unai Martínez

  • Posts: 9
  • Joined: Thu Apr 13, 2017 10:39 pm
  • Location: London

Re: Convolve

PostTue Sep 05, 2017 5:42 pm

Done. It was just a matter of initialising the float4s in the CL code. Here’s the updated version:

Code: Select all
-- Convolve v1.2 by Unai Martínez Barredo - unaimb.com
FuRegisterClass('Convolve', CT_Tool, {
  REGS_Category = 'Blur',
  REGS_OpIconString = 'Cnv',
  REGS_OpDescription = 'OpenCL Convolve Fuse',
  REG_OpNoMask = true,
  REG_NoBlendCtrls = true,
  REG_NoObjMatCtrls = true,
  REG_NoMotionBlurCtrls = true
})
function Create()
  Input = self:AddInput('Input', 'Input', {
    LINKID_DataType = 'Image',
    LINK_Main = 1
  })
  Filter = self:AddInput('Filter', 'Filter', {
    LINKID_DataType = 'Image',
    LINK_Main = 2
  })
  Output = self:AddOutput('Output', 'Output', {
    LINKID_DataType = 'Image',
    LINK_Main = 1
  })
  Normal = self:AddInput('Normalise', 'Normalise', {
    LINKID_DataType = 'Number',
    INP_Integer = true,
    INPID_InputControl = 'CheckboxControl',
    INP_Default = 1
  })
end
local mgr
local clsource = [[
kernel void normFind(FuReadImage_t inpImg, FuWriteImage_t outMul,
                      int2 inpSiz){
  int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
  if(inpPos.x == 0 && inpPos.y == 0){
    float4 gdiv = (float4)(0.0, 0.0, 0.0, 0.0);
    for(int y = 0; y < inpSiz.y; y ++){
      for(int x = 0; x < inpSiz.x; x ++){
        int2 cPos = (int2)(x, y);
        gdiv += FuReadImagef(inpImg, cPos, inpSiz);
      }
    }
    FuWriteImagef(outMul, (int2)(0, 0), (int2)(1, 1), 1 / gdiv);
  }
}
kernel void normMult(FuReadImage_t inpImg, FuWriteImage_t outImg,
                     FuReadImage_t mulImg, int2 inpSiz){
  int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
  float4 mul = FuReadImagef(mulImg, (int2)(0, 0), (int2)(1, 1));
  FuWriteImagef(outImg, inpPos, inpSiz,
                FuReadImagef(inpImg, inpPos, inpSiz) * mul);
}
kernel void convolve(FuReadImage_t inpImg, FuReadImage_t filImg,
                     FuWriteImage_t outImg, int2 inpSiz, int2 filSiz){
  int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
  float4 outVal = (float4)(0.0, 0.0, 0.0, 0.0);
  for(int y = 0; y < filSiz.y; y ++){
    for(int x = 0; x < filSiz.x; x ++){
      int2 filPos = (int2)(x, y);
      float4 inpVal = FuReadImageCf(inpImg, inpPos - filPos + filSiz / 2,
                                    inpSiz);
      float4 filVal = FuReadImagef(filImg, filPos, filSiz);
      outVal = mad(inpVal, filVal, outVal);
    }
  }
  FuWriteImagef(outImg, inpPos, inpSiz, outVal);
}
]]
function OnAddToFlow()
  mgr = OCLManager()
  if mgr then
    prog = mgr:BuildCachedProgram('Convolve', debug.getinfo(1).source:sub(2),
                                  clsource)
  end
end
function OnRemoveFromFlow()
   prog = nil
   mgr = nil
end
function Process(req)
  local filter = Filter:GetValue(req)
  local filcl
  if prog then
    filcl = prog:CreateImage(filter, 'read')
  end
  local success = true
  local nrmcl
  local out
  if Normal:GetValue(req).Value > 0.5 then
    success = false
    local mul = Image({
      IMG_Width = 1,
      IMG_Height = 1,
      IMG_Depth = 8
    })
    local mulcl
    if prog and mul then
      mulcl = prog:CreateImage(mul, 'readwrite')
      if filcl and mulcl then
        local kernel = prog:CreateKernel('normFind')
        if kernel then
          prog:SetArg(kernel, 0, filcl)
          prog:SetArg(kernel, 1, mulcl)
          prog:SetArgInt(kernel, 2, filter.Width, filter.Height)
          success = prog:RunKernel(kernel)
        end
      end
    end
    if success then
      success = false
      local nrm = Image({IMG_Like = filter})
      if nrm then
        nrmcl = prog:CreateImage(nrm, 'readwrite')
        if nrmcl then
          local kernel = prog:CreateKernel('normMult')
          if kernel then
            prog:SetArg(kernel, 0, filcl)
            prog:SetArg(kernel, 1, nrmcl)
            prog:SetArg(kernel, 2, mulcl)
            prog:SetArgInt(kernel, 3, filter.Width, filter.Height)
            success = prog:RunKernel(kernel)
            if mulcl then
              mulcl:ReleaseCLObject()
            end
          end
        end
      end
    end
    if success then
      filcl = nrmcl
    end
  end
  if success then
    success = false
    local input = Input:GetValue(req)
    out = Image({IMG_Like = input})
    if out then
      local incl = prog:CreateImage(input, 'read')
      local outcl = prog:CreateImage(out, 'write')
      if filcl and incl and outcl then
        local kernel = prog:CreateKernel('convolve')
        if kernel then
          prog:SetArg(kernel, 0, incl)
          prog:SetArg(kernel, 1, filcl)
          prog:SetArg(kernel, 2, outcl)
          prog:SetArgInt(kernel, 3, input.Width, input.Height)
          prog:SetArgInt(kernel, 4, filter.Width, filter.Height)
          success = prog:RunKernel(kernel)
          if success then
            success = prog:Download(outcl, out)
          end
          if incl then
            incl:ReleaseCLObject()
          end
          if filcl then
            filcl:ReleaseCLObject()
          end
          if nrmcl then
            nrmcl:ReleaseCLObject()
          end
          if outcl then
            outcl:ReleaseCLObject()
          end
        end
      end
    end
  end
  if prog and not success then
    OnRemoveFromFlow()
    collectgarbage()
    OnAddToFlow()
  end
  Output:Set(req, out)
end
Fake dinosaurs in real cities… and such.
Offline
User avatar

Tony Gallardo

  • Posts: 21
  • Joined: Tue Aug 23, 2016 7:04 pm
  • Location: San Antonio, Texas

Re: Convolve

PostTue Sep 05, 2017 6:19 pm

Hi Uni

I really love the idea of this...I tried to paste this into my flow, doesn't seem to want to paste. How should I go about "installing this in Fusion?"

on 2015MacBookPro, MacOS 10.11.6, Fusion Studio 9
Offline
User avatar

Unai Martínez

  • Posts: 9
  • Joined: Thu Apr 13, 2017 10:39 pm
  • Location: London

Re: Convolve

PostTue Sep 05, 2017 6:27 pm

Make a file called Convolve.fuse with the code in it. On macOS, place it inside ~/Library/Application Support/Blackmagic Design/Fusion/Fuses, restart Fusion and you’re ready to go!
Fake dinosaurs in real cities… and such.
Offline

john barclay

  • Posts: 8
  • Joined: Wed Nov 02, 2016 2:37 pm

Re: Convolve

PostWed Sep 06, 2017 10:30 am

Hey,

This is super cool. I'll need to take a look at your code soon as I'd like to see if it's altering the bokeh shape across the frame.

In other news,

Lenscare accepts an 'iris' that I think is used when convolving to create a custom bokeh shape. Both FL_outOfFocus and FL_depthOfField can have an iris.

http://www.frischluft.com/lenscare/

Sapphire suite has z_convolve as well.
Offline
User avatar

Unai Martínez

  • Posts: 9
  • Joined: Thu Apr 13, 2017 10:39 pm
  • Location: London

Re: Convolve

PostWed Sep 06, 2017 10:47 am

It’s super basic; the shape is always the same and there’s no way of changing it. I’d love to try and implement that, but I have to make it stable yet— if the images are too big, it crashes, instead of just taking very long. Not cool.
Fake dinosaurs in real cities… and such.

Return to Fusion

Who is online

Users browsing this forum: No registered users and 24 guests