Page 1 of 1
Convolve
Posted:
Thu Apr 13, 2017 11:10 pm
by Unai Martínez
Hi! New here, as well as with Fusion
Coming from Nuke, there’s really only one thing I miss: being able to convolve an image with another, eg. to mimic more complex bokeh, or to apply the captured artefacts of a real lens on a CG image.
Sure the Defocus tool lets you tweak the lens model to be bladed and fade from the centre etc. but it still looks very artificial; and the Custom Filter only lets you convolve by kernels of up to 7 × 7 pixels, which is really tiny.
So, is there a tool that I’m missing, or a 3rd party Fuse or plugin for this?
Thanks!
Re: Convolve
Posted:
Fri Apr 14, 2017 6:40 am
by Bryan Ray
I don't remember having seen one, but a convolve is simple enough that it would make a good first project for someone interested in learning how to make Fuses.
Fusion's Custom Filter has all kinds of problems, anyway. You can type in decimals, but it just rounds them off to integers. If you switch to the Common Controls tab and back to the main tab, it will reset to a 3x3 matrix. And, as you pointed out, it's limited to a very small kernel.
Re: Convolve
Posted:
Fri Apr 14, 2017 8:07 am
by Unai Martínez
Thanks Bryan. I wanted to do an FFT Fuse since it would be valuable for both convolutions and lens diffraction patterns. But there are two things putting me off:
- Lack of structured documentation. I started with the User, Tool and Scripting manuals in my installation, and had to jump over to a clone of the VFXPedia before BMD ditched it, but this one refers to eyeonScript instead of fusionScript. I know they’re the same in principle, but for example the manuals refer to some “optimised” functions like Blur() that are not documented anywhere else. Is this because they were added after VFXPedia, or because VFXPedia’s Fuse Tool Script Reference isn’t complete? Is there an official updated reference?
- If you look around it seems no-one has done it. This might be because it’s so simple it wasn’t worth posting, or because it’s so complex no-one’s managed to do it, and frankly, seeing the code of some more complex OpenCL Fuses scares me a bit.
EDIT: VFXPedia
does explain everything I’ve stumbled upon so far.
Re: Convolve
Posted:
Thu Apr 20, 2017 11:31 pm
by Unai Martínez
Nevermind, I made it
- Code: Select all
-- Convolve v1.1 by Unai Martínez Barredo - unaimb.com
FuRegisterClass('Convolve', CT_Tool, {
REGS_Category = 'Blur',
REGS_OpIconString = 'Cnv',
REGS_OpDescription = 'OpenCL Convolve Fuse',
REG_OpNoMask = true,
REG_NoBlendCtrls = true,
REG_NoObjMatCtrls = true,
REG_NoMotionBlurCtrls = true
})
function Create()
Input = self:AddInput('Input', 'Input', {
LINKID_DataType = 'Image',
LINK_Main = 1
})
Filter = self:AddInput('Filter', 'Filter', {
LINKID_DataType = 'Image',
LINK_Main = 2
})
Output = self:AddOutput('Output', 'Output', {
LINKID_DataType = 'Image',
LINK_Main = 1
})
Normal = self:AddInput('Normalise', 'Normalise', {
LINKID_DataType = 'Number',
INP_Integer = true,
INPID_InputControl = 'CheckboxControl',
INP_Default = 1
})
end
local mgr
local clsource = [[
kernel void normFind(FuReadImage_t inpImg, FuWriteImage_t outMul,
int2 inpSiz){
int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
if(inpPos.x == 0 && inpPos.y == 0){
float4 gdiv;
for(int y = 0; y < inpSiz.y; y ++){
for(int x = 0; x < inpSiz.x; x ++){
int2 cPos = (int2)(x, y);
gdiv += FuReadImagef(inpImg, cPos, inpSiz);
}
}
FuWriteImagef(outMul, (int2)(0, 0), (int2)(1, 1), 1 / gdiv);
}
}
kernel void normMult(FuReadImage_t inpImg, FuWriteImage_t outImg,
FuReadImage_t mulImg, int2 inpSiz){
int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
float4 mul = FuReadImagef(mulImg, (int2)(0, 0), (int2)(1, 1));
FuWriteImagef(outImg, inpPos, inpSiz,
FuReadImagef(inpImg, inpPos, inpSiz) * mul);
}
kernel void convolve(FuReadImage_t inpImg, FuReadImage_t filImg,
FuWriteImage_t outImg, int2 inpSiz, int2 filSiz){
int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
float4 outVal;
for(int y = 0; y < filSiz.y; y ++){
for(int x = 0; x < filSiz.x; x ++){
int2 filPos = (int2)(x, y);
float4 inpVal = FuReadImageCf(inpImg, inpPos - filPos + filSiz / 2,
inpSiz);
float4 filVal = FuReadImagef(filImg, filPos, filSiz);
outVal = mad(inpVal, filVal, outVal);
}
}
FuWriteImagef(outImg, inpPos, inpSiz, outVal);
}
]]
function OnAddToFlow()
mgr = OCLManager()
if mgr then
prog = mgr:BuildCachedProgram('Convolve', debug.getinfo(1).source:sub(2),
clsource)
end
end
function OnRemoveFromFlow()
prog = nil
mgr = nil
end
function Process(req)
local filter = Filter:GetValue(req)
local filcl
if prog then
filcl = prog:CreateImage(filter, 'read')
end
local success = true
local nrmcl
local out
if Normal:GetValue(req).Value > 0.5 then
success = false
local mul = Image({
IMG_Width = 1,
IMG_Height = 1,
IMG_Depth = 8
})
local mulcl
if prog and mul then
mulcl = prog:CreateImage(mul, 'readwrite')
if filcl and mulcl then
local kernel = prog:CreateKernel('normFind')
if kernel then
prog:SetArg(kernel, 0, filcl)
prog:SetArg(kernel, 1, mulcl)
prog:SetArgInt(kernel, 2, filter.Width, filter.Height)
success = prog:RunKernel(kernel)
end
end
end
if success then
success = false
local nrm = Image({IMG_Like = filter})
if nrm then
nrmcl = prog:CreateImage(nrm, 'readwrite')
if nrmcl then
local kernel = prog:CreateKernel('normMult')
if kernel then
prog:SetArg(kernel, 0, filcl)
prog:SetArg(kernel, 1, nrmcl)
prog:SetArg(kernel, 2, mulcl)
prog:SetArgInt(kernel, 3, filter.Width, filter.Height)
success = prog:RunKernel(kernel)
if mulcl then
mulcl:ReleaseCLObject()
end
end
end
end
end
if success then
filcl = nrmcl
end
end
if success then
success = false
local input = Input:GetValue(req)
out = Image({IMG_Like = input})
if out then
local incl = prog:CreateImage(input, 'read')
local outcl = prog:CreateImage(out, 'write')
if filcl and incl and outcl then
local kernel = prog:CreateKernel('convolve')
if kernel then
prog:SetArg(kernel, 0, incl)
prog:SetArg(kernel, 1, filcl)
prog:SetArg(kernel, 2, outcl)
prog:SetArgInt(kernel, 3, input.Width, input.Height)
prog:SetArgInt(kernel, 4, filter.Width, filter.Height)
success = prog:RunKernel(kernel)
if success then
success = prog:Download(outcl, out)
end
if incl then
incl:ReleaseCLObject()
end
if filcl then
filcl:ReleaseCLObject()
end
if nrmcl then
nrmcl:ReleaseCLObject()
end
if outcl then
outcl:ReleaseCLObject()
end
end
end
end
end
if prog and not success then
OnRemoveFromFlow()
collectgarbage()
OnAddToFlow()
end
Output:Set(req, out)
end
Re: Convolve
Posted:
Fri Apr 21, 2017 12:10 am
by Bryan Ray
Very nice! You certainly learned OpenCL faster than I did! I haven't yet been successful in that arena.
Re: Convolve
Posted:
Fri Apr 21, 2017 1:00 am
by Bryan Ray
Wow. That crashed my video driver! (And Fusion, too, of course.)
Re: Convolve
Posted:
Fri Apr 21, 2017 8:14 am
by Unai Martínez
Thanks for the kind words, and sorry I crashed your GPU.
…On that note, I’d appreciate any tips for optimising the OCL kernels
Re: Convolve
Posted:
Sat Apr 22, 2017 7:55 pm
by michael vorberg
great work!
Re: Convolve
Posted:
Sat Apr 22, 2017 8:56 pm
by Bryan Ray
Previous test was in Fusion 7.7.1, where it crashed. Testing in Fusion 8.2.1b6 gives me interesting, but incorrect results:
- Untitled.jpg (226.55 KiB) Viewed 6227 times
It's definitely close to operational—changing the filter image changes the shape of the blooms.
I'm on Windows 7 with a Titan X.
I'm looking forward to the next time I have downtime at work so I can tear the fuse apart to see what makes it tick.
Re: Convolve
Posted:
Sun Apr 23, 2017 10:29 am
by Unai Martínez
That’s very interesting! In one of the iterations I tried using CreateBuffer instead of CreateImage and the output was jammed in a similar way.
Michael, I’m interested in knowing if it worked for you.
Re: Convolve
Posted:
Tue Sep 05, 2017 4:35 pm
by Unai Martínez
Hey!
So I was trying to get this working on Fusion 9, and it was erroring until I hand picked the GPU in the OpenCL settings (choosing Select in the Device checkboxes). I decided giving the CPU a go too, and to my surprise, it showed something similar to yours, Bryan.
I’ll try and fix that
Re: Convolve
Posted:
Tue Sep 05, 2017 5:42 pm
by Unai Martínez
Done. It was just a matter of initialising the float4s in the CL code. Here’s the updated version:
- Code: Select all
-- Convolve v1.2 by Unai Martínez Barredo - unaimb.com
FuRegisterClass('Convolve', CT_Tool, {
REGS_Category = 'Blur',
REGS_OpIconString = 'Cnv',
REGS_OpDescription = 'OpenCL Convolve Fuse',
REG_OpNoMask = true,
REG_NoBlendCtrls = true,
REG_NoObjMatCtrls = true,
REG_NoMotionBlurCtrls = true
})
function Create()
Input = self:AddInput('Input', 'Input', {
LINKID_DataType = 'Image',
LINK_Main = 1
})
Filter = self:AddInput('Filter', 'Filter', {
LINKID_DataType = 'Image',
LINK_Main = 2
})
Output = self:AddOutput('Output', 'Output', {
LINKID_DataType = 'Image',
LINK_Main = 1
})
Normal = self:AddInput('Normalise', 'Normalise', {
LINKID_DataType = 'Number',
INP_Integer = true,
INPID_InputControl = 'CheckboxControl',
INP_Default = 1
})
end
local mgr
local clsource = [[
kernel void normFind(FuReadImage_t inpImg, FuWriteImage_t outMul,
int2 inpSiz){
int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
if(inpPos.x == 0 && inpPos.y == 0){
float4 gdiv = (float4)(0.0, 0.0, 0.0, 0.0);
for(int y = 0; y < inpSiz.y; y ++){
for(int x = 0; x < inpSiz.x; x ++){
int2 cPos = (int2)(x, y);
gdiv += FuReadImagef(inpImg, cPos, inpSiz);
}
}
FuWriteImagef(outMul, (int2)(0, 0), (int2)(1, 1), 1 / gdiv);
}
}
kernel void normMult(FuReadImage_t inpImg, FuWriteImage_t outImg,
FuReadImage_t mulImg, int2 inpSiz){
int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
float4 mul = FuReadImagef(mulImg, (int2)(0, 0), (int2)(1, 1));
FuWriteImagef(outImg, inpPos, inpSiz,
FuReadImagef(inpImg, inpPos, inpSiz) * mul);
}
kernel void convolve(FuReadImage_t inpImg, FuReadImage_t filImg,
FuWriteImage_t outImg, int2 inpSiz, int2 filSiz){
int2 inpPos = (int2)(get_global_id(1), get_global_id(0));
float4 outVal = (float4)(0.0, 0.0, 0.0, 0.0);
for(int y = 0; y < filSiz.y; y ++){
for(int x = 0; x < filSiz.x; x ++){
int2 filPos = (int2)(x, y);
float4 inpVal = FuReadImageCf(inpImg, inpPos - filPos + filSiz / 2,
inpSiz);
float4 filVal = FuReadImagef(filImg, filPos, filSiz);
outVal = mad(inpVal, filVal, outVal);
}
}
FuWriteImagef(outImg, inpPos, inpSiz, outVal);
}
]]
function OnAddToFlow()
mgr = OCLManager()
if mgr then
prog = mgr:BuildCachedProgram('Convolve', debug.getinfo(1).source:sub(2),
clsource)
end
end
function OnRemoveFromFlow()
prog = nil
mgr = nil
end
function Process(req)
local filter = Filter:GetValue(req)
local filcl
if prog then
filcl = prog:CreateImage(filter, 'read')
end
local success = true
local nrmcl
local out
if Normal:GetValue(req).Value > 0.5 then
success = false
local mul = Image({
IMG_Width = 1,
IMG_Height = 1,
IMG_Depth = 8
})
local mulcl
if prog and mul then
mulcl = prog:CreateImage(mul, 'readwrite')
if filcl and mulcl then
local kernel = prog:CreateKernel('normFind')
if kernel then
prog:SetArg(kernel, 0, filcl)
prog:SetArg(kernel, 1, mulcl)
prog:SetArgInt(kernel, 2, filter.Width, filter.Height)
success = prog:RunKernel(kernel)
end
end
end
if success then
success = false
local nrm = Image({IMG_Like = filter})
if nrm then
nrmcl = prog:CreateImage(nrm, 'readwrite')
if nrmcl then
local kernel = prog:CreateKernel('normMult')
if kernel then
prog:SetArg(kernel, 0, filcl)
prog:SetArg(kernel, 1, nrmcl)
prog:SetArg(kernel, 2, mulcl)
prog:SetArgInt(kernel, 3, filter.Width, filter.Height)
success = prog:RunKernel(kernel)
if mulcl then
mulcl:ReleaseCLObject()
end
end
end
end
end
if success then
filcl = nrmcl
end
end
if success then
success = false
local input = Input:GetValue(req)
out = Image({IMG_Like = input})
if out then
local incl = prog:CreateImage(input, 'read')
local outcl = prog:CreateImage(out, 'write')
if filcl and incl and outcl then
local kernel = prog:CreateKernel('convolve')
if kernel then
prog:SetArg(kernel, 0, incl)
prog:SetArg(kernel, 1, filcl)
prog:SetArg(kernel, 2, outcl)
prog:SetArgInt(kernel, 3, input.Width, input.Height)
prog:SetArgInt(kernel, 4, filter.Width, filter.Height)
success = prog:RunKernel(kernel)
if success then
success = prog:Download(outcl, out)
end
if incl then
incl:ReleaseCLObject()
end
if filcl then
filcl:ReleaseCLObject()
end
if nrmcl then
nrmcl:ReleaseCLObject()
end
if outcl then
outcl:ReleaseCLObject()
end
end
end
end
end
if prog and not success then
OnRemoveFromFlow()
collectgarbage()
OnAddToFlow()
end
Output:Set(req, out)
end
Re: Convolve
Posted:
Tue Sep 05, 2017 6:19 pm
by Tony Gallardo
Hi Uni
I really love the idea of this...I tried to paste this into my flow, doesn't seem to want to paste. How should I go about "installing this in Fusion?"
on 2015MacBookPro, MacOS 10.11.6, Fusion Studio 9
Re: Convolve
Posted:
Tue Sep 05, 2017 6:27 pm
by Unai Martínez
Make a file called Convolve.fuse with the code in it. On macOS, place it inside ~/Library/Application Support/Blackmagic Design/Fusion/Fuses, restart Fusion and you’re ready to go!
Re: Convolve
Posted:
Wed Sep 06, 2017 10:30 am
by john barclay
Hey,
This is super cool. I'll need to take a look at your code soon as I'd like to see if it's altering the bokeh shape across the frame.
In other news,
Lenscare accepts an 'iris' that I think is used when convolving to create a custom bokeh shape. Both FL_outOfFocus and FL_depthOfField can have an iris.
http://www.frischluft.com/lenscare/Sapphire suite has z_convolve as well.
Re: Convolve
Posted:
Wed Sep 06, 2017 10:47 am
by Unai Martínez
It’s super basic; the shape is always the same and there’s no way of changing it. I’d love to try and implement that, but I have to make it stable yet— if the images are too big, it crashes, instead of just taking very long. Not cool.