Issue creating video frames using C#

Hi,

I'm currently trying to output a frame buffer to a decklink in C# using the 12.0 SDK. I have the following code running on each "ScheduledFrameCompleted" callback:

Code: Select all

byte[] framePixelData;
if (Program.frameBuffer.TryDequeue(out framePixelData))
{
    IDeckLinkMutableVideoFrame scheduleFrame;

    selectedDevice.DeckLinkOutput.CreateVideoFrame(frameWidth, frameHeight, BytesPerRow, selectedPixelFormat, _BMDFrameFlags.bmdFrameFlagDefault, out scheduleFrame);

    FillRenderFrame(scheduleFrame, framePixelData);

    selectedDevice.DeckLinkOutput.ScheduleVideoFrame(scheduleFrame, (totalFramesScheduled * frameDuration), frameDuration, frameTimescale);
} else
{
    // frame buffer was empty or locked elsewhere
    ScheduleEmptyFrame();
}

totalFramesScheduled++;

This process runs fine for between 4 and 6 frames, before the "CreateVideoFrame" call returns E_FAIL (and throws an exception). This makes me think there's some kind of memory management step I'm missing?

My program is based of the SignalGenCSharp sample, which I had working fine on the 12.0 SDK, but I guess the key difference is I'm running CreateVideoFrame for each new frame?

For more info, the FillRenderFrame function does a simple Marshal.Copy of the pixel data into the Decklink frame.

Would be grateful for any advice/input on this! Let me know if there's any other code snippets you need that could help work out what the issue is. Thanks!

I've found a workaround for this issue by sub-classing IDecklinkVideoFrame and using some unsafe pointer trickery to pass the pixel data into it without needing to copy anything before ScheduleVideoFrame.

It would still be useful to understand why CreateVideoFrame returns E_FAIL when called too many times, but the working snippet is as follows.

Upon ScheduledFrameCompleted:

Code: Select all

byte[] framePixelData;
if (Program.frameBuffer.TryDequeue(out framePixelData))
{
    unsafe
    {
        fixed (byte* p = framePixelData)
        {
            IntPtr ptr = (IntPtr)p;

            CustomDecklinkFrame frame = new CustomDecklinkFrame(frameWidth, frameHeight, ptr);

            selectedDevice.DeckLinkOutput.ScheduleVideoFrame(frame, (totalFramesScheduled * frameDuration), frameDuration, frameTimescale);
        }
    }
} else
{
    // frame buffer was empty or locked elsewhere
    ScheduleEmptyFrame();
}

totalFramesScheduled++;

CustomDecklinkFrame class:

Code: Select all

public class CustomDecklinkFrame : IDeckLinkVideoFrame {
    private int width;
    private int height;
    private IntPtr bufferPtr;

    public CustomDecklinkFrame (int width, int height, IntPtr bufferPtr)
    {
        this.width = width;
        this.height = height;
        this.bufferPtr = bufferPtr;
    }

    public int GetWidth () => width;
    public int GetHeight () => height;
    public int GetRowBytes() => width * 4;
    public _BMDPixelFormat GetPixelFormat() => _BMDPixelFormat.bmdFormat8BitBGRA;
    public _BMDFrameFlags GetFlags() => _BMDFrameFlags.bmdFrameFlagDefault;

    public void GetBytes(out IntPtr buffer)
    {
        buffer = bufferPtr;
    }

    public void GetTimecode(_BMDTimecodeFormat format, out IDeckLinkTimecode timecode)
    {
        timecode = null;
    }
    public void GetAncillaryData(out IDeckLinkVideoFrameAncillary ancillaryData)
    {
        ancillaryData = null;
    }
}

[Cameron Nichols]

Hi Daniel,

When using the default memory allocator, IDeckLinkOutput::CreateVideoFrame will allocate up to approx 35 frames of the maximum resolution supported by the device. The SignalGenCSharp sample only allocates a frame for colour bars, one for black frame, and optionally 2 frames for pixel format conversions of each frame.

To allocate more frames, you can generate your own custom memory allocator by creating a class that implements IDeckLinkMemoryAllocator[1], and registering it with IDeckLinkOutput::SetVideoOutputFrameMemoryAllocator[2].

But what you have implemented is also an acceptable solution. An advantage of this solution it is a zero-copy wrapper to the IDeckLinkVideoFrame interface. Please take care that after CustomDecklinkFrame constructor that the buffer remains static until IDeckLinkVideoOutputCallback::ScheduledFrameCompleted callback.

Regards
Cameron

References:
[1] 2.5.18 IDeckLinkMemoryAllocator Interface
[2] 2.5.3.8 IDeckLinkOutput::SetVideoOutputFrameMemoryAllocator method

Thanks for the reply Cameron, that's cleared things up for me.