Issue creating video frames using C#

Ask software engineering and SDK questions for developers working on Mac OS X, Windows or Linux.
  • Author
  • Message
Offline

DanS98

  • Posts: 5
  • Joined: Thu Oct 04, 2018 10:53 pm
  • Real Name: Daniel Shields

Issue creating video frames using C#

PostThu Feb 25, 2021 1:17 am

Hi,

I'm currently trying to output a frame buffer to a decklink in C# using the 12.0 SDK. I have the following code running on each "ScheduledFrameCompleted" callback:
Code: Select all
byte[] framePixelData;
if (Program.frameBuffer.TryDequeue(out framePixelData))
{
    IDeckLinkMutableVideoFrame scheduleFrame;

    selectedDevice.DeckLinkOutput.CreateVideoFrame(frameWidth, frameHeight, BytesPerRow, selectedPixelFormat, _BMDFrameFlags.bmdFrameFlagDefault, out scheduleFrame);

    FillRenderFrame(scheduleFrame, framePixelData);

    selectedDevice.DeckLinkOutput.ScheduleVideoFrame(scheduleFrame, (totalFramesScheduled * frameDuration), frameDuration, frameTimescale);
} else
{
    // frame buffer was empty or locked elsewhere
    ScheduleEmptyFrame();
}

totalFramesScheduled++;

This process runs fine for between 4 and 6 frames, before the "CreateVideoFrame" call returns E_FAIL (and throws an exception). This makes me think there's some kind of memory management step I'm missing?

My program is based of the SignalGenCSharp sample, which I had working fine on the 12.0 SDK, but I guess the key difference is I'm running CreateVideoFrame for each new frame?

For more info, the FillRenderFrame function does a simple Marshal.Copy of the pixel data into the Decklink frame.

Would be grateful for any advice/input on this! Let me know if there's any other code snippets you need that could help work out what the issue is. Thanks!
Offline

DanS98

  • Posts: 5
  • Joined: Thu Oct 04, 2018 10:53 pm
  • Real Name: Daniel Shields

Re: Issue creating video frames using C#

PostThu Feb 25, 2021 7:48 pm

I've found a workaround for this issue by sub-classing IDecklinkVideoFrame and using some unsafe pointer trickery to pass the pixel data into it without needing to copy anything before ScheduleVideoFrame.

It would still be useful to understand why CreateVideoFrame returns E_FAIL when called too many times, but the working snippet is as follows.

Upon ScheduledFrameCompleted:
Code: Select all
byte[] framePixelData;
if (Program.frameBuffer.TryDequeue(out framePixelData))
{
    unsafe
    {
        fixed (byte* p = framePixelData)
        {
            IntPtr ptr = (IntPtr)p;

            CustomDecklinkFrame frame = new CustomDecklinkFrame(frameWidth, frameHeight, ptr);

            selectedDevice.DeckLinkOutput.ScheduleVideoFrame(frame, (totalFramesScheduled * frameDuration), frameDuration, frameTimescale);
        }
    }
} else
{
    // frame buffer was empty or locked elsewhere
    ScheduleEmptyFrame();
}

totalFramesScheduled++;

CustomDecklinkFrame class:
Code: Select all
public class CustomDecklinkFrame : IDeckLinkVideoFrame {
    private int width;
    private int height;
    private IntPtr bufferPtr;

    public CustomDecklinkFrame (int width, int height, IntPtr bufferPtr)
    {
        this.width = width;
        this.height = height;
        this.bufferPtr = bufferPtr;
    }

    public int GetWidth () => width;
    public int GetHeight () => height;
    public int GetRowBytes() => width * 4;
    public _BMDPixelFormat GetPixelFormat() => _BMDPixelFormat.bmdFormat8BitBGRA;
    public _BMDFrameFlags GetFlags() => _BMDFrameFlags.bmdFrameFlagDefault;

    public void GetBytes(out IntPtr buffer)
    {
        buffer = bufferPtr;
    }

    public void GetTimecode(_BMDTimecodeFormat format, out IDeckLinkTimecode timecode)
    {
        timecode = null;
    }
    public void GetAncillaryData(out IDeckLinkVideoFrameAncillary ancillaryData)
    {
        ancillaryData = null;
    }
}
Offline

Cameron Nichols

Blackmagic Design

  • Posts: 343
  • Joined: Mon Sep 04, 2017 4:05 am

Re: Issue creating video frames using C#

PostFri Mar 05, 2021 4:59 am

Hi Daniel,

When using the default memory allocator, IDeckLinkOutput::CreateVideoFrame will allocate up to approx 35 frames of the maximum resolution supported by the device. The SignalGenCSharp sample only allocates a frame for colour bars, one for black frame, and optionally 2 frames for pixel format conversions of each frame.

To allocate more frames, you can generate your own custom memory allocator by creating a class that implements IDeckLinkMemoryAllocator[1], and registering it with IDeckLinkOutput::SetVideoOutputFrameMemoryAllocator[2].

But what you have implemented is also an acceptable solution. An advantage of this solution it is a zero-copy wrapper to the IDeckLinkVideoFrame interface. Please take care that after CustomDecklinkFrame constructor that the buffer remains static until IDeckLinkVideoOutputCallback::ScheduledFrameCompleted callback.

Regards
Cameron

References:
[1] 2.5.18 IDeckLinkMemoryAllocator Interface
[2] 2.5.3.8 IDeckLinkOutput::SetVideoOutputFrameMemoryAllocator method
Offline

DanS98

  • Posts: 5
  • Joined: Thu Oct 04, 2018 10:53 pm
  • Real Name: Daniel Shields

Re: Issue creating video frames using C#

PostSat Mar 27, 2021 5:45 pm

Thanks for the reply Cameron, that's cleared things up for me.

Return to Software Developers

Who is online

Users browsing this forum: No registered users and 2 guests