IDeckLinkGLScreenPreviewHelper in C++ on OSX

Hi all,

I just wanted to check that I am approaching my problem correctly before going too far down the rabbit hole. I have a c++ OpenGL app (made with libcinder) and want to render the captured input into an FBO, before then using Apple's AVFoundation to work with it further. The FBO is already set up and rendering to screen.

I have some old capture code that loops through the pixels in IDeckLinkInputCallback::VideoInputFrameArrived() and creates a texture. This works but isn't as performative as I need.

Am I correct in thinking that I should use IDeckLinkGLScreenPreviewHelper::PaintGL() to draw directly to my FBO instead? There is no OSX equivalent of the LoopThroughWithOpenGLCompositing example. Any advice on how to go about doing this or a sample?

Thanks,
Rene.

[Cameron Nichols]

Hi Rene,

To improve performance you need to move rendering functions out of IDeckLinkInputCallback::VideoInputFrameArrived() callback. The issue is that it is expected that VideoInputFrameArrived completes within a frame period to avoid queuing of frames and eventual dropped frames.

Instead, create a class that subclasses IDeckLinkScreenPreviewCallback and implement functions to update your preview window. The key difference of the DrawFrame callback is that the DeckLinkAPI does not queue frames if in case you don't return from the callback within the frame period. This makes the callback more suitable for rendering tasks. You register your callback with IDeckLinkInput::SetScreenPreviewCallback() method. Inside the DrawFrame callback you call IDeckLinkGLScreenPreviewHelper::SetFrame, set GL rendering context and call PaintGL(). Have a look at the Windows CapturePreview sample for an example how this is done.

Alternatively, we have the IDeckLinkCocoaScreenPreviewCallback helper that abstracts the OpenGL calls for displaying the preview in a Cocoa application. Please see the Mac CapturePreview sample for usage.

Regards
Cameron

Thanks Cameron,

I spent some time with this but got stuck. The libcinder app template I use creates an Open GL context on the main thread. I want to get the preview into an FBO in this Open GL context. However, it seems that IDeckLinkScreenPreviewCallback::DrawFrame gets called on a different thread and I get a crash when trying to bind a FBO or call IDeckLinkGLScreenPreviewHelper::PaintGL inside it.

What is the recommended approach here? I have not had to do this before but do I need to set up a shared Open GL context or use a PBO for this? Is that even possible when calling IDeckLinkGLScreenPreviewHelper::PaintGL or do I need to subclass IDeckLinkGLScreenPreviewHelper and override PaintGL to do something?

Any advice appreciated!
Rene.

Just wanted to add that I have tried calling PaintGL from the main thread, and although it no longer crashes it doesn't appear to render anything. Pseudo code snippets below.

I pass a pointer of a PreviewHelper instance to my PreviewCallback class then:

Code: Select all

virtual HRESULT STDMETHODCALLTYPE   DrawFrame(IDeckLinkVideoFrame* theFrame){
        mtx1.lock();
        PreviewHelper->SetFrame(theFrame);
        mtx1.unlock();
}


The on main thread:

Code: Select all

    //bind fbo
    //translate
    //scale
    //set fill colour to white
        mtx2.lock();
        MyObject.PreviewHelper->PaintGL();
        mtx2.unlock();
    //unbind fbo


A Rect with co-ordinates (-1,-1,1,1) draws fine.