DrawFrame method

Ask software engineering and SDK questions for developers working on Mac OS X, Windows or Linux.
  • Author
  • Message
Offline

makoe29

  • Posts: 16
  • Joined: Wed Apr 14, 2021 11:53 am
  • Real Name: Marco Könemann

DrawFrame method

PostTue Aug 17, 2021 12:42 pm

In the CapturePreview example under PreviewWindow a preview box is calculated (and generated?) manually. In the OpenGLOutput example the size of the preview window is not defined. How does this works? Is the size of the window calculated automatically? In the doc I read that DrawFrame() is called when a Frame arrived, but I found no call of this method in VideoInputFrameArrived().

My second question is if it is possible to use the IDeckLinkScreenPreviewCallback and the ScreenPreview Helper together with SDL or GLFW? Is it possible to create a window in the DrawFrame method? Or have I to use the Win32 API?

Thank you.
Offline

Cameron Nichols

Blackmagic Design

  • Posts: 442
  • Joined: Mon Sep 04, 2017 4:05 am

Re: DrawFrame method

PostFri Sep 10, 2021 4:36 am

Hi Marco,

You are referring to 2 different callbacks, let me explain the difference:
  • IDeckLinkInputCallback::VideoInputFrameArrived: All frames received are queued with an internal frame buffer, then delivered to the callback. It is the responsibility of the application to ensure prompt return from the callback to prevent overflow of the frame buffer.
  • IDeckLinkScreenPreviewCallback::DrawFrame: This callback is designed for rendering, where the render task may take longer than a frame time period. If further frame(s) are received before the return of callback, then they are dropped.
The CapturePreview (Win32 GL) and QuadPreview (Qt QOpenGLWidget) samples demonstrate how to draw to OpenGL context with orthographic projection using the OpenGL helper (IDeckLinkGLScreenPreviewHelper). These samples call glViewport to set viewport size - the rendering is done to GL space between (-1,-1) and (1,1) and projected to viewport.

Please also refer to section 2.5.26 of the DeckLink SDK manual how to configure IDeckLinkGLScreenPreviewHelper object interface and how it is used with IDeckLinkScreenPreviewCallback.

Regards
Cameron
Offline

makoe29

  • Posts: 16
  • Joined: Wed Apr 14, 2021 11:53 am
  • Real Name: Marco Könemann

Re: DrawFrame method

PostWed Sep 22, 2021 1:32 pm

Hi Cameron,

thanks for information. There is another question.

In the QuadPreview there is used a own defined com_ptr template. Why is this necassery? Is there a disadvantage using smart pointers?

Kind Regards
Marco
Offline

Cameron Nichols

Blackmagic Design

  • Posts: 442
  • Joined: Mon Sep 04, 2017 4:05 am

Re: DrawFrame method

PostThu Sep 23, 2021 7:41 am

Hi Marco,

We use com_ptr<> template to provide automatic reference counting of IDeckLink objects while the object remains in scope. Otherwise you need to manage reference counts manually via calls to IUnknown::AddRef() and IUnknown::Release(). We have examples of both automatic and manual reference counting throughout the SDK samples.

The com_ptr template was chosen for the QuadPreview sample as it is a Qt cross-platform sample (Linux/Mac/Windows). A few other SDK samples specific to Windows (CapturePreview, SignalGenerator, FileCapture, FilePlayback) use CComPtr from the ATL library[1]. Both smart pointer templates implement the same principles for automatic reference counting.

Regards
Cameron

References:
[1] https://docs.microsoft.com/en-us/cpp/at ... w=msvc-160
Offline

makoe29

  • Posts: 16
  • Joined: Wed Apr 14, 2021 11:53 am
  • Real Name: Marco Könemann

Re: DrawFrame method

PostSat Oct 02, 2021 10:51 am

Hi Cameron,
thanks for information.
I write a appliction which should display a camera input (detected with Ultra Studio Recorder) in a GUI. For this I use Qt. I use the Qt event loop with signal and slots for the frame transfer. The "architcture" of the event loop is "inspired" by the QuadPreview example from the SDK but instead of emit I use QMetaObject::invokeMethod and the QThreading module.
Actually the constrcutor of the class (which initiaizes the ScreenHelperPreview and manages OpenGL, the GUI-Class (which handles starting the video when a button is clicked) and the slot function, that calles SetFrame() runs in the same thread. So, I think there should no problem. But when I start the video I get a black screen. When I debug the program I see that the event loop runs.
I will be very thankful if you, or someone else, have an idea why this did not work.

Code: Select all
/**
* Videoscreen.cpp
*/
VideoScreen::VideoScreen(QWidget* parent) :
   m_previewHelper(nullptr)
{
   qDebug() << "VideoScreen constructor was called.";

   //Registering the type name of type CComPtr<IDeckLinkVideoFrame>, requires include<QMetaType>
   //Create and destroy objects of the type dyamically at runtime after registration
   qRegisterMetaType<CComPtr<IDeckLinkVideoFrame>>("CComPtr<IDeckLinkVideoFrame>");

   //Initializing Screen Preview Helper
   ScreenPreviewHelperInstance(m_previewHelper);

   m_vsThread = new QThread();
   m_drawFrame = new VideoScreenHelper(parent);
   m_drawFrame->moveToThread(m_vsThread);
   //QObject::connect(const QObject *sender, PointerToMemberFunction signal, const QObject *receiver, PointerToMemberFunction method, Qt::ConnectionType type = Qt::AutoConnection)
   QObject::connect(m_drawFrame, &VideoScreenHelper::FrameChanged, this, &VideoScreen::HandleFrame, Qt::QueuedConnection);
   QObject::connect(m_vsThread, &QThread::finished, m_drawFrame, &QObject::deleteLater); //Deletes VideoScreenHelper object
   QObject::connect(m_vsThread, &QThread::finished, m_vsThread, &QObject::deleteLater); //Deletes QThread object
   m_vsThread->start();
   qDebug() << "VideoScreen Ctor: Thread-ID is " << QThread::currentThreadId();
}

/**
* Function that inializes a ScreenPreviewHelper instance. Function is called in the constructor.
*/
CComPtr<IDeckLinkGLScreenPreviewHelper> VideoScreen::ScreenPreviewHelperInstance(CComPtr<IDeckLinkGLScreenPreviewHelper> previewHelper) {

   HRESULT result = CoCreateInstance(CLSID_CDeckLinkGLScreenPreviewHelper, NULL, CLSCTX_ALL, IID_IDeckLinkGLScreenPreviewHelper, (void**)&m_previewHelper);
   if (result == S_OK) {
      qDebug() << "ScreenPreviewHelper installed.";
   }
   return m_previewHelper;

   
}

/**
* Getter function
*/
IDeckLinkScreenPreviewCallback* VideoScreen::GetScreenPreviewCallback() {
   qDebug() << "VideoScreen: GetScreenPeviewCallback was called.";
   return m_drawFrame;
   
}

/**
* Slot function - Sets up the frame
*/
void VideoScreen::HandleFrame(CComPtr<IDeckLinkVideoFrame> frame) {
   if (m_previewHelper != nullptr) {
      m_previewHelper->SetFrame(frame);
      qDebug() << "VideoScreen: HandleFrame method is called. Thread-ID: " << QThread::currentThreadId();
   }
   
}

//QOpenGLWidget Interface functions
/**
*
*/
void VideoScreen::initializeGL() {
   if (m_previewHelper != nullptr) {
      m_previewHelper->InitializeGL();
      qDebug() << "VideoScreen: initalizeGL works.";
   }
}

void VideoScreen::paintGL() {
   if (m_previewHelper != nullptr) {
      m_previewHelper->PaintGL();
      qDebug() << "VideoScreen: paintGL was called.";
   }
}

void VideoScreen::resizeGL(int w, int h) {
   QOpenGLFunctions* m_gLFuncs = QOpenGLContext::currentContext()->functions();
   m_gLFuncs->glViewport(0, 0, w, h);
}


Code: Select all
/**
* VideoScreenHelper.cpp
*/
VideoScreenHelper::VideoScreenHelper(QObject* parent)
   : m_referenceCounter(1)
{}

/**
* Dtor
*/
VideoScreenHelper::~VideoScreenHelper()
{}

/**
* The method is called on every incoming frame and is derived from the IDeckLinkScreenPreviewCallback
*/
HRESULT VideoScreenHelper::DrawFrame(IDeckLinkVideoFrame* frame) {
   QMetaObject::invokeMethod(this, "FrameChanged", Qt::QueuedConnection, Q_ARG(CComPtr<IDeckLinkVideoFrame>, frame));
   qDebug() << "VideoScreenHelper: DrawFrame is called. Thread-ID is " << QThread::currentThreadId();
   return S_OK;
}


Code: Select all
/**
* QVideoMeter
*/
QVideoMeter::QVideoMeter(QWidget* parent)
    : QWidget(parent)
{
    //initializes ControlVideo with a decklink instance
    m_control = std::make_unique<ControlVideo>(m_init->CreateDeckLinkInstance());
    m_videoScreen = new VideoScreen(parent);

    ui.setupUi(this);
    //Observer pattern in Qt style. If the start button is released, the video stream starts
    QObject::connect(ui.startBtn, &QPushButton::clicked, this, &QVideoMeter::QStartVideo);
    QObject::connect(ui.stopBtn, &QPushButton::released, this, &QVideoMeter::QStopVideo);

    qDebug() << "QVideoMeter CTOR" << QThread::currentThreadId();

}

/**
* Dtor
*/
QVideoMeter::~QVideoMeter() {
    //delete ui;
}

/**
* Button interaction. If the start button is pressed the StartVideo function is called
*/
void QVideoMeter::QStartVideo() {

    HRESULT inputVideoData = m_control->StartVideo(bmdModeHD1080p25, bmdFormat8BitYUV, bmdAudioSampleRate48kHz, bmdAudioSampleType16bitInteger, 2, false, m_videoScreen->GetScreenPreviewCallback());

    if (inputVideoData == S_OK) {
        ui.startBtn->setText("Running");
        qDebug() << "QVideoMeter: StartButton clicked. Stream is running.";
    }
    else {
        QMessageBox::information(this, "Kein Input", "Es wurde kein Eingangssignal erkannt.");
    }
}
Offline

makoe29

  • Posts: 16
  • Joined: Wed Apr 14, 2021 11:53 am
  • Real Name: Marco Könemann

Re: DrawFrame method

PostThu Oct 07, 2021 2:38 pm

Let me correct my last post. I think manageing threads is not necessary.

I remove QThread, so my constructor looks like this (I reduce it to the elementary):

Code: Select all
VideoScreen::VideoScreen(QWidget* parent) : m_previewHelper(nullptr), m_drawFrame(nullptr)
{
   qRegisterMetaType<CComPtr<IDeckLinkVideoFrame>>("CComPtr<IDeckLinkVideoFrame>");

   m_drawFrame = new VideoScreenHelper(parent);

   QObject::connect(m_drawFrame, &VideoScreenHelper::FrameChanged, this, &VideoScreen::HandleFrame, Qt::AutoConnection);
}

Actully the loop runs. The camera device also reacts to the capture process. But I get a black screen. When I implement the com_ptr template in my project, what makes my project similar to the sample code I also get a black screen.
Offline

makoe29

  • Posts: 16
  • Joined: Wed Apr 14, 2021 11:53 am
  • Real Name: Marco Könemann

Re: Program renders single frame after minimizing the window

PostWed Oct 13, 2021 1:16 pm

Hi,
I found by accident that my program renders a single frame of the camera input, after I start the loop and minimize the GUI of the program. When I reopen the GUI a new frame is rendered and also, as I can see in the console, the paintGL() method of the QOpenGLWidget interface is called. So the widget in the GUI, that displays the video, did not update the state of the frame when the window is open. My idea is that there is either a problem with the Qt OpenGL interface or with the videoinputformat method /auto-mode-detection?

Kind regards,
Marco

Return to Software Developers

Who is online

Users browsing this forum: SerhiiK and 11 guests