Decklink SDK & QtQuick

Ask software engineering and SDK questions for developers working on Mac OS X, Windows or Linux.
  • Author
  • Message
Offline

tautin

  • Posts: 108
  • Joined: Thu Oct 18, 2018 3:11 pm
  • Real Name: Tim Autin

Decklink SDK & QtQuick

PostThu May 23, 2019 2:48 pm

Hello,

I successfully used the Decklink SDK samples to display a video from my Decklink card to a DeckLinkOpenGLWidget (subclass of QOpenGLWidget, from the SDK samples).

Now I'm trying to do the same in a QtQuick application. I'm able to draw some OpenGL stuff, so that part is working, but I can't get the video to show.

Here are some pieces of code:

VideoView.h
Code: Select all
class VideoView : public QQuickItem {

Q_OBJECT

public:
    VideoView();

public slots:
    void sync();
    void cleanup();
    void refresh();

private slots:
    void handleWindowChanged(QQuickWindow *win);

private:
    VideoViewRenderer *m_renderer;
};


VideoView.cpp
Code: Select all
VideoView::VideoView()
{
    m_renderer = nullptr;

    connect(this, &QQuickItem::windowChanged, this, &VideoView::handleWindowChanged);
}

void VideoView::handleWindowChanged(QQuickWindow *win)
{
    if (win) {
        connect(win, &QQuickWindow::beforeSynchronizing, this, &VideoView::sync, Qt::DirectConnection);
        connect(win, &QQuickWindow::sceneGraphInvalidated, this, &VideoView::cleanup, Qt::DirectConnection);
        // If we allow QML to do the clearing, they would clear what we paint and nothing would show.
        win->setClearBeforeRendering(false);
    }
}

void VideoView::cleanup()
{
    if (m_renderer) {
        delete m_renderer;
        m_renderer = nullptr;
    }
}

void VideoView::sync()
{
    if (!m_renderer) {
        m_renderer = new VideoViewRenderer(this, window()->size() * window()->devicePixelRatio(), window());
        connect(window(), &QQuickWindow::beforeRendering, m_renderer, &VideoViewRenderer::paint, Qt::DirectConnection);
    }
}

void VideoView::refresh() {
    update();
}


VideoViewRenderer.h
Code: Select all
class VideoView;

class VideoViewRenderer : public QObject, protected QOpenGLFunctions, public IDeckLinkScreenPreviewCallback {

Q_OBJECT

private:

    void initOpenGl();

    QSize                               m_viewportSize;
    QQuickWindow*                       m_window;
    QAtomicInt                      m_refCount;
    QMutex                        m_mutex;
    IDeckLinkGLScreenPreviewHelper*      m_deckLinkScreenPreviewHelper;
    DeckLinkDeviceDiscovery*            m_deckLinkDiscovery;
    DeckLinkInputDevice*              m_selectedDevice;
    VideoView*                          m_videoView;
    QOpenGLShaderProgram*               m_program = nullptr;

public:
    explicit VideoViewRenderer(VideoView* videoView, QSize viewportSize, QQuickWindow* window);
    ~VideoViewRenderer() override;

    // IUnknown
    HRESULT      QueryInterface(REFIID iid, LPVOID *ppv) override;
    ULONG      AddRef() override;
    ULONG      Release() override;

    // IDeckLinkScreenPreviewCallback
    HRESULT      DrawFrame(IDeckLinkVideoFrame* theFrame) override;

    // QObject
    void customEvent(QEvent* event) override;

public slots:

    void paint();
};


VideoViewRenderer.cpp
Code: Select all
VideoViewRenderer::VideoViewRenderer(VideoView* videoView, QSize viewportSize, QQuickWindow* window)
{
    m_videoView = videoView;
    m_viewportSize = viewportSize;
    m_window = window;

    m_selectedDevice = nullptr;
    m_deckLinkScreenPreviewHelper = CreateOpenGLScreenPreviewHelper();

    if (m_deckLinkScreenPreviewHelper != nullptr)
    {
        // Create and initialise DeckLink device discovery and profile objects
        m_deckLinkDiscovery = new DeckLinkDeviceDiscovery(this);

        if (m_deckLinkDiscovery == nullptr || !m_deckLinkDiscovery->enable())
        {
            std::cout << "This application requires the DeckLink drivers installed. Please install the Blackmagic DeckLink drivers to use the features of this application." << std::endl;
        }
        else
        {
            initOpenGl();
        }
    }
}

VideoViewRenderer::~VideoViewRenderer()
{
    if (m_deckLinkDiscovery != nullptr)
    {
        m_deckLinkDiscovery->Release();
        m_deckLinkDiscovery = nullptr;
    }
}

void VideoViewRenderer::initOpenGl()
{
    m_mutex.lock();
    m_deckLinkScreenPreviewHelper->InitializeGL();
    QOpenGLFunctions::initializeOpenGLFunctions();
    m_mutex.unlock();
}

void VideoViewRenderer::paint()
{
    m_mutex.lock();

    glLoadIdentity();
    glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
    glClear(GL_COLOR_BUFFER_BIT);

    m_deckLinkScreenPreviewHelper->PaintGL();

    m_mutex.unlock();
}

/// IUnknown methods

HRESULT VideoViewRenderer::QueryInterface(REFIID, LPVOID *ppv)
{
    *ppv = nullptr;
    return E_NOINTERFACE;
}

ULONG VideoViewRenderer::AddRef ()
{
    return (ULONG) m_refCount.fetchAndAddAcquire(1);
}

ULONG VideoViewRenderer::Release()
{
    auto newRefValue = (ULONG) m_refCount.fetchAndAddAcquire(-1);
    if (newRefValue == 0)
    {
        delete this;
        return 0;
    }

    return newRefValue;
}

/// IDeckLinkScreenPreviewCallback methods

HRESULT VideoViewRenderer::DrawFrame(IDeckLinkVideoFrame* theFrame)
{
    if (m_deckLinkScreenPreviewHelper != nullptr)
    {
        m_deckLinkScreenPreviewHelper->SetFrame(theFrame);
        m_videoView->refresh();
    }
    return S_OK;
}

/// QObject

void VideoViewRenderer::customEvent(QEvent *event)
{
    if (event->type() == kAddDeviceEvent)
    {
        auto* discoveryEvent = dynamic_cast<DeckLinkDeviceDiscoveryEvent*>(event);
        auto* newDevice = new DeckLinkInputDevice(this, discoveryEvent->DeckLink());

        // Initialise new DeckLinkDevice object
        if (!newDevice->Init())
        {
            // Device does not have IDeckLinkInput interface, eg it is a DeckLink Mini Monitor
            newDevice->Release();
            return;
        }

        if (newDevice->GetDeviceName() == "DeckLink 8K Pro (2)")
        {
            m_selectedDevice = newDevice;
            m_selectedDevice->StartCapture(bmdMode4K2160p25, this, true);
        }
    }
}


The DrawFrame is called, the problem is on the drawing part. If I replace the initOpenGl & paint functions by those:

Code: Select all
void VideoViewRenderer::initOpenGl()
{
    QOpenGLFunctions::initializeOpenGLFunctions();

    const char* vertexShaderSource = "#version 330 core\n"
                                     "layout (location = 0) in vec3 aPos;\n"
                                     "void main()\n"
                                     "{\n"
                                     "   gl_Position = vec4(aPos.x, aPos.y, aPos.z, 1.0);\n"
                                     "}\0";

    const char* fragmentShaderSource = "#version 330 core\n"
                                       "out vec4 FragColor;\n"
                                       "void main()\n"
                                       "{\n"
                                       "   FragColor = vec4(1.0f, 0.5f, 0.2f, 1.0f);\n"
                                       "}\n\0";


    m_program = new QOpenGLShaderProgram();
    m_program->addCacheableShaderFromSourceCode(QOpenGLShader::Vertex,vertexShaderSource);
    m_program->addCacheableShaderFromSourceCode(QOpenGLShader::Fragment, fragmentShaderSource);

    m_program->bindAttributeLocation("vertices", 0);
    m_program->link();
}


Code: Select all
void VideoViewRenderer::paint()
{
    glClearColor(0.0f, 0.4f, 0.0f, 0.0f);
    glClear(GL_COLOR_BUFFER_BIT);

    m_program->bind();
    m_program->enableAttributeArray(0);

    float values[] = {
            -0.5f, -0.5f, 0.0f, // left
            0.5f, -0.5f, 0.0f, // right
            0.0f,  0.5f, 0.0f  // top
    };

    m_program->setAttributeArray(0, GL_FLOAT, values, 3);
    m_program->setUniformValue("t", (float) 0);

    glViewport(0, 0, m_viewportSize.width(), m_viewportSize.height());
    glDrawArrays(GL_TRIANGLE_STRIP, 0, 3);

    m_program->disableAttributeArray(0);
    m_program->release();
}


The triangle is correctly displayed.

Any idea?

Thanks in advance!
Offline

Cameron Nichols

Blackmagic Design

  • Posts: 442
  • Joined: Mon Sep 04, 2017 4:05 am

Re: Decklink SDK & QtQuick

PostThu May 30, 2019 4:12 am

Hi Tautin,

I think you need VideoViewRenderer constructor to call parent QOpenGLWidget constructor and implement overrides for initializeGL(), paintGL() and resizeGL() methods.

Please have a look at the Linux CapturePreview sample, or the cross-platform QuadPreview sample to understand how this is implemented.

Regards
Cameron
Offline

tautin

  • Posts: 108
  • Joined: Thu Oct 18, 2018 3:11 pm
  • Real Name: Tim Autin

Re: Decklink SDK & QtQuick

PostMon Jun 03, 2019 4:02 pm

Hi Cameron,

Thanks for your answer.

QtQuick and QWidgets are two differents GUI APIs that can hardly be mixed, so I can't use the QOpenGLWidget at all, and my VideoViewRenderer is not inheriting QOpenGLWidget.

I'm very surprised that there's no QML / QQuickItem sample in the SDK samples (like the DeckLinkOpenGLWidget for QWidgets). QtQuick has been there for 8 years now, it seems to be the way to go, and we're no longer using QWidgets in my company.

Thanks
Offline

tautin

  • Posts: 108
  • Joined: Thu Oct 18, 2018 3:11 pm
  • Real Name: Tim Autin

Re: Decklink SDK & QtQuick

PostFri Jun 14, 2019 8:44 am

In the end I used GStreamer and the qmlglsink element. This allows to display my 4K video in a QML app in ~ 150 lines of code.

A sample of the qmlglsink is available here: https://cgit.freedesktop.org/gstreamer/ ... k?h=master

To display a video from a Blackmagic card you need to add a glcolorconvert element after the glupload.

The version of the decklinkvideosrc element provided with GStreamer 1.16 (needed for the qmlglsink element) has a few bugs. Thanks to Sebastian Dröge from GStreamer I've been able to get a working solution while he finds the bug. See the discussion here: https://gitlab.freedesktop.org/gstreame ... issues/986

I also have another non blocking problem described here: https://gitlab.freedesktop.org/gstreame ... issues/988 (the video mode needs to be set when starting the video, auto-detection leads to shaky video).

It's likely possible to do it with Blackmagic SDK if you are ready to read the ~ 2000 lines of the LoopThroughWithOpenGLCompositing sample and to debug it (it just shows an OpenGL scene on a black background for me)...

Return to Software Developers

Who is online

Users browsing this forum: No registered users and 24 guests