Minimising capture latency

Ask software engineering and SDK questions for developers working on Mac OS X, Windows or Linux.
  • Author
  • Message
Offline

Baldur Gislason

  • Posts: 13
  • Joined: Sat Jan 06, 2018 7:55 pm

Minimising capture latency

PostWed Nov 09, 2022 3:46 pm

Hello. I am developing a real time video processing application, using Decklink 8k pro cards.
Right now I'm just using the Capture.cpp examples as the basis for getting video out of the cards on a DecklinkCaptureDelegate VideoInputFrameArrived callback, where I copy the video frame straight to an Nvidia graphics card for processing by another process.
Are there any optimisations I can do in the capture process to minimise the latency between the last pixel of the last video line being clocked in to the SDI input until the callback is called?

I noticed a few years ago, when I was outputting video via SDI, I was able to shave off a considerable amount of latency by disabling the audio and thus eliminating preroll in the output. Are there any tricks like this that help with input latency? It doesn't seem logical that there should be any latency at all, as buffering video frames that are ready to be processed is just a waste of memory when there is no timing requirement on the receiving end.
Baldur Gislason
Offline

Cameron Nichols

Blackmagic Design

  • Posts: 442
  • Joined: Mon Sep 04, 2017 4:05 am

Re: Minimising capture latency

PostFri Nov 11, 2022 5:06 am

Hi Baldur,

So long as the IDeckLinkInputCallback::VideoInputFrameArrived[1] callback is being returned promptly, then you should be able to receive callback shortly after the end of frame on the wire (a few ms). You should run the InputLoopThrough SDK sample to see this measurement.

If the VideoInputFrameArrived callback is not promptly returned, then you may see queuing of input frames, and IDeckLinkInput::GetAvailableVideoFrameCount[2] and latency will grow.

As best practice:
  • Maintain the frame reference by calling IDeckLinkVideoInputFrame::AddRef
  • Push the captured frame to a queue (eg SampleQueue in InputLoopThrough)
  • Return from IDeckLinkInputCallback::VideoInputFrameArrived
  • In processing thread, pop the frame from queue
  • Once completed processing release frame with IDeckLinkVideoInputFrame::Release
For select NVIDIA cards (eg Quadro, RTX A families), to minimise latency you should consider implementing with GPU Direct for Video [3] that will pin the video frame buffer in system memory that can be accessed by the GPU. Please refer to the LoopThroughWithOpenGLCompositing or LoopThroughWithDX11Compositing SDK samples for an example implementation of GPU Direct.

Regards
Cameron

References:
[1] DeckLink SDK Manual 2.5.10.1 IDeckLinkInputCallback::VideoInputFrameArrived method
[2] DeckLink SDK Manual 2.5.4.6 IDeckLinkInput::GetAvailableVideoFrameCount method
[3] https://developer.nvidia.com/gpudirectforvideo

Return to Software Developers

Who is online

Users browsing this forum: No registered users and 22 guests