Page 1 of 1

Minimum processing time for scheduled frames

PostPosted: Fri Apr 09, 2021 3:30 pm
by Steinar H. Gunderson

I'm generating video for playout (on various DeckLink cards), with (ideally) low latency. Currently, I'm using the scheduled playback from IDeckLinkOutput.

What I'd like to know is exactly when ScheduleVideoFrame() picked up my frame, or put differently: How far ahead of the frame start must I be for it to “reach the bus” and play out at the scheduled time? (A programmatic answer is fine, a static answer is fine. “Just use three frames” is not fine, because it's obviously possible to go much lower than that in practice.)

In general, the behavior seems to be very erratic as my algorithm dials down the latency (it's trying to find the minimum possible queue size that keeps no errors). Say that I've scheduled frame 100 and 101, but 101 was sent a bit too late from me to the driver. I will then almost immediately get a bmdOutputFrameDropped for frame 101, and then frame 100 gets delayed one frame! But frame 100 is not marked as bmdOutputFrameDisplayedLate; the callback just comes a frame later, with status bmdOutputFrameCompleted, and then 102 etc. are similarly delayed. It's very puzzling.

So what I'd like to know is, for any frame called back, when did the driver complete processing of it so that it was “safe” and put in the hardware queue? Or is there a better way to solve this?

Of course, I will have my own jitter in frame generation, but my code will measure it and compensate for it, as long as I know when the frame submission deadline is. :-)

Re: Minimum processing time for scheduled frames

PostPosted: Fri Apr 23, 2021 6:12 am
by Cameron Nichols
Hi Steinar,

Can I suggest experimenting using the InputLoopThrough SDK sample, and modify the constants:
  • kOutputVideoPreroll - set the preroll size of scheduled output
  • kWaitForReferenceToLock - set whether you are running with genlock reference
  • kProcessingAdditionalTimeMean, kProcessingAdditionalTimeStdDev - inject processing time to the loop
    through to simulate application
This will help understand whether your application can maintain a minimum preroll size.

If a video frame is scheduled, but the stream time has already passed and there is a frame at requested stream time, then you will get bmdOutputFrameDropped result immediately and the earlier frame is not output. If the stream time has passed, but there is not a scheduled frame at requested stream time, then the frame is output with bmdOutputFrameDisplayedLate completion result. This will occur if you have no scheduled frames buffered and scheduling has fallen behind output stream time.