Missing/Dropped frames during capture

I am using the SDK to capture VANC data. The source is 1080i 59.94 and I am using the DeckLink Mini Recorder.

What I discovered is a large number of missing/dropped frames as evidenced by skips in the time info (along with missing VANC data that I know to be in there). A relevant code fragment is given at the end. Capturing with bmdModeHD1080i5994 and bmdFormat10BitYUV.

I am running on a dedicated Windows 7 PC. Though not the swiftest, it isn't ancient either (~3 yrs old). CPU is at 11%.

I've also confirmed, by extracting EIA 608 captions from VANC line 9, that there are missing characters whereas the same source dual fed to an SDI monitor shows captions w/o any missing characters.

The output for the test program looks like:

Frame time 78078 (on scale 60000)
Frame time 80080 (on scale 60000)
Frame time 82082 (on scale 60000)
Frame time 84084 (on scale 60000)
Frame time 86086 (on scale 60000)
Frame time 90090 (on scale 60000)
Missing frame. Time delta is (on scale 60000): 4004
Frame time 92092 (on scale 60000)
Frame time 94094 (on scale 60000)
Frame time 96096 (on scale 60000)
Frame time 100100 (on scale 60000)
Missing frame. Time delta is (on scale 60000): 4004
Frame time 102102 (on scale 60000)
Frame time 104104 (on scale 60000)
Frame time 106106 (on scale 60000)
Frame time 108108 (on scale 60000)
Frame time 110110 (on scale 60000)
Frame time 114114 (on scale 60000)
Missing frame. Time delta is (on scale 60000): 4004
Frame time 116116 (on scale 60000)
Frame time 118118 (on scale 60000)
Frame time 120120 (on scale 60000)
Frame time 124124 (on scale 60000)
Missing frame. Time delta is (on scale 60000): 4004
Frame time 126126 (on scale 60000)
Frame time 128128 (on scale 60000)
Frame time 130130 (on scale 60000)
Frame time 132132 (on scale 60000)
Frame time 134134 (on scale 60000)
Frame time 138138 (on scale 60000)
Missing frame. Time delta is (on scale 60000): 4004
Frame time 140140 (on scale 60000)
Frame time 142142 (on scale 60000)
Frame time 144144 (on scale 60000)
Frame time 146146 (on scale 60000)
Frame time 148148 (on scale 60000)
Frame time 152152 (on scale 60000)
Missing frame. Time delta is (on scale 60000): 4004
Frame time 172172 (on scale 60000)
Missing frame. Time delta is (on scale 60000): 20020
Frame time 174174 (on scale 60000)
Frame time 176176 (on scale 60000)
Frame time 178178 (on scale 60000)
Frame time 182182 (on scale 60000)
Missing frame. Time delta is (on scale 60000): 4004
Frame time 184184 (on scale 60000)
Frame time 186186 (on scale 60000)
Frame time 188188 (on scale 60000)
Frame time 192192 (on scale 60000)
Missing frame. Time delta is (on scale 60000): 4004
Frame time 194194 (on scale 60000)
Frame time 196196 (on scale 60000)
Frame time 198198 (on scale 60000)
Frame time 200200 (on scale 60000)
Frame time 202202 (on scale 60000)
Frame time 206206 (on scale 60000)
Missing frame. Time delta is (on scale 60000): 4004
Frame time 208208 (on scale 60000)
Frame time 210210 (on scale 60000)


Relevant code excerpt:

BMDTimeValue lastFrameTime = 0;

....

HRESULT STDMETHODCALLTYPE VideoInputFrameArrived(IDeckLinkVideoInputFrame *videoFrame, IDeckLinkAudioInputPacket *audioPacket) {

if (videoFrame != NULL) {
if (videoFrame->GetFlags() & bmdFrameHasNoInputSource) {
printf("VideoInputFrameArrived: NoInputSource\n");
fflush(stdout);
return S_OK;
}
BMDTimeValue frameTime, frameDuration;
videoFrame->GetStreamTime(&frameTime, &frameDuration, 60000);
printf("Frame time %ld (on scale 60000)\n", frameTime);
if (frameTime > lastFrameTime + 2100) {
printf(" Missing frame\n");
}
lastFrameTime = frameTime;
fflush(stdout);
}
return S_OK;
}

[Matt Jefferson]

Ray -
Are you using 608 close captions as your source? Or are you using 708 as below from our SDK using the format specified below?

To capture or output CEA-708 captions, a 10 bit pixel format such as bmdFormat10BitYUV must be used

The reason why you don't see 608 is that Standard Def close captioning starts in line 21 which is not vertical blanking but in the active picture region. So you would not use the VBI of the API and would need to use a 3rd party decoder.

If you are having a problem with 10bit YUV using as you state with Cea708 then please contact us on the developer support for SDK email support. The email is on page 13 of the SDK Manual.

I actually figured this out late last night.

It turned out that though the card physically fits into a x2 PCIe slot, for Windows at least, there were reliability issues. Moving to an x4 slot resolved the problems.

For those curious: The VANC line 9 data contains both CEA 608 (2 bytes per frame) and 708 (many more bytes per frame), so that wasn't it. The 608 now decodes correctly for me.

And for what it's worth -- compared against a pro VANC decoder appliance, the DeckLink card is 2 frames behind -- pretty nifty!

-Ray

Just wanted to add -- I now have something working with both VANC line 9 and Line 21 in the video itself. So I can now handle all US formats.


To help others, I have a list of pixel locations for reading the data out. Below is a code excerpt with the relevant info. Capture with bmdModeNTSC/bmdFormat8BitYUV. I consider Y values of 0 to 38 (inclusive) to be zero bit, 90 to 165 (inclusive) to be one bit. This is slightly more generous than the minimum required decoder IRE ranges specified in CEA-608 (assuming that IRE 0 maps to 8-bit value 0 and IRE 100 maps to 255).

In practice, I find Line 21 on line index 1&2 of captured content.

UNIT8 b[2];

// 7 clock run in
if (!checkPixelValues(pixelValues, 10, 14, 0)) break;
if (!checkPixelValues(pixelValues, 22, 29, 1)) break;
if (!checkPixelValues(pixelValues, 37, 41, 0)) break;
if (!checkPixelValues(pixelValues, 49, 56, 1)) break;
if (!checkPixelValues(pixelValues, 64, 67, 0)) break;
if (!checkPixelValues(pixelValues, 76, 82, 1)) break;
if (!checkPixelValues(pixelValues, 90, 94, 0)) break;
if (!checkPixelValues(pixelValues, 103, 109, 1)) break;
if (!checkPixelValues(pixelValues, 117, 121, 0)) break;
if (!checkPixelValues(pixelValues, 129, 136, 1)) break;
if (!checkPixelValues(pixelValues, 144, 148, 0)) break;
if (!checkPixelValues(pixelValues, 156, 163, 1)) break;
if (!checkPixelValues(pixelValues, 171, 175, 0)) break;
if (!checkPixelValues(pixelValues, 183, 190, 1)) break;
// 001 start
if (!checkPixelValues(pixelValues, 198, 243, 0)) break;
if (!checkPixelValues(pixelValues, 249, 172, 1)) break;

// character 1
if (!updateBitReturnValid(pixelValues, 276, 296, 0, &b[0])) break;
if (!updateBitReturnValid(pixelValues, 303, 323, 1, &b[0])) break;
if (!updateBitReturnValid(pixelValues, 330, 350, 2, &b[0])) break;
if (!updateBitReturnValid(pixelValues, 357, 377, 3, &b[0])) break;
if (!updateBitReturnValid(pixelValues, 384, 404, 4, &b[0])) break;
if (!updateBitReturnValid(pixelValues, 411, 431, 5, &b[0])) break;
if (!updateBitReturnValid(pixelValues, 437, 457, 6, &b[0])) break;
if (!updateBitReturnValid(pixelValues, 464, 484, 7, &b[0])) break;

// character 2
if (!updateBitReturnValid(pixelValues, 491, 511, 0, &b[1])) break;
if (!updateBitReturnValid(pixelValues, 518, 538, 1, &b[1])) break;
if (!updateBitReturnValid(pixelValues, 545, 565, 2, &b[1])) break;
if (!updateBitReturnValid(pixelValues, 572, 592, 3, &b[1])) break;
if (!updateBitReturnValid(pixelValues, 598, 618, 4, &b[1])) break;
if (!updateBitReturnValid(pixelValues, 625, 645, 5, &b[1])) break;
if (!updateBitReturnValid(pixelValues, 652, 672, 6, &b[1])) break;
if (!updateBitReturnValid(pixelValues, 679, 699, 7, &b[1])) break;

// trail off at end
if (!checkPixelValues(pixelValues, 707, 715, 0)) break;

.... now b has the 2 bytes to be decoded by a 608 decoder..... if there were any errors, break would have occurred.

Ray, could you read CEA-708 closed captions with success?

Best regards,

André

Were you able to output captions? Chasing unicorns over here..