Capture frames, processing and display

Ask software engineering and SDK questions for developers working on Mac OS X, Windows or Linux.
  • Author
  • Message
Offline

Bukaj25

  • Posts: 2
  • Joined: Wed Jan 08, 2020 3:50 pm
  • Real Name: Jakub Zbikowski

Capture frames, processing and display

PostWed Jan 08, 2020 4:00 pm

Hello,

I would like to make software which capture frames from camera connected to Blackmagic DeckLink Quad 2, process these with OpenCV and display they on PC screen. I analized CapturePreview and LiveProcessing simple programs, but still I haven't any idea to resolve this problem. Eventually i will be forced to create my own schedule playback, but maybe someone has some clue how resolve this problem.
Offline

Cameron Nichols

Blackmagic Design

  • Posts: 442
  • Joined: Mon Sep 04, 2017 4:05 am

Re: Capture frames, processing and display

PostThu Jan 09, 2020 2:38 am

Hi Jakub,

Could you please elaborate on the issue you are seeing. The Live Video Processing with OpenCV sample should be the best example to start from. You should be particularly interested in the movement of IDeckLinkVideoInputFrame to cv::Mat in the IDeckLinkInputCallback::VideoInputFrameArrived callback.

Were you able to get the sample running?

Regards
Cameron
Offline

Bukaj25

  • Posts: 2
  • Joined: Wed Jan 08, 2020 3:50 pm
  • Real Name: Jakub Zbikowski

Re: Capture frames, processing and display

PostThu Jan 09, 2020 11:40 am

I build Live Video Processing project and it return error "Could not start video output", because I have not any output. As output I would like some dialog window like in sample "Capture Preview". I comment in Live Video Processing code responsible for output so I can stop debugging in VideoInputFrameArrived, but unfortunately I get black/empty frames. To resolve my problem I think about to create my own ScheduledPlayback which will display video on PC display with using cv::imshow function. Before I start program it I wonder there is some ready solution of this issue, because own ScheduledPlayback looks very hard to do.
Offline

DtrRon

  • Posts: 6
  • Joined: Tue Jan 28, 2020 7:13 pm
  • Real Name: Ron Valli

Re: Capture frames, processing and display

PostWed Jan 29, 2020 4:08 pm

I'm having problems with the "Live Video With OpenCV" sample code. I got it to compile after I changed the reference from cv::CV_HAAR_SCALE_IMAGE to cv::CASCADE_SCALE_IMAGE to match my code release.

During runtime I get this error:

Capture failed: getWidthHeightForDisplayMode: did not find mode 879440688

I don't see "getWidthHeightForDisplayMode" reference in the SDK.

I'm using a Intel, Ubuntu 18.04, OpenCV2 with a Decklink Duo 2 card.

Any help would be greatly appreciated.
Offline

Cameron Nichols

Blackmagic Design

  • Posts: 442
  • Joined: Mon Sep 04, 2017 4:05 am

Re: Capture frames, processing and display

PostThu Jan 30, 2020 7:17 am

Hi Ron,

Capture::getWidthHeightForDisplayMode is defined in LiveVideoWithOpenCV.cpp, it checks the requested display mode (enum BMDDisplayMode) against the IDeckLinkDisplayMode interface objects that are supported by the device. Here BMDDisplayMode = 879440688 = 0x346b3330 = '4k30' (see DeckLinkAPIModes.h header) which is not supported by the DeckLink Duo 2.

Change the following line (LiveVideoWithOpenCV.cpp|line 551):
Code: Select all
auto        capture       = Capture::CreateInstance(deckLinkInput, deckLinkOutput, bmdMode4K2160p30)
to bmdModeHD1080p6000 or similar.

Regards
Cameron
Offline

DtrRon

  • Posts: 6
  • Joined: Tue Jan 28, 2020 7:13 pm
  • Real Name: Ron Valli

Re: Capture frames, processing and display

PostThu Jan 30, 2020 3:52 pm

Hi Cameron,

Thanks for the quick comeback. Okay that worked, but now I am getting the error: "Capture failed: Could not enable video output" from...

Code: Select all
   if (FAILED(m_deckLinkOutput->EnableVideoOutput(m_displayMode, bmdVideoOutputFlagDefault)))
      throw std::runtime_error("Could not enable video output");

Without visibility into that method, I'm stuck once again. Is the "output" to the DeckLink card?

My goal is 12 cameras on three DeckLink cards that I can manipulate the images using OpenCV. Media Express works great with all 12 cameras.

Thanks,
Ron
Offline

Cameron Nichols

Blackmagic Design

  • Posts: 442
  • Joined: Mon Sep 04, 2017 4:05 am

Re: Capture frames, processing and display

PostFri Jan 31, 2020 5:27 am

Hi Ron,

The LiveVideoWithOpenCV demo provides both a capture interface and an output interface. The output displays the captured video with rectangle overlay for face detection, as well as metrics on the classification.

The reason for EnableVideoOutput failing will likely be one of 2 things:
  • You have already enabled the IDeckLinkOutput in another process
  • You have configured the DeckLink Duo 2 device as half duplex, so you cannot enable input and output simultaneously.
If it is the latter, then you may want to temporary reconfigure the Duo 2 for full-duplex operation. Later when you want to achieve 12-inputs (with all devices in half-duplex mode), then you will need to strip the output processing from the code.

Kind Regards
Cameron Nichols
Offline

DtrRon

  • Posts: 6
  • Joined: Tue Jan 28, 2020 7:13 pm
  • Real Name: Ron Valli

Re: Capture frames, processing and display

PostFri Jan 31, 2020 2:49 pm

Hi Cameron,

Well, that makes sense, but when I change it to Full Duplex (2 inputs, 2 outputs) I get "Could not enable video input".

Using Desktop Video Setup:
Code: Select all
[b]FULL DUPLEX:      Code Error: "Could not enable video input[/b]"
Label             Video in          Video out       Gen Lock   
Decklink Duo (1)    SDI:1080p30     1080p30           -             
Decklink Duo (2)    SDI:1080p30     1080p30           -       
Decklink Duo (3)    SDI              -                -   
Decklink Duo (4)    SDI              -                -         


[b]HALF DUPLEX:           Code Error: "Could not enable video output[/b]"
Label             Video in          Video out       Gen Lock     
Decklink Duo (1)    SDI             1080p30           -             
Decklink Duo (2)    SDI             1080p30           -       
Decklink Duo (3)    SDI             1080p30           -   
Decklink Duo (4)    SDI             1080p30           -

In the half duplex mode: it can enable the inputs, but fails to enable the output. This seems correct because there isn't any outputs to enable.

In full duplex mode: It can't enable any inputs. Is the code picking the wrong inputs?

Media Express (Full Duplex) finds two devices: 1 & 2 and can show the camera images.

Thanks again for your help,
Ron
Offline

DtrRon

  • Posts: 6
  • Joined: Tue Jan 28, 2020 7:13 pm
  • Real Name: Ron Valli

Re: Capture frames, processing and display

PostFri Jan 31, 2020 8:46 pm

Hello,

Here is a follow-on post to my last one. I decided that since I don't even want an output port, I commented out the lines to "enable" and "start" the output. Then I had some OpenCV install issues. After solving those, I have an empty (black) window with the key command list.

The waitKey set at waitKey(), it just waits for a keystroke and never update the image. Mat and imshow are outside any while (true) loop.

I must be missing something.
Code: Select all
capture->StartDeckLinkCapture();

// Display an image with instructions for the key press commands
int      lineNumber = 1;
cv::Mat image      = cv::Mat::zeros(776, 800, CV_8UC1);
.
.
.
imshow("Live Video Command Window", image);

// Loop forever, processing key presses
int keyCode;

while ((keyCode = cv::waitKey()) != 'q')
{
   switch (keyCode)
   {
      case '0': capture->setProcessMode(kProcessModeBlack); break;
      .
      .
      .
      default: break;
   }
}

// Quit was pressed so stop capture
capture->StopDeckLinkCapture();


Thanks for the help,
Ron
Offline

Cameron Nichols

Blackmagic Design

  • Posts: 442
  • Joined: Mon Sep 04, 2017 4:05 am

Re: Capture frames, processing and display

PostThu Feb 06, 2020 11:51 pm

Hi Ron,

Please ensure you checked that there is actually a valid signal on the wire, you may want to run SDK sample CapturePreview to test this. Next check that your incoming signal is YUV 422 - which if expected by demo, and not RGB 444.

If you are debugging, you can check this by adding breakpoint inside Capture::VideoInputFormatChanged, you will want to look at the BMDDetectedVideoInputFormatFlags parameter (replace /* unused */ with variable), mask against bmdVideoInputColorspaceChanged to check whether RGB 444 is detected.

Also, in same callback, what is value that is returned by newDisplayMode->GetDisplayMode() ?

Please let me know whether this helps.

Regards
Cameron
Offline

DtrRon

  • Posts: 6
  • Joined: Tue Jan 28, 2020 7:13 pm
  • Real Name: Ron Valli

Re: Capture frames, processing and display

PostThu Feb 13, 2020 10:24 pm

Hi Cameron,

I have not had any luck with getting this to work. It doesn't look like that callback is being accessed. I wrote up a more detailed email and sent it in to the support address. If you can take a look at that I would greatly appreciate it.

Best Regards,
Ron

Return to Software Developers

Who is online

Users browsing this forum: No registered users and 2 guests