Blackmagic Forum

Hi,

I bought a decklink mini recorder and I'm trying to create an application in WPF - C# (.NET) (vs2013 s.o. windows 10)

My application has two preview sections called "preview hdmi" and "preview sdi" and 2 buttons called "hdmi in" and "sdi in".
When I click on the "hdmi in" button (connected external camera with 1920i60 resolution) I reproduce the video in input in the "preview hdmi" section.
When I click on the "sdi in" button (connected external camera with 1920i60 resolution) I play the video in input in the "preview sdi" section.

I'd like to get something similar to the CapturePreview example, but unfortunately I do not know the C++ language

I helped with the only example in C# SignalGenCSharp

I have read Chapter 2.4.1 Capture Manual and relying on the SignalGenCSharp project have been able to receive calls to IDeckLinkInputCallback::VideoInputFrameArrived with video frame and corresponding audio packet.

In the next steps I have difficulty understanding how to display (on monitor) frames received from callback VideoInputFrameArrived


Seeing the CapturePreview project within the VideoInputFrameArrived method calls the GetAncillaryDataFromFrame Method 5 times by referencing frame timecodes and userbits and after calling the UpdateAncillaryData method.
 
My problem is I do not know how to translate these 2 methods into C# and also i do not know which element to use to preview (mediaelement? PictureBox?) (I've read Chapter 2.4.7 Ancillary Data Functionality and VANC Capture, but unfortunately I did not help much)

Thanks in advance for the help.

Good morning,

After 10 days no one can give me an answer.

The question is not clear?

Thanks in advance.

The challenge with your question is that it covers such a lot of ground, it is difficult to answer quickly. I can attempt to summarise and then you could refine the question.

IDeckLinkInputCallback::VideoInputFrameArrived delivers you a IDeckLinkVideoInputFrame object.

This frame object has GetRowBytes to retrieve a pointer to the raw pixel data and a GetStreamTime method to retrieve the timecode.

The pixel data is an array of pixels in whatever pixel format you selected arranged as an array of width x height x sizeof of a pixel in bytes.

In principal you then point your display method at the raw pixel data and then it copies it somewhere for display. Alternatively you create a bitmap, copy and transform the pixel format of the input to the bitmap format.

Once you have copied the pixels, you return from the arrived function.

Some other thread then has to display the copy on the screen.

Wow... this is more complicated than I expected. Like the original poster, I've been trying to create a similar (winform) app. I too got to the point of receiving video and audio data in the VideoInputFrameArrived callback. However, I could not find any examples explaining how to render the raw data.

I've cobbled something together with Directshow using the Blackmagic WDM Capture Device, AVI Decompressor, VMR9 and default audio device. While this solution works, I don't particularly like it. Just wondering if the original poster or anyone else has a more elegant solution they'd be willing to share.

Hi Olias + Giovanni,

Please refer to the sample application CapturePreviewCSharp, this has been active in the SDK since 10.9.5.

This sample application differs from previous CapturePreview samples, in that rendering is performed with DirectX.Direct3D via the IDeckLinkDX9ScreenPreviewHelper interface[1][2].

The render task should be called from IDeckLinkScreenPreviewCallback::DrawFrame callback[3], rather than from VideoInputFrameArrived

Regards
Cameron

Refs
[1] Blackmagic DeckLink SDK - 2.5.25 IDeckLinkDX9ScreenPreviewHelper Interface
[2] Blackmagic DeckLink SDK - 2.5.25.2 IDeckLinkDX9ScreenPreviewHelper::Render method
[3] Blackmagic DeckLink SDK - 2.5.22.1 IDeckLinkScreenPreviewCallback::DrawFrame method

For what it's worth, I know you're using Winforms but if you have the ability to use WPF you could use a WriteableBitmap which allows you to write raw data to its internal buffer and display that on screen. In WPF, it's quite easy for that reason.

I tried the Csharp Capture Preview sample in the new 10.9.5 SDK.
I am getting a black screen???
No errors! Any ideas?

LarsWesselius wrote:For what it's worth, I know you're using Winforms but if you have the ability to use WPF you could use a WriteableBitmap which allows you to write raw data to its internal buffer and display that on screen. In WPF, it's quite easy for that reason.

Thanks for the suggestion. Unfortunately I'm limited to winforms. I suppose I could always try hosting a WriteableBitmap with an ElementHost within my form. Not sure if it would work... but it feels like I'd be asking for trouble

Salem Jabri wrote:I tried the Csharp Capture Preview sample in the new 10.9.5 SDK.
I am getting a black screen???
No errors! Any ideas?

To be honest, I haven't spent too much time with the C# CapturePreview sample yet. Initially, it wouldn't even compile for me due to the Microsoft.DirectX references. When I have some time, I may try refactoring it to use ShapDX or SlimDX. Sorry I cannot be of more help at this time.

Olias Becker wrote:

Salem Jabri wrote:I tried the Csharp Capture Preview sample in the new 10.9.5 SDK.
I am getting a black screen???
No errors! Any ideas?

To be honest, I haven't spent too much time with the C# CapturePreview sample yet. Initially, it wouldn't even compile for me due to the Microsoft.DirectX references. When I have some time, I may try refactoring it to use ShapDX or SlimDX. Sorry I cannot be of more help at this time.

I went ahead and updated my copy of the C# CapturePreview sample to use SharpDX. I don't see a way to attach my solution here, so I'll post my code below. You'll also need to include the SharpDX and SharpDX.Direct3D9 Nuget packages. I'm using VS 2017 btw. I've had some trouble including SharpDX packages in 2015.

Code: Select all

// PreviewWindow.cs

using System;
using System.Drawing;
using System.Windows.Forms;
using DeckLinkAPI;
using Direct3D = SharpDX.Direct3D9;
using SharpDX.Mathematics.Interop;

namespace CapturePreviewCSharp
{
    public partial class PreviewWindow : Control, IDeckLinkScreenPreviewCallback
    {
        private IDeckLinkDX9ScreenPreviewHelper m_previewHelper;
        private string                          m_timeCodeString;

        private Direct3D.Device                 m_d3DDevice;
        private Direct3D.Font                   m_d3DFont;

        public PreviewWindow()
        {
            m_previewHelper = new CDeckLinkDX9ScreenPreviewHelper();
            InitializeComponent();
        }

        public void InitD3D()
        {
            var d3dpp = new Direct3D.PresentParameters();
            d3dpp.BackBufferFormat = Direct3D.Format.Unknown;
            d3dpp.BackBufferCount = 2;
            d3dpp.Windowed = true;
            d3dpp.SwapEffect = Direct3D.SwapEffect.Discard;
            d3dpp.DeviceWindowHandle = this.Handle;
            d3dpp.PresentationInterval = Direct3D.PresentInterval.Default;

            m_d3DDevice = new Direct3D.Device(new Direct3D.Direct3D(), 0, Direct3D.DeviceType.Hardware, this.Handle, Direct3D.CreateFlags.HardwareVertexProcessing | Direct3D.CreateFlags.Multithreaded, d3dpp);

            m_d3DDevice.Reset(d3dpp);
            device_DeviceReset(m_d3DDevice, null);

            unsafe
            {
                m_previewHelper.Initialize(m_d3DDevice.NativePointer);
            }
        }

        void device_DeviceReset(object sender, EventArgs e)
        {
            Direct3D.FontDescription systemfont = new Direct3D.FontDescription() { FaceName = "Arial",  PitchAndFamily = Direct3D.FontPitchAndFamily.Mono, Width = 14, Height = 14, Weight = Direct3D.FontWeight.Regular };
            m_d3DFont = new Direct3D.Font(m_d3DDevice, systemfont);
        }

        protected override void OnPaint(PaintEventArgs e)
        {
            base.OnPaint(e);
        }

        void Render()
        {
            m_d3DDevice.BeginScene();

            tagRECT rect;
            rect.top = m_d3DDevice.Viewport.Y;
            rect.left = m_d3DDevice.Viewport.X;
            rect.bottom = m_d3DDevice.Viewport.Y + m_d3DDevice.Viewport.Height;
            rect.right = m_d3DDevice.Viewport.X + m_d3DDevice.Viewport.Width;

            m_previewHelper.Render(rect);

            // Draw the timecode top-center with a slight drop-shadow
            RawRectangle rc = m_d3DFont.MeasureText(null, m_timeCodeString, Direct3D.FontDrawFlags.Center);
            int x = (m_d3DDevice.Viewport.Width / 2) - (Math.Abs(rc.Right - rc.Left) / 2);
            int y = 10;
            m_d3DFont.DrawText(null, m_timeCodeString, x + 1, y + 1, new RawColorBGRA(Color.Black.B, Color.Black.G, Color.Black.R, Color.Black.A));
            m_d3DFont.DrawText(null, m_timeCodeString, x, y, new RawColorBGRA(Color.White.B, Color.White.G, Color.White.R, Color.White.A));

            m_d3DDevice.EndScene();
            m_d3DDevice.Present();
        }

        void SetTimecode(IDeckLinkVideoFrame videoFrame)
        {
            IDeckLinkTimecode timecode;

            m_timeCodeString = "00:00:00:00";

            videoFrame.GetTimecode(_BMDTimecodeFormat.bmdTimecodeRP188Any, out timecode);

            if (timecode != null)
                timecode.GetString(out m_timeCodeString);
        }

        void IDeckLinkScreenPreviewCallback.DrawFrame(IDeckLinkVideoFrame theFrame)
        {
            // First, pass the frame to the DeckLink screen preview helper
            m_previewHelper.SetFrame(theFrame);
            SetTimecode(theFrame);

            // Then draw the frame to the scene
            Render();

            System.Runtime.InteropServices.Marshal.ReleaseComObject(theFrame);
        }
    }
}


One more thing... the CapturePreview samples (both C# and C++) display video but have no audio. Does anyone know how to get the audio working and synced with the video?

Thank you.
I changed my code as well and used SharpDX.
It is running with no errors. But I am still getting a black video screen with timecode running!!!
The only way to see the video is using openGL. the Visual C++ sample is working.
Any one has a C#code with openGL code?

Any one has tried the CSharp Capture Preview???
I guess I am the only one having this issue.
No video being displayed!!!!

Hi Olias,

The DeckLink API consists of 2 callbacks interfaces that can be called to draw incoming frame:
1) IDeckLinkScreenPreviewCallback [1]:

• This is implemented in CapturePreviewCSharp sample application.
• Callback Delivery rate may be limited by DrawFrame rendering, so frames may be dropped
• Audio packets are not synchronised in callback

2) IDeckLinkInputCallback [2]:

• Callback delivery rate is same as frame rate
• Audio packets are synchronised to video frame in callback

To allow processing input audio packets synchronised to video, I suggest moving to IDeckLinkInputCallback::VideoInputFrameArrived callback.

Even without calling IDeckLinkInput::SetScreenPreviewCallback, you can still create a member of type IDeckLinkScreenPreviewCallback and call DrawFrame with the incoming frame. Please take caution though, the call to DrawFrame could take longer than input video frame period, so you will need to call DrawFrame from a separate thread than the IDeckLinkInputCallback::VideoInputFrameArrived callback.

Kind Regards
Cameron Nichols

Refs
[1] Blackmagic DeckLink API - 2.5.22 IDeckLinkScreenPreviewCallback Interface
[2] Blackmagic DeckLink API - 2.5.10 IDeckLinkInputCallback Interface

Thanks for your reply Cameron... coincidentally, what you've outlined is precisely what I'm doing, so at least I know I'm on the right track You've also confirmed that I am not dealing with some sort audio / video timecode sync issue. That said, what I'm observing seems to be approximately a 100 millisecond latency between my audio and video, where the video is ahead of the audio. I'm trying to determine if the delay is increasing over time versus being consistently 100 milliseconds. What I can't figure out is why I have this delay in the first place given that I'm receiving synchronized audio and video in the IDeckLinkInputCallback.VideoInputFrameArrived callback. Hardware issue perhaps? Do you have any suggestions, other than introducing some sort configurable offset (video delay)? Again, your continued assistance is appreciated. I've posted some relevant snippets from my code below. Thanks

DeckLinkDevice.cs

Code: Select all

   
        private CSCore.Streams.WriteableBufferingSource WritableBufferingSource { get; set; }
        private CSCore.SoundOut.ISoundOut SoundOut { get; set; }

        public DeckLinkDevice(IDeckLink deckLink)
        {
            m_deckLink = deckLink;
            m_deckLinkInput = (IDeckLinkInput)m_deckLink;

            this.WritableBufferingSource = new CSCore.Streams.WriteableBufferingSource(new CSCore.WaveFormat((int)_BMDAudioSampleRate.bmdAudioSampleRate48kHz, (int)_BMDAudioSampleType.bmdAudioSampleType16bitInteger, 2));
            this.SoundOut = new CSCore.SoundOut.WasapiOut(false, CSCore.CoreAudioAPI.AudioClientShareMode.Shared, 1, ThreadPriority.Highest, SynchronizationContext.Current);
            this.SoundOut.Initialize(this.WritableBufferingSource);
            this.SoundOut.Play();
        }
     
        void IDeckLinkInputCallback.VideoInputFrameArrived(IDeckLinkVideoInputFrame videoFrame, IDeckLinkAudioInputPacket audioPacket)
        {
            Parallel.Invoke(() => this.ProcessAudio(audioPacket), () => this.ProcessVideo(videoFrame));
        }

        private void ProcessAudio(IDeckLinkAudioInputPacket audioPacket)
        {
            if (audioPacket != null)
            {
                IntPtr payload;
                audioPacket.GetBytes(out payload);
                var dataBlockSize = 4 * audioPacket.GetSampleFrameCount();
                var bytes = new byte[dataBlockSize];
                Marshal.Copy(payload, bytes, 0, dataBlockSize);
                this.WritableBufferingSource.Write(bytes, 0, bytes.Length);

                System.Runtime.InteropServices.Marshal.ReleaseComObject(audioPacket);
            }
        }

        private void ProcessVideo(IDeckLinkVideoFrame videoFrame)
        {
            if (videoFrame != null)
            {
                bool inputSignal = videoFrame.GetFlags().HasFlag(_BMDFrameFlags.bmdFrameHasNoInputSource);
                if (inputSignal != m_validInputSignal)
                {
                    m_validInputSignal = inputSignal;
                    InputSignalChanged(m_validInputSignal);
                }
                else
                {
                    this.PreviewWindow.RenderFrame(videoFrame);
                }

                System.Runtime.InteropServices.Marshal.ReleaseComObject(videoFrame);
            }
        }


PreviewWindow.cs

Code: Select all

        public void RenderFrame(IDeckLinkVideoFrame theFrame)
        {
            m_previewHelper.SetFrame(theFrame);
            m_d3DDevice.BeginScene();
            m_d3DDevice.Viewport = new RawViewport() { X = this.Bounds.Left, Y = this.Bounds.Top, Width = this.Bounds.Width, Height = this.Bounds.Height };
            m_previewHelper.Render(new tagRECT { left = this.Bounds.Left, top = this.Bounds.Top, right = this.Bounds.Width, bottom = this.Bounds.Height });
            m_d3DDevice.EndScene();
            m_d3DDevice.Present();

            System.Runtime.InteropServices.Marshal.ReleaseComObject(theFrame);
        }


Hi Olias,

Can you please confirm the measured offset via DeckLinkVideoInputFrame::GetStreamTime() [1] IDeckLinkAudioInputPacket::GetPacketTime() [2] commands?

Regards
Cameron

Refs:
[1] DeckLink SDK Manual, 2.5.11.1 IDeckLinkVideoInputFrame::GetStreamTime method
[2] DeckLink SDK Manual, 2.5.12.3 IDeckLinkAudioInputPacket::GetPacketTime method

LarsWesselius wrote:For what it's worth, I know you're using Winforms but if you have the ability to use WPF you could use a WriteableBitmap which allows you to write raw data to its internal buffer and display that on screen. In WPF, it's quite easy for that reason.

Dear Lars,

Can you post us some code example in order to show your idea ? I don't sucess to display picture with WPF...

Thanks in advance.