Blackmagic Forum

Hi, I am new to decklink SDK and I am trying to capture&preview video+audio from camera using UltraStudio HD Mini.

I made video preview working but I had issues with paying sound from camera in real time. Sound is out of sync and faster than real sound. I noticed this when I buffered multiple packets of sound and played them together.

I am capturing audio and video in IDeckLinkInputCallback.VideoInputFrameArrived. For audio playback I am using NAudio.

Code: Select all


        public uint channelCount { get; set; } = 2;
        public _BMDAudioSampleType sampleType { get; set; } = _BMDAudioSampleType.bmdAudioSampleType16bitInteger;
        public int sampleTypeInt { get; set; } = 16;
        private WaveFormat waveFormatTarget = new WaveFormat(48000, sampleTypeInt, (int) channelCount);
        private BufferedWaveProvider waveProvider;
        private WaveOut waveOut = new();
        private Queue<byte[]> audioSamplesQueue = new();



Code: Select all

            waveOut.Init(waveProvider);
            waveOut.Play();
            waveOut.Volume = 1;


Code: Select all

private void BufferThread()
        {
            Task.Run(() =>
            {
                while (true)
                {
                    Thread.Sleep(10000);
                    try
                    {
                        while (audioSamplesQueue.Count > 0)
                        {
                            var buffer = audioSamplesQueue.Dequeue();

                            waveProvider.AddSamples(buffer, 0, buffer.Length);

                            memoryStream.Write(buffer);
                            memoryStream.Flush();
                        }
                    }
                    catch (Exception e)
                    {
                        Console.WriteLine(e);
                    }
                }
            });
        }


Code: Select all

            if (audioPacket != null)
            {
                audioPacket.GetBytes(out var inputAudioBuffer);

                if (inputAudioBuffer != IntPtr.Zero)
                {
                    var sampleCount = audioPacket.GetSampleFrameCount();
                    var bufferLength = waveFormatTarget.BitsPerSample / 8 * waveFormatTarget.Channels * sampleCount;
                    var tempBuffer = new byte[bufferLength];

                    audioPacket.GetPacketTime(out var packetTime, 10 * 1000 * 1000);

                    Marshal.Copy(inputAudioBuffer, tempBuffer, 0, bufferLength);

                    Marshal.ReleaseComObject(audioPacket);

                    audioSamplesQueue.Enqueue(tempBuffer);
                }
                else
                {
                    Console.WriteLine();
                }
            }

Hi Michal,

Can you instead try holding onto the IDeckLinkAudioInputPacket reference and store this in a queue. Later you can call Marshal.Copy/Queue.Enqueue in your processing thread.

My concern is that you should keep IDeckLinkInputCallback::VideoInputFrameArrived as lightweight as possible and that these methods will be relatively processor intensive with copy and reallocations.

You can test this theory by keeping log of the difference in the timestamps from IDeckLinkInput::GetHardwareReferenceClock called at the start of callback with IDeckLinkVideoInputFrame::GetHardwareReferenceTimestamp. This will provide the time for callback to occur as compared to the End of Frame on the wire.

If you see this difference grow, then it would suggest the callback is not keeping up with the frame rate. This may lead to later dropping audio samples as the input buffer fills.

Regards
Cameron

I made example with winui 3 preview. It just work (for a while). Feel free to inspire. Repository here:
https://github.com/nono3551/BM-Capture