- Posts: 6
- Joined: Fri Sep 03, 2021 3:33 pm
- Real Name: Kerry Loux
Tue Mar 04, 2025 5:30 pm
Hello,
I am attempting to render audio using the ScheduleAudioSamples() method. Roughly, playback is configured by doing (in this order):
At this point, I can check the pre-rolled frame counts (note that my video frame rate is 29.97 Hz):
which prints:
Playback continues with:
I continue to schedule video frames in the ScheduledFrameCompleted() callback, and video playback works fine.
At the same time, I intend to schedule video frames in the RenderAudioSamples() callback. The callback fires as expected, however the buffer size I observe indicates that the audio is never pulled from the buffer. No audio is actually rendered. My RenderAudioSamples() callback is below:
During execution, the following output is generated:
Oddly, bufferedFrameCount does drop by one right after scheduled playback begins, then there are no further changes to the value bufferedFrameCount during playback. Video playback occurs as expected. I expected to see bufferedFrameCount drop as playback occurs, where we would periodically add frames to the buffer to maintain our nominal buffer size.
What steps am I missing to get audio to actually play?
Thanks,
Kerry
I am attempting to render audio using the ScheduleAudioSamples() method. Roughly, playback is configured by doing (in this order):
- Code: Select all
IDeckLinkOutput::SetScheduledFrameCompletionCallback()
IDeckLinkOutput::EnableVideoOutput()
IDeckLinkOutput::ScheduleVideoFrame() (three times)
IDeckLinkOutput::SetAudioCallback()
IDeckLinkOutput::EnableAudioOutput()
IDeckLinkOutput::BeginAudioPreroll()
within the RenderAudioSamples() callback:
IDeckLinkOutput::ScheduleAudioSamples() (three times, as shown below)
IDeckLinkOutput::EndAudioPreroll()
At this point, I can check the pre-rolled frame counts (note that my video frame rate is 29.97 Hz):
- Code: Select all
unsigned int a, v;
deckLinkOutput->GetBufferedAudioSampleFrameCount(&a);
deckLinkOutput->GetBufferedVideoFrameCount(&v);
outStream << "Buffered frame counts are audio = " << a << " and video = " << v << std::endl;
which prints:
- Code: Select all
Buffered frame counts are audio = 4803 and video = 3
Playback continues with:
- Code: Select all
IDeckLinkOutput::StartScheduledPlayback()
I continue to schedule video frames in the ScheduledFrameCompleted() callback, and video playback works fine.
At the same time, I intend to schedule video frames in the RenderAudioSamples() callback. The callback fires as expected, however the buffer size I observe indicates that the audio is never pulled from the buffer. No audio is actually rendered. My RenderAudioSamples() callback is below:
- Code: Select all
HRESULT STDMETHODCALLTYPE AudioOutput::RenderAudioSamples(BOOL preroll)
{
if (!playbackRunning)
return S_OK;
unsigned int bufferedFrameCount;
if (APIUtilities::CallHasError(deckLinkOutput->GetBufferedAudioSampleFrameCount(&bufferedFrameCount), _T("Failed to check audio buffer size"), outStream))
return E_FAIL;
outStream << "AudioOutput::RenderAudioSamples: bufferedFrameCount = " << bufferedFrameCount << "; target = " << targetBufferedAudioCount << std::endl;
if (bufferedFrameCount < targetBufferedAudioCount)
{
const unsigned int framesToAdd((targetBufferedAudioCount - bufferedFrameCount + audioSamplesPerVideoFrame - 1) / audioSamplesPerVideoFrame);
for (unsigned int i = 0; i < framesToAdd; ++i)
{
VideoReader::AudioPacketData nextPacket;
if (videoReader.GetNextAudioPacket(nextPacket))
{
if (!nextPacket.buffer || nextPacket.sampleCount == 0)
{
playbackRunning = false;
return S_OK;
}
outStream << "Scheduling frame with " << nextPacket.sampleCount << " samples at " << scheduledFrameCount * audioSamplesPerVideoFrame << std::endl;
if (APIUtilities::CallHasError(deckLinkOutput->ScheduleAudioSamples(nextPacket.buffer, nextPacket.sampleCount,
scheduledFrameCount * audioSamplesPerVideoFrame, sampleRateHz, nullptr), _T("Failed to schedule audio samples"), outStream))
return E_FAIL;
if (nextPacket.buffer)
delete[] nextPacket.buffer;
++scheduledFrameCount;
}
}
}
else if (preroll)
{
outStream << "Audio preroll complete" << std::endl;
if (APIUtilities::CallHasError(deckLinkOutput->EndAudioPreroll(), _T("Failed to end audio preroll"), outStream))
return E_FAIL;
prerollComplete = true;
prerollCompleteCondition.notify_all();
}
return S_OK;
}
During execution, the following output is generated:
- Code: Select all
2025-03-04 11:35:31.567 : Video source frame rate = 30000 / 1001
2025-03-04 11:35:31.671 : Configuring audio output for 2 channels and 16 bits per sample
2025-03-04 11:35:31.678 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 0; target = 4803
2025-03-04 11:35:31.683 : Scheduling frame with 1601 samples at 0
2025-03-04 11:35:31.688 : Scheduling frame with 1601 samples at 1601
2025-03-04 11:35:31.692 : Scheduling frame with 1601 samples at 3202
2025-03-04 11:35:31.699 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 4803; target = 4803
2025-03-04 11:35:31.702 : Audio preroll complete
2025-03-04 11:35:31.710 : Buffered frame counts are audio = 4803 and video = 3
2025-03-04 11:35:31.716 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 4802; target = 4803
2025-03-04 11:35:31.721 : Scheduling frame with 1601 samples at 4803
2025-03-04 11:35:31.741 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 6404; target = 4803
2025-03-04 11:35:31.763 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 6404; target = 4803
2025-03-04 11:35:31.780 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 6404; target = 4803
2025-03-04 11:35:31.799 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 6404; target = 4803
2025-03-04 11:35:31.826 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 6404; target = 4803
2025-03-04 11:35:31.838 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 6404; target = 4803
2025-03-04 11:35:31.858 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 6404; target = 4803
2025-03-04 11:35:31.878 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 6404; target = 4803
2025-03-04 11:35:31.899 : AudioOutput::RenderAudioSamples: bufferedFrameCount = 6404; target = 4803
...
Oddly, bufferedFrameCount does drop by one right after scheduled playback begins, then there are no further changes to the value bufferedFrameCount during playback. Video playback occurs as expected. I expected to see bufferedFrameCount drop as playback occurs, where we would periodically add frames to the buffer to maintain our nominal buffer size.
What steps am I missing to get audio to actually play?
Thanks,
Kerry