Screen tearing with minimal example

Ask software engineering and SDK questions for developers working on Mac OS X, Windows or Linux.
  • Author
  • Message
Offline

LarsWesselius

  • Posts: 4
  • Joined: Wed Apr 19, 2017 8:33 am

Screen tearing with minimal example

PostThu May 18, 2017 12:40 pm

Hello,

I'm encountering some screen tearing when outputting 1920x1080p30 to a TV screen. I have included the full code below (C#). It shows a vertical bar 25 pixels wide moving across the screen. If I increase my CPU usage (artificially) the screen tearing starts to become extremely apparent.
Code: Select all
using DeckLinkAPI;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;

namespace DeckLinkTestFrameTearing
{
    class Program : IDeckLinkVideoOutputCallback
    {
        static void Main(string[] args)
        {
            new Program();
        }

        public Program()
        {
            Start();
            Console.ReadLine();
        }

        private IDeckLink _deckLinkCard;
        private IDeckLinkOutput _deckLinkOutput;

        private long _totalFramesScheduled;
        private long _frameDuration, _frameTimescale;

        public int Width = 1920;
        public int Height = 1080;


        private int _barLoc = 0;
        private object _lock = new object();
        private int _frameCounter = 0;


        private IDeckLinkMutableVideoFrame _deckLinkVideoFrame, _deckLinkTest;

        public void Start()
        {
            IDeckLinkIterator deckLinkIterator = new CDeckLinkIterator();
           
            deckLinkIterator.Next(out _deckLinkCard);


            _deckLinkOutput = (IDeckLinkOutput)_deckLinkCard;
            _deckLinkOutput.SetScheduledFrameCompletionCallback(this);
            _frameDuration = 1000;
            _frameTimescale = 30000;
            _deckLinkOutput.EnableVideoOutput(_BMDDisplayMode.bmdModeHD1080p30, _BMDVideoOutputFlags.bmdVideoOutputFlagDefault);

            _deckLinkOutput.CreateVideoFrame(Width, Height, Width * 4, _BMDPixelFormat.bmdFormat8BitBGRA, _BMDFrameFlags.bmdFrameFlagDefault, out _deckLinkVideoFrame);
            _deckLinkOutput.CreateVideoFrame(Width, Height, Width * 4, _BMDPixelFormat.bmdFormat8BitBGRA, _BMDFrameFlags.bmdFrameFlagDefault, out _deckLinkTest);

            FillBlackBGRA(_deckLinkVideoFrame);

            for (uint i = 0; i != 25; ++i)
            {
                _deckLinkOutput.ScheduleVideoFrame(_deckLinkVideoFrame, _totalFramesScheduled * _frameDuration, _frameDuration, _frameTimescale);
                ++_totalFramesScheduled;
            }


            _deckLinkOutput.StartScheduledPlayback(0, 100, 1.0);
        }
       

        public void ScheduledFrameCompleted(IDeckLinkVideoFrame completedFrame, _BMDOutputFrameCompletionResult result)
        {
            lock (_lock)
            {
                IDeckLinkVideoFrame frame = _deckLinkTest;

                IntPtr buffer;
                frame.GetBytes(out buffer);

                int width, height;
                int index = 0;

                width = frame.GetWidth();
                height = frame.GetHeight();


                int barWidth = 25;
                unchecked
                {
                    for (uint y = 0; y < height; y++)
                    {
                        for (uint x = 0; x < width; x++)
                        {
                            if (x > _barLoc && x < _barLoc + barWidth)
                            {
                                Marshal.WriteInt32(buffer, index * 4, (int)0xFFFFFFFF);
                            }
                            else
                            {
                                Marshal.WriteInt32(buffer, index * 4, (int)0x00000000);
                            }
                            ++index;
                        }
                    }
                }

                _barLoc += barWidth;
                if (_barLoc >= width)
                {
                    _barLoc = 0;
                }


                _deckLinkOutput.ScheduleVideoFrame(frame, _totalFramesScheduled * _frameDuration, _frameDuration, _frameTimescale);



                ++_totalFramesScheduled;

                ++_frameCounter;
                if (_frameCounter >= 25)
                {
                    _frameCounter = 0;
                }
            }
        }

        public void ScheduledPlaybackHasStopped()
        {
        }

        private void FillBlackBGRA(IDeckLinkVideoFrame frame)
        {
            IntPtr buffer;
            frame.GetBytes(out buffer);

            int size = Width * Height * 4;
            byte[] bytes = new byte[size];

            int index = 0;
            for (int x = 0; x != Height; ++x)
            {
                for (int y = 0; y != Width; ++y)
                {
                    bytes[index] = 0;
                    ++index;
                    bytes[index] = 0;
                    ++index;
                    bytes[index] = 0;
                    ++index;
                    bytes[index] = 255;
                    ++index;
                }
            }

            Marshal.Copy(bytes, 0, buffer, size);
        }
    }
}


Does anyone have an idea why this could be happening? Much appreciated
Offline

Martin Kibsgaard

  • Posts: 4
  • Joined: Wed Jul 02, 2014 1:00 pm

Re: Screen tearing with minimal example

PostThu May 18, 2017 7:00 pm

My guess is that ScheduledFrameCompleted might be called on a new thread even if the previous call to the function hasn't returned yet, resulting in ScheduleVideoFrame outputting the frame from the old thread while the new thread is in the middle of writing to the same buffer.
To avoid that you could have multiple buffers, but in any case you would probably need to drop frames if your processing is slower than the time between frames being output (e.g. more than 33 ms for 1080i5994). It might be easier to not use scheduled playback in cases where you can't keep up the frame rate and instead just use DisplayVideoFrameSync as often as you have a frame available. Although, this will most likely cause a bit of stutter because once in a while the same frame will be displayed twice to fill in the gaps.
Offline

Nicholas Gill

Blackmagic Design

  • Posts: 168
  • Joined: Mon May 04, 2015 10:28 pm

Re: Screen tearing with minimal example

PostTue May 23, 2017 5:21 am

Hi Lars,

Thanks for providing a small code sample.

In the code shown, note that the preroll loop is scheduling the same IDeckLinkVideoFrame instance, 25 times, e.g. after this preroll loop, the frame queue looks like this (Simplified to three frames):

A = _deckLinkVideoFrame

Frame queue:
Code: Select all
AAA...


When playback is started, the DeckLink starts outputting the head item in the frame queue (A), when this completes, this frame instance is returned to the application:
Code: Select all
ScheduledFrameCompleted(A)


The application then proceeds to overwrite the video buffer for completed frame A.

Frame queue:
Code: Select all
AA...


(@Martin Kibsgaard: Just as a point of clarification, ScheduledFrameCompleted will not be called concurrently from different thread contexts.)

And the DeckLink device starts outputting the next video frame (A).

The issue occurs because the reference to A which your application is overwriting is the same as the reference that the DeckLink device is reading from, the application effectively is overwriting the video frame buffer while the API is using it.

The solution in this case is to create and schedule _distinct_ IDeckLinkVideoFrame instances.

E.g. Remove references to _deckLinkVideoFrame & _deckLinkTest, and call CreateVideoFrame in the preroll loop.

Now, after preroll the frame queue will be similar to:
Code: Select all
ABC...


Head of queue is output and completes:
Code: Select all
ScheduledFrameCompleted(A)


Application then fills video buffer for completed frame A.

Frame queue:
Code: Select all
BC...

The DeckLink is then outputting frame B while the application is overwriting frame A.

Hope that helps,

-nick
Offline

LarsWesselius

  • Posts: 4
  • Joined: Wed Apr 19, 2017 8:33 am

Re: Screen tearing with minimal example

PostMon May 29, 2017 1:33 pm

Hello,

Thank you both for your explanations!

Based on your feedback, I'm now using a ringbuffer type system as found in the LoopThroughWithDX11Compositing example and that seems to work very very well. Adjusting the preroll now allows me to tweak 'stability' as it will become less CPU dependant the higher the preroll (more time to prepare frames).

Now I've got to time the audio to the (now delayed) video frames..

Thanks again!

Return to Software Developers

Who is online

Users browsing this forum: No registered users and 3 guests