ATEM: Confusing YUVA Pixel Format Documentation

The BMDSwitcherPixelFormat::bmdSwitcherPixelFormat10BitYUVA is documented as follows

BMDSwitcherPixelFormat::bmdSwitcherPixelFormat10BitYUVA
Eight bytes per two pixels, a0, cb, y0, a1, cr, y1.

Says 8 bytes but has 6 values. Is there padding? The name is enum value contains YUV, but the explanatory sentence has Cb and Cr values. Below are the values I'm getting back assuming a char size byte and casting the result as an (int).

There are 2 stills created in Photoshop the first is RGB(0,0,0) and the 2nd is RGB(255,255,255) uploaded with the BM PS tool.

Here is the code:

Code: Select all

//Grab the frame above ^
auto pixelF = frame->GetPixelFormat();

 
  switch (pixelF) {
   case bmdSwitcherPixelFormat8BitARGB:
    puts("Four bytes per pixel, alpha, red, green, blue.");
    break;
   case bmdSwitcherPixelFormat8BitXRGB:
    puts("Four bytes per pixel, padding, red, green, blue.");
    break;
   case bmdSwitcherPixelFormat8BitYUV:
    puts("Four bytes per two pixels, cb, y0, cr, y1.");
    break;
   case bmdSwitcherPixelFormat10BitYUVA:
    puts("Eight bytes per two pixels, a0, cb, y0, a1, cr, y1.");
    break;
  }

  char* frameData = nullptr;
  frame->GetBytes((void**)&frameData);
 
  for(int i = 0; i < 8; i++){

    std::cout << (int)frameData[i] << "\n";

  }


When the frame is targeting the black still output is:

Code: Select all

Eight bytes per two pixels, a0, cb, y0, a1, cr, y1.
58
-104
0
64
58
-104
0
64


White Still:

Code: Select all

Eight bytes per two pixels, a0, cb, y0, a1, cr, y1.
58
-105
-1
-87
58
-104
3
-87


These number seem no nonsensical. I would expect everything to be zero except the padding byte for black still and everything to be zero except matching large Y (indicating the luminance upper bound) values in the white still.

Any ideas?

Here is what I think I know:

BMDSwitcherPixelFormat::bmdSwitcherPixelFormat10BitYUVA is 64bit little-endian word that is a double of the bit layout of BMDPixelFormat::bmdFormat10BitYUV.

Since there is no documentation on the bit layout for BMDSwitcherPixelFormat::bmdSwitcherPixelFormat10BitYUVA, I've been printing out the binary in 8 bit chunks from different single color stills and looking for patterns. For White, Black, and Red solids there was a 00 in the padding position of the BMDPixelFormat::bmdFormat10BitYUV documentation. I using this assumption took 6 10bit chunks and calculated the base ten ints from this pattern, some of the values seemed sensible.

However, I'm still getting wacky colors when I output a png using Magick++; White appears pink and black dark green. This could be due to a misunderstanding of Magick++/Converting to RGB from YCbCr or being incorrect in my assumptions concerning the bit layout.

Wish I could get confirmation from BM on the bit layout

I got something that worked for me for downloading images from de media pool.

What worked for me is:
Read an UInt32 from the frame, then
bits 0-9 is for y0
bits 10-19 is for cb
bits 20-29 is for a0

again read an UInt32 from the frame, then
bits 0-9 is for y1
bits 10-19 is for cr
bits 20-29 is for a1

Then make your pixels with (y0,cb,cr) and (y1,cb,cr)

you can also add the alpha channels with a0 and a1 for each pixel. But that jus gave me a washed out image, so I ignored that, and used 255 instead.

Here is some very basic code that worked for me, and also did the RGB conversion:

Code: Select all

                        for (int yIndex = 0; yIndex < frame.GetHeight(); yIndex++)
                        {
                            for (int xIndex = 0; xIndex < frame.GetWidth(); xIndex+=2)
                            {
                                //Read 32 bits for the first pixel of a pair
                                UInt32 Pixel0 = (UInt32)((UInt32)buffer[subPixIndex++] << 24);
                                Pixel0 |= (UInt32)((UInt32)buffer[subPixIndex++] << 16);
                                Pixel0 |= (UInt32)((UInt32)buffer[subPixIndex++] << 8);
                                Pixel0 |= (UInt32)((UInt32)buffer[subPixIndex++]);

                                //Read 32 bits for the second pixel of a pair
                                UInt32 Pixel1 = (UInt32)((UInt32)buffer[subPixIndex++] << 24);
                                Pixel1 |= (UInt32)((UInt32)buffer[subPixIndex++] << 16);
                                Pixel1 |= (UInt32)((UInt32)buffer[subPixIndex++] << 8);
                                Pixel1 |= (UInt32)((UInt32)buffer[subPixIndex++]);

                                //Get parts from first pixel
                                UInt16 y0 = (UInt16)(Pixel0 & 0x3ff);
                                UInt16 cb = (UInt16)((Pixel0 >> 10) & 0x3ff);
                                UInt16 a0 = (UInt16)((Pixel0 >> 20) & 0x3ff);

                                //Get parts from second pixel
                                UInt16 y1 = (UInt16)(Pixel1 & 0x3ff);
                                UInt16 cr = (UInt16)((Pixel1 >> 10) & 0x3ff);
                                UInt16 a1 = (UInt16)((Pixel1 >> 20) & 0x3ff);

                                //YUV to RGB conversion 1st pixel
                                int C = y0 - 64;
                                int D = cb - 512;
                                int E = cr - 512;

                                int R = (1192 * C + 1636 * E + 512) >> 10;
                                int G = (1192 * C - 400 * D - 832 * E + 512) >> 10;
                                int B = (1192 * C + 2064 * D + 512) >> 10;
                                if (R < 0) R = 0; else if (R > 1023) R = 1023;
                                if (G < 0) G = 0; else if (G > 1023) G = 1023;
                                if (B < 0) B = 0; else if (B > 1023) B = 1023;

                                //Create a color pixel
                                Color pixel = new Color();
//                                pixel = Color.FromArgb(a0 / 4, R / 4, G / 4, B / 4);
                                pixel = Color.FromArgb(255, R / 4, G / 4, B / 4);//Ignore alpha
                                //Set pixel in the image
                                image.SetPixel(xIndex, yIndex, pixel);

                                //YUV to RGB conversion 2nd pixel
                                C = y1 - 64;
                                R = (1192 * C + 1636 * E + 512) >> 10;
                                G = (1192 * C - 400 * D - 832 * E + 512) >> 10;
                                B = (1192 * C + 2064 * D + 512) >> 10;
                                if (R < 0) R = 0; else if (R > 1023) R = 1023;
                                if (G < 0) G = 0; else if (G > 1023) G = 1023;
                                if (B < 0) B = 0; else if (B > 1023) B = 1023;

                                //Create a color pixel
//                                pixel = Color.FromArgb(a1 / 4, R / 4, G / 4, B / 4);
                                pixel = Color.FromArgb(255, R / 4, G / 4, B / 4);//Ignore alpha

                                //Set pixel in the image
                                image.SetPixel(xIndex+1, yIndex, pixel);

                            }
                        }


[Brendan Dower]

Hi Michael,

Due to the nature of YUV, you need 2 pixels worth of information to define each pixel.
The first pixel is actually a0, y0, cb0, cr0 and the second pixel is a1, y1, cb1, cr1.

However, cb1 and cr1 can be calculated by interpolating cb0, cr0 with cb3, cr3 (from the next pixel).
Therefore we only need a0, cb(0) y0 and a1, y1, cr(0).

You need 10 bits per component, which comes to 30 bits total per pixel (4 bytes).
Adding the 2nd pixel gives you 8 bytes per 2 pixels, which matches the documentation.

Peter is correct with his breakdown of which bits are used for each component.

Kind regards,
Brendan Dower
Developer Support Engineer