IDeckLinkDeviceNotificationCallback does not work properly

In order to be notified when USB or Thunderbolt DeckLink/Ultrastudio devices are added or removed from a system, I'm using IDeckLinkDeviceNotificationCallback, as described in the DeckLink SDK documentation, to be informed of device arrival or removal. Device arrival works fine, and my callback object's DeckLinkDeviceArrived() method is called appropriately, but device removal doesn't work, and my callback object's DeckLinkDeviceRemoved() method is never called. Even though DeckLinkDeviceRemoved() isn't called, if I iterate through the DeckLink objects (using IDeckLinkIterator) immediately after device removal, it is clear that the object has been removed, and that's actually how I'm working around the issue, by periodically checking to see if the device has been removed through iteration in the case that I'm dealing with a USB or Thunderbolt device. Plus, the device disappears from Device Manager and Intel's Thunderbolt software device list. Plus, I see a WM_DEVICECHANGE windows message sent to my window when the device is unplugged from the system. So, it would seem that all the information should be available to Blackmagic to detect device removal, and this tends to point to a bug in the Blackmagic software.

This happens on Windows 10 Professional 64-bit on a Intel NUC NUC6i7KYK, on which I'm running the latest BIOS and latest drivers, including the latest Thunderbolt driver from Intel. Since the computer has a Thunderbolt 3 port, I'm using a Startech Thunderbolt 3 to Thunderbolt adapter to connect a Blackmagic UltraStudio Mini Recorder to the system.

On a separate note, there isn't a good way to determine if a particular IDeckLink object is associated with the same underlying hardware device as another IDeckLink object. For example, when I use IDeckLinkIterator to iterate through all the IDeckLink objects, the object pointer associated with the same underlying hardware device typically changes from call to call, as would generally be expected. I tried to use the BMDDeckLinkPersistentID attribute ID as a unique ID, but, at least when it comes to the UltraStudio Mini Recorder, this device doesn't have a persistent ID. The approach that I've settled upon is to use IDeckLink::GetModelName()--if the model names returned are the same, then I interpret this to mean that I'm dealing with the same underlying hardware device. But, I don't think that this approach is guaranteed to produce uniqueness, and it would be preferable if a better approach were available. I also tried using the undocumented BMDDeckLinkDeviceHandle, which supposedly returns a string, but the software crashed in DeckLink code when I attempted to use it, and I believe that I passed in correct arguments.

[Nicholas Gill]

Hi Aaron,

The behaviour of IDeckLinkDeviceNotificationCallback that you describe is the expected behaviour.

DeckLinkDeviceArrived[1] is called to inform an application of the arrival of a DeckLink device.

If the application is interested in this device, it is expected to call AddRef on the provided IDeckLink device and store the device for processing.

If the application is not interested in this device, it can simply ignore the device by not retaining a reference to the device.

DeckLinkDeviceRemoved[2] is then called for any IDeckLink instance for which the application currently holds a reference. If the application does not currently hold a reference to the mentioned IDeckLink instance, it will not receive notification of the removal of that device.

To achieve the desired behaviour, your application should call AddRef on the IDeckLink instance in DeckLinkDeviceArrived in order to hold a reference to the instance, then if the device is removed the application will receive a DeckLinkDeviceRemoved callback as long as it holds a reference to that device.

Regarding identifying devices, devices which don't support BMDDeckLinkPersistentID may support BMDDeckLinkTopologicalID - an identifier for the device based on how it is connected to the system. For the purposes of identifying two references to the same device this is the next best option when BMDDeckLinkPersistentID is unavailable.

As not all devices support BMDDeckLinkTopologicalID, your workaround to identify devices by the model name will work for situations involving multiple distinct device models.

BMDDeckLinkDeviceHandle is an opaque identifier that conceptually combines the topological and persistent identifiers to represent the best available identifier for a particular device.

Could you please show how your application attempts to query BMDDeckLinkDeviceHandle?

It should be possible to query BMDDeckLinkDeviceHandle as follows, where deviceHandle is a platform specific string type, and noting that the application must check for success before dereferencing the deviceHandle string:

Code: Select all

   HRESULT result = deckLinkAttributes->GetString(BMDDeckLinkDeviceHandle, &deviceHandle);
   if (result == S_OK)
      //... use and release when finished


Hope that helps,

-nick

[1] 2.5.29.1 IDeckLinkDeviceNotificationCallback::DeckLinkDeviceArrived method
[2] 2.5.29.2 IDeckLinkDeviceNotificationCallback::DeckLinkDeviceRemoved method

The behaviour of IDeckLinkDeviceNotificationCallback that you describe is the expected behaviour.

DeckLinkDeviceArrived[1] is called to inform an application of the arrival of a DeckLink device.

If the application is interested in this device, it is expected to call AddRef on the provided IDeckLink device and store the device for processing.

If the application is not interested in this device, it can simply ignore the device by not retaining a reference to the device.

DeckLinkDeviceRemoved[2] is then called for any IDeckLink instance for which the application currently holds a reference. If the application does not currently hold a reference to the mentioned IDeckLink instance, it will not receive notification of the removal of that device.

I can work with this, although I think it is non-intuitive and also not documented in the SDK manual. By non-intuitive, I mean that this approach is inconsistent with other similar notification APIs available, such as similar notification APIs from Microsoft. The closest that the SDK comes to mentioning this is the following, from the documentation for the DeckLinkDeviceArrived() method: "The IDeckLink reference will be released when the callback returns. To hold on to it beyond the callback, call AddRef. Your application then owns the IDeckLink reference and is responsible for managing the IDeckLink object’s lifetime." But, that isn't anywhere close to saying that you must call AddRef() in order for DeckLinkDeviceRemoved() to ever be called for the same underlying hardware device. There is also no mention of this AddRef business in section 2.4.4 of the manual, which discusses the general procedure for DeckLink device notification. Also, presumably, in order for it to call DeckLinkDeviceArrived(), an IDeckLink object must be constructed by the DeckLink libraries internally--I don't see why another one couldn't be constructed for DeckLinkDeviceRemoved(), and using this approach would eliminate the need to call AddRef().

BMDDeckLinkDeviceHandle is an opaque identifier that conceptually combines the topological and persistent identifiers to represent the best available identifier for a particular device.

Could you please show how your application attempts to query BMDDeckLinkDeviceHandle?

It should be possible to query BMDDeckLinkDeviceHandle as follows, where deviceHandle is a platform specific string type, and noting that the application must check for success before dereferencing the deviceHandle string:

Code: Select all

   HRESULT result = deckLinkAttributes->GetString(BMDDeckLinkDeviceHandle, &deviceHandle);
   if (result == S_OK)
      //... use and release when finished


I used the following, on a Windows platform:

Code: Select all

CComBSTR deviceHandle;
HRESULT hr = pAttributes->GetString(BMDDeckLinkDeviceHandle, &deviceHandle);


It crashed in the call to GetString()--when I stepped through the disassembly, it looked like it crashed when it was trying to retrieve something from the stack. This is with Desktop Video 10.6.8 and the 10.6.6 SDK files, all the latest as of 6/24/2016. This is also with a Blackmagic UltraStudio Mini Recorder.

Thanks,
Aaron

I'm experiencing the same problems as the original poster.

Thank god i found this discussion.

However, now I cannot find a way to query my devices to verify that they are still healthy.

I receive the Removed call back, and increment a status flag.

Later on, asynchronously, I check my flag to see if anything changed, but how do I check to see that my
IDeckLinkInput or my IDeckLink reference is still valid ??

I don't want to tear down and rebuild my input streams unless I absolutely have to.
I don't want to end up in a situation where I tear down and rebuild over and over again, especially
because your APIs have massive memory leaks and I have no help tracking them down.
Why don't you just open source the entire API? The value of your product is your hardware. This opaque API just makes it impossible to track down bugs.

[Nicholas Gill]

Hi Keith,

The DeckLinkDeviceRemoved callback indicates that the IDeckLink instance passed as a parameter has been removed, and all objects created from it (via QueryInterface, etc.) must be released.

The application will generally maintain the relationship between the IDeckLink instance and related objects (e.g. IDeckLinkInput), so that this cleanup happens automatically as the video operation is stopped.

Could you please provide a code sample which reproduces the memory leaks you observe?

Thanks,

-nick

It sounds like what you are saying is that there is no way to query an existing IDeckLink or IDeckLinkInput reference to verify that it is still alive.
If I'm wrong about this, please post sample code that does this.

In general,
It would be *much* more useful if *you* were to post sample code that *doesn't* leak.
What code do you use to test your API?
Please post the code that you use to test your API.
I can only assume that you are unwilling to post any test code because

A) you haven't written it
B) it is not bulletproof because of race conditions and multi-threading issues.
C) it is so complicated and intertwined that there is no hope of understanding it.

It would take me a long time to post my DeckLink specific sample code, as it is over 2000 lines of code that is tied into the rest of our codebase, and contains safe multi-threading.
However, here are a few snippets:

Code: Select all

if([key hasPrefix:@"VideoInput"]) { performBlockOnDrawThread( ^{ ControlVideoInput(key, val); } ); return; }

Code: Select all

   if([[params objectForKey:@"SourceType"] isEqual:@"Blackmagic"]) mode = kDeckLinkMode;
   else if([[params objectForKey:@"SourceType"] isEqual:@"DeckLink"]) mode = kDeckLinkMode;
   else if([[params objectForKey:@"SourceType"] isEqual:@"Phoenix"]) mode = kASLphoenixMode;
   else if([[params objectForKey:@"SourceType"] isEqual:@"QuickTime"]) mode = kQTKitMode;

Code: Select all

                    if (mode == kDeckLinkMode && interested &&
                        discoveryIncrement != gDecklinkDiscoveryIncrement )
                    {
                        discoveryIncrement = gDecklinkDiscoveryIncrement;
                        if ( decklinkinput ) decklinkinput->CheckStatus();
                    }

Code: Select all

void DeckLinkInput::CheckStatus(void)
{
    if ( lostSourceCount > 150 || !decklinkinput )
    {
        parent->newFormat = [[parent->setupDict objectForKey:@"SourceFormat"] retain];
        parent->formatChanged = [NSDate timeIntervalSinceReferenceDate] + 1;
        return; // already reported lost source
    }
   
    if ( decklinkinput )
    {
        /* gah there does not appear to be any way to verify the status of a decklinkinput
        HRESULT result = decklinkinput->StartStreams();
       
        // card name still valid??? NSString *verifyCardName = nil;
        //HRESULT result = decklink->GetModelName((CFStringRef*)&verifyCardName);
        // if ( cardName && ![cardName isEqualToString:@"Unknown DeckLink Card"] )
        // {
        //
        if ( result != S_OK )
        {
            printf( "result 0x%x\n", result );

            parent->setErrorString((char*)"(No Video Input)");
           
            if(parent) parent->useSubstituteImage(YES);
           
            updateStatus();
           
            //   printf("lost video %d\n", inputnum);
           
            lostSource = YES;
            lostSourceCount = 200;
        }
         */
    }
}

Code: Select all

static IDeckLinkDiscovery* sDeckLinkDiscovery;
UInt32 gDecklinkDiscoveryIncrement;
class DeckLinkDiscovery : IDeckLinkDeviceNotificationCallback
{
    virtual HRESULT DeckLinkDeviceArrived (/* in */ IDeckLink* deckLinkDevice)
    {
        gDecklinkDiscoveryIncrement++;
        // according to blackmagic forum post, you must call AddRef() in order for
        // DeckLinkDeviceRemoved to be called!
        deckLinkDevice->AddRef();
        if (!gLastDisplayCallback ) gLastDisplayCallback = mach_absolute_time();
        return S_OK;
    }
   
    virtual HRESULT DeckLinkDeviceRemoved (/* in */ IDeckLink* deckLinkDevice)
    {
        gDecklinkDiscoveryIncrement++;
        if (!gLastDisplayCallback ) gLastDisplayCallback = mach_absolute_time();
        return S_OK;
    }
   
    virtual HRESULT QueryInterface (REFIID iid, LPVOID *ppv) {return E_NOINTERFACE;}
    virtual ULONG AddRef (void) {return 1;}
    virtual ULONG Release (void) {return 1;}
};
static DeckLinkDiscovery *sDeckLinkDiscoveryNotify;

still struggling here... please have an answer before I return from lunch

To answer Nicholas Gill's question about memory leaks, I'm not aware of any memory leaks in the implementation of the core Blackmagic APIs on Windows, i.e., those implemented as COM objects on Windows and installed by the Blackmagic DeckLink driver package. It will consume a certain amount of memory when Blackmagic is initialized by software, but when de-initialized, I see the memory footprint go down by approximately the same amount it went up by when Blackmagic was initialized. Perhaps there are some memory leaks in Blackmagic's sample code that is included with the SDK, but even if there are, while they ought to be fixed if they exist, that's not such a big deal. We are talking about sample code, and I've seen much worse sample code out there. However, overall, I've found Blackmagic's sample code in the SDK to be pretty good and very helpful at determining how to use certain Blackmagic programming facilities.

Keith: you expected to get a response from someone an hour after you posted your code, which I think is a bit unreasonable. Based on when Nicholas Gill has posted in the past, my guess is that Nicholas isn't based in the US, so if you were expecting a response from him earlier than this evening, you are going to have to wait.

Nevertheless, I've basically already alluded to the answer to your question for how to detect if a Blackmagic device is no longer available. You used the term "healthy" in your first post, but, based on my own experience, if the Blackmagic device is still plugged into the system, it is going to be working, and only when it is unplugged is it not working. Here's how you can detect that a device has been removed from the system or is no longer working without using IDeckLinkDeviceNotificationCallback:

Code: Select all

         bool bFoundSameDevice = false;

         CComPtr<IDeckLinkIterator> pDeckLinkIterator;
         pDeckLinkIterator.CoCreateInstance(CLSID_CDeckLinkIterator);
         if (pDeckLinkIterator)
         {
            CComPtr<IDeckLink> pDeckLink;
            HRESULT hr = pDeckLinkIterator->Next(&pDeckLink);
            while (hr == S_OK)
            {
               CComBSTR modelName;
               pDeckLink->GetModelName(&modelName);
               if (modelName == m_pDeckLinkInputCallback->m_modelName)
               {
                  bFoundSameDevice = true;
                  break;
               }

               pDeckLink.Release();
               hr = pDeckLinkIterator->Next(&pDeckLink);
            }
         }

         if (!bFoundSameDevice)
         {
            CComPtr<IDeckLinkInput> pDeckLinkInput(m_pDeckLinkInputCallback->m_pDeckLinkInput);
            HRESULT hr = pDeckLinkInput->StopStreams();
            hr = pDeckLinkInput->DisableVideoInput();
            hr = pDeckLinkInput->DisableAudioInput();
            hr = pDeckLinkInput->SetCallback(0);

            m_pDeckLinkInputCallback = 0;
         }


For this to work, it needs to be called at regular intervals. Basically, you are continually iterating through the DeckLink input devices to detect when a device is removed. Alternatively, you can use IDeckLinkDeviceNotificationCallback to do this, although I personally only use IDeckLinkDeviceNotificationCallback to detect when devices are added, since I had already implemented this workaround for the removal issue when I wrote the original post and saw no need to remove it when Nicholas provided his response.

AaronL

Thx AaronL for the response.

On Mac OS, I'm convinced the leaks related to the DeckLink drivers
cause my app to become a zombie process...
implying that its beyond a memory leak... its a system resource leak.

I've posted about leaks, thread safety, and lag in the past,
and have gotten vague responses at best.
Every time I revisit this SDK its hours of frustration and guesswork.

They need to provide complete sample applications with source code.

BlackMagic is missing out on $$$$$ because its too painful to manage
multiple BlackMagic devices plugged into the same computer.

Ok I think I finally solved my immediate problem.
Still not certain about memory leaks, but at least this should uniquely identify devices.

Code: Select all

           int64_t persist_id = 0;
           if((err = card->QueryInterface(IID_IDeckLinkAttributes, (void**)&attr) == S_OK))
            {
                attr->GetInt(BMDDeckLinkPersistentID, &persist_id);
                if(!persist_id) attr->GetInt(BMDDeckLinkTopologicalID, &persist_id);
                attr->Release();
            }

[Nicholas Gill]

Hi Keith,

(I've responded to some of the comments out of order, but hopefully the discussion flows in context)

It sounds like what you are saying is that there is no way to query an existing IDeckLink or IDeckLinkInput reference to verify that it is still alive.
If I'm wrong about this, please post sample code that does this.

Your understanding is correct that there is no method to determine if the device has been removed; this is the specific purpose of the DeckLinkDeviceRemoved [1] callback.

However, here are a few snippets:

Thanks for providing the sample code, based on a quick reading, I observe some unusual API usage and behaviour by the application.

For example in DeckLinkDeviceArrived [2], the application holds a reference to every discovered DeckLink device, but appears to do nothing with the discovered object. How does your application track and release the references to these objects later?

I presume this is related to the unusual behaviour in the implementation of DeckLinkDeviceRemoved where no explicit action is performed to trigger removal of the device nor release the reference held in DeckLinkDeviceArrived (i.e. the reference is leaked by the application).

Ok I think I finally solved my immediate problem.
Still not certain about memory leaks, but at least this should uniquely identify devices.

The code shown suggests using the Persistent / Topological IDs [3] to associate the DeckLink device from IDeckLinkDiscovery [4] with the DeckLink device used elsewhere in your application.

This is a workable solution, but I believe it is only necessary because your application does not use the DeckLink device as notified via the IDeckLinkDiscovery interface. If it did, pointer equality of the provided parameter would be sufficient to associate the IDeckLink [5] instances.

Guessing that your application is using a combination of IDeckLinkDiscovery and IDeckLinkIterator [6], the solution using the available IDs is workable where the device in use has either a Persistent or Topological ID.

It would be *much* more useful if *you* were to post sample code that *doesn't* leak.

On Mac OS, I'm convinced the leaks related to the DeckLink drivers
cause my app to become a zombie process...
implying that its beyond a memory leak... its a system resource leak.

I am not aware of any leaks present with correct DeckLink API usage, nor aware of any leaks in the provided DeckLink API samples and examples included in the DeckLink SDK package.

If you observe a leak in these sample applications, could you please identify which sample / example (including the platform) has a memory / resource leak?

If you observe a leak in otherwise correct DeckLink API usage (i.e. where the application releases all the DeckLink API objects), could you please provide a small piece of sample code which reproduces the issue?

I've posted about leaks, thread safety, and lag in the past,
and have gotten vague responses at best.

If the communication was direct with Blackmagic Design, I apologise that you have received vague responses in the past. I also prefer precise technical communication, particularly relating to DeckLink API reference counting and thread safety.

Unfortunately I have been unable to correct previous responses you have received, as I have been unable to locate any previous correspondence on these topics either on the forum, nor via the Developer Support email address.

Could you please elaborate on the issues you have encountered?

Every time I revisit this SDK its hours of frustration and guesswork.

They need to provide complete sample applications with source code.

BlackMagic is missing out on $$$$$ because its too painful to manage
multiple BlackMagic devices plugged into the same computer.

I hope that this is simply a misunderstanding. The DeckLink SDK package includes a number of small examples, and some larger, platform-specific sample applications, all provided with source.

The DeckLink SDK Manual makes reference to samples which demonstrate particular behaviour, and contains sections describing both high-level overviews of common operations, and detailed semantics of individual methods.

I observe that your application is written in Objective-C, please see the SDK Mac CapturePreview sample application (Mac/Samples/CapturePreview). Relating to this topic, I recommend in particular reviewing the implementation of IDeckLinkDeviceNotificationCallback methods DeckLinkDeviceArrived & DeckLinkDeviceRemoved in DeckLinkController.mm.

Cheers,

-nick

[1] 2.5.29.2 IDeckLinkDeviceNotificationCallback::DeckLinkDeviceRemoved method
[2] 2.5.29.1 IDeckLinkDeviceNotificationCallback::DeckLinkDeviceArrived method
[3] 2.7.17 DeckLink Attribute ID
[4] 2.5.28 IDeckLinkDiscovery Interface
[5] 2.5.2 IDeckLink Interface
[6] 2.5.1 IDeckLinkIterator Interface

Thanks for the reply, Nick. (belated thanks 3 years later)
The deckLinkDevice->AddRef() call was the key piece of undocumented info I needed.

virtual HRESULT DeckLinkDeviceArrived (/* in */ IDeckLink* deckLinkDevice)
{
// according to blackmagic forum post, you must call AddRef() in order for
// DeckLinkDeviceRemoved to be called!
deckLinkDevice->AddRef();

ChangeDetection(deckLinkDevice);
return S_OK;
}