ATEM Mini Pro drops On-Air mode

A couple of Sundays ago our ATEM mini-pro dropped its On-Air connection mid-service, and is the second time in a month this has happened. Before this we never had a problem with it. The mini-pro outputs directly to the our website host, who then sends it to FB. We have 3 camera inputs and one input from our ProPresenter 7 computer, which is a Mac mini that outputs from its Thunderbolt output through a UltraStudio Monitor 3G then to the ATEM. The ATEM drops out of its On-Air connection, and we cannot re-enable the On-Air connection by pressing the ON-AIR button on the ATEM switcher. The only way we can re-connect is by power cycling the ATEM, and after we do that, and then pressing the ON-AIR button, it connects just fine. I don't know if this is a hardware issue or firmware issue, or something else. I'm hoping BMD can offer some help. I'm also hoping that someone here has had the same or similar problem with their Mini Pro and can offer some direction. Thanks everyone!

We have a similar setup and similar problems, as does another church that has the same setup as our church. Many others have reported issues like this. For us the cache will start filling up and there are no known upload issues when this occurs. The workaround for us is to pull the network cable from the atem when cache gets in the 40's(it empties faster if you don't let it get to high), after it empties plug it back in and it will resume.

I have a ticket in with support, we are convinced their is a bug in the streaming engine.

Ed

I'm going to demo my mini pro to a church in a few weeks. They are not streaming and want to avoid facebook and youtube due to political reasons. Are you only streaming to youtube? From your post and others about this I'm wondering if it's youtube receiving only that has problems.

A friend who owns a mini says he thinks the mini changes output resolution based on input. I have not looked into this but thought it was worth mentioning.

If you never change camera inputs does the stream drop out?

I would start by saving your settings on the computer that runs the ATEM software, and then opening up the .XML file and looking at the "user name" and "password" for the streaming service that you are trying to use. This is pre-loaded with generic entries, which in our case, worked fine with Facebook back in Sept but failed every time since then.

When I entered the actual "user name" and "password" into the .XML file, saved it and then "restored" that into the ATEM, Facebook was then able to use them to check against the streaming key, and only after that happened was the ATEM stream visible inside of Facebook. This also stopped the blinking "On-Air" button (bad connection) and cache filling (streaming permission) issues, at least in our case.

Hey jay thompson,

We stream simultaneously to our website host and FB. Since the original post, this hasn't happened again. I don't believe it is an input related error; we were on a camera for a while when it dropped out. The cache building up as mentioned by eheiger seems like a possible cause of our problem. We'll keep checking in on it and let you all know if we find out anything.

This is most likely related to the network / streaming / recording issue first identified and discussed as far back as July 2020…

…see https://forum.blackmagicdesign.com/viewtopic.php?f=4&t=117185&start=50#p828312

We are rapidly approaching the two-year mark for this (IMHO) critical ATEM bug.

In a thread started in July 2020, Henry Kim/@bigfundj wrote up a detailed analysis of the problem in October 2020:
https://forum.blackmagicdesign.com/viewtopic.php?f=4&t=117185#p675388

I added information in October 2020 as well, on how the ATEM Mini Pro ISO had a cascade failure during a wedding:
https://forum.blackmagicdesign.com/viewtopic.php?f=4&t=117185#p678105

In the *year and a half* since then, I’ve avoided using the ATEM in any streaming situations where I did not have a dedicated direct Ethernet connection at the venue, greatly limiting its usefulness as an event streaming platform. My multiple ATEM mobile rigs have become almost worthless.

I was very excited about the direct ATEM-Extreme-ISO-to-4G/5G-phone-connection feature….but the next time I tried it in May of 2021, it blew up in my face yet again….after streaming successfully for 20-30 minutes at some point it detected temporary network congestion, immediately filled the cache, locking up and causing failure of the stream AND the ISO recordings being done at the time.

I pulled out my iPhone and gimbal and got a stream and recording of the last 1/4 of the event, but all recordings of the middle half were lost forever, due to the complete failure of the ATEM.

I haven’t heard any updates addressing this problem from Blackmagic.

I have the same problem when the MTU of the internet connection is smaller than 1500.
I suspect the (misguided / broken) IP stack in the atem to not work properly in that case.

This seems like a really good theory on a potential key factor of the underlying problem. Doing a cursory search, I saw some people quoting 4G/LTE connections often having maximum MTUs in the 1400 and 1300 range

me too...unplug the network cable from the atem for 2-3 seconds when cache starts to fill then plug it back in and it recovers every time. Not acceptable but it's the only way to make the streaming engine useable. have ticket open with support, they sort of hint engineers are aware so hopefully they are tuly aware. I provide feedback to them via the ticket to help them narrow down the problem. I'm convinced it's a bug, my internet has no issues when this happens. Have fios 300/300 and barely using it. Constant ping to my provider ny.castr.io showed no issues or changes when cache fills. When you take the network cable out it seems it resets the ip stack because cache will empty very quickly and the stream rolls along flawlessy afterwards.

I did another event in the fall of 2021 using the ATEM Extreme ISO, and now that I think back at one point I did start getting the cache full syndrome, even though it was wired (but to a slightly congested Venue network. Thankfully, unplugging and re-plugging the Ethernet cable did eventually get the streaming cache un-frozen before total lock (and ISO recording loss) started to occur. But it really shouldn’t be this way!

Two years is a very very long time for this known critically fatal problem with a key functionality(streaming and recording) to go without being addressed. It is concerning that the issue can be documented and debugged, by Blackmagic customers and ATEM owners wishing the device to be successful, on the company’s user forums….but with no meaningful feedback or mitigation plans from the company.

It really feels like it’s being ignored or de-prioritized for some reason, and it shouldn’t be that the only solution for action is to document these issues in reviews on external vendor sites such as Amazon, B&H, etc. I hope that Blackmagic will try to address this issue ASAP.

I am having a very similar problem with an ATEM Mini Pro.

It is streaming 24x7 from a remote site and has been very stable, except when you connect to it via the ATEM software control and whilst the media files are downloading the stream buffer fills up. Quite often the Software control app then restarts the connection and the media file download occurs again only for the process to be repeated.

If I start a "ping" on the LAN local to the ATEM between a PC and the router gateway (normally 1ms response) the pings begin to fail. It looks like the ATEM is causing a local broadcast storm. I am going to try rate limiting the switch port to see if that helps.

On other occasions the stream crashes, never to be recovered without a power cycle, the ATEM Software Control stream button does nothing and neither does the physical button. You can prompt this behavior by uploading a new MACRO file, it seems to crash a couple of hours after uploading the file.

I wonder if the reason for this problem not being resolved is because its a hardware issue and the fix is physical surgery?

This problem is never solved and the worst thing about the atem mini series. If there goes something wrong with the streaming and the buffer fills to the max, then the atem crashes and can only recoverd with a total power off and on. Imagine the consequences if simultaniously you are using the atem to record as well? For this reason I use two atems. one for streaming. And I always chose a quality way below the available network speed. I hate this thingy most in all of the further great machines. Reliable operating is the most important thing of any devide. network congestion should never crash the atem.