Page 1 of 1

BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Tue Oct 19, 2021 5:28 pm
by DerPaule13
I have searched but found nothing like this in the forum yet.

Yesterday we had an interview shooting with BMPCC 6K & 6K Pro cameras. Luckily it was an inhouse production for ourselves. After the interview we noticed that both cameras had an exclamation mark blinking inside the still running recording button.

Touching it wouldn't stop the recording. We also tried the hardware button. The recording went on and the SSDs got written on. (Blue LEDs on the SSDs were blinking)
Only way to stop this was to shut down the cameras.

After restarting the footage was gone! We only had about 2 minutes on the 6K and about 3 minutes on the 6K Pro of interview and about 40 minutes missing! (Also checked the SSDs via computer)

So we did the whole interview again :oops:
Now we paid attention to this and on the 6K Pro it happend once more after about 10 minutes!


How is this possible?

The only thing we have changed on the 6K (our workhorse) is updating the firmware so the color profile will match our new 6K Pro. We never had this issue before.
As of right now, we are really scared to use these cameras on our next production.
Has this happend to anyone else?

We have experienced this exclamation mark before, trying to record 6K 50 fps to an SD-Card and after 3 seconds the recording would stop. That's because the SD card was too slow for this.
But we can't explain what's going on here.


Here are the parameters we worked with:

BMPCC 6K
Firmware: 7.3
Resolution: 6K, 25 fps
Codec: BM RAW 8:1
Drive: Samsung T5 SSD 1 TB via USB-C (Tilta cable)
Timecode: Tentacles Sync E

BMPCC 6K Pro
Firmware: 7.3.1
Resolution: 6K, 25 fps
Codec: BM RAW 12:1
Drive: Samsung T5 SSD 1 TB via USB-C (Tilta cable)
Timecode: Tentacles Sync E

Thanks for any help!

_________________________
EDIT 30-03-2022

Solution:
We are using the Tilta Side Handles with the Battery & Samsung T5 SSD holder.
Here's the thing: Tilta's USB-C to USB-C cables are not fast enough for BM color science V5. :shock: :|
Using the original Samsung USB-C to USB-C cables fixed the issue for us.

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Tue Oct 19, 2021 7:17 pm
by Jamie LeJeune
The 1TB Samsung T5 is not on BMD's supported list for recording 6K BRAW:
https://www.blackmagicdesign.com/support/faq/59026

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Tue Oct 19, 2021 8:31 pm
by rick.lang
It might be helpful to plug the SSD into the computer and run the BMD Speedtest to see where the drive should work. But the list of supported drives as Jamie provided is a more reliable guidance than the Speedtest. Thankfully there are several approved drives available for the 6K/Pro.

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Wed Oct 20, 2021 6:48 am
by DerPaule13
Thanks for your answers. This could be it.
Unfortunately we're on windows, so we can't use the disk speed test.

The thing is, up until now the 6K worked perfectly fine with the Samsung T5 1TB SSDs. 6K, 5:1, 50 fps.

Now with the new firmware 6K, 12:1, 25 fps is too much? We have lots of those SSDs for our productions. It would be quite unfortunate having to replace all of them now.

I guess we'll try rolling back to the previous firmware with both cameras for now :cry:

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Wed Oct 20, 2021 7:23 am
by Robert Niessner
DerPaule13 wrote:Thanks for your answers. This could be it.
Unfortunately we're on windows, so we can't use the disk speed test.


BMD Speedtest is also available for Windows.
You need to install BMD Desktop Video:
https://www.blackmagicdesign.com/suppor ... c1/Windows

There is a second speedtest utility from AJA you should also use:
https://www.aja.com/en/products/aja-system-test

Make sure to set them to write the biggest file in the settings, otherwise you are just testing caches.

DerPaule13 wrote:The thing is, up until now the 6K worked perfectly fine with the Samsung T5 1TB SSDs. 6K, 5:1, 50 fps.


The Samsung T5 SSD have never been super reliable in conjunction with the cameras as a recording media. The T5 has been made for computer use with very different workload cases than the demanding use case of constantly writing video streams from a camera at high speed.

If you want reliable media then buy CFast 2.0 cards. There is a reason those are more expensive, while the T5s are one of the cheapest media.

DerPaule13 wrote:Now with the new firmware 6K, 12:1, 25 fps is too much? We have lots of those SSDs for our productions. It would be quite unfortunate having to replace all of them now.

I guess we'll try rolling back to the previews firmware with both cameras for now :cry:


The T5 will dramatically degrade during long write periods and with less and less free cells. That's the way flash memory works. You need to do a secure erase for getting the SSD back to factory default speeds.

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Wed Oct 20, 2021 12:31 pm
by DerPaule13
Thanks! I didn't know that Speedtest is part of the "Desktop Video" application.

I'll run some tests and hopefully the fault are the SSDs - in which case a secure erase may solve the problem.

Before updating the firmware we've been using the T5 SSDs for about 1 1/2 years without any hiccups. Two different SSDs failing at the same time seems a bit odd, considering the firmware is the only thing that has changed. And shouldn't the recording just stop in case the media can't write fast enough?
Not being able to stop the recording seems more like a software bug, but who knows.

I'll report back as soon as we're done with our tests. Thanks again for your help so far guys! :)

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Wed Oct 20, 2021 1:58 pm
by Robert Niessner
DerPaule13 wrote:Thanks! I didn't know that Speedtest is part of the "Desktop Video" application.

I'll run some tests and hopefully the fault are the SSDs - in which case a secure erase may solve the problem.


Most super fast SSDs are using a small part of the cells in SLC (single level cell) mode as write cache to achieve those claimed speeds. When that gets filled up, speed falls dramatically. Also when the drives get hot they begin to throttle.

Before a cell can be written, it has to be deleted (except it has been never be used before or a secure erase has been performed). This will also affect speed.

So it might be that those test tools still show that write speeds are ok because they write into the SLC cache.

DerPaule13 wrote:Before updating the firmware we've been using the T5 SSDs for about 1 1/2 years without any hiccups. Two different SSDs failing at the same time seems a bit odd, considering the firmware is the only thing that has changed. And shouldn't the recording just stop in case the media can't write fast enough?
Not being able to stop the recording seems more like a software bug, but who knows.


It could be an coincidence that just now the drives have been enough degraded.

But I remember now that someone wrote about an answer he got from Adata or Wise support (can't remember) that BMD changed something with the power delivered over USB-C in the latest firmware and that this would cause hiccups with external drives.
So this could be the case here too.

If your T5 doesn't get enough power suddenly it might hang in an undefined state which in turn might hang the camera's bus system.

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Wed Oct 20, 2021 2:20 pm
by DerPaule13
Robert Niessner wrote:But I remember now that someone wrote about an answer he got from Adata or Wise support (can't remember) that BMD changed something with the power delivered over USB-C in the latest firmware and that this would cause hiccups with external drives.
So this could be the case here too.

If your T5 doesn't get enough power suddenly it might hang in an undefined state which in turn might hang the camera's bus system.

This is interesting and sounds to me like a very likely reason. One SSD is being used for 1 1/2 years now, the other is relatively new.

In case this is an issue with power delivery over USB-C, I really hope BMD will fix this with the next update :idea:

We'll do some tests none the less just to be sure and I'll report back. Thanks again!

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Wed Oct 20, 2021 3:05 pm
by Robert Niessner
Found the posting - alas it was about the UMP12k and SSD recorder over USB-C - so this might or might not apply to the BMPCC6k too:

Source

Patrick Spectra wrote:So Exascend have been great and are sending an updated drive immediately. I have to send mine to their office in California but they are sending the replacement without needing mine back before they send out the new one. Anyways here’s what they said:

“We found the recognition problem is caused by insufficient power supply from URSA Mini Pro 12K.

We did several cross test and found latest URSA recorder firmware (FW7.4.1) decrease power voltage to SSD when connecting with URSA Mini Pro 12K and the restricted power voltage is less than industrial standard. When power management IC detects insufficient power voltage, the SSD won’t be active to prevent data loss.
The restriction instruction comes from camera (via Type-C connector), that’s why the detective problem only happens when recorder connects to the camera, but SSD works normal when connecting to the laptop.


We’ll update the power management IC firmware to ensure the SSD could still be active even detects low power voltage from Mini Pro 12K. At the same time we’ll raise the issue to Blackmagic Design since the restricted power voltage do not fit industrial standard.”

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Wed Oct 20, 2021 4:16 pm
by Mark Foster
the new BRAW 2.x files (gen5 from firmware 7.3.x) need a higher throughput

.

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Thu Oct 21, 2021 10:06 am
by carlomacchiavello
I follow the Robert’s note about power. I use different ssd and found that with external battery (12v power) I never had problems on recording, when I need to use internal battery lp6, with the same ssd (different brand and size), I found problems to reach maximum speed recording.
I did many tests with pocket4k and 6k and power of usb-c is not so stable with internal battery against external battery, few ssd work fine with internal battery at maximum speed, I suspect that also ssd lower their speed when there is low power in.


Inviato dal mio iPhone utilizzando Tapatalk

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Fri Oct 22, 2021 5:28 am
by DerPaule13
Mark Foster wrote:Source for UMP12k and SSD recorder over USB-C

Thanks! Interesting none the less. We'll keep that in mind.

carlomacchiavello wrote:I follow the Robert’s note about power. I use different ssd and found that with external battery (12v power) I never had problems on recording, when I need to use internal battery lp6, with the same ssd (different brand and size), I found problems to reach maximum speed recording.
I did many tests with pocket4k and 6k and power of usb-c is not so stable with internal battery against external battery, few ssd work fine with internal battery at maximum speed, I suspect that also ssd lower their speed when there is low power in.

Makes sense. Another factor to keep in mind, thanks.
We actually power our cameras either with the power cable or with an external battery (tilta battery handle, NP-F970)

Mark Foster wrote:the new BRAW 2.x files (gen5 from firmware 7.3.x) need a higher throughput
.

Thanks for this information. But shouldn't this result in stopping the recording, instead of continuous SSD-writing without being able to stop the recording?

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Fri Oct 22, 2021 9:19 am
by Uli Plank
The latter you can change in the settings.

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Wed Mar 30, 2022 11:02 am
by DerPaule13
Finally! I have found the solution.

We are using the Tilta Side Handles with a Battery & Samsung T5 SSD holder.
The best thing about this setup is that the USB-C cable can't get loose, as both sides are fixed with screws. Here's the thing: Tilta's cable is not fast enough. :shock: :|

With BM color science V4 everything was fine. But V5 has a slightly higher throughput and that seems to be enough to reach the limits of Tilta's USB-C cables. As soon as we used the stock USB-C to USB-C cables that come with Samsung's SSD, everything works fine again. No more recording errors.

Thanks everyone for helping out here!

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Wed Mar 30, 2022 11:33 am
by Will Vazquez
Good to hear that you solved it. I only use the cables that came with the T5 drive and have had wonderful success even with multi camera shoots. I've had file corruption or drive failure twice with Angelbird CFast cards. So after that I only trust SD Cards or T5.

BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Wed Mar 30, 2022 3:09 pm
by carlomacchiavello
Will Vazquez wrote:Good to hear that you solved it. I only use the cables that came with the T5 drive and have had wonderful success even with multi camera shoots. I've had file corruption or drive failure twice with Angelbird CFast cards. So after that I only trust SD Cards or T5.
Usually cfast are more sure than sd card, usually problems happen with bad card reader or for bad attitude of windows to mess up external support with chksdk command, which pretend to repair thing that not need repair (I miss in past, many time, data from sd, cfast, external hdd, before to disable permanently from boot of windows.
And today I prefer offload by Mac or linux device.


Inviato dal mio iPhone utilizzando Tapatalk

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Thu Oct 05, 2023 9:07 pm
by Niculescu
Hello,

Yesterday I had the same problem but I am 100% sure that the issue is not SSD or cable related; I will explain why.

I lost unique footage with our Prime Minister and the CEO of Coca-Cola so I would like a detailed technical answer from the BMD team. Please have in mind that we wore the official video team and we don't have other teams to ask them for some footage (journalists, TV, etc). Situation is as bad as it can get.

As I was saying, SSD is not the problem: I use T7 with Apple thunderbolt 4 cable (I think is the best cable that you can get). The drives are not used extensively but we have used them, filming government and parliament meetings, non stop for hours, using RAW Q1 and we never had any issues. Yesterday we set 4K RAW Q3, meaning less data bandwidth than usual, and around 45 minutes of footage. I personally saw the exclamation sign inside the recording button but I could not find a written error message. I shot some test footage and all was good. "!" appeared and disappeared sporadically, during stop and also during record time. Files were created on the ssd but some have the extension .mcc and 0 Kb. We have 7 files, file 0002 and 0005 are .mcc, the other ones are good.

Please, let's find what generates this problem and how to address it. Personally I would recommend inserting a more obvious and detailed error message flashing red in the middle of the screen in the next firmware update. A "!" can mean many things...

We lost the most important footage: closeup camera (Murphy laws).

Looking forward for you answer.

Thank you.
Best regards

BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Fri Oct 06, 2023 4:39 pm
by rick.lang
The Samsung T7 is not an approved SSD; the Samsung T7 Shield is approved and I use the T7 Shield 2TB drive successfully shooting with the BMPCC4K.

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Fri Oct 06, 2023 5:03 pm
by carlomacchiavello
Niculescu wrote:Hello,

Yesterday I had the same problem but I am 100% sure that the issue is not SSD or cable related; I will explain why.

I lost unique footage with our Prime Minister and the CEO of Coca-Cola so I would like a detailed technical answer from the BMD team. Please have in mind that we wore the official video team and we don't have other teams to ask them for some footage (journalists, TV, etc). Situation is as bad as it can get.

As I was saying, SSD is not the problem: I use T7 with Apple thunderbolt 4 cable (I think is the best cable that you can get). The drives are not used extensively but we have used them, filming government and parliament meetings, non stop for hours, using RAW Q1 and we never had any issues. Yesterday we set 4K RAW Q3, meaning less data bandwidth than usual, and around 45 minutes of footage. I personally saw the exclamation sign inside the recording button but I could not find a written error message. I shot some test footage and all was good. "!" appeared and disappeared sporadically, during stop and also during record time. Files were created on the ssd but some have the extension .mcc and 0 Kb. We have 7 files, file 0002 and 0005 are .mcc, the other ones are good.

Please, let's find what generates this problem and how to address it. Personally I would recommend inserting a more obvious and detailed error message flashing red in the middle of the screen in the next firmware update. A "!" can mean many things...

We lost the most important footage: closeup camera (Murphy laws).

Looking forward for you answer.

Thank you.
Best regards

I'm afraid for you, loss of shooting is ever a big problem.
you write t7, you mean samsung t7 or samsung t7 shield? are very different support, first is source of infinity problems, second is in the approved list.
have you tried to check disk from a recovery software?
in this thread there is an address about a society specialised in recovery, especially for braw.

Re: BMPCC 6K & 6K Pro Recording Failure & Data Loss

PostPosted: Fri Oct 06, 2023 8:26 pm
by Robert Niessner
If there is a chance to recover your footage, your best bet would be to contact them:
https://aeroquartet.com/treasured/index.en.html