Multicam audio sync failure example!

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

Dan Sherman

  • Posts: 1185
  • Joined: Fri Jul 01, 2016 11:07 pm

Multicam audio sync failure example!

PostMon Feb 12, 2018 1:03 am

Yes, I know everyone says it doesn't work very well, but as a developer who's aware of how complicated this can be, I'm still shocked it's this bad. I've been playing with this for a few days now, trying various setting combinations, and nothing has worked.

This morning I devised one last test to help Resolve as much as possible. I took two cameras, a Panasonic GH5 and G7 and routed the GH5's audio out into the G7's audio in. I adjusted the gain (in camera) to get levels within 1 db of each other. I shot a short sequence where the GH5 ran continually, and the G7 was used to generate short clips between 30 and 90 seconds in length.

When setting Angle Sync to sound this is what Resolve came up with.

1.jpg
1.jpg (161.43 KiB) Viewed 2763 times

2.jpg
2.jpg (134.04 KiB) Viewed 2763 times


This is what it should have looked like.
3.jpg
3.jpg (98.27 KiB) Viewed 2763 times
Last edited by Dan Sherman on Mon Feb 12, 2018 1:12 am, edited 2 times in total.
AMD 7950X | AMD 7900XTX (23.20.24) | DDR5-6000 CL30-40-40-96 2x32 GB | Multiple PCIe 4.0 X4 NVME | ASUS x670e HERO | Win 11 Pro 23H2 | Resolve Studio 18.6.5 B7
Offline

Dan Sherman

  • Posts: 1185
  • Joined: Fri Jul 01, 2016 11:07 pm

Re: Multicam audio sync failure example!

PostMon Feb 12, 2018 1:05 am

Whats bothersome about this, is that if it can't match waveforms that are this similar, I don't see how this can possibly be useful in real world use cases.

4.jpg
4.jpg (116.11 KiB) Viewed 2686 times


5.jpg
5.jpg (80.73 KiB) Viewed 2686 times
AMD 7950X | AMD 7900XTX (23.20.24) | DDR5-6000 CL30-40-40-96 2x32 GB | Multiple PCIe 4.0 X4 NVME | ASUS x670e HERO | Win 11 Pro 23H2 | Resolve Studio 18.6.5 B7
Offline
User avatar

Uli Plank

  • Posts: 21278
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: Multicam audio sync failure example!

PostMon Feb 12, 2018 7:52 am

No NLE beats PluralEyes in this respect.
No, an iGPU is not enough, and you can't use HEVC 10 bit 4:2:2 in the free version.

Studio 18.6.5, MacOS 13.6.5
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G, iMac 2017, eGPU
Offline
User avatar

Glenn Venghaus

  • Posts: 1358
  • Joined: Wed Jan 01, 2014 9:56 pm
  • Location: Amsterdam , The Netherlands

Re: Multicam audio sync failure example!

PostMon Feb 12, 2018 8:10 am

Absolutely. I love resolve to the death , but would not dream of usings it or any other nle's for sound sync duties based on waveform. Pluraleyes is a few billion lightyears ahaed and never failed me once , where others where also consistent in failing all the time.
Beatstep & APC-40 Resolve Edition Controllers https://posttools.tachyon-consulting.com
Test Rig : 2xXeon (24c) | UNRAID KVM OSX VM's | 128GB | 5700XT | 40Gbe
Prod Rig : i9-7940X (14c) | OSX 10.15 | 64GB | 2xVega 56 | 40Gbe | Tb3 | V:Eizo | A:5.1RME
Offline

Dan Sherman

  • Posts: 1185
  • Joined: Fri Jul 01, 2016 11:07 pm

Re: Multicam audio sync failure example!

PostMon Feb 12, 2018 3:54 pm

I had originally planned on a Zoom F4, and some tentacle Sync Es or NanoLockits, but that's now on hold until my wife and I's first child arrives in a few months. I thought I could make do with the audio sync, but my tests have proven to me, that's just not feasible.
AMD 7950X | AMD 7900XTX (23.20.24) | DDR5-6000 CL30-40-40-96 2x32 GB | Multiple PCIe 4.0 X4 NVME | ASUS x670e HERO | Win 11 Pro 23H2 | Resolve Studio 18.6.5 B7
Offline

Sam Steti

  • Posts: 2470
  • Joined: Tue Jun 17, 2014 7:29 am
  • Location: France

Re: Multicam audio sync failure example!

PostMon Feb 12, 2018 4:34 pm

I've been seeing more and more topics popping up about audio syncing issues, but actually Resolve is not more nor less in the same situation as other NLEs : just 3 levels under Plural Eyes, whose unique job is to achieve this.

I'm no developer in this field, and I didn't realize it was so hard achieve the implementation of this tool. But since around 2-3 years now, I understand the reason why Plural Eyes is such a powerful tool : it always worked great, allows drag'n'dop, is fast and clear, does the job and exports what you need... It was not as easily doable as I thought it was at the end of the day...
*MacMini M1 16 Go - Ext nvme SSDs on TB3 - 14 To HD in 2 x 4 disks USB3 towers
*Legacy MacPro 8core Xeons, 32 Go ram, 2 x gtx 980 ti, 3SSDs including RAID
*Resolve Studio everywhere, Fusion Studio too
*https://www.buymeacoffee.com/videorhin
Offline
User avatar

Uli Plank

  • Posts: 21278
  • Joined: Fri Feb 08, 2013 2:48 am
  • Location: Germany and Indonesia

Re: Multicam audio sync failure example!

PostTue Feb 13, 2018 2:45 pm

Well, if it's about money, if I remember well you can get a 10 day fully functioning demo of PluralEyes.

I'm not sure any more, since I bought it right away…
No, an iGPU is not enough, and you can't use HEVC 10 bit 4:2:2 in the free version.

Studio 18.6.5, MacOS 13.6.5
MacBook M1 Pro, 16 GPU cores, 32 GB RAM and iPhone 15 Pro
Speed Editor, UltraStudio Monitor 3G, iMac 2017, eGPU
Offline

Sam Steti

  • Posts: 2470
  • Joined: Tue Jun 17, 2014 7:29 am
  • Location: France

Re: Multicam audio sync failure example!

PostTue Feb 13, 2018 3:01 pm

+ 1 : the PE 4 version I own is a must have for me, and saves a lot of precious time...
*MacMini M1 16 Go - Ext nvme SSDs on TB3 - 14 To HD in 2 x 4 disks USB3 towers
*Legacy MacPro 8core Xeons, 32 Go ram, 2 x gtx 980 ti, 3SSDs including RAID
*Resolve Studio everywhere, Fusion Studio too
*https://www.buymeacoffee.com/videorhin
Offline

Rohit Gupta

Blackmagic Design

  • Posts: 1628
  • Joined: Wed Aug 22, 2012 5:00 am

Re: Multicam audio sync failure example!

PostTue Feb 13, 2018 3:15 pm

Dan Sherman wrote:Yes, I know everyone says it doesn't work very well, but as a developer who's aware of how complicated this can be, I'm still shocked it's this bad. I've been playing with this for a few days now, trying various setting combinations, and nothing has worked.

This morning I devised one last test to help Resolve as much as possible. I took two cameras, a Panasonic GH5 and G7 and routed the GH5's audio out into the G7's audio in. I adjusted the gain (in camera) to get levels within 1 db of each other. I shot a short sequence where the GH5 ran continually, and the G7 was used to generate short clips between 30 and 90 seconds in length.

When setting Angle Sync to sound this is what Resolve came up with.

1.jpg

2.jpg


This is what it should have looked like.
3.jpg


Could you PM me a link to your test files, and we’ll check it out.
Rohit Gupta

DaVinci Resolve Software Development
Blackmagic Design
Offline

Dan Sherman

  • Posts: 1185
  • Joined: Fri Jul 01, 2016 11:07 pm

Re: Multicam audio sync failure example!

PostTue Feb 13, 2018 3:52 pm

Rohit Gupta wrote:
Dan Sherman wrote:Yes, I know everyone says it doesn't work very well, but as a developer who's aware of how complicated this can be, I'm still shocked it's this bad. I've been playing with this for a few days now, trying various setting combinations, and nothing has worked.

This morning I devised one last test to help Resolve as much as possible. I took two cameras, a Panasonic GH5 and G7 and routed the GH5's audio out into the G7's audio in. I adjusted the gain (in camera) to get levels within 1 db of each other. I shot a short sequence where the GH5 ran continually, and the G7 was used to generate short clips between 30 and 90 seconds in length.

When setting Angle Sync to sound this is what Resolve came up with.

1.jpg

2.jpg


This is what it should have looked like.
3.jpg


Could you PM me a link to your test files, and we’ll check it out.


I'll upload them to my server this evening and send you a link.
AMD 7950X | AMD 7900XTX (23.20.24) | DDR5-6000 CL30-40-40-96 2x32 GB | Multiple PCIe 4.0 X4 NVME | ASUS x670e HERO | Win 11 Pro 23H2 | Resolve Studio 18.6.5 B7
Offline

Dan Sherman

  • Posts: 1185
  • Joined: Fri Jul 01, 2016 11:07 pm

Re: Multicam audio sync failure example!

PostTue Feb 13, 2018 10:38 pm

Rohit Gupta wrote:Could you PM me a link to your test files, and we’ll check it out.


pm sent!
AMD 7950X | AMD 7900XTX (23.20.24) | DDR5-6000 CL30-40-40-96 2x32 GB | Multiple PCIe 4.0 X4 NVME | ASUS x670e HERO | Win 11 Pro 23H2 | Resolve Studio 18.6.5 B7
Offline

Johannes Hoffmann

  • Posts: 274
  • Joined: Mon Aug 26, 2013 5:22 am
  • Location: Germany

Re: Multicam audio sync failure example!

PostWed Feb 14, 2018 8:17 am

Sam Steti wrote:+ 1 : the PE 4 version I own is a must have for me, and saves a lot of precious time...


Can you tell me more about your workflow?

The reason I ask: I tested PE a couple of weeks back. And while it was amazing how well it matched the audio, the overall workflow took more time than syncing by hand. Maybe I missed something obvious, but this are the things that slow me down:

- PE often gets the cameras wrong (in my case insisting that files from the BMCC and the Pocket are from the same camera unless I place them in separate folders).
- No way in PE to re-order wrong matches to an other track in the timeline
- PE can replace the original audio with one external audio source only
- PE exports only an older version of FCPX-xml (that resolve does not read)
- Using Premiere-xml does not support multicam

So the simple situation of a two camera files and two audio files in a folder leads to a lot of additional work until I finally have my multicam in Resolve.

After a few days testing I really wondered: do I miss something or why is everyone praising an application that does costs so much and lacks so much in workflow usability?

Johannes
Apple M1 Max, 32GB, 32 GB GPU – macOS Monterey
almost retired: MacPro 5.1, 48GB, RX 580 – macOS Mojave
Offline

Sam Steti

  • Posts: 2470
  • Joined: Tue Jun 17, 2014 7:29 am
  • Location: France

Re: Multicam audio sync failure example!

PostWed Feb 14, 2018 11:57 am

Johannes Hoffmann wrote:
Sam Steti wrote:+ 1 : the PE 4 version I own is a must have for me, and saves a lot of precious time...

Can you tell me more about your workflow? Johannes

I can understand, there's something "special" in the Resolve workflow, which also took me time in the beginning (I was faster in manual sync in FCP7 than in Resolve with PE for weeks !)...
Well, let's not take into account errors, mistakes etc in PE, here's roughly my WF in Resolve :
- drag'n'drop right files > sync > export XML
- In Resolve : file > import xml
Now beware : you have stacked tracks properly synced from PE, but no real multicam file in Resolve so far; and here's the weirdness of Resolve now : to make the multicam file with all the stacked angles (called ISOs here by BMD), Resolve doesn't take these tracks TC or markers here on the TL into account, but from these very files open in the media pool only. See below.

Now I want to create the multi file by designing markers I will put on each track as representing the point to sync to make the real multicam file.
BUT - and this is weird and dumb - if you put the playhead anywhere and decide to put markers on every tracks here, it won't match in the end (at least it never did for me). So :
- I put the playhead around the beginning, where every angles are involved
- I note (!) the TC of each track at the playhead place
- I go to the media pool and open each track one after another there, to put a marker at their noted TC
- right click > multi > sync by marker

Usually I then duplicate the XML TL, overwrite the muticam file on top of the imported stacked tracks : this allows me to wipe all stacked tracks in the end but to keep some audio ones if needed....

Of course, it seems long to read, but I can do that very fast :lol:
If Resolve allowed me to put markers directly on the TL of PE's imported XML to result in a really synced sequence (because it's not in this case), I'll gain 40% again...
*MacMini M1 16 Go - Ext nvme SSDs on TB3 - 14 To HD in 2 x 4 disks USB3 towers
*Legacy MacPro 8core Xeons, 32 Go ram, 2 x gtx 980 ti, 3SSDs including RAID
*Resolve Studio everywhere, Fusion Studio too
*https://www.buymeacoffee.com/videorhin
Offline

Dan Sherman

  • Posts: 1185
  • Joined: Fri Jul 01, 2016 11:07 pm

Re: Multicam audio sync failure example!

PostWed Feb 14, 2018 5:36 pm

Sam Steti wrote:I'm no developer in this field, and I didn't realize it was so hard achieve the implementation of this tool.


It's an interesting topic, this a short paper on a possible method that can be used.
https://static.googleusercontent.com/me ... /42193.pdf
AMD 7950X | AMD 7900XTX (23.20.24) | DDR5-6000 CL30-40-40-96 2x32 GB | Multiple PCIe 4.0 X4 NVME | ASUS x670e HERO | Win 11 Pro 23H2 | Resolve Studio 18.6.5 B7

Return to DaVinci Resolve

Who is online

Users browsing this forum: Alexandre Sadowsky, edouardjeanjean, Google [Bot], pperquin, ReginaldVanSeveren and 197 guests