Auto Sync Audio > Based on Waveform | Hardly ever works

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 8:52 pm

For our videoshoots we use one (or multiple) cameras and a ZOOM H6 recorder for audio.
I should specify that the audio from camera and ZOOM are both very clear and audible.

After importing the footage and audio, I select all of the clips, right click and choose "Auto Sync Audio >> Based on Waveform and Append Tracks".

The software does its analysis and that's when the trouble starts.

Often times, it says some clips could not be matched. Even though the exact same number of audio and video clips are selected, all of which can easily synced by, for example Premiere Pro.

The appended tracks aren't even remotely related. Sometimes it syncs the "clap" sound from the slate as significant audio event but all the rest is wrong.

Here's a screenshot of one of the appended tracks, as you can see, the waveform doesn't match at all:
https://cp.sync.com/dl/0fa05f450#4tbphd9j-j7wr9y32-gw4wvtbi-x2yj7vfa

My "work-around" has been to sync audio in Premiere Pro, throw it on one timeline and export the timeline back to Davinci. Which is quite inconvenient.

In the previous betas it didn't work well either and I'm not sure if Resolve14 was any better.
In short: auto sync based on waveform is quite unusable.
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 9:25 pm

Further exploration:

The ZOOM audio is imported as 44100 Hz.
The camera audio is imported as 48000 Hz.

When selecting ONLY one audio and video clip that are supposed to by sync-able (rather than letting the algorithm figure out which clips need to sync) there are still major problems:

It tries to sync the timeline in the beginning but quickly the audio shifts out of proportion. The further in the timeline, the more out of sync.

But here's the kicker: the audio tracks are still equal in length. Although one seems to play much faster, the tracks are still ending at the same spot.

HOW IS THIS POSSIBLE?

Is there a difference in tempo between 44.1 and 48 kHz playback?

I'm also noticing that the Fairlight tab has much more detailed Waveforms than the Edit tab. But, when zooming in on the Fairlight tab, certain zoom levels make the waveform "pop" out of place.

It's crazy!

I don't know what to do here.

Unless I'm supposed to do something special to make 44.1 and 48 kHz tracks play back properly, it's definitely a bug.
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 9:29 pm

Further exploring:

The audio files seem to have a "FPS" of 29.970 DF.
The video files have an FPS of 24.000

The auto-synced clips have a fps of 24.000.

Does this have anything to do with it?

Is there a reason that audio has a "Frames Per Second" setting when there are no frames in audio?
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 9:35 pm

Further exploring:

I've converted the audio to 48 kHz instead of 44.1.

Resolve still imports it with a 29.970 FPS.

I don't know what to do until someone replies here.
Meanwhile I'll have to revert to Premiere Pro. CRAP. I hate Premiere.
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 9:57 pm

Further exploring:

I've re-encoded the video to be 29.970, since the Resolve project was set to that. This is probably why the audio is given the same FPS.

However, even syncing the video clip with the same FPS as the audio, still is wildly out of sync.
So perhaps it has nothing to do with the framerate.

(Makes no sense to have FPS for audio still, but whatever).
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 11:03 pm

Further exploring:

I've edited the video in Premiere and exported it as XML, then imported that timeline in Resolve.

Result? The video and audio clips are there, but the audio is completely offset. Unusable.

Sigh. Guess I can't use Resolve until this gets fixed.
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

Rohit Gupta

Blackmagic Design

  • Posts: 1011
  • Joined: Wed Aug 22, 2012 5:00 am

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostThu Jun 21, 2018 2:58 am

TornadoTwins wrote:For our videoshoots we use one (or multiple) cameras and a ZOOM H6 recorder for audio.
I should specify that the audio from camera and ZOOM are both very clear and audible.

After importing the footage and audio, I select all of the clips, right click and choose "Auto Sync Audio >> Based on Waveform and Append Tracks".

The software does its analysis and that's when the trouble starts.

Often times, it says some clips could not be matched. Even though the exact same number of audio and video clips are selected, all of which can easily synced by, for example Premiere Pro.

The appended tracks aren't even remotely related. Sometimes it syncs the "clap" sound from the slate as significant audio event but all the rest is wrong.

Here's a screenshot of one of the appended tracks, as you can see, the waveform doesn't match at all:
https://cp.sync.com/dl/0fa05f450#4tbphd9j-j7wr9y32-gw4wvtbi-x2yj7vfa

My "work-around" has been to sync audio in Premiere Pro, throw it on one timeline and export the timeline back to Davinci. Which is quite inconvenient.

In the previous betas it didn't work well either and I'm not sure if Resolve14 was any better.
In short: auto sync based on waveform is quite unusable.


Are you able to send a few clips and the audio files so we can check it out? Please send the originals.

Please upload to Dropbox, or similar, and PM me a link. We'll investigate.
Rohit Gupta

DaVinci Resolve Software Development
Blackmagic Design
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostThu Jun 21, 2018 4:38 am

Certainly, I’ll send you the files tomorrow morning by replying to your DM.
Thanks for looking into this!
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

Uli Plank

  • Posts: 3538
  • Joined: Fri Feb 08, 2013 2:48 am

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostThu Jun 21, 2018 6:04 am

As much as I like DR, I’ve never found waveform syncing acceptable in any NLE.
Rather give PluralEyes a try, it works very well for us. If I remember correctly, they have a fully functional 10 days demo.
Resolve Studio and Fusion Studio
iMac 2017 Radeon Pro 580 8 GB VRAM and 32 GB RAM
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostThu Jun 21, 2018 10:44 pm

@Rohit Gupta - Thanks, I've sent you an Enrico a DM with the project and audio/video files.

@Uli Plank - I do like the idea of PluralEyes... however, since it's about the same price as Davinci Resolve Studio, and since it does work fine in Premiere Pro, it would be really great if I didn't have to buy anything extra.
That said, great tip, thanks for sending this along!
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

malkazoid

  • Posts: 16
  • Joined: Tue Jul 03, 2018 10:07 pm
  • Real Name: Malcolm Wright

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostFri Aug 24, 2018 3:18 am

Hello all,

I'm using DR 15 Studio, and experiencing similar woes with waveform syncing.
Namely zero sync attempts have been successful in DR, always resulting in all clips being declared as not having matches AND my attempts to use Pluraleyes 4.1.6 result in the same problems experienced by others.

If I try to export an xml of a timeline and bring it into Pluraleyes - Pluraleyes says there are invalid elements on the timeline and will not import.

If I use Pluraleyes on its own, pulling the media directly into it, sync, then export an xml, this xml results in clips with portions declared as offline, and not lining up with the audio. It appears to be some weird timecode offset in the clips affected. The clips have proper timecode and are Sony MXFs shot on a FS7m2.

Like others, if I first load the xml from pluraleyes into Premiere, then export a new xml from Premiere, DR happily brings in this new xml, and I'm good to go.

I'm not a software engineer, but there is clearly something about the Pluraleyes written xml, that Premiere is happily reading correctly, but DR is not.

The whole point of editing in DR is to no longer need to use or pay for Premiere, for a lot of us, so I think this needs to be looked at?

I can stomach the waveform syncing in DR being pretty much non-functional - I bought Pluraleyes long ago... but it becomes super frustrating to realise DR won't even play nicely with Pluraleyes.
Offline

Uli Plank

  • Posts: 3538
  • Joined: Fri Feb 08, 2013 2:48 am

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostFri Aug 24, 2018 5:30 am

I've seen that before, but I was under the impression it was fixed – well obviously only for the formats we use regularly.
Could you try to re-wrap (not transcode) those offending files into another wrapper? I admit it should be fixed, but it might be a workaround. Did you check the TC with a program like Media Info? Is there a different codec shown in PluralEyes vs. Resolve if you just look at the clips?

BTW, I had footage in the past that was not synced by Premiere, but by PluralEyes. Fortunately, we are getting more stuff with proper TC and/or clappers recently.
Resolve Studio and Fusion Studio
iMac 2017 Radeon Pro 580 8 GB VRAM and 32 GB RAM
Offline

malkazoid

  • Posts: 16
  • Joined: Tue Jul 03, 2018 10:07 pm
  • Real Name: Malcolm Wright

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostSat Aug 25, 2018 12:40 pm

Uli Plank wrote:I've seen that before, but I was under the impression it was fixed – well obviously only for the formats we use regularly.
Could you try to re-wrap (not transcode) those offending files into another wrapper? I admit it should be fixed, but it might be a workaround. Did you check the TC with a program like Media Info? Is there a different codec shown in PluralEyes vs. Resolve if you just look at the clips?

BTW, I had footage in the past that was not synced by Premiere, but by PluralEyes. Fortunately, we are getting more stuff with proper TC and/or clappers recently.


Thanks Uli. I was wondering if it had to do with the format.
In the future, if all it takes is for me to rewrap the footage so I can let go of Premiere for good, that could be workable.
Like you say though, this should be fixed!
Working to a deadline so I can't test the rewrap right now - I'll get to it!
Offline

Vi-Dan

  • Posts: 3
  • Joined: Wed Aug 15, 2018 1:52 pm
  • Real Name: Vi-Dan Tran

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostSat Sep 15, 2018 8:30 am

OH THANKS GOD!! Finally found a post about this problem!

I have exactly the same issue!
Working with davinci resolve 15.1 now and hoped the sound problem would be fixed.
But still same problem, especially when u are working with 15-20 audio tracks in the timeline (narrative projects with dialouge, foleys, atmosphere, music).
Its driving me nuts....

Working sound design in davinci is not really consistant.
Working with ursa mini 4.6 and zoom h6 and noticed the same.
For longer takes > clips getting out of sync
Further more:
- sometimes some audio clips re making a weird echo stereo sound , after restart the project bug is gone
- sometimes some audio clips gettin quieter, same again, after restart normal again
- sometimes crossfades + noise reduction makes weird cracky sounds
- sometimes audio clip is just damaged, cracky, making weird sounds

Return to DaVinci Resolve

Who is online

Users browsing this forum: etang77, steveremich and 35 guests