Auto Sync Audio > Based on Waveform | Hardly ever works

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 8:52 pm

For our videoshoots we use one (or multiple) cameras and a ZOOM H6 recorder for audio.
I should specify that the audio from camera and ZOOM are both very clear and audible.

After importing the footage and audio, I select all of the clips, right click and choose "Auto Sync Audio >> Based on Waveform and Append Tracks".

The software does its analysis and that's when the trouble starts.

Often times, it says some clips could not be matched. Even though the exact same number of audio and video clips are selected, all of which can easily synced by, for example Premiere Pro.

The appended tracks aren't even remotely related. Sometimes it syncs the "clap" sound from the slate as significant audio event but all the rest is wrong.

Here's a screenshot of one of the appended tracks, as you can see, the waveform doesn't match at all:
https://cp.sync.com/dl/0fa05f450#4tbphd9j-j7wr9y32-gw4wvtbi-x2yj7vfa

My "work-around" has been to sync audio in Premiere Pro, throw it on one timeline and export the timeline back to Davinci. Which is quite inconvenient.

In the previous betas it didn't work well either and I'm not sure if Resolve14 was any better.
In short: auto sync based on waveform is quite unusable.
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 9:25 pm

Further exploration:

The ZOOM audio is imported as 44100 Hz.
The camera audio is imported as 48000 Hz.

When selecting ONLY one audio and video clip that are supposed to by sync-able (rather than letting the algorithm figure out which clips need to sync) there are still major problems:

It tries to sync the timeline in the beginning but quickly the audio shifts out of proportion. The further in the timeline, the more out of sync.

But here's the kicker: the audio tracks are still equal in length. Although one seems to play much faster, the tracks are still ending at the same spot.

HOW IS THIS POSSIBLE?

Is there a difference in tempo between 44.1 and 48 kHz playback?

I'm also noticing that the Fairlight tab has much more detailed Waveforms than the Edit tab. But, when zooming in on the Fairlight tab, certain zoom levels make the waveform "pop" out of place.

It's crazy!

I don't know what to do here.

Unless I'm supposed to do something special to make 44.1 and 48 kHz tracks play back properly, it's definitely a bug.
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 9:29 pm

Further exploring:

The audio files seem to have a "FPS" of 29.970 DF.
The video files have an FPS of 24.000

The auto-synced clips have a fps of 24.000.

Does this have anything to do with it?

Is there a reason that audio has a "Frames Per Second" setting when there are no frames in audio?
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 9:35 pm

Further exploring:

I've converted the audio to 48 kHz instead of 44.1.

Resolve still imports it with a 29.970 FPS.

I don't know what to do until someone replies here.
Meanwhile I'll have to revert to Premiere Pro. CRAP. I hate Premiere.
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 9:57 pm

Further exploring:

I've re-encoded the video to be 29.970, since the Resolve project was set to that. This is probably why the audio is given the same FPS.

However, even syncing the video clip with the same FPS as the audio, still is wildly out of sync.
So perhaps it has nothing to do with the framerate.

(Makes no sense to have FPS for audio still, but whatever).
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostWed Jun 20, 2018 11:03 pm

Further exploring:

I've edited the video in Premiere and exported it as XML, then imported that timeline in Resolve.

Result? The video and audio clips are there, but the audio is completely offset. Unusable.

Sigh. Guess I can't use Resolve until this gets fixed.
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

Rohit Gupta

Blackmagic Design

  • Posts: 1045
  • Joined: Wed Aug 22, 2012 5:00 am

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostThu Jun 21, 2018 2:58 am

TornadoTwins wrote:For our videoshoots we use one (or multiple) cameras and a ZOOM H6 recorder for audio.
I should specify that the audio from camera and ZOOM are both very clear and audible.

After importing the footage and audio, I select all of the clips, right click and choose "Auto Sync Audio >> Based on Waveform and Append Tracks".

The software does its analysis and that's when the trouble starts.

Often times, it says some clips could not be matched. Even though the exact same number of audio and video clips are selected, all of which can easily synced by, for example Premiere Pro.

The appended tracks aren't even remotely related. Sometimes it syncs the "clap" sound from the slate as significant audio event but all the rest is wrong.

Here's a screenshot of one of the appended tracks, as you can see, the waveform doesn't match at all:
https://cp.sync.com/dl/0fa05f450#4tbphd9j-j7wr9y32-gw4wvtbi-x2yj7vfa

My "work-around" has been to sync audio in Premiere Pro, throw it on one timeline and export the timeline back to Davinci. Which is quite inconvenient.

In the previous betas it didn't work well either and I'm not sure if Resolve14 was any better.
In short: auto sync based on waveform is quite unusable.


Are you able to send a few clips and the audio files so we can check it out? Please send the originals.

Please upload to Dropbox, or similar, and PM me a link. We'll investigate.
Rohit Gupta

DaVinci Resolve Software Development
Blackmagic Design
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostThu Jun 21, 2018 4:38 am

Certainly, I’ll send you the files tomorrow morning by replying to your DM.
Thanks for looking into this!
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

Uli Plank

  • Posts: 4192
  • Joined: Fri Feb 08, 2013 2:48 am

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostThu Jun 21, 2018 6:04 am

As much as I like DR, I’ve never found waveform syncing acceptable in any NLE.
Rather give PluralEyes a try, it works very well for us. If I remember correctly, they have a fully functional 10 days demo.
Resolve Studio and Fusion Studio
iMac 2017 Radeon Pro 580 8 GB VRAM and 32 GB RAM
Offline

TornadoTwins

  • Posts: 19
  • Joined: Sat Apr 14, 2018 6:01 pm
  • Real Name: Efraim Meulenberg

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostThu Jun 21, 2018 10:44 pm

@Rohit Gupta - Thanks, I've sent you an Enrico a DM with the project and audio/video files.

@Uli Plank - I do like the idea of PluralEyes... however, since it's about the same price as Davinci Resolve Studio, and since it does work fine in Premiere Pro, it would be really great if I didn't have to buy anything extra.
That said, great tip, thanks for sending this along!
Davinci Resolve Version: 15.0.0B.057
OS: Windows 10 Pro
Processor: i7-6700k CPU @4.00Ghz 4.01Ghz
RAM: 32 GB
Video Card 1: GeForce GTX 970 – GUI Only
Video Card 2: GeForce GT 740
Hard Drives: Three SSD’s of 1TB, one of SSD of 2 TB
Offline

malkazoid

  • Posts: 38
  • Joined: Tue Jul 03, 2018 10:07 pm
  • Real Name: Malcolm Wright

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostFri Aug 24, 2018 3:18 am

Hello all,

I'm using DR 15 Studio, and experiencing similar woes with waveform syncing.
Namely zero sync attempts have been successful in DR, always resulting in all clips being declared as not having matches AND my attempts to use Pluraleyes 4.1.6 result in the same problems experienced by others.

If I try to export an xml of a timeline and bring it into Pluraleyes - Pluraleyes says there are invalid elements on the timeline and will not import.

If I use Pluraleyes on its own, pulling the media directly into it, sync, then export an xml, this xml results in clips with portions declared as offline, and not lining up with the audio. It appears to be some weird timecode offset in the clips affected. The clips have proper timecode and are Sony MXFs shot on a FS7m2.

Like others, if I first load the xml from pluraleyes into Premiere, then export a new xml from Premiere, DR happily brings in this new xml, and I'm good to go.

I'm not a software engineer, but there is clearly something about the Pluraleyes written xml, that Premiere is happily reading correctly, but DR is not.

The whole point of editing in DR is to no longer need to use or pay for Premiere, for a lot of us, so I think this needs to be looked at?

I can stomach the waveform syncing in DR being pretty much non-functional - I bought Pluraleyes long ago... but it becomes super frustrating to realise DR won't even play nicely with Pluraleyes.
------------------------
Davinci Resolve Studio
Mini panel
Windows 10
Nvidia GTX1080ti
Offline

Uli Plank

  • Posts: 4192
  • Joined: Fri Feb 08, 2013 2:48 am

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostFri Aug 24, 2018 5:30 am

I've seen that before, but I was under the impression it was fixed – well obviously only for the formats we use regularly.
Could you try to re-wrap (not transcode) those offending files into another wrapper? I admit it should be fixed, but it might be a workaround. Did you check the TC with a program like Media Info? Is there a different codec shown in PluralEyes vs. Resolve if you just look at the clips?

BTW, I had footage in the past that was not synced by Premiere, but by PluralEyes. Fortunately, we are getting more stuff with proper TC and/or clappers recently.
Resolve Studio and Fusion Studio
iMac 2017 Radeon Pro 580 8 GB VRAM and 32 GB RAM
Offline

malkazoid

  • Posts: 38
  • Joined: Tue Jul 03, 2018 10:07 pm
  • Real Name: Malcolm Wright

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostSat Aug 25, 2018 12:40 pm

Uli Plank wrote:I've seen that before, but I was under the impression it was fixed – well obviously only for the formats we use regularly.
Could you try to re-wrap (not transcode) those offending files into another wrapper? I admit it should be fixed, but it might be a workaround. Did you check the TC with a program like Media Info? Is there a different codec shown in PluralEyes vs. Resolve if you just look at the clips?

BTW, I had footage in the past that was not synced by Premiere, but by PluralEyes. Fortunately, we are getting more stuff with proper TC and/or clappers recently.


Thanks Uli. I was wondering if it had to do with the format.
In the future, if all it takes is for me to rewrap the footage so I can let go of Premiere for good, that could be workable.
Like you say though, this should be fixed!
Working to a deadline so I can't test the rewrap right now - I'll get to it!
------------------------
Davinci Resolve Studio
Mini panel
Windows 10
Nvidia GTX1080ti
Offline

Vi-Dan

  • Posts: 3
  • Joined: Wed Aug 15, 2018 1:52 pm
  • Real Name: Vi-Dan Tran

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostSat Sep 15, 2018 8:30 am

OH THANKS GOD!! Finally found a post about this problem!

I have exactly the same issue!
Working with davinci resolve 15.1 now and hoped the sound problem would be fixed.
But still same problem, especially when u are working with 15-20 audio tracks in the timeline (narrative projects with dialouge, foleys, atmosphere, music).
Its driving me nuts....

Working sound design in davinci is not really consistant.
Working with ursa mini 4.6 and zoom h6 and noticed the same.
For longer takes > clips getting out of sync
Further more:
- sometimes some audio clips re making a weird echo stereo sound , after restart the project bug is gone
- sometimes some audio clips gettin quieter, same again, after restart normal again
- sometimes crossfades + noise reduction makes weird cracky sounds
- sometimes audio clip is just damaged, cracky, making weird sounds
Offline

efeldmaz

  • Posts: 1
  • Joined: Thu Nov 29, 2018 1:12 am
  • Real Name: ellen feldman

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostThu Nov 29, 2018 1:19 am

[Solved my issue]

Not sure if this will help solve the problem on the waveform end but, I had my audio coming in at 24 FPS and my video at 23.97. I let davinci's auto promo to change my multiple frame rate and it then was able to sync properly by timecode. It only changes the audio's frame rate not the video. Maybe the waveform is also weird because your mixed sound is a different frame rate then you cameras?

Hope this may help someone too -

Ellen
Offline

Kevin Good

  • Posts: 14
  • Joined: Thu Feb 06, 2014 10:52 pm
  • Location: DC/Maryland/Los Angeles

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostThu Dec 20, 2018 6:20 pm

Any progress on this? I'm trying to use this feature for the first time as I try to migrate from cutting in Premiere > XML > Grading in Resolve, to just doing the whole thing in Resolve. I'm off to a bad start. It won't sync based on waveform at all.

The footage is two cameras, with one sound recorder. The footage does NOT have any audio on tracks 1 & 2, the reference audio for sync is on tracks 3 & 4 (Sony Fs7), which I think might be what's causing the trouble. But if so I don't know how to fix it.

The external recorder audio was just one boom (mono files) to an external recorder with timecode that resets to 0 on every take. I assume in the 'waveform' sync the timecode would be totally ignored, but worth noting as we try to figure out what is causing this.

I usually get an error message saying the clips could not be synced. Every once in a while it thinks it's successful, but has done it completely wrong matching entirely wrong clips/timing.

I even selected just 1 matching audio & video clip so it didn't have to guess what goes with what, just find the sync, and it failed as well.

So basically I cannot get any auto sync based on waveform to work at all under any circumstances and it's very frustrating.

Running Resolve Studio 15.2.2.

Thanks for any input y'all!
Kevin
Offline

ohimbz

  • Posts: 23
  • Joined: Wed Nov 14, 2018 8:54 am
  • Location: Romania
  • Real Name: Musetoiu Florin

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostThu Dec 20, 2018 7:19 pm

My work-flow for synchronizing 2 cameras (Sony A7III) and 1 audio recorder (Zoom H4n) is as follows:

1. make timeline with the desired files to be syncronized

Image

2. Export Timeline as FCP 7 XML v5 file

3. Import in Syncaila to be synchronized (i found that this program works regardless of different FPS files 25 or 100 in the same timeline, Plural Eyes does not work with different FPS files)

4. Import back in Resolve the timline .

Do no use - Import Media from XML, use Import Timeline from XML Ctrl+Shift+I (You can uncheck the Automatically Import Source clips into media pool, this way you don`t get duplicated video files in your project)

Image
Offline

Freelosof

  • Posts: 3
  • Joined: Sat Feb 02, 2019 6:27 pm
  • Real Name: Sergey Nesterov

Re: Auto Sync Audio > Based on Waveform | Hardly ever works

PostMon Feb 11, 2019 10:32 am

Hi. I have the same issue.
My friend shot his first movie and I help him with editing. They shot H.264 at 23.98 fps with 48000 sound and recorded sound to zoom as 44100. I choose DVR to skip conforming step from another NLE before grading. The sound from camera and the sound from zoom differs a little bit because they were in different places phisically and sometimes the sound from camera better to hear and vice versa so I need to use sound from both sources while editing.
I tried 2 ways to got 24fps finally.
1. Edit 23.98 in 23.98 project than finally export as 24p. This export option preserve frames but change length so I got a little sound pitch shift.
2. Edit 23.98 clips as 24p conformed in 24p project. This setting I choose.
But in both cases I can't use autosync options based on waveform.
If i sync h.264 clip with audio from zoom on timeline manually they match perfectly within the length of video. But if I try to autosync them by waveform and append tracks than sync doesn't match. I noticed what the audio from zoom tracks in synced clips sounds faster. It seems what 44100 sample rate just plays as 48000.
And it is impossible to sync clips manually and keep all audio tracks. Even if I try to correct the sync of clips with appended tracks as it is described in chapter "offseting the Sync of Previously Synced Clips" (turn off link, adjust and than turn on linked clip back) I immediately loose the audio from camera. So the only way to add audio tracks and to keep original audio is to use autosync and if autosync doesn't work so you have no options.
Also I noticed the length of zoom audio is different in 23.98 and in 24 projects, I don't know what this means. Not just a few frames because of 23.98/24 frame rates difference but second or two.

Return to DaVinci Resolve

Who is online

Users browsing this forum: Benoit Breton, Bing [Bot], Chris Mierzwinski and 21 guests