VST plugins pushing audio out of sync

I am mixing music in Fairlight for a hobby project with 15+ contributors singing/playing to a music track.

Everybody is in sync, and playback is working just fine. (All videos are in a 4x4 grid that is cached, so there's no video overhead)

All source tracks are mono, and they're going into submixes (vocals/guitar/bass/etc), then into a stereo master bus.

Some tracks have one or two (maximum) VST track fx on them (breath removal, de-ess, for example). Most tracks have Fairlight compression and/or EQ on them.

Most of the time things are fine but, sometimes, I'll start playback and one track will be delayed by a couple of seconds. Turn the Inserts off and it's back in sync. Turn it on, and it's out of sync again.

It's not laggy, it's not stuttering, it's just one of the tracks with fx jumps out of sync.

Any reason why that is?

The videos are mostly H.264 files, of varying framerates and sizes. Some of the audio is compressed audio direct from the video, some are seperately supplied .WAV files.

System spec in signature

Audio plugins introduce latency in the path which may affect sync.

You do not specifically mention which plugins are instantiated but from the description these may be iZotope RX plugins.

The iZotope RX Breath Control plugin, for example, introduces 26,623 samples (555 ms) of latency.

If the plugin is instantiated as a Clip-based effect, right click the Audio Clip and select either:
"Cache Audio Effects"
"Bounce Audio Effects"

For Track-based effects, use:
Timeline menu > Bounce Selected Track to New Layer
Timeline menu > Bounce Mix to Track

Alternatively, configure iZotope RX as an External Audio Process and use the Breath Control module in the standalone application: Preferences > System > Audio Plugins > Setup External Audio Processes

Yeah, as Reynaud rightly suggested, it will be the izotope breath control pushing it late. It has mega latency. Even on my Pro Tools rig with full auto delay compensation it is out of time, so izoptope stuff like that I use audiosuite and print it to the clip, or send it out to the stand alone RX using RX Connect, process and send it back to the timeline.. the external audio process setup Reynaud suggested is the same sort of thing on Fairlight. The other plugins like the de-esser will be fine. I'd try what he suggested, putting it on as a clip effect and cache audio effects on the clip, or the external process.
I think if you render it as a track effect it may still be out of time,,,but not 100% on that.
Or, just don't worry about debreathing the vocals, they are part of the emotion of a vocal performance... or if they really worry you use key frames in the clip volume and turn them down.

I think to get full delay compensation with Fairlight you need to have the audio accelerator card installed (I think), but as I say even on other DAW systems with ADC, some of the Izotope RX stuff is way latent. Izotope themselves told me not to try running dialogue dereverb in real time. The only ones I use regularly in realtime are the dialogue denoising plugins.

Fairlight in version 17 provides latency compensation with both Fixed and Flexi Bus modes in native mode.
The path latency is reflected in the Tracks Index's ADC column.

A Fairlight Audio Accelerator card is required for latency compensation in version 16.

Thanks for clarifying Reynaud.
Cheers.

If the plugin is instantiated as a Clip-based effect, right click the Audio Clip and select either:
"Cache Audio Effects"
"Bounce Audio Effects"

For Track-based effects, use:
Timeline menu > Bounce Selected Track to New Layer
Timeline menu > Bounce Mix to Track

In my experience, caching audio will not correct latency or other anomalies related to third-party VSTs. You actually got this to work with Izotope plugins?

Yeah, as Reynaud rightly suggested, it will be the izotope breath control pushing it late. It has mega latency. Even on my Pro Tools rig with full auto delay compensation it is out of time, so izoptope stuff like that I use audiosuite and print it to the clip, or send it out to the stand alone RX using RX Connect, process and send it back to the timeline.. the external audio process setup Reynaud suggested is the same sort of thing on Fairlight. The other plugins like the de-esser will be fine. I'd try what he suggested, putting it on as a clip effect and cache audio effects on the clip, or the external process.
I think if you render it as a track effect it may still be out of time,,,but not 100% on that.
Or, just don't worry about debreathing the vocals, they are part of the emotion of a vocal performance... or if they really worry you use key frames in the clip volume and turn them down.

I think to get full delay compensation with Fairlight you need to have the audio accelerator card installed (I think), but as I say even on other DAW systems with ADC, some of the Izotope RX stuff is way latent. Izotope themselves told me not to try running dialogue dereverb in real time. The only ones I use regularly in realtime are the dialogue denoising plugins.

Great info, thank you both, gentlemen!