Delay compensation | Effective management of plugin latencies in complex mix environments

10 replies [Last post]
jburtner
User offline. Last seen 6 years 47 weeks ago. Offline
Joined: 2007-02-15
Posts:

Hi,

I am interested in understanding various methods to manage delay compensation (plugin latency) using Ardour as the master mixer.

The goal is to have the ability to determine and control (manually and/or automatically) the amount of delay-compensation induced in certain channels so that certain critical specific components of the mix (processed separately) arrive at the master stereo/surround bus with phase coherency for an audiophile grade level of detail.

Part of the aim would be to consider that in addition to LADSPA plugs, the mix would also utilize VST plugins as well as physical I/O routed via jackd through various other outboard hardware including other dsp based processing systems as well as various analog eq's and compressors.

The various routings would potentially include post&pre-fader sends, post&pre-fader inserts, send and insert processing loops fed from both original source channels as well as parallel audio group/bus sub-masters where the original set of audio sources is comprised of sound sources recorded simultaneously in an acoustic space with audible bleed from adjacent microphones.

A common example of this mix situation would be the mixing of a recording of an acoustic drum set with the following complement of audio sources on separate channels (as well as the other instrumentation in the song):

-01- KICK INSIDE
-02- KICK OUTSIDE
-03- SNARE-TOP
-04- SNARE-BOTTOM
-05- HAT
-06- RACK TOM 1
-07- RACK TOM 2
-08- FLOOR TOM 1
-09- RIDE CYMBAL
-10- CRASH CYMBAL
-11- OVERHEAD L
-12- OVERHEAD R
-13- ROOM 1 L
-14- ROOM 1 R
-15- ROOM 2 L
-16- ROOM 2 R

- Each channel has various inserts as needed (Gates, EQ's, Compression) and is sent to the master stereo bus.

- Delay-compensation is applied to each channel as required (depending on the inserts and processing loops) so that the mix of signals at the stereo bus are phase coherent.

- In addition to the stereo bus, a post fader send of each channel is sent to an audio subgroup bus. For example Bus 1&2 as a stereo pair.

- Bus 1/2 then has an external analog compressor and an EQ inserted and brought back into an Ardour mixer channel and sent to the stereo bus. The level of the drum-subgroup audio is brought up so that it is just under the original mix of drum audio.

- In addition to Bus 1/2, several more stereo post-fader drum subgroups are set up which is also post insert from the original source tracks. These are matrixed and remixed to create a L+R and an L-R which can be processed differently then de-matrixed back to a stereo signal which is then sent to the stereo bus like this:

----------------------------
L (+) ->
SUMMED = L+R -> [EQ + COMPRESSION] = L+R-PROC
R (+) ->

L (+) ->
SUMMED = L-R -> [EQ + COMPRESSION] = L-R-PROC
R (-) ->

Then bussed and mixed so that

L+R (+) ->
SUMMED and sent to Stereo Bus L
L-R (+) ->

L+R (+) ->
SUMMED and sent to Stereo Bus R
L-R (-) ->
----------------------------

The matrixed/de-matrixed audio is then brought up to the desired level in the stereo bus.

The problem to solve here is how set up the mix so that it is possible to determine and apply delay-compensation so that there is no comb filtering (phase-coherency/latency differences) in the stereo bus between the three groups of mix sources (Original, Sub 1/2, and matrixed/dematrixed).

This is an example and could be applied to various mixing situations with varying levels of complexity and numbers of audio sources and subgroups which must be phase coherent in the stereo bus and because of the signal flow delaying the source tracks causes the rest of the tracks to also be delayed and retain their latency offset.

I appreciate your input, feedback, and postings.

Cheers,
jonathan

jburtner
User offline. Last seen 6 years 47 weeks ago. Offline
Joined: 2007-02-15
Posts:

The idea is to be able to get the delay-comp right so that if each of the three paths had no processing and were set to unity such that if a discrete mono bounce of each was created then mixed with a polarity-reversed version of any of the other two then it would cancel and be silent.

Thx,
jonathan

Reuben
User offline. Last seen 4 years 32 weeks ago. Offline
Joined: 2006-03-30
Posts:

Ardour already automatically accounts for latencies introduced by ladspa plugins. I don't think that is really possible though with VST plugins since, to my knowlege, vst plugins don't have a built in interface to report their latencies. With those you will have to manually adjust for the delays using the ladspa "Artificial Latency" plugin.

Reuben
User offline. Last seen 4 years 32 weeks ago. Offline
Joined: 2006-03-30
Posts:

Ardour already automatically accounts for latencies introduced by ladspa plugins. I don't think that is really possible though with VST plugins since, to my knowlege, vst plugins don't have a built in interface to report their latencies. With those you will have to manually adjust for the delays using the ladspa "Artificial Latency" plugin.

paul
paul's picture
User offline. Last seen 1 day 1 hour ago. Offline
Joined: 2006-03-16
Posts:

reuben, actually VST has the mechanism, and its LADSPA that sort of doesn't. Steve Harris and I came up with a kludge solution for LADSPA (an output control port called "latency"), and a few other LADSPA developers supported it. in VST, its a host->plugin callback called getInitialDelay().

paul
paul's picture
User offline. Last seen 1 day 1 hour ago. Offline
Joined: 2006-03-16
Posts:

jburtner, i thought i had already replied to this, but i must never have submitted it after a preview.

in theory, Ardour already does everything you need for this to work (i spent months working on this in the spring of 2006). that is, it will detect both:

* plugins with internal latency (e.g. FFT-based FX)
* signal routing paths that cause latency

and will adjust the timing of other tracks to make everything line up.

there are 2 provisos:

a) it requires that every plugin accurately report its internal latency. there are a few that have non-zero values and do not report it. there isn't much you can do here. there is an artificial latency plugin which reports latency without causing any, so you could play around with this to adjust the sync but its time consuming to get it dead on.

b) if any of your signal routing leaves JACK to go out through external hardware, then you will need to have told jackd about the systemic latency of your A/D and D/A converters. there is a nice little command line utility called jdelay that will measure this in a loopback setup. you will numbers from it that can be passed to JACK and will then be passed along to an app like ardour for use when calculating signal path latencies.

Basically, you should be able to just do what you want, and find that it works. If not, please feel free to ask. IRC is much faster to solve problems with this kind of thing, however.

jburtner
User offline. Last seen 6 years 47 weeks ago. Offline
Joined: 2007-02-15
Posts:

Hey Paul & Reuben:

Thanks for the responses. I did read some other threads about this before posting which explain the same thing about using the artificial latency plug. That would help determine.

Would the jdelay cmd-line tool mentioned automatically store and pass that latency to ardour or does that need to be managed manually?

Paul, I would like to understand better how ardour's latency compensation works. If you could point me in the direction of the relevant sections in the source files I might be able to better understand and ask more intelligent questions.

It sound's complex based on the amount of time you spent sorting it out. A couple questions do come to mind now though:

- Where is Ardour's automatic-latency-compensation inserted in the signal flow?
- If it is automatically inserted somewhere in the source track/mixer-channel do the sends from that channel also incur the same latency?
- If multiple busses are fed from an Ardour-mixer-channel and further down the signal flow chain with different latencies incurred along the way how does Ardour know which to delay more/less?

I appreciate your help in understanding this.

Thx in advance,
jonathan

paul
paul's picture
User offline. Last seen 1 day 1 hour ago. Offline
Joined: 2006-03-16
Posts:

Unless you have a lot of experience with C++, threads, realtime code and more, there is absolutely no reason for you trying to look into this at the source code level.

Latency compensation works by delaying track playback. If there are two tracks, and one of them has an apparent latency (either because of plugins or signal routing or both) or 15ms, then the other track will delay the onset of its playback by 15ms. If there are two delayed tracks with latencies of 23msec and 6 msec, and one non-delayed track, the non-delayed one will delay itself 23 msec, the less delayed track will delay itself 17 msec and the more delayed track will not delay its playback at all.

Sends will experience the same delay, because the signal is delivered to them "after" the playback signal has been set up. See http://ardour.org/files/manual/sn-tracks-and-busses.html for more details on this.

Ardour can measure latency at all points in the signal flow. If you set up a feedback loop anywhere, there is no single answer to the question of the "correct" latency, so Ardour will pick one. In all other cases, a bunch of trivial but deep code figures out latencies for all routing/plugin possibilities and seeks to do the right thing. Basically, you should just Ardour, and if it doesn't work correctly, file a bug in mantis so that I can work on it.

Numbers determined by jdelay have to manually passed to jackd at startup time. If you use the ALSA backend, then the -I and -O options for the backend supply "systemic" or "extra" latency figures for input and output respectively. QJackctl has fields in its setup dialog to specify these. Note that to use jdelay, you must set up a loopback signal path (so that the output from the audio interface is fed through whatever devices you use and then back into the audio interface).

Reuben
User offline. Last seen 4 years 32 weeks ago. Offline
Joined: 2006-03-30
Posts:

@Paul: What about LV2? Is this more of a standard feature in the LV2 spec?

roaldz
User offline. Last seen 1 year 39 weeks ago. Offline
Joined: 2008-11-12
Posts:

I'm sorry about gravedigging this thread, but I have a question about latency as well.
If I put JAMin on an insert of a track, I can clearly hear it introduces latency. Do I have to use the "Artificial Latency" plugin on this track? I know that if JAMin is used on the master bus, it wouldn't matter.
If this is true, I assume that if I use an external hardware insert (like a reverb), and my jack setup is correct by using Jdelay, I'd have to use the artificial latency plugin to compensate for the system-latency introduced by the external hardware (digital reverb AD-DSP-DA) on the specific track.

Or am I missing it completely, and do I have to use "Artificial Lateny" on every other track?
Please let me know if I'm on the right track.:)

Cheers,

Roald

nowhiskey
User offline. Last seen 4 weeks 6 days ago. Offline
Joined: 2006-09-30
Posts:

hallo,
i suppose that in your set up, jamin is connected to ardours master bus, but to your sound cards outputs as well.
either disconnect jamin from the your playback device, or simple start it with 'jamin -p'.that case jamin want connect to the playback.
other ways, i might be completely wrong, but since you say that you hear the latency, it was the first which came into my mind.

cheers,
doc