Purists vs pragmatists

When music is transmitted through real-time wireless digital technology – such as in live shows, music production, virtual choirs, and more – the performer’s experience is impacted by latency. The contention lies in what degree of delay is tolerable, and therefore where we should be setting our sights as audio engineers.

Previously I worked in Bluetooth® to improve audio quality. Some of my colleagues held a frustrating position on sampled audio. Their argument was if most people can’t hear above 10kHz, what’s the point in aiming for 44.1kHz. In their eyes (or ears), aiming for 96kHz or 192kHz was ridiculous – comedic.

Much energy was burned in winning the debate. Happily, the purists won; the nay-sayers moved on. Not without stalling Bluetooth® as a reputable medium for transporting audio, though.

Even when Bluetooth® LE is becoming more ubiquitous – the same argument exists.

I’m starting to hear a similar response regarding latency. One side argues, “10 milliseconds (ms) here or there doesn’t really matter” while those at the other end of the spectrum say, “anything over 0 ms won’t be tolerated.”

Given we can measure latency empirically, as opposed to the highly subjective nature of audio, I’ll begin by outlining where delays are introduced in a high-quality wireless audio link.

I’ll also consider what degrees of latency different users can work with, notice, or find debilitating.

This may go some way in avoiding several wasted years of pushing rocks uphill again!

Examples of when audio latency makes an experience less enjoyable.

Here are examples where audio latency disrupts the enjoyment of the experience – to varying degrees:

• Live performance (musicians, DJs, theatre)
• Gaming
• Phone and video calls
• Music production (multichannel recording, editing, mixing, processing)
• Watching TV/movies (extending into surround sound synchronisation)
• Lip syncing
• Multi-person online performance (jamming online/virtual choirs)
• Real-world audio/in-person conversations (using hearing aids)

Broadly speaking, each application will have a range of tolerance levels depending on the user’s competence, usage, and sensitivity to latency. A professional vocalist, for example, who performs live for more than 200 days a year, will notice latency significantly more than an occasional karaoke singer.

The natural delay of sound travel due to frequency, temperature and physical obstacles.

Listening to music over Classic Bluetooth® with its 200+ ms latency is totally acceptable as there’s no reference point. However, apply this amount of latency for live musicians and it becomes a huge problem.

Latency is a serious issue for live musicians, as they can experience other players performing at a slower tempo, even if they are not. Natural latency means musicians can’t rely solely on their ears to keep in time and is one of the main reasons why larger bands and orchestras are led by conductors – for visual time representation.

Ironically, musicians are trained to deal with some level of latency, but only at amounts caused by natural acoustics. Frequency, temperature and physical obstacles – which absorb or reflect sound – make sound travel at different speeds. Generally, sound waves on a stage or from a speaker can be measured as speeds travelling at around 342 metres per second. If performers are 3.4 metres apart, this can cause a natural delay between them of about 10 ms.

A musician on stage with several band members must contend with multiple levels of latency caused by different instruments, frequency ranges and their distance from each other.

Still, many performers say they notice even a 10 ms delay, especially if it’s coming from their own instrument.

Research reveals the type of musical instrument – and monitoring environment – impacts the perception of latency.

AES conducted a subjective listening test to identify how intolerable various amounts of latency are for performers in live monitoring scenarios. They discovered the perception of latency depends on the type of musical instrument and monitoring environment (wedges vs in-ear-monitors {IEMs}). This experiment showed the acceptable range of latency is between 42 ms and less than 1.4 ms.

Vocalists can tolerate the least amount of latency (less than 3 ms), followed by drummers (less than 6ms), pianists (less than 10 ms), guitarists (less than 12 ms) and keyboardists (less than 20 ms).

What else do we need to consider?

The audio chain (without a wireless link and for a one-way link) can have a total latency of 10-20 ms.

Why does this happen? This buffering is largely due to:

• Conversion issues: Typically, between analogue and digital domains or frequency and time domains.
• Inconsistent block size across a chain of processing modules: Various digital signal processing algorithms work on blocks of samples, rather than one sample at a time, which requires buffers processed as a single unit. If the block size isn’t consistent across a chain of processing modules, further buffering is needed.
• The challenges of transferring between systems: Wired or wireless transfer of data between systems – such as between different software algorithms, different chips within a single product, different products, or different locations over a network – requires buffers. Transfers typically happen in blocks and can require retransmissions. This can be further compounded when the data size of the radio transfers doesn’t match the typical size of the audio blocks – requiring further buffering.
• Mismatched clocks: Some systems require crossing clock domains to transfer data which can need additional buffers to handle the slightly different rates, and the processing required. Fortunately, some professional systems are designed to run from a common clock to avoid this problem.

Where do latencies occur in a radio frequency (RF) link?

Let’s next look at Bluetooth® LE and UWB (ultra-wideband) technology as possible RF options.

Sometimes the product designer has the luxury of designing their own protocol and using discrete components, but that’s a topic for another day.

For an off-the-shelf Bluetooth® LE SoC-based design, the lowest latency appears to be around 19 ms using a standard protocol and LC3+. Now we have a one-way link (microphone or IEM) that is at best 29 ms. Double that and we’ve got a compounding issue where the 58ms is way beyond the 42 ms cited by the AES paper.

Virscient technology can lower the RF portion to around 3.8 ms.         

The good news is we can achieve ultra-low latency audio over low-power wireless links thanks to Virscient’s LiveOnAir – a technology we introduced in 2023 for wireless mics and IEMs.

Using the LiveOnAir Bluetooth® LE link with LC3+ can reduce the RF part from 19 ms to 12 ms. This can be reduced even further with a low latency audio solution. This now delivers an RF figure of around 3.8 ms.

The entire production link is now 13.8 ms and a round trip under 27 ms is achievable.

Moving on to UWB: Super-low latencies make ‘magic’ possible.

When I first heard about UWB, I thought it had almost magical properties – super-low latencies and enough bandwidth to support Linear PCM at 24bit, 96kHz sampled audio (remember that hallowed level I mentioned earlier!).

Big-name device manufacturers like Samsung and Apple are starting to support this technology, raising the profile of UWB as an RF solution. More radio vendors are starting to supply it, and currently UWB is largely used for location and access control. However, due to the frequencies UWB works at, it’s prone to body blocking and de-tuning dropout.

For non-real-time applications or when you can rely on reflections for a signal to arrive, this isn’t a cause for concern.

But for real-time audio, glitching is a cardinal sin. That’s why UWB didn’t used to be an option for real-time audio.

 

AntennaWare directly addresses body blocking.

With their patented techniques AntennaWare has rendered body blocking a non-issue – by adding up to an extra 20dB of gain.

UWB is now a realistic option for real-time audio.

From audio input to audio output, latencies are measurable below 3ms with Linear PCM audio. If we return to the live performance set-up, the entire chain is now close to 10 ms which equates to a distance of 10 feet.

While it’s not up to me to say if this level of latency is enough – only performers can make that call – what I can say is performers can remove cables for mics and protect their hearing with IEMs.

What’s equally exciting is, if we have cracked one of the most demanding use cases in live music, then gamers and DJs can also enjoy wireless audio connectivity without being held back by latency.

We’ve come a long way.