Keeping audio and visuals in sync with the Web Audio API

There was an issue with the animated piano keyboard I built for my website JazzKeys.fyi in that the audio and visuals were out of sync when listening on Bluetooth headphones. I had assumed that latency would be taken care of automatically by the OS — there are no such issues when viewing an embedded HTML <video>, for example — but it seems that the Web Audio API does not handle this by default.

It turns out, however, that the Web Audio API’s AudioContext interface has a new property that returns an estimate in seconds of the output latency. The property, outputLatency, is currently supported in newer versions of Firefox and Chrome¹, and allows us to delay starting the visuals and have them be in sync with the audio.

Testing it out permalink

You can test it by running the code below in the browser console.

First check the latency while your audio output is set to the built-in speakers or a pair of wired headphones:

var audioCtx = new AudioContext()
audioCtx.outputLatency

I get 0 (zero) when I just checked in Firefox on macOS.²

Now set the audio output to a pair of Bluetooth headphones and check the latency again:

audioCtx.outputLatency

I get 0.17780041666666666 seconds.

If I then switch back to the built-in speaker it‘s 0.02485258333333333 seconds. I can’t explain that; feel free to leave a comment if you know the reason.

You can alternatively test it on an updated version of the animated keyboard (below). Output latency will be shown beneath the controls when you press Play.

Safari and older browsers permalink

Because browser support is incomplete, when using outputLatency we should first check that the property exists:³

const hasOutputLatency = window.AudioContext && 'outputLatency' in window.AudioContext.prototype ? true : false;

if (hasOutputLatency) {
  console.log('AudioContext.outputLatency is available');
}

Regarding Safari, I see there is a fixme in the WebKit code, so hopefully the feature will be available soon on iPhones and iPads.

Further reading permalink

There is some great technical background in this post by a Paul Adenot, an engineer at Mozilla who added outputLatency (along with baseLatency and getOutputTimestamp) to Firefox.