Removing background noise from audio: what works

A fan in the room, the hum of a cheap microphone cable, the hiss of a laptop's built-in mic: most home recordings have some of it, and it is what makes them sound amateur. Noise removal tools promise to take it out without touching the voice. We mixed five kinds of noise into real recordings of people speaking, cleaned them with our noise remover and measured what came out.

How the noise remover works

Our noise remover uses RNNoise, a small neural network made at Mozilla and Xiph.org, the people behind the Opus audio codec. Every 10 milliseconds it looks at the sound split into frequency bands and estimates, band by band, how much of it is voice and how much is noise, then turns down the noisy bands. The whole program, network included, is a 110 KB download, which is why it runs in real time even on phones.

This design has a clear strength and a clear limit. It learns what voices sound like, so anything that does not sound like a voice and stays roughly the same over time is easy to remove. Sounds that change quickly, or that resemble voices, such as music, are much harder.

The test

We joined six short English recordings from Mozilla Common Voice, real people reading sentences into their own microphones, with short pauses between them: 40 seconds in all. To that we added five kinds of noise, each at a level 10 decibels below the voice, a typical noisy home recording:

• Hiss: even, broadband noise, like a cheap microphone or a recording turned up too far.
• Mains hum: a 50 Hz tone with its overtones, as from a poorly shielded cable.
• Fan: a low rumble with a faint tone, like a computer or air conditioner.
• Keyboard: sharp clicks several times a second.
• Music: chords from an instrument playing in the background.

We also tried hiss at 0 decibels, as loud as the voice, and at 20 decibels, barely audible. Two numbers describe each result. The first is how much quieter the pauses became, which is what you hear between sentences. The second is a standard score (SI-SDR) of how close the whole result is to the original voice, in decibels: higher is better, and every 10 decibels means ten times less leftover noise and distortion.

Results

Steady noise is where this kind of tool shines. Hiss and fan noise dropped by 35 to 37 decibels in the pauses, from clearly audible to essentially silent. Hum went down by 25 decibels; its tones sit where the lowest notes of a male voice are, which may be why the model removed less of it.

Clicks and music are another story. The clicks are too short and too sudden for a model that adapts over tens of milliseconds, and music has the harmonic structure of a voice. Both were reduced by only 6 to 7 decibels in the pauses, and both remained clearly audible.

Light noise: use less strength

One result looks odd at first: with light hiss, 20 decibels below the voice, the score went down from 20 to 17.6 at full strength. The pauses still got 30 decibels quieter, but the network also removed the faint room sound that was in the original recordings and slightly reshaped the voice, and the score counts both as differences.

The Strength setting blends the cleaned sound with the original. On the light hiss, 50 percent gave a score of 21.4 dB, better than both the original and full strength. With the noise 10 decibels below the voice, 75 percent (16.6 dB) and full strength (16.2 dB) were about the same.

Tips for cleaner recordings

• Remove noise before anything else. Compressing or boosting the volume first makes noise harder to separate from the voice.
• Start at full strength for heavy noise and around 50 to 75 percent for light noise, then listen to the pauses and to the end of words, where over-cleaning shows first.
• Fix the source when you can. Turn off the fan, move away from the computer, and record closer to the microphone. A few centimetres closer raises the voice above the noise more than any tool can.
• Do not expect miracles with music or crosstalk. For a voice over a band or another speaker, this kind of tool helps a little, not a lot.

The noise remover works on audio files and on the sound of videos, keeping the picture untouched. Everything runs in your browser, about 30 times faster than real time on the 2013 desktop processor of our test, so an hour of audio takes around two minutes per channel and never leaves your device.