What is audio fingerprinting?
Say you're out with your friends at a bar or a club. As you're enjoying an evening of friends and music, you hear a great new song. You'd love to know what it is, and who it's by, but what are your options to find out?
Sure, you could start shouting over the music to ask your friends what it is, taking the chance that they don't know either and missing the music in the process.
Maybe you could walk up to a nearby hipster and ask them, although then you run the risk of an exasperated response whether (a) you didn't know that the music was their favourite underground post-acid-grime artiste or (b) it was actually Lady Gaga.
Or, finally, you could simply reach in your pocket for your smartphone, record a few seconds of the song, and send it off to a music identification service, and within seconds it will tell you the song title, the artist that recorded it, and even the year it was released.
Have you ever wondered what kind of electronic wizardry makes that bit of magic possible? Well, it's called audio fingerprinting, and it's carving out a market for itself that extends not only to music lovers, but also copyright holders hoping to safeguard their own holdings. It works by combining two basic elements: a database of unique audio fingerprints of millions of audio files, and software that can process and analyze entries while searching that database for a match.
Problems and solutions
Traditionally, using digital approaches to identify electronic sound files has proved problematic due to the nature of digital audio itself. Depending on the individual and the listening equipment/environment the human ear will hear a recording of Elvis' Heartbreak Hotel on a CD as very similar to one played from an MP3 encoded at a half decent bit rate.
Putting all audiophile quality arguments aside, as far as the typical music listener can tell both recordings are equally Heartbreak Hotel. In contrast, a computer being asked to open, store, or catalog those same two recordings will see them as completely different. If you consider adding formats like FLAC and AAC to the equation, the same musical work could easily appear to be several different pieces depending on what format the music was encoded in. Even though the audio that eventually finds your ear might sound the same, to your computer they might as well be in completely different languages.
To work around that discrepancy, the process of audio fingerprinting uses a software algorithm to take a recorded piece of audio from any medium and create a digital summary of some of that particular recording's attributes. Using parameters like frequency, intensity, and time, the fingerprinting software creates a virtual map of peaks and anchor points for those attributes.
For instance, if a recording has a spike of 823 Hz at the 1 minute mark, the program will place a 'marker' there. This is, of course, happening at a rate of several times a second. Over the course of the audio sample, those marks will be connected by a line by the fingerprinting program, and that line will become one of many, the unique combination of which will become that recording's audio fingerprint. The resulting audio fingerprint, also known in technical circles as an acoustic fingerprint, is not altogether different from an actual fingerprint. Much like the kind you find on your own fingertips, every audio fingerprint is unique, and every piece of recorded audio has them.
The fingerprint is then stored in a database. Some of these databases are available online, a good example being Acoustid which is the fingerprinting service bliss uses.
Back to the club
When you want to know what that song is and you record that audio sample with your smartphone, the algorithm runs in the same way that was used to create the original database of fingerprints. The created audio fingerprint is compared to the online database until it finds a match. It doesn't even take an entire song's worth of data to do this. Typically, a few seconds is more than enough. The program only needs to find enough similarities between your entry and the service database to provide a positive match up.
For music lovers, there are already a plethora of services available to make use of audio fingerprinting technology. bliss, for example, can help you investigate some of those unlabelled tracks in your collection that you've long since given up on remembering where they came from. It can take each track and compare it against MusicBrainz extensive user maintained free access database, and add the appropriate tags to the music files once it makes a match. The correct title, artist, album, and even release date are added to your library automatically.
Companies like Shazam offer apps that you can install directly on your mobile devices, such as an Android smartphone, iPad, or iPhone. After recording a few seconds worth of music from a particular song, the program sends the file to Shazam to find a match in their database. If it finds one, you'll receive a message back providing you with all of the track's pertinent information. From there, you can even opt to search for YouTube videos related to the song, or even buy the track and have it downloaded to your device library.
In addition to the home listener, audio fingerprinting benefits copyright holders as well by offering commercial software that can be used to help find and identify copyrighted work. Companies like ACRCloud already operate a database containing over 40m tracks, which it uses to not only track copyrighted work online, but even from radio and television.
Thanks to audio fingerprinting, identifying and cataloging digital music has never been easier or more precise.
Thanks to a2gemma, BotheredByBees and paulius.malinovskis for the images above.