Setting up audio search tools

Setting Up Audio Search Tools

One of our particular areas of interest in computational musicology here at Goldsmiths is audio search: building a database of music recordings and being able to search in that database using audio queries.

From an earlier collaborative project, OMRAS2, we have a database system that can be used for searching audio collections. AudioDB stores numerical data extracted from audio signals and is able to match fragments of such numerical data against each other. This numerical data we call feature vectors, essentially a list of numbers that represent some (usually) time-varying property—or feature—of the music such as harmonic content or timbre. In order to judge the similarity of two feature vectors, AudioDB uses a distance measure which determines how close the two vectors are to each other in the mathematical space they occupy. Doing this distance matching alone would be very computationally expensive, so AudioDB employs a technique called approximate nearest neighbour matching. It uses a so-called randomized algorithm called Locality Sensitive Hashing to work out close matches within a given probability of correctness, sacrificing accuracy for speed.

In this post we'll show how to install AudioDB and an audio feature extractor called Sonic Annotator. The procedure is slightly different depending on your platform (Windows, Mac OS 10, GNU/Linux).

The AudioDB system comes in two parts: the core library, libaudioDB which implements feature ingestion and storage and feature distance matching, and a command line interface to allow a user to interact with a database. The code for both parts is now hosted on Github.

Installing libaudioDB on GNU/Linux
Installing audioDB on GNU/Linux
Installing libaudioDB on Mac OS
Installing audioDB on Mac OS
Installing Sonic Annotator
Windows

Installing libaudioDB on GNU/Linux

We'll start with installing the core library on GNU/Linux.

Ensure that the following tools are installed on your system:
- git
- gcc
On Debian and derivatives, you can install them like this:
```
 $ sudo apt-get install git gcc
```
Install the GNU Scientific Library. On Debian and derivatives, you can install it like this:
```
  $ sudo apt-get install libgsl0-dev
```
Download the libaudioDB code from https://github.com/TransformingMusicology/libaudioDB:
```
 $ git clone https://github.com/TransformingMusicology/libaudioDB.git
```
Compile libaudioDB:
```
 $ cd libaudioDB
 $ make
```
Run the tests:
```
 $ make test
```
Install the shared library:
```
 $ sudo make install
```
By default, it installs into /usr/local. You can change the PREFIX variable to choose a different location:
```
 $ sudo make PREFIX=$HOME/.local install
```
Clean up the build files and test outputs:
```
 $ make clean
```

You should now have libaudioDB.so.0.0 in /usr/local/lib.

Installing audioDB on GNU/Linux

Next, we'll install the audioDB command line interface on GNU/Linux.

Install help2man and gengetopt. On Debian and derivatives, you can install them like this:
```
  $ sudo apt-get install gengetopt help2man
```
Download the AudioDB code from https://github.com/TransformingMusicology/audioDB:
```
 $ git clone https://github.com/TransformingMusicology/audioDB.git
```
Compile AudioDB:
```
 $ cd audioDB
 $ make
```
If you chose to install libaudioDB in a non-standard location, you'll need to make sure the linker can find it and the compiler can find the includes. For example, you may want to specify the libaudioDB include and linker flags to make:
```
 $ make ADB_INCLUDE=-I$HOME/.local/include LIBAUDIODB="-L$HOME/.local/lib -laudioDB"
```
Run the tests:
```
 $ make test
```
Install the executable and man page:
```
 $ sudo make install
```
By default, it installs into /usr/local. You can change the PREFIX variable to choose a different location:
```
 $ sudo make PREFIX=$HOME/.local install
```
Clean up the build files and test outputs:
```
 $ make clean
```

You should now have audioDB in /usr/local/bin and you should be able to execute it:

    $ audioDB

You should see a short help message. Again, if you've installed libaudioDB in a non-standard location, you'll need to make sure audioDB can find it, for example, by setting LD_LIBRARY_PATH:

    $ LD_LIBRARY_PATH=$HOMR/.local/lib audioDB

Installing libaudioDB on Mac OS

Now for installation on Mac OS. One thing to be aware of before we start is that the standard filesystem on MacOS (HFS+) does not support sparse files. I.e. on other file systems you can allocate a large file (say 4GB) but any blocks containing just zeros will not occupy any actual disk space. On HFS+, on the other hand, a newly allocated 4GB file will take up 4GB of space. This is not generally a problem except for running the test suite which creates about 30 such 4GB files and so consumes 120GB of disk space before it gets cleaned.

On Mac OS the default C compiler has recently changed from gcc to clang. AudioDB compiles with clang without problems, but you will need to make sure the GSL dependency links against the same C++ library as clang uses. (gcc uses libstdc++ whereas clang uses libc++.) This just means that you need to either compile GSL yourself using clang or, if you use a binary distribution, make sure it was compiled with clang. AudioDB will also compile on older Mac OS versions where gcc is still the default compiler. In all the commands below, I'll explicitly set the compiler to clang although this isn't actually necessary as it's the default anyway.

The procedure I'll describe here involves installing everything from source and to make things clean, we'll do it all in a directory I'll call $ADB_SB for audioDB sandbox.

Create a sandbox:

 $ ADB_SB=~/adb-sandbox
 $ mkdir -p $ADB_SB/root

Install GSL

 $ cd $ADB_SB
 $ curl http://ftp.heanet.ie/mirrors/gnu/gsl/gsl-latest.tar.gz > gsl-latest.tar.gz
 $ cd gsl-1.16/
 $ ./configure --prefix=$ADB_SB/root/ CC=clang
 $ make
 $ make install

Install libaudioDB

 $ cd $ADB_SB
 $ git clone https://github.com/TransformingMusicology/libaudioDB.git
 $ cd libaudioDB
 $ make PREFIX=$ADB_SB/root CC=clang CXX=clang++ GSL_INCLUDE=-I$ADB_SB/root/include LIBGSL="-L$ADB_SB/root/lib -lgsl -lgslcblas  -lm"
 $ make PREFIX=$ADB_SB/root CC=clang CXX=clang++ install

You should now have libaudioDB.dylib.0.0 in $ADB_SB/root/lib and compilation flags in $ADB_SB/lib/pkgconfig/audioDB.pc.

Installing audioDB on Mac OS

Again, all the caveats about HFS+ and clang above apply here. And we'll be using the $ADB_SB directory for installation.

Install XZ utils on Mac OS 10.6 and earlier; if you're using a new Mac OS XZ utils is available so you don't need to do this. You can check with the command:
```
 $ tar -Jcf test.tar.xz foo
```
If the error is just that it can't find foo then XZ is available. If it tells you that the command usage is wrong, then there's no -J option meaning no XZ support.
```
 $ cd $ADB_SB
 $ curl http://tukaani.org/xz/xz-5.2.1.tar.gz > xz-5.2.1.tar.gz
 $ cd xz-5.2.1
 $ ./configure --prefix=$ADB_SB/root/ CC=clang
 $ make
 $ make install
```

Install help2man. If you're using Mac OS 10.6 or earlier (i.e. no built-in XZ support) then do this:

 $ cd $ADB_SB
 $ curl http://ftp.heanet.ie/mirrors/gnu/help2man/help2man-1.46.5.tar.xz > help2man-1.46.5.tar.xz
 $ $ADB_SB/bin/unxz help2man-1.46.5.tar.xz
 $ tar xf help2man-1.46.5.tar
 $ cd help2man-1.46.5
 $ ./configure --prefix=$ADB_SB/root/ CC=clang
 $ make
 $ make install

Otherwise, do this:

 $ cd $ADB_SB
 $ curl http://ftp.heanet.ie/mirrors/gnu/help2man/help2man-1.46.5.tar.xz > help2man-1.46.5.tar.xz
 $ tar -Jxf help2man-1.46.5.tar.xz
 $ cd help2man-1.46.5
 $ ./configure --prefix=$ADB_SB/root/ CC=clang
 $ make
 $ make install

Install gengetopt

 $ cd $ADB_SB
 $ curl ftp://ftp.gnu.org/gnu/gengetopt/gengetopt-2.22.6.tar.gz > gengetopt-2.22.6.tar.gz
 $ tar -zxf gengetopt-2.22.6.tar.gz
 $ cd gengetopt-2.22.6
 $ ./configure --prefix=$ADB_SB/root/ CC=clang
 $ make
 $ make install

Install audioDB

 $ cd $ADB_SB
 $ git clone https://github.com/TransformingMusicology/audioDB.git
 $ cd audioDB
 $ DYLD_LIBRARY_PATH=$ADB_SB/root/lib make PREFIX=$ADB_SB/root CC=clang CXX=clang++ HELP2MAN=$ADB_SB/root/bin/help2man GENGETOPT=$ADB_SB/root/bin/gengetopt GSL_INCLUDE=-I$ADB_SB/root/include LIBGSL="-L$ADB_SB/root/lib -lgsl -lgslcblas  -lm" ADB_INCLUDE=-I$ADB_SB/root/include LIBAUDIODB="-L$ADB_SB/root/lib -laudioDB"
 $ DYLD_LIBRARY_PATH=$ADB_SB/root/lib make PREFIX=$ADB_SB/root CC=clang CXX=clang++ HELP2MAN=$ADB_SB/root/bin/help2man GENGETOPT=$ADB_SB/root/bin/gengetopt GSL_INCLUDE=-I$ADB_SB/root/include LIBGSL="-L$ADB_SB/root/lib -lgsl -lgslcblas  -lm" ADB_INCLUDE=-I$ADB_SB/root/include LIBAUDIODB="-L$ADB_SB/root/lib -laudioDB" install

As we're installing everything outside of standard system locations, the make options for AudioDB are pretty long. But once you've got the audioDB executable and libaudioDB.dylib they're both self-contained and can be moved into a standard location.

I'd be very pleased to receive any reports on how this installation procedure is working for others; either comment on this post or submit an issue on Github.

Installing Sonic Annotator

The other part of our audio search toolchain is an audio feature extractor. For several years, our colleagues at Queen Mary University of London have been developing an audio feature extraction framework called Vamp. It allows developers to write audio feature extraction plugins implementing specific features. It also provides a tool to execute those plugins against some audio files, Sonic Annotator.

Sonic Annotator can be downloaded as a binary distribution for GNU/Linux, Mac OS, and Windows from the Vamp site.

Windows

I haven't yet tested the recent AudioDB on Windows. However, earlier versions compiled using the GNU toolchain for Windows, MinGW. Once I've tested this I'll publish a post with the details.

Next steps

In the following post, I will describe how to create a simple database with AudioDB and how to execute a simple query against that database.