Musical Score Annotation Kit (MuSAK)

18 February 2016
Richard Lewis

MuSAK

MuSAK is the Musical Score Annotation Kit, assembled from off-the-shelf hardware and applications combined with bespoke software.

The existing hardware used includes an iPad, for the live annotation of digitised images of a pre-annotated musical score using a set of purpose-designed symbols. This time-efficient but reductionist approach was complemented by the use of a Livescribe Echo digital pen, which allowed for the creation of additional, qualitative annotations. The entire annotation process was also captured on video. Applications and bespoke software include the Union Platform 3 server with custom room module, which ran on a laptop to store the annotation events. The iPad and server were networked using a small battery powered wireless router with a private IP address.

For further details, please refer to Page, at al, ISMIR 2015.

Data

Get started here.

The data was created through various stages. Firstly, PDFs of the musical score were downloaded from IMSLP, and manually annotated by a Wagner expert to capture those musicological elements (such as leitmotifs) that could be determined directly from the score itself. For this purpose, a set of symbols were created. These pages were then digitised, and served to the iPad for re-annotation during the live performance using another custom-designed set of symbols, which capture the idiosyncratic features of the performance. The data is stored as CSV, and another version of the score was produced by rendering these annotations onto the score (resulting in a new digital image).

A Livescribe Echo digital pen was used to create either audio files (from dictation of notes), or interactive ‘pencast’ documents. The annotation process was also captured on video, kept in multiple formats.

To explore the graph using the ‘Follow your Nose’ method, clicking on links and URIs in the Q&D Browser, you could start here.

To access the publicly available data and all RDF triples see http://performance.data.t-mus.org/performance/. For navigating through the graph, please see above.

Please note, the identifiers correspond to the following performances:

f3xsaAfdb0I (via browser)
to a performance of Das Rheingold given by the Mariinsky Opera (led by Valery Gergiev) at the Birmingham Hippodrome, UK on November 5th, 2014.
gvX3hrDeTEA (via browser)
to a performance of Die Walküre given by the Mariinsky Opera (led by Valery Gergiev) at the Birmingham Hippodrome, UK on November 6th, 2014.
gvX3hrDeTEA (via browser)
to a performance of Siegfried given by the Mariinsky Opera (led by Valery Gergiev) at the Birmingham Hippodrome, UK on November 8th, 2014.
gvX3hrDeTEA (via browser)
to a performance of Götterdämmerung given by the Mariinsky Opera (led by Valery Gergiev) at the Birmingham Hippodrome, UK on November 9th, 2014.

For further details, please refer to Page, at al, ISMIR 2015.

metaMuSAK

The annotation process resulted in a heterogeneous multimedia dataset, with various and diverse data granularities, inter-dependencies and provenances. We connected and interlinked these resources and their metadata using RDF, including the capture of property qualifiers such as direction and number.

We created a semantic hyper-structure, which bridges several domain-specific OWL ontologies, and forms the foundation for the production of the resulting Linked Data resource. Each resource in the dataset that has a dereferenceable URI has an associated RDF sidecar, providing metadata and describing the resource, linking to other resources in the dataset, and pointing to external authorities where applicable.

For further details please refer to Nurmikko-Fuller, et al, DLfM 2015.

Code

metamusak.py is available from https://github.com/musicog/metamusak.

Papers

  • Kevin R. Page, Terhi Nurmikko-Fuller, Carolin Rindfleisch, David M. Weigl, Richard Lewis, Laurence Dreyfus, David De Roure, 2015. A Toolkit for Live Annotation of Opera Performance: Experiences Capturing Wagner’s Ring Cycle. Proceedings of the 16th ISMIR Conference, Malaga, Spain, October 26-30. Available at http://ismir2015.uma.es/articles/311_Paper.pdf
  • Terhi Nurmikko-Fuller, David M Weigl, Kevin R Page, 2015. On organising multimedia performance corpora for musicological study using Linked Data. Proceedings of the 2nd International Workshop on Digital Libraries for Musicology, Knoxville, TN, USA, June 24, 2015. Available at http://dl.acm.org/citation.cfm?id=2785532