Marking up opera

One of Transforming Musicology's core musicological work packages addresses the leitmotive technique of Richard Wagner. In this work we are applying a mixed disciplines approach comprising: humanistic literature study of the ways the leitmotives have been identified and discussed since the operas' first performances up to the present; computational audio similarity experiments to find known leitmotives and potential leitmotive candidates; and pyschological listening tests of participants with a range of Wagner experience to understand how they hear the operas and the leitmotives.

Opera, of course, is a large and multi-modal phenomenon. For the audio search experiments it's going to be important to be able to understand search results within such a mass of material quickly and easily. To aid this, we have been working on developing an opera navigation tool. Of the various modes that make up opera, we're going to be concentrating on the libretto, the score, and audio recordings of performances for our search system. What we'd ideally like to have is time-aligned encodings of each of these for an opera. As it's relatively easy to acquire and to work with, we've decided to start with the libretto as a kind of 'spine' for this time alignment and we've produced a TEI encoding of the libretto of Das Rheingold.

We've developed a customisation of TEI which adapts the existing drama module for opera libretti, adding a few extra elements:

<sg>: Indicates that the enclosed text is sung. This element is very similar in use to the core <sp> element.
<lyric>: Contains a line of sung text, quite similar to the core <l> element.
<singer>: Contains the name of a cast member who sings in the containing <sg>. Again, this is very similar to the <speaker> element.
<voice>: Indicates the voice (e.g. soprano, alto, tenor, bass) of a singing cast member. This element is included in the <castItem> element that describes the singer.
<musicalPrelude>: Indicates music at the start of an opera which does not include sung text. The prelude may also include stage directions.
<musicalInterlude>: Indicates music which does not include sung text occurring at any point during an opera.

These, along with elements from the drama module such as <castList>, <castItem>, and <stage> and core structural and metadata elements have proved sufficient for marking up the libretto text we're using. Here's a small excerpt from the first scene:

<sg xml:id="sg1">
    <singer who="WOGLINDE">WOGLINDE</singer>
    <lyric n="1">Weia! Waga! Woge, du Welle,</lyric>
    <lyric n="2">walle zur Wiege! Wagalaweia!</lyric>
    <lyric n="3">Wallala, weiala weia!</lyric>
</sg>
<sg xml:id="sg2">
    <singer who="WELLGUNDE">WELLGUNDE</singer>
    <stage>Stimme von oben</stage>
    <lyric n="1">Woglinde, wachst du allein?</lyric>
</sg>

Our key aim here is alignment of the libretto (as spine) with media of other modes such as score and performance. To align with performance we have manually inserted <anchor type="audio-cue"> elements which bear an xml:id attribute of the form: cue-{conductor}{year}-CD{n}-{track}, e.g. cue-Neuhold1993-CD1-1:

<sg xml:id="sg9">
    <anchor type="audio-cue" xml:id="cue-Haitink1988-CD1-3" />
    <anchor type="audio-cue" xml:id="cue-Neuhold1993-CD1-3" />
    <singer who="ALBERICH">ALBERICH</singer>
    <lyric n="1">Hehe! Ihr Nicker!</lyric>
    <lyric n="2">Wie seid ihr niedlich, neidliches Volk!</lyric>
    <lyric n="3">Aus Nibelheims Nacht naht' ich mich gern,</lyric>
    <lyric n="4">neigtet ihr euch zu mir!</lyric>
</sg>

In the <back> of the document we then automatically generate (using a simple Perl script) a <linkGrp> of <link>s each of which relates an <anchor> to a track in the MusicBrainz database. This gives us our alignment of libretto to performances:

<linkGrp type="audio-cues" domains="#das-rheingold #recording-Neuhold1993 http://musicbrainz.org/release/468d27f6-b1c2-4c4e-be5e-591703800a60">
    <link target="#cue-Neuhold1993-CD1-1 #when-Neuhold1993-CD1-1 #2846b8f9-c9d6-3dea-82dd-57e20479c456"/>
    <link target="#cue-Neuhold1993-CD1-2 #when-Neuhold1993-CD1-2 #d7ec4c5b-8406-38c5-b6c9-3fb991149d26"/>
    <link target="#cue-Neuhold1993-CD1-3 #when-Neuhold1993-CD1-3 #4e65b26d-1d24-3475-9cac-3305ddec2454"/>

The target is made up of three parts: the xml:id of the cue; the xml:id of a time offset for the track (described below); and the MusicBrainz ID of the track. These correspond to the domains attribute value of the <linkGrp> in which the last element is the full URL to the MusicBrainz release from which the tracks are taken. In order for the Perl script to find the tracks it's necessary to supply it with the MusicBrainz release ID of the recording; this has to be searched for manually from the MusicBrainz database.

As MusicBrainz includes the track lengths, we also automatically generate from these a <timeline> containing a series of <when> elements with IDs matchable with the <anchor>s and capturing the lengths of each track as an interval in seconds:

<timeline xml:id="recording-Neuhold1993" unit="s">
    <when xml:id="when-Neuhold1993-CD1-1" absolute="unknown"/>
    <when xml:id="when-Neuhold1993-CD1-2" interval="260" since="#when-Neuhold1993-CD1-1"/>
    <when xml:id="when-Neuhold1993-CD1-3" interval="94" since="#when-Neuhold1993-CD1-2"/>

In a future post I'll describe how we publish this marked up libretto on the Web as a next step towards our goal of an opera navigation tool.

And we'll eventually be looking at how to align with scores. The real show-stopper here, of course, is that we don't have any encoded scores of Wanger's operas so sourcing those will be a significant challenge.