Addressing the Music Information Needs of Musicologists

As a research project progresses it's quite common to get the point where you feel ready to explain rather than just report your findings. This is the point where you're ready to give a tutorial. The ISMIR (International Society for Music Information Retrieval) conference has run a tutorials track for several years and many of the topics in MIR which have been the subject of research eventually solidify into practical techniques which can be presented in this tutorial format. ISMIR tutorials are three hours long and often presented by just a small number of people.

In Transforming Musicology, following our very successful workshop at the DH@Ox Summer School, and our research into digital musicology techniques--not only over the course of the project, but in preceding years--we felt that we had reached the point where we were able to offer something to the ISMIR community on the subject of musicology. So we proposed a tutorial for ISMIR 2015 with the title, "Addressing the Music Information Needs of Musicologists." This titles makes use of the idea of "information needs" from information science which is essentially the idea that users of information retrieval systems come to such systems with an information need that they wish to satisfy. And so what we proposed to share was our experience of what musicologists might be able to gain from music information retrieval systems.

David Weigl and Kevin Page presenting.

The tutorial was held on Monday 26 October and attracted an audience of about 40 delegates of the ISMIR conference. We took the approach of presenting several different topics each with examples of work we've done in the area. The idea was that our audience of MIR researchers would be well-versed in the MIR techniques that they use, but may not have considered how those techniques could be applied to musicological problems. So each of the topics was centred around MIR techniques. The tutorial started with an introduction from Alan Marsden who spoke about nature of musical research and potential synergies between MIR and musicology.

Metadata: (Richard Lewis and David Lewis) Here we made the argument that much of the work that goes on in historical musicology is actually work done with materials that the MIR community would consider "metadata", i.e. that, for historical musicology, metadata is data. We showed some examples from the A Big Data History of Music project in which data on music publishing was collected and analysed in order to show trends in the first two centuries of music publishing. We argued that rich metadata and sophisticated metadata search is just as important for musicology as the sophisticated content-based search that is more common in the MIR community.
Networks: (Ben Fields) The practice of network analysis has become quite central to the work we're doing in Transforming Musicology and so we included a section in which we described analysing an online community using these techniques. A lot of the work that goes on in historical musicology can be framed as network questions, although unlike the metadata ideas, very few musicologists would currently identify network analytics as an information need.
Performance/real-time: (Kevin Page and David Weigl) Again, this is a more niche interest in musicology, but we've been working on techniques in the project to enable a kind of empirical reception studies. This has included a suite of technologies that allow an analyst to annotate a score in real-time during a performance, especially focusing on details of the performance that are not captured in audio recordings. There's a very tantalising information need here in being able to open up access to detailed, multi-modal data surrounding a musical performances.
Symbolic music data: (Richard Lewis) Although scores would seem to be a staple of musicological research, we took the opportunity in our tutorial to emphasise how abstract musical notation is not often the source material for musicology. More often, scholars are interested in notation more as documentary evidence; it's in the details of where musical documents have been, who had access to them, what alterations they have been subject to, and what versions of pieces they contain that the real musicological interest lies. And so we argued for the need for rich digital score encodings that allow these kinds of details to be captured, highlighting especially the affordances of the Music Encoding Initiative.
Audio search: (Christophe Rhodes, Ben Fields, and Tim Crawford) The ideas of large-scale search in audio collections and of audio similarity are things which don't feature much in contemporary musicology. We highlighted how some of the document-level work that's quite common in MIR (e.g. things like classifying the genre or mood of a whole track) might not suit musicological information needs so well. Whereas passage-level retrieval could become a useful tool. We gave the example of identifying occurrences of leitmotives in recordings of Wagner's Ring Cycle. But we especially highlighted the deficiencies of existing techniques for doing (e.g. that chroma features don't work well for leitmotives that are distinctive only in their melodic content, but have static harmony).

We closed the tutorial with a structured group discussion. We asked all the participants to form into pairs and spend five minutes discussing the MIR techniques that they already use and, following our presentations of how techniques could be applied to musicological problems, think about what kinds of musicological tasks their own techniques could be applied to. Next, we asked the pairs to join into groups of four or six and repeat the discussion. Finally, each of the groups reported back to the whole group.

The ideas that came up from these groups included:

The problem that a lot of the existing tooling for MIR research is very tied up with the research question it was designed to answer. Consequently, as a musicologist with a research question, it's hard to find any tools that could be adopted. The group suggested a move towards tools made out of simple "building blocks" that could be more easily selected and combined by non-specialists.
The problem of segmenting musical data in meaningful and musicologically credible ways. For MIR applications that are not just document-level, a segmentation of musical documents is always necessary. But how can this be done without manual intervention?
The challenges of working with non-Western musics and how suitable existing MIR techniques are for things such as non-tonal tuning systems and unfamiliar timbres.
The possibility of variation and similarity being central issues affecting the applicability of MIR techniques to musicological research. Many of the common tasks in MIR such as score following and cover song detection are based on assumptions about the nature of variation and similarity.
The kinds of visualisations that could be made for large music collections and how they might be useful for music research.
The extent to which computational pattern discovery in music aligns with psychological notions of musical salience. Are we actually able to come up with notions of similarity that make sense to listeners?

Tim Crawford concluded the group discussion pointing out that a lot of the areas of interest seem to be concerned with putting the humans back in the loop of MIR. He noted the importance of systems which reflect people's actual understandings of and reactions to music and also highlighted the continuing challenge of building user interfaces to MIR systems that fit musicological needs.

Following the tutorial, several of the Transforming Musicology team attended the rest of the ISMIR conference, presenting a few posters and late-breaking demos. We also formed a group discussing musicology and MIR in the Unconference on the closing day. One particular suggestion that came out of this was that we encourage the programme committee for ISMIR 2016 to include some papers which are about musical findings, rather than just about MIR techniques and developments. Hopefully T-Mus will be able to push this idea and even come up with relevant papers.