Music Information Retrieval Algorithms for Oral History Collections

Digital humanities, as a largely text based domain, often treats audio files as texts, retrieving semantic information in order to categorise, sort, and discover audio. This workshop will treat audio as audio. Taking oral history collections from the University of Sussex Archive of Resistance Testimony as a test case, participants will be lead through the use of Music Information Retrieval (MIR) approaches to categorise, sort, and support their discovery of an audio collection. Participants will also be supported in planning the extension of these approaches to explore audio collections that they know or work with.

All instructors for this workshop are from the new inter-disciplinary Sussex Humanities Lab at the University of Sussex. The workshop combines the diverse interests of the instructors to focus on the application of music technology to history and stems from this inherently collaborative environment.

Oral history best practice publications and resources often focus on the application and use of digital methods and tools to create, store and manage audio, audio-visual, and subsequent text files. They recommend, for example, standards for file formats, metadata and text encoding, software for audio to text conversion, and database and content management systems. However, while a number of projects provide innovative and useful tools that challenge the privilege of the text (i.e. Oral History Metadata Synchroniser), the majority of projects rely on the ability to encode an oral history interview to carry out further analysis using digital tools and methods. The analysis, therefore, is based on the text surrogate rather than the original audio source, but as Alessandro Portellii states this focus denies the ‘orality of the oral source’.

Text encodings or transcripts of oral history interviews have their obvious advantages - they are easier to anonymise, distribute and store, and we have established techniques for text analysis. However, there are indications within the community that the privilege of this text based approach should be questioned given the ever increasing possibility for computational analysis of audio. This is evident in the Oral History Society’s recent call for papers which remarks that the ‘auditory dimension of oral history was for decades notoriously underused’.

In light of this loss of context and information this workshop will explore the original sources for the richer datasets which they afford. The field of MIR provides this opportunity. MIR draws from digital audio signal processing, pattern recognition, psychology of perception, software system design, and machine learning to develop algorithms that enable computers to ‘listen’ to and abstract high-level musical information from low-level audio data. Just as human listeners can recognize pitch, tempo, chords, genre, song structure etc, MIR algorithms are capable of recognizing and extracting this information, enabling systems to perform extensive sorting, searching, music recommendation, metadata generation, transcription on large data sets. Deployed initially in musicology research and more recently for automatic recommender systems, the research potential for MIR tools in non-musical audio data mining is being recognised (e.g. analysing bio-acoustic data for ecological purposes) but yet to be fully explored in the humanities.

The aim of this workshop is to help the digital humanities community explore the possibilities of MIR in a practical and methodological fashion within context of oral history collections. Our participatory design approach enables development for a broader scope of problems and support participants to challenge current methodologies for oral history analysis. We seek to apply an objective analysis based on the content of the source material in its entirety and complement or challenge human and text based analysis with computational methods. By applying MIR algorithms to a field that traditionally privileges text we hope to add new understandings and interpretations to rich audio resources.