Arduino Circuits and Javanese Puppets: ‘Re-materializing’ Digital Archives through Tangible Interfaces

The creation and analysis of digital collections is a key aspect of the digital humanities. Deciding what to archive, how to organize the documents, and how to present them to the public are never easy questions. But in the case of theatre and performance archives, these questions are especially complex since theatre performances are necessarily transient and they depend on the embodied co-presence of actors and spectators. In order to document the embodied and ephemeral nature of theatre performances, digital archives often try to include a wide array of documents (video recordings, photographs, motion capture data, 3d models of theatrical spaces and technical scripts). The makers of digital theatre archives are at pains to contextualize their materials and account for aspects that cannot be easily documented and transmitted. Can interaction design help communicate these aspects? Is it possible to imagine and construct interfaces that can communicate cultural context, transcending the limitations of a computer screen, mouse and keyboard?

To investigate these questions within the context of an Indonesian performance archive, I developed a Tangible User Interface (TUI) using open source hardware (Arduino microcontrollers and an array of sensors) and culturally-coded objects that are important for the performances in the archive. This interface was developed as an interactive artwork for educational museums and as a thought experiment on how tangible, culturally-specific interfaces can constitute instances of academic research outputs in the digital humanities.

The archive in question is the Contemporary Wayang Archive (CWA, http://cwa-web.org), a collection of digital recordings and metadata pertaining to Javanese wayang kulit (shadow puppetry), which I have been developing with my collaborators at the National University of Singapore since 2012. Wayang kulit is the oldest theatre form of Indonesia and one of the most important theatre traditions of Southeast Asia. It consists of a form of puppetry where a dalang (narrator-puppeteer) singlehandedly moves all the puppets, speaks all the character parts, jokes with the audience and directs the musicians. He is at the same time a puppeteer, a storyteller, an orchestra conductor and a stand-up comedian. In order to communicate with the audience and the orchestra, he uses different objects in order to control the progression of the story. For this project, I wired different sensors into these objects in order to develop a TUI that recreates the spatial setup of a conventional wayang performance and three key material components of this setup:

Figure 1. Conventional wayang kulit setup
Figure 1. Conventional wayang kulit setup
Figure 2. IMU sensor attached to the kayon (left) and piezoelectric sensor attached to the cempala (right)
Figure 2. IMU sensor attached to the kayon (left) and piezoelectric sensor attached to the cempala (right)
Figure 3. A user interacting with the TUI
Figure 3. A user interacting with the TUI

Bibliography
  1. Fishkin, K. P. (2004). A taxonomy for and analysis of tangible interfaces. Personal Ubiquitous Computing, 8(5): 347–58.
  2. Bonanni, L. et al. (2010). Tangible interfaces for art restoration. International Journal of Creative Interfaces and Computer Graphics, 1(1): 54-66.
  3. Jones, S. et al. (2009). Redefining the Performing Arts Archive. Archival Science, 9(3): 165-71.