Florian Grond & Adriana Olmos: Audible sculptures: A CIRMMT Strategic Innovation Fund report

ABSTRACT

Urban citizens of Montreal enjoy a great wealth of artistic creations in public space; amongst them we find a great collection of sculptures. These creations visually enrich our daily lives, for example, while walking through an urban park or in a museum. While the visually impaired community engages strongly with their audible environment, artworks such as sculptures generally remain inaccessible. Motivated by observations during our field study with the blind community within the In Situ Audio Services (ISAS) project, we are intrigued by the possibility of rendering for them an acoustic description of physical art forms as an analogue to the experience that sighted individuals receive as they walk down the street. As a concrete example, we wanted to convey through auditory signals the physical features of one of the many sculptures up in the Mont-Royal mountain, so that a blind individual gains a mental representation of its form, and ideally, an appreciation of its aesthetic.

Inspired by early work on sonification of shapes and textures (Meijer, 1992;  Seung-Yeo and Berger J., 2006; Jung-Kyong and Zatorre, 2010), and psychomechanics research of simulated sound sources related to the intuition of object sizes (McAdams S, Chaigne A, Roussarie V., 2004), we proposed to address the question above. In order to develop a sonic representation of the sculptures that is useful and interesting for the blind, we make use of an interaction paradigm that is closely related to echolocation. This mode of gathering information about the surroundings, is familiar to many visually impaired individuals, and is based on sending out short impulses and listening to the reflections of nearby objects. In brief the audible sculptures listen for your sounds like clapping hands, snapping fingers or tongue clicking and render back these short impulses convolved with a sound-buffer. This convolution kernel corresponds to the actual auditory representation of the 3D sculpture and is inspired by the model-based sonification method known as a data-sonogram (T. Hermann and H. Ritter, 1999). 

Our methodology for the selection of the sound synthesis and mapping strategy followed an ethnographic and participatory design approach. To this end, we have developed a prototype that allows a visually impaired individual to listen to the sonic responses of the short impulse sounds. Our visually impaired team member has been evaluating this sonic experience through the exploration of the sculpture by touch and together we have been exploring what could help to create an audible mental representation of the sculpture. Future steps would involve evaluating the sound synthesis approach in a more qualitative manner (following a similar approach as used by Jung-Kyong and Zatorre, 2010). 

In this talk we will be presenting our prototype and our findings during the participatory design phases.  This is a CIRMMT strategic innovation funded project submitted by: Jeremy R. Cooperstock, Gary P. Scavone, Jean Piché, Zack Settel, Adriana Olmos. 

 

ABOUT FLORIAN GROND

Florian studied at the DeMontfort University in Leicester (UK) and the Eberhard Karls University in Tübingen (Germany) and holds an MSc in Chemistry (2002) from the Karl Franzens University in Graz (Austria).

After his academic studies he worked at the ZKM (Center for Art an Media) (2003 - 2007) in Karlsruhe (Germany) as a research assistant and guest artist, where he developed his interdisciplinary artistic practice working with new media participatin in exhibitions in Europe, North America and Asia.

He was research trainee at the shared reality lab in the Center for Intelligent machines (CIM) (2010) and guest researcher at the Input Devices and Music Interaction Laboratory (IDMIL) (2008).  He is currently a PhD candidate in the Ambient Intelligence research group at the Cognitive Interaction Technology Center of Excellence (CITEC) Bielefeld University. 

 

ABOUT ADRIANA OLMOS

Adriana received her Bachelors degree in Electronics Engineering from the Universidad Iberoamericana Leon, Mexico and her Ph.D. degree from the Computing and Electrical Engineering Department at Heriot-Watt University, Edinburgh, Scotland. She developed visual perceptual models and designed experiments while working as a Postdoctoral Research Fellow at the McGill Vision Research Laboratory and the Biomotion Lab at Queen's University. 

At the Shared Reality Lab she collaborated in the design and development of the Health Services Virtual Organization project, Open Orchestra and the Services Audio In Situ. In the past, she collaborated in the conception and design of various projects including the Canadian Centre for Architecture (CCA) online, 33MAG, and the Historical Website of the Montreal Canadiens while working at Bluesponge.  She organizes the Iterati lecture series, bringing in speakers at the intersection of ethnography, empathy and iterative design.