Workshop on gesture as language - Abstracts

  • 09:30-10:00 - Étienne Thoret: Hearing gestures and drawing sounds: Auditory perception of biological movements

Understanding the links between the acoustical properties of sounds produced by our actions and the properties of the action imagined from them is a crucial challenge to understand the fundamental interplay between audition and action. Here I'll present results of works on the acoustic characterization of specific biological movements: the drawing movements. If we listen carefully to the friction sounds produced by someone drawing, a movement can be perceived. How do we interpret this movement perceived from a friction sound? Is it possible to recognize whether or not it is a human movement? Can the drawn shapes be deduced from the dynamics of such sounds? And conversely, can we create sounds to influence and guide the dynamic and the geometry of someone drawing? By addressing these issues we showed the crucial role of our ears may have to control our movements..

 

  • 10:00-10:30 – Ian Marci: Handedness in percussion performance

Percussionists spend their entire careers striving for parity of sound between their hands, trying to overcome their natural laterality. Tenets of percussion pedagogy prescribe that the most effect method of creating an even sound is to match the movements of each hand. While experiments have demonstrated percussionists' extreme precision in timing experiments, little work has been done to explore their success in the attempt for symmetry in movement. This work analyzes the gestures of percussionist using motion capture technology to identify asymmetries in motion caused by handedness. The impulsive nature of percussion gestures limits the interaction between player and instrument. However, by subtly adjusting the mallet trajectory using their wrists and fingers, percussionists are able to create a wide range of musical articulations. Both hands use the same strategies for each articulation, but asymmetries caused by handedness are evident.

 

  • 10:30-11:00 - Fabrice Marandola: Encoding/decoding music and space through language: percussion as a case study

I will present preliminary results from several ongoing studies related to percussion performance. Different examples will illustrate how performers use different strategies to transform the musical code (the score) into a personal code that helps them to control their gestures during a musical performance.

 

  • 11:15-11:45 - Jullian Hoff & Charlotte Layec: Interact with the transfigured tube

With the clarinetist Charlotte Layec, we will present the tools and strategies deployed to create an audiovisual work which consists of a co-improvisation between a musician and an artificial agent. The system was developed for the production of a work that was presented in public in Canada, the United States and which is part of the programming of the next festival RE: FLUX in New Brunswick. The artificial agent consists of three Max programs with distinct roles:

-Acquisition of the musical gesture via audio descriptors and subsequent mapping strategy to control the generative audiovisual agent.

-Musical algorithms to accompany the musician or taking the role of soloist

-Generative visual synthesizer reacting to the musician.

The proposed presentation consists of a brief explanation of the tools developed followed by a real-time demonstration.

 

  • 11:45-12:15 - Diego Quiroz: Gestural control for 3D audio

Digital musical instruments (DMIs) have been evolving over the past several decades, and much research has been done on the subject of capturing the gestures of performers in an effort to re-map them to digital instruments. The practice of audio mixing is no different from a musical performance in this respect. Ever since three-dimensional sound entered the audio industry scene, there has been a need for research in the area of gestural control for 3D audio processes like mixing, panning, spatialization and post-production automation. 3D sound systems emerging lately, like Dolby Atmos (for movie theaters), Auro3D and 22.2 NHK, and binaural audio formats for virtual-3D headphone listening, are very suitable to be benefited from the development of input devices and software applications for 3D audio. When using traditional faders and pots, three-dimensional panning poses a problem of using separable controllers for what is an integral action, such as reaching a point in space in all three dimensions simultaneously in one motion. Traditional stereo and surround-driven techniques and tools are becoming obsolete for new 3D audio systems, and new gestural controllers that work in three dimensions need to be researched, developed and applied so the quality of 3D sound recording and post-production can advance.

  

  • 13:15-14:30 - Caroline Traube, Felipe Verdugo & Justine Pelletier: Towards an overarching research approach for the study of instrumental gestures in piano performance

Piano performance involves an indivisible complex of psychomotor, mechanical, psychoacoustical, linguistic and cognitive phenomena. Scientific research on pianist’s gestures faces therefore a major challenge since the study of each of these phenomena needs to be developed in an overarching perspective integrating knowledge from several and heterogeneous disciplines. For instance, analysing the role of the numerous body segments involved in piano performance calls for an understanding not only of the biomechanical principles underlying pianist’s gestures but also of the pianist's musical intention, the structural and mechanical parameters of the instrument’s action as well as the psychoacoustical and perceptual features of the resulting sound. In this talk, we will first present a multidisciplinary approach inspired from the paradigm of phonetics for the study of the gesture-to-sound relationship. We will then discuss the general outline of an ongoing research project built on a constructive juxtaposition of current literature on the biomechanics of piano performance and basic principles of an evidence-based approach to piano technique developed and taught at Université de Montréal. We will also present some of our experimental studies aiming to acquire a better understanding of concepts such as the use of « weight » to control piano timbre.