fMRI-Compatible Musical Interfaces
Return to the projects main page
Summary
This long-term, 8-year, collaborative project between CIRMMT and BRAMS researchers focused on developing musical interfaces for use within MRI (magnetic resonance imaging) scanners. Among the interfaces created in this project, a piano-like interface in two versions and a cello-like interface allow neuroscientists studying motor learning of musical tasks to perform functional scans of a subject's brain.
Objectives
Develop safe and reliable functional musical interfaces to study the brain activity of music performers while performing inside an fMRI scanner.
Timeline
fMRI-Compatible Piano Keyboard: 2005 - 2007
fMRI-Compatible Cello: 2008 - 2014
Status: completed
Outcomes & Impacts
Two functional versions of the piano and one cello have been built, and several full-fledged MRI studies have been completed.
The fMRI-compatible piano-like interfaces have limited ranges (up to 2 octaves) due to physical limitations of the fMRI scanner. Data captured by the interfaces, e.g., note onset, offset, and velocity of the piano keys, is synchronized with the scanner, auditory and visual stimuli, and auditory feedback.
The optically sensed fMRI-compatible electro-acoustic cello, the first ever developed, enables neuroscientists studying motor learning in musical tasks to perform functional brain scans while capturing detailed information on the cellist’s instrumental gestures, such as fingering, bow speed, and bow pressure.
All interfaces have been thoroughly tested on 3T fMRI scanners, with no image artifacts observed.
People Involved
- Avrum Hollinger, McGill University (Music Technology)*
Left: Melanie Segado playing cello interface. Right: Subject inside MRI scanner with piano interface. - Melanie Segado, McGill University (Neuroscience)*
- Marcelo M. Wanderley, McGill University*
- Virginia Penhune, Concordia University*
- Christopher J. Steele, Concordia University
- Robert Zatorre, McGill & Montreal Neurological Institute*
*CIRMMT regular, collaborator and student members
Granting Agencies / Funding Sponsors
- Canadian Foundation for Innovation (CFI)
- CIRMMT Student Award 2007-2008
- NSERC Discovery Grant, M. Wanderley
- Concordia University
Resources
Video Recording
- Playing musical instruments in the MRI - the brain on music, a 5-minute documentary produced by McGill University
In the Press
- Outside the Music Box, McGill Reporter, Andrew Mullins, 2010-01-28
- Invention of musical instruments that can be played in a Magnetic Resonance Imaging (MRI) scanner, McGill Bicentennial, 2011
- Playing a cello (inside an MRI scanner), Improbable Research, Martin Gardiner, 2013-10-11
- Hacked Instruments Help Measure Musicians' Brain Activity, Wired, Danielle Venton, 2015-06-28
Selected Publications
Technical Developments:
- Avrum Hollinger, Virginia Penhune, Robert Zatorre, Christopher Steele, Marcelo M. Wanderley. "fMRI-Compatible Electronic Controllers." In Proceedings of the 2007 International Conference on New Interfaces for Musical Expression (NIME07), pp. 246-249.
- Hollinger, A. (2008). "Design of fMRI-compatible electronic musical interfaces." M.A. thesis, Music Technology, McGill University, Montreal
- A. Hollinger and M. M. Wanderley. “Optoelectronic Acquisition and Control Board for Musical Applications.” In Proc. of the 2012 International Conference on New Interfaces for Musical Expression (NIME12), 2012.
- A. Hollinger and M. M. Wanderley. “MRI-Compatible Optically-Sensed Cello.” In Proceedings of IEEE SENSORS Conference, 2013.
- A. Hollinger. Optical Sensing, Embedded Systems, and Musical Interfaces for Functional Neuroimaging. Ph.D. Thesis, McGill University, 2014.
- A. Hollinger and M. M. Wanderley. “The Design, Implementation, and Testing of a Sensorized MRI-Compatible Cello.” IEEE Sensors Journal, 15(11):6125-6134, 2015.
Experiments:
- R. Brown, J. Chen, A. Hollinger, V. Penhune, C. Palmer, and R. Zatorre, “Neurological and behavioral basis for auditory-motor transformations in music performance,” in Proceedings of the 11th International Conference on Music Perception and Cognition, August 2010.
- M. Segado, A. Hollinger, J. Thibodeau, V. Penhune, and R. Zatorre. “Partially Overlapping Brain Networks for Singing and Cello Playing.“ Front. Neurosci. 12:351Sec. Auditory Cognitive Neuroscience, 27 May 2018.
- I. Wollman, V. Penhune, M. Segado, T. Carpentier, and R.J. Zatorre. “Neural network retuning and neural predictors of learning success associated with cello training.“ Proc. Natl. Acad. Sci. U.S.A. 115(26):E6056-E6064, 2018.
Keywords
Research-creation, Motor learning, Optical sensing, Music neuroscience, MRI-safe instrument design, Gesture and performance