To view the list of presenters: CIRMMT Student Symposium 2018 - List of Presenters
General Assembly and Student Symposium 2018 Overview
10:05am-12:10pm: ORAL PRESENTATIONS / PRÉSENTATIONS ORALES
10:05 - 10:30: Louis Commère, Visual data sonification to assist blind people
Visual to Auditory Sensory substitution devices are designed to help blind people to understand visual information with the use of the auditory modality. We first developed at NECOTIS lab (Sherbrooke) a sensory substitution system which sonifies in real time 2D images taken from an iPhone camera. We then test our system to investigate the ability of blindfolded participants to perform basic visual tasks like locomotion and object detection after a short time of training. Considering our promising results we are currently developing a more elaborate system by using depth information which is very important for understanding the environment and interacting with. In this context, we propose a low-level sonification of 3D point cloud data taking into account the psychoacoustic limit of auditory perception. With this new system, we will test if blind people can better perceive the environment with tasks like shape recognition and distance evaluation.
10:30 - 10:55: David Rafferty, Structured cell: A work in space
Spatialization as a compositional parameter in multi-speaker environments opens many new opportunities in music research while simultaneously highlighting several important questions. How can space be used as an effective compositional tool? On a technical level, in which manner will the spatialization be implemented in a space? How would one capture an acoustic space in a recording that would emulate an environment accurately? How can we reimagine a performance space to enhance the experience of contemporary music? These were the questions driving the composition ‘structured cell’. The piece was written for 10 performers, a 22.2 speaker dome surrounding the musicians performed with Maxmsp and live electronics, in collaboration with sound engineer and 3D audio specialist Will Howie captured in a 22.2 3D private audio recording session.
10:55 - 11:20: André Martins de Oliveira, Katelyn Richardson, The KKPM database: Revolutionizing musical instrument training through crowdsourced data
The Kinetic-Kinematic-Physiological Musicians Database combines movement and muscle function data in search for patterns in instrumental performance. Related publications are limited and often lack an extensive sample size or a replicable methodology, inhibiting both the extraction of clear conclusions and the ability to compare results. This void highlights the pressing need for systematic, musician-movement research in order to fully elucidate, evaluate, and improve playing techniques. The KKPM Database begins satisfying this void by proposing a platform to share measurements. Results of the project include selecting a host of the multi-modal data, establishing standardized collection methods, and uploading an initial set of data from violinists and flautists. Future steps involve the online release of the database for expansion through worldwide collaboration. Such a resource will support research addressing the “unknowns” surrounding function and ideal use of the body during music production.
11:20 - 11:45: Etienne Richan, Towards translating perceptual distance from hearing to vision using neural networks
This research project proposes a system mapping sounds to images in a manner that respects perceptual distances in both modalities. In other words, the degree of perceived difference between two sounds should be reflected in the dissimilarity of the images generated from those sounds. In our experiments, timbral features are used to measure perceived auditory distance and an embedding learned by neural networks for computer vision provides a measure for visual distance. A convolutional neural network for style transfer is used for the synthesis of a wide variety of textures and patterns. Different methods of controlling this network are explored and a system for mapping timbral features to textures is presented. This architecture allows users to choose their own pairings of images and sounds, which can then be extrapolated to generate visual representations for a larger set of sounds.
11:45 - 12:10: Etienne Thoret, Perceptually inspired representations for the analysis of musical timbre
There is a growing interest in mathematical representations, known as Modulation Power Spectrum and inspired by the auditory system physiology, to analyze and synthesize musical signals. Here, I’ll first present results of perceptual experiments in which we highlighted the most relevant parts of these representations for musical instrument timbre identification. By using a methodology inspired from vision studies, known as the bubbles method, we showed that low temporal and spectral modulations are the most relevant parts of these representations. Finally, I’ll present a meta-analysis of former studies on musical timbre perception based on these new relevant perceptually inspired representations. We showed in particular that these perceptually inspired representations are better suited to account for these perceptual results. Overall, all these studies demonstrate the relevance of these new representations for the study of the acoustical correlates of musical timbre perception.
12:10-2:00pm: LUNCH & POSTER AND AUDIO DEMOS / DÎNER & AFFICHES ET DEMONSTRATIONS D'AUDIO
Johnty Wang, Informing the development of a mapping interface for alternative control of pipe organs
The aim of project was to support the conception, implementation and use of a system that interfaces with MIDI enabled pipe organs in churches. In essence, the sound producing system of the organ becomes the "synthesizer" component of a digital musical instrument where the input and mapping system can be arbitrarily chosen to correspond to any performance gesture. A number of experimentations were performed using the system including control with new interfaces, the playback of pre-sequenced arrangements, as well as remote actuation of the instrument demonstrating a tele-performance. Another key development was the configuration of a virtual organ synthesis system so that artists can sketch out and design compositions in a simulated environment without constant physical access to the church space.The poster and video provides a summary of the various explorations performed throughout the project, and also presents future work surrounding the system.
Matthew Boerum, Jack Kelly, Diego Quiroz, The effect of virtual environments on localization during a 3D audio production task
In a perceptual study of three-dimensional audio for virtual reality (VR), an experiment was conducted to investigate the influence of virtual environments on localization during a production-based panning task. It was hypothesized that performing this task in VR would take longer and be less accurate than if conducted in a real environment. Using a 3D loudspeaker array and hardware panning controls, 80 participants were asked to match probe sounds to the location of randomly positioned target sounds. Participants performed the task blind or aware of loudspeaker position. Half were presented the task in a VR replica of the environment using a head mounted display. Results showed no significant difference in accuracy for virtual vs. real conditions, but VR decreased duration.
Pierre Grandjean, Philippe-Aubert Gauthier, Alain Berry, Spherical double-layer microphone array for 3D recordings and directional metrics measurements: Design and fabrication
Spatial audio in multimedia, digital arts as well as industrial applications, is in a boom. After some decades of stereophonic illusion ubiquitous, many 3D spatial audio alternative solutions have appeared, such as Wave Field Synthesis, High Order Ambisonic, or Directional Audio Coding. One problem on these systems is the sound mixing step that is more challenging. A solution is to design a microphone able to record 3D sound field. Moreover, it can be used to evaluate reproduction deviation between these systems, 3D recording system is necessary. Therefore, this project deals with prototyping an ambisonic spherical microphone, which is able to records sound pressure as well as to evaluate directional metrics. A 50-node grid, according to Lebedev theory, enable microphone array to record at ambisonic order 5. In order to compensate frequency limitations, a combination of two concentric 50-node microphone arrays, rigid and empty, is investigated. Finally, parameter-selection methods are presented.
Ian Marci, Handedness in percussion performance
Percussionists spend their entire careers striving for parity of sound between their hands, trying to overcome their natural laterality. Tenets of percussion pedagogy prescribe that the most effect method of creating an even sound is to match the movements of each hand. While experiments have demonstrated percussionists’ extreme precision in timing experiments, little work has been done to explore their success in the attempt for symmetry in movement. This work analyzes the gestures of percussionist using motion capture technology to identify asymmetries in motion caused by handedness. The impulsive nature of percussion gestures limits the interaction between player and instrument. However, by subtly adjusting the mallet trajectory using their wrists and fingers, percussionists are able to create a wide range of musical articulations. Both hands use the same strategies for each articulation, but asymmetries caused by handedness are evident.
Moe Touizrar, The sunrise metaphor in orchestral music: An analysis of transmodal perception from creation to reception
Composers from Haydn to Schoenberg purported to depict sunrises using the orchestra. But, what does it mean to hear a sunrise? And, how do listeners understand this visual-to-auditory metaphor? My project uses three methods of analysis: a corpus study, qualitative interviews, and empirical perceptual testing. First, I create a corpus of examples of orchestral sunrises drawn mainly from the 19th and early 20th century literature (N=75). Second, three practice-based factors contribute to feature identification and weighting within the corpus: my own listening, group listening sessions with subject matter experts, and semi-structured interviews conducted with internationally recognized orchestral composers and conductors (living in Scandinavia and North America). In-depth interviews will be structured using the Verbal-Probing technique combined with the subject’s descriptive report on directed listening to orchestral sunrise examples. Additionally, interviewees will be asked to describe, in their own terms, how they might orchestrate a sunrise – i.e. what would the most important features be, and in what order? Using an integrated approach to text analysis, I mix inductive ‘grounded’ coding with a small ‘start list framework’ made up of previously identified features of the corpus. Quantitative Concept Analysis will then be performed on the entire collection of interviews to determine relational concept-clustering across participants. This second stage will result in a list of weighted features and stimuli for subsequent empirical testing, where the features and examples will be selected based on the analysis and grouping of similar descriptive terms, and their correlates in the music. The testing phase seeks to understand the relative contribution of both perceptual features such as loudness and spectral centroid, as well as orchestral devices such as diachronic changes to instrumentation and layering. Subjects will rate sound stimuli for perceived fit with visual stimuli that vary in brightness and intensity, simulating the dynamic temporal evolution of a sunrise. Test results will allow analytical descriptions of sunrises using weighted classifications of features and orchestral devices. Features and devices can then be discussed in terms of effective strategies for future artistic representations of sunrises.
2:00-3:00pm: CIRMMT STUDENT AWARD LIGHTNING TALKS
2:00 - 2:10: Yaolong Ju, Generating multi-style harmonic analysis using artificial intelligence for homorhythmic music
In this project, we propose an innovative workflow to generate high-quality harmonic analysis with multiple analytical styles using artificial intelligence, starting with two largely homorhythmic music sets: chorales by Praetorius (1571 – 1621) and Bach. The project contains four steps. First, we will define two distinct harmonic analysis rubrics -- maximally melodic and maximally harmonic -- with a definitive set of rules. Second, instead of requiring annotators to follow the rules and label the whole dataset from scratch, we will develop a rule-based algorithm to generate preliminary harmonic analyses. Although the algorithm cannot deal with sophisticated passages as well as annotators, it can generate 100% consistent analyses in each style. Third, annotators will modify a subset of the analyses. To make the modification consistent, multiple annotators will work on the same passage, and modify analyses based on consensus. Last, these highly consistent and accurate analyses will serve as training data for machine learning models to generate analyses automatically. In this way, not only are we able to minimize the work of the annotators with artificial intelligence techniques, and create consistent, accurate harmonic analysis with multiple interpretive styles, but also we will have machine learning models to generate harmonic analyses automatically for the unannotated, homorhythmic music. Furthermore, the resulting homorhythmic corpus will be made freely available online, where all the musical metadata (e.g., keys, notes, and harmonies) is searchable, serving as an invaluable resource for musical information retrieval (MIR), music theoretic, and musicological research.
2:10 - 2:20: Tanor Bonin, Kseniya Degtyareva, Ephraim Hahn, Investigating semantic and technical coordination among composers, performers and sound engineers
Successful music production depends on the effective communication and execution of a shared artistic vision among the composer, performer, and sound engineer. The translation of complex artistic concepts in language and the coordination of technical actions among these creative disciplines present challenges to this interdisciplinary collaboration. How does one formulate semantic analogies for musical ideas? How does a creator translate these analogies into technical actions to shape the musical sound in the direction of the desired artistic outcome? We will investigate these questions with an iterative production-analysis procedure, in which basic musical motifs will first be generated, then classified with acoustic and semantic modelling, then developed in the creative process by composers, performers and sound engineers, and finally reanalyzed to allow for semantic and acoustic comparison of initial musical concepts and final musical products. Our results will provide empirical insight into the creative process of music production.
2:20 - 2:30: Laura Jacyna, Kit Soden, Thesaurus of orchestration
Our goal is to build a prototype thesaurus of orchestration terminology related to instrumentation and perception. This digital humanities project will draw from and add to research being done as part of the international Orchestration and Perception Project that is based at McGill. The thesaurus of orchestration will be built from terminology extracted from orchestration treatises and textbooks, as well as audience perception research. These terms will then be used to construct a model database, using the thesaurus management software MultiTes. A faceted classification system will be used so that terms may be assigned more than one hierarchical term and can be found through different searches. We plan to build a relational database of the thesaurus so that the terms can be accessed, browsed through, and searched easily, for continued use as a research tool and subsequent online orchestration resource.
2:30 - 2:40: Noam Bierstone, Takuto Fukuda, Mylène Gioffredo, Reimagining Kagel’s Exotica in a contemporary technological and cultural context
Commissioned for the exhibition Weltkulturen und moderne Kunst, held in parallel with the 1972 Olympics in Munich, Exotica is a work of instrumental theatre for six classically trained performers playing several "exotic" instruments of their own choice. The piece features various degrees of indeterminacy both at the micro-level (timbre, pitch and dynamics) and the formal level: the score contains five sections with various subsections that can be reordered, each of them featuring different musical textures and several types of interactions between performers. Two of the sections involve ethnomusicological recordings chosen by the performers which serve as models that the performers imitate in specified ways. Through various musical situations the piece questions the meaning of “exoticism” and the definition of the Other. However, since Kagel’s approach to intercultural exchanges and exoticism was developed when globalization and its cultural impact were still at an early stage, we propose, forty-seven years later, to develop a performance of Exotica relevant to the contemporary multicultural society of Montreal.
Such performance will be the result of investigations led on the compositional processes, research made possible through the study of the sketches held by the Paul Sacher Stiftung, the exploration of various possible performances and their political meanings emanating from the various possible interpretative choices. Going beyond the updated performance of Kagel’s work, we will also propose a new piece, thought as an answer to Kagel’s musical proposition, developing the main compositional principles of Exotica through the use of new interactive technologies.
2:40 - 2:50: John Sullivan, Building performance practice around new instruments: A longitudinal study of the Noisebox
This project will study how the use of participatory design methods can positively contribute to development of novel digital musical instruments (DMIs) for use in long-term, engaged performance practice. The project has two main objectives. First, working with a core group of performers, we will actively support the participants' creative explorations of an existing DMI called the Noisebox by iterating on its design based on their input and feedback. Second, by documenting the performers' development over an extended period of time, we hope to elucidate how new performance techniques and expertise emerge and are codified into practice on a novel instrument. User-driven research methods will include group workshops and rehearsals, interviews and documentation of individual practice sessions, and the evaluation of new instrument prototypes through video and motion capture analysis of performance. Our goals for the project are to develop an improved, mature family of Noiseboxes that have been successfully integrated into an ongoing performance practice, culminating in the generation of original works and live performances featuring the instrument. Furthermore, we aim to identify an effective set of guidelines and research methods towards the development of new instruments intended for long-term use in performance.
3:20-4:20: KEYNOTE ADDRESS / CONFÉRENCE INVITÉE
Eric Heller: Journey into psychoacoustics
Psychoacoustics has become accessible. Nearly everyone with a laptop computer and headphones can become an informed participant or indeed an investigator. Research level experiments that required a few hundred thousand dollars for a laboratory 25 or 30 years ago can now be done almost for free. This talk will focus on examples from pitch perception, waveform perception, and source localization, illustrated with classic and also very new findings. To the extent possible, live audio examples will be provided. Recent results obtained by students in the Harvard undergraduate course, Why You Hear What You Hear (SPU13) will also be discussed.
