Speech Production in Challenging Listening Conditions

Return to the Projects main page

Summary

This project investigates how noise, ear occlusion, and hearing impairment shape speech production by altering self-auditory feedback. By developing and analyzing the Hearing-Integrated Bilingual Speech Corpus (HIBiSCus), the project provides new insights into speech motor control and supports innovations in hearables, audiology, and speech communication research.

Objectives

• Characterize how noise, ear occlusion, and hearing thresholds—individually and together—affect speech production.
• Develop the HIBiSCus corpus, a bilingual (English–French) speech database collected across different noise and ear occlusion conditions for various speech-rated tasks.
• Examine individual variability in the effect of noise, ear occlusion, and hearing threshold on speech level.
• Advance understanding of how hearing impairment influences speech level and its adaptation in different noise and ear occlusion conditions.
• Provide high-quality multimicrophone speech recordings for cross-disciplinary research on communication, hearing health, and hearables.

Timeline

Approach

The methodology integrates experiments, corpus development, and statistical analysis:

• Speech was recorded in controlled acoustic conditions within an audiometric booth using multiple microphones (reference, in-ear, outer-ear).
• Listening conditions systematically varied across three noise levels (silent, 70, 85 dBA) and four ear occlusion states (open, simulated open, low occlusion, high occlusion).
• Participants performed three tasks: reading sentences, sustained vowel production, and picture description.
• Audiometric pure-tone hearing thresholds were measured to allow continuous modelling of hearing impairment.
• Linear mixed-effects modelling was used to evaluate how noise, occlusion, and hearing thresholds influence speech level and adaptation patterns.

Outcomes & Impact

• Creation of HIBiSCus, a French-English speech corpus examining noise, occlusion, and hearing thresholds together.
• New evidence on how hearing thresholds modulates the effect of noise and ear occlusion on speech level control.
• Insights supporting improved design of hearables, hearing aids, and communication technologies.
• Contributions to theories of speech motor control and auditory-feedback mechanisms.
• Open-access database enabling cross-disciplinary collaborations involving acoustics, audiology, speech science, and engineering.

People Involved

• Xinyi Zhang — Lead Researcher / PhD Student*
• Dr. Rachel Bouserhal — Supervisor (ÉTS, CIRMMT)*
• Dr. Ingrid Verduyckt — Co-investigator (Université de Montréal)

*CIRMMT Regular and student members

Partners

• The Research in Hearing Health and Assistive Devices (RHAD) Lab
• Centre for Interdisciplinary Research in Music, Media and Technology (CIRMMT)
• École de technologie supérieure (ÉTS)
• Université de Montréal — École d’orthophonie et d’audiologie
  

Granting Agencies / Funding Sponsors

• Natural Sciences and Engineering Research Council of Canada (NSERC - Discovery grant)
• Fonds de recherche du Québec's Master's training scholarship
• CIRMMT student award 2024-25
• ETS Marcelle Gauvreau Engineering Research Chair in Multimodal Health Monitoring and Early Disease Detection with Hearables

Resources

• Published paper: "Hearing-integrated bilingual speech corpus: a French-English corpus including hearables for studying speech production under challenging listening conditions". https://doi.org/10.1121/10.0042355 
• Hearing-Integrated Bilingual Speech Corpus (HIBiSCus)

Keywords

Scientific/technological research, Speech production, Noise & ear occlusion, Hearing impairment, Auditory feedback

Learn More

• RHAD Lab website
• Google Scholar
  • Xinyi Zhang
  • Rachel Bouserhal
• ORCiD
  • Xinyi Zhang
  • Rachel Bouserhal
• LinkedIn Xinyi Zhang