Connor Adams
Pronouns: he/him
Research Mentor(s): David Brang
Research Mentor School/College/Department: Psychology / LSA
Program:
Authors: Emma Zhang, Connor Adams, Iain DeWitt, Yike (Echo) Li
Session: Session 1: 9:00 am – 9:50 am
Poster: 1
Abstract
Effective spoken communication requires listeners to accurately decode auditory speech content. Multisensory processing can enhance auditory speech intelligibility. This research examines the influence of visual information on the neural mechanisms of auditory speech perception. Our main hypothesis is that visual information facilitates speech processing by increasing neuronal response selectivity in the auditory cortex. To study neural responses to multimodal speech stimuli, electrocorticography (ECoG) was used to record high-gamma local-field potentials in intractable epilepsy patients while they watched naturalistic audiovisual movies derived from the Human Connectome Project (HCP; Cutting et.al, 2014). To assess multisensory phoneme-related neuronal responses to naturalistic stimuli, HCP movies were annotated for phoneme events and the visibility of speaker and non-speaker faces. All electrodes were assessed for speech responsiveness, upon which only responsive contacts were included in selectivity analyses. To quantify response selectivity at single contacts for audiovisual and audio-only stimuli, we calculated unimodal and multimodal phoneme selectivity index vectors (PSI; Mesgarani et al., 2014). The entropy of each PSI vector was used to assess overall neuronal response selectivity at each contact.
