| Contents | Cortical oscillations play a key role in segmenting, parsing, and encoding continuous speech, as brain rhythms align with speech rhythms - a process known as speech entrainment. Recent evidence indicates that such entrainment supports speech understanding. Using magnetoencephalography (MEG), we have shown that visual speech (lip movements) similarly entrains slow neural rhythms, prompting new questions about how auditory and visual information are represented in the brain - jointly or independently. Which systems convey shared information from multisensory inputs, and which support synergistic representations? And can we causally modulate speech comprehension using non-invasive brain stimulation technique?
In this talk, I will first demonstrate how slow cortical rhythms (delta, theta) track speech dynamics and relate to comprehension. Second, I will show how entrained auditory and visual speech interact to facilitate understanding, leveraging a novel information-theoretic framework - Partial Information Decomposition (PID) - to quantify redundancy, synergy, and unique information over time. Third, I will present how the brain encodes high-level semantic gist (topic keywords) using an NLP-based topic-model approach, and I will highlight ongoing MEG/OPM-MEG work on multi-modal speech development. Finally, I will introduce rapid invisible frequency tagging (RIFT) as a non-invasive technique to enhance speech processing in combination with MEG, and I will briefly discuss ongoing efforts to improve the decoding of imagined speech.
Together, these studies link multiscale neural dynamics to perceptual and semantic aspects of speech, reveal how auditory and visual channels jointly shape comprehension, and suggest principled avenues for improving communication in challenging listening conditions and clinical populations. |
