The human brain has the remarkable ability to process speech and language cues that often occur in just fractions of a second. While our eye-tracking techniques can often measure very early processes, they leave open the neural mechanisms that may underlie speech perception, and they can really only measure representations that are relatively late in speech processing like words.

However, with advanced neurophysiological recording techniques, we can overcome some of these problems to measure responses to these cues in the auditory cortex as unfold they in real-time. This allows us to ask questions about the encoding of established acoustic cues for speech like voice onset time for voicing and second formant transitions for place of articulation, and how this low-level encoding changes as a function of context factors like who the talker is, what the neighboring sounds are, and what the speaking rate. In a sense, when do we quite literally “hear” this cue differently as a function of context, or do we just interpret it differently?

We are also interested in exploring lexical activation and context effects, and electrophysiology allows us to measure the stages of these processes with fine temporal resolution. Understanding the cognitive neuroscience of speech and language can help us establish neurobiologically plausible psychological models, and provide critical insight into theoretical issues that are not clearly amenable to behavioral investigation.

Some of the topics we study are:

• When and how are speech features and cues encoded in the auditory cortex?
• Do neural responses pattern with categorical behavioral responses?
• At what level does context information affect speech perception?
• What is the role of lexical feedback in the perception of speech sounds?

Perhaps the classic neuroscientific technique for answering these questions it scalp-recorded electroencephalography (EEG) and the closely related event-related-potentials (ERP) technique. Toby Mordkoff in the Department of Psychology, we've been using these techniques. However, thanks to a collaboration the Human Brain Research Laboratory in the Department of Neurosurgery, we can take advantage of much more cutting edge approaches. The HRBL works with patients who are being treated for epilepsy and other brain disorders and have electrodes implanted directly on and in their brains This intracranial electrocorticography (ECoG) technique allows us to directly measure brain responses in a highly localized way for awake behaving individuals listening to speech.

Recent Publications

Toscano, J., McMurray, B., Dennhardt, J., & Luck, S. (2010) Continuous perception and graded categorization: Electrophysiological evidence for a linear relationship between the acoustic signal and perceptual encoding of speech. Psychological Science, 21(10), 1532-1540.