cognition, brain and consciousness: an introduction to cognitive neuroscience

Cognition, Brain and Consciousness: An Introduction to Cognitive NeuroscienceEdited by Bernard J. Baars and Nicole M. Gage2007 Academic Press

Chapter 7 Hearing and Speech“it was very fortunate that, even in Helmholtz’ time, the great anatomical discoveries by Corti (and others) had already made it clear that the vibrating tissue most important for hearing is the basilar membrane of the inner ear, because the cells on which the nerve endings terminate are seated on this membrane … the problem of how we hear was reduced largely to a mechanical question: how does the basilar membrane vibrate when the eardrum is exposed to a sinusoidal sound pressure?”

Bekesy, Nobel Prize Lecture (online)


Chapter Outline

1.0 Introduction

2.0 The central auditory system

3.0 Functional mapping of auditory processing

4.0 Speech Perception

5.0 Music Perception

6.0 Learning and plasticity

7.0 Auditory awareness and imagery

8.0 Summary


1.0 Introduction

A model for sound processing

Sound inputs enter the system and there is a very brief storage or buffer called echoic memory.

There may be many competing sounds occurring at the same time -- such as the sounds in a noisy lecture hall


1.0 Introduction


Selective attention allows the system to direct resources to a subset of the sensory inputs that are competing for attention -- such as attending to a lecturer and ignoring the sounds of someone whispering behind you


1.0 Introduction


Complex interactions occur between the new inputs and existing memories and experiences, as well with other sensory systems.


1.0 Introduction


The ultimate goal or ‘action’ to be performed is important as well, and will affect how information is encoded and stored.

These processes are not one-way: there are interactions that occur through the encoding of sounds, both within the auditory system and across other sensory, cognitive, memory, and motor systems.


1.0 Introduction

It’s about time …

Time is a critical aspect of auditory processing: the auditory system differs from the visual system in that all sound processing occurs over time.

A spectrogram provides a ‘picture’ of speech, with time on the x-axis and frequency on the y-axis. Speech contains harmonics -- formants - at specific regions on the spectrogram. Darker bands indicate more energy at that region in the speech signal.

A spectrogram showing the energy at different frequencies in the speech signal


1.0 Introduction


Three time scales are critical for decoding speech -- 20, 200, 2000 milliseconds (ms):

Formants in consonants such as ‘b’ and ‘d’ change on a time scale of about 20/1000 of a second, or 20 ms

A syllable lasts about 200 ms

A typical sentence lasts around 2000 ms



1.0 Introduction


The auditory system must be able to decode information at each of these timescales in order to accurately perceive speech.



1.0 IntroductionSound and hearing basics

Two ways to represent the physical features of a sound pictorially:

In the time-domain: time is on the x-axis and pressure (or amplitude) is on the y-axisIn the spectral-domain: time is on the x-axis and frequency (in Hz) is on the y-axis

time-domain

spectral-domain


1.0 Introduction

Sound and hearing basics

Physical features of sounds: Frequency, intensity, and time

The frequency of a sound is the rate of sound wave vibration and is calculated in terms of cycles completed per second, or hertz (Hz). A sinusoid with 1000 cycles per second has a frequency of 1000 Hz


1.0 Introduction


The intensity of a sound reflects the amount of amplitude (or displacement) within its cycle and over time.

Intensity is reported in units of decibels (dB). Hearing thresholds for humans and cats are shown, along with the dB level for various types of sounds. Sounds over 120 dB present a high risk for damage to the hearing system.


1.0 Introduction


The peripheral hearing system: external, middle and inner ear

The cochlea is located in the inner ear. Within the cochlea lies the basilar membrane


1.0 IntroductionA cut-away view of the external, middle, and inner ear showing the transmission of sound (red arrows). Note the basilar membrane curving within the cochlea


1.0 Introduction

Innervation patterns of afferent (ascending pathway) and efferent (descending pathway) neurons in the Organ of Corti


2.0 The Central Auditory System

Auditory pathways

The auditory system is complex, with many stages and pathways on the way to auditory cortex.The two main flow of information are in the ascending and descending pathways.

The ascending (afferent) pathways transmit information about sounds from the periphery to cortex. This is not a simple delivery system, but entails many stages where information is encoded and recoded



Auditory pathways

The descending (efferent) pathways extend from regions in the cortical and subcortical auditory system to the periphery. These pathways are under direct or indirect cortical control.

An important function of the descending pathways is to provide ‘top-down’ information that aids in selective attention processes.



Auditory cortex

Auditory cortex is the region within the cortex that is specialized for sound processing. It is located within the Sylvian fissure on the surface of the supratemporal plan and the upper banks of the superior temporal gyrus,so it is difficult to see from a lateral view of the brain.

Brodmann areas for auditory and receptive language processing include Brodmann 22, 41, 42, and 52.



Auditory cortex Much of what we know about auditory cortex comes from studies in non-human primates. Here is a map of auditory regions in the left hemisphere of the macaque.

The core, belt, and parabelt regions that form auditory cortex are tucked inside the Sylvian fissure and shown in an opened cutaway.



Auditory cortex

Human brain, showing auditory cortical regions within the Sylvian fissure in the left hemisphere.


2.0 The Central Auditory SystemAn axial slice of the brain showing a schematic of regions within auditory cortex. Note the hemispheric asymmetries for each region.



Auditory cortex

Top: MRI coronal slice showing the location of the supratemporal plane

Bottom: MRI axial slice showing the transverse gyrus of Heschl and the planum temporale



Receptive field properties of neurons in auditory cortex

Here are receptive fields of two auditory cortical neurons plotted as a function of sound pressure level and azimuth (a ‘map’ of auditory space with respect to the two ears).

Notice that the top map shows a cell that is broadly tuned across much of the azimuth on the contralateral side, while the bottom map shows a cell with a much narrower tuning.


3.0 Functional Mapping of Auditory Processing

Auditory stream analysis

A core function of the auditory system is to localize sound. Another central function is to recognize auditory ‘objects’ such as a friend’s voice or the sound of a car braking. An important role for the auditory system in humans is the perception of speech.

These differing functions have led to the suggestion that there are multiple -- but highly interactive -- auditory processing ‘streams’ that underlie these functions and that interface with other sensory modalities.



Heschl’s gyrus and the Planum temporale

Heschl’s gyrus and the planum temporale have been the focus of many fMRI studies in order to understand their functional role in sound processing and speech perception.

Heschl’s gyrus is the site for primary auditory cortex (A1). Far less is known about the functional role of the planum temporale. Recent studies have suggested that it serves as a nexus or hub for auditory processing streams.



‘What’ and ‘where’ processing streams

‘What’ and ‘where’ processing streams have been suggested based on studies in non-human primate. Similar to the pathways or streams evidenced in the visual system, the ‘what’ pathway is engaged in object recognition while the ‘where’ pathway is engaged in sound localization



‘What’ and ‘where’ processing streams

Human studies have provided evidence that auditory streams may not simply form two pathways: this topic remains an open one, but recent studies have provided evidence for a ‘how’ stream in addition to the ‘what’ and ‘where’ streams. Other studies have provided evidence for a ‘who’ stream for voice recognition.



Speech perception

The decoding of human speech is a central and important function of the human auditory system. Here is a spectrogram showing time on the x-axis, frequency on the y-axis, with darkened shadings showing increased energy at that time and frequency.

Speech perceptual systems must be able to decode information in short time windows (20 ms) as well as longer ones (2000 ms).



Distinctive features

Early work using the spectrogram to study the sound-based (acoustic phonetics) aspects of speech identified distinctive features of consonants and vowels.

Here is a schematic illustration of the formant patterns for types of consonants in American English



The Lack of Invariance Problem

Speech scientists were searching for the basic ‘building blocks’ or primitives that enabled us to decode speech. The distinctive features in speech sounds seemed like a promising candidate for speech primitives -- however the lack of invariant formant patterns for consonants, differing depending on the following vowel, made it obvious that encoding distinctive feature information could not be the only way speech decoded.



A process model

Other models for understanding how speech perception was accomplished by the brain took into account other processes -- such as semantic decoding -- that were closely association with speech perception



Speech perception in the brain

The advent of functional neuroimaging techniques allow us to investigate brain regions involved both in speech perception and production



19th and 20th Century models for speech and language

The groundbreaking work of 19th century scientists such as Broca and Wernicke identified regions that appeared critical for speech perception (temporal lobe, ‘Wernicke’s area’) and production (frontal lobe, ‘Broca’s area’).

Functional imaging studies have confirmed these general findings, although new discoveries are still occurring to clarify the roles of these brain regions in speech.



Beyond Broca …

A recent model extends the earlier views of receptive and productive language regions beyond Broca’s and Wernicke’s areas. This is an active area of study and new models are constantly being advanced.



Music perception

The perception of music is complex and likely involves many interactive stages



Music perception

Music and language are unique to human beings

Recent studies have investigated if same or similar brain areas are involved in encoding music and language.

Music perception in the brain is a relatively new field of study -- like speech, there are no simple ‘building blocks’ or primitives.


6.0 Learning and Plasticity

Sensory memory formation

We are continually learning to recognize new auditory ‘objects’ in our environment: a new friend’s voice, a new cellphone ringtone … how, and where, does the auditory system accomplish this?

Sensory memory formation: a large body of evidence indicates that sensory memories are both formed and stored in auditory cortex.


7.0 Auditory Awareness and Imagery

The hearing system is the last to fall asleep and the first to awaken.

Why is it that the best way to wake someone up is to call their name? Is there a ‘wake-up’ area in the brain?

A neuroimaging study investigated that question by playing simple sounds (beeps) and one’s own name in two conditions: while the subject was awake and while the subject was asleep



A brain wake-up call?

Results: areas in the middle temporal and frontal lobes were more active for hearing one’s name vs. beeps in the sleep condition.

It might be that these regions monitor sound during sleep and activate other brain regions upon hearing one’s name.



Are the same brain areas active when you imagine a sound vs. hearing it?



An fMRI study investigated that question: primary auditory cortex (A1) was active during the perception phase but not during the imagery phase of the study.


8.0 Summary

•While much work has been accomplished to understand how the brain decodes complex sound patterns such as those in human speech and music, much more work needs to be done to understand how and where this happens in the brain

•There are likely multiple auditory streams that interact in encoding auditory sensory inputs, that are also highly interactive with other sensory, memory, and action systems

•The advent of neuroimaging techniques have made it possible to investigate aspects of auditory function that were previously difficult or impossible to study, such as auditory imagery and auditory awareness during sleep

cognition, brain and consciousness: an introduction to cognitive neuroscience

Documents

cognitive neuroscienceedited

time time

introductiona model

auditory awareness

helmholtz time

sound processingsound

visual system

sensory inputs