Comprehension of Prosodic Cues in Typical & Atypical Populations

Dr. Rachel Mitchell

Dept. of Psychology, Durham Univ.

∂

What is prosody?

Higher order language function:Paralinguistic or pragmatic phenomenon that accompanies words

& can modify or influence meaning

(Suprasegmental) features of speech such as intonation, loudness, & tempo

Conveyed by gradient features e.g. pitch height & range, & rhythmicity

“That faculty of speech which conveys different shades of meaning by means of variations in stress + pitch – irrespective

of the words + grammatical construction” (Monrad-Kohn, 1947).

∂

What’s its role in language?

Intrinsic prosody: dialectical and idiosyncratic differences, question vs. declarative statement

Intellectual prosody: speaker attitude about info conveyed (enthusiasm, boredom etc)

Emotional prosody: emotion conveyed (happiness, fear etc)

Inarticulate prosody: nonlinguistic sounds (sighs, groans etc)

∂

Why is it important?

Crucial to understanding true communicative intent (societal integration)

How something said as imp. as what’s said (sarcasm etc)

Deficits impair ability to interpret others’ emotional states & behave in socially appropriate manner (Trauner et al., 1996)

Ability to interpret auditory emotion cuesplays important role in maintaining successful relationships & healthy psychological fx (Carton et al., 1999).

∂

What happens in the brain?

Mitchell et al. (2003) Neuropsychologia, 41(10), 1410-1421.

Variable lesion studies: R hem damage not always disruptive, sometimes L hem damage, multiple skills across both hem’s

Solely associated with early association cortex unlikelyStimuli: H & S sentences, H, S & N intonation

1. Pure: MTG (R>L), L MFG

2. Cong v rest: R STG, R MFG, L SFG, CG

3. Cong v neutral: R MTG, R IFG, LSTG

4. Incong v rest: MTG (R>L)

5. Attend semantics: R STG, L SPL, L MFG, L ITG, L SPL

6. Attend prosody: R MTG, R MedFG

∂

How do we process mixed signals?Mitchell R (2006) Cognitive, Affective & Behavioral Neurosci, 6(4), 298-305.

Modulation of speech content by prosody.Why wouldn’t conflicting cues have similar effects to Stroop tasks?Congruence effects? YesRelative importance of semantics & prosody. No effect

Mitchell R (2006) Eur J Neurosci, 24(12), 3611-18

Activation of conflict regions? - dlPFC, ACC- See next slideModulation of ‘prosodic emotion’ regions? - Attention influences

activity in sensory cortical regions assoc with specific modality. - R lat T lobe ROI parameter estimates suggest lower activity in

interference contrast than in facilitation contrast

∂

Facilitation Contrast  Interference Contrast

IFG (vlPFC) L 47 -50 29 -6 3.92 IFG L 44 -45 18 13 5.03

STG L 22 -62 -40 18 4.32 IFG (vlPFC) L 45 -50 32 1 3.63

STG

R 22 65 -20 4 4.11 IFG (vlPFC) L 47 -45 26 -14 3.42

MTG L 21 -59 0 -3 4.72 STG R 38 39 13 -33 4.05

MTG L 21 -53 -18 -7 4.86

MTG R 21 56 -4 -15 3.75

ParaHG L 36 -33 -10 -20 4.24

BG L -18 -15 -2 3.89

∂

Why do neuroimaging and lesion studies differ?Ross & Mesulam (1979)– 2 R-handed patients with frontoparietal ischemic strokes – Neither aphasic but displayed flattened affective demeanour– Ability to identify emotion portrayed in normal speech unaffected– series papers confirmed association, while damage involving posterior T

region associated with inability to comprehend affective prosody– Organisation prosodic functions in R hem mirrors propositional language

functions in L?

Unexpected F lobe activations in early neuroimaging studies.

Schirmer & Kotz (2006) 3 stage model® Can TMS resolve this discrepancy? (LUCY ALBA-FERRARA)

∂

What happens when we age?

Mitchell R (2007) Cogn Emot, 21(7), 1435 – 54

Ageing affects facial emotion recognitionBrosgole & Weisman (1995) asymptote age 12, maintain until mid

age (mid 40s), then ↓ age 65+ - resembles 3-to-6 yr oldsTheoretical doubts: lifetime of practiceMediators and mechanisms:– Frontal Lobe Load (N-back task)– Hearing sensitivity (Pure Tone Average)– IQ (NART) – only covariate– Depression (Beck Depression Iventory)– Verbal Working Memory (Daneman & Carpenter)– Aged 61 to 83 chronological age covariate

∂

How specific is this?

Are prosodic emotion decoding deficits caused by inability to generally decode prosodic cues?

Compare emotional prosody with non-emotional prosody (e.g. changes in stress that distinguish bet declarative & question)

Which constituents of emotional prosody do older adults have particular difficulty processing?

Are prosodic emotion decoding deficits caused by inaccurate processing of elemental constituents?

∂

What are the Consequences?

Little explicit investigation in normal populationsAim = determine which aspects social well-being decline in tandem

with age-related decline

Study A. – Which aspects social well-being associated with norm variations in ability to

decode prosodic emotion? – Which aspects decline in tandem with age-related decline?

Study B. – Does depressed mood in older adults leads to prosodic emotion comprehension

dysfunction or v.v.? – Depressed mood may be means by which poor comprehension exerts effects

on social function. – Low social function often correlated with depressed mood in older adults

∂

What happens in schizophrenia?

∂

Experiment 1Filtered emotional prosody vs. rest

Healthy controlsL: Z=4.25, 188 voxels

R: Z=4.42, 362 voxels

SchizophreniaL: Z=3.76, 455 voxels

R: Z=3.17, 10 voxels

Z=3.95, 329 voxels

Bipolar DisorderL: Z=5.20, 1090 voxels

R: Z=3.47, 27 voxels

Z=5.03, 665 voxels

∂

Experiment 2Unfiltered emotional prosody vs. rest

Healthy controlsL: Z=4.52, 819 voxels

R: Z=5.39, 599 voxels

SchizophreniaL: Z=3.46, 293 voxels

Z=3.88, 51 voxels

R: Z=3.64, 61 voxels

Z=3.54, 153 voxels

Bipolar DisorderL: Z=4.43, 792 voxels

R: Z=4.13, 569 voxels

∂

Experiment 3

Attention to semantics:– HC activated large network L hemisphere regions, S did not– BP didn’t activate any regions in within-group analyses but not

abnormal in between-group analyses

Attention to emotional prosody:– S activated large network R hemisphere regions, HC did not– BP didn’t activate any regions in within-group analyses but not

abnormal in between-group analyses

∂

What happens in Parkinson’s disease?Patients with PD may find it more difficult to ‘read’ emotion signals

transmitted via tone of voice.

PD neuropathology suggests altered basal ganglia function may explain this, but it could be explained by altered frontal lobe function instead.

Patients with PD performed 4 tasks:

1. A standard task of working (‘online’) memory

2. An emotion version of task 1

3. A standard attention task

4. An emotion version of task 3

Patients not worse on emotion tasks than older adults, even when tasks demanding only when frontal lobe pathology above a threshold?

But, PD patients didn’t perform standard tasks as well more prominent?

∂

Laterality and intensity issues

Lateralisation

Motor symptoms usually worse on one side of body.

Tone of voice findings inconsistent – a subgroup only?

Word content mediated by L half brain, emotional tone of voice by R half.

Would expect different patterns behaviour depending whether pathology in L or R half brain.

e.g. visuo-spatial problems in patients whose L side body affected

Compare comprehension word content with comprehension tone of voice.

Patients studied initially in on and off state medication 

∂

Intensity

1. Sensitivity to emotional tone of voice in sentences

2. Sensitivity to non-emotional tone of voice in sentences

3. Sensitivity to emotional prosody in words

4. Sensitivity to facial emotions

5. Sensitivity to varying degrees of facial emotion.

We will adjust the data for changes in response speed.

Explore Medication level, Chronicity of disease, Severity motor symptoms

∂

New directions

?