Download - Charles Clifton, Jr. grateful acknowledgments to Mohamed Mohamed Matt Traxler Rihana Williams
On using eyetracking data to evaluate theories of on-line sentence processing:
The case of reduced relative clausesCharles Clifton, Jr.grateful acknowledgments to
Mohamed Mohamed
Matt Traxler
Rihana Williams
Keith Rayner
Robin Morris
Sungryong Koh
Lyn Frazier
Eye movements and sentence comprehension
• Existing measures not a transparent window into cognitive processes– Lexical processing: some good ideas about
mapping– Comprehension: another story– Consider one case of garden-path sentences…
Garden Paths in Reduced Relative Clause Sentences
(the curse of Tom Bever)
• The horse raced past the barn fell.
• But sometimes reduced relative clauses are easy to comprehend.
McKoon & Ratcliff, 2003
• The window broken by John couldn’t be repaired
• The window examined by John needed repairing
• Both cases: (easy) reduced relative construction denotes an entity that participates in an externally caused event
Trueswell, Tanenhaus, & Garnsey, 1994
• The defendant examined by the lawyer proved to be unreliable– Produces clear reading disruption in disambiguating
region compared to full relative clause (The defendant who was…)
• The evidence examined by the lawyer proved to be unreliable– Initial NP inanimate, and a good theme but a poor agent– Claimed to eliminate difficulty of reduced relative clause
• TTG 1994 actually a followup to Ferreira & Clifton, 1986
Animacy
Animate Inanimate
Fir
st P
ass
Tim
e, M
s/ch
ar
0
20
30
40
50
Ferreira & Clifton 1987, Disambiguating Region
ReducedRel
FullRel
Animate: The defendant examined by the lawyer…
Inanimate: The evidence examined by the lawyer…
Criticism: Trueswell, Tanenhaus, & Garnsey, 1994
• Some of the Ferreira & Clifton inanimate sentences weren’t all that implausible as main clause sentences– The trash smelled…. (pretty bad)– The car towed…(the trailer)
• TTG, better materials, first noun normed for plausibility as agent and as patient of first verb
Subject Noun
Animate Inanimate
Fir
st P
ass
Tim
e, m
s
0
400
500
600
Trueswell, Tanenhaus, & GarnseyDisambiguating Region
RedRel
FullRel
Why the differences?
• Maybe TTG were right – the F&C materials were bad
• But maybe something else: parafoveal preview of “by the…”– Evidence from Burgess, Spivey, McRae, etc, in
self-paced reading– Possible display limitations in F&C (42
character display, line break typically before PP would have prevented effective preview)
Clifton, Traxler, Mohamed, Williams, Morris, Rayner, in press
• Redo Ferreira & Clifton using Trueswell et al. materials (thanks to John et al.)
• Add boundary change manipulation to permit or deny parafoveal preview of “by the noun”
• Perform additional analyses
Clifton et al. details
• Used materials from Trueswell, Tanenhaus, & Garnsey
• 8 conditions– Animate (“difficult”) vs inanimate (“easy”) initial noun
– Reduced vs full (control) relative clause
– Parafoveal preview vs. no preview of “by the” (reader saw random 6-letter string instead of “by the”)
• Measured eye movements while reading for comprehension (24 Ss)
Fir
st P
ass
Tim
e, m
s
0
400
500
600
700
800Disambiguating Region, FP
Animate Inanimate Animate Inanimate Preview No Preview
RedRel
FullRel
Fir
st P
ass
Tim
e, m
s
0
400
500
600
700
800Disambiguating Region, FP
Animate Inanimate Animate Inanimate Preview No Preview
RedRel
FullRel
Overall: 66 ms ambiguity effect for animate nouns, 34 ms ambiguity effect for inanimate nouns (sig. smaller, but > 0)
Reg
ress
ion
Pat
h D
ura
tion
, ms
0
400
500
600
700
800Disambiguating Region, RPD
Animate Inanimate Animate Inanimate Preview No Preview
RedRel
FullRel
Overall: 68 ms ambiguity effect for animate nouns, 83 ms ambiguity effect for inanimate nouns (n.s. different)
Interim Conclusion and Question
• Looking only at first pass times gave an incomplete picture of sentence comprehension– RPD measure indicated garden-pathing even
with inanimate initial noun– Trueswell et al. experiment not an adequate
basis to reject modular serial parsing model• But do the increased RPDs reflect the usual
behavior of the eyes?
Pro
por
tion
Fir
st P
ass
Reg
ress
ion
s O
ut
0.0
5.0
10.0
15.0
20.0
25.0Disambiguating Region, Reg Out
Animate Inanimate Animate Inanimate Preview No Preview
RedRel
FullRel
More regressions for No Preview than Preview, and for Animate than Inanimate, but apparent effect of Reduction and interactions not significant.
Pro
por
tion
Fir
st P
ass
Reg
ress
ion
s O
ut
0.0
5.0
10.0
15.0
20.0
25.0Disambiguating Region, Reg Out
Animate Inanimate Animate Inanimate Preview No Preview
RedRel
FullRel
Reg
ress
ion
Pat
h D
ura
tion
, ms.
, Ou
tsid
e R
egio
n
0
20
40
60
80
100
120Disambiguating Region, RPD Outside
Animate Inanimate Animate Inanimate Preview No Preview
RedRel
FullRel
Sec
ond
Pas
s T
ime,
ms.
, fro
m L
eft
0
20
40
60
80
100
120Disambiguating Region, Second Pass from Left
Animate Inanimate Animate Inanimate Preview No Preview
RedRel
FullRel A similar pattern for: regression path duration (outside the region) second pass on target (coming from left)
Combined data: First pass regressions
• Combine data with parallel experiment conducted at U South Carolina– Always preview of “by the” phrase– Vary high vs low span readers
• Total n = 68, more stable data
Condition
Anim InanPro
por
tion
Fir
st P
ass
Reg
ress
ion
s
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0Combined Data, Regressions Out
RedRel
FullRel
Significant effects of animacy, ambiguity, no interaction
Summary
• Eye movement measures show comparable ambiguity effects for sentences with animate and with inanimate subjects– It is premature to conclude that inanimacy
blocks garden-pathing– Other data are needed to evaluate models that
give logical priority to syntactic structure
A problem for understanding eye movements
• Regression path durations are inflated by processing difficulty– This inflation can only come from trials where
there were regressions
• But regressions were very infrequent, < 10% of the trials
A similar effect in other experiments
• Reduced relative clauses with various classes of verbs (manner of motion, theta-grid changing, unaccusatives)
• No garden-pathing in first pass times, big garden-pathing in RPD, but < 30% regressions
Conclusions
• Sentence processing difficulty is only variably mirrored in the ET record– Doesn’t always show up in FPT
• One clear response to difficulty: a regression– But only happens 5-30% of time
• A common response to difficulty: fuggettabottit– Just get the words, `mam
I’d like to be shown wrong, but…
• ET response to comprehension difficulty is variable: linger, regress, nothing
• Effects seen in means may actually occur on only a minority of trials
• Eye movements may be controlled by word recognition, and only exceptionally affected by higher-level factors
Put on those rose-colored glasses
• Maybe all these problems will disappear if people read texts they really care about, not just single sentences
• But I wouldn’t bet on it
Stevenson & Merlo, 1997
• The troops marched across the fields all day resented the general.– Manner of movement verbs hard in reduced
relatives; transitive use of these verbs requires an operation of “syntactic causativisation”
• But…The witch melted in the Wizard of Oz was played by a famous actress.– Seems easy. Suggestion: Unaccusative verbs
OK in reduced relatives
Eyetracking Experiment
• Sentences with Manner of Movement (MOM) verbs
• Sentences with verbs whose thematic grid changes (agent-theme to goal-theme) with passivization
• Sentences with unaccusative verbs (theme in both active and passive subject position)
MOM (manner of movement) sentences
• The horse raced along the beach frightened the onlookers
• The craft sailed to the island served as a makeshift ferry
• Controls– The horse that was raced to to the beach frightened the
onlookers. (full relative clause)– The horse raced along the beach and frightened the
onlookers. (main clause)– Disambiguating region indicated in boldface
Thematic Grid Change Sentences
• The secretary brought the forms filled them out.
• The client asked the question received no thanks.– Main clause structure: Agent – verb – theme– Relative clause structure: Goal – verb – theme– Control: The client asked the question but
received no thanks
Unaccusative Verbs
• The leaves burned today smoked a lot.
• The potatoes baked in the oven smelled terrific.– Main clause structure: theme – verb– Relative clause structure: theme – verb– Control: The potatoes baked in the oven and
smelled terrific.
Predictions
• Manner of movement sentences: Hard, on everybody’s story
• Thematic grid change sentences: Comparably hard, if thematic grid, not causal structure, the root
• Unaccusative sentences: Relatively easy, if thematic grid change not syntactic structure change the underlying problem
First Pass Times
0
400
500
600
Fir
st P
ass
Tim
e, D
isam
big
Reg
,ms
MOM Thematic Unaccusative Change
Legend
RedRelFullRelMainCl
Regression Path Durations
Verb Type
MOM ThemeChg Unaccus
Reg
ress
ion
Pat
h D
urat
ion,
ms
0
400
600
800
1000
1200
1400
Disambiguating Region
Legend
RedRelFullRelMainCl
Comprehension Accuracy
• Questions were asked following 2/3 of the items (half of these were relevant to the RC)– MOM questions: The boy marched to school looked unhappy.
• How did the boy get to school?Someone marched him there/He marched there on his own
– Theta-grid changing questions: The client asked the question received no thanks.
• Who asked the question?The client / someone
– Unaccusative questions: The potatoes baked in the oven smelled terrific
• What was happening with the potatoes?Someone had baked them in the oven / They were sitting in the oven,
baking.
Verb Type
MOM ThemeChg
Pro
port
ion
Cor
rect
0.0
0.2
0.4
0.6
0.8
1.0Question-answering Accuracy
RedRel
FullRel
MainCl
Note: accuracy for Unaccusative verb sentences is not shown. The questions were bad – only 67% correct for main clause items.
Conclusions
• Intuitions of sentence processing difficulty not transparently mirrored in the ET record
• Typical response to difficulty: a regression– May be specific to this experiment; lots of hard,
tricky RC sentences– But even here, only about 1/3 of the trials
• Another response to difficulty: fughetabottit– Just get the words, `mam
For the future: I’d like to be shown wrong, but…
• ET response to comprehension difficulty seems to be variable: linger, regress, nothing
• Eye movements may generally be controlled by word recognition, and only exceptionally affected by higher-level factors
• Effects seen in means may actually occur on only a minority of trials
• Maybe all these problems will disappear if people read texts they really care about, not just single sentences
• But I wouldn’t bet on it
Reg
ress
ion
Pat
h D
ura
tion
, ms.
, Ou
tsid
e R
egio
n
0
20
40
60
80
100
120Disambiguating Region, RPD Outside
Animate Inanimate Animate Inanimate Preview No Preview
RedRel
FullRel
Seco
nd P
ass
Tim
e, m
s., f
rom
Lef
t
0
20
40
60
80
100
120Disambiguating Region, Second Pass from Left
Animate Inanimate Animate Inanimate Preview No Preview
RedRel
FullRel
MacDonald, Pearlmutter, Seidenberg, 1994
• In all cases, the examples cited here were not the only reduced relatives in these articles.
• The pattern of results obtained with global reading times is reflected by differences located on the critical disambiguating phrase.
Ferreira & Clifton 1986 – An early attempt to find easy RRCs
• Ambiguous animate: The defendant examined by the lawyer was unreliable
• Ambiguous inanimate: The evidence examined by the lawyer was unreliable
• Controls: The defendant/evidence who/that was examined by the lawyer was unreliable.– Disambiguating region indicated in boldface