incremental, predictive parsing with psycholinguistically motivated … · 2011-04-13 ·...
TRANSCRIPT
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Incremental, Predictive Parsing withPsycholinguistically Motivated Tree-Adjoining
Grammar
Frank Keller
Institute for Language, Cognition and ComputationSchool of Informatics, University of Edinburghhttp://homepages.inf.ed.ac.uk/keller/
Joint work with Vera Demberg and Alexander Koller
Frank Keller Incremental, Predictive Parsing with PLTAG 1
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
1 IntroductionIncrementalityPredictionCase Study: Syntactic Prediction
2 Prediction and GrammarConceptual IssuesFormalismComparison with TAGModeling Prediction
3 Predictive ParsingTreebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
4 EvaluationParsing PerformanceCognitive Plausibility
Frank Keller Incremental, Predictive Parsing with PLTAG 2
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
1 IntroductionIncrementalityPredictionCase Study: Syntactic Prediction
2 Prediction and GrammarConceptual IssuesFormalismComparison with TAGModeling Prediction
3 Predictive ParsingTreebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
4 EvaluationParsing PerformanceCognitive Plausibility
Frank Keller Incremental, Predictive Parsing with PLTAG 3
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Incrementality
Text and speech are perceived serially.
Human language processing is adapted to this: sentencecomprehension proceeds incrementally:
the interpretation of a sentence is built word by word;
each new word is integrated as fully as possible into arepresentation of the sentence thus far;
processing effort depends on the properties of the word and itsrelationship to the preceding context.
Frank Keller Incremental, Predictive Parsing with PLTAG 4
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Discourse Prediction
Not only is processing word-by-word, it is also predictive:comprehenders anticipate upcoming linguistic material.
van Berkum et al. (2005) show that contextual information is usedto predict specific lexical items; processing difficulty ensues if inputis incompatible with the prediction (ERP study).
Frank Keller Incremental, Predictive Parsing with PLTAG 5
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Structural Prediction
Staub & Clifton (2006) show that the sentence processor can alsomake structural predictions:
(1) Peter read either a book or an essay in the school magazine.
(2) Peter read a book or an essay in the school magazine.
The presence of either leads to shorter reading times on or and onthe NP that follows it (eye-tracking study).
The word either makes it possible to anticipate an upcoming NPconjunction (rather than VP conjunction).
Frank Keller Incremental, Predictive Parsing with PLTAG 6
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Semantic Prediction
Visual world paradigm:
image and speech presented synchronously;
eye-movements reflect listeners’ interpretation of input;
they can also indicate predictions about upcoming input.
Altmann & Kamide (1999) use this paradigm to provided evidencefor semantic prediction. They presented sentences such as:
(3) a. The boy will eat . . .b. The boy will move . . .
together with a scene that contained one edible but severalmovable objects.
Frank Keller Incremental, Predictive Parsing with PLTAG 7
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Semantic Prediction
When participants heard eat, they looked more at the cake.Evidence for prediction induced by semantic restrictions of the verb.
Frank Keller Incremental, Predictive Parsing with PLTAG 8
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Granularity of Prediction
What is the granularity of prediction? We saw predictions can betriggered by:
discourse context;
specific collocations (either . . . or);
semantic restrictions of a lexical item.
But can we get predictions from lexically specific syntacticinformation?
We will look at an experiment in detail that shows prediction basedon verb subcategorization (joint work with Manabu Arai).
Frank Keller Incremental, Predictive Parsing with PLTAG 9
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Design
Compare obligatory transitive verbs (e.g., offend) and intransitiveverbs (e.g., frown):
(4) a. All of the sudden, the inmate offended the judge.b. All of the sudden, the inmate frowned at the judge.c. All of the sudden, the inmate frowned and the judge threw
the gloves.
Listeners’ looks at the verb indicate which verb frame they assume.
Frank Keller Incremental, Predictive Parsing with PLTAG 10
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Materials
(5) All of the sudden, the inmate offended the judge.
Listeners predict upcoming patient information on hearing the verband look at the judge.
Frank Keller Incremental, Predictive Parsing with PLTAG 11
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Materials
(5) All of the sudden, the inmate frowned at the judge.
No prediction on hearing the verb. But listeners predict upcomingpatient information on hearing at and look at the judge.
Frank Keller Incremental, Predictive Parsing with PLTAG 11
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Materials
(5) All of the sudden, the inmate frowned and the judge threwthe gloves.
No prediction on hearing the verb and and.
Frank Keller Incremental, Predictive Parsing with PLTAG 11
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Results
Gazes after the verb onset:
-0.8
-0.7
-0.6
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.22
0
10
0
18
0
26
0
34
0
42
0
50
0
58
0
66
0
74
0
82
0
90
0
98
0
10
60
11
40
12
20
13
00
13
80
14
60
Time from the verb onset (ms)
log
it o
f p
ati
en
t lo
ok
Transitive Intransitive + preposition Intransitive + conjunction
ROI: PATIENT (judge)
Participants looked more at a patient picture on hearing transitiveverbs (offended) than intransitive verbs (frowned).
Frank Keller Incremental, Predictive Parsing with PLTAG 12
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Results
Gazes after hearing at or and :
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.520
100
180
260
340
420
500
580
660
740
820
900
980
1060
1140
1220
1300
1380
1460
Time from the preposition/conjunction onset (ms)
log
ito
fp
ati
en
tlo
ok
s
Intransitive + preposition Intransitive + conjunction
ROI: PATIENT (judge)
Participants predicted and looked more at a patient picture onhearing at than on hearing and.
Frank Keller Incremental, Predictive Parsing with PLTAG 13
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
IncrementalityPredictionCase Study: Syntactic Prediction
Summary
Evidence for the use of verb-specific subcategorization informationin prediction:
participants predicted a direct object following a transitiveverb more than following an intransitive verb;
they made a similar predictions at the preposition following anintransitive verb;
but not if a conjunction followed the intransitive verb.
Frank Keller Incremental, Predictive Parsing with PLTAG 14
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
1 IntroductionIncrementalityPredictionCase Study: Syntactic Prediction
2 Prediction and GrammarConceptual IssuesFormalismComparison with TAGModeling Prediction
3 Predictive ParsingTreebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
4 EvaluationParsing PerformanceCognitive Plausibility
Frank Keller Incremental, Predictive Parsing with PLTAG 15
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Conceptual Issues
Challenge: develop a model of prediction in sentence processingthat accounts for these experimental results. Assumptions:
structures are built incrementally (word by word);
partial structures do not contain unconnected nodes;
upcoming syntactic material is predicted.
Evidence for connectedness: Sturt & Lombardo (2005). Existingincremental parsers don’t build fully connected structures.
Our approach: devise a grammar formalism that supportsincrementality and connectedness; prediction then follows.
Frank Keller Incremental, Predictive Parsing with PLTAG 16
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Implementing Prediction
Experimental results inform our model regarding the granularity ofprediction. The model predicts:
lexical items when they are syntactically required (e.g., either. . . or, pick . . . up);
syntactic structure when required by subcat frames;
syntactic structure when required by connectedness.
Frank Keller Incremental, Predictive Parsing with PLTAG 17
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Formalism
We propose Psycholinguistically Motivated TAG (PLTAG), avariant of tree-adjoining grammar:
in standard TAG, the lexicon consists of initial trees andauxiliary trees (both are lexicalized);
we add unlexicalized predictive trees to achieve connectivity;
the standard TAG operations are substitution and adjunction;
we add verification to verify predictive trees;
we use TAG’s extended domain of locality for lexicalprediction.
PLTAG supports parsing with incremental, fully connectedstructures (Demberg & Keller 2008).
Frank Keller Incremental, Predictive Parsing with PLTAG 18
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Formalism
Lexicon:
Standard TAG lexicon
Predictive lexicon(PLTAG)
Operations:
Substitution
Adjunction
Verification (PLTAG)
Frank Keller Incremental, Predictive Parsing with PLTAG 19
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Formalism
Lexicon:
Standard TAG lexicon
Predictive lexicon(PLTAG)
Operations:
Substitution
Adjunction
Verification (PLTAG)
Example
Initial Tree: S
���
HHH
NP↓ VP
�� HHV
meets
NP↓
Auxiliary Tree: VP
�� HHAP
yesterday
VP*
Frank Keller Incremental, Predictive Parsing with PLTAG 19
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Formalism
Lexicon:
Standard TAG lexicon
Predictive lexicon(PLTAG)
Operations:
Substitution
Adjunction
Verification (PLTAG)
Example
DT
the
substitutes into NP�� HH
DT↓ NN
man
resulting in NP�� HH
DT
the
NN
man
Frank Keller Incremental, Predictive Parsing with PLTAG 19
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Formalism
Lexicon:
Standard TAG lexicon
Predictive lexicon(PLTAG)
Operations:
Substitution
Adjunction
Verification (PLTAG)
Example
VP
�� HHVP* AP
yesterday
adjoins into S
���
HHH
NP��HH
DT
the
NN
man
VP
V
slept
resulting in S
���
HHH
NP��HH
DT
the
NN
man
VP
�� HHVP
V
slept
AP
yesterday
Frank Keller Incremental, Predictive Parsing with PLTAG 19
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Formalism
Lexicon:
Standard TAG lexicon
Predictive lexicon(PLTAG)
Operations:
Substitution
Adjunction
Verification (PLTAG)
Example
Initial Prediction Tree: Ss
�� HHNPs VPs
s
Auxiliary Prediction Tree: VPs
�� HHAPs
s VP*s
Index s marks predicted node.
Frank Keller Incremental, Predictive Parsing with PLTAG 19
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Formalism
Lexicon:
Standard TAG lexicon
Predictive lexicon(PLTAG)
Operations:
Substitution
Adjunction
Verification (PLTAG)
Example
Ss
�� HHNPs
the man
VPss
is verified by S
���
HHH
NP↓ VP
�� HHV
meets
NP↓
resulting in S
���
HHH
NP
the man
VP
�� HHV
meets
NP↓
All nodes indexed with s have to be verified.
Frank Keller Incremental, Predictive Parsing with PLTAG 19
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Comparison with and TAG
TAG derivations are not always incremental.
a
DT
initial(subst)
Peter
NP VP
AP
often
VP
S
V
reads
NP
DT
book
NN
a
VP
S
V NP
reads
NP
Peter
NP
V
reads
VP
NP
S
Peter
NP VP
AP
often
VP
S
V
reads
NP
Peter
NP VP
AP
often
VP
S
V
reads
NP
DT
book
NN
substitution
VPNP
S
V NP
reads
VP
VP*AP
often
adjunction
NP
Peter
NP
NNDT
book
substitution substitution
TAG
Frank Keller Incremental, Predictive Parsing with PLTAG 20
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Comparison with and TAG
PLTAG derivation are always incremental and fully connected.
Peter
NP
ssVPNP
S
s
s
initial(subst)
s
ss
NP
NNDTs
Peter
NP VP
AP
often
VP
S
s s
s
s
Peter
NP VP
AP
often
VP
S
V
reads
NP
Peter
NP VP
AP
often
VP
S
sNPV
reads DT NNsss
Peter
NP VP
AP
often
VP
S
sNPV
reads DT NNsss
a
Peter
NP VP
AP
often
VP
S
V
reads
NP
DT
book
NN
a
Peter
NP
Peter
NP
PLTAG
substitution
VP
VP*AP
often
adjunction
VPNP
S
V NP
reads
verification substitution
a
DT
substitution
NP
NNDT
book
verification
s s
s
VP
S
Frank Keller Incremental, Predictive Parsing with PLTAG 20
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Modeling Prediction
PLTAG assumes three types of prediction:
predictive nodes (required for connectivity);
open substitution nodes (subcategorization);
lexical prediction (e.g., either . . . or).
Connectedness and prediction interact closely:
in order to achieve incrementality with full connectedness,upcoming nodes have to be predicted;
in a fully connected structure, predictions can be read offstraightforwardly (all open prediction and substitution nodes).
Frank Keller Incremental, Predictive Parsing with PLTAG 21
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Modeling Prediction
Predictive Nodes:
Peter
NP
initial(subst)
s
ss
NP
NNDTs
Peter
NP VP
AP
often
VP
S
sNPV
reads DT NNsss
Peter
NP VP
AP
often
VP
S
sNPV
reads DT NNsss
a
Peter
NP VP
AP
often
VP
S
V
reads
NP
DT
book
NN
a
Peter
NP
Peter
NP
Peter
NP
Peter
NP
PLTAGVP
VP*AP
often
adjunction
VPNP
S
V NP
reads
substitution
a
DT
substitution
NP
NNDT
book
verification
NP VPsss
Ss
s
Ss
sVPs
NP
AP
often
VP
V
reads
s
AP
often
VP
Ss
VPs
s
verificationsubstitution
S
VP
Frank Keller Incremental, Predictive Parsing with PLTAG 22
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Modeling Prediction
Open Substitution Nodes:
Peter
NP
ssVPNP
S
s
s
initial(subst)
s
ss
NP
NNDTs
Peter
NP VP
AP
often
VP
S
s s
s
s
Peter
NP VP
AP
often
VP
S
sNPV
reads DT NNsss
a
Peter
NP VP
AP
often
VP
S
V
reads
NP
DT
book
NN
a
Peter
NP
Peter
NP
Peter
NP
Peter
NP VP
AP
often
VP
S
V
reads
PLTAG
substitution
VP
VP*AP
often
adjunction
VPNP
S
V NP
reads
verification substitution
a
DT
substitution
NP
NNDT
book
verification
s s
s
VP
S
ssNPV
reads NNss
VP
AP
often
VP
S
NP
DT
Frank Keller Incremental, Predictive Parsing with PLTAG 22
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Conceptual IssuesFormalismComparison with TAGModeling Prediction
Modeling Prediction
Lexical prediction based on TAG’s extended domain of locality:
NP CC NP
NP
DT
either or
Peter
NP VP
read a book
NPV
S
Peter
NP
DT NP CC NP
either a book or
V
read
VP
S
NP
CC SS*or
CC NPNP*or
NP
(b) derivation at "or" in either−case(a) lexicon entry for "either"
(c) ambiguity at "or"adjunction
(d) lexicon entries for "or"S
S1
S1S1
Frank Keller Incremental, Predictive Parsing with PLTAG 23
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
1 IntroductionIncrementalityPredictionCase Study: Syntactic Prediction
2 Prediction and GrammarConceptual IssuesFormalismComparison with TAGModeling Prediction
3 Predictive ParsingTreebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
4 EvaluationParsing PerformanceCognitive Plausibility
Frank Keller Incremental, Predictive Parsing with PLTAG 24
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
An Incremental Parser for PLTAG
In order to construct an incremental parser for PLTAG, we need to:
1 convert the Penn Treebank into PLTAG format;
2 induce a lexicon from it;
3 develop an incremental parsing algorithm;
4 devise a probability model;
5 formulate a linking theory.
Frank Keller Incremental, Predictive Parsing with PLTAG 25
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 1: Treebank Conversion
Convert Penn Treebank into TAG format (Xia et al. 2000) using:
head percolation table for determining how to cut up a treeinto elementary trees (Magerman 1995);
Propbank for distinguishing arguments and modifiers (Palmeret al. 2003);
noun phrase annotation to derive NP-internal structure(Vadas & Curran 2007).
The resulting trees are less flat, contain head information, andargument/modifier distinction.
Frank Keller Incremental, Predictive Parsing with PLTAG 26
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 2: Lexicon Induction
A standard TAG lexicon can be derived from the TAG Treebank bycutting up the trees into initial trees and adjunction trees.
For the predictive lexicon, we need the notion of connection path.
Connection Path
The connection path of w1 is the minimal amount of structureneeded to connect words w1 . . .wi under one node (Sturt et al.2003).
Essentially, we determine which parts of the tree we need topredict to achieve connectivity.
Frank Keller Incremental, Predictive Parsing with PLTAG 27
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 2: Lexicon Induction
predictive lexiconentries generatedfrom tree
The
DET
Italian
ADJ
N
people
N
NP
vote
V
VP
The
DET
Italian
ADJ
N
people
N
NP
vote
V
VP
Berlusconi
NPoften
ADVP
VP
Berlusconi
NPoften
ADVP
VP
SS
none
Frank Keller Incremental, Predictive Parsing with PLTAG 28
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 2: Lexicon Induction
predictive lexiconentries generatedfrom tree
The
DET
Italian
ADJ
N
people
N
NP
vote
V
VP
Berlusconi
NPoften
ADVP
VP
S
The
DET
Italian
ADJ
N
people
N
NP
vote
V
VP
Berlusconi
NPoften
ADVP
VP
S
Frank Keller Incremental, Predictive Parsing with PLTAG 28
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 2: Lexicon Induction
predictive lexiconentries generatedfrom tree
vote
V
VP
vote
V
VP
Berlusconi
NPoften
ADVP
VP
S
people
N
NP
DET
NP
Ns
s
ss
Frank Keller Incremental, Predictive Parsing with PLTAG 28
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 2: Lexicon Induction
predictive lexiconentries generatedfrom tree
The
DET
Italian
ADJ
N
people
N
NP
vote
V
VP
Berlusconi
NPoften
ADVP
VP
S
The
DET
Italian
ADJ
N
people
N
NP
vote
V
VP
Berlusconi
NPoften
ADVP
VP
S
DET
NP
Ns
s
ss
Frank Keller Incremental, Predictive Parsing with PLTAG 28
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 2: Lexicon Induction
predictive lexiconentries generatedfrom tree
Berlusconi
NP
no new predictive entry
SS
Berlusconi
NP
vote
V
VP
vote
V
VP
often
ADVP
VP
often
ADVP
VP
DET
NP
Ns
s
ss
Frank Keller Incremental, Predictive Parsing with PLTAG 28
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 2: Lexicon Induction
predictive lexiconentries generatedfrom tree
The
DET
Italian
ADJ
N
people
N
NP
vote
V
VP
The
DET
Italian
ADJ
N
people
N
NP
vote
V
VP
Berlusconi
NPoften
ADVP
VP
Berlusconi
NPoften
ADVP
VP
SS
DET
NP
Ns
s
ss
Frank Keller Incremental, Predictive Parsing with PLTAG 28
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 2: Lexicon Induction
predictive lexiconentries generatedfrom tree
vote
V
VP
vote
V
VP
Berlusconi
NP
Berlusconi
NP
SS
DET
NP
N
S
VPNP s
s
ss
s
s
ss
Frank Keller Incremental, Predictive Parsing with PLTAG 28
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 2: Lexicon Induction
predictive lexiconentries generatedfrom tree
The
DET
Italian
ADJ
N
people
N
NP
vote
V
VP
Berlusconi
NPoften
ADVP
VP
S
The
DET
Italian
ADJ
N
people
N
NP
vote
V
VP
Berlusconi
NPoften
ADVP
VP
S
DET
NP
Ns
s
ss
S
VPNP s
s
ss
Frank Keller Incremental, Predictive Parsing with PLTAG 28
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 3: Parsing Algorithm
Properties:
incrementally builds fully connected partial structures;
only allows valid partial PLTAG structures;
constructs all possible structures in parallel.
At word wi , retrieve elementary tree ε for wi and connect it to theprefix tree β for w1 . . .wi−1:
parsing operations: substitution, adjunction, verification;
dependent on status of β and ε: standard or predictive tree.
Frank Keller Incremental, Predictive Parsing with PLTAG 29
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 4: Probability Model
The following properties need to hold (Chiang 2000):
Substitution:∑ε
P(ε|ηβ) = 1
Adjunction:∑ε
P(ε|ηβ) + P(NONE |ηβ) = 1
Verification:∑ε
P(ε|πβ) = 1
where P(ε|ηβ) = P(τε|ηβ)P(λε|τε, λβ)and P(ε|πβ) = P(τε|πβ)P(λε|τε)
elementary tree ε prefix tree β prediction tree πtree structure τ integration point node η a tree’s head leaf λ
Frank Keller Incremental, Predictive Parsing with PLTAG 30
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Treebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
Step 5: Linking Theory
The linking theory translates parser states into processing difficulty:
elementary tree εwi is integrated with prefix tree βw1...wi−1 ;
processing difficulty proportional to change in distributionP(β) from wi−1 to wi ;
each predicted tree π has a time-stamp t;
at verification, decay d is calculated based on t (recentlyaccessed structures are easier to integrate).
Surprisal
Dwi =
︷ ︸︸ ︷− log
∑βw1...wi
P(βw1...wi ) + log∑
βw1...wi−1
P(βw1...wi−1)
− log∑π
P(π)(1−d tπ )
}Verification Cost
Frank Keller Incremental, Predictive Parsing with PLTAG 31
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Parsing PerformanceCognitive Plausibility
1 IntroductionIncrementalityPredictionCase Study: Syntactic Prediction
2 Prediction and GrammarConceptual IssuesFormalismComparison with TAGModeling Prediction
3 Predictive ParsingTreebank Conversion and Lexicon InductionParsing Algorithm and Probability ModelLinking Theory
4 EvaluationParsing PerformanceCognitive Plausibility
Frank Keller Incremental, Predictive Parsing with PLTAG 32
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Parsing PerformanceCognitive Plausibility
Parsing Performance
Computational evaluation of PLTAG parser:
train and test on standard Penn Treebank data (converted toPLTAG), with sentences of length 40 or less;
assume gold-standard POS tags;
use a supertagger to choose prediction trees (one wordlookahead);
coverage on the test set is not perfect: beam search; missinglexical entries.
Frank Keller Incremental, Predictive Parsing with PLTAG 33
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Parsing PerformanceCognitive Plausibility
Results
Model Precision Recall F-score CoverageBaseline 44.39 52.38 48.06 85.10PLTAG Parser 78.01 78.83 78.42 92.73Prediction Tree Oracle 79.88 80.51 80.19 89.54
Baseline: pick most frequent tree (highest combined frequency ofall subtrees).
Oracle: assume correct prediction tree (instead of supertagging).
Frank Keller Incremental, Predictive Parsing with PLTAG 34
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Parsing PerformanceCognitive Plausibility
Comparison with other TAG Parsers
imple- incre-Model mented mental connected predictive F-scoreMazzei et al. (2007) – + + + n/aThis work + + + + 78.4Kato et al. (2004) + + + – 79.7Sarkar (2001) + – – – 79.8Chiang (2000) + – – – 86.7Shen & Joshi (2005) + + – – 87.4∗
∗evaluated on dependencies
Frank Keller Incremental, Predictive Parsing with PLTAG 35
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Parsing PerformanceCognitive Plausibility
Comparison with other TAG Parsers
Performance not directly comparable with parsers that use theoriginal Treebank structure (simpler NP structure, etc.);
there are structural differences even with other TAG parsers(LTAG, spinal TAG);
Mazzei et al. (2007) parser conceptually most similar, but notimplemented and evaluated;
Kato et al. (2004) make strong simplifying assumptions (nomodifier/argument distinction);
Sarkar (2001) and Chiang (2000) parsers are not incremental;
Shen & Joshi (2005) don’t build connected structures.
Frank Keller Incremental, Predictive Parsing with PLTAG 36
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Parsing PerformanceCognitive Plausibility
Cognitive Plausibility
Psycholinguistic evaluation of PLTAG parser:
train on Penn Treebank;
take experimental materials from psycholinguistic experiments;
parse them using the PLTAG parser, compute processingdifficulty values for each sentence;
compare to published reading time results.
Baseline: standard surprisal model (PLTAG without prediction andverification component).
Frank Keller Incremental, Predictive Parsing with PLTAG 37
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Parsing PerformanceCognitive Plausibility
Either . . . or Constructions
PLTAG model predicts difficulty in either . . . or constructions:
(6) Peter read either a book or an essay in the school magazine.
(7) Peter read a book or an essay in the school magazine.
Frank Keller Incremental, Predictive Parsing with PLTAG 38
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Parsing PerformanceCognitive Plausibility
Relative Clause Asymmetry
Classic result in psycholinguistics: subject relative clauses are easierto process than object relative clauses.
(8) SRC: The reporter that attacked the senator admitted the error.
(9) ORC: The reporter that the senator attacked admitted the error.
340
360
380
400
420
440
460
Rea
ding
tim
e in
mse
c
Reading Times for SRC vs. ORC (Gibson, 1998)
SRCORC
admitted the error.who attacked thewho the senator
senatorattacked
The reporter
Frank Keller Incremental, Predictive Parsing with PLTAG 39
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Parsing PerformanceCognitive Plausibility
The relative clause asymmetry
PLTAG model predicts difficulty at verb region:
Correct predictions, but verification component necessary, resultsnot predicted by surprisal-only baseline.
Frank Keller Incremental, Predictive Parsing with PLTAG 40
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Parsing PerformanceCognitive Plausibility
Conclusions
Human sentence processing is incremental and predictive;
evidence for lexical syntactic prediction (subcat frames);
we presented a version of TAG that models these properties;
the model comes with a parser, a probability model, and alinking theory;
performance comparable to parsers with similar properties inthe TAG literature;
cognitive evaluation using experimental data: either . . . orprediction and relative clauses.
Frank Keller Incremental, Predictive Parsing with PLTAG 41
IntroductionPrediction and Grammar
Predictive ParsingEvaluation
Parsing PerformanceCognitive Plausibility
References
Altmann, G. T. M., & Kamide, Y. (1999). Incremental interpretation at verbs: Restricting the domain ofsubsequent reference. Cognition, 73 , 247–264.
Chiang, D. (2000). Statistical parsing with an automatically-extracted tree adjoining grammar. In Proceedings ofthe 38th Annual Meeting of the Association for Computational Linguistics, (pp. 463–470), Hong Kong.
Demberg, V., & Keller, F. (2008). A psycholinguistically motivated version of TAG. In Proceedings of the 9thInternational Workshop on Tree Adjoining Grammars and Related Formalisms, (pp. 25–32), Tubingen.
Kato, Y., Matsubara, S., & Inagaki, Y. (2004). Stochastically evaluating the validity of partial parse trees inincremental parsing. In F. Keller, S. Clark, M. Crocker, & M. Steedman (eds.), Proceedings of the ACLWorkshop on Incremental Parsing: Bringing Engineering and Cognition Together , (pp. 9–15), Barcelona.
Magerman, D. (1995). Statistical decision-tree models for parsing. In Proceedings of the 33rd Annual Meeting ofthe Association for Computational Linguistics, (pp. 276–283), Cambridge, MA.
Mazzei, A., Lombardo, V., & Sturt, P. (2007). Dynamic TAG and lexical dependencies. Research on Language andComputation, 5 , 309–332.
Palmer, M., Gildea, D., & Kingsbury, P. (2003). The proposition bank: An annotated corpus of semantic roles.Computational Linguistics, 31 , 71–106.
Sarkar, A. (2001). Applying co-training methods to statistical parsing. In Proceedings of the 2nd Conference of theNorth American Chapter of the Association for Computational Linguistics, (pp. 175–182), Pittsburgh, PA.Association for Computational Linguistics.
Shen, L., & Joshi, A. K. (2005). Incremental LTAG parsing. In Proceedings of the Human Language TechnologyConference of the North American Chapter of the Association for Computational Linguistics, (pp. 811–818),Vancouver.
Staub, A., & Clifton, C. (2006). Syntactic prediction in language comprehension: Evidence from either . . . or.Journal of Experimental Psychology: Learning, Memory, and Cognition, 32 , 425–436.
Sturt, P., Costa, F., Lombardo, V., & Frasconi, P. (2003). Learning first-pass structural attachment preferenceswith dynamic grammars and recursive neural networks. Cognition, 88 , 133–169.
Sturt, P., & Lombardo, V. (2005). Processing coordinated structures: Incrementality and connectedness. CognitiveScience, 29 , 291–305.
Vadas, D., & Curran, J. (2007). Adding noun phrase structure to the Penn Treebank. In Proceedings of the 45thAnnual Meeting of the Association for Computational Linguistics, (pp. 240–247), Prague.
van Berkum, J. J. A., Brown, C. M., Zwitserlood, P., Kooijman, V., & Hagoort, P. (2005). Anticipating upcomingwords in discourse: Evidence from erps and reading times. Journal of Experimental Psychology: Learning,Memory, and Cognition, 31 , 443–467.
Xia, F., Palmer, M., & Joshi, A. (2000). A uniform method of grammar extraction and its applications. InProceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and VeryLarge Corpora, (pp. 53–62), Hong Kong.
Frank Keller Incremental, Predictive Parsing with PLTAG 42