cs626-449: speech, nlp and the web/topics in ai
DESCRIPTION
CS626-449: Speech, NLP and the Web/Topics in AI. Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 13: Deeper Adjective and PP Structure; Structural Ambiguity and Parsing. Types of Grammar. Prescriptive Grammar Taught in schools Emphasis is on usage Descriptive Grammar - PowerPoint PPT PresentationTRANSCRIPT
CS626-449: Speech, NLP and the Web/Topics in AI
Pushpak BhattacharyyaCSE Dept., IIT Bombay
Lecture 13: Deeper Adjective and PP Structure; Structural Ambiguity and
Parsing
Types of Grammar
Prescriptive Grammar Taught in schools Emphasis is on usage
Descriptive Grammar Also known as Linguistic Grammar Describes Language
Types of Languages SVO
Subject – Verb – Object English E.g. Ram likes music. S V O
SOV Subject- Object-Verb Indian Languages E.g. रा�म पा�नी� पा� राहा� हा� | S O V
More deeply embedded structureNP
PP
AP
big
The
of poems
with the blue cover
N’1
Nbook
PP
N’2
N’3
Bar-level projections Add intermediate structures
NP (D) N’ N’ (AP) N’ | N’ (PP) | N (PP)
() indicates optionality
New rules produce this treeNP
PP
AP
big
The
of poems
with the blue cover
N’1
Nbook
PP
N’2
N’3
N-bar
As opposed to this tree
NP
PPAP
big
The
of poems
with the blue coverbook
PP
V-bar
What is the element in verbs corresponding to one-replacement for nouns
do-so or did-so
I [eat beans with a fork]
VP
NP
beans
eat
with a fork
PP
No constituent that groups together V and NP and excludesPP
Need for intermediate constituents
I [eat beans] with a fork but Ram [does so] with a spoon
V2’
NP
beans
eat
with a fork
PP
VP
V1’
V
VPV’V’ V’ (PP)V’ V (NP)
How to target V1’
I [eat beans with a fork], and Ram [does so] too.
V2’
NP
beans
eat
with a fork
PP
VP
V1’
V
VPV’V’ V’ (PP)V’ V (NP)
Case of conjunction
V3’
NP
beans
eat
In the afternoon
PP
VP
V1’
V
V4’
NP
coffee
drink
V
V2’
Conjand
A-bar: adjectives
A3’
A4’
blue
Very
AP
bright
A5’
A6’
green
A1’
AP
A2’
Conjand
AP AP
dull
AP A’A’ (AP) A’A’ A (PP)
So-replacement for adjectives
Ram is very serious about studies , but less so than Shyam
P-bar: prepositions
P2’
NP
the table
right
AP
off
P3’
NP
the trash
A1’
AP
P1’
Conjand
P P
into
PP P’P’ P’ (PP)P’ P (NP)
PP
So-replacement for Prepositions
Ram is utterly in debt, but Shyam is only partly so.
Complements and Adjuncts orArguments and Adjuncts
Rules in bar notation: Noun
NP (D) N’ N’ (AP) N’ N’ N’ (PP) N’ N (PP)
Rules in bar notation: Verb
VP V’ V’ V’ (PP) V’ V (NP)
Rules in bar notation: Adjective
AP A’ A’ (AP) A’ A’ A (PP)
Rules in bar notation: Preposition
PP P’ P’ P’ (PP) P’ P (NP)
Introducing the “X factor”
Let X stand for any category N, V, A, P
Let XP stand for NP, VP, AP and PP Let X’ stand for N’, V’, A’ and P’
XP to X’
Collect the first level rules NP (D) N’ VP V’ AP A’ PP P’
And produce XP (YP) X’
X’ to X’
Collect the 2nd level rules N’ (AP) N’ or N’ (PP) V’ V’ (PP) A’ (AP) A’ P’ P’ (PP)
And produce X’ (ZP) X’ or X (ZP)
X’ to X
Collect the 3rd level rules N’ N (PP) V’ V (NP) A’ A (PP) P’ P (NP)
And produce X’ X (WP)
Basic observations about X and X’
X’ X (WP) X’ X’ (ZP) X is called Head Phrases must have Heads:
Headedness property Category of XP and X must match:
Endocentricity
Basic observations about X and X’
X’ X (WP) X’ X’ (ZP) Sisters of X are complements
Roughly correspond to objects Sisters of X’ are Adjuncts
PPs and Adjectives are typical adjuncts We have adjunct rules and
complement rules
Structural difference between complements and adjuncts
X’
WP
Complement
X’
X
ZP
XP
Adjunct
Complements and Adjuncts in NPs
N’
PP
of poems
N’
N
ZP
NP
with red cover
book
Any number of Adjuncts
N’
PP
of poems
N’
N
ZP
N’
with red cover
book
NP
from Oxford Press
Parsing Algorithm
A simplified grammar
S NP VP NP DT N | N VP V ADV | V
A segment of English Grammar
S’(C) S S{NP/S’} VP VP(AP+) (VAUX) V (AP+)
({NP/S’}) (AP+) (PP+) (AP+) NP(D) (AP+) N (PP+) PPP NP AP(AP) A
Example Sentence
People laugh1 2 3
Lexicon:People - N, V Laugh - N, V
These are positions
This indicate that both Noun and Verb is
possible for the word “People”
Top-Down Parsing
State Backup State Action
-----------------------------------------------------------------------------------------------------
1. ((S) 1) - -
2. ((NP VP)1) - -
3a. ((DT N VP)1) ((N VP) 1) -
3b. ((N VP)1) - -
4. ((VP)2) - Consume “People”
5a. ((V ADV)2) ((V)2) -
6. ((ADV)3) ((V)2) Consume “laugh”
5b. ((V)2) - -
6. ((.)3) - Consume “laugh”
Termination Condition : All inputs over. No symbols remaining.
Note: Input symbols can be pushed back.
Position of input pointer
Discussion for Top-Down Parsing This kind of searching is goal driven. Gives importance to textual precedence
(rule precedence). No regard for data, a priori (useless
expansions made).
Bottom-Up Parsing
Some conventions:N12
S1? -> NP12 ° VP2?
Represents positions
End position unknownWork on the LHS done, while the work on RHS remaining
Bottom-Up Parsing (pictorial representation)
S -> NP12 VP23 °
People Laugh 1 2 3
N12 N23
V12 V23
NP12 -> N12 ° NP23 -> N23 °
VP12 -> V12 ° VP23 -> V23 °
S1? -> NP12 ° VP2?
Problem with Top-Down Parsing
• Left Recursion• Suppose you have A-> AB rule. Then we will have the expansion as
follows:• ((A)K) -> ((AB)K) -> ((ABB)K) ……..
Combining top-down and bottom-up strategies
Top-Down Bottom-Up Chart Parsing
Combines advantages of top-down & bottom-up parsing.
Does not work in case of left recursion. e.g. – “People laugh”
People – noun, verb Laugh – noun, verb
Grammar – S NP VPNP DT N | N
VP V ADV | V
Transitive Closure
People laugh
1 2 3
S NP VP NP N VP V
NP DT N S NPVP S NP VP NP N VP V ADV success
VP V
Arcs in Parsing
Each arc represents a chart which records Completed work (left of ) Expected work (right of )
Example
People laugh loudly
1 2 3 4
S NP VP NP N VP V VP V ADVNP DT N S NPVP VP VADV S NP VPNP N VP V ADV S NP VP
VP V
Dealing With Structural Ambiguity
Multiple parses for a sentence The man saw the boy with a telescope. The man saw the mountain with a
telescope. The man saw the boy with the ponytail.
At the level of syntax, all these sentences are ambiguous. But semantics can disambiguate 2nd & 3rd sentence.
Prepositional Phrase (PP) Attachment Problem
V – NP1 – P – NP2
(Here P means preposition)NP2 attaches to NP1 ?
or NP2 attaches to V ?
Parse Trees for a Structurally Ambiguous Sentence
Let the grammar be – S NP VPNP DT N | DT N PPPP P NPVP V NP PP | V NPFor the sentence,“I saw a boy with a telescope”
Parse Tree - 1S
NP VP
N V NP
Det N PP
P NP
Det N
I saw
a boy
with
a telescope
Parse Tree -2S
NP VP
N V NP
Det N
PP
P NP
Det NI saw
a boy with
a telescope
Parsing Structural Ambiguity
Topics to be covered
Dealing with Structural Ambiguity Moving towards Dependency
Parsing
Parsing for Structurally Ambiguous Sentences Sentence “I saw a boy with a telescope” Grammar:
S NP VPNP ART N | ART N PP | PRONVP V NP PP | V NP
ART a | an | theN boy | telescopePRON IV saw
Ambiguous Parses Two possible parses:
PP attached with Verb (i.e. I used a telescope to see)
( S ( NP ( PRON “I” ) ) ( VP ( V “saw” ) ( NP ( (ART “a”) ( N “boy”))( PP (P “with”) (NP ( ART “a” ) ( N
“telescope”))))) PP attached with Noun (i.e. boy had a telescope) ( S ( NP ( PRON “I” ) ) ( VP ( V “saw” )
( NP ( (ART “a”) ( N “boy”) (PP (P “with”) (NP ( ART “a” ) ( N
“telescope”))))))
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |
PRON
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |
PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”,
backup state (b) used
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |
PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”,
backup state (b) used
3B
( ( PRON VP ) 1 ) − −
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |
PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”,
backup state (b) used
3B
( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |
PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”,
backup state (b) used
3B
( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |
PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”,
backup state (b) used
3B
( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
6 ( ( NP PP ) 3 ) − Consumed “saw”
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |
PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”,
backup state (b) used
3B
( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
6 ( ( NP PP ) 3 ) − Consumed “saw”
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |
PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”,
backup state (b) used
3B
( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
6 ( ( NP PP ) 3 ) − Consumed “saw”
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |
PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”,
backup state (b) used
3B
( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
6 ( ( NP PP ) 3 ) − Consumed “saw”
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed “boy”
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |
PRON
3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )
(b) ( ( PRON VP ) 1)
− ART does not match “I”,
backup state (b) used
3B
( ( PRON VP ) 1 ) − −
4 ( ( VP ) 2 ) − Consumed “I”
5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used
6 ( ( NP PP ) 3 ) − Consumed “saw”
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed “boy”
10
( ( P NP ) ) − −
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
… … … … …
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed “boy”
10
( ( P NP ) 5 ) − −
11
( ( NP ) 6 ) − Consumed “with”
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
… … … … …
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed “boy”
10
( ( P NP ) 5 ) − −
11
( ( NP ) 6 ) − Consumed “with”
12
( ( ART N ) 6 ) (a) ( ( ART N PP ) 6 )
(b) ( ( PRON ) 6)
−
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
… … … … …
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed “boy”
10
( ( P NP ) 5 ) − −
11
( ( NP ) 6 ) − Consumed “with”
12
( ( ART N ) 6 ) (a) ( ( ART N PP ) 6 )
(b) ( ( PRON ) 6)
−
13
( ( N ) 7 ) − Consumed “a”
Top Down ParseState Backup State Action Comments
1 ( ( S ) 1 ) − − Use S NP VP
… … … … …
7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )
(b) ( ( PRON PP ) 3 )
8 ( ( N PP) 4 ) − Consumed “a”
9 ( ( PP ) 5 ) − Consumed “boy”
10
( ( P NP ) 5 ) − −
11
( ( NP ) 6 ) − Consumed “with”
12
( ( ART N ) 6 ) (a) ( ( ART N PP ) 6 )
(b) ( ( PRON ) 6)
−
13
( ( N ) 7 ) − Consumed “a”
14
( ( − ) 8 ) − Consume “telescope”
Finish Parsing
Top Down Parsing - Observations
Top down parsing gave us the Verb Attachment Parse Tree (i.e., I used a telescope)
To obtain the alternate parse tree, the backup state in step 5 will have to be invoked
Is there an efficient way to obtain all parses ?
I saw a boy with a telescope1 2 3 4 5 6 7 8
Colour Scheme : Blue for Normal Parse Green for Verb Attachment Parse Purple for Noun Attachment Parse Red for Invalid Parse
Bottom Up Parse
I saw a boy with a telescope
1 2 3 4 5 6 7 8
Bottom Up Parse
NP12 PRON12
S1?NP12VP2?
I saw a boy with a telescope
1 2 3 4 5 6 7 8
Bottom Up Parse
NP12 PRON12
S1?NP12VP2? VP2?V23NP3?PP??
VP2?V23NP3?
I saw a boy with a telescope
1 2 3 4 5 6 7 8
Bottom Up Parse
NP12 PRON12
S1?NP12VP2? VP2?V23NP3?PP??
VP2?V23NP3? NP35 ART34N45
NP3?ART34N45PP5?
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP3?ART34N45PP5?
Bottom Up Parse
NP12 PRON12
S1?NP12VP2? VP2?V23NP3?PP??
VP2?V23NP3? NP35 ART34N45
NP3?ART34N45PP5?
NP35ART34N45
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP3?ART34N45PP5?
Bottom Up Parse
NP12 PRON12
S1?NP12VP2? VP2?V23NP3?PP??
VP2?V23NP3? NP35 ART34N45
NP3?ART34N45PP5?
NP35ART34N45
VP25V23NP35 S15NP12VP25
VP2?V23NP35PP5?
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP3?ART34N45PP5?
Bottom Up Parse
NP12 PRON12
S1?NP12VP2? VP2?V23NP3?PP??
VP2?V23NP3? NP35 ART34N45
NP3?ART34N45PP5?
PP5?P56NP6?NP35ART34N45
VP25V23NP35 S15NP12VP25
VP2?V23NP35PP5?
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP3?ART34N45PP5?
Bottom Up Parse
NP12 PRON12
S1?NP12VP2? VP2?V23NP3?PP??
VP2?V23NP3? NP35 ART34N45
NP3?ART34N45PP5?
PP5?P56NP6?NP35ART34N45 NP68ART67N7?
NP6?ART67N78PP8?
VP25V23NP35 S15NP12VP25
VP2?V23NP35PP5?
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP3?ART34N45PP5?
Bottom Up Parse
NP12 PRON12
S1?NP12VP2? VP2?V23NP3?PP??
VP2?V23NP3? NP35 ART34N45
NP3?ART34N45PP5?
PP5?P56NP6?NP35ART34N45 NP68ART67N7?
NP6?ART67N78PP8?
NP68ART67N78
VP25V23NP35 S15NP12VP25
VP2?V23NP35PP5?
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP3?ART34N45PP5?
Bottom Up Parse
NP12 PRON12
S1?NP12VP2? VP2?V23NP3?PP??
VP2?V23NP3? NP35 ART34N45
NP3?ART34N45PP5?
PP5?P56NP6?NP35ART34N45 NP68ART67N7?
NP6?ART67N78PP8?
NP68ART67N78
VP25V23NP35 S15NP12VP25
VP2?V23NP35PP5? PP58P56NP68
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP3?ART34N45PP5?
Bottom Up Parse
NP12 PRON12
S1?NP12VP2? VP2?V23NP3?PP??
VP2?V23NP3? NP35 ART34N45
NP3?ART34N45PP5?
PP5?P56NP6?NP35ART34N45 NP68ART67N7?
NP6?ART67N78PP8?
NP68ART67N78
VP25V23NP35 S15NP12VP25
VP2?V23NP35PP5? PP58P56NP68
NP38ART34N45PP58
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP3?ART34N45PP5?
Bottom Up Parse
NP12 PRON12
S1?NP12VP2? VP2?V23NP3?PP??
VP2?V23NP3? NP35 ART34N45
NP3?ART34N45PP5?
PP5?P56NP6?NP35ART34N45 NP68ART67N7?
NP6?ART67N78PP8?
NP68ART67N78
VP25V23NP35 S15NP12VP25
VP2?V23NP35PP5? PP58P56NP68
NP38ART34N45PP58
VP28V23NP38VP28V23NP35PP58
I saw a boy with a telescope
1 2 3 4 5 6 7 8
NP3?ART34N45PP5?
Bottom Up Parse
NP12 PRON12
S1?NP12VP2? VP2?V23NP3?PP??
VP2?V23NP3? NP35 ART34N45
NP3?ART34N45PP5?
PP5?P56NP6?NP35ART34N45 NP68ART67N7?
NP6?ART67N78PP8?
NP68ART67N78
VP25V23NP35 S15NP12VP25
VP2?V23NP35PP5? PP58P56NP68
NP38ART34N45PP58
VP28V23NP38VP28V23NP35PP58
S18NP12VP28
Bottom Up Parsing - Observations
Both Noun Attachment and Verb Attachment Parses obtained by simply systematically applying the rules
Numbers in subscript help in verifying the parse and getting chunks from the parse
Exercise
For the sentence,“The man saw the boy with a
telescope” & the grammar given previously, compare the performance of top-down, bottom-up & top-down chart parsing.
Verb Alternation (1/2) (ref: Natural Language Understanding, James Allan)
Verb ComplementStructure
Example
laugh Empty (in transitive) Ram laughed
find NP (transitive) Ram found the key
give NP+NP (di transitive) Ram gave Sita the paper
give NP+PP [to] Ram gave the paper to Sita
Reside Loc Phrase Ram resides in Mumbai
put NP+loc phrase Ram put the book inside the box
speak PP [with]+PP[about] Ram with Sita about floods
try VP[to] Ram tried to apologise
tell NP+VP[to] Ram told the man to go
Verb Alternation (1/2)Verb Complement
StructureExample
wish S [to] Ram wished for the man to go
keep VP [ing] Ram keeps hoping for the best
catch NP+VP [ing] Ram caught Shyam looking in his desk
Watch NP+VP [base] Ram watched Shyam eat the pizza
regret S [that] Ram regretted that he had eaten the whole thing
Tell NP+S [that] Ram told Sita that he was sorry
Seem ADJP Ram seems unhappy in his new job
Think NP+ADJP Ram thinks Sita is happy
Know S [wh] Ram knows where to go