cs626-449: speech, nlp and the web/topics in ai

CS626-449: Speech, NLP and the Web/Topics in AI

Pushpak BhattacharyyaCSE Dept., IIT Bombay

Lecture 13: Deeper Adjective and PP Structure; Structural Ambiguity and

Parsing

Types of Grammar

Prescriptive Grammar Taught in schools Emphasis is on usage

Descriptive Grammar Also known as Linguistic Grammar Describes Language

Types of Languages SVO

Subject – Verb – Object English E.g. Ram likes music. S V O

SOV Subject- Object-Verb Indian Languages E.g. रा�म पा�नी� पा� राहा� हा� | S O V

More deeply embedded structureNP

PP

AP

big

The

of poems

with the blue cover

N’1

Nbook

PP

N’2

N’3

Bar-level projections Add intermediate structures

NP (D) N’ N’ (AP) N’ | N’ (PP) | N (PP)

() indicates optionality

New rules produce this treeNP

PP

AP

big

The

of poems

with the blue cover

N’1

Nbook

PP

N’2

N’3

N-bar

As opposed to this tree

NP

PPAP

big

The

of poems

with the blue coverbook

PP

V-bar

What is the element in verbs corresponding to one-replacement for nouns

do-so or did-so

I [eat beans with a fork]

VP

NP

beans

eat

with a fork

PP

No constituent that groups together V and NP and excludesPP

Need for intermediate constituents

I [eat beans] with a fork but Ram [does so] with a spoon

V2’

NP

beans

eat

with a fork

PP

VP

V1’

V

VPV’V’ V’ (PP)V’ V (NP)

How to target V1’

I [eat beans with a fork], and Ram [does so] too.

V2’

NP

beans

eat

with a fork

PP

VP

V1’

V

VPV’V’ V’ (PP)V’ V (NP)

Case of conjunction

V3’

NP

beans

eat

In the afternoon

PP

VP

V1’

V

V4’

NP

coffee

drink

V

V2’

Conjand

A-bar: adjectives

A3’

A4’

blue

Very

AP

bright

A5’

A6’

green

A1’

AP

A2’

Conjand

AP AP

dull

AP A’A’ (AP) A’A’ A (PP)

So-replacement for adjectives

Ram is very serious about studies , but less so than Shyam

P-bar: prepositions

P2’

NP

the table

right

AP

off

P3’

NP

the trash

A1’

AP

P1’

Conjand

P P

into

PP P’P’ P’ (PP)P’ P (NP)

PP

So-replacement for Prepositions

Ram is utterly in debt, but Shyam is only partly so.

Complements and Adjuncts orArguments and Adjuncts

Rules in bar notation: Noun

NP (D) N’ N’ (AP) N’ N’ N’ (PP) N’ N (PP)

Rules in bar notation: Verb

VP V’ V’ V’ (PP) V’ V (NP)

Rules in bar notation: Adjective

AP A’ A’ (AP) A’ A’ A (PP)

Rules in bar notation: Preposition

PP P’ P’ P’ (PP) P’ P (NP)

Introducing the “X factor”

Let X stand for any category N, V, A, P

Let XP stand for NP, VP, AP and PP Let X’ stand for N’, V’, A’ and P’

XP to X’

Collect the first level rules NP (D) N’ VP V’ AP A’ PP P’

And produce XP (YP) X’

X’ to X’

Collect the 2nd level rules N’ (AP) N’ or N’ (PP) V’ V’ (PP) A’ (AP) A’ P’ P’ (PP)

And produce X’ (ZP) X’ or X (ZP)

X’ to X

Collect the 3rd level rules N’ N (PP) V’ V (NP) A’ A (PP) P’ P (NP)

And produce X’ X (WP)

Basic observations about X and X’

X’ X (WP) X’ X’ (ZP) X is called Head Phrases must have Heads:

Headedness property Category of XP and X must match:

Endocentricity

Basic observations about X and X’

X’ X (WP) X’ X’ (ZP) Sisters of X are complements

Roughly correspond to objects Sisters of X’ are Adjuncts

PPs and Adjectives are typical adjuncts We have adjunct rules and

complement rules

Structural difference between complements and adjuncts

X’

WP

Complement

X’

X

ZP

XP

Adjunct

Complements and Adjuncts in NPs

N’

PP

of poems

N’

N

ZP

NP

with red cover

book

Any number of Adjuncts

N’

PP

of poems

N’

N

ZP

N’

with red cover

book

NP

from Oxford Press

Parsing Algorithm

A simplified grammar

S NP VP NP DT N | N VP V ADV | V

A segment of English Grammar

S’(C) S S{NP/S’} VP VP(AP+) (VAUX) V (AP+)

({NP/S’}) (AP+) (PP+) (AP+) NP(D) (AP+) N (PP+) PPP NP AP(AP) A

Example Sentence

People laugh1 2 3

Lexicon:People - N, V Laugh - N, V

These are positions

This indicate that both Noun and Verb is

possible for the word “People”

Top-Down Parsing

State Backup State Action

-----------------------------------------------------------------------------------------------------

1. ((S) 1) - -

2. ((NP VP)1) - -

3a. ((DT N VP)1) ((N VP) 1) -

3b. ((N VP)1) - -

4. ((VP)2) - Consume “People”

5a. ((V ADV)2) ((V)2) -

6. ((ADV)3) ((V)2) Consume “laugh”

5b. ((V)2) - -

6. ((.)3) - Consume “laugh”

Termination Condition : All inputs over. No symbols remaining.

Note: Input symbols can be pushed back.

Position of input pointer

Discussion for Top-Down Parsing This kind of searching is goal driven. Gives importance to textual precedence

(rule precedence). No regard for data, a priori (useless

expansions made).

Bottom-Up Parsing

Some conventions:N12

S1? -> NP12 ° VP2?

Represents positions

End position unknownWork on the LHS done, while the work on RHS remaining

Bottom-Up Parsing (pictorial representation)

S -> NP12 VP23 °

People Laugh 1 2 3

N12 N23

V12 V23

NP12 -> N12 ° NP23 -> N23 °

VP12 -> V12 ° VP23 -> V23 °

S1? -> NP12 ° VP2?

Problem with Top-Down Parsing

• Left Recursion• Suppose you have A-> AB rule. Then we will have the expansion as

follows:• ((A)K) -> ((AB)K) -> ((ABB)K) ……..

Combining top-down and bottom-up strategies

Top-Down Bottom-Up Chart Parsing

Combines advantages of top-down & bottom-up parsing.

Does not work in case of left recursion. e.g. – “People laugh”

People – noun, verb Laugh – noun, verb

Grammar – S NP VPNP DT N | N

VP V ADV | V

Transitive Closure

People laugh

1 2 3

S NP VP NP N VP V

NP DT N S NPVP S NP VP NP N VP V ADV success

VP V

Arcs in Parsing

Each arc represents a chart which records Completed work (left of ) Expected work (right of )

Example

People laugh loudly

1 2 3 4

S NP VP NP N VP V VP V ADVNP DT N S NPVP VP VADV S NP VPNP N VP V ADV S NP VP

VP V

Dealing With Structural Ambiguity

Multiple parses for a sentence The man saw the boy with a telescope. The man saw the mountain with a

telescope. The man saw the boy with the ponytail.

At the level of syntax, all these sentences are ambiguous. But semantics can disambiguate 2nd & 3rd sentence.

Prepositional Phrase (PP) Attachment Problem

V – NP1 – P – NP2

(Here P means preposition)NP2 attaches to NP1 ?

or NP2 attaches to V ?

Parse Trees for a Structurally Ambiguous Sentence

Let the grammar be – S NP VPNP DT N | DT N PPPP P NPVP V NP PP | V NPFor the sentence,“I saw a boy with a telescope”

Parse Tree - 1S

NP VP

N V NP

Det N PP

P NP

Det N

I saw

a boy

with

a telescope

Parse Tree -2S

NP VP

N V NP

Det N

PP

P NP

Det NI saw

a boy with

a telescope

Parsing Structural Ambiguity

Topics to be covered

Dealing with Structural Ambiguity Moving towards Dependency

Parsing

Parsing for Structurally Ambiguous Sentences Sentence “I saw a boy with a telescope” Grammar:

S NP VPNP ART N | ART N PP | PRONVP V NP PP | V NP

ART a | an | theN boy | telescopePRON IV saw

Ambiguous Parses Two possible parses:

PP attached with Verb (i.e. I used a telescope to see)

( S ( NP ( PRON “I” ) ) ( VP ( V “saw” ) ( NP ( (ART “a”) ( N “boy”))( PP (P “with”) (NP ( ART “a” ) ( N

“telescope”))))) PP attached with Noun (i.e. boy had a telescope) ( S ( NP ( PRON “I” ) ) ( VP ( V “saw” )

( NP ( (ART “a”) ( N “boy”) (PP (P “with”) (NP ( ART “a” ) ( N

“telescope”))))))

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |

PRON

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |

PRON

3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )

(b) ( ( PRON VP ) 1)

− ART does not match “I”,

backup state (b) used

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |

PRON

3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )

(b) ( ( PRON VP ) 1)

− ART does not match “I”,

backup state (b) used

3B

( ( PRON VP ) 1 ) − −

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |

PRON

3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )

(b) ( ( PRON VP ) 1)

− ART does not match “I”,

backup state (b) used

3B

( ( PRON VP ) 1 ) − −

4 ( ( VP ) 2 ) − Consumed “I”

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |

PRON

3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )

(b) ( ( PRON VP ) 1)

− ART does not match “I”,

backup state (b) used

3B

( ( PRON VP ) 1 ) − −

4 ( ( VP ) 2 ) − Consumed “I”

5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |

PRON

3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )

(b) ( ( PRON VP ) 1)

− ART does not match “I”,

backup state (b) used

3B

( ( PRON VP ) 1 ) − −

4 ( ( VP ) 2 ) − Consumed “I”

5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used

6 ( ( NP PP ) 3 ) − Consumed “saw”

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |

PRON

3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )

(b) ( ( PRON VP ) 1)

− ART does not match “I”,

backup state (b) used

3B

( ( PRON VP ) 1 ) − −

4 ( ( VP ) 2 ) − Consumed “I”

5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used

6 ( ( NP PP ) 3 ) − Consumed “saw”

7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )

(b) ( ( PRON PP ) 3 )

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |

PRON

3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )

(b) ( ( PRON VP ) 1)

− ART does not match “I”,

backup state (b) used

3B

( ( PRON VP ) 1 ) − −

4 ( ( VP ) 2 ) − Consumed “I”

5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used

6 ( ( NP PP ) 3 ) − Consumed “saw”

7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )

(b) ( ( PRON PP ) 3 )

8 ( ( N PP) 4 ) − Consumed “a”

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |

PRON

3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )

(b) ( ( PRON VP ) 1)

− ART does not match “I”,

backup state (b) used

3B

( ( PRON VP ) 1 ) − −

4 ( ( VP ) 2 ) − Consumed “I”

5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used

6 ( ( NP PP ) 3 ) − Consumed “saw”

7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )

(b) ( ( PRON PP ) 3 )

8 ( ( N PP) 4 ) − Consumed “a”

9 ( ( PP ) 5 ) − Consumed “boy”

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

2 ( ( NP VP ) 1 ) − − Use NP ART N | ART N PP |

PRON

3 ( ( ART N VP ) 1 ) (a) ( ( ART N PP VP ) 1 )

(b) ( ( PRON VP ) 1)

− ART does not match “I”,

backup state (b) used

3B

( ( PRON VP ) 1 ) − −

4 ( ( VP ) 2 ) − Consumed “I”

5 ( ( V NP PP ) 2 ) ( ( V NP ) 2 ) − Verb Attachment Rule used

6 ( ( NP PP ) 3 ) − Consumed “saw”

7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )

(b) ( ( PRON PP ) 3 )

8 ( ( N PP) 4 ) − Consumed “a”

9 ( ( PP ) 5 ) − Consumed “boy”

10

( ( P NP ) ) − −

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

… … … … …

7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )

(b) ( ( PRON PP ) 3 )

8 ( ( N PP) 4 ) − Consumed “a”

9 ( ( PP ) 5 ) − Consumed “boy”

10

( ( P NP ) 5 ) − −

11

( ( NP ) 6 ) − Consumed “with”

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

… … … … …

7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )

(b) ( ( PRON PP ) 3 )

8 ( ( N PP) 4 ) − Consumed “a”

9 ( ( PP ) 5 ) − Consumed “boy”

10

( ( P NP ) 5 ) − −

11

( ( NP ) 6 ) − Consumed “with”

12

( ( ART N ) 6 ) (a) ( ( ART N PP ) 6 )

(b) ( ( PRON ) 6)

−

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

… … … … …

7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )

(b) ( ( PRON PP ) 3 )

8 ( ( N PP) 4 ) − Consumed “a”

9 ( ( PP ) 5 ) − Consumed “boy”

10

( ( P NP ) 5 ) − −

11

( ( NP ) 6 ) − Consumed “with”

12

( ( ART N ) 6 ) (a) ( ( ART N PP ) 6 )

(b) ( ( PRON ) 6)

−

13

( ( N ) 7 ) − Consumed “a”

Top Down ParseState Backup State Action Comments

1 ( ( S ) 1 ) − − Use S NP VP

… … … … …

7 ( ( ART N PP ) 3 ) (a) ( ( ART N PP PP ) 3 )

(b) ( ( PRON PP ) 3 )

8 ( ( N PP) 4 ) − Consumed “a”

9 ( ( PP ) 5 ) − Consumed “boy”

10

( ( P NP ) 5 ) − −

11

( ( NP ) 6 ) − Consumed “with”

12

( ( ART N ) 6 ) (a) ( ( ART N PP ) 6 )

(b) ( ( PRON ) 6)

−

13

( ( N ) 7 ) − Consumed “a”

14

( ( − ) 8 ) − Consume “telescope”

Finish Parsing

Top Down Parsing - Observations

Top down parsing gave us the Verb Attachment Parse Tree (i.e., I used a telescope)

To obtain the alternate parse tree, the backup state in step 5 will have to be invoked

Is there an efficient way to obtain all parses ?

I saw a boy with a telescope1 2 3 4 5 6 7 8

Colour Scheme : Blue for Normal Parse Green for Verb Attachment Parse Purple for Noun Attachment Parse Red for Invalid Parse

Bottom Up Parse

I saw a boy with a telescope

1 2 3 4 5 6 7 8

Bottom Up Parse

NP12 PRON12

S1?NP12VP2?

I saw a boy with a telescope

1 2 3 4 5 6 7 8

Bottom Up Parse

NP12 PRON12

S1?NP12VP2? VP2?V23NP3?PP??

VP2?V23NP3?

I saw a boy with a telescope

1 2 3 4 5 6 7 8

Bottom Up Parse

NP12 PRON12

S1?NP12VP2? VP2?V23NP3?PP??

VP2?V23NP3? NP35 ART34N45

NP3?ART34N45PP5?

I saw a boy with a telescope

1 2 3 4 5 6 7 8

NP3?ART34N45PP5?

Bottom Up Parse

NP12 PRON12

S1?NP12VP2? VP2?V23NP3?PP??

VP2?V23NP3? NP35 ART34N45

NP3?ART34N45PP5?

NP35ART34N45

I saw a boy with a telescope

1 2 3 4 5 6 7 8

NP3?ART34N45PP5?

Bottom Up Parse

NP12 PRON12

S1?NP12VP2? VP2?V23NP3?PP??

VP2?V23NP3? NP35 ART34N45

NP3?ART34N45PP5?

NP35ART34N45

VP25V23NP35 S15NP12VP25

VP2?V23NP35PP5?

I saw a boy with a telescope

1 2 3 4 5 6 7 8

NP3?ART34N45PP5?

Bottom Up Parse

NP12 PRON12

S1?NP12VP2? VP2?V23NP3?PP??

VP2?V23NP3? NP35 ART34N45

NP3?ART34N45PP5?

PP5?P56NP6?NP35ART34N45

VP25V23NP35 S15NP12VP25

VP2?V23NP35PP5?

I saw a boy with a telescope

1 2 3 4 5 6 7 8

NP3?ART34N45PP5?

Bottom Up Parse

NP12 PRON12

S1?NP12VP2? VP2?V23NP3?PP??

VP2?V23NP3? NP35 ART34N45

NP3?ART34N45PP5?

PP5?P56NP6?NP35ART34N45 NP68ART67N7?

NP6?ART67N78PP8?

VP25V23NP35 S15NP12VP25

VP2?V23NP35PP5?

I saw a boy with a telescope

1 2 3 4 5 6 7 8

NP3?ART34N45PP5?

Bottom Up Parse

NP12 PRON12

S1?NP12VP2? VP2?V23NP3?PP??

VP2?V23NP3? NP35 ART34N45

NP3?ART34N45PP5?

PP5?P56NP6?NP35ART34N45 NP68ART67N7?

NP6?ART67N78PP8?

NP68ART67N78

VP25V23NP35 S15NP12VP25

VP2?V23NP35PP5?

I saw a boy with a telescope

1 2 3 4 5 6 7 8

NP3?ART34N45PP5?

Bottom Up Parse

NP12 PRON12

S1?NP12VP2? VP2?V23NP3?PP??

VP2?V23NP3? NP35 ART34N45

NP3?ART34N45PP5?

PP5?P56NP6?NP35ART34N45 NP68ART67N7?

NP6?ART67N78PP8?

NP68ART67N78

VP25V23NP35 S15NP12VP25

VP2?V23NP35PP5? PP58P56NP68

I saw a boy with a telescope

1 2 3 4 5 6 7 8

NP3?ART34N45PP5?

Bottom Up Parse

NP12 PRON12

S1?NP12VP2? VP2?V23NP3?PP??

VP2?V23NP3? NP35 ART34N45

NP3?ART34N45PP5?

PP5?P56NP6?NP35ART34N45 NP68ART67N7?

NP6?ART67N78PP8?

NP68ART67N78

VP25V23NP35 S15NP12VP25

VP2?V23NP35PP5? PP58P56NP68

NP38ART34N45PP58

I saw a boy with a telescope

1 2 3 4 5 6 7 8

NP3?ART34N45PP5?

Bottom Up Parse

NP12 PRON12

S1?NP12VP2? VP2?V23NP3?PP??

VP2?V23NP3? NP35 ART34N45

NP3?ART34N45PP5?

PP5?P56NP6?NP35ART34N45 NP68ART67N7?

NP6?ART67N78PP8?

NP68ART67N78

VP25V23NP35 S15NP12VP25

VP2?V23NP35PP5? PP58P56NP68

NP38ART34N45PP58

VP28V23NP38VP28V23NP35PP58

I saw a boy with a telescope

1 2 3 4 5 6 7 8

NP3?ART34N45PP5?

Bottom Up Parse

NP12 PRON12

S1?NP12VP2? VP2?V23NP3?PP??

VP2?V23NP3? NP35 ART34N45

NP3?ART34N45PP5?

PP5?P56NP6?NP35ART34N45 NP68ART67N7?

NP6?ART67N78PP8?

NP68ART67N78

VP25V23NP35 S15NP12VP25

VP2?V23NP35PP5? PP58P56NP68

NP38ART34N45PP58

VP28V23NP38VP28V23NP35PP58

S18NP12VP28

Bottom Up Parsing - Observations

Both Noun Attachment and Verb Attachment Parses obtained by simply systematically applying the rules

Numbers in subscript help in verifying the parse and getting chunks from the parse

Exercise

For the sentence,“The man saw the boy with a

telescope” & the grammar given previously, compare the performance of top-down, bottom-up & top-down chart parsing.

Verb Alternation (1/2) (ref: Natural Language Understanding, James Allan)

Verb ComplementStructure

Example

laugh Empty (in transitive) Ram laughed

find NP (transitive) Ram found the key

give NP+NP (di transitive) Ram gave Sita the paper

give NP+PP [to] Ram gave the paper to Sita

Reside Loc Phrase Ram resides in Mumbai

put NP+loc phrase Ram put the book inside the box

speak PP [with]+PP[about] Ram with Sita about floods

try VP[to] Ram tried to apologise

tell NP+VP[to] Ram told the man to go

Verb Alternation (1/2)Verb Complement

StructureExample

wish S [to] Ram wished for the man to go

keep VP [ing] Ram keeps hoping for the best

catch NP+VP [ing] Ram caught Shyam looking in his desk

Watch NP+VP [base] Ram watched Shyam eat the pizza

regret S [that] Ram regretted that he had eaten the whole thing

Tell NP+S [that] Ram told Sita that he was sorry

Seem ADJP Ram seems unhappy in his new job

Think NP+ADJP Ram thinks Sita is happy

Know S [wh] Ram knows where to go