earley’s algorithm (1970)

8/14/2019 Earleys Algorithm (1970)

1/89

600.465 - Intro to NLP - . Eisner 1

Earleys Algorithm (1970)

Nice combo of our parsing ideas so far:no restrictions on the form of the grammar:

A B C spoon D x incremental parsing (left to right, like humans)

left context constrains parsing of subsequentwordsso waste less time building impossible thingsmakes it faster than O(n 3) for many grammars


2/89


Overview of Earleys Algorithm

Finds constituents and partial constituents in inputA B C . D E is partial: only the first half of the A

A

B C D E

A B C . D E

D+ = A

B C D E

A B C D . E


3/89

600.465 - Intro to NLP - . Eisner

Overview of Earleys Algorithm

Proceeds incrementally , left-to-rightBefore it reads word , it has already built all hypothesesthat are consistent with first wordsReads word & attaches it to immediately precedinghypotheses. Might yield new constituents that are thenattached to hypotheses immediately preceding them

E.g., attaching D to A B C . D E gives A B C D . EAttaching E to that gives A B C D E .Now we have a complete A that we can attach tohypotheses immediately preceding the A, etc.


4/89


Our Usual Example Grammar

ROOT SS NP VP NP PapaNP Det N N caviar

NP NP PP N spoonVP VP PP V ate

VP V NP P withPP P NP Det theDet a


5/89


First Try: Recursive Descent

ROOT . SS . NP VP

NP . PapaNP Papa .

S NP . VP VP . VP PP

VP . VP PPVP . VP PP

VP . VP PP oops, stack overflowed

OK, lets pretend that didnt happen.Lets suppose we didnt see VP VP PP, and used VP V NP instead.

ROOT

SVP

VP PP NP

PapaV ateS NP VP VP V NP N caviar P withNP Det N PP P NP N spoon Det theNP NP PP Det a

Papa ate the 3 caviar with a spoo

goal stack


6/89



VP . V NP after dot = nonterminal, so recursively look for it ( predict ) V . ate after dot = terminal, so look for it in the input ( scan )V ate . after dot = nothing, so parents subgoal is completed ( attach )

VP V . NP predict (next subgoal) NP . ... do some more parsing and eventually ... NP ... . 7 we complete the parents NP subgoal, so attach

VP V NP . 7 attach againS NP VP . 7 attach again

ROOT

SVP

V NP NP

PapaV ateS NP VP VP VP PP N caviar P withNP Det N PP P NP N spoon Det theNP NP PP Det a


ROOT . S

S . NP VPNP . PapaNP Papa .

S NP . VP after dot = nonterminal, so recursively lookfor it ( predict )


7/89

600.465 - Intro to NLP - . Eisner


ROOT . SS . NP VP

NP . PapaNP Papa .

S NP . VP VP . V NP

V . ateV ate .

VP V . NP

NP . ... NP ... . 7

VP V NP . 7S NP VP . 7

ROOT

SVP

V NP NP


But how about the other parse

must backtrack to try predicting

a different VP rule here instead


implement by function calls:S() calls NP() and VP(), which recurse


8/89


9/89



VP . VP PP VP . VP PP VP . VP PP

oops, stack overflowedno fix after all

must transform grammar to eliminate left-recursiverules

ROOT

SVP

V NP NP



ROOT . S

S . NP VPNP . PapaNP Papa .

S NP . VP VP . VP PP

VP . VP PP


10/89600.465 - Intro to NLP - . Eisner 10

Use a Parse TableEarleys algorithm resembles recursive descent, but solves the left-recursion problem.No recursive function calls.Use a parse table as we did in CKY, so we can look up anything weve discovered so far.

Dynamic programming.

Entries in column look like (3, S NP . VP) (but well omit the etc. tosave space)Built while processing word Means that the input substring from 3 to matches the initial NP portion of a S NP VP ruleDot shows how much weve matched as of column Perfectly fine to have entries like (3, S is it . true that S)



Use a Parse Table

Entries in column look like (3, S NP . VP)What does it mean if we have this entry?

Unknown right context: Doesnt mean well necessarily beable to find a VP starting at column to complete the S.Known left context: Does mean that some dotted rule backin column 3 is looking for an S that starts at 3.

So if we actually do find a VP starting at column , allowing us tocomplete the S, then well be able to attach the S to something.And when that something is complete, it too will have a customer toits left just as in recursive descent!In short, a top-down (i.e., goal-directed) parser: it chooses to startbuilding a constituent not because of the input but because thatswhat the left context needs. In the spoon , wont build spoon as averb because theres no way to use a verb there.So any hypothesis in column could get used in the correct parse, if words - are continued in just the right way by words -n.



Operation of the Algorithm

Process all hypotheses one at a time in order.(Current hypothesis is shown in blue.)

This may add to the end of the to-do list, or try toadd again.new hypotheses

old hypothesesProcess a hypothesis according to whatfollows the dot just as in recursivedescent:

If a word, scan input and see if it matchesIf a nonterminal, predict ways to match it

(well predict blindly, but could reduce # of predictionsby looking ahead k symbols in the input and onlymaking predictions that are compatible with this

limited right context )


13/89

0 ROOT . S

initialize

Remember this stands for ( , ROOT . S)


14/89

0 ROOT . S

S . NP VP

predict the kind of S we are looking for

Remember this stands for ( , S . NP VP)


15/89

0 ROOT . S

S . NP VP

NP . Det

N NP . NPPP

NP . Papa

predict the kind of NP we are looking for(actually well look for 3 kinds: any of the 3 will do


16/89

0 ROOT . S

S . NP VP

NP . Det

N NP . NPPP

NP . Papa

Det . the

Det . a

predict the kind of Det we are looking for (2 kind


17/89

0 ROOT . S

S . NP VP

NP . Det

N NP . NPPP

NP . Papa

Det . the

Det . a

predict the kind of NP were looking for but we were already looking for these sodont add duplicate goals! Note that this happenewhen we were processing a left-recursive rule.


18/89

0 Papa1

ROOT . S NP Papa .

S . NP VP

NP . DetN

NP . NPPP

NP . Papa

Det . the

Det . a

scan : the desired word is in the input!


19/89

0 Papa1

ROOT . S NP Papa .

S . NP VP

NP . DetN

NP . NPPP

NP . Papa

Det . the

Det . a

scan : failure


20/89


21/89

0 Papa1

ROOT . S NP Papa .

S . NP VP S NP . VP

NP . DetN NP NP .PP

NP . NPPP

NP . Papa

Det . the

Det . a

attach the newly created NP(which starts at ) to its customers (incomplete constituents that end at and have NP after the dot)


22/89

0 Papa1

ROOT . S NP Papa .

S . NP VP S NP . VP

NP . DetN NP NP .PP

NP . NPPP

VP . V NP

NP . Papa VP . VPPP

Det . the

Det . a

predict


23/89


24/89

0 Papa1

ROOT . S NP Papa .

S . NP VP S NP . VP

NP . DetN NP NP .PP

NP . NPPP

VP . V NP

NP . Papa VP . VPPP

Det . the PP . P NP

Det . a V . ate

predict


25/89

0 Papa1

ROOT . S NP Papa .

S . NP VP S NP . VP

NP . DetN NP NP .PP

NP . NPPP

VP . V NP

NP . Papa VP . VPPP

Det . the PP . P NP

Det . a V . ate

predict


26/89


27/89

0 Papa 1 ate2

ROOT . S NP Papa . V ate .

S . NP VP S NP . VP

NP . DetN NP NP .PP

NP . NPPP

VP . V NP

NP . Papa VP . VPPP

Det . the PP . P NP

Det . a V . ate

P . with

scan : success!


28/89

0 Papa 1 ate2


S . NP VP S NP . VP

NP . DetN NP NP .PP

NP . NPPP

VP . V NP

NP . Papa VP . VPPP

Det . the PP . P NP

Det . a V . ate

P . with

scan : failure


29/89

0 Papa 1 ate2


S . NP VP S NP . VP VP V . NP

NP . DetN NP NP .PP

NP . NPPP

VP . V NP

NP . Papa VP . VPPP

Det . the PP . P NP

Det . a V . ate

P . with

attach


30/89


31/89

0 Papa 1 ate2



NP . DetN NP NP .PP NP . DetN

NP . NPPP

VP . V NP NP . NPPP

NP . Papa VP . VPPP

NP . Papa

Det . the PP . P NP Det . the

Det . a V . ate Det . a

P . with

predict (these next few stepsshould look familiar)


32/89

0 Papa 1 ate2




NP . NPPP

VP . V NP NP . NPPP

NP . Papa VP . VPPP

NP . Papa



P . with

predict


33/89

0 Papa 1 ate2




NP . NPPP

VP . V NP NP . NPPP

NP . Papa VP . VPPP

NP . Papa



P . with

scan (this time we fail sincePapa is not the next word)


34/89

0 Papa 1 ate 2 the 3

ROOT . S NP Papa . V ate . Det the .



NP . NPPP

VP . V NP NP . NPPP

NP . Papa VP . VPPP

NP . Papa



P . with

scan: success!


35/89

0 Papa 1 ate 2 the 3




NP . NPPP

VP . V NP NP . NPPP

NP . Papa VP . VPPP

NP . Papa



P . with


36/89


37/89

0 Papa 1 ate 2the 3


S . NP VP S NP . VP VP V . NP NP Det .N

NP . DetN NP NP .PP NP . DetN 3 N . caviar

NP . NPPP

VP . V NP NP . NPPP

3 N . spoon

NP . Papa VP . VP

PP

NP . Papa



P . with


38/89

0 Papa 1 ate 2 the3 caviar 4

ROOT . S NP Papa . V ate . Det the . 3 N caviar .


NP . DetN NP NP .PP NP . DetN 3 N . caviar

NP . NPPP

VP . V NP NP . NPPP

3 N . spoon

NP . Papa VP . VP

PP

NP . Papa



P . with


39/89


40/89


41/89




NP Det N.

NP . DetN NP NP .PP NP . DetN 3 N . caviar VP V NP .

NP . NPPP

VP . V NP NP . NPPP

3 N . spoon NP NP .PP

NP . Papa VP . VP

PP

NP . Papa



P . with

attach(again!)


42/89


43/89




NP Det N.


NP . NPPP

VP . V NP NP . NPPP


NP . Papa VP . VP

PP

NP . Papa S NP VP .

Det . the PP . P NP Det . the VP VP .PP

Det . a V . ate Det . a PP . P NP

P . with


44/89




NP Det N.


NP . NPPP

VP . V NP NP . NPPP


NP . Papa VP . VP

PP

NP . Papa S NP VP .



P . with ROOT S .

attach(again!)


45/89




NP Det N.


NP . NPPP

VP . V NP NP . NPPP


NP . Papa VP . VP

PP

NP . Papa S NP VP .



P . with ROOT S .


46/89




NP Det N.


NP . NPPP

VP . V NP NP . NPPP


NP . Papa VP . VP

PP

NP . Papa S NP VP .



P . with ROOT S .


47/89




NP Det N.


NP . NPPP

VP . V NP NP . NPPP


NP . Papa VP . VP

PP

NP . Papa S NP VP .



P . with ROOT S .


48/89

0 Papa 1 ate 2 the 3caviar 4 with 5

ROOT . S NP Papa . V ate . Det the . 3 N caviar . P with .


NP Det N.


NP . NPPP

VP . V NP NP . NPPP


NP . Papa VP . VP

PP

NP . Papa S NP VP .



P . with ROOT S .


49/89




NP Det N.

PP P . NP


NP . NPPP

VP . V NP NP . NPPP


NP . Papa VP . VP

PP

NP . Papa S NP VP .



P . with ROOT S .


50/89


51/89




NP Det N.

PP P . NP

NP . DetN NP NP .PP NP . DetN 3 N . caviar VP V NP . NP . DetN

NP . NPPP

VP . V NP NP . NPPP


NP . NPPP

NP . Papa VP . VP

PP

NP . Papa S NP VP . NP . Papa


Det . the

Det . a V . ate Det . a PP . P NP Det . a

P . with ROOT S .


52/89




NP Det N.

PP P . NP


NP . NPPP

VP . V NP NP . NPPP


NP . NPPP

NP . Papa VP . VP

PP



Det . the


P . with ROOT S .


53/89




NP Det N.

PP P . NP


NP . NPPP

VP . V NP NP . NPPP


NP . NPPP

NP . Papa VP . VP

PP



Det . the


P . with ROOT S .


54/89


55/89

1 ate 2 the 3 caviar 4a 6

a . V ate . Det the . 3 N caviar . P with . Det a .

P VP V . NP NP Det .N

NP Det N.

PP P . NP

NP . DetN 3 N . caviar VP V NP . NP . DetN

P NP . NPPP


NP . NPPP


P Det . the VP VP .PP

Det . the

Det . a PP . P NP Det . a

ROOT S .


56/89




NP Det N.

PP P . NP NP Det .N

NP . DetN 3 N . caviar VP V NP . NP . DetN

P NP . NPPP


NP . NPPP



Det . the


ROOT S .


57/89




NP Det N.

PP P . NP NP Det .N

NP . DetN 3 N . caviar VP V NP . NP . DetN N . caviar

P NP . NPPP


NP . NPPP

N . spoon



Det . the


ROOT S .


58/89




NP Det N.

PP P . NP NP Det .N


P NP . NPPP


NP . NPPP

N . spoon



Det . the


ROOT S .


59/89

1 ate 2 the 3 caviar 4 withspoon 7

a . V ate . Det the . 3 N caviar . P with . Det a . N spoon .


NP Det N.

PP P . NP NP Det .N


P NP . NPPP


NP . NPPP

N . spoon



Det . the


ROOT S .


60/89

1 ate 2 the 3 caviar 4 withspoon 7

a . V ate . Det the . 3 N caviar . P with . Det a . N spoon .


NP Det N.

PP P . NP NP Det .N

NP Det N.


P NP . NPPP


NP . NPPP

N . spoon



Det . the


ROOT S .


61/89


62/89

0 Papa 1 ate 2 the 3aviar 4 with a spoon 7

ROOT . S NP Papa . V ate . Det the . 3 N caviar . N spoon .


NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN

3 N . caviar VP V NP . PP P NP .

NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa VP . VP

PP

NP . Papa S NP VP . NP NP PP

. Det . the PP . P NP Det . the VP VP .

PP VP VPPP .


P . with ROOT S .


63/89




NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN


NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa VP . VP

PP



PP VP VPPP .

Det . a V . ate Det . a PP . P NP 7 PP . P NP

P . with ROOT S .


64/89




NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN


NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa VP . VP

PP



PP VP VPPP .


P . with ROOT S . VP V NP .


65/89




NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN


NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa VP . VP

PP



PP VP VPPP .




66/89




NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN


NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa VP . VP

PP



PP VP VPPP .




67/89


68/89




NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN


NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa VP . VP

PP



PP VP VPPP .




69/89




NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN


NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa VP . VP

PP



PP VP VPPP .




70/89




NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN


NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa VP . VP

PP



PP VP VPPP .




71/89




NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN


NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa VP . VP

PP



PP VP VPPP .




72/89


73/89




NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN


NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa VP . VP

PP



PP VP VPPP .



Left Recursion Kills Pure


74/89

. - Intro to NLP - . Eisner 7

Top-Down Parsing

VP



75/89


Top-Down Parsing

VP

VP PP



76/89


Top-Down Parsing

VP PP

VP

VP PP



77/89


Top-Down Parsing

VP PP

VP

VP PP

VP PP

makes new hypothesesad infinitum before weveseen the PPs at all

hypotheses try to predictin advance how manyPPs will arrive in input


78/89


but Earleys Alg is Okay!VP

PPVP

VP . VP PP

(in column 1)


79/89


but Earleys Alg is Okay!

VP

V NP VP V NP .

ate the caviar

VP

PPVP

VP . VP PP

(in column 1)

(in column 4)


80/89

. - Intro to NLP - . Eisner


VP

V NP

VP

PPVP

V NP

attach

ate the caviar

VP VP . PP

VP

PPVP

VP . VP PP

(in column 1)

(in column 4)


81/89



VP

V NP

VP

PPVP

V NPate the caviar

with a spoon

VP VP PP .

VP

PPVP

VP . VP PP

(in column 1)

(in column 7)


82/89


83/89



VP

V NP

VP

PPVP

V NP

VP

PP

VP VP . PP

ate the caviar

with a spoon

VP

PPVP

VP . VP PP

can be reused(in column 1)

(in column 7)

attach


84/89


85/89



VP

V NP

VP

PPVP

V NP

VP

PP

ate the caviar

with a spoon

in his bed

VP VP PP .

VP

PPVP

VP . VP PP

can be reused again(in column 1)

(in column 10)


86/89



VP

V NP

VP

PPVP

V NP

VP

PPVP

VP . VP PP

can be reused again

VP

PP

VP

PP

VP VP . PP

ate the caviar

with a spoon

in his bed

(in column 1)

(in column 10)

attach


87/89




NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN


NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa 1 VP . VP

PP


. Det . the PP . P NP Det . the 1 VP VP .

PP VP VPPP .



o mpleted a VP in col ol lets us use it in a VP PP structure


88/89




NP DetN .

NP Det N.

NP . DetN

NP NP .PP

NP . DetN


NP . NPPP

VP . V NP NP . NPPP


NP NP .PP

NP . Papa 1 VP . VP

PP



PP VP VPPP .



o mpleted that VP = VP PP in col 7ol would let us use it in a VP PP structurean reuse col as often as we need


89/89

earley’s algorithm (1970)

Documents