l10
DESCRIPTION
l10TRANSCRIPT
CFG [1]
Language of a Grammar
If G is a grammar we write
L(G) = { w∈ T∗ | S ⇒∗ w }
Definition: A language L is context-free iff there is a grammar G such that L= L(G)
start symbol corresponds to start state
variable symbols correspond to states
terminal symbols T correspond to the alphabet Σ
1
CFG [2]
Context-Free Languages and Regular Languages
Theorem: If L is regular then L is context-free.
Proof: We know L = L(A) for a DFA A. From A we can build a CFG Gsuch that L(A) = L(G)
The variables are A,B,C, with start symbol A, the terminal token are 0, 1and the productions are
A→ 1A | 0B B → 0B | 1C C → ε | 0B | 1C
2
CFG [3]
Context-Free Languages and Regular Languages
Let LX be the language generated by the grammar with X as a start symbolwe prove (mutual induction!) that w ∈ LX iff δ̂(X,w) = C by induction on |w|
Such a CFG is called right regular
It would be possible also to define L by a left regular language with start stateC
A→ ε | C1 | A1 B → A0 | C0 | B0 C → B1
The intuition here is that LX represents the path from A to X
3
CFG [4]
Example of a derivation
Given the grammar for english above we can generate (leftmost derivation)
SENTENCE ⇒ SUBJECT VERB OBJECT
⇒ ARTICLE NOUN VERB OBJECT ⇒ the NOUN VERB OBJECT
⇒ the NOUN VERB OBJECT ⇒ the cat VERB OBJECT
⇒ the cat caught OBJECT ⇒ the cat caught ARTICLE NOUN
⇒ the cat caught a NOUN ⇒ the cat caught a dog
4
CFG [5]
Derivation Tree
Notice that the following generation is possible (rightmost derivation)
SENTENCE ⇒ SUBJECT VERB OBJECT
⇒ SUBJECT VERB ARTICLE NOUN
⇒ SUBJECT VERB ARTICLE dog
⇒ SUBJECT VERB a dog ⇒ SUBJECT caught a dog
⇒ ARTICLE NOUN caught a dog ⇒ ARTICLE cat caught a dog
⇒ the cat caught a dog
5
CFG [6]
Derivation Tree
Both generation corresponds to the same derivation tree or parse tree whichreflects the internal structure of the sentence
Number of left derivations of one word
= number of right derivations
= number of parse trees
6
CFG [7]
A grammar for arithmetical expressions
S → (S) | S + S | S × S | I
I → 1 | 2 | 3
The terminal symbols are {(, ),+,×, 1, 2, 3}
The variable symbols are S and I
7
CFG [8]
Ambiguity
Definition: A grammar G is ambiguous iff there is some word in L(G) whichhas two distinct derivation trees
Intuitively, there are two possible meaning of this word
Example: the previous grammar for arithmetical expression is ambigiuoussince the word 2 + 1× 3 has two possible parse trees
8
CFG [9]
Ambiguity
An example of ambiguity in programming language is else with the followingproduction
C → if b then C else C
C → if b then C
C → s
9
CFG [10]
Ambiguity
A word like
if b then if b then s else s
can be interpreted as
if b then (if b then s else s)
or
if b then (if b then s) else s
10
CFG [11]
Context-Free Languages and Inductive Definitions
Each CFG can be seen as an inductive definition
For instance the grammar for arithmetical expression can be seen as thefollowing inductive definition
• 1, 2, 3 are arithmetical expressions
• if w is an arithmetical expression then so is (w)
• if w1, w2 are arithmetical expressions then so are w1 + w2 and w1 × w2
A natural way to do proofs on context-free languages is to follow this inductivestructure
11
CFG [12]
Context-Free Languages and Regular Languages
The following language L = {anbn | n > 1} is context-free
We know that it is not regular
Proposition: The following grammar G generates L
S → ab | aSb
12
CFG [13]
Context-Free Languages and Regular Languages
We prove w ∈ L(G) implies w ∈ L by induction on the derivation of w ∈ L(G)
• ab ∈ L(G)
• if w ∈ L(G) then awb ∈ L(G)
We can prove also w ∈ L(G) implies w ∈ L by induction on the length of aderivation S ⇒∗ w
We prove anbn ∈ L(G) by induction on n
13
CFG [14]
Abstract Syntax
The parse tree has often too much information w.r.t. the internal structure ofa document. This structure is best reflected by an abstract syntax tree. We giveonly an example here.
Here is BNF for arithmetic expression
E → E + E | E × E | (E) | I I → 1 | 2 | 3
Parse tree for 2 + (1× 3)
14
CFG [15]
Abstract Syntax
This can be compared with the abstract syntax tree for the expression 2+(1×3)
Concrete syntax describes the way documents are written while abstract syntaxdescribes the pure structure of a document.
15
CFG [16]
Abstract Syntax
In Haskell, use of data types for abstract syntax
data Exp = Plus Exp Exp | Times Exp Exp | Num Atom
data Atom = One | Two | Three
ex = Plus Two (Times One Three)
16
CFG [17]
Ambiguity
Definition: A grammar G is ambiguous iff there is some word in L(G) whichhas two distinct derivation trees
Intuitively, there are two possible meaning of this word
Example: the previous grammar for arithmetical expression is ambigiuoussince the word 2 + 1× 3 has two possible parse trees
17
CFG [18]
Ambiguity
Let Σ be {0, 1}.
The following grammar of parenthesis expressions is ambiguous
E → ε | EE | 0E1
18
CFG [19]
A simple example
Σ = {0, 1}
L = {uuR | u ∈ Σ∗}
This language is not regular: using the pumping lemma on 0k10k
L = L(G) for the grammar
S → ε | 0S0 | 1S1
We prove that if S ⇒∗ v then v ∈ L by induction on the length of S ⇒∗ v
We prove uuR ∈ L(G) if u ∈ Σ∗ by induction on the length of u
19
CFG [20]
A simple example
Theorem: The grammar for S is not ambiguous
Proof: By induction on |v| we prove that there is at most one productionS ⇒∗ v
20
CFG [21]
Polish notation
The following is a grammar for arithmetical expressions
E → ∗EE | + EE | I, I → a | b
Theorem: This grammar is not ambiguous
We show by induction on |u| the following.
Lemma: for any k there is at most one leftmost derivation of Ek ⇒∗ u
21
CFG [22]
Polish notation
The proof is by induction on |u|. If |u| = n + 1 with n > 1 there are threecases
(1) u = +v then the derivation has to be of the form
Ek ⇒ +EEEk−1 ⇒∗ +v
for a derivation Ek+1 ⇒∗ v and we conclude by induction hypothesis
(2) u = ∗v then the derivation has to be of the form
Ek ⇒ ∗EEEk−1 ⇒∗ ∗v
for a derivation Ek+1 ⇒∗ v and we conclude by induction hypothesis
22
CFG [23]
Polish notation
(3) u = iv with i = a or i = b, then the derivation has to be of the form
Ek ⇒ iEk−1 ⇒∗ iv
for a derivation Ek−1 ⇒∗ v and we conclude by induction hypothesis
23
CFG [24]
Polish notation
It follows from this result that we have the following property.
Corollary: If ∗u1u2 = ∗v1v2 ∈ L(E) then u1 = v1 and u2 = v2. Similarly if+u1u2 = +v1v2 ∈ L(E) then u1 = v1 and u2 = v2.
but the result says also that if u ∈ L(E) then there is a unique parse tree foru.
24
CFG [25]
Ambiguity
Now, a more complicated example. Let Σ be {0, 1}.
The following grammar of parenthesis expressions is ambiguous
E → ε | EE | 0E1
We replace this by the following equivalent grammar
S → 0S1S | ε
Lemma: L(S) = L(E)
Theorem: The grammar for S is not ambiguous
25
CFG [26]
Ambiguity
Lemma: L(S)L(S) ⊆ L(S)
Proof: we prove that if u ∈ L(S) then uL(S) ⊆ L(S) by induction on |u|
If u = ε then uL(S) = L(S)
If |u| = n+1 then u = 0v1w with v, w ∈ L(S) and |v|, |w| 6 n. By inductionhypothesis, we have wL(S) ⊆ L(S) and so
uL(S) = 0v1wL(S) ⊆ 0v1L(S) ⊆ L(S)
since v ∈ L(S) and 0L(S)1L(S) ⊆ L(S) Q.E.D.
26
CFG [27]
Ambiguity
We can also do an induction on the length of a derivation S ⇒∗ u
Using this lemma, we can show L(E) ⊆ L(S)
27
CFG [28]
Ambiguity
Lemma: L(E) ⊆ L(S)
Proof: We prove that if u ∈ L(E) then u ∈ L(S) by induction on the lengthof a derivation E ⇒∗ u
If E ⇒ ε = u then u ∈ L(S)
If E ⇒ EE ⇒∗ vw = u then by induction v, w ∈ L(S) and by the previousLemma we have u ∈ L(S)
If E ⇒ 0E1 ⇒∗ 0v1 = u then by induction v ∈ L(S) and so u = 0v1ε ∈ L(S).Q.E.D.
28
CFG [29]
Ambiguity
The proof that the grammar for S is not ambiguous is difficult
One first tries to show that there is at most one left-most derivation
S ⇒∗lm u
for any string u ∈ Σ∗
If u is not ε we have that u should be 0u1 and then the derivation should be
S ⇒ 0S1S ⇒ 0u1
with S1S ⇒ u1
29
CFG [30]
Ambiguity
This suggests the following statement ψ(u) to be proved by induction on thelength of u
For any k there exists at most one leftmost derivation S(1S)k ⇒∗ u
We can then prove ψ(u) by induction on |u|
If u = ε we should have k = 0 and the derivation has to be S ⇒ ε
30
CFG [31]
Ambiguity
If ψ(v) holds for |v| = n and |u| = n+ 1 then u = 0v or u = 1v with |v| = n.We have two cases
(1) u = 1v and S(1S)k ⇒∗ 1v, the derivation has the form
S(1S)k ⇒ ε(1S)k ⇒∗ 1v
for a derivation S(1S)k−1 ⇒∗ v and we conclude by induction hypothesis
(2) u = 0v and S(1S)k ⇒∗ 0v, the derivation has the form
S(1S)k ⇒ 0S1S(1S)k ⇒∗ 0v
for a derivation S(1S)k+1 ⇒∗ v and we conclude by induction hypothesis
31
CFG [32]
Inherent Ambiguity
There exists a context-free language L such that for any grammar G ifL = L(G) then G is ambiguous
L = {anbncmdm | n,m > 1} ∪ {anbmcmdn | n,m > 1}
L is context-free
S → AB | C A→ aAb | ab
B → cBd | cd C → aCd | aDd D → bDc | bc
32
CFG [33]
Eliminating ε- and unit productions
Definition: A unit production is a production of the form A→ B with A,Bnon terminal symbols.
This is similar to ε-transitions in a ε-NFA
Definition: A ε-production is a production of the form A→ ε
Theorem: For any CFG G there exists a CFG G′ with no ε- or unit productionssuch that L(G′) = L(G)− {ε}
33
CFG [34]
Elimination of unit productions
Let P1 be a system of productions such that if A→ B and B → β are in P1
then so is A→ β and G1 = (V, T, P1, S).
Let P2 the set of non unit productions of P1 and G2 = (V, T, P2, S)
Theorem: L(G1) = L(G2)
34
CFG [35]
Elimination of unit productions
Proof: If u ∈ L(G1) and S ⇒∗ u is a derivation of minimal length then thisderivation is in G2. Otherwise it has the shape
S ⇒∗ α1Aα2 ⇒ α1Bα2 ⇒n β1Bβ2 ⇒ β1ββ2 ⇒∗ u
and we have a shorter derivation
S ⇒∗ α1Aα2 ⇒n β1Aβ2 ⇒ β1ββ2 ⇒∗ u
contradiction.
35
CFG [36]
Elimination of unit productions
S → CBh | D
A→ aaC
B → Sf | ggg
C → cA | d | C
D → E | SABC
E → be
36
CFG [37]
Elimination of unit productions
We eliminate unit productions
S → SABC | be | CBh
A→ aaC
B → Sf | ggg
C → cA | d
37
CFG [38]
Elimination of ε-productions
If G = (V, T, P, S) build the new system P1 closing P by adding rules
If A→ αBβ and B → ε then A→ αβ
We have L(G1) = L(G). Let P2 the system obtained from P1 by taking awayall ε-productions
Theorem: L(G2) = L(G)− {ε}
Proof: We clearly have L(G2) ⊆ L(G1). We prove that if S ⇒∗ u, u ∈ T ∗
and u 6= ε is a production of minimal length then it does not use any ε-production,so it is a derivation in G2. Q.E.D.
38
CFG [39]
Eliminating ε- and unit productions
Starting from G = (V, T, P, S) we build a larger set P1 of productionscontaining P and closed under the two rules
1. if A→ w1Bw2 and B → ε are in P1 then A→ w1w2 is in P1
2. if A→ B and B → w are in P1 then so is A→ w
We add only productions whose right-handside is a subtring of an old right-handside, so this process stops.
It can be shown that if L(V, T, P1, S) = L(G) and that if P ′ is the set ofproductions in P1 that are not ε- neither unit production then L(V, T, P ′, S) =L(G)− {ε}
39
CFG [40]
Eliminating ε- and unit productions
Example: If we start from the grammar
S → aSb | SS | ε
we get first the new productions
S → ab | S | S
and if we eliminate the ε- and unit productions we get
S → aSb | SS | ab
40
CFG [41]
Eliminating ε- and unit productions
Example: If we start from the grammar
S → AB A→ aAA | ε B → bBB | ε
we get first the new productions
S → A | B A→ aA | a B → bB | b
and if we eliminate the ε- and unit productions we get
S → AB | aAA | aA | a | bB | b A→ aAA | aA | a B → bBB | bB | b
41
CFG [42]
Eliminating Useless Symbols
A symbol X is useful if there is some derivation S ⇒∗ αXβ ⇒∗ w where wis in T ∗
X can be in V or T
X is useless iff it is not useful
X is generating iff X ⇒∗ w for some w in T ∗
X is reacheable iff S ⇒∗ αXβ for some α, β
42
CFG [43]
Reachable Symbols
By analogy with accessible states, we can define accessible or reachablesymbols. We give an inductive definition
• BASIS: The start symbol S is reachable
• INDUCTION: If A is reachable and A→ w is a production, then all symbolsoccuring in w are reachable.
43
CFG [44]
Reachable Symbols
Example: Consider the following CFG
S → aB | BC A→ aA | c | aDb
B → DB | C C → b D → B
Then s is accessible, hence also B and C, and hence D is accessible.
But A is not accessible.
We can take away A from this grammar and we get the same language
S → aB | BC B → DB | C C → b D → B
44
CFG [45]
Generating Symbols
We define when an element of V ∪ T (terminal or non terminal symbols) isgenerating by an inductive definition
• BASIS: all elements of T are generating
• INDUCTION: if there is a production X → w where all symbols occuring inw are generating then X is generating
This gives exactly the generating variables
45
CFG [46]
Generating Symbols
Example: We consider
S → aS | W | U W → aW
U → a V → aa
Then U, V are generating because U → a V → aa
Hence S is generating because S → U
W is not generating, we have only W → aW for production for W
46
CFG [47]
Eliminating Useless Symbols
To eliminate useless symbols in a grammar G, first eliminate all nongeneratingsymbols we get an equivalent grammar G1 and then eliminate all symbols in G1
that are non reachable.
We get a grammar G2 that is equivalent to G1 and to G
We have to do this in this order
Examples: For the grammar
S → AB | a A→ b
47
CFG [48]
B is not generating, we get the grammar
S → a A→ b
and then A is not reachable we get the grammar
S → a
48
CFG [49]
Elimination of useless variables
S → gAe | aY B | CY, A→ bBY | ooC
B → dd | D, C → jV B | gi
D → n, U → kW
V → baXXX | oV, W → c
X → fV, Y → Y hm
49
CFG [50]
Elimination of useless variables
Simplified grammar
S → gAe
A→ ooC
C → gi
50
CFG [51]
Linear production systems
Several algorithms we have seen are instances of graph searchingalgorithm/derivability in linear production systems
51
CFG [52]
Linear Production systems
For testing for accessibility, for the grammar
S → aB | BC, A→ aA | c | aDb
B → DB | C, C → b | B
we associate the production system
→ S, S → B, S → C
A→ A, A→ D, B → B
B → D, B → C, C → B
and we can produce S,B,D,C
52
CFG [53]
Linear Production systems
A lot of problems in elementary logic are of this form
A→ B, B → C, A,C → D
What can we deduce from A?
53
CFG [54]
Linear Production systems
For computing generating symbols we have a more general form of productionsystem
For instance for the grammar
A→ ABC, A→ C, B → Ca, C → a
we can associate the following production system
A,B,C → A, C → A, C → B, → C
and we can produce C,B,A. There is an algorithm for this kind of problemin 7.4.3
54
CFG [55]
Chomsky Normal Form
Definition: A CFG is in Chomsky Normal Form (CNF) iff all productions areof the form A→ BC or A→ a
Theorem: For any CFG G there is a CFG G′ in Chomsky Normal Form suchthat L(G′) = L(G)− {ε}
55
CFG [56]
Chomsky Normal Form
We can assume that G has no ε- or unit productions. For each terminal a weintroduce a new nonterminal Aa with the production
Aa → a
We can then assume that all productions are of the form A → a or A →B1B2 . . . Bk with k > 2
If k > 2 we introduce C with productions A→ B1C and C → B2 . . . Bk untilwe have only right-hand sides of length 6 2
56
CFG [57]
Chomsky Normal Form
Example: For the grammar
S → aSb | SS | ab
we get firstS → ASB | SS | AB A→ a B → b
and thenS → AC | SS | AB A→ a B → b C → SB
which is in Chomsky Normal Form
57
CFG [58]
The Chomsky Hierarchy
Noam Chomsky 1956
Four types of grammars
Type 0: no restrictions
Type 1: Context-sensitive, rules αAβ → αγβ
Type 2: Context-free or context-insensitive
Type 3: Regular, rules of the form A→ Ba or A→ aB or A→ ε
Type 3 ⊆ Type 2 ⊆ Type 1 ⊆ Type 0
Grammars for programming languages are usually Type 2
58
CFG [59]
Context-Free Languages and Regular Languages
Theorem: If L is regular then L is context-free.
Proof: We know L = L(A) for a DFA A. We have A = (Q,Σ, δ, q0, F ).We define a CFG G = (Q,Σ, P, q0) where P is the set of productions q → aq′
if δ(q, a) = q′ and q → ε if q ∈ F . We have then q ⇒∗ uq′ iff δ̂(q, u) = q′ andq ⇒∗ u iff δ̂(q, u) ∈ F . In particular u ∈ L(G) iff u ∈ L(A).
A grammar where all productions are of the form A→ aB or A→ ε is calledleft regular
59
CFG [60]
Pumping Lemma for Left Regular Languages
Let G = (V, T, P, S) be a left regular language, and let N be |V |.
If a1 . . . ar is a string of length ≥ N any derivation
S ⇒ a1B1 ⇒ a1a2B2 ⇒ · · · ⇒ a1 . . . aiA
⇒ · · · ⇒ a1 . . . ajA⇒ · · · ⇒ a1 . . . an
has length n and there is at least one variable A which is used twice (pigeon-holeprinciple)
If x = a1 . . . ai and y = ai+1 . . . aj and z = aj+1 . . . an we have |xy| 6 N andxykz ∈ L(G) for all k
60
CFG [61]
Pumping Lemma for Context-Free Languages
Let L be a context-free language
Theorem: There exists N such that if z ∈ L and N 6 |z| then one can writez = uvwxy such that
z = uvwxy, |vx| > 0, |vwx| 6 N, uvkwxky ∈ L for all k
61
CFG [62]
Pumping Lemma for Context-Free Languages
Theorem: The language {akbkck | k > 0} is not context-free
Proof: Assume L to be context-free. Then we have N as stated in thePumping Lemma. Consider z = aNbNcN . We have N 6 |z| so we can writez = uvwxy such that
z = uvwxy, |vx| > 0, |vwx| 6 N, uvkwxky ∈ L for all k
Since |vwx| 6 N there is one letter d ∈ {a, b, c} that occurs not in vwx, andsince |vx| > 0 there is another letter e 6= d that occurs in vx. Then e has moreoccurence than d in uv2wx2y, and this contradicts uv2wx2y ∈ L. Q.E.D.
62
CFG [63]
Proof of the CFL Pumping Lemma
We can assume that the language is presented by a grammar in ChomskyNormal Form, working with L− {ε}
The crucial remark is that a binary tree with height p + 1 has at most 2p
leaves
The height of a binary tree is the number of nodes from the root to thelongest path
63
CFG [64]
Proof of the CFL Pumping Lemma
Example: the Chomsky grammar
S → AC | AB, A→ a, B → b, C → SB
consider a parse tree for a4b4 corresponding to the derivation
S ⇒ AC ⇒ aC ⇒ aSB ⇒ aACB ⇒ aaCB ⇒ aaSBB
⇒ a2ACBB ⇒ a3CBB ⇒ a3SBBB ⇒ a3ABBBB ⇒ a4BBBB
⇒ a4BBBB ⇒ a4bBBB ⇒ a4b2BB ⇒ a4b3B ⇒ a4b4
The symbol S appears twice on a path u = aa, v = a, w = ab, x = b, y = bb
64
CFG [65]
Non closure under intersection
T = {a, b, c}
L1 = {akbkcm | k,m > 0}
L2 = {ambkck | k,m > 0}
L1 and L2 are CFL, but the intersection
L1 ∩ L2 = {akbkck | k > 0}
is not CF
65
CFG [66]
Non closure under intersection
However one can show (we will not do the proof in this course, but you shouldknow the result)
Theorem: If L1 ⊆ Σ∗ is context-free and L2 ⊆ Σ∗ is regular then L1 ∩ L2 iscontext-free
Application: The following language, for Σ = {0, 1}
L = {uu | u ∈ Σ∗}
is not context-free, by considering the intersection with L(0∗1∗0∗1∗)
One can show that the complement of L is context-free!
66
CFG [67]
Closure under union
If L1 = L(G1) and L2 = L(G2) with disjoint set of variables V1 and V2, andsame alphabet T , we can define
G = (V1 ∪ V2 ∪ {S}, T, P1 ∪ P2 ∪ {S → S1 | S2}, S)
It is then direct to show that L(G) = L(G1) ∪ L(G2) since a derivation hasthe form
S ⇒ S1 ⇒∗ u
orS ⇒ S2 ⇒∗ u
67
CFG [68]
Non-Closure Under Complement
L1 ∩ L2 = L1 ∪ L2
So CFL cannot be closed under complement in general. Otherwise they wouldbe closed under intersection.
68
CFG [69]
Closure Under Concatenation
If L1 = L(G1) and L2 = L(G2) with disjoint set of variables V1 and V2, andsame alphabet T , we can define
G = (V1 ∪ V2 ∪ {S}, T, P1 ∪ P2 ∪ {S → S1S2}, S)
It is then direct to show that L(G) = L(G1)L(G2) since a derivation has theform
S ⇒ S1S2 ⇒∗ u1u2
withS1 ⇒∗ u1, S2 ⇒∗ u2
69
CFG [70]
LL(1) parsing
A grammar is LL(1) if in a sequence of leftmost production we can decidewhat is the production looking only at the first symbol of the string to be parsed
For instance S → +SS | a | b is LL(1)
Any regular grammar S → aA , A→ bA | ε is LL(1) iff it corresponds toa deterministic FA
There are algorithms to decide if a grammar is LL(1) (not done in this course)
Any LL(1) grammar is unambiguous (because by definition there is a at mostone left most derivation for any string)
70
CFG [71]
Grammar transformations
The grammar
S → AB, A→ aA | a, b→ bB | c
is equivalent to the grammar
S → aAB, A→ aA | ε, b→ bB | c
71
CFG [72]
Grammar transformations
The grammar
S → Bb B → Sa | a
which is not LL(1) is equivalent to the grammar
S → abT T → abT | ε
which is LL(1)
72
CFG [73]
Grammar transformations
The grammar
A→ Aa | b
is equivalent to the grammar
A→ bB, B → aB | ε
In general however there is no algorithm to decide L(G1) = L(G2)
For regular expression, we have an algorithm to decide L(E1) = L(E2)
73
CFG [74]
The CYK Algorithm
We present now an algorithm to decide if w ∈ L(G), assuming G to be inChomsky Normal Form.
This is an example of the technique of dynamic programming
Let n be |w|. The natural algorithm (trying all productions of length < 2n)may be exponential. This technique gives a O(n3) algorithm!!
74
CFG [75]
dynamic programming
fib 0 = fib 1 = 1
fib (n+ 2) = fib n+ fib (n+ 1)
fib 5? calls fib 4, fib 3 and fib 4 calls fib 3
So in a top-down computation there is duplication of works (if one does notuse memoization)
75
CFG [76]
dynamic programming
For a bottom-up computation
fib 2 = 2, fib 3 = 3, fib 4 = 5, fib 5 = 8
What is going on in the CYK algorithm or Earley algorithm is similar
S → AB | BC, A→ BA | a, B → CC | b, C → AB | a
bab ∈ L(G)?? and aba ∈ L(G)?
76
CFG [77]
dynamic programming
The idea is to represent bab as the collection of the facts b(0, 1), a(1, 2), b(2, 3)
We compute then the facts X(i, k) for i < k by induction on k − i
Only one rule:
If we have a production C → AB and A in X(i, j) and B in X(j, k) then Cis in X(i, k)
77
CFG [78]
The CYK Algorithm
The algorithm is best understood in term of production systems
Example: the grammar
S → AB | BA | SS | AC | BD
A→ a, B → b, C → SB, D → SA
becomes the production system
78
CFG [79]
The CYK Algorithm
A(x, y), B(y, z) → S(x, z), B(x, y), A(y, z) → S(x, z)
S(x, y), S(y, z) → S(x, z), A(x, y), C(y, z) → S(x, z)
B(x, y), D(y, z) → S(x, z), S(x, y), B(y, z) → C(x, z)
S(x, y), A(y, z) → D(x, z), a(x, y) → A(x, y), b(x, y) → B(x, y)
79
CFG [80]
The CYK Algorithm
The problem if one can one derive S ⇒∗ aabbab is transformed to the problem:can one produce S(0, 6) in this production system given the facts
a(0, 1), a(1, 2), b(2, 3), b(3, 4), a(4, 5), b(5, 6)
80
CFG [81]
The CYK Algorithm
For this we apply a forward chaining/bottom up sequence of productions
A(0, 1), A(1, 2), B(2, 3), B(3, 4), A(4, 5), B(5, 6)
S(1, 3), S(3, 5), S(4, 6)
S(1, 5), C(1, 4), C(3, 6)
S(0, 4), . . .
S(0, 6)
81
CFG [82]
The CYK Algorithm
For instance the fact that C(3, 6) is produced corresponds to the derivation
C ⇒ SB ⇒ BAB ⇒ bAB ⇒ baB ⇒ bab
In this way, we get a solution in O(n3)!
82
CFG [83]
Forward-chaining inference
This idea works actually for any grammar. For instance
S → SS | aSb | ε
is represented by the production system
→ S(x, x), S(x, y), S(y, z) → S(x, z)
a(x, y), S(y, z), b(z, t) → S(x, t)and the problem to decide S ⇒∗ aabb is replaced by the problem to derive S(0, 4)from the facts
a(0, 1), a(1, 2), b(2, 3), b(3, 4)
83
CFG [84]
Forward-chaining inference
This is the main idea behind Earley algorithm
Mainly used for parsing in computational linguistics
Earley parsers are interesting because they can parse all context-free languages
84
CFG [85]
Complement of a CLF
We have seen that CLF are not closed under intersection, are closed underunion
It follows that they are not closed under complement
Here is an explicit example: one can show that the complement of
{anbncn | n > 0}
is a CFL
85
CFG [86]
Undecidable Problems
We have given algorithm to decide L(G) 6= ∅ and w ∈ L(G). What issurprising is that it can be shown that there are no algorithms for the followingproblems
Given G1 and G2 do we have L(G1) ⊆ L(G2)? Do we have L(G1) = L(G2)?Given G and R regular expression, do we have L(G) = L(R)? L(R) ⊆ L(G)?Do we have L(G) = T ∗ where T is the alphabet of G? (Compare to the case ofregular languages)
Given G is G ambiguous??
86
CFG [87]
Undecidable Problems
One reduces these problems to the Post Correspondance Problem
Given u1, . . . , un and v1, . . . , vn in {0, 1}∗ is it possible to find i1, . . . , ik suchthat
ui1 . . . uik = vi1 . . . vik
Example: 1, 10, 011 and 101, 00, 11
Challenge example: 001, 01, 01, 10 and 0, 011, 101, 001
87
CFG [88]
Haskell Program
isPrefix [] ys = TrueisPrefix (x:xs) (y:ys) = x == y && isPrefix xs ysisPrefix xs ys = False
isComp (xs,ys) = isPrefix xs ys || isPrefix ys xs
exists p [] = Falseexists p (x:xs) = p x || exists p xs
exhibit p (x:xs) = if p x then x else exhibit p xs
88
CFG [89]
Haskell Program
addNum k [] = []addNum k (x:xs) = (k,x):(addNum (k+1) xs)
nextStep xs ys =concat (map (\ (n,(s,t)) ->
map (\ (ns,(u,v)) -> (ns++[n],(u ++ s,v ++ t)))ys)
xs)
89
CFG [90]
Haskell Program
mainLoop xs ys =letbs = filter (isComp . snd) ysprop (_,(u,v)) = u == vinif exists prop bs then exhibit prop bselse if bs == [] then error"NO SOLUTION"
else mainLoop xs (nextStep xs bs)
90
CFG [91]
Haskell Program
post xs =letas = addNum 1 xsin mainLoop as (map (\ (n,z) -> ([n],z)) as)
xs1 = [("1","101"),("10","00"),("011","11")]
xs2 = [("001","0"),("01","011"),("01","101"),("10","001")]
91
CFG [92]
Haskell Program
Main> post xs1([1,3,2,3],("101110011","101110011"))
Main> post xs2
ERROR - Garbage collection fails to reclaim sufficient space[2,2,2,3,2,2,2,3,3,4,4,6,8,8,15,21,15,17,18,24,15,12,12,18,18,24,24,45,63,66,84,91,140,182,201,346,418,324,330,321,423,459,780
92
CFG [93]
Post Correspondance Problem and CFL
To the sequence u1, . . . , un we associate the following grammar GA
The alphabet is {0, 1, a1, . . . , an}
The productions are
A→ u1a1 | . . . | unan | u1Aa1 | . . . | unAan
This grammar is non ambiguous
93
CFG [94]
Post Correspondance Problem and CFL
To the sequence v1, . . . , vn we associate the following grammar GB
The alphabet is the same {0, 1, a1, . . . , an}
The productions are
B → v1a1 | . . . | vnan | v1Ba1 | . . . | vnBan
This grammar is non ambiguous
94
CFG [95]
Post Correspondance Problem and CFL
Theorem: We have L(GA)∩L(GB) 6= ∅ iff the Post Correspondance Problemfor u1, . . . , un and v1, . . . , vn has a solution
95
CFG [96]
Post Correspondance Problem and CFL
Finally we have the grammar G with productions
S → A | B
Theorem: The grammar G is ambiguous iff the Post Correspondance Problemfor u1, . . . , un and v1, . . . , vn has a solution
96
CFG [97]
Post Correspondance Problem and CFL
The complement of L(GA) is CF
We see this on one example u1 = 0, u2 = 10
The complement of L(GB) is CF
Hence we have a grammar GC for the union of the complement of L(GA)and the complement of L(GB)
97
CFG [98]
Post Correspondance Problem and CFL
Theorem: We have L(GC) = T ∗ iff L(GA) ∩ L(GB) = ∅
Hence the problems
L(E) = L(G)
L(E) ⊆ L(G)
are in general undecidable
98