finite-state automata & recursive transition...
TRANSCRIPT
![Page 1: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/1.jpg)
1
Finite-State Automata& Recursive Transition
Network
J. SavoyUniversité de Neuchâtel
R. Mitkov (Ed): The Oxford Handbook of Computational Linguistics. Oxford University Press, Oxford, 2005.
G. Gaznar & C. Mellish: Natural Language Processing in PROLOG: An Intorduction to Computational Linguistics. Addison-Wesley, 1989.
![Page 2: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/2.jpg)
2
Purpose
� Finite-State Automata (FSA)� Very simple automaton� Efficient / effective� Use to recognize or to generate (an answer)� Applications in various sublanguages� A first step (and a first model)
![Page 3: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/3.jpg)
3
Basic Notation
� Language: set of strings of any kind� String: concatenation of zero (null string) or more symbols� Symbols: single char (e.g., a) or multi-char (e.g., +noun)
empty string (ε or 0) → atomic entity� Relation: a set of ordered string pairs e.g., {(a,bb), (cd,ε)}.
The first member is the upper string (domain)The second member is the lower string (range)
� Identity relation: {(a,a), (c,c), …}.
![Page 4: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/4.jpg)
4
Basic Notation
� Network: a finite-state automata or networkdirected graphs composed of states and arcs
� A single initial state (start state)� Any number of final states
� Arc: may be labeled either by a single symbol (a) or a symbol pair (a:b) (or e.g., (car:voiture))
� Path: a sequence of arcs from the start state to a final state
![Page 5: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/5.jpg)
5
ExampleA finite-state automata to recognize the laughing language (ha!, haha!, hahaha!, etc.)Where are the states?Where are the arcs? And their labels?
10h
The initial state will be placed on the left.When we’re processing a string, we remove the front symbol corresponding to the label on the arc.
![Page 6: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/6.jpg)
6
FSA Processing
If starting in the initial state, we reach a final state and the input string is empty at this point, we have found a path and the entry string is valid. If we cannot reach a final state, the input string is not valid (well-formed formula, wff).If we have a choice, we can come back (backtracking) and try the alternative way later.
![Page 7: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/7.jpg)
7
Example
Example: A finite-state automaton to recognize the laughing language (ha!, haha!, hahaha!, etc.)If you find a path, you find a valid string
210h !
a
h3
The final state will be placed on the right (in bold).At any state, you have no real choice (deterministic automaton). Is this the single solution?
![Page 8: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/8.jpg)
8
Deterministic & non-deterministic
Example of a non-deterministic automaton
210!
h
a3
Do we recognize the same strings?Why non-deterministic?
a
![Page 9: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/9.jpg)
9
More Complex Transition
Draw a FSA to accept from state i to state j …a) the symbol “a”b) any symbolc) the symbols “ed”d) with, at least, once the symbol “b”e) with 0, 1, or n occurrences of the symbol “c”
We can define a metachar (e.g., *)
ji
?
![Page 10: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/10.jpg)
10
More Complex Examples
Draw the FSA to accept numbers…a) Real numbers (e.g., 123.45)b) With the scientific notations (e.g., 3.14E+02)c) A simple English grammar
Hint: Replace a set of symbols by a LABELLABEL: set of symbols
![Page 11: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/11.jpg)
11
Simple Grammar
We have the following lexicon:NNP (proper noun): Kim, Mary, Ann.DT (determiner): a, the, her.NN (noun): consumer, man, woman.VB (verb): is, wasJJ (adjective): happy, stupid.MD (modal): veryRB (adverb): often, always, sometimes.CC (conjunction): and, or.
(We have used the Penn Treebank POS tags)
![Page 12: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/12.jpg)
12
Simple Grammar
3
2
1
9
4
DTNNP
NN
VB
CC CC
5
6
7
8
NN
DT
JJ
DTε
JJ
RB
ε MD
MD
![Page 13: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/13.jpg)
13
Simple Grammar
• Verify your grammar with the following string:a. Kim is happyb. Ann is often a consumerc. Mary is a happy consumer and Ann is happyd. the consumer is very happy • What are the strings you can generate?• Can you add the plural form?
• Can you transform the grammar to admit also interrogative string?
![Page 14: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/14.jpg)
14
Lexical Analysis
• Other application: Recognize a set of words belonging to a sublanguage.
• Build a finite-state automaton that will recognize the following words• Clear• Clever• Ear• Ever• Fat• Fatter
![Page 15: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/15.jpg)
15
Example
FSA for the lexical analysis
4 53210
6 7 8 9
fa t
c ael r
e
t
ve
![Page 16: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/16.jpg)
16
Transducers
• Instead of verifying the string (valid or not), we may transform it (more precisely, return a new output string if the input string is valid).
• Use the relation definition (e.g., {(a,bb), (cd,ε)}) between states (but one could be the empty symbol ε).
• We may change our name from Finite-State Automaton to Finite-State Transducer (FST).
• You need to propose a FST to produce the translation for the string (where is the exit) into (où est la sortie).
![Page 17: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/17.jpg)
17
Translation withTansducerA simple translation example
4 3
21
0
where:oùis:est
the:la
exit:sortie
![Page 18: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/18.jpg)
18
Transducer
1. Generalize the previous example.
• We may replace a specific symbol pair by a set possible symbols (e.g, NN)ExampleNNexit:sortie, shop:boutique,policeman:gendarme, toilet:toilette
2. Can you take account for the gender (the:la) and (the:le)?
![Page 19: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/19.jpg)
19
Translations with Transducer
NNexit:sortie, shop:boutique, policeman:gendarme, toilet:toiletteBut “où est la gendarme”!
4 3
21
0
where:oùis:est
the:la
NN
![Page 20: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/20.jpg)
20
Translations with Gender
NNexit:la_sortie,shop:la_boutique, policeman:le_gendarme,toilet:la_toilette
Closed association between DT & NN!
4 3
21
0
where:oùis:est
the:ε
NN
![Page 21: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/21.jpg)
21
Translations with Gender
NFexit:sortie, shop:boutique, toilet:toiletteNMpoliceman:gendarme
5
3
21
0
where:oùis:est
the:la
NF
4
the:le
NM
![Page 22: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/22.jpg)
22
Overview of FSA and FST
• The simplest approach to NLP• Of very little use by themselves• Map one string of symbols into another• Can be used for sublanguage translation• Can be used for morphological processing• Are easy to implement• Can handle expression like anbm but not anbn
• But not sufficient for NLP
![Page 23: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/23.jpg)
23
Recursive Transition Network
• We could name a network (for future use) and thus we may simplify the design process (we may reuse existing networks)(Recursive Transition Network, RTN)
• We have: NN: woman, house, table, mouse, man, genius, …NNP: Mary, John, Washington, Ben, …DT: a, the, that, …VB: sees, hits, sings, loves, saw, …WH: who, which, that, …
• A valid Sentence is a string with a NounPhrasefollows by a VerbPhrase.
![Page 24: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/24.jpg)
24
Recursive Transition Network
• A valid NounPhrase is a string composed of a determinant follows by a noun and possibly follows by a WH and VerbPhrase
210NP VPS:
210DT NNNP:
3WH
VP
NNP
![Page 25: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/25.jpg)
25
Recursive Transition Network
• And for the VerbPhrase network (VP):
• Can you recognize the sentences:John sees the house, Mary loves John,John says that Mary says that Washington is a genius
210VB NPVP:
3
Sthat
![Page 26: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/26.jpg)
26
Recursive Transition Network
• We clearly take account for recursive patterns in the language
• Very common in all natural languages• But with limited use• “The man who the woman sings sees Ben” .
“The man who the woman who the boy plays sings hits Washington” .
• Useful to named a network• Handle recursive expressions easily
“The rapidity that the motion that the wing that the hummingbird has has has is remarkable”
• Can recognize expression such as anbn but not anbncn
![Page 27: Finite-State Automata & Recursive Transition Networkmembers.unine.ch/jacques.savoy/lectures/SemCL/FSACL.pdf · 2009-03-24 · Example: A finite-state automaton to recognize the laughing](https://reader034.vdocuments.site/reader034/viewer/2022042711/5f820bed2e072408331ec147/html5/thumbnails/27.jpg)
27
Exercice
• Using FSAcan you represent the Latin morphology E.g., the first declensionnominative, vocative, accusative, genitive, dative, ablativerosa, rosa, rosam, rosae, rosae, rosae;rosae, rosae, rosas, rosarum, rosis, rosis.
• Using the RTN method, can you recognize a grammar having the following ruleS: a S band that can generate ab, aabb, aaabbb, ..., anbn