1 formal languages and theory of computation wen-hsiang tsai 蔡文祥 講座教授 2011/9 taj mahal...
Post on 12-Jan-2016
221 Views
Preview:
TRANSCRIPT
1
Formal Languages and Formal Languages and Theory of ComputationTheory of Computation
Wen-Hsiang TsaiWen-Hsiang Tsai蔡文祥 講座教授蔡文祥 講座教授
20120111/9/9
Taj Mahal in India
2
Chapter 0Chapter 0
IntroductionIntroduction
Opera House in Sidney, Australia
Chapter. 0 – p. 3
OutlineOutline
0.0 Reasons for Taking This Course0.0 Reasons for Taking This Course
0.1 General Concepts0.1 General Concepts
0.2 Problems Studied in Theory of Computation0.2 Problems Studied in Theory of Computation
0.3 Fields Related to Theory of Computation0.3 Fields Related to Theory of Computation
0.4 Applications of Theory of Computation0.4 Applications of Theory of Computation
0.5 Brief History of Theory of Computation0.5 Brief History of Theory of Computation
0.6 Textbooks and Course Grading0.6 Textbooks and Course Grading
Chapter. 0 – p. 4
0.0 Reasons for Taking This Course0.0 Reasons for Taking This Course
Philosophy of computer science (CS)Philosophy of computer science (CS) Indispensable for complete study of CSIndispensable for complete study of CS Broadening views in all fields of CSBroadening views in all fields of CS Enhancing problem solving capabilityEnhancing problem solving capability Basic training for undergraduate studentsBasic training for undergraduate students Background course for graduate studentsBackground course for graduate students Qualification exam topic for Ph. D. students.Qualification exam topic for Ph. D. students.
Chapter. 0 – p. 5
0.1 General Concepts0.1 General Concepts
Course title ---Course title ---““Formal Languages and Theory of CompuFormal Languages and Theory of Compu
tation” (tation” ( 正規語言與計算理論正規語言與計算理論 )) Scope of course:Scope of course:
Formal languagesFormal languages Automata theoryAutomata theory ComputabilityComputability Computational complexityComputational complexity
Chapter. 0 – p. 6
0.1 General Concepts0.1 General Concepts Formal Languages ---Formal Languages ---
Definition --- Definition ---
Study of “Study of “strings of symbolsstrings of symbols,” including their,” including their
descriptions, properties,descriptions, properties,
generations, recognitions,generations, recognitions,
compiling, applications, …compiling, applications, …
Chapter. 0 – p. 7
0.1 General Concepts0.1 General Concepts Formal Languages ---Formal Languages ---
Concepts (1/2) ---Concepts (1/2) ---
Strings of symbols include those ofStrings of symbols include those of
natural languages,natural languages,
computer languages,computer languages,
discrete signals, …discrete signals, …
Chapter. 0 – p. 8
0.1 General Concepts0.1 General Concepts
Formal Languages ---Formal Languages --- Concepts (2/2) --- Concepts (2/2) ---
Results of study of Results of study of automatic processing of automatic processing of
symbolic informationsymbolic information
Parallel to Parallel to numerical mathematical processinumerical mathematical processi
ngng
Similar to the role of Similar to the role of set theoryset theory ( (集合論集合論 )) in in
mathematicsmathematics
Chapter. 0 – p. 9
0.1 General Concepts0.1 General Concepts Automata Theory ---Automata Theory ---
A question --- A question ---
Do you know how a vending machine Do you know how a vending machine
works? Can you design one?works? Can you design one?
Vending machine room seen in Hokkaido, Japan 2004
Chapter. 0 – p. 10
0.1 General Concepts0.1 General Concepts
Automata Theory ---Automata Theory ---
An example (1/3) --- An example (1/3) ---
How to design a vending machine?How to design a vending machine?
Use a Use a finite automatonfinite automaton!!
Chapter. 0 – p. 11
0.1 General Concepts0.1 General Concepts
Automata Theory ---Automata Theory ---
An example (2/3) --- An example (2/3) ---
Assumptions (for simplicity):Assumptions (for simplicity):
Only NT 5-dollar and 10-dollar coins are used.Only NT 5-dollar and 10-dollar coins are used.
Only drinks all of 20 dollars are sold.Only drinks all of 20 dollars are sold.
Chapter. 0 – p. 12
0.1 General Concepts0.1 General Concepts Automata Theory ---Automata Theory ---
An example (3/3) ---An example (3/3) ---
Requiring “memory” called “states” for the design.Requiring “memory” called “states” for the design.
Notes: see next page. Notes: see next page.
$0startstart
$5 $5
$10 $10
$5
$15
$5
$10
$10
$10$10 $5
$20transition diagram
Chapter. 0 – p. 13
0.1 General Concepts0.1 General Concepts
$10$10 $5 is returned as “output” (not shown)
$20 “Final state”
Automata Theory ---Automata Theory ---
Notes for the last diagram:Notes for the last diagram:
$0startstart “Start state”
Chapter. 0 – p. 14
0.1 General Concepts0.1 General Concepts
Automata Theory ---Automata Theory ---
Definition --- Definition ---
Study of dynamic behaviors of “discrete-Study of dynamic behaviors of “discrete-
parameter information systems” in form ofparameter information systems” in form of
“abstract computing devices” or “abstract computing devices” or
“machines.”“machines.”
Chapter. 0 – p. 15
0.1 General Concepts0.1 General Concepts
Automata Theory ---Automata Theory ---
Examples of discrete-parameter Examples of discrete-parameter
information systems ---information systems ---
digital systems, nerve systems, languagesdigital systems, nerve systems, languages
information transmission systemsinformation transmission systems
human-environment interactions, …human-environment interactions, …
Chapter. 0 – p. 16
0.1 General Concepts0.1 General Concepts
Automata Theory ---Automata Theory --- Three major models of automata ---Three major models of automata ---
generatorgenerator --- with output and --- with output and withoutwithout input input
acceptoracceptor --- with input and --- with input and withoutwithout output output
transducertransducer --- --- bothboth with input and with output with input and with output
generator acceptor transducer
triggtrigger er signasignall
Yes or No
Chapter. 0 – p. 17
0.1 General Concepts0.1 General Concepts Automata Theory ---Automata Theory ---
Examples ---Examples --- generator generator --- ---
““natural language” grammarnatural language” grammar (generating “sentences” spoken by people)(generating “sentences” spoken by people)
reception robotreception robot
(speaking organized words and sentences)(speaking organized words and sentences)
context-free grammarcontext-free grammar
(generating strings of symbols)(generating strings of symbols) ******
(*** abstract models studied in this course)(*** abstract models studied in this course)
Reception robot --- Expo 2005
Chapter. 0 – p. 18
0.1 General Concepts0.1 General Concepts
Automata Theory ---Automata Theory --- Examples ---Examples ---
acceptor acceptor --- --- digital lockdigital lock (accepting digits)(accepting digits)
lexical analyzerlexical analyzer (recognizing computer language (recognizing computer language
keywords)keywords)
finite automatonfinite automaton
(accepting valid strings of symbols)(accepting valid strings of symbols) ******(*** abstract models studied in this course)(*** abstract models studied in this course)
Digital lock
Chapter. 0 – p. 19
0.1 General Concepts0.1 General Concepts
Automata Theory ---Automata Theory ---
Examples ---Examples --- transducer transducer --- ---
InterpreterInterpreter (translating natural languages)(translating natural languages)
CompilerCompiler (translating high-level languages into (translating high-level languages into
machine codes)machine codes)
Turing machineTuring machine
(transforming strings of symbols)(transforming strings of symbols) ******(*** abstract models studied in this course)(*** abstract models studied in this course)
“Interpreter”雙面翻譯 (a movie)
Chapter. 0 – p. 20
0.1 General Concepts0.1 General Concepts
Computability ---Computability ---
Definition --- Definition ---
Study of problem solving capabilities of Study of problem solving capabilities of
computational models.computational models.
Chapter. 0 – p. 21
0.1 General Concepts0.1 General Concepts
Computability ---Computability ---
Problem types based on Problem types based on resourcesresources --- ---
Impossible problemsImpossible problems
Possible-with-unlimited-resources-but-impoPossible-with-unlimited-resources-but-impo
ssible-with-limited-resourcesssible-with-limited-resources problems problems
Possible-with-limited-resources problemsPossible-with-limited-resources problems
Chapter. 0 – p. 22
0.1 General Concepts0.1 General Concepts
Computability ---Computability ---
Problem types based on Problem types based on timetime --- ---
Undecidable problemsUndecidable problems
Intractable Intractable problemsproblems
Tractable problemsTractable problems
Chapter. 0 – p. 23
0.1 General Concepts0.1 General Concepts
Computability ---Computability --- Advantage ---Advantage ---
Studies of computability help us Studies of computability help us not to not to
waste timewaste time on “unsolvable problems” on “unsolvable problems”
already investigated before.already investigated before.
Chapter. 0 – p. 24
0.1 General Concepts0.1 General Concepts
Computational complexity ---Computational complexity ---
Definition --- Definition ---
Study of “Study of “efficiencyefficiency” of problem solving.” of problem solving.
Chapter. 0 – p. 25
0.1 General Concepts0.1 General Concepts
Computational complexity ---Computational complexity --- Concepts (1/2) ---Concepts (1/2) ---
To unify comparison, an To unify comparison, an abstract modelabstract model is is
needed as the machine for executing needed as the machine for executing
problem solutions.problem solutions.
Usually the most famous “Usually the most famous “Turing machineTuring machine” ”
(an automaton) is used.(an automaton) is used.
Chapter. 0 – p. 26
0.1 General Concepts0.1 General Concepts
Computational complexity ---Computational complexity ---
Concepts (2/2) ---Concepts (2/2) ---
Turing machine, though simple, has been Turing machine, though simple, has been
proved to be able to proved to be able to simulate any problem simulate any problem
solving steps (“algorithms”) solving steps (“algorithms”) designed by designed by
human beings! human beings!
Turing machine is the Turing machine is the foundationfoundation for for
development of modern computation theory!development of modern computation theory!
Chapter. 0 – p. 27
0.2 Problems Studied in Theory of 0.2 Problems Studied in Theory of ComputationComputation
Essential problems studied ---Essential problems studied ---
““What are the fundamental capabilities and What are the fundamental capabilities and
limitations of computers?”limitations of computers?”
What can a computer do at all? --- What can a computer do at all? ---
studied in the domain of studied in the domain of ComputabilityComputability!!
What can a computer do efficiently? --- studied What can a computer do efficiently? --- studied
in the domain of in the domain of Computational complexityComputational complexity!!
Chapter. 0 – p. 28
0.2 Problems Studied in Theory of 0.2 Problems Studied in Theory of ComputationComputation
A basic concept ---A basic concept --- A Turing machine conceptually is no A Turing machine conceptually is no
more than more than a pen and a paper with a pen and a paper with several reasoning rulesseveral reasoning rules!!
=?=?
Turing machine=
?
=?
Chapter. 0 – p. 29
0.3 Fields Related to Scope of 0.3 Fields Related to Scope of Theory of ComputationTheory of Computation
Fields Related theory
Compiling theory Formal languages
Switching circuit theory Automata theory
Algorithm analysis Computational complexity
Natural language processing Formal languages
Syntactic pattern recognition Formal languages
Programming languages Formal languages
Artificial intelligence Formal languages and automata theory
Neural networks Automata theory
Chapter. 0 – p. 30
0.4 Applications of Theory of 0.4 Applications of Theory of ComputationComputation
Applications ---Applications --- Text analysis ---Text analysis ---
text search (text search ( 文辭搜尋文辭搜尋 ))
text editingtext editing ( (文句編輯文句編輯 ))
Compiler design (Compiler design ( 編譯器設計編譯器設計 )) lexical analysis (lexical analysis ( 辭彙分析辭彙分析 ))
parser generation (parser generation ( 剖析器產生剖析器產生 ))
Chapter. 0 – p. 31
0.4 Applications of Theory of 0.4 Applications of Theory of ComputationComputation
Applications ---Applications --- Language designLanguage design
programming language design (programming language design ( 程式語言設程式語言設計計 ))
document description language design (document description language design ( 公公文文描述語言設計描述語言設計 ))
e.g., HTML, XML, …e.g., HTML, XML, … picture language design (picture language design ( 圖畫語言設計圖畫語言設計 ))
e.g., SVG, VHML, …e.g., SVG, VHML, … special language design (special language design ( 特別語言設計特別語言設計 ) )
Chapter. 0 – p. 32
0.4 Applications of Theory of 0.4 Applications of Theory of ComputationComputation
Applications ---Applications --- Digital system designDigital system design
computer design (computer design ( 計算機設計計算機設計 )) special digital system design (special digital system design ( 特殊數位系統特殊數位系統
設計設計 )) Protocol modeling and verification (Protocol modeling and verification ( 網路網路
協定模式化與驗証協定模式化與驗証 )) Expert system design(Expert system design( 專家系統設計專家系統設計 )) Cryptography(Cryptography( 密碼學密碼學 ) …) …
Chapter. 0 – p. 33
0.4 Applications of Theory of 0.4 Applications of Theory of ComputationComputation
A specific application --- A specific application --- special digital special digital system designsystem design
Mainly based on automata theory --- Mainly based on automata theory --- using finite automata or “using finite automata or “sequential machinessequential machines” ”
(finite automata (finite automata both both with with input and input and with with outputoutput))
Some specific examples of digital Some specific examples of digital systems --- systems --- Elevators, vending machines, traffic signal Elevators, vending machines, traffic signal
control systems …control systems …
Chapter. 0 – p. 34
0.5 Brief History of Theory of 0.5 Brief History of Theory of Computation Computation
At beginning of 19th century, mathemAt beginning of 19th century, mathematicians believe: aticians believe:
Axiomatization of mathematicsAxiomatization of mathematics(( 數學公理化數學公理化 ))
That is,That is,
it is possible to decide the truth of any it is possible to decide the truth of any mathematical statement thru use of mathematical statement thru use of infinference rules erence rules andand axiom systems axiom systems..
Chapter. 0 – p. 35
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
In 1931, Kurt GIn 1931, Kurt Göödel proposed the del proposed the incompleteness theorem incompleteness theorem which says:which says:
the truth of certain mathematical the truth of certain mathematical statements statements cannot cannot be proved by the use be proved by the use of mathematical axioms.of mathematical axioms.
That is, That is,
some mathematical problems some mathematical problems cannotcannot be be solved by algorithms.solved by algorithms.
Chapter. 0 – p. 36
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
From then, the ideal of From then, the ideal of axiomatization axiomatization
of mathematicsof mathematics was broken. was broken.
Then, mathematicians started to Then, mathematicians started to
think the scope of algorithm capabilities, think the scope of algorithm capabilities,
design theoretical computational machinesdesign theoretical computational machines, & , &
study their capability limitations.study their capability limitations.
Chapter. 0 – p. 37
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
Many theories were proposed, but the Many theories were proposed, but the
corresponding machines corresponding machines could not be could not be
manufacturedmanufactured at that time. at that time.
Chapter. 0 – p. 38
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
In 1936, In 1936, Turing Turing proposed the model proposed the model of of
universal algorithm machineuniversal algorithm machine,,
which later was calledwhich later was called
Turing machineTuring machine
mentioned before.mentioned before.
Chapter. 0 – p. 39
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
Some similar ideas were proposed by Some similar ideas were proposed by
other scholars, includingother scholars, including
Stephen C. Stephen C. KleeneKleene,,
Alonzo Alonzo ChurchChurch,,
Emil Emil PostPost, etc.,, etc.,
in different forms of models.in different forms of models.
Chapter. 0 – p. 40
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
In 1943, neural physiologistsIn 1943, neural physiologists
Warren S. Warren S. McCulloch McCulloch andand
Walter Walter PittsPitts
developed developed finite-state systems finite-state systems to to simulate neural networks in biological simulate neural networks in biological systems.systems.
They are pioneers of They are pioneers of automata theoryautomata theory..
Chapter. 0 – p. 41
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
In late 1940’sIn late 1940’s ,, Von Neumann Von Neumann
proposed the idea of proposed the idea of
stored programstored program
for computer models.for computer models.
Chapter. 0 – p. 42
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
In 1951, a In 1951, a real computer real computer following the idfollowing the idea of “stored program” was constructed:ea of “stored program” was constructed:
UNIVAC IUNIVAC I,,
the world's first commercially available cthe world's first commercially available c
omputeromputer (manufactured (manufactured by Eckert-Mauchby Eckert-Mauch
ly Computer Company) .ly Computer Company) .
Chapter. 0 – p. 43
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
In late 1950’s, linguist In late 1950’s, linguist N.N. Chomsky Chomsky proposed proposed aa mathematical model for mathematical model for grammars grammars of natural languages.of natural languages.
In 1956, he proposed further the In 1956, he proposed further the concept of concept of context-free grammarcontext-free grammar, , which may be used for defining which may be used for defining computer languages.computer languages.
Chapter. 0 – p. 44
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
In 1956, In 1956, Kleene Kleene proposed the concept proposed the concept of of finite automaton finite automaton for simulating neurafor simulating neural networks proposed by McCulloch and l networks proposed by McCulloch and Pitts.Pitts.
He also proposed He also proposed regular expressions regular expressions tto describe strings of symbols, and provo describe strings of symbols, and proved them ed them equivalent equivalent to finite automata.to finite automata.
Chapter. 0 – p. 45
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
In 1959 and 1960, In 1959 and 1960, John W. Backus & John W. Backus & Peter Naur Peter Naur proposed sequentially a spproposed sequentially a special expression for computer languagecial expression for computer language grammars, called later e grammars, called later Backus-Naur Backus-Naur formform (BNF) . (BNF) .
It was used to describe computer languaIt was used to describe computer language ge ALGOL-60ALGOL-60, leading to development of , leading to development of compiling theorycompiling theory..
Chapter. 0 – p. 46
0.5 Brief History of Theory of 0.5 Brief History of Theory of ComputationComputation
In 1969, In 1969, Stephen A.Stephen A. Cook Cook found that found that computation problems may be computation problems may be separated into separated into tractabletractable and and intractable intractable ones.ones. Intractable problems are also called Intractable problems are also called NP-NP-
hardhard problems. problems. Such problems cannot be solved by Such problems cannot be solved by
computers except very computers except very small instances small instances (with only small-sized inputs).(with only small-sized inputs).
Chapter. 0 – p. 47
0.6 Textbooks and Course Grading0.6 Textbooks and Course Grading
Textbook:Textbook: Hopcroft, Motwani, Ullman, Hopcroft, Motwani, Ullman, Introduction to AutoIntroduction to Auto
mata Theory, Languages, and Computationmata Theory, Languages, and Computation (3r (3rd Edition), Addison-Wesley, Boston, MA, USA, d Edition), Addison-Wesley, Boston, MA, USA, 20072007
Reference textbook:Reference textbook: 蔡文祥蔡文祥 , , 正規語言與計算理論正規語言與計算理論 (Formal Languag(Formal Languag
es & The Theory of Computation)(es & The Theory of Computation)( 尚未出版書尚未出版書籍;中文籍;中文 ) () ( 參考用參考用 ))
Chapter. 0 – p. 48
0.6 Textbooks and Course Grading0.6 Textbooks and Course Grading
Course grading:Course grading: Homeworks 33.33%Homeworks 33.33%
mid-term exam 33.33%mid-term exam 33.33%
final exam 33.33%final exam 33.33%
Website for downloading course material aWebsite for downloading course material and quizzes :nd quizzes :
http://www.cis.nctu.edu.tw/~whtsai/http://www.cis.nctu.edu.tw/~whtsai/
top related