CS4026 Formal Models of Computation Part III Computability & Complexity Part III-A – Computability Theory

Download CS4026 Formal Models of Computation Part III Computability & Complexity Part III-A – Computability Theory

Post on 05-Jan-2016




3 download

Embed Size (px)


  • DecidabilityOverview (Chapter 4 of Sipsers Book)Decidable languagesThe Halting ProblemMotivationInvestigate the power of algorithms to solve problemsSome problems can be solved, others cannot!If you know a problem is algorithmically unsolvable then it must be simplified before we propose an algorithmInstead of Hilberts problem, you might build an algorithm for deciding whether any roots exist between -n and +n.Computers have limitations we must consider these if we want to use computers well

  • Decidable LanguagesLets look at problems concerning finite automataWe shall use languages to represent problemsWe already have a terminology to deal with languagesIn this area we present mere sketches of proofs More precise proofs will re-appear when we turn to undecidable languages

  • Decidable Problems on Reg. Langs.Decidable problems concerning regular languagesRegular languages: accepted by finite automataDeterministic Finite Automata (DFAs):L = {01*01*}

  • Decidable Problems on R.L. (Contd)The acceptance problem for DFAs: givenA deterministic finite automaton (DFA) andA string check if the automaton accepts the stringLanguage contains the encodings of all DFAs coupled with strings that the DFAs accept:ADFA = {B, w | B is a DFA that accepts input string w }DFA B accepts input w same as B, w ADFAOther computational problems can be formulated in terms of testing membership of a setShowing that the language is decidable is the same as showing the computational problem is decidable

  • Decidable Language 1Theorem: ADFA is a decidable languageProof sketch: We only need to present a TM M that decides ADFAM = On input B, w , where B is a DFA and w is a string: 1. Simulate B on input w 2. If the simulation ends in an accepting state of B , then accept; If the simulation ends in a non-accepting state of B, then reject.We dont prove that simulation is possible ...

  • ... but note that the transitions of an DFA can be encoded as follows:

    Each of the states qi is encoded as a string: q1=DA, q2=DAA, q3=DAAA, ...Each of the symbols Sj is encoded as a string: S1=DC, S2=DCC, S3=DCCC, ...Now each transition rule can be encoded as a string. E.g. d(q1,S2)=q2 is endoded as DADCCDAAE (with E for end of transition) The algorithm terminates because the number of steps needed by a DFA to reach accept/reject is finite (bounded by the number of symbols in the string w).

  • Decidable Language 2Is there an algorithm to check if a DFA accepts any strings at all?In terms of languages:EDFA = {A | A is a DFA and L (A) = }

  • Decidable Language 2 (Contd)Theorem: EDFA is a decidable languageProof: a DFA accepts some string if, and only if, it is possible to go from the start state to an accept state following the arrowsA TM T that decides EDFA:T = On input A , where A is a DFA: 1. Mark the start state of A 2. Repeat until no new states get marked: 3. Mark any state that has a transition coming into it from any marked state 4. If an accept state is marked, accept; otherwise reject

  • Decidable Language 3Is there an algorithm to check if two DFAs are equivalent, i.e., they recognise the same language?In terms of languages:EQDFA = {A, B | A and B are DFAs and L (A) = L (B )}In the true spirit of software reuse, lets devise a solution to this problem using the previous MT!

  • Decidable Language 3 (Contd)We build a new DFA C from A and B such thatC accepts only those strings that are accepted either by A or B but not both;A and B recognise the same language iff C is empty The language of C isL (C ) = (L (A ) L (B )) (L (A ) L (B )) This is the symmetric difference of L (A ) and L (B )L (A ) is the complement of L (A )Diagrammatically: in grey, L (C ) L (A )L (B )

  • Decidable Language 3 (Contd)We build C from A and B with operations on DFAs:We can build a DFA that recognises L (A ) L (B )We can build a DFA that recognises L (A ) L (B )We can build a DFA that recognises L (A )We can therefore build a DFA that recognises CFor DFAs, there is no difference between recognizing and decidingDFAs can be implemented using TMsTMs can perform them!Once we have built C, we can use previous TM to check if L (C ) is emptyif L (C ) is empty then L (A ) = L (B )

  • Decidable Language 3 (Contd)The proof is an MT:F = On input A,B where A and B are DFAs: 1. Build DFA C as described 2. Run TM T from previous example on C 4. If T accepts, accept; If T rejects, rejectBringing together CS3012 and CS4026:

  • A Hierarchy of LanguagesClasses of languages and how they relate:

  • Before we get to the Halting ProblemPrerequisites in set theory

    The Russell Paradox. (See CS3511, or any text book on set theory.)

    Classification of functions using the concepts of an injection, surjection, bijection/one-to-one correspondence (See CS3511, or any text book on set theory)

    You may need to study these for yourself. Details are not repeated in these lectures.

  • The Diagonalisation Method

    Proposed by Georg Cantor (1873)Can we compare the size of infinite sets?For finite sets this is straightforward: count!But how about infinite sets?Example: card(N ) > card({0*1*})?Both are infiniteBut is one larger than the other?Cantors idea:The size (cardinality) of a set should not depend on the identity of its elementsTwo finite sets A and B have the same size if we can pair the elements of A with elements of BWe can extend the idea to infinite sets

  • CorrespondencesAssume sets A and B and function f from A to Bf is one-to-one (also: injective) if it does not map a, b A onto the same element in B , that is:if a b then f (a) f (b) f is onto (also: surjective) if for every b B there is an a A such that f (a) = bFunctions that are both one-to-one and onto are (one-to-one) correspondences (Also: bijections)Sets A and B have same size if we can show there is a correspondence between their elements:Every element of A maps to a unique element of B Each element of B has a unique elem. of A mapping to it

  • Correspondences (Contd)Example: LetN be the set of natural numbers {1, 2, 3, }E be the set of even natural numbers {2, 4, 6, }Using Cantors definition of size, we can show that N and E have the same size:Correspondence f (n) = 2nA table showing the pairingIntuitively, E is smaller than N !However, Pairing each element of N with its corresponding element in E is possible, So we declare these two set to be the same sizeThis is even though E N (E is a real subset of N )

  • Countable setsWe say: a set is countable if either It is finite orIt has the same size as N, the natural numbers

    For example,N is countable, and so are all its subsets:E is countable{0,1,2,3} is countable is countable

    How about supersets of N?

  • An Even Stranger ExampleLet Q be the set of positive rational numbersQ = { m/n | m,n N }Just like E, the set Q has the same size as N !We show this giving a correspondence from Q to N Q is thus countable One way to give a correspondence between a set X and N is to enumerate (i.e., to list) Xs elements. For example, for X=Q :List all the elements of Q Pair the first element of the list with 1 from N Pair the second element of the list with 2 from N And so on, making sure every member of Q appears only once in the list

  • An Even Stranger Example (Contd)To build a list with the elements of Qmake infinite matrix with all positive rational numbersi -th row contains all numbers with numerator ij -th column has all numbers with denominator ji /j is in i -th row and j -th column

  • An Even Stranger Example (Contd)Now we turn the previous matrix into a listA bad way: begin list with first rowSince rows are infinite, we will never get to 2nd row!

  • An Even Stranger Example (Contd)Instead, we list the elements along diagonals:We should, however, eliminate repeated elements

  • An Even Stranger Example (Contd)We list elements along diagonals w/o repetitions:1/1, 2/1, 1/2, 3/1, 1/3,

  • Uncountable sets

    Some infinite sets have no correspondence with NThese sets are simply too big!They are not countable: we say uncountableTheorem:The set of real numbers between 0 and 1 (e.g., 0.244, 0.3141592323....) is uncountable Call this set R0,1

    (Some sets are even larger. Serious set theory is all about calculations with infinite sets. Most of this is irrelevant for this course.)

  • Finally: diagonalisationTheorem: | R0,1 | > |N|. Proof strategy: | R0,1 |>=|N|. Suppose | R0,1 |=|N| and derive a contradiction: Each member of R0,1 can be written as a zero followed by a dot and a countable sequence of digits. Suppose there existed a complete enumeration of R, (using whatever order) . For example,

    e1. 0.000000000000000000000....e2. 0.010000000000000000000....e3. 0.820000000000000000000....e4. 0.171000000000000000000....e5. ...

  • Cantors diagonalisation trick:e1. 0.000000000000000000000....e2. 0.0100000000000000000000....e3. 0.8200000000000000000000....e4. 0.1710000000000000000000....... Now construct a Real number n not in the enumeration, as follows: ns first digit (after the dot) = [e1s first digit] + 1ns second digit = [e2s second digit] + 1 ...General: ns i-th difit = [e-is i-th digit] + 1 i: n differs from e-i in its i-th digitContradiction: is not a (complete) enumeration after all. QED

  • A corollary for computingThere are uncountably many languages/problemsThere are countably many TMsSince Each TM can only recognise a single languageThere are more languages than TMswe can conclude thatSome languages are not recogni