david m keil, framingham state university csci 460 theory of
TRANSCRIPT
1. Sets, languages, and models David Keil Theory of computing 6/12
David Keil Theory of Computing 12/12 1
1. Sets, languages,
and models 1. Countable sets and formal languages
2. Uncountable sets
3. Streams and coinduction
4. Circuit and lookup computation
David M Keil, Framingham State University
CSCI 460 Theory of Computing
Inquiry
• Are the sets of natural numbers and
strings of comparable size?
• Are all infinite sets the same size?
• Can we define infinite sets of infinitely
large data streams?
David Keil Theory of Computing 12/12 2
1. Sets, languages, and models David Keil Theory of computing 6/12
Objectives
David Keil Theory of Computing 12/12 3
1a. Use the notation of formal language
theory*
1b. Describe the logic-circuit model of
computation*
1c. Prove that a set is countable
1d. Prove that a set is uncountable
1e. Explain or write a coinductive definition
How Topic 1 relates to computing
• We show that some problems are
algorithmically unsolvable
• We show a diagonal proof method that is
used again in topic 4
• Computation deals with finite, infinite, and
uncountable objects or sets
• Computations are operations on sets of
strings (languages)
David Keil Theory of Computing 12/12 4
1. Sets, languages, and models David Keil Theory of computing 6/12
• How big is a set?
• What does countability have to do with
computing?
• How many of the following exist?
– natural numbers
– rational numbers
– real numbers
– functions
– programs
– sets David Keil Theory of Computing 12/12 5
1. Countable sets and formal languages
Countability • The natural numbers are countable, i.e.,
they can be arranged in a particular order
• A bijection is a one-to-one correspondence,
i.e., a relation between two sets that is both
surjective and injective
• A set is countable iff there exists a bijection
between it and the natural numbers (ℕ)
• Countable sets tend to be sets of finite
objects, defined inductively
David Keil Theory of Computing 12/12 6
1. Sets, languages, and models David Keil Theory of computing 6/12
Peano’s axioms: definition by induction
1. 0 is a natural number (0 N)
2. Every natural number n has a unique
successor, n, also a natural number
(n N) n N
3. All natural numbers follow (1) or (2)
(n N) n = 0 (m N) n = m
• Significance: These axioms, or assumptions, provide a formal logical basis to work with counting numbers. Note: (2) is recursive
• Computation is a formal way to manipulate numbers and objects representable by them.
7 David Keil Theory of Computing 12/12
Theorem: ℕ is infinite Proof by contradiction:
1. Suppose ℕ is finite
2. Then let n be the largest element of ℕ
3. Consider the number (n + 1). It cannot be in ℕ,
because it is larger than ℕ’s largest element
4. But by definition of ℕ, any successor of an
element of ℕ is in ℕ.
5. Hence by the contradiction of (3) and (4), we
must reject (1).
6. Hence ℕ is infinite.
David Keil Theory of Computing 12/12 8
1. Sets, languages, and models David Keil Theory of computing 6/12
Strings • Alphabet: a finite set of symbols, e.g.,
{0,1},
{0, 1, …9}, {a, b, c, …, z}
• A string is a sequence of symbols over a
finite alphabet
• Notation:
– xR is the string x reversed
– n0(x) is the number of zeroes in string x
David Keil Theory of Computing 12/12 9
The set of strings over an alphabet
• * is the set of strings over finite alphabet
• Recursive definition:
– Base: * ( = = null string)
– Recursive: a s * sa *
– Restriction: only objects defined as above
are strings
• In set notation,
* = {} { ax | a , x *}
David Keil Theory of Computing 12/12 10
1. Sets, languages, and models David Keil Theory of computing 6/12
A bijection ℕ {0,1}*
David Keil Theory of Computing 12/12 11
Str(n) =
if n = 0
Str(n/2) + ‘0’ if Odd(n) n > 0
Str(n/2) + ‘1’ otherwise
Num(s) =
0 if s =
2 Num(s[1 .. |s| - 1]) +
s[|s|] + 1 otherwise
• This bijection is the core of a proof that
{0,1}* is countable
Cardinalities of sets • Informally, their sizes
• For infinite sets, we say that two sets have
the same cardinality iff there is a bijection
(1-to-1 correspondence) between their
elements
• Example: Card (EVEN) = Card (ℕ)
Proof: EVEN0 = 0, EVEN1 = 2, etc.
• Example: Card (ℚ) = Card (ℕ)
Proof: Zig-zag through a table of rationals
David Keil Theory of Computing 12/12 12
1. Sets, languages, and models David Keil Theory of computing 6/12
David Keil Theory of Computing 12/12 13
Formal languages • Language: a set of strings over an alphabet
• Languages may be enumerated:
, 0, 1, 00, 01, 10, 11, …
• * is the set of all strings over
• For any language L over , L *
• Inductively, * = {xy | x y *}
Defining languages
• 0n1n is the language of strings with n zeroes
followed by n ones
• Let alphabet be {0,1}, let be the null
string
• Then 0 = {}, 1 = {(0), (1)}
2 = {00, 01, 10, 11}
k = the set of strings of length k
* = Uk N k
David Keil Theory of Computing 12/12 14
1. Sets, languages, and models David Keil Theory of computing 6/12
Operations on languages
• Concatenation (L1 L2 ): strings that are a
string in L1 followed by one in L2
• L1 L2 = {xy | x L1 y L2}
• Union (selection, L1 L2, L1 | L2):
strings either in L1 or L2
L1 L2 = {x | x L1 x L2}
• Intersection (L1 L2) strings that are in
both L1 and L2
David Keil Theory of Computing 12/12 15
Kleene star languages
• Kleene star (Iteration, L*): strings that are
sequences of strings in L
L* = {xy | x L y L*}
• Alternative definition:
– L0 = {}, L1 = L, L2 = LL, L3 = LLL, etc.
– LL ={xx | x L}
– L* = Uk N Lk
– Example: If L = {0, 00} then
LL = {00, 000, 0000}
David Keil Theory of Computing 12/12 16
1. Sets, languages, and models David Keil Theory of computing 6/12
Countability of strings • To help prove equivalence of number-based
and string-based models of computation, we
show mappings of any string to a natural
number and any natural number to a string
• Enumeration according to bit representation:
N {0,1}* 0
1 0
2 1
3 00
4 01
David Keil Theory of Computing 12/12 17
Languages and countability
• Computation operates on strings;
instances of models of computation
accept sets of strings
• A language may be represented as an
infinite bit sequence, with a 1 bit in a
certain bit location denoting a string’s
membership in the language:
{0, 01, 11, 100} = 10101100…
David Keil Theory of Computing 12/12 18
1. Sets, languages, and models David Keil Theory of computing 6/12
Countable set: the rational numbers
• The rationals may
be enumerated as
at right, where the
numerator is the
row number and
the denominator is
the column number
• Reference: “The set of rational numbers is countable,”
(http://www.homeschoolmath.net/teaching/rational-
numbers-countable.php)
David Keil Theory of Computing 12/12 19
2. Uncountable sets
• Are all infinite sets the same size?
• Does infinity matter in the real
finite world?
David Keil Theory of Computing 12/12 20
1. Sets, languages, and models David Keil Theory of computing 6/12
Uncountability
• We show that some sets, such as the
real numbers, are not countable
• We prove this by showing that no
bijection between such a set and the set
of natural numbers can exist
• This proof and its method imply limits
on the power of algorithmic
computation
David Keil Theory of Computing 12/12 21
The set of real numbers • Intuitively, the real numbers express analog or
continuous quantities; they give all the
possible weights or distances.
• We may consider the size of the set ℝ, of reals,
as corresponding to all the points on a line
segment, line, plane, space, or space/time
• Another definition: The reals are all the
numbers that can be expressed using an
infinite sequence of digits after the decimal or
binary point
David Keil Theory of Computing 12/12 22
1. Sets, languages, and models David Keil Theory of computing 6/12
Theorem: |ℝ| > |ℕ| Proof (Cantor) :
1.Suppose ℝ were countable
2.Then the interval (0, 1] could be enumerated as:
r1 = 0 . b1,1 b1,2 b1,3 …
r2 = 0 . b2,1 b2,2 b2,3 …
… where bm,n {0,1}
3.Now, consider the real number s =
0. b1,1 b2,2 b3,3 …
That is, for all n, bit n is bn,n (bitwise negation
of the diagonal of r)
David Keil Theory of Computing 12/12 23
Cantor’s proof (cont’d)
4. Now, since for every i, the ith bit of s is
different from the ith bit of ri, so s differs
from every element of r, so there is no
element of sequence r that matches s
5. Hence r does not contain s, which is
clearly a real number
6. Hence we have a contradiction and must
reject the supposition
7. Hence ℝ is not countable
David Keil Theory of Computing 12/12 24
1. Sets, languages, and models David Keil Theory of computing 6/12
Cantor’s proof, illustrated Supposed enumeration of ℝ:
r1 = (0, 0, 0, 0, 0, 0, 0, ...)
r2 = (1, 1, 1, 1, 1, 1, 1, ...)
r3 = (0, 1, 0, 1, 0, 1, 0, ...)
r4 = (1, 0, 1, 0, 1, 0, 1, ...)
r5 = (1, 1, 0, 1, 0, 1, 1, ...)
r6 = (0, 0, 1, 1, 0, 1, 1, ...)
r7 = (1, 0, 0, 0, 1, 0, 0, ...)
... David Keil Theory of Computing 12/12 25
Real number that differs
with every element in
enumeration in at least
one bit:
s = (1, 0, 1, 1, 1, 0, 1, ...)
Summary of Cantor’s proof
• “The proof shows that no matter how a list
of real numbers were arranged, a real
number could still be defined that would not
be in the list”
T. DiRienzo, J. Bartlett, 2/09
• A diagonal proof is both by construction and
by contradiction
David Keil Theory of Computing 12/12 26
1. Sets, languages, and models David Keil Theory of computing 6/12
The predicates are uncountable • Start with the set of functions
{ f : {0} {0,1}}
• This is a set of two functions:
{{(0,1)}, {(0,0)}}
• The set { f : {0, 1} {0,1}} has four elements,
{ f : {0, 1, 2} {0,1}} has eight, etc.
• There are 2|ℕ| predicates f : ℕ {0,1}
• Question: Is this the same as, or more than, the
cardinality of the natural numbers?
• Compare with the cardinality of reals
David Keil Theory of Computing 12/12 27
Can all predicates on ℕ be
computed in Java? 1. Enumerate all Java methods that take an integer
parameter and return 0 or 1, as J1, J2, J3, … in a
bitwise ordering
2. Consider the predicate f : ℕ {0,1} s.t. f (n) = 1
if Jn(n) = 0 or Jn(n) hangs, f (n) = 0 if Jn(n) = 1
3. The predicate f differs in its behavior from each
element of our enumeration of Java methods
4. Hence f is a predicate not computed by any Java
method
David Keil Theory of Computing 12/12 28
1. Sets, languages, and models David Keil Theory of computing 6/12
Results
• ℕ is countable (cardinality 0, aleph-null)
• ℝ is uncountable (cardinality 1)
• The set of Java methods is countable using
their bit representations
• The predicates f : ℕ {0,1} are uncountable
• There are 20 = 1 predicates on ℕ
• The sets of natural numbers are uncountable
(see handout)
David Keil Theory of Computing 12/12 29
3. Streams and coinduction
30 David Keil Theory of Computing 12/12
• Can we define infinite sets of infinitely
large data streams?
• How can infinite I/O be described
mathematically?
• What is induction without a base case?
1. Sets, languages, and models David Keil Theory of computing 6/12
Streams
• A reactive system may receive an infinite
stream of inputs and may emit the
corresponding infinite stream of outputs
• Sets of streams express interactive behavior
• Coinduction defines sets of infinite objects
as induction defines sets of finite ones
David Keil Theory of Computing 12/12 31
ProgramEnvironmentinputs
Interaction stream
outputs
Inductively defined sets • (i) 0 is a natural number;
(ii) Every n ℕ has a unique successor, n; (iii) i and ii are the only ways to obtain a
natural number (Peano)
• Set of expressions using numerals, +, and ( ):
expression numeral |
numeral + expression | ( expression )
• The definition of such a set may use itself
• But such a set does not contain itself
32 David Keil Theory of Computing 12/12
1. Sets, languages, and models David Keil Theory of computing 6/12
David Keil Theory of Computing 12/12 33
Inductive and coinductive definitions of languages
• * = {} U {ax | a , x *}
(note base case )
• “L is a language over alphabet *”
means that L *
• A stream is an infinite string
• A stream language is a set of streams
• Example, defined coinductively:
= {ax | a , x }
(note lack of base case)
Well-founded sets • (B. Russell) Let the village hair stylist be
the person who cuts the hair of everyone
who doesn’t cut own hair
• Question: Who cuts the hair-stylist’s hair?
• Similar Questions: Is there a set of all sets?
If so, does it contain itself ? Is there a set of
all sets that don't contain themselves?
• In classical (well founded) set theory, it is
meaningless to say a set belongs or doesn't
belong to itself (Foundation Axiom)
34 David Keil Theory of Computing 12/12
1. Sets, languages, and models David Keil Theory of computing 6/12
David Keil Theory of Computing 12/12 35
Another diagonal proof • Theorem: The notion “set of all sets” leads
to a contradiction
• Proof: Consider = { A : A A } (sets not members of themselves)
• ( ) ( ) , ( ) ( ) , contradictions
• Note role of “Spoiler” instance
• We say that , are not well founded (NWF)
Streams and non-well-founded sets
• A non-well-founded set may contain itself,
directly or indirectly
• Example: A = { B, C }; B = { A, D }
• Every stream of characters is a character,
followed by a stream of characters
• Streams contain streams, which contain
streams, which contain streams, ...
• Ex.: The set of all bit streams {0,1} consists
of any infinite sequence that consists of 0 or 1
followed by a stream
36 David Keil Theory of Computing 12/12
1. Sets, languages, and models David Keil Theory of computing 6/12
Streams and coinduction
• Induction defines countable sets of finite
objects: * = {} { ax | a , x *}
• Coinduction
– is a dual of induction (recursion), lacking a base case
– is used to define sets of infinite objects
• The set of streams over an alphabet, express-
ing ongoing processes such as interaction:
= { ax | a , x }
37 David Keil Theory of Computing 12/12
Least and greatest fixed points • A fixed point x of function f is a value x
(which may be a set), for which f (x) = x
• Whereas induction is characterized by
minimality conditions (least fixed points),
coinduction has a maximality condition
(greatest fixed point)
• Minimality example: + is the smallest set that
fits the spec {ax | a , x *}
• Maximality example: is the largest set that
fits the spec {ax | a , x }
David Keil Theory of Computing 12/12 38
1. Sets, languages, and models David Keil Theory of computing 6/12
• What is a function on a finite set?
• What is a property of a finite set?
• How can a function on a finite domain be
defined?
• What’s a way to compute such a function?
• What’s a simple model of simple
computation?
David Keil Theory of Computing 12/12 39
4. Circuit and lookup computation
Problems and languages • A computational problem with a Boolean
(yes/no) solution is called a decision problem
• Computing devices that solve them are called
acceptors or recognizers
• A decision problem corresponds to a
language: all the strings that get a “yes”
decision
• Other problems, with string solutions, are
called transduction problems
David Keil Theory of Computing 12/12 40
1. Sets, languages, and models David Keil Theory of computing 6/12
Decision and transduction • A decision problem requires yes/no responses
to input
• Hence it is also known as a language-
recognition problem
• A transduction computation produces a string
as output
• Hence its solution is the computation of a
function
• We speak of decidable languages (problems)
or computable functions
41 David Keil Theory of Computing 12/12
Some language-decision problems
• Some languages:
– 1 | 0 {0, 1}
– (1 | 0) 1 {11, 01}
– 1* {, 1, 11, …}
– 1*0 {0, 10, 110, 1110, …}
– 0(1|0)* {0, 00, 01, 000, 001,
010, 011, …}
• Q: How do we build a simple machine to
accept strings in one of these languages?
David Keil Theory of Computing 12/12 42
1. Sets, languages, and models David Keil Theory of computing 6/12
No-loop computation • The simplest type of computation uses no
loops, but maps from finite domains
• Examples: lookup table; logic circuit
• Other equivalent models:
– Finite computation trees
– Flowcharts with bounded input, no loops
– Formulas in propositional logic
– Finite languages L k for any or k
David Keil Theory of Computing 12/12 43
Logic circuits • A model for the computation
of functions f : {0,1}m {0,1}n
where m is the arity of f
(number of inputs to circuit)
and n is the number of outputs
• Proposition: for any such function f, a logic
circuit that computes f may be constructed from a
finite number of , , or gates
• A logic circuit computes a function on strings of
bounded size, i.e., functions with a finite domain
David Keil Theory of Computing 12/12 44
1. Sets, languages, and models David Keil Theory of computing 6/12
Lookup tables
David Keil Theory of Computing 12/12 45
• A function on a finite domain may be
represented and effectively computed
using a lookup table
a b a xor b
0 0 0 0 1 1 1 0 1 1 1 0
Square function XOR operator
Labeled transition systems • Definition: a digraph, where vertices denote
state, labeled edges denote transitions
• Example (right): an input string
of bits is read from the
top, following edges
• A loopless transition
system can accept
any finite language of strings that leave the
system in an accepting state
David Keil Theory of Computing 12/12 46
1. Sets, languages, and models David Keil Theory of computing 6/12
Transducers and accepters • Theorem:Any computation of a function
f : {0, 1}* {0, 1}n can be done by n accepters
• Proof:
1. for i n, let Li = {x | bit i of f(x) is 1}
2. Construct accepters M1 .. i for L1 .. i as follows.
3. Mi decides the ith bit of f (x) for any string x.
4. Let Pi be the predicate computed by Mi
5. So f (x) = Concati m(Pi(x))
6. Hence f (x) is computable by M1..n
David Keil Theory of Computing 12/12 47
Barwise-Moss. Vicious Circles, 1996
S. Epp. Discrete Mathematics with
Applications. Brooks/Cole, 2011.
Finsler, 1920s (NWF sets)
Goldin-Wegner (at www.engr.uconn.edu/~dqg)
Peano, 1889 (induction)
J. Savage.
David Keil Theory of Computing 12/12 48
References