Lesson 2 Countable and Uncountable Sets

Cardinality of a set

Sets: S = {a, b}, T = {a, b, c}

|S| = 2, |T| = 3

Counting the elements in a set – one establishes a bijection (1 –1 and onto mapping) from that set into N’ where N’ N (the natural numbers, i.e. 0, 1, 2, 3, …).

Definition: A function f from a set D (the domain to a set C ( codomain ), written f: D C is a relation such that:

i. f(di) = cj and f(di) = ck cj = ck

alternately, one may write

(di, cj) f and (di, ck) f cj = ck di D

ii. di D, cj C : f(di) = cj

i.e. every element in the domain has an image

In the definition for a function, if only property (i) is satisfied then we have a partial function as opposed to a (total) function.

f1

D CThis arrow diagram depicts a relation that is not a function. Explain why not. Is f1 a partial function?

f2

D C

Is this relation a function? Why or why not? Is f2 a partial function?

1

d1

d2

c1

c2

c3

d1

d2

c1

c2

Definition: A function f is said to be injective ( or 1 – 1 ) if f(di) = ck and f(dj) = ck implies that di = dj. ck is said to be the image of di under f, and di is called the pre-image of ck, written f-1(ck) = di. So, we may say that a function f is injective if pre-images (where they exist!) are unique.

f3 f4

D C D C

Diagrams of f3 and f4

Are both of these relations examples of functions? … Are you sure?

Is either (or both?) an example of an injective function?

Definition: A function is said to be surjective (onto) if cj C, di D, : f-1(cj) = di

i.e. Every element in the codomain has a (not necessarily unique) pre-image.

Is either f3 or f4 an example of a surjective (onto) function?

Definition: The range of a function, R is that subset of the codomain for which pre-images exist.

i.e. R = {cj C | di D with f(di) = cj}subset o

What is the range of f3? And that of f4?

Give an alternative definition for a surjective function.

Definition: A function f is said to be bijective if it is both injective and surjective, ie. There is a 1 – 1 correspondence between the elements in D and C.

2

d1

d2

c1

d1

d2

c1

c2

c3

f5

D C

A bijection from D to C

Functions – Practice Problems

1. Which of the following relations are functions?

a) = {(x, x2 + x) | x is a real number }b) = {(x2, x) | x is a real number}c) = {( x3, x) | x is a real number }d) = {(log x, x) | x is a real number }e) = {(x, log x) | x is a real number }

2. Let A be the set {0, 1, 2, 3, 4}. For each relation on A given below, decide if P is ai) partial functionii) total functioniii) one-to-one functioniv) onto function

a) = {(0, 4), (3, 2), (1, 3), (2, 3), (4,1)}b) = {(3, 1), (2, 0), (1, 4), (4, 3), (0, 2)}c) = {(0, 3), (1, 4), (3, 2), (2, 0), (1, 4)}d) = {(1, 3), (3, 2), (4, 3), (0, 0), (3, 2)}

We now return to our previous discussion concerning the cardinality of a set. Suppose S = {a, b}, then to obtain the cardinality of S we count its elements , i.e. we establish a bijection from the elements of S into N’ where N’ N, the set of natural numbers.

S: f(a) = 1 //less trouble to start at 1f(b) = 2

Hence we conclude that the cardinality of S is two. Problems from Machine, Languages, and Computation by Denning, Dennis, and Qualitz; Prentice - Hall

3

d1

d2

c1

c2

We observe that it is often possible to compare the cardinality of two sets without explicitly counting the elements of either.

Example: Let P be the set of people at Yankee StadiumLet C be the set of seats at Yankee Stadium

Then we know:| P | < | C | when ……………..…..……| P | > | C | when ………………………| P | = | C | when ………………………

Definition: A set A is finite with cardinality n N if there is a bijection from the set {0, 1, …, n – 1} to A. A set is infinite if it is not finite.

Theorem: The set N of natural numbers is infinite.

Proof: To prove N is not finite, we must show that there is no n N such that a bijection exists from {0, 1, 2, …, n – 1} to N. Let n be any element of N and f an arbitrary function from {0, 1, 2, …, n – 1} to N . Let

K = 1 + max {f(0), f(1), …, f(n – 1)}.

Then K N, but for every x {0, 1, 2, …, n – 1}, f(x) K. Hence, f cannot be a surjection and not a bijection. Since n and f were chosen arbitrarily, we conclude that N is infinite.

It is often easier to prove that a set is infinite by using the following alternative definition.

Definition: A set A is infinite if there exists an injection f : A A such that f (A) is a proper subset of A. A set is finite if it is not infinite.

Theorem: The set N of natural numbers is an infinite set.

Proof: The map f: N N defined by f(x) = 2x is an injection whose image is the proper subset of nonnegative even integers.

4

Examples

a) The set of real numbers, R is infinite Consider:

f: R R,f(x) = x + 1 if x 0f(x) = x if x < 0

Then f is an injection and f(R) = {x | x R and x [0, 1)}

b) The closed interval [0, 1] is infinite. The function f: [0, 1] [0, 1] defined by f(x) = x/2 is an injection whose image is the proper subset [0, ½]

c) Let = {a, b}. Then * is infinite.

Let f: * * be defined by f(x) = ax, then f is an injection and the image of f is the proper subset of * which contains all strings beginning with the letter a.

Definition: A set A is of cardinality 0, denoted |A | = 0, if there is a bijection from N to A.

// 0 Aleph-naught is the first letter of the Hebrew alphabet

Definition: A set A is countably infinite if |A | = 0. The set A is countable, or denumerable, if it is either finite or countably infinite. The set A is uncountably, or uncountably infinite, if it is not countable.

Examples:The set of positive integers, Z+ = {1, 2, 3, …} is countably infinite (has cardinality 0)

The function f : N Z defined by f(x) = x + 1 is a bijection.

Would you argue that |Z| > |N| - i.e. Z is uncountably infinite based on the following:

5

f : N Z+ { 0 } defined by f(x) = x, i.e. the identity function is a bijection from N to just part of Z, the non-negative part. And hence |Z| 2 |N| ????

Hint – re-read the first definition above

Definition: Let A be a set. An enumeration of A is a surjective function f from an initial segment of N to A. if f is injective as well (and therefore bijective), then f is an enumeration without repetitions; if f is not injective , the f is an numeration with repetitions.

Examples:(i) If A = {a, b, c}, then a, b, a, c and b, c,

a are both finite enumerations of A, the first with repetitions and the second without.

(ii) Let A be the set of even natural numbers. Then0, 2, 4, … and2, 0, 6, 4, 10, 8, … are both enumerations of A.

The second enumeration is f(n) = 2n + 1 if n is even and fn = 2n - 1 if n is odd.

Theorem: A set A is countable if and only if there exist an enumeration of A.

Definition: Let be a finite alphabet with an associated alphabetic (linear) order, and let || x || denote the length of x . Then x y in the standard ordering of if: (i) ||x|| < ||y|| or (ii) ||x|| = ||y|| and x precedes y in the lexicographical ordering of .

Examples:(i) The set is countably infinite for any finite alphabet . The enumeration of

in standard order is (when = {a, b})

, a, b, aa, ab, ba, bb, aaa, aab, …

(ii) The set of positive rational numbers Q + is countably infinite. Clearly, Q + is not finite (why not?!).

6

We can show that Q+ is countable by exhibiting an enumeration with repetitions. The order of the enumeration is specified by the directed path in the following array: (Cantor’s First Diagonalization Proof)

NUMERATOR

1 2 3 4 5 …

1 1/1 2/1 3/1 4/1 5/1 …

2 1/2 2/2 3/2 4/2 …

DENOMINATOR 3 1/3 2/3 3/3 4/3

4 1/4 2/4 3/4

5 1/5 2/5

6 1/6

This enumeration will include every integer ratio mn, therefore it is an enumeration of Q+, and hence Q+ is countably infinite. The enumeration is with repetitions e.g. 1/2 and 2/4 denote the same element of Q+. Therefore we know a bijection from N to Q + .

Theorem: The union of a countable collection of countable sets is countable.

How might we employ this theorem to prove that Q is countably infinite?

Not all sets are countable! We employ Cantor’s Second Diagonalization Proof below.

Theorem: The subset of real numbers over [0, 1] is uncountable (or uncountably infinite)

Proof: Recall that [0, 1] denotes the set {x | x R 0 x 1}. Each x [0, 1]

can be represented by an infinite decimal expressionx = .x0x12x3………

where each xi is a decimal digit. We note that the representation is not unique, for example:

.5000…….. = .4999………

Let x = .4999….. then 10x = 4.999………

7

100x= 49.999………

and then 100x – 10x = 45 from which it follow that x = .5

f: N [0, 1] be an arbitrary function from the natural numbers to the set [0, 1]. Arrange the elements f(0), f(1), … in a vertical array, using a decimal representation for each value f(x)

The resulting array appears as follows:f(0) : .x00x01x02 … …f(1) : .x10x11x12 … … :

: : f(n) : .xn0xn1xn2 … …

where xni is the ith digit in the decimal expansion of f(n).

We now specify a real number y [0, 1] as follows:Y = .y0y1y2 … …, where

yi = xii (the 9’s complement of xii)

The number y is determined by the digits on the diagonal of the array. Clearly, y [0, 1].

However, y differs from each f(n) in at least one digit of the expansion (namely the nth digit). Hence, y f(n), for any n , and we conclude that the mapping f : N [0, 1] is not surjective f is not an enumeration of [0, 1]. Since f was arbitrary, we may conclude | [0, 1] | 0

Theorem: If is a finite nonempty alphabet, then is uncountably infinite.

(Recall that a language L , hence may be viewed as a set of all languages over , i.e. = L = {Li | Li }).

Proof: Start by letting w0, w1, w2, … be an enumeration of and let L0, L1, L2, … be an enumeration of languages over .

One must show that there is a language over which is not in the enumeration.

w0 w1 w2 ……

8

L0 a00 a01 a02 ……

L1 a10 a11 a12 ……

L2 a20 a21 a22 …… : : :

This is a binary matrix (possibly infinite). Let the ith row represent the characteristic function of Li. Then a1j = XLi(wj), i.e. aij = 1 if wj Li, otherwise aij = 0.

Finish this proof.

When you are done with the preceding proof, you will have an important result.

Recall that is countably infinite. But we write our algorithm with strings of symbols (whether in VB, Java, C++ etc.). Hence the number of possible algorithms is countably infinite.

We also showed in lesson 1 an equivalence between problems and languages. Hence we have shown that the number of languages over some alphabet (and correspondingly the number of problems) is uncountably infinite.

Hence there are more problems than there are possible algorithms. Therefore, there must exist problems with no possible solution. We shall investigate several such unsolvable problems later in this course.

Definition: A set A is of cardinality c (is uncountably infinite, or equivalently has the same cardinality as the continuum) if there is a bijection from [0, 1] to A.

Examples/problems(i) | [a, b] | = c where [a, b] is any closed interval in R with a < b.

Note: f(x) = (b – a)x + a is a bijection from [0, 1] to [a, b].

(ii) Show that is countably infinite where = {a}.

(iii) Show that the set {x, y | x, y R x2 + y2 = 1} is uncountably infinite (has cardinality c).

9

Post Correspondence Problem

An instance of Post’s Correspondence Problem (PCP) consists of two lists

A = w1, w2, …, wk

B = x1, x2, …, xk

of strings over some alphabet . This instance of the PCP has a solution if there is any sequence of integers i1, i2, i3, …, im, with m 1 such that:

wi1, wi2, …, wim = xi1, xi2, …, xim.

The sequence i1, i2, i3, …, im is a solution to this instance of the PCP.

Examples

(i) Let = {0, 1} and let A and B be lists of three strings each as defined below.

List A List Bi wi xi

123

1101110

111100

This instance of the PCP has a solution.

Let m = 4, i1 = 2, i2 = 1, i3 = 1 and i4 = 3, then

w2w1w1w3 = x2x1x1x3 = 101111110

10

(ii) Let = {1, 0}. Let A and B be lists of three strings as shown below:

List A List Bi wi xi

123

10011101

10111011

Suppose that this instance of the PCP has a solution:

i1, i2, …, im.

i1 must equal 1 //Why?

So far we have from list A: 10

from list B: 101

The next selection from A must begin with a 1. Therefore, i2 = 1 or i3 = 3. But i2 = 1 won’t work. //Why not?

With i2 = 3 we have 10101101011

Clearly i3 = i4 = … = 3. However string B is always one character longer. This instance of PCP has no solution.

It is undecidable whether an arbitrary instance of the PCP has a solution.

A proof of this will have to wait several months.

However, for now we can employ some intuition

We require first one unsolvable problem. In 304 we have seen that the Turing machine halting problem is undecidable, i.e. given an arbitrary Turing machine M with arbitrary input tape w, it is undecidable if M(w) will halt.

Second, we must reduce the Turing machine halting problem to the PCP. Then a solution for the PCP would mean that a solution exists for the Turing machine halting problem.

But it doesn’t.

11

Hence the PCP must be undecidable as well.

Essentially, the string in the PCP will correspond to portions of Turing machine instantaneous descriptions.

Then two i.d.’s will match iff the Tm would enter its halt state. But we cannot a priori know this

… hence ………..

12