appendix a - guy lebanon's websitetheanalysisofdata.com/probability/a/a.pdfappendix a set...

14
Appendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern mathematics. A.1 Basic Definitions Definition A.1.1. A set is an unordered collection of elements. Sets may be described by listing their elements between curly braces, for example {1, 2, 3} is the set containing the elements 1, 2, and 3. Alternatively, we an describe a set by specifying a certain condition whose elements satisfy, for example {x : x 2 =1} is the set containing the elements 1 and 1 (assuming x is a real number). We make the following observations. There is no importance to the order in which the elements of a set appear. Thus {1, 2, 3}, is the same set as {3, 2, 1}. An element may either appear in a set or not, but it may not appear more than one time. Sets are typically denoted by an uppercase letter, for example A or B. It is possible that the elements of a set are sets themselves, for example {1, 2, {3, 4}} is a set containing three elements (two scalars and one set). We typically denote such sets with calligraphic notation, for example U . Definition A.1.2. If a is an element in a set A, we write a 2 A. If a is not an element of A, we write a 62 A. The empty set, denoted by ; or {}, does not contain any element. 223

Upload: others

Post on 14-Oct-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

Appendix A

Set Theory

This chapter describes set theory, a mathematical theory that underlies all ofmodern mathematics.

A.1 Basic Definitions

Definition A.1.1. A set is an unordered collection of elements.

Sets may be described by listing their elements between curly braces, forexample {1, 2, 3} is the set containing the elements 1, 2, and 3. Alternatively,we an describe a set by specifying a certain condition whose elements satisfy, forexample {x : x2 = 1} is the set containing the elements 1 and −1 (assuming x isa real number).

We make the following observations.

• There is no importance to the order in which the elements of a set appear.Thus {1, 2, 3}, is the same set as {3, 2, 1}.

• An element may either appear in a set or not, but it may not appear morethan one time.

• Sets are typically denoted by an uppercase letter, for example A or B.

• It is possible that the elements of a set are sets themselves, for example{1, 2, {3, 4}} is a set containing three elements (two scalars and one set).We typically denote such sets with calligraphic notation, for example U .

Definition A.1.2. If a is an element in a set A, we write a 2 A. If a is notan element of A, we write a 62 A. The empty set, denoted by ; or {}, does notcontain any element.

223

Page 2: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

224 APPENDIX A. SET THEORY

Definition A.1.3. A set A with a finite number of elements is called a finiteset and its size (number of elements) is denoted by |A|. A set with an infinitenumber of elements is called an infinite set.

Definition A.1.4. We denote A ⇢ B if all elements in A are also in B. Wedenote A = B if A ⇢ B and B ⇢ A, implying that the two sets are identical.The di↵erence between two sets A \ B is the set of elements in A but not in B.The complement of a set A with respect to a set ⌦ is Ac = ⌦ \A (we may omitthe set ⌦ if it is obvious from context). The symmetric di↵erence between twosets A,B is

A4B = {x : x 2 A \B or x 2 B \A}.Example A.1.1. We have {1, 2, 3} \ {3, 4} = {1, 2} and {1, 2, 3}4{3, 4} ={1, 2, 4}. Assuming ⌦ = {1, 2, 3, 4, 5}, we have {1, 2, 3}c = {4, 5}.

In many cases we consider multiple sets indexed by a finite or infinite set. Forexample U

,↵ 2 A represents multiple sets, one set for each element of A.

Example A.1.2. Below are three examples of multiple sets, U↵

,↵ 2 A. Thefirst example shows two sets: {1} and {2}. The second example shows multiplesets: {1,−1}, {2,−2}, {3,−3}, and so on. The third example shows multiplesets, each containing all real numbers between two consecutive natural numbers.

Ui

= {i}, i 2 A = {1, 2},Ui

= {i,−i}, i 2 A = N = {1, 2, 3, . . .},U↵

= {↵+ r : 0 r 1}, ↵ 2 A = N = {1, 2, 3, . . .}.

Definition A.1.5. For multiple sets U↵

,↵ 2 A we define the union and inter-section operations as follows:

[

↵2A

U↵

= {u : u 2 U↵

for one or more ↵ 2 A}\

↵2A

U↵

= {u : u 2 U↵

for all ↵ 2 A}.

Figure A.1 illustrates these concepts in the case of two sets A,B with a non-empty intersection.

Definition A.1.6. The sets U↵

,↵ 2 A are disjoint or mutually disjoint if\↵2A

U↵

= ; and are pairwise disjoint if ↵ 6= β implies U↵

\ Uβ

= ;. A union ofpairwise disjoint sets U

, ↵ 2 A is denoted by ]↵2A

U↵

.

Example A.1.3.

{a, b, c} \ {c, d, e} = {c}{a, b, c} [ {c, d, e} = {a, b, c, d, e}{a, b, c} \ {c, d, e} = {a, b}.

Page 3: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

A.1. BASIC DEFINITIONS 225

A BA \B

Figure A.1: Two circular sets A,B, their intersection A\B (gray area with hori-zontal and vertical lines), and their union A[B (gray area with either horizontalor vertical lines or both). The set ⌦\(A[B) = (A[B)c = Ac\Bc is representedby white color.

Example A.1.4. If A1

= {1}, A2

= {1, 2}, A3

= {1, 2, 3} we have

{Ai

: i 2 {1, 2, 3}} = {{1}, {1, 2}, {1, 2, 3}}[

i2{1,2,3}

Ai

= {1, 2, 3}\

i2{1,2,3}

Ai

= {1}.

The properties below are direct consequences of the definitions above.

Proposition A.1.1. For all sets A,B,C ⇢ ⌦,

1. Union and intersection are commutative and distributive:

A [B = B [A, (A [B) [ C = A [ (B [ C)

A \B = B \A, (A \B) \ C = A \ (B \ C)

2. (Ac)c = A, ;c = ⌦, ⌦c = ;3. ; ⇢ A

4. A ⇢ A

5. A ⇢ B and B ⇢ C implies A ⇢ C

Page 4: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

226 APPENDIX A. SET THEORY

6. A ⇢ B if and only if Bc ⇢ Ac

7. A [A = A = A \A

8. A [ ⌦ = ⌦, A \ ⌦ = A

9. A [ ; = A, A \ ; = ;.

Definition A.1.7. The power set of a set A is the set of all subsets of A, includingthe empty set ; and A. It is denoted by 2A.

Proposition A.1.2. If A is a finite set then

|2A| = 2|A|.

Proof. We can describe each element of 2A by a list of |A| 0 or 1 digits (1 if thecorresponding element is selected and 0 otherwise). The proposition follows sincethere are 2|A| such lists (see Proposition 1.6.1).

Example A.1.5.

2{a,b} = {;, {a, b}, {a}, {b}}|2{a,b}| = 4

X

A22

{a,b}

|A| = 0 + 2 + 1 + 1 = 4.

The R package sets is convenient for illustrating basic concepts.

library(sets)A = set("a", "b", "c")2ˆA

## {{}, {"a"}, {"b"}, {"c"}, {"a", "b"},## {"a", "c"}, {"b", "c"}, {"a", "b",## "c"}}

A = set("a", "b", set("a", "b"))2ˆA

## {{}, {"a"}, {"b"}, {{"a", "b"}}, {"a",## "b"}, {"a", {"a", "b"}}, {"b", {"a",## "b"}}, {"a", "b", {"a", "b"}}}

A = set(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)length(2ˆA) # = 2ˆ10

## [1] 1024

Page 5: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

A.1. BASIC DEFINITIONS 227

Proposition A.1.3 (Distributive Law of Sets).0

@

[

↵2Q

A↵

1

A

\

C =[

↵2Q

A↵

\

C⌘

0

@

\

↵2Q

A↵

1

A

[

C =\

↵2Q

A↵

[

C⌘

.

Proof. We prove the first statement for the case of |Q| = 2. The proof of thesecond statement is similar, and the general case is a straightforward extension.

We prove the set equality U = V by showing U ⇢ V and V ⇢ U . Let xbelong to the set (A [ B) \ C. This means that x is in C and also in either Aor B, which implies x 2 (A \ C) [ (B \ C). On the other hand, if x belongs to(A \ C) [ (B \ C), x is in A \ C or in B \ C. Therefore x 2 C and also x is ineither A or B, implying that x 2 (A [B) \ C.

Proposition A.1.4.

S \[

↵2Q

A↵

=\

↵2Q

(S \A↵

)

S \\

↵2Q

A↵

=[

↵2Q

(S \A↵

).

Proof. We prove the first result. The proof of the second result is similar.Let x 2 S \ [

↵2Q

A↵

. This implies that x is in S but not in any of the A↵

sets, which implies x 2 S \A↵

for all ↵, and therefore x 2 \↵2Q

(S \A↵

). On theother hand, if x 2 \

↵2Q

(S \A↵

), then x is in each of the S \A↵

sets. Thereforex is in S but not in any of the A

sets, hence x 2 S \ [↵2Q

A↵

.

Corollary A.1.1 (De-Morgan’s Law).0

@

[

↵2Q

A↵

1

A

c

=\

↵2Q

Ac

0

@

\

↵2Q

A↵

1

A

c

=[

↵2Q

Ac

.

Proof. This is a direct corollary of Proposition A.1.4 when S = ⌦.

Definition A.1.8. We define the sets of all natural numbers, integers, and ra-tional numbers as follows:

N = {1, 2, 3, . . .}Z = {. . . ,−1, 0, 1, . . .}Q = {p/q : p 2 Z, q 2 Z \ {0}}.

Page 6: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

228 APPENDIX A. SET THEORY

The set of real numbers1 is denoted by R.

Definition A.1.9. We denote the closed interval, the open interval and the twotypes of half-open intervals between a, b 2 R as

[a, b] = {x 2 R : a x b}(a, b) = {x 2 R : a < x < b}[a, b) = {x 2 R : a x < b}(a, b] = {x 2 R : a < x b}.

Example A.1.6.

[

n2N[0, n/(n+ 1)) = [0, 1),

\

n2N[0, n/(n+ 1)) = [0, 1/2).

Definition A.1.10. The Cartesian product of two sets A and B is

A⇥B = {(a, b) : a 2 A, b 2 B}.In a similar way we define the Cartesian product of n 2 N sets. The repeatedCartesian product of the same set, denoted as

Ad = A⇥ · · ·⇥A, d 2 N,

is the set of d-tuples or d-dimensional vectors whose components are elements inA.

Example A.1.7. Rd is the set of all d dimensional vectors whose componentsare real numbers.

Definition A.1.11. A relation R on a set A is a subset of A ⇥ A, or in otherwords a set of pairs of elements in A. If (a, b) 2 R we denote a ⇠ b and if(a, b) 62 R we denote a 6⇠ b. A relation is reflexive if a ⇠ a for all a 2 A. It issymmetric if a ⇠ b implies b ⇠ a for all a, b 2 A. It is transitive if a ⇠ b andb ⇠ c implies a ⇠ c for all a, b, c 2 A. An equivalence relation is a relation thatis reflexive, symmetric, and transitive.

Example A.1.8. Consider relation 1 where a ⇠ b if a b over R and relation2 where a ⇠ b if a = b over Z. Relation 1 is reflexive and transitive but notsymmetric. Relation 2 is reflexive, symmetric, and transitive and is therefore anequivalence relation.

1One way to rigorously define the set of real numbers is as the completion of the rationalnumbers. The details may be found in standard real analysis textbooks, for example [37]. Wedo not pursue this formal definition here.

Page 7: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

A.2. FUNCTIONS 229

Definition A.1.12. The sets U↵

,↵ 2 A form a partition of U if]

↵2A

U↵

= U.

(see Definition A.1.6.) In other words, the union of the pairwise disjoint setsU↵

,↵ 2 A is U . The sets U↵

are called equivalence classes.

An equivalence relation ⇠ on A induces a partition of A as follows: a ⇠ b ifand only if a and b are in the same equivalence class.

Example A.1.9. Consider the set A of all cities and the relation a ⇠ b if thecities a, b are in the same country. This relation is reflexive, symmetric, andtransitive, and therefore is an equivalence relation. This equivalence relationinduces a partition of all cities into equivalence classes consisting of all cities inthe same country. The number of equivalence classes is the number of countries.

A.2 Functions

Definition A.2.1. Let A,B be two sets. A function f : A ! B assigns oneelement b 2 B for every element a 2 A, denoted by b = f(a).

Definition A.2.2. For a function f : A ! B, we define

range f = {f(a) : a 2 A} (A.1)

f−1(b) = {a 2 A : f(a) = b} (A.2)

f−1(H) = {a 2 A : f(a) 2 H}. (A.3)

If range f = B, we say that f is onto. If for all b 2 B, |f−1(b)| 1 we say thatf is 1-1 or one-to-one. A function that is both onto and 1-1 is called a bijection.

If f : A ! B is one-to-one, f−1 is also a function f−1 : B0 ! A whereB0 = range f ⇢ B. If f : A ! B is a bijection then f−1 : B ! A is also abijection.

Definition A.2.3. Let f : A ! B and f : B ! C be two functions. Theirfunction composition is a function f ◦ g : A ! C defined as (f ◦ g)(x) = f(g(x)).

Proposition A.2.1. For any function f and any indexed collection of setsU↵

,↵ 2 A,

f−1

\

↵2A

U↵

!

=\

↵2A

f−1(U↵

)

f−1

[

↵2A

U↵

!

=[

↵2A

f−1(U↵

)

f

[

↵2A

U↵

!

=[

↵2A

f(U↵

).

Page 8: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

230 APPENDIX A. SET THEORY

Proof. We prove the result above for a union and intersection of two sets. Theproof of the general case of a collection of sets is similar.

If x 2 f−1(U \ V ) then f(x) is in both U and V and therefore x 2 f−1(U) \f−1(V ). If x 2 f−1(U) \ f−1(V ) then f(x) 2 U and f(x) 2 V and thereforef(x) 2 U \ V , implying that x 2 f−1(U \ V ).

If x 2 f−1(U [ V ) then f(x) 2 U [ V , which implies f(x) 2 U or f(x) 2 Vand therefore x 2 f−1(U)[ f−1(V ). On the other hand, if x 2 f−1(U)[ f−1(V )then f(x) 2 U or f(x) 2 V implying f(x) 2 U [ V and x 2 f−1(U [ V ).

If y 2 f(U [ V ) then y = f(x) for some x in either U or V and y = f(x) 2f(U) [ f(V ). On the other hand, if y 2 f(U) [ f(V ) then y = f(x) for some xthat belongs to either U or V , implying y = f(x) 2 f(U [ V ).

Interestingly, the statement f(U \ V ) = f(U) \ f(V ) is not true in general.

Example A.2.1. For f : R ! R, f(x) = x2, we have f−1(x) = {px,−px}

implying that f is not one-to-one. We also have f−1([1, 2]) = [−p2, 1] [ [1,

p2].

Definition A.2.4. Given two functions f, g : A ! R we denote f g if f(x) g(x) for all x 2 A, f < g if f(x) < g(x) for all x 2 A, and f ⌘ g if f(x) = g(x)for all x 2 A.

Definition A.2.5. We denote a sequence of functions fn

: A ! R, n 2 Nsatisfying f

1

f2

f3

· · · as fn

%.

Definition A.2.6. Given a set A ⇢ ⌦ we define the indicator function IA

: ⌦ !R as

IA

(!) =

(

1 ! 2 A

0 otherwise, ! 2 ⌦.

Example A.2.2. For any set A, we have IA

= 1− IA

c .

Example A.2.3. If A ⇢ B, we have IA

IB

.

A.3 Cardinality

The most obvious generalization of the size of a set A to infinite sets (see Defini-tion A.1.3) implies the obvious statement that infinite sets have infinite size. Amore useful generalization can be made by noticing that two finite sets A,B havethe same size if and only if there exists a bijection between them, and generalizingthis notion to infinite sets.

Definition A.3.1. Two sets A and B (finite or infinite) are said to have thesame cardinality, denoted by A ⇠ B if there exists a bijection between them.

Note that the cardinality concept above defines a relation that is reflexive(A ⇠ A) symmetric (A ⇠ B ) B ⇠ A), and transitive (A ⇠ B and B ⇠ Cimplies A ⇠ C) and is thus an equivalence relation. The cardinality relation

Page 9: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

A.3. CARDINALITY 231

thus partitions the set of all sets to equivalence classes containing sets with thesame cardinality. For each natural number k 2 N, we have an equivalence classcontaining all finite sets of that size. But there are also other equivalence classescontaining infinite sets, the most important one being the equivalence class thatcontains the natural numbers N.

Definition A.3.2. Let A be an infinite set. If A ⇠ N then A is a countablyinfinite set. If A 6⇠ N then A is a uncountably infinite set.

Proposition A.3.1. Every infinite subset E of a countably infinite set A iscountably infinite.

Proof. A is countably infinite, so we can construct an infinite sequence x1

, x2

, . . .containing the elements of A. We can construct another sequence, y

1

, y2

, . . .,that is obtained by omitting from the first sequence the elements in A \ E. Thesequence y

1

, y2

, . . . corresponds to a bijection between the natural numbers andE.

Proposition A.3.2. A countable union of countably infinite sets is countablyinfinite.

Proof. Let An

, n 2 N be a collection of countably infinite sets. We can arrangethe elements of each A

n

as a sequence that forms the n-row of a table with infiniterows and columns. We refer to the element at the i-row and j-column in thattable as A

ij

. Traversing the table in the following order: A11

, A21

, A12

, A31

,A

22

, A13

, and so on (traversing south-west to north-east diagonals of the tablestarting at the north-west corner), we express the elements of A = [

n2NAn

asan infinite sequence. That sequence forms a bijection from N to A.

Corollary A.3.1. If A is countably infinite then so is Ad, for d 2 N.

Proof. If A is countably infinite, then A ⇥ A corresponds to one copy of A foreach element of A, and thus we have a bijection between A2 and a countablyinfinite union of countably infinite sets. The previous proposition implies thatA2 is countably infinite. The general case follows by induction.

It can be shown that countably infinite sets are the “smallest” sets (in termsof the above definition of cardinality) among all infinite sets. In other words, ifA is uncountably infinite, then there exists an onto function f : A ! N but noonto function f : N ! A.

Proposition A.3.3. Assuming a < b and d 2 N,

N ⇠ Z ⇠ QN 6⇠ [a, b]

N 6⇠ (a, b)

N 6⇠ RN 6⇠ Rd.

Page 10: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

232 APPENDIX A. SET THEORY

We first define the concept of a binary expansion of a number, which will beused in the proof of the proposition below.

Definition A.3.3. The binary expansion of a number r 2 [0, 1] is defined as0.b

1

b2

, b3

. . . where bn

2 {0, 1}, n 2 N and

r =X

n2Nbn

2−n.

Example A.3.1. The binary expansions of 1/4 is 0.01 and the binary expansionof 3/4 is 0.11 = 1/2 + 1/4.

Proof. Obviously there exists a bijection between the natural numbers and thenatural numbers. The mapping f : N ! Z defined as

f(n) =

(

(n− 1)/2 n is odd

−n/2 n is even

maps 1 to 0, 2 to -1, 3 to 1, 4 to -2, 5 to 2, etc., and forms a bijection betweenN and Z.

A rational number is a ratio of a numerator and a denominator integers (notehowever that two pairs of numerators and denominators may yield the samerational number). We thus have that Q is a subset of Z2, which is countably infi-nite by Proposition A.3.2. Using Proposition A.3.1, we have that Q is countablyinfinite.

We next show that there does not exist a bijection between the natural num-bers and the interval [0, 1]. If there was such a mapping f , we could arrangethe numbers in [0, 1] as a sequence f(n), n 2 N and form a table A with infiniterows and columns where column n is the binary expansion of the real numberf(n) 2 [0, 1].

We could then create a new real number whose binary expansion is bn

, n 2 Nwith b

n

6= Ann

for all n 2 N by traversing the diagonal of the table and choosingthe alternative digits. This new real number is in [0, 1] since it has a binaryexpansion, and yet it is di↵erent from any of the columns of A. We have thusfound a real number that is di↵erent from any other number2 in the range of f ,contradicting the fact that f is onto.

Since there is no onto function from the naturals to [0, 1] there can be no ontofunction from the natural numbers to R or its Cartesian products Rd.

We extend below the definition of a Cartesian product (Definition A.1.10) toan infinite number of sets.

Definition A.3.4. Let A, T be sets. The notation AT denotes a Cartesianproduct of multiple copies of A, one copy for each element of the set T . In otherwords, AT is the set of all functions from T to A. The notation A1 denotes AN,a product of a countably infinite copies of A.

2The setup described above is slightly simplified. A rigorous proof needs to resolve the factthat some numbers in [0, 1] have two di↵erent binary expansions, for example, 0.0111111 . . . =0.1. See for example [37].

Page 11: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

A.4. LIMITS OF SETS 233

Example A.3.2. The set R1 is the set of all infinite sequences over the realline R

R1 = {(a1

, a2

, a2

, . . . , ) : an

2 R for all n 2 N}and the set {0, 1}1 is the set of all infinite binary sequences. The set R[0,1] isa Cartesian product of multiple copies of the real numbers — one copy for eachelement of the interval [0, 1], or in other words the set of all functions from [0, 1]to R.

Example A.3.3. The set {0, 1}A is the set of all functions from A to {0, 1},each such function implying a selection of an arbitrary subset of A (the selectedelements are mapped to 1 and the remaining elements are mapped to 0). A similarinterpretation may given to sets of size 2 that are di↵erent than {0, 1}. RecallingDefinition A.1.7, we thus have if |B| = 2, then BA corresponds to the power set2A, justifying its notation.

A.4 Limits of Sets

Definition A.4.1. For a sequence of sets An

, n 2 N, we define

infk≥n

Ak

=

1\

k=n

Ak

supk≥n

Ak

=

1[

k=n

Ak

lim infn!1

An

=[

n2Ninfk≥n

Ak

=[

n2N

1\

k=n

Ak

lim supn!1

An

=\

n2Nsupk≥n

Ak

=\

n2N

1[

k=n

Ak

.

Applying De-Morgan’s law (Proposition A.1.1) we have

lim infn!1

An

c

= lim supn!1

Ac

n

.

Definition A.4.2. If for a sequence of sets An

, n 2 N, we have lim infn!1 A

n

=lim sup

n!1 An

, we define the limit of An

, n 2 N to be

limn!1

An

= lim infn!1

An

= lim supn!1

An

,

The notation An

! A is equivalent to the notation limn!1 A

n

= A.

Example A.4.1. For the sequence of sets Ak

= [0, k/(k+1)) from Example A.1.6

Page 12: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

234 APPENDIX A. SET THEORY

we have

infk≥n

Ak

= [0, n/(n+ 1))

supk≥n

Ak

= [0, 1)

lim supn!1

An

= [0, 1)

lim infn!1

An

= [0, 1)

limAn

= [0, 1).

We have the following interpretation for the lim inf and lim sup limits.

Proposition A.4.1. Let An

, n 2 N be a sequence of subsets of ⌦. Then

lim supn!1

An

=

(

! 2 ⌦ :X

n2NIA

n

(!) = 1)

lim infn!1

An

=

(

! 2 ⌦ :X

n2NIA

c

n

(!) < 1)

.

In other words, lim supn!1 A

n

is the set of ! 2 ⌦ that appear infinitely often(abbreviated i.o.) in the sequence A

n

, and lim infn!1 A

n

is the set of ! 2 ⌦ thatalways appear in the sequence A

n

except for a finite number of times.

Proof. We prove the first part. The proof of the second part is similar. If ! 2lim sup

n!1 An

then by definition for all n there exists a kn

such that ! 2 Ak

n

.For that ! we have

P

n2N IA

n

(!) = 1. Conversely, ifP

n2N IA

n

(!) = 1, thereexists a sequence k

1

, k2

, . . . such that ! 2 Ak

n

, implying that for all n 2 N,! 2 [

i≥n

Ai

.

Corollary A.4.1.

lim infn!1

An

⇢ lim supn!1

An

.

Definition A.4.3. A sequence of sets An

, n 2 N is monotonic non-decreasing ifA

1

⇢ A2

⇢ A3

⇢ · · · and monotonic non-increasing if · · · ⇢ A3

⇢ A2

⇢ A1

. Wedenote this as A

n

% and An

&, respectively. If limAn

= A, we denote this asA

n

% A and An

& A, respectively.

Proposition A.4.2. If An

% then limn!1 A

n

= [n2NAn

and if An

& thenlim

n!1 An

= \n2NAn

.

Proof. We prove the first statement. The proof of the second statement is similar.We need to show that if A

n

is monotonic non-decreasing, then lim supAn

=lim inf A

n

= [n

An

. Since Ai

⇢ Ai+1

, we have \k≥n

Ak

= An

, and

lim infn!1

An

=[

n2N

\

k≥n

Ak

=[

n2NA

n

lim supn!1

An

=\

n2N

[

k≥n

Ak

⇢[

k2NA

k

= lim infn!1

An

⇢ lim supn!1

An

.

Page 13: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

A.5. NOTES 235

The following corollary of the above proposition motivates the notationslim inf and lim sup.

Corollary A.4.2. Since Bn

= [k≥n

Ak

and Cn

= \k≥n

Ak

are monotonic se-quences

lim infn!1

An

= limn!1

infk≥n

An

lim supn!1

An

= limn!1

supk≥n

An

.

A.5 Notes

Most of the material in this section is standard material in set theory. Moreinformation may be found in any set theory textbook. A classic textbook is[20]. Limits of sets are usually described at the beginning of measure theory orprobability theory textbooks, for example [5, 1, 33].

A.6 Exercises

1. Prove the assertions in Example A.1.1.

2. Prove the assertion in Example A.1.6.

3. Prove that f(U \ V ) = f(U) \ f(V ) is not true in general.

4. Let A0

= {a} and define Ak

= 2Ak1 for k 2 N. Write down the elementsof the sets A

k

for all k = 1, 2, 3.

5. Let A,B,C be three finite sets. Describe intuitively the sets A(B

C

) and(AB)C . What are the sizes of these two sets?

6. Let A be a finite set and B be a countably infinite set. Are the sets A1

and B1 countably infinite or uncountably infinite?

7. Find a sequence of sets An

, n 2 N for which lim inf An

6= lim supAn

.

8. Describe an equivalence relation with an uncountably infinite set of equiv-alence classes, each of which is a set of size 2.

Page 14: Appendix A - Guy Lebanon's websitetheanalysisofdata.com/probability/A/A.pdfAppendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern

236 APPENDIX A. SET THEORY