probability 2013 ch01 axioms of probability

8/18/2019 Probability 2013 Ch01 Axioms of Probability

1/28

Probability

Chapter Outline

Chapter one

xioms of probability

1.1 Introduction

1.2 Sample space and events1.3 Axioms of probability

1.4 Basic Theorems

1.5 Continuity of probability function

1.6 Probabilities 0 and 1

1.7

Random

selection

of

points

from

intervals


2/28

Probability

1.1 Introduction

Event: a set of outcomes (to which a probability is assigned)

– Event may or may not occur Random event

– Occurrence of event is inevitable Certain event

– Event never occur Impossible event Probability theory aims at determining the exact value, or an

estimate of the chance of occurrence of a random event

Humans have been interested in games of chance and gambling

– 4‐sided die: Ancient Egypt; 3500 B.C.

– 6‐sided die: 1600 B.C.

– Dice: China; 7th‐10th centuries

– Playing cards: Much more recent

Advent of probability as a math discipline is relatively recent

Chien-Chao Tseng 2


3/28

Probability

Early Era of Probability Theory

Studies of Chances of Events (begins at fifteen century)

Italian scholars tried to construct a math foundation for Prob.

– Luca Paccioli (1445‐1514)

– Niccolo Tartaglia (1499‐1557) – Girolamo Cardano (1501‐1576)

– Galileo Galielei (1564‐1642)

Real progress (started in France) (in 1654) General methods for the calculation of probabilities

– Blaise Pascal (1623‐1662)

– Pierre de Fermat (1601‐1665)

First book on probability (in 1657 )

– Christian Huygens (1629‐1695) (Dutch)

– “On Calculations in Games of Chance"

Chien-Chao Tseng 3


4/28

Probability

Growth of Probability Theory

Major breakthrough – James Bernoulli (1654‐1705) (book published in 1917) – Abraham de Moivre (1667‐1754) (book published in 1930)

Growth of probability and applications in various directions – Pierre‐Simon Laplace (1749‐1827) – Simeon Denis Poisson (1781‐1840) – Karl Friedrich Gauss (1777‐1855)

Advance of probability theory (19th century)

Russian mathematicians advanced works of Laplace, DeMoivre, and Bernoulli

– Pafnuty Chebyshev (1821‐1894) – Andrei Markov (1856‐1922) – Aleksandr Lyapunov (1857‐1918)

Chien-Chao Tseng 4


5/28

Probability

Relative Frequency Interpretation of Probability

By the early 20th century, prob. is already a developed theory – But, foundation was not firm

Relative Frequency Interpretation (most satisfactory that time)

Relative frequency of occurrences as the probability p of the occurrence of an event A of an experiment

To define p, we – study a series of sequential or simultaneous performances

of the experiment and – observe the proportion of times that A occurs approaches a

constant.

• : number of times A occurs during performances of the experiment

This definition is mathematically problematic

Cannot be the basis of a rigid probability theoryChien-Chao Tseng 5

lim→

/


6/28

Probability

Problem with Relative Frequency Interpretation

1) In practice, lim→

/ cannot be computed and no

way to analyze the error of the approximation

2) No reason to believe the limit of ⁄ , as → ∞ exists.

Furthermore, if we accept it exists many dilemmas arise

E.g., uniqueness of the probability is not guaranteed

Cannot justify ⁄ approaches the same limit in a

different series of experiments.

3) Probabilities that based on our personal belief and knowledge is not justifiable

Statements such as the following would be meaningless.• “The probability that the price of oil will be raised in the

next six months is 60%.”

• “The probability that it will snow next Christmas is 30%.”

Chien-Chao Tseng 6


7/28

Probability

Firm Mathematical Grounds

Axiomatic treatment of the theory of probability (in 1900)

Pointed out by David Hilbert (1862‐1943) Crucial to the advance of probability

Firm mathematical foundation of probability Some work toward this goal done by

– Émile Borel (1871‐1956), Serge Bernstein (1880‐1968),

Richard von Mises (1883‐1953) Successful creation of the axioms of probability (in 1933) by

– Russian mathematician Andrei Kolmogorov (1903‐1087) – Took 3 self ‐evident and indisputable properties as axiomsEntire theory of probability is developed and rigorously

based on axioms

Chien-Chao Tseng 7

• “Existence of a constant p as → ∞" is shown

•Subjective probabilities may be modeled and studied


8/28

Probability

1.2 Sample space and events

Experiment: “Tossing a die,” …

– Outcome of an experiment is not certain, but – All of the possible outcomes are predictable

Sample Space: set of all possible outcomes (denoted as S) Sample points (or points) of the sample space:

– Possible outcomes of the experiment

E.g., 1.1 Tossing a coin once: sample space S = {H, T} E.g., 1.2 Flipping a coin and

– tossing a die if T or flipping a coin again if H

S={T1, T2, T3, T4, T5, T6, HT, HH}

Events: certain subsets of sample space S

– Set of points of the sample space

an event E occurs if outcome of experiment belongs to E

Chien-Chao Tseng 8


9/28

Probability

Relations of Events (of an Experiment) (1/3)

Subset

event E is said to be a subset of event F if, whenever E occurs, F also occurs.

– all sample points of E are contained in F . – in set‐theory, E ⊆ F .

Equality

Events E and F are said to be equal if occurrence of E implies

occurrence of F , and vice versa;

– that is, if E ⊆ F and F ⊆ E , hence E = F .

Intersection

An event is called the intersection of two events E and F if it occurs only whenever E and F occur simultaneously.

– In set‐theory, this event denoted by EF or E ∩ F

Chien-Chao Tseng 9


10/28


11/28


12/28

Probability

Useful Relations

= , ∪ = , and = ∅

Commutative laws: E UF = F UE, EF = FE

Associative laws: EU(FUG)=(EUF)UG

Distributive laws: (EF)UH=(EUH)(FUH), (EUF)H=(EH)U(FH) De Morgan’s 1st law: (EUF)c = EcFc

⋃

= ⋂

, ⋃

= ⋂

De Morgan’s 2nd law: (EF)c = EcUFc

⋂

= ⋃

, ⋂

= ⋃

∪ : ∪ = ∪

Elementwise method for identity proof : – Events on both sides formed by the same sample points – Prove set inclusion in both directions

• E.g.,

=

if

and

Chien-Chao Tseng 12

Distributive Law

E E C

E

F C

F EF

EF C


13/28

Probability

1.3 Axioms of Probability

If a result is false, a counterexample exists to refuse it If a result is valid, a proof must be found.

Venn diagrams (sec. 1.2): to give intuitive justification, or to

create counterexample and show invalidity of relations Proofs in probability theory: by axiomatic method

1) Adopt certain simple, indisputable, and consistent statements without justifications (axioms or postulates).

2) Agree on how and when one statement is a logical consequence of another one

3) Obtain new results (theorems) using the terms already

clearly understood, axioms and definitions Theorems are statements that can be proved.

– Can be used for discovery of new theorems,

– Process

continues

and

a

theory

evolves.Chien-Chao Tseng 13


14/28

Probability

Axioms of probability

Definition (Probability Axioms)

– S: a sample space of a random phenomenon.

– A: an event of S

– P : a function for each event A, i.e., P : 2S R (or (S) R) P ( A): a number associated with A

If P satisfies the following axioms,

– P is

called

a

probability

and

– P ( A) is said to be the probability of A

Axiom 1: P ( A) 0

Axiom 2: P (S) = 1

Axiom 3: If { A1, A2, A3, …} is a sequence of mutually exclusive events ( = ∅ when ) then

Chien-Chao Tseng 14

)()(11

i iii AP AP


15/28

Probability

Equally Likely, Empty Set

Equally likely

– Let S be the sample space of an experiment.

– Let A and B be events of S.

– A and B are equally likely if P(A) = P(B).

Theorem 1.1 Probability of empty set ∅ is 0. (P (∅) = 0.)

Proof:

– Let A1 = S and Ai = ∅ for i ≥ 2; , , , . . . is a sequence of mutually exclusive events.

– By Axiom 3,

⋃

∑ ∑ ∅

Implying that ∑ ∅ 0 ∅ 0

Chien-Chao Tseng 15

Axiom 3


16/28

Probability

Finite Additivity (Theorem 1.2)

Axiom 3 is stated for a countably infinite

collection of mutually exclusive events.

Axiom 3 is also called the axiom of countable additivity .

Theorem 1.2 (Finite

Additivity .) – Let { , , . . . , } be a mutually exclusive set of events.

– Then ⋃ ∑

Proof : – For i , let Ai = ∅.

, , , . . .: a sequence of mutually exclusive events.

By Axiom 3 and Theorem 1.1, ⋃

= ⋃

= ∑

= ∑ ∑

∑

Chien-Chao Tseng 16

Axiom 3

∅ 0


17/28

Probability

Implication of Theorem 1.2

By theorem 1.2,

if and are two mutually exclusive events

∪ = + (1.2)

Implication of equation 1.2: for any event A, 1

Proof :

– ∪

=

+

– By Axiom 2,

∪ = = 1

+ = 1

For any event A, 0 1

Chien-Chao Tseng 17

Axiom 2


18/28

Probability

Probability of Equal Likely Outcomes


– If S has N points that are all

equally

likely

to occur

Probability of each outcome (sample point) is 1 /N .

Inference: – Let S = {s1 , s2 , . . . , sN} the sample space of an experiment;

– All sample points are equally likely to occur,

P({s1}) = P({s2})= ∙ ∙ ∙ = P({sN}) – P(S) = 1, and {s1}, {s2}, …, {sN} are mutually exclusive

1 = P(S) = P({s1, s2, ∙ ∙ ∙, sN})

= P({s1}) + P({s2}) + ∙ ∙ ∙ +P({sN})

= NP({sN})

P({sN}) = 1/N

Chien-Chao Tseng 18


19/28

Probability

Theorem 1.3 Classical Definition of Probability


– If S has N points that are all

equally

likely

to occur

For any event A of S ,

P( A) = N( A)/N, where N( A) is the number of points of A. Proof:

– Let S = {s1 , s2 , . . . , sN}, is an outcome of the experiment

– Equiprobable

P({sN})

=

1/N for

all

i ,

1 ≤

i ≤

N. – Let A = { , , . . . , }, where , ∈ S for all .

– { , , . . . , } are mutually exclusive

P( A) = P({

,

, . . . ,

})

= P({}) + P({}) + ∙ ∙ ∙ +P({})

= 1/N+1/N + .. +1/N = N( A)/N

Chien-Chao Tseng 19 N ( A) terms


20/28

Probability

1.4 Basic Theorems

Theorem 1.4 = 1 – A, mutually exclusive

1 ∪

Theorem

1.5 If

A

B,

then

P(B A) = P(B ) = P(B) P( A)

Proof :

– A BB = (B A) ∪ A

– Events (B A) and A are mutually exclusive (∵ (B A) A = ∅)

P(B) = P((B A) ∪ A) = P(B A) + P( A)

P(B A) = P(B) P( A)

Corollary If A B , then P( A) P(B)

Theorem 1.6 P( A ∪ B) = P( A) + P(B) P( AB)

Theorem

1.7

P( A)

=

P( AB)

+

P( A

)Chien-Chao Tseng 20


21/28

Probability

Proof of Theorem 1.6

Theorem 1.6 P( A ∪ B) = P( A) + P(B) P( AB) Proof :

̶ A ∪ B = A ∪ (B AB)

̶ A and (B AB) are mutually exclusive P( A ∪ B ) = P( A ∪ (B AB))

= P( A ) + P(B AB)

– By Theorem 1.5,

AB ⊆ B P(B AB) = P(B) P( AB)

P( A ∪ B )

=

P( A

) + P(B)

P( AB)

Chien-Chao Tseng 21


22/28

Probability

Inclusive‐Exclusive Principle (1/2)

Three Events:

P( ∪∪) = P( )+P( )+P( )

P( )P( )P( )

+P( )

Proof :

̶ By associative laws: P(

∪

∪

) = P(

∪

∪

)

̶ By Theorem 1.6 and distributive Laws:

̶ P( ∪∪) = P(( ∪)∪)

= P( )+P( )P( )+P( )P( ∪ )= P( )+P( )P( )+P( )

P( )P( )+P( )

Chien-Chao Tseng 22


23/28

Probability

Inclusive‐Exclusive Principle (2/2)

Four Events:

P( ∪∪∪)

= P( )+P( )+P( )+P( )

P( )P( )P( )P( )P( )P( )

+P( )+P( )+P( ) +P( )

P( )

Inclusive‐Exclusive Principle ( events)

⋃ ∑

∑ ∑

+ ∑ ∑ ∑

⋯

+ 1 ⋯

Chien-Chao Tseng 23


24/28

Probability

Theorem 1.7

Theorem 1.7 P( A) = P( AB) + P( A) Proof :

̶ A = AS = A ∩ (B ∪ ) = AB ∪ A

̶ AB and A are mutually exclusive P( A) = P( AB ∪ A) = P( AB) + P( A)

Chien-Chao Tseng 24


25/28

Probability

1.5 Continuity of probability function

Recall: probability is a set function from (S) to [0 , 1].– (S), power set (set of all possible events) of S.

A sequence events { , ≥ 1} of a sample space is called

– increasing if ⊆ ⊆ ⊆ ∙ ∙ ∙ ⊆ ⊆ ∙ ∙ ∙ ; – decreasing if ⊇ ⊇ ⊇ ∙ ∙ ∙ ⊇ ⊇ ⊇ ∙ ∙ ∙ .

For an increasing sequence of events { , ≥ 1}, – lim

→

: event that at least one , 1 ≤


26/28

Probability

Theorem 1.8 Continuity of Probability Function

– For any increasing or decreasing seq. of events, { , ≥ 1}

lim→

) = lim→

roof:

– ’s: circular disks

– ’s: shaded circular annuli, except .

• ⋃ = ⋃

= , 1, 2, 3, ⋯,

⋃ = ⋃

= lim

→

– lim→

= ⋃ = ⋃

= ∑

= lim→ ∑ = lim→ ⋃

= lim→ ⋃

= lim→

Chien-Chao Tseng 26

Referred to textbook for the proof for decreasing sequence

Mutually exclusive

Mutually exclusive

Mutually exclusive


27/28

Probability

1.6 Probabilities 0 and 1

Misinterpretation of events with probabilities 1 and 0.

– If event E with probabilities 1 then it is the sample space S.

– If event F with probabilities 0 then it is the empty set ∅.

In some experiments, there exist infinitely many events – each with probability 1, and

– each with probability 0.

Ex. (Experiment: Random selection of points from (0,1)) – Probability of the occurrence of any particular point is 0.

(Can be shown by Continuity of Probability, page 29, 30)

– For a point t ∈ (0 , 1), let = (0 , 1) − {t }.

• P({t }) = 0, but {t } ∅

• P() = P() 1 1, but S (0 , 1).

– Infinitely many events {t }’s and ’s exist

Chien-Chao Tseng 27


28/28

Probability

1.7 Random selection of points from intervals

Definition: A point is said to be randomly selected from an interval (a, b):

– if any two subintervals of (a, b) that have the same length

are

equally likely to

include

the

point.

Probability that the subinterval (, ) contains the point is defined to be ( )/(ba).

(Sketch of Proof: divide interval into (ba)/ ( )

subintervals, apply Axiom 2 and Thm 1.2)

In a random selection of points from intervals,

– Probability of the occurrence of any particular point is 0.

If [α , β] ⊆ (a, b) , events that the point falls in [α , β] , (α , β) , [α , β), and (α , β] are all equiprobable.

– Probability of the occurrence of a finite or countablyinfinite set is 0.

Chien-Chao Tseng 28

probability 2013 ch01 axioms of probability

Documents