11

ProbabilityProbability

Ch2Ch2

22

The Universal Set , denoted S , is the set of all objects under consideration at a given moment. Also known as the outcome space or the population or the sample space .

An Element of a set is any member of the set. A Subset is any part of a set; A B means A is a subset

of B . An Event is a collection of possible outcomes and must

be contained in S . The Empty (Null) Set is a notational idea representing

the set (or subset) with no members; we use Ø to denote it.

33

44

The basic set operations: Union , Intersection, Complement , Set Difference .

A B is the union of A and B; A B is the intersection of A and B;

A’ is the complement of A; A − B is the difference of A and B.

Two sets are Mutually Exclusive or disjoint if they have no elements in common (their intersection is disjoint).

A1,A2, . . . ,Ak are mutually exclusive events means that Ai Aj = Ø

, i j , that is, A1,A2, . . . ,Ak are disjoint sets. A1,A2, . . . ,Ak are exhaustive events means A1 A2 ...

Ak = S . Venn diagrams are useful is graphically representing s

ets. Laws on Set operations. A Set Function is a function whose domain is a collecti

on of sets.

55

The events and outcome space are sets. Fundamental set operations can be applied in the proba

bility computation. They are ∩, , and the like. ∪

For event A in an experiment of n trials, The occurrences of A are denoted as N(A), A's frequency. N(A)/n is the relative frequency of A .A's probability P(A), as n↑.

Ex2.1-1: rolling a dice 6 times–success: there is a match if dice=k on Roll k. Let A = {success}. P(A)=1-(1-1/6)6=0.665. N(A)/n can be verified by simulation, and will approach .665 as n↑.

Ex2.1-2: throwing a disk with diameter 2 inches on a tiled floor, where each tile is a square with sides 4 inches in length. Let C = {the disk is placed entirely on one tile.}. P(C) = 4/16 = 0.25.

66

Definition and Theorem Def.1.2-1: Probability is a real-valued set function

P that assigns to each event A .the sample space S a number P(A), called the probability of event A. P(A) ≥0, P(S)=1 If A1, A2, …, Ak are disjoint events, then P(A1 A∪ 2 …∪ ∪

Ak) = P(A1) + P(A2) + …+ P(Ak) for any integer k.

Thm1.2-1: For each event A, P(A) = 1 –P(A’). Thm1.2-2: P()=0. Thm1.2-3: If event A event B, then P(A) ≤P(B). Thm1.2-4: For each event A, P(A) ≤1.

77

Thm1.2-5: For any two events A & B, P(A B) ∪= P(A) + P(B) –P(A ∩B).

Thm1.2-6: For any three events A, B & C, P(A B C) = P(A) + P(B) + P(C) –P(A ∩B) –P(B ∪ ∪

∩C) –P(C ∩A) + P(A ∩B ∩C) Ex1.2-6: Draw a card at random from a deck of 52 c

ards. Let A denote the outcome is a king. Then, P(A)=4/52=1/13.

88

Enumeration Methods Multiplication Principle:

An experiment E1 has n1 outcomes, and for each of them, an experiment E2 has n2 outcomes.

The composite experiment E1E2 for doing first E1 and then E2 has n1n2 outcomes. Ex2.2-1: E1 denotes the rat selection from a cage –{F, M}, and E1

denotes applying drug to the selected rate –{A, B, P}. The set of all possible composite outcomes is {(F, A), (F, B), (F, P), (M, A), (M, B), (M, P)}, 6 in total.

Tree diagram:

The composite experiment E1E2…Em has n1n2…nm outcomes (different combinations).

99

Definition Def.1.3-1: Each of the n! arrangements (in a row) of n differ

ent objects is called a permutation of the n objects. There are n positions to be filled with n different objects n! Ex2.2-3: The number of permutations of the 4 letters a, b, c, and d c

learly 4!=24. Particularly, the number of possible 4-letter code words using these 4 letters is 44=256 if letters can be repeated (reused).

Def.1.3-2: Each of the nPr arrangements is called a permutation of n objects taken r at a time.

nPr= n(n-1)(n-2)…(n-r+1)=n!/(n-r)! Ex1.3-5: The number of ways of selecting a president, a vice presid

ent, a secretary, and a treasurer in a club with 10 persons is 10P4=5040.

These 4 persons chosen are distinguishable.

Def.1.3-3: If r objects chosen from a set of n objects are distinguishable or the selection order is noted, the selected set of r objects is called an ordered sample of size r.

1010

Definition (cont'd) Def.1.3-4: Sampling with replacement occurs when an obj

ect is selected and then replaced before the next object is selected. The no. of possible ordered samples of size r taken from n objects is

nr. Ex1.3-6: Rolling a dice 5 times. The number of possible ordered sample

s is 65= 7776. Rolling a dice is equivalent to sampling with replacement.

Def.1.3-5: Sampling without replacement occurs when an object is not replaced after it has been selected. There are exactly nPr possible ordered samples.

If the selection order of r objects is ignored, the number of (unordered) subsets of size r is nCr= nPr/r! = n!/[r!(n-r)!].

1111

Def.1.3-6: Each of the nCr unordered subsets is called a combination of n objects taken r at a time.

1212

Example Ex1.3-11: Drawing a five-card hand from a

deck of 52 cards. Event A: the five cards in a hand are all spades.

Event B: there are exactly three kings and two queens.

Event C: there are exactly two kings, two queens, and one jack.

1313

Distinguishable Permutations Def.1.3-7: Each of the nCr permutations of n objects, r of on

e type and n-r of another type, is called a distinguishable permutation.

Ex1.3-12: Flipping a coin 10 times to yield a head/tail sequence.The number of possible outcomes having 4 heads and 6 tails is

Ex1.3-13: Arranging 7 colored flags on a vertical flagpole.The number of different signals by 4 orange and 3 blue flags is

Extension: For a set of n objects, n1are similar, n2 are similar, …, n

s are similar, where n1+ n2+ …+ ns= n.The number of distinguishable permutations of the n objects is

multinomial coefficients

1414

Selecting r objects out of n objects For ordered samples,

there are nr possible outcomes when sampling with replacement. Each of r different objects is arbitrarily mapped to one of n object

s.

there are nPr outcomes when sampling without replacement.

For unordered samples (the order is irrelevant), there are nCr outcomes when sampling without replacem

ent. there are n-1+rCr outcomes when sampling with replaceme

nt. This can be computed as the number of ways to insert (n-1) “|”int

o a series of r “0”. For instance, 00||000|0|000|0 is an example for n=6 and r=10.

Each of r similar objects is arbitrarily mapped to one of n objects.

1515

Ex1.3-15: Rolling a pair of six-sided dices. The number of distinguishable outcomes is

Note: each outcome occurs unequally likely!

1616

More Examples Ex: Giving a, a, a, a, b, c, d, e arbitrarily to 3 persons.

The number of distinguishable outcomes is H3434

Ex: Giving a, b, c, d, e, f items arbitrarily to 4 persons, each with exactly one item. The number is P6

4

10 persons, each with at most one item. The number is 4 persons. The number is 4 persons, but the first person gets at least 2 items. The number

is

Ex: Giving a, a, a, a, a, a items arbitrarily to 4 persons, each with exactly one item. The number is 1. 10 persons, each with at most one item. The number is C10

6

4 persons. The number is H46

4 persons, but the first person gets at least 3 items. The number is H4

3

(As giving a, a, a items arbitrarily to 4 persons.)

1717

Conditional Probability Def.1.4-1: P(A|B) is a conditional

probability, defined as the probability of A among the occurrences of B. Mathematically, the relationship is Note: P(B)>0.

Ex.1.4-2: Suppose P(A)=0.4, P(B)=0.5, and P(A∩B)=0.3.P(A|B)=0.3/0.5=0.6; P(B|A)=0.3/0.4=0.75

1818

Ex.1.4-4: Rolling a pair of four-sided dice and determine the sum. Let A be the event that a sum of 3 is rolled, and B be the event that a sum of 3 or 5 is rolled. Compute P(A|B).

P(A|B): a sum of 3 occurs before a sum of 5.

1919

Conditional Probability Derivation With P(B)>0, Def.1.2-1 can be specified for c

onditional probability. P(A|B) ≥0, P(B|B)=1. If A1, A2, …, Ak are disjoint (mutually exclusive) events, th

en P(A1 A∪ 2 … A∪ ∪ k |B) = P(A1|B) + P(A2|B) + …+ P(Ak|B) for any integer k.

Ex1.4-5: There are 25 balloons: 10 yellow, 8 red, and 7 g

reen. Given the balloon sold is yellow, what is the probability that the next balloon selected at random is also yellow? This conditional probability is 9/24.

A: the 1st balloon sold is yellow; B: the 2nd balloon selected is yellow. P(B|A)=9/24=P(A∩B)/P(A) .P(A∩B)= P(B|A)P(A)=(9/24)(10/25).

2020

Def.1.4-2: The probability that two events, A and B, both occur is given by the multiplicative rule P(A∩B)= P(B|A)P(A)=P(A|B)P(B). Individual probabilities, P(A), P(B), P(B|

A), P(A|B), may be easier to get.[an factorization idea: divide/conquer]

2121

Examples Ex.1.4-6: Drawing 2 out of 10 chips: 7 blue and 3 red. What

is the probability that (A) the first drawn is red,

and (B) the second drawn is blue? It is easier to get P(A)=3/10 and P(B|A)=7/9. Thus P(A∩B)=7/30.

Ex.1.4-7: Drawing cards at random w/o replacement. What is the probability the third spade appears on the sixth draw? A: two replacement spades in the first five draws. B: a spade is drawn on the sixth draw.

Different factorizations are possible, if reasoning is consistent with assumptions.

2222

Ex.1.4-8: Rolling a pair of 4-sided dice. What is the probability for rolling a sum of 3 on the first roll and then, rolling a sum of 3 before rolling a sum of 5? Ex.1.4-4) (2/16) (2/6)=1/24

2323

Chained Conditional Probability P(AB C)=P(A B)P(C|A B) =P(A)P(B|A)P(C|A B)

Ex1.4-9: Drawing 4 cards from an ordinary deck. The probability of receiving in order a spade, a heart, a diamond, and a club is (13/52)(13/51)(13/50)(13/49)

Ex1.4-10: The left pocket has 5 blue and 4 white marbles and the right pocket has 4 blue and 5 white. If one marble is transferred from left to right, then the probability of subsequently drawing a blue marble from the right is BL: drawing a blue from the left. BR: right. WL: white left.

P(BR)=P(BL BR)+P(WL BR) =P(BL)P(BR|BL)+P(WL)P(BR|WL) =(5/9)(5/10)+(4/9)(4/10)=41/90

2424

Independent Events If the occurrence of one event does not change the probability o

f the other's happening, both are said to be independent events.

Def.1.5-1: Events A & B are independent iff P(A∩B)=P(A)P(B).Otherwise, they are called dependent events. Ex1.5-1: Flip a coin twice; the sample space of the sequence is S={HH,

HT, TH, TT}, each with probability of 1/4. A={heads on the first flip}={HH, HT} B={tails on the second flip}={HT, TT} C={tails on both flips}={TT}

P(B)=1/2; P(B|C)=1 since C B. C and B are dependent. P(B|A)=1/2=P(B). A and B are independent. P(A|B)=P(A). P(A∩B)=P(A)P(B).

Ex1.5-2: Roll a red die and a white die. A={4 on red}; B={sum is odd} P(A)P(B)=(6/36)(18/36)=3/36=P(AB) Independent

Ex1.5-3: C={5 on red}; D={sum is 11} P(C)P(D)=(6/36)(2/36)=1/108P(C)P(D)=(6/36)(2/36)=1/1081/36=P(C1/36=P(CD) D) DependentDependent

2525

Derived Independences Thm1.5-1: If A & B are independent, the following

are too: A & B'; A' & B; A' & B'.

Ex1.5-4: Draw a ball from an urn with 4 balls: 1, 2, 3, 4. A={1, 2}, B={1, 3}, C={1, 4}. P(A)=P(B)=P(C)=1/2. A∩B={1}, B∩C={1}, A∩C={1}. P(A∩B)=P(B∩C)=P(C∩

A)=1/4.P(A∩B)= P(A)P(B), … A, B, C are independent in p∵ ∴

airs. (pairwise independence)

However, P(A∩B∩C)=P({1})=1/4 ≠1/8=P(A)P(B)P(C). There is no complete independence of A, B, and C.

P(AB’)=P(A)P(B’|A) =P(A)[1-P(B|A)] =P(A)[1-P(B)]=P(A)P[B’]

2626

Def.1.5-2: A, B & C are mutually independent iff They are pairwise independent, and P(A∩B∩C)=P(A)P(B)P(C).

Generally, each pair, triple, quartet, etc. must be all independent.

A', B' & C' are mutually independent; A and (B∩C); A and (B C); A' and (B∩C'); ∪

…are independent.

2727

Representation of Sample Space Ex1.5-6: Rolling a 6-sided dice 6 times.

Ai: Side i is observed on Roll i –a match. P(Ai)=1/6 & P(Ai')=5/6. B: at least one match occurs; B': no matches occur.

P(B)=1-P(B')=1-P(A1'∩…∩A6')=1-(5/6)6.

Roll a 6-sided die n times. S = {(O1, …, On): Oi=1,2,…,6, for i = 1,2,…,n}.

Toss a coin n times. S = {(O1, …, On): Oi=H or T, for i = 1,2,…,n}.

For simplicity, (O1, …, On) = O1…On; (H, T) = HT.

Ex1.5-9: On 5 consecutive days, a lottery with the winning probability of 1/5 is purchased on each day.

Assume independence, P(WWLLL)=(1/5)2(4/5)3=P(LWLWL). The probability of purchasing 2 winning tickets and 3 losing tickets is

2828

Bayes' Theorem Posterior Probability of Bk by some A!

Note: B1, …, Bm are mutually exclusive and exhaustive. The prior probability of event Bi: P(Bi)>0, for i=1..m.

2929

Ex.1.6-2: Machines A, B, C are all producing springs of the same length. Of the total productions, they produce 35, 25, and 4

0%, with the defective rates of 2, 1, and 3%, respectively.

The probability of selecting a spring that is defective is P(D) = P(A)P(D|A) + P(B)P(D|B) + P(C)P(D|C) = 35% 2% + 25% 1% + 40% 3% = 215/10000.

The conditional probability that the defective spring is produced from C is