reviewch2-basic concept probability
TRANSCRIPT
-
8/3/2019 ReviewCh2-Basic Concept Probability
1/92
6/7/2011
1
241-460 Introduction to Queueing
Netw orks : Engineering Approach
Probability TheoryProbability Theory
Assoc. Prof. Thossaporn Kamolphiwong
Centre for Network Research (CNR)
Department of Computer Engineering, Faculty of Engineering
r nce o ong a n vers y, a anEmail : [email protected]
Outline
Random Experiments
Type of sample space
Event Set Theory
Chapter 2 : Basic Concepts of Probability Theory
2
-
8/3/2019 ReviewCh2-Basic Concept Probability
2/92
6/7/2011
2
Specifying Random Experiments
A random experiment is an experiment in
unpredictable fashion when theexperiment is repeated under the samecondition
Chapter 2 : Basic Concepts of Probability Theory
3
Example of Random Experiments
Experiment E1 : Toss a coin three times andnote the sequence of heads and tails
Experiment E2 : Toss a coin three times andnote the number of heads
Experiment E3 : Count the number of voicepackets containing only silence procedure from agroup of N speakers in a 10-ms period
xper men : oc o n orma on stransmitted repeatedly over a noisy channel untilan error-free block arrives at the receiver. Countthe number of transmissions required.
Chapter 2 : Basic Concepts of Probability Theory
4
-
8/3/2019 ReviewCh2-Basic Concept Probability
3/92
6/7/2011
3
(Continue)
Experiment E5 : Measure the time between twomessa e arrives at a messa e center
Experiment E6 : Determine the value of avoltage waveform at time t1 and t2
Experiment E7 : Pick a number at randombetween zero and one.
between zero and one, then pick a number Yatrandom between zero andX
Chapter 2 : Basic Concepts of Probability Theory
5
(Continue)
Note:
xper men an xper men are esame procedure but they are different
observations different Experiments
Chapter 2 : Basic Concepts of Probability Theory
6
-
8/3/2019 ReviewCh2-Basic Concept Probability
4/92
6/7/2011
4
Set and P robability
Outcome or sample pointany possible observation is an element or point in Sample Space
Sample SpaceS
The set of all possible outcomes
Set notation, tables, diagrams, intervals of the
rea ne, reg ons o e p aneFinite, Countably infinite, Uncountably infinite
Chapter 2 : Basic Concepts of Probability Theory
7
Sample Space
Sample spaces corresponding to theex erim ent
S1 = {HHH, HHT, HTH, THH, TTH, THT, HTT, TTT}S2 = {0, 1, 2, 3}
S3 = {0, 1, 2, ,N}
S4 = {0, 1, 2, }
S5 = {t: t> 0} = [0, )
S6 = {(v1, v2) : - < v1 < and- < v2 < }
Chapter 2 : Basic Concepts of Probability Theory
8
-
8/3/2019 ReviewCh2-Basic Concept Probability
5/92
6/7/2011
5
(Continue)
S7 = {x : 0
-
8/3/2019 ReviewCh2-Basic Concept Probability
6/92
6/7/2011
6
Event
Event
Event Sample Space
Chapter 2 : Basic Concepts of Probability Theory
11
Event examples
Event examples
E1 : The three tosses gives the same outcome
A1 = {HHH, TTT}E2 : The three tosses gives # of head equals #
of tails
A2 = {}=
E8 : The two numbers differ by less than 1/10
A8 = {(x,y) : (x,y) S8 and |xy| < 1/10}
Chapter 2 : Basic Concepts of Probability Theory
12
-
8/3/2019 ReviewCh2-Basic Concept Probability
7/92
6/7/2011
7
Example
Flip four coins, 0.5 baht(red), 1 baht(yellow), 5baht white and 10 baht silver .
Examine the coins in order (0.5 baht, 1 baht, 5baht, 10 baht) and observe whether each coinshows a head (h) or a tail (t).
What is the sample space?
LetBi = {outcomes with i heads}
Chapter 2 : Basic Concepts of Probability Theory
13
Solution
Outcome
r y w s , r y w s , r y w s , r y w s , r y w s ,
(hrtyhwts), (hrtytwhs), (hrtytwts), (trhyhwhs), (trhyhwts),
(trhytwhs), (trhytwts), (trtyhwhs), (trtyhwts), (trtytwhs),
(trtytwts)
Chapter 2 : Basic Concepts of Probability Theory
14
-
8/3/2019 ReviewCh2-Basic Concept Probability
8/92
6/7/2011
8
(Continue)
There are 16 members of the sample space
S = { (hrhyhwhs), (hrhyhwts), (hrhytwhs), (hrhytwts),
(hrtyhwhs), (hrtyhwts), (hrtytwhs), (hrtytwts),
(trhyhwhs), (trhyhwts), (trhytwhs), (trhytwts),
(trtyhwhs), (trtyhwts), (trtytwhs), (trtytwts) }
Chapter 2 : Basic Concepts of Probability Theory
15
(Continue)
EventB0 = { (trtytwts) }
=1 r y w s , r y w s , r y w s , r y w s
EventB2 = { (hrhytwts), (hrtyhwts), (hrtytwhs), (trtyhwhs),
(trhytwhs), (trhytwhs)} EventB3 = {(hrhyhwts),(hrhytwhs),(hrtyhwhs), (trhyhwhs)
}
EventB4 = { (hrhyhwhs) }
Event B = {B0,B1,B2, B3, B4} is Event space
Chapter 2 : Basic Concepts of Probability Theory
16
-
8/3/2019 ReviewCh2-Basic Concept Probability
9/92
6/7/2011
9
Event Space
Event Space
Collective exhaustive
S = {1,2,3,4,5,6}
A = {2,4,6}
= , , Mutually exclusive
Chapter 2 : Basic Concepts of Probability Theory
17
Applying Set Theory to P robability
Probability is a number that describes a
Set Algebra Probability
Set Event
Universal set Sample space
Element Outcome
Chapter 2 : Basic Concepts of Probability Theory
18
-
8/3/2019 ReviewCh2-Basic Concept Probability
10/92
6/7/2011
10
Set Theory
The mathematical basis of probability is the theoryof sets
Set is a collection of things
Things that together make up the set areelement
= x x = , , , ,= {1, 4, 9, 16, 25}
Chapter 2 : Basic Concepts of Probability Theory
19
Set Operation
Union
A B : the set of outcomes either inA or inB orboth
Intersection
A B : the set of outcomes in bothA andB IfA B = , thenA andB are mutually exclusive
Complement
A = The set of all outcomes not inA Difference
A - B contains all elements of A that are not elementsofB
Chapter 2 : Basic Concepts of Probability Theory
20
-
8/3/2019 ReviewCh2-Basic Concept Probability
11/92
6/7/2011
11
A
Venn diagram: Set Operation
A BSS A BA
B
SA B = S
Chapter 2 : Basic Concepts of Probability Theory
BA21
(Continue)
SA -BS
A BA B
A
Chapter 2 : Basic Concepts of Probability Theory
22
-
8/3/2019 ReviewCh2-Basic Concept Probability
12/92
6/7/2011
12
Mutually exclusive
Mutually exclusive AS
Ai Aj = for i jA B = Disjoint
B
A8SA5
A1 A2 An = SChapter 2 : Basic Concepts of Probability Theory
A1A7A6
A2
A4
A1
A3
23
Properties of set operation
Commutative Properties:
A B = B A and A B = B A
Associative Properties:
A (B C) = (A B) C and
A (B C) = (A B) C
Distributive Properties:
=
A (B C) = (A B) (A C)
De Morgans law:
(A B) = A B and(A B) = A B
Chapter 2 : Basic Concepts of Probability Theory
24
-
8/3/2019 ReviewCh2-Basic Concept Probability
13/92
6/7/2011
13
(Continue)
Union and intersection operation can bere eated
n
n
k
k AAAAA 3211
n
n
k AAAAA 321
Chapter 2 : Basic Concepts of Probability Theory
25
Ax ioms of Probability
Axiom 1 : For any event A, P[A] 0
Axiom 2 : P[S] = 1
Axiom 3 :If A B = , then P[A B] = P[A] + P[B]
Axiom 4 :If A1, A2, is a sequence of events such that
Ai Aj, = for all i j, then
Chapter 2 : Basic Concepts of Probability Theory
...2111
APAPAPAPk
k
k
k26
-
8/3/2019 ReviewCh2-Basic Concept Probability
14/92
6/7/2011
14
Some Consequences of The
Axioms The probability measure P[] satisfies
=
Corollary 2 : P[A] < 1
Corollary 3 : P[] = 0
Corollary 4 : IfA1,A2, ,An are pairwise mutually
exclusive, then
Chapter 2 : Basic Concepts of Probability Theory
27
(Continue)
Corollary 5 : P[A B] = P[A] + P[B] P[A B]
A BSB
A
Chapter 2 : Basic Concepts of Probability Theory
28
-
8/3/2019 ReviewCh2-Basic Concept Probability
15/92
6/7/2011
15
(Continue)
Corollary 6 :
Chapter 2 : Basic Concepts of Probability Theory
29
(Continue)
Corollary 7 : IfA B, then P[A] < P[B]
A BA
B
S
Chapter 2 : Basic Concepts of Probability Theory
30
-
8/3/2019 ReviewCh2-Basic Concept Probability
16/92
6/7/2011
16
Example
A company has a model of telephone usage. Itclassifies all calls as either long (L), if they last more
, .whether calls carry voice (V), data (D) or fax (F).This model implies an experiment in which theprocedure is to monitor a call and the observationconsists of the type of call, V, D, or F, and thelength,L or B. The corresponding table entry is theprobability of the outcome. The table is
V D F
L 0.3 0.12 0.15B 0.2 0.08 0.15
Chapter 2 : Basic Concepts of Probability Theory
31
Solution
V D F
B 0.2 0.08 0.15
From the table we can read that the probability of
a brief data call is
P[BD] = 0.08.
The probability of a long call is
P[L] = P[LV] + P[LD] + P[LF] = 0.57
Chapter 2 : Basic Concepts of Probability Theory
32
-
8/3/2019 ReviewCh2-Basic Concept Probability
17/92
6/7/2011
17
Types of Sample Space
Discrete Sample Space
Chapter 2 : Basic Concepts of Probability Theory
33
Discrete Sample Space
Finite sample space : S = {a1, a2, , an }
All distinct events are mutually exclusive
Event ''2'1 ,...,, maaaA
'''
Chapter 2 : Basic Concepts of Probability Theory
''2'121
...
,...,,
m
m
aPaPaP
aaaPAP
34
-
8/3/2019 ReviewCh2-Basic Concept Probability
18/92
6/7/2011
18
(Continue)
Countable infinite, S = {b1, b2, }
All distinct events are mutually exclusive
Event
''
,..., '2'1 bbB
Chapter 2 : Basic Concepts of Probability Theory
...21 bPbPBP
35
(Continue)
Sample space has n elements,
S = {a1, a2, , an }
Probability assignment is equally likelyoutcomes
Chapter 2 : Basic Concepts of Probability Theory
naPaPaP n
1}][{}][{}][{ 21
36
-
8/3/2019 ReviewCh2-Basic Concept Probability
19/92
6/7/2011
19
Example
Selecting a ball from urn containing 10 identicalballs numbered 0 1 9.
A = number of ball selected is odd
B = number of ball selected is a multiple of 3
C = number of ball selected is less than 5
Find
P[A B C]
Chapter 2 : Basic Concepts of Probability Theory
37
Solution
S = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
= , , , ,
B = {3, 6, 9}
C= {0, 1, 2, 3, 4} Assume that outcome are equally likely
5}]9[{}]7[{}]5[{}]3[{}]1[{][ PPPPPAP
Chapter 2 : Basic Concepts of Probability Theory
10
3}]9[{}]6[{}]3[{][ PPPBP
10
10
5}]4[{}3[{}2[{}]1[{}]0[{][ PPPPPCP
38
-
8/3/2019 ReviewCh2-Basic Concept Probability
20/92
6/7/2011
20
(Continue)
2
9,3][ PBAP 2
3,1][ PCAP
10
13][ PCBP
10
13][ PCBAP
][][][][ BAPBPAPBAP
Chapter 2 : Basic Concepts of Probability Theory
][][][
][][][][][
CBAPCBPCAP
BAPCPBPAPCBAP
39
(Continue)
6235][][][][ BAPBPAPBAP
91122535
][][][
][][][][][
CBAPCBPCAP
BAPCPBPAPCBAP
Chapter 2 : Basic Concepts of Probability Theory
1010101010101010
40
-
8/3/2019 ReviewCh2-Basic Concept Probability
21/92
-
8/3/2019 ReviewCh2-Basic Concept Probability
22/92
6/7/2011
22
(Continue)
Relative frequency
= = j =j , , ,
P[j toss till first head] = (1/2)j j = 1, 2, 3,
Heads Tails
n = 100 trials
N 50
50
25
100/8
Chapter 2 : Basic Concepts of Probability Theory
N2 25
N3 100/8
N4 100/1643
Continuous Sample Spaces
Outcomes are numbers.
line, rectangular regions in the plane
Chapter 2 : Basic Concepts of Probability Theory
44
-
8/3/2019 ReviewCh2-Basic Concept Probability
23/92
6/7/2011
23
Example
Pick two numberx andy at random between
zero and one.
Find the probability of the following events:
A = {x > 0.5},
B = {y > 0.5}
C= {x >y}
Chapter 2 : Basic Concepts of Probability Theory
45
Solution
y
1
y
1
Sx
10x
10
x>1/2
(a) Sample space (b)Event(x > )
P A = 1 2
y y
Chapter 2 : Basic Concepts of Probability Theory
(c)Event(y > ) (d)Event(x >y)1
y > 1/2
x0
P[B] = 1/2
x10
x >yP[C] = 1/2
46
-
8/3/2019 ReviewCh2-Basic Concept Probability
24/92
6/7/2011
24
Example
Suppose that the lifetime of a computer memorychi is measured and we find that the
proportion of chips whose lifetime exceeds tdecreases exponentially at a rate .
Find the probability of arbitrary intervals in S.
Chapter 2 : Basic Concepts of Probability Theory
47
Solution
Probability that chips lifetime exceeds tdecreases
e-t
,0for, tetP t
S = (0,)1
0.5
Lifetime
Chapter 2 : Basic Concepts of Probability Theory
t time
P S = P 0, = 1
48
-
8/3/2019 ReviewCh2-Basic Concept Probability
25/92
6/7/2011
25
(Continue)
LetA : event of chips lifetime in interval (r,s)
P[(r, )] = P[(r,s]] + P[(s, )]
r s time( ](
P[A] = P[(r,s]] = P[(r, )] - P[(s, )] = e-r e-s
Chapter 2 : Basic Concepts of Probability Theory
49
Computing probabilities usingCounting M ethods
The outcome of finite sample can be assumed tobe e ui robable
Calculation of probabilities reduces to countingthe number of outcome in an event
Chapter 2 : Basic Concepts of Probability Theory
50
-
8/3/2019 ReviewCh2-Basic Concept Probability
26/92
6/7/2011
26
Basic principle of Counting rules
Addition rule
Chapter 2 : Basic Concepts of Probability Theory
51
Addition Rule
Addition Rule
1 2
Let eventEdescribe the situation where either
eventE1 or eventE2 will occur. The number of times event Ewill occur can be
given by the expression:
n(E) =n(E1) +n(E2)
Chapter 2 : Basic Concepts of Probability Theory
52
-
8/3/2019 ReviewCh2-Basic Concept Probability
27/92
6/7/2011
27
Example
Set of numbers S = {1, 2, 3, 4, 5, 6, 7, 8, 9}
1
E2 = choosing an even number from S.
Find n(E) when
E= choosing an odd or an even number from S;
n(E) = n(E1) + n(E2) = 5 + 4
Chapter 2 : Basic Concepts of Probability Theory
53
Multiplication Rule
Multiplication Rule
If one event can occur in m ways and another
event can occur in n, independently of eachother, then there are mn ways in which bothevents can occur.
Chapter 2 : Basic Concepts of Probability Theory
54
-
8/3/2019 ReviewCh2-Basic Concept Probability
28/92
6/7/2011
28
Example
Multiple-choice test has 2 questionsst ,
answers and 2nd question the student select oneof 5 possible answers.
What is the total number of ways of answering
Total number of ways =4x5
Chapter 2 : Basic Concepts of Probability Theory
55
(Continue)
Multiple-choice test has 2 questions
ipossible answers.
What is the total number of ways of answeringthe entire test?
Chapter 2 : Basic Concepts of Probability Theory
56
-
8/3/2019 ReviewCh2-Basic Concept Probability
29/92
6/7/2011
29
Solution
xi : question number i
i
b1
x1a1 a2 1n
a
(a1,b1) (a2,b1)
(a ,b ) (a ,b )
1,1 ban
,bab
Total number of
ways of answering
the test = n1n2
Chapter 2 : Basic Concepts of Probability Theory
2,1 nba
x2
2nb
21, nn ba
1
57
Counting Method
The number of distinct ordered k-tuples (x1,,xk)
i i
element is
Number of distinct ordered k-tuples = n1n2nk
Chapter 2 : Basic Concepts of Probability Theory
58
-
8/3/2019 ReviewCh2-Basic Concept Probability
30/92
6/7/2011
30
Example
There are two subexperiments. The first .
outcomes,Hand T. The second
subexperiment is Roll a die. It has sixoutcomes, 1, 2, , 6. The experiment,
Flip a coin and roll a die, has 2x6 = 12
ou comes:(H,1), (H,2), (H,3), (H,4), (H,5), (H,6)
(T,1), (T,2), (T,3), (T,4), (T,5), (T,6)
Chapter 2 : Basic Concepts of Probability Theory
59
Sampling
Sampling
Without Replacement
With Ordering Without Ordering
With replacement Without Rep lacement
RR -
RB RB
BB -
BR BR Chapter 2 : Basic Concepts of Probability Theory
60
-
8/3/2019 ReviewCh2-Basic Concept Probability
31/92
6/7/2011
31
Sampling w ith Replacement and
w ith OrderingChoose kobject from a setA that has n
distinct ob ects with re lacement andordering
The experiment produces an ordered k-tuple,
(x1,,xk) wherexi A and i = 1,, k
Chapter 2 : Basic Concepts of Probability Theory
Number of distinct ordered k-tuples = nk
61
Example
An urn contains five ball number 1 to 5
How many distinct ordered pairs are possible? What is the probability that the two draws yield
the same number?
Chapter 2 : Basic Concepts of Probability Theory
62
-
8/3/2019 ReviewCh2-Basic Concept Probability
32/92
6/7/2011
32
Solution
Ordered pairs for sampling with replacement
, , , , ,
(2,1) (2,2) (2,3) (2,4) (2,5)
(3,1) (3,2) (3,3) (3,4) (3,5)
(4,1) (4,2) (4,3) (4,4) (4,5)
(5,1) (5,2) (5,3) (5,4) (5,5)
(a) Total distinct ordered pairs are 52 = 25
(b) Probability that two draws yield the samenumber is 5/25 = 0.2
Chapter 2 : Basic Concepts of Probability Theory
63
Example
A laptop computer has PCMCIA expansion cardslots A and B. Each slot can be filled with eithera modem card (m), a SCSI interface (i), or aGPS card (g). From the set {m, i, g} of possible
cards, what is the set of possible ways to fill thetwo slots when we sample with replacement?Inother words, What is the probability that bothslots hold the same type of card?
Chapter 2 : Basic Concepts of Probability Theory
64
-
8/3/2019 ReviewCh2-Basic Concept Probability
33/92
6/7/2011
33
Solution
Letxy : the outcome that card type x is used inslotA and card t e is used in slotB. The
possible outcomes are
S = {mm, mi, mg, im, ii, ig, gm, gi, gg}
e num er o poss e ou comes s n ne
The probability that both slots hold the same typeof card is 3/9
Chapter 2 : Basic Concepts of Probability Theory
65
Sampling w ithout Replacement andw ith Ordering
Choose kobjects in succession withoutreplacement formA ofn distinct object. k< n.
The number of possible outcomes in the first draw
is n1 = n; the number of possible outcomes inthe second draw is n2 = n -1, all n object exceptthe one selected in the first draw; and so on, up
k -
Chapter 2 : Basic Concepts of Probability Theory
Number of distinct ordered k-tuples
= n(n-1)(n k +1)66
-
8/3/2019 ReviewCh2-Basic Concept Probability
34/92
6/7/2011
34
Example
A college club has 25 members and is electingfour officers res v secretar treasurer
How many ways can the officers be filled?
Order matters Bill as pres and Bob as vice-presis different from Bob as pres and Bill as vice-pres
,= 25!/21!
25 possibilities for the first officer, 24 for thesecond, 23 for the third, 22 for the fourth
Chapter 2 : Basic Concepts of Probability Theory
67
Permutations
Sampling without replacement with k< n
Use the multiplication rule
Number of ways
!nP n
is called the number of permutation ofk
objects out ofnChapter 2 : Basic Concepts of Probability Theory
!kn
n
kP68
-
8/3/2019 ReviewCh2-Basic Concept Probability
35/92
6/7/2011
35
Sampling w ithout Replacement and
w ithout Ordering
Pickkobject from a set ofn distinct objects .
The number of ways to choose kobjects out ofn
distinguishable objects is
= n n-1 n k +1 /k!
Chapter 2 : Basic Concepts of Probability Theory
69
Example
A college club has 25 members and is sendingfour students to meetin
How many ways can the students be chosen?
Order does not matter sending Bill and Bob isthe same as sending Bob and Bill
# ways = 25x24x23x22/(4x3x2x1)
= , = x
Permutations divided by 4!, the number of waysto mix around the four chosen
Chapter 2 : Basic Concepts of Probability Theory
70
-
8/3/2019 ReviewCh2-Basic Concept Probability
36/92
6/7/2011
36
Combinations
Sampling without replacement with k< n
Use the multiplication rule
Number of ways
!!!
knk
n
k
nCnk
is call the number ofcombinations ofk
objects out ofn distinguishable objects, and is
also called the binomial coefficientChapter 2 : Basic Concepts of Probability Theory
n
kC
71
When P and When C
Permutations
Change the order its a different outcome
CombinationsOrder does not matter
Change the order its the same outcome
Chapter 2 : Basic Concepts of Probability Theory
72
-
8/3/2019 ReviewCh2-Basic Concept Probability
37/92
6/7/2011
37
Permutations and Combination
A batch of 50 items contains 10 defective items.Su ose 12 items are selected at random andtested.
How many ways can the items be chosen sothat exactly 5 of the items tested are defective?
A more complicated permutation / combinationproblem
Chapter 2 : Basic Concepts of Probability Theory
73
Solution
5 defective items from the batch of 10
7 nondefective items from the batch of 40
25212345/678910105 C
34353637383940407
C
Chapter 2 : Basic Concepts of Probability Theory
560,643,18
74
-
8/3/2019 ReviewCh2-Basic Concept Probability
38/92
6/7/2011
38
Permutations and Combination
# of ways to selecting 5 defective and 7
nondefective items from the batch of 50 =
252x18,643,560 = 4,698,177,120
# of ways each 12 items can be ordered = 12!
nondefective items with ordering from the batch
of 50 = 4,698,177,120 x12!
Chapter 2 : Basic Concepts of Probability Theory
75
Partitioning
Combinations how to choose k out of n
The ones that are chosen
The ones that are not chosen
Two results from this observation
#1 : the number of ways of partitioning n n -
#2 :
Chapter 2 : Basic Concepts of Probability Theory
n
kC
n
kn
n
k CC 76
-
8/3/2019 ReviewCh2-Basic Concept Probability
39/92
6/7/2011
39
Multinom ial coefficient
How about partitioning n items into more than
two rou s?
Partition n items into m groups containing k1 ,k2, , km objects (where k1 + k2 + + km = n)
This is a more general version of thecombinations problem
This is called themultinomial coefficientChapter 2 : Basic Concepts of Probability Theory
!!...!!!...!
ways#2121 mm
kkkn
kkkn
77
Example
Number of distinctB1
n = 9
B3
B2
223
56789!2!3!4
!9
2,3,4
9
Chapter 2 : Basic Concepts of Probability Theory
k1 = 4 k3 = 2k2 = 3
m = 3
1260
78
-
8/3/2019 ReviewCh2-Basic Concept Probability
40/92
6/7/2011
40
Example
A six-sided die is tossed 12 times.
from the set {1, 2, 3, 4, 5, 6}) have eachnumber appearing exactly twice?
400,484,72
!12
!2!2!2!2!2!2
!126
a s e pro a y o o a n ng suc asequence?
Chapter 2 : Basic Concepts of Probability Theory
3
12
6
104.3336,782,176,2
400,484,7
6
2!12 79
Example
For five subexperiments with sample, ,
sequences are there in which 0 appears 2
times and 1 appears 3 times?
Chapter 2 : Basic Concepts of Probability Theory
80
-
8/3/2019 ReviewCh2-Basic Concept Probability
41/92
6/7/2011
41
Solution
The set of five-letter words with 0 appearing twiceand 1 a earin three times is
{00111, 01011, 01101, 01110, 10011, 10101, 10110,
11001, 11010, 11100}
ere are exac y suc wor s
Multinomial coefficient
Chapter 2 : Basic Concepts of Probability Theory
10!3!2
!5
3,2
5
81
Sampling w ith Replacement andw ithout Ordering
Pickkobjects from set ofn distinct with
Number of different ways of picking kobjectsfrom a set ofn distinct objects with
Chapter 2 : Basic Concepts of Probability Theory
1
11
n
kn
k
kn
82
-
8/3/2019 ReviewCh2-Basic Concept Probability
42/92
6/7/2011
42
Example
Consider the following program segment, wherea b c are inte er variables
for a = 1 to 20 do
for b = 1 to a do
for c = 1 to b do
print(a *b + c)
How many times is the print statement executein this program segment?
Chapter 2 : Basic Concepts of Probability Theory
83
(Continue)
The selections ofa, b, and c where the print
statement is executed satisfies the condition 1
-
8/3/2019 ReviewCh2-Basic Concept Probability
43/92
6/7/2011
43
Conditional Probability
Interested in determining whether two events,
A andB are related in sense that knowledgeB,
alters the likelihood of occurrence ofA
P A|B : Probabilit of A iven B
Chapter 2 : Basic Concepts of Probability Theory
85
(Continue)
Conditional Probability
the occurrence of the event B is
S
BPBAP
BAP
|
,P[B] > 0
Chapter 2 : Basic Concepts of Probability Theory
AA B
86
-
8/3/2019 ReviewCh2-Basic Concept Probability
44/92
6/7/2011
44
(Continue)
Probability as Relative frequency
B
BA
n
n
BAP
Beventintimeofnumber
BAeventintimeofnumber|
Chapter 2 : Basic Concepts of Probability Theory
nn
nn
BAPB
BA
|
87
Correlated Events
some definitions describing the conditionalrobabilities of correlated events.
P[AB] orP[AB] : Joint Probability The probability that both events A andB occur
P[A|B] : Condition Probability
The probability of event A, given that some other
eventB has occurred
P[B] : Probability of eventB
Chapter 2 : Basic Concepts of Probability Theory
88
-
8/3/2019 ReviewCh2-Basic Concept Probability
45/92
6/7/2011
45
(Continue)
BAPBAP
BP
Chapter 2 : Basic Concepts of Probability Theory
P[A B] = P[A|B]P[B]= P[AB]
P[B A] = P[B|A]P[A]= P[BA]=
89
Example
Communication systems can be modeled :User inputs a 0 or a 1 into system,
Receiver makes a decision about what was the inputto the system, based on the signal it received.
Suppose thatUser sends 0s with probability 1-p and 1s with
probabilityp,
Receiver makes random decision errors withprobability .
Chapter 2 : Basic Concepts of Probability Theory
R0
R1
1-
1-
T01-p
T1
p
90
-
8/3/2019 ReviewCh2-Basic Concept Probability
46/92
6/7/2011
46
(Continue)
R01-
T0
1-p
Given P(R0|T0) = (1- ), P(R1|T0) = ,
P(R1|T1) = (1- ), P(R0|T1) =
R11-
T1
p
Find P(R0)
P(R0) = P(R0/T0)P(T0) + P(R0/T1)P(T1)
= (1- )(1-p) + p = 1 - -p + 2p
Chapter 2 : Basic Concepts of Probability Theory
91
(Continue)
For i = 0, 1, let
i ,
Rj be the event receiver decision wasj.
Find probabilities P[Ti Rj] for i =0,1 andj = 0,1.
0 0 =
P[T0 R1] = ?
P[T1 R0] = ?
P[T1 R1] = ?
Chapter 2 : Basic Concepts of Probability Theory
92
-
8/3/2019 ReviewCh2-Basic Concept Probability
47/92
6/7/2011
47
10
Solution
Input into binary channelTransmitter
0 1 0 1Output form binary channel
Receiver
1-
- p
1-
(1-p)(1-) (1-p) p p(1-)
P[T0 R0] = (1 - )(1 p)
Chapter 2 : Basic Concepts of Probability Theory
P[T0 R1] = (1 p)
P[T1 R0] = p P[T1 R1] = (1 - )p93
Example
Ball is selected from urn containing
,
Two white balls, numbered 3 and 4
EventsA : black ball selected
B : even-numbered ball selected
: num er o a s grea er an
Find P[A|B] and P[A|C]
Chapter 2 : Basic Concepts of Probability Theory
94
-
8/3/2019 ReviewCh2-Basic Concept Probability
48/92
6/7/2011
48
Solution
S = {(1,b), (2,b) , (3,w) , (4,w)}
A = {(1,b), (2,b)}
B = {(2,b), (4,w)}
C = {(3,w), (4,w)}
5.05.0
25.0
BP
BAPBAP
00 CAPCAP P[AB] = P[(2,b)]
P[AC] = P[] = 0
Chapter 2 : Basic Concepts of Probability Theory
.
95
Example
An urn contains
three white ball
Two balls are selected at random withoutreplacement and the sequence of colors in
.
Find the probability that both balls are black.
Chapter 2 : Basic Concepts of Probability Theory
96
-
8/3/2019 ReviewCh2-Basic Concept Probability
49/92
6/7/2011
49
Solution
0
W1
5
2
B1
B2 W2
5
3
4
1
4
34
2
4
2
u come o rs raw
B2 W2
1 2
Outcome of second draw
Chapter 2 : Basic Concepts of Probability Theory
101
103
10
3
10
1
5
2
4
121 BBP
103
212 | BPBBP97
Total Probability
LetB1, B2, , Bn be mutually exclusive event
= 1 2 n B1, B2, , Bn form a partition of S
Chapter 2 : Basic Concepts of Probability Theory
98
-
8/3/2019 ReviewCh2-Basic Concept Probability
50/92
6/7/2011
50
(Continue)
SB1
B3 Bn-1
A = A S =A (B1 B2 Bn)
= (A B1) (A B2) (A Bn)
A
B2Bn
P[A] = P[A B1] + P[A B2] + + P[A Bn]
= P[A|B1]P[B1]+P[A|B2]P[B2]+ + P[A|Bn]P[Bn]
Chapter 2 : Basic Concepts of Probability Theory
99
Example
An urn contains
three white ball
Two balls are selected at random withoutreplacement and the sequence of colors in
.
Find the probability of the event that the
second ball is white.
Chapter 2 : Basic Concepts of Probability Theory
100
-
8/3/2019 ReviewCh2-Basic Concept Probability
51/92
6/7/2011
51
Solution
B1
0
W1
1B2 W2 B2 W2
5 5
4
1
4
34
2
4
2
10
1
10
3
10
3
2
10
3
P[W2] = P[W2|B1]P[B1] + P[W2|W1]P[W1]
Chapter 2 : Basic Concepts of Probability Theory
5
3
10
3
10
3
5
3
2
1
5
2
4
3
101
Example
A manufacturing process produces a mix of ood memor chi s and bad memor chi s.The lifetime of good chips follows theexponential law with rate of failure . The
lifetime of bad chips also follows the exponentiallaw, but the rate of failure is 1000. Suppose
that the fraction of good chips is 1 p and of
bad chips,p.
Find the probability that a randomly selected
chip is still functioning after tsecond.
Chapter 2 : Basic Concepts of Probability Theory
102
-
8/3/2019 ReviewCh2-Basic Concept Probability
52/92
6/7/2011
52
(Continue)
Let1Lifetime
B : Chip is bad
C: Chip still
functioning aftert
second
e-1000t
0
e-t
2 4 6 8 10
0.2
0.4
0.6
.
timet
Chapter 2 : Basic Concepts of Probability Theory
BPBCPGPGCPCP ||
pBCPpGCP |1| tt peep 10001 103
Example
Three machines B1, B2, and B3 for making 1 kresistors.
B1 produces 80% of resistors within 50 of thenominal value.
B2 produces 90% of resistors within 50 of thenominal value.
Percentage for machineB3 is 60%.
, 1 , 2produces 4000 resistors, and B3 produces 3000resistors.
Chapter 2 : Basic Concepts of Probability Theory
104
-
8/3/2019 ReviewCh2-Basic Concept Probability
53/92
6/7/2011
53
Solution
What is the probability that the company ships aresistor that is within50 of the nominalvalues?
Chapter 2 : Basic Concepts of Probability Theory
105
(Continue)
LetA = {resistor is within 50 of the nominalvalue
B1 produces 80% of resistors within 50 of thenominal value.
P[A|B1] = 0.8,
B2 produces 90% of resistors within 50 of thenominal value.
P[A|B2] = 0.9,
Percentage for machineB3 is 60%.
P[A|B3] = 0.6
Chapter 2 : Basic Concepts of Probability Theory
106
-
8/3/2019 ReviewCh2-Basic Concept Probability
54/92
6/7/2011
54
(Continue)
Each hour,B1 produces 3000 resistors,B2 produces4000 resistors, andB3 produces 3000 resistors.
Total product = 3000 + 4000 + 3000 = 10,000
P[B1] = 3000/10,000 = 0.3,
P[B2] = 0.4,
3 = .
Chapter 2 : Basic Concepts of Probability Theory
107
(Continue)
A = {resistor is within 50 of the nominal value}
S
B1
B3 B2
A
P[A] = P[A|B1]P[B1] + P[A|B2]P[B2] + P[A|B3]P[B3]
= (0.8)(0.3) + (0.9)(0.4) + (0.6)(0.3) = 0.78
Chapter 2 : Basic Concepts of Probability Theory
108
-
8/3/2019 ReviewCh2-Basic Concept Probability
55/92
6/7/2011
55
Bayes Rule
Situations
Advance in formation aboutP[A|B]
Chapter 2 : Basic Concepts of Probability Theory
Need to calculateP[B|A]
109
(Continue)
LetB1, B2, , Bn be a partition of a sample space S.
prob.condition,|
AP
ABPABP
BAPABP
Chapter 2 : Basic Concepts of Probability Theory
APBPBAP
ABP|
|
110
-
8/3/2019 ReviewCh2-Basic Concept Probability
56/92
6/7/2011
56
Example of Bayes Rule
Communication system
0.92R0
R1
0.95
T00.45
T1
0.55
Given : P(R0|T0)
Want to know : P(T0|R0)
Chapter 2 : Basic Concepts of Probability Theory
111
Example: Communication System
Communication system
R0
R1
.
0.95
T00.45
T1
0.55
Given P(R0|T0) = 0.92;
Chapter 2 : Basic Concepts of Probability Theory
P(R1|T1) = 0.95
P(T0) = 0.45; P(T1) = 0.55112
-
8/3/2019 ReviewCh2-Basic Concept Probability
57/92
6/7/2011
57
(Continue)
Given P R0 T0 = 0.92
P(R1|T1) = 0.95
P(T0) = 0.45
P(T1) = 0.55
Find P(R0), P(R1), P(T1|R1), P(T0|R0), P(Error)
Find which input is more probable given that thereceiver has output a 1
Chapter 2 : Basic Concepts of Probability Theory
113
Solution
P(R0) = P(R0/T0)P(T0) + P(R0/T1)P(T1)
= 0.92x0.45 + 0.05x0.55 = 0.4415
P(R1) = P(R1/T1)P(T1) + P(R1/T0)P(T0)
= 0.95x0.55 + 0.08x0.45 = 0.5585P(R1) = 1 P(R0)
rror = 0 1 1 + 1 0 0= 0.05x0.55 + 0.08x0.45 = 0.0635
Chapter 2 : Basic Concepts of Probability Theory
114
-
8/3/2019 ReviewCh2-Basic Concept Probability
58/92
6/7/2011
58
(Continue)
P(T1/R1) = P(R1/T1)P(T1)/P(R1)= 0.95x0.55/0.5585
= 0.9355
P(T0/R0) = P(R0/T0)P(T0)/P(R0)= 0.98x0.45/0.4415
= 0.9988
Chapter 2 : Basic Concepts of Probability Theory
115
(Continue)
P[T0 |R1] = P[T0R1]/P[R1] = [R1|T0]P[T0]/P[R1]
T1 is more probable given that
the receiver has output a 1
= 0.08x0.45/0.5585 = 0.0645
P[T1 |R1] = P[T1R1]/P[R1] = P[R1|T1]P[T1]/P[R1]
= 0.95x0.55x/0.5585 = 0.9355
Chapter 2 : Basic Concepts of Probability Theory
116
-
8/3/2019 ReviewCh2-Basic Concept Probability
59/92
6/7/2011
59
Independence of Events
Two Independent Events
][][
][][
][
][]|[ AP
BP
BPAP
BP
BAPBAP
If P[A B] = P[A]P[B] thenA andB are independent.
Since
Chapter 2 : Basic Concepts of Probability Theory
Implies that P[A |B] = P[A],
P[B |A] = P[B]
Implies that P[A] 0 and P[B] 0117
Independent & Disjoint
Two eventsA andBP[A] 0, P[B] 0, P[A B] = 0A andB cannot be independent.
AS
A
S
Independent Disjoint
Chapter 2 : Basic Concepts of Probability Theory
B
P[AB] 0P[AB] = P[A]P[B] P[AB] = 0118
-
8/3/2019 ReviewCh2-Basic Concept Probability
60/92
6/7/2011
60
Example
A short-circuit tester has a red light (r) to indicate
to indicate that there is no short circuit. Consider
an experiment consisting of a sequence of three
tests. Suppose that for the three lights, each
outcome (a sequence of three lights, each either
. 2that the second light was red and G2 that the
second light was green independent?Are the
eventsR1 andR2 independent?
Chapter 2 : Basic Concepts of Probability Theory
119
(Continue)
A short-circuit tester
green light (g) : no short circuit.
Consider a sequence of three tests.
The sequence of three lights is equally likely.
Are R2 and G2 independent?
Are R1 andR2 independent?
Chapter 2 : Basic Concepts of Probability Theory
120
-
8/3/2019 ReviewCh2-Basic Concept Probability
61/92
6/7/2011
61
Solution
Circuit 3Circuit 2Circuit 1
R1 = the first circuit is red
R2 = the second circuit is red
G2 = the second circuit is red
Chapter 2 : Basic Concepts of Probability Theory
Outcomerrr rrg rgr rgg grr grg ggr ggg121
(Continue)
Are R2 and G2 independent?
, , , , , , ,
P[R2] = 4/8 = 1/2
P[G2] = 4/8 =1/2
srrr rgr
R2 G2
R2 G2 =
P[R2G2] = 0
Chapter 2 : Basic Concepts of Probability Theory
grr
grgggr
ggg
122
-
8/3/2019 ReviewCh2-Basic Concept Probability
62/92
6/7/2011
62
(Continue)
Are R2 and G2 independent?
P[R2]P[G2] = ()() =
P[R2G2] = 0
P[R2G2] P[R2]P[G2]
R2 and G2 are not independent
R2 and G2 are disjoint
Chapter 2 : Basic Concepts of Probability Theory
123
(Continue)
Are R1 and R2 independent?
, , , , , , ,
P[R1] = 4/8 = 1/2
P[R2] = 4/8 =1/2
srrr
rrg
rgr
rgg
R1
R1 R2 = {rrr, rrg}
P[R1R2] = 2/8 = 1/4
Chapter 2 : Basic Concepts of Probability Theory
grr
grg
ggr
ggg
R2124
-
8/3/2019 ReviewCh2-Basic Concept Probability
63/92
6/7/2011
63
Solution
Are R1 and R2 independent?
P[R1]P[R2] = ()() =
P[R1R2] =
P[R1R2] = P[R1]P[R2]
R1 andR2 are independent
Chapter 2 : Basic Concepts of Probability Theory
125
Example
Two numberx andy are selected at random
between zero and one
A = {x > 0.5}B = {y > 0.5}
C= x >
AreA andB independent?
AreA and Cindependent?
Chapter 2 : Basic Concepts of Probability Theory
126
-
8/3/2019 ReviewCh2-Basic Concept Probability
64/92
6/7/2011
64
Solution
y
1
y
1B
A
1x
0 0x
1
P[A B] = P[A] =
P[B] =
P[A C] = + 1/8 = 3/8
P[A] =
P[C] =
AC
Chapter 2 : Basic Concepts of Probability Theory
=P[A B] = P[A]P[B ] =
Event A and B are
independent
=P[A B] P[A]P[B ]
Event A and C are not
independent127
3 Independent Events
Event A, B and C are independent if the
triplet of events is equal to the product of the
probabilities of the individual events
P A B C = P A P B P C
Chapter 2 : Basic Concepts of Probability Theory
128
-
8/3/2019 ReviewCh2-Basic Concept Probability
65/92
6/7/2011
65
Example
Two numberx andy are selected at random
between zero and one
A = {x > 0.5}
B = {y > 0.5}
C= x >
AreA andB and Cindependent?
Chapter 2 : Basic Concepts of Probability Theory
129
Solution
y
1A = {x > 0.5}
P[A ] =
P[B ] = P[A B C] = P[A]P[B]P[C]
A
1 x0C
B = {y > 0.5}
C= {x >y}
Chapter 2 : Basic Concepts of Probability Theory
P C =
P[A B C] = 1/8P[A]P[B]P[C] = ()()()
= 1/8
Event A and B and
C are independent
130
-
8/3/2019 ReviewCh2-Basic Concept Probability
66/92
6/7/2011
66
Example
LetB = {y > }
=
F= {x < andy < } {x > andy > }
AreB,D and Fare independent?
Chapter 2 : Basic Concepts of Probability Theory
131
Solution
y
1
y
1F
y
1B
0x
1
D
D = {x < }
0 x1
F
F= {x}
1x
0
B = {y > }
=
P[D] =
P[F] =
Chapter 2 : Basic Concepts of Probability Theory
132
-
8/3/2019 ReviewCh2-Basic Concept Probability
67/92
6/7/2011
67
Solution
B = {y > }
y
1
P[B] = P[B D F] P[B]P[D]P[F]
D = {x < }
F= {x}
1x
0
DF
P[F] =
P[B D F] = 0
P[B]P[D]P[F] = ()()() =1/8
Chapter 2 : Basic Concepts of Probability Theory
ree even s are no n epen en
133
Sequential Experiments
Sequential experiment consist of a
Sequences of Independent Experiment
Sequences of Dependent Experiment
Chapter 2 : Basic Concepts of Probability Theory
134
-
8/3/2019 ReviewCh2-Basic Concept Probability
68/92
6/7/2011
68
Sequences of Independent
Experiments
Experiment subexperiments subexperiments
A1,A2,,An are subexperiments
IfA1,A2,,An are independent
Thus
P[A1 A2 An] = P[A1]P[A2] P[An]
Chapter 2 : Basic Concepts of Probability Theory
135
Example
Suppose that 10 numbers are selected atrandom from the interval 0 1 . Find theprobability that the first 5 numbers are less than and the last 5 numbers are greater than .
Chapter 2 : Basic Concepts of Probability Theory
136
-
8/3/2019 ReviewCh2-Basic Concept Probability
69/92
6/7/2011
69
Solution
Let x1, x2, , x10 : the sequence of 10 numbers
Ak= {xk< } for k= 1,,5
Ak= {xk> } for k= 6,,10
0 1
P[A1 A2 A10] = P[A1]P[A2] P[A10]
= ()5()5
Chapter 2 : Basic Concepts of Probability Theory
137
Bernoulli trial
Bernoulli trial
two possible outcomes, "success with probability
p and "failure with probability q = 1 -p.
In practice it refers to a single experiment whichcan have one of two possible outcomes.
Toss a coin
Head (success)
Tail (fail)
Chapter 2 : Basic Concepts of Probability Theory
138
-
8/3/2019 ReviewCh2-Basic Concept Probability
70/92
6/7/2011
70
Binomial Experiment
Abinomial experiment is one that possessesthe followin ro erties:
The experiment consists ofn repeated trials;
Each trial results in an outcome that may beclassified as a successor a failure(hence thename binomial
The probability of a success, denoted byp,
remains constant from trial to trial and repeatedtrials are independent.
Chapter 2 : Basic Concepts of Probability Theory
139
Binomial P robability Law
k : the number of successes inn independent
The probabilities of k successes in n independent
repetitions is :
knkn pp
nkP
1
Chapter 2 : Basic Concepts of Probability Theory
n = the number of trials
k= 0, 1, 2, ... np = the probability of success in a single trial140
-
8/3/2019 ReviewCh2-Basic Concept Probability
71/92
6/7/2011
71
(Continue)
nn
n ppk
kP
)1()(
)!(!
!
knk
n
k
n
ro a ty o ksuccessesin n trials
Binomial coefficient
Chapter 2 : Basic Concepts of Probability Theory
n
kn Ck
nkN
)( Picking kpositions out ofn
for the success141
Example
Coin is tossed three time
heads isp
Let k: the number of head in three trials
Find the probability of k success in head.
Chapter 2 : Basic Concepts of Probability Theory
142
-
8/3/2019 ReviewCh2-Basic Concept Probability
72/92
6/7/2011
72
Solution
Sequence of heads and tail is
P HHH = P H P H P H = 3
P[{HHT}] = P[{H}]P[{H}]P[{T}] =p2(1-p)
P[{HTH}] = P[{H}]P[{T}]P[{H}] =p2(1-p)
P[{THH}] = P[{T}]P[{H}]P[{H}] =p2(1-p)
P[{TTH}] = P[{T}]P[{T}]P[{H}] =p (1-p)2
P[{THT}] = P[{T}]P[{H}]P[{T}] =p (1-p)2
P[{HTT}] = P[{H}]P[{T}]P[{T}] =p (1-p)2
P[{TTT}] = P[{T}]P[{T}]P[{T}] = (1-p)3
Chapter 2 : Basic Concepts of Probability Theory
143
Solution
P3(0) = P[k= 0] = P[{TTT}] = (1 p)3
P3(1) = P[k= 1] = P[{TTH, THT, HTT}] = 3p(1 p)2
3303 110
30 pppP
Chapter 2 : Basic Concepts of Probability Theory
13 1311
1 ppppP
144
-
8/3/2019 ReviewCh2-Basic Concept Probability
73/92
6/7/2011
73
(Continue)
P3(2) = P[k= 2] = P[{TTH, THT, HTT}] = 3p2(1 p)
P3(3) = P[k= 3] = P[{HHH}] =p3
ppppP
131
2
32
212
3
Chapter 2 : Basic Concepts of Probability Theory
3033 13
33 pppP
145
Example
Communication system transmits binaryinformation over channel that introduced
random bit errors with probability = 10-3
Transmitter transmits each information bit threetimes
Receiver take majority vote of received bit todecide on that the transmitted bit was
Find the probability that the receiver will makean incorrect decision
Chapter 2 : Basic Concepts of Probability Theory
146
-
8/3/2019 ReviewCh2-Basic Concept Probability
74/92
6/7/2011
74
Solution
Let k: number of errors that the receiver will
6
0312999.0001.0
3
3999.0001.0
2
32
kP
Chapter 2 : Basic Concepts of Probability Theory
147
Example
To communicate one bit of information reliably,
symbol five times.
zero is transmitted as 00000
one is 11111.
The receiver detects the correct information ifthree or more binar s mbols are receivedcorrectly.
What is the information error probability P[E], ifthe binary symbol error probability is q = 0.1?
Chapter 2 : Basic Concepts of Probability Theory
148
-
8/3/2019 ReviewCh2-Basic Concept Probability
75/92
6/7/2011
75
Solution
k: # of errors that the receiver will make correct
The probability of a successes isp
p = 1- q = 0.9
The information error probability P[E]
Chapter 2 : Basic Concepts of Probability Theory
00856.02
5
1
5
0
53245
555
qppqq
149
Multinomial Experiment
Amultinomial experiment has the followingro erties:
The experiment consists ofnrepeated trials.
Each trial has a discrete number of possibleoutcomes.
On any given trial, the probability that aparticular outcome will occur is constant.
The trials are independent; that is, the outcomeon one trial does not affect the outcome on othertrials.
Chapter 2 : Basic Concepts of Probability Theory
150
-
8/3/2019 ReviewCh2-Basic Concept Probability
76/92
6/7/2011
76
Example
Toss two dice three times, and record the outcomeon each toss. This is a multinomial ex erimentbecause:
The experiment consists of repeated trials. We tossthe dice three times.
Each trial can result in a discrete number of outcomes- 2 through 12.
T e pro a i ity o any outcome is constant; it oesnot change from one toss to the next.
The trials are independent; that is, getting aparticular outcome on one trial does not affect theoutcome on other trials.
Chapter 2 : Basic Concepts of Probability Theory
151
Multinomial Probability Law
A multinomial experiment consists ofn trials, and eachtrial can result in any ofkpossible outcomes:B ,B ,. . . ,BM. Suppose that each possible outcome canoccur with probabilitiesp1,p2, . . . , pk.
The probability that B1 occurs k1 times, B2 occurs k2 times, .
. . , and BMoccurs km times is
where n = k1 + k2 + . . . + kM.
Chapter 2 : Basic Concepts of Probability Theory
MkMkk
M
M pppkkk
nkkkP ...
!!...!
!,...,, 21 21
21
21
152
-
8/3/2019 ReviewCh2-Basic Concept Probability
77/92
6/7/2011
77
(Continue)
Mkkkn!
21
M
M
Mkkk
...!!...!
,...,, 2121
21
Multinomial Coefficient
Chapter 2 : Basic Concepts of Probability Theory
153
Example
Pick 10 telephone numbers at random from atele hone book and note the last di it in each ofthe numbers. What is the probability that weobtain each of the integers from 0 to 9 only
once?
Solution
Chapter 2 : Basic Concepts of Probability Theory
410 106.31.0!1!...1!1
!10
154
-
8/3/2019 ReviewCh2-Basic Concept Probability
78/92
6/7/2011
78
Geometric Probability Law
Geometric Probability Law
occurrence of the first success
Letm : # of trials carried out until the occurrence of
the first success
,...2,11...1''' mAAAAPmP
m
Where p :probability of success for the Bernoulli trial.
Ai : event success in ith trial
Chapter 2 : Basic Concepts of Probability Theory
155
(Continue)
P(m) = (1-p)m-1p Geometric probability
= q - p q = -p
1
1
1 m
m
mqpmP
= -
Chapter 2 : Basic Concepts of Probability Theory
=x
n
i
i
ni
i
m
qpqpmP001
lim
156
-
8/3/2019 ReviewCh2-Basic Concept Probability
79/92
6/7/2011
79
(Continue)
n
i
iqx0
x = 1 + q + q2 ++ qn
qx = q + q2 ++ qn + qn+1
(1-q)x = 1 - qn+1
qx
n
1 1
q
qq
nn
i
i
1
1 1
0
Chapter 2 : Basic Concepts of Probability Theory
11
1limlim
1
11
q
qpqpmP
n
n
n
m
i
nm
157
(Continue)
K= # of Bernoulli trial
1
11
...Km
mKK
qppqpqKmP
Chapter 2 : Basic Concepts of Probability Theory
01
1
i
iK
Km
mqqq
i = m 1 - k158
-
8/3/2019 ReviewCh2-Basic Concept Probability
80/92
6/7/2011
80
(Continue)
qqq
nnii
111
The probability that more than Ktrial :
Kq
pqKmP
1
1
qqnini 00
Chapter 2 : Basic Concepts of Probability Theory
Where p = 1 q
Kq
159
Example
ComputerA sends a message to computer B.
WhenB detects an error
RequestA to retransmit
Probability of a message transmission error isq = 0.1
What is the probability that a message needs tobe transmitted more than two times?
Chapter 2 : Basic Concepts of Probability Theory
160
-
8/3/2019 ReviewCh2-Basic Concept Probability
81/92
6/7/2011
81
Solution
Probability ofmth success is
P(m) = qm-1p m = 1,2,
The probability that a message needs to betransmitted more than two times
Chapter 2 : Basic Concepts of Probability Theory
161
(Continue)
1st p
2nd The probability that a
Trial more than 2 times
3rd q2p
mth qm-1p
message nee s o etransmitted more than
two times
P[m > 2] = q2 = 10-2
1=p + qp + q2p+ q3p +
q2p+ q3p + = 1 (p + qp) = 1 p qp
= q q(1 - q) = q q + q2 = q2
Chapter 2 : Basic Concepts of Probability Theory
162
-
8/3/2019 ReviewCh2-Basic Concept Probability
82/92
6/7/2011
82
Reliabil ity Problem
Independent trials can be used to describereliabilit roblem calculate the robabilit
that a particular operation succeeds
Operation consist ofn components
Each component succeeds withp, independent ofany other component.
Components in series
Components in parallel
Chapter 2 : Basic Concepts of Probability Theory
163
Type of Operation
Components in series Components in parallel
W1 W3W2
W1
W2
Chapter 2 : Basic Concepts of Probability Theory
P[W] = P[W1W2Wn]
= p xp xxp
= pn
nn
p
WWWPWP
1
... ''2'
1
3
164
-
8/3/2019 ReviewCh2-Basic Concept Probability
83/92
6/7/2011
83
Sequential Experiments
Sequential experiment consist of a
Sequences of Independent Experiment
Sequences of Dependent Experiment
Chapter 2 : Basic Concepts of Probability Theory
165
Sequences of Dependent Experiment
The outcome of a given experiment determineswhich subex eriment is erformed next.
It can be represented by a tree diagram.
0 0 0 0h 1
0
1 1
0 0
Chapter 2 : Basic Concepts of Probability Theory
1 1 1 1t
0 0 0111
1 2 3 4166
-
8/3/2019 ReviewCh2-Basic Concept Probability
84/92
6/7/2011
84
Example
Sequential experiment involves repeatedly drawing aball from one of two urns, noting the number on theball and replacing the ball in its urn. The urn fromwhich the first draw is made is selected at randomby flipping a fair coin. Urn 0 is used if the outcomeis heads and urn 1 if the outcome is tails. Thereafterthe urn used in a subexperiment corresponds to the
subexperiment.
Chapter 2 : Basic Concepts of Probability Theory
167
(Continue)
0
0h 01
00
1
00
1
0
0011
11
0
11
Chapter 2 : Basic Concepts of Probability Theory
1t
1
01
1 2 3 4
1
01
1
01
168
-
8/3/2019 ReviewCh2-Basic Concept Probability
85/92
6/7/2011
85
Probabil ity of Dependent
Experiment Compute the probability of a particular sequence
of outcomes sa s s s where ,
s0 is result from 1st outcome
s1 is result from 2nd outcome
s2 is result from 3rd outcome
P[{s0}{s1}{s2}] = ??
Chapter 2 : Basic Concepts of Probability Theory
169
(Continue)
P[{s0}{s1}{s2}] = ??
LetA = {s2},B = {s1}{s0}
P[{s0}{s1}{s2}] = P[AB]P[{s0}{s1}{s2}] = P[{s2}{s1}{s0}] = P[AB]
Since P[A B] = P[A|B]P[B]
P[{s0}{s1}{s2}] = P[{s2}|{s1}{s0}]P[{s1}{s0}]
= P[{s2}|{s1}{s0}]P[{s1}|{s0}]P[{s0}]
Chapter 2 : Basic Concepts of Probability Theory
170
-
8/3/2019 ReviewCh2-Basic Concept Probability
86/92
6/7/2011
86
(Continue)
P[{s0}{s1}{s2}] = P[{s2}|{s1}{s0}]P[{s1}|{s0}]P[{s0}]
P[{sn}|{sn-1}{s1}{s0}] = ??
depends on only on {sn-1}]
P[{sn}|{sn-1} {s1} {s0}]{s0}] = P[{sn}|{sn-1}]
Chapter 2 : Basic Concepts of Probability Theory
ar ov a n
171
(Continue)
P[{s0}{s1}{s2}] = P[{s2}|{s1}{s0}]P[{s1}{s0}]
= 2 1 0 1 0 0
ThereforeP[{s0}{s1}{s2}] = P[{s2}|{s1}]P[{s1}|{s0}]P[{s0}]
{s1}
P[s0s1s2] = P[s2|s1]P[s1|s0]P[s0]
Chapter 2 : Basic Concepts of Probability Theory
172
-
8/3/2019 ReviewCh2-Basic Concept Probability
87/92
6/7/2011
87
(Continue)
P[s0s1sn] = P[sn|sn-1]P[sn-1|sn-2]P[s1|s0]P[s0]
Chapter 2 : Basic Concepts of Probability Theory
173
Example
10 1
0h 010
0
010
0
010
0
001 10 1
Find the probability of the sequence 0011
Chapter 2 : Basic Concepts of Probability Theory
1t
1 11 2 3 4
11 11
174
-
8/3/2019 ReviewCh2-Basic Concept Probability
88/92
6/7/2011
88
(Continue)
The probability of the sequence 0011
Meaning
P[0011] = P[{s0}{s1}{s2}{s3}]
= P[{0}{0}{1}{1}]
Chapter 2 : Basic Concepts of Probability Theory
P[0011] = P[1|1] P[1|0] P[0|0]P[0]
= P[{s3}|{s2}] P[{s2}|{s1}]P[{s1}|{s0}]P[{s0}]175
0h
(Continue)
01
00
1
00
1
0
31
1t
1
0
11 2 3 4
32
32
32
1
0
11
0
1
001
Chapter 2 : Basic Concepts of Probability Theory
1 1
6
1
65 1 1
31 3
1
61
61
65
65
1
11
0
11
176
-
8/3/2019 ReviewCh2-Basic Concept Probability
89/92
6/7/2011
89
(Continue)
P[0011] = P[1|1]P[1|0]P[0|0]P[0]
2 2 2
=
31
0
1
0
1
31
3
61
65
0
1
0
1
31
3 3
61
61
65
65
h
1t
1
1
0
11 2 3 4
1
1
0
1 1
1
0
1
P 1 0 = 1/3
P[0|0] = 2/3
Chapter 2 : Basic Concepts of Probability Theory
54
5
2
1
3
2
3
1
6
50011
P
P[0] = = P[1]
177
Example
Suppose traffic engineers have coordinated the timingof two traffic lights to encourage a run of greenlights. In particular, the timing was designed so thatwith probability 0.8 a driver will find the second light
to have the same color as the first. Assuming thefirst light is equally likely to be red or green, what isthe probability P[G2] that the second light is green?
, ,at least one light? Lastly, what is P[G1|R2], the
conditional probability of a green first light given ared second light?
Chapter 2 : Basic Concepts of Probability Theory
178
-
8/3/2019 ReviewCh2-Basic Concept Probability
90/92
6/7/2011
90
Solution
Given P[G2|G1] = P[R2|R1] = 0.8
= =1 1 .
0.5
0.8
0.2
P[G1G2] = 0.4
P[G1R2] = 0.1
G1
R2
G2
Chapter 2 : Basic Concepts of Probability Theory
0.5
0.8
0.2
P[R1R2] = 0.4
P[R1G2] = 0.1
R1
R2
G2
179
(Continue)
What is the probability P[G2] that the second
li ht is reen?
0.5
0.8
0.2
0.2
P[G1G2] = 0.4
P[G1R2] = 0.1
P[R1G2] = 0.1
G1
R2
G2
G2
P[G2] = P[G1G2] + P[R1G2] = 0.4 + 0.1 = 0.5
Chapter 2 : Basic Concepts of Probability Theory
.
0.8 P[R1R2] = 0.4
R1
R2
180
-
8/3/2019 ReviewCh2-Basic Concept Probability
91/92
6/7/2011
91
(Continue)
What is the probability P[W] that you wait for at
least one li ht?
0.5
0.5
0.8
0.2
0.2
P[G1G2] = 0.4
P[G1R2] = 0.1
P[R1G2] = 0.1R1
G1
R2
G2
G2
W = {G1R2 R1G2 R1R2}
P[W] = P[R1G2]+P[G1R2]+P[R1R2] = 0.1+0.1+0.4
= 0.6Chapter 2 : Basic Concepts of Probability Theory
. P[R1R2] = 0.42
181
(Continue)
What is P[G1|R2], the conditional probability of a
reen first li ht iven a red second li ht?
0.5
0.8
0.2
0.2
P[G1G2] = 0.4
P[G1R2] = 0.1
P[R1G2] = 0.1
G1
R2
G2
G2
Chapter 2 : Basic Concepts of Probability Theory
2.05.0
1.0|
2
2121
RP
RGPRGP
.
0.8 P[R1R2] = 0.4
R1
R2
182
-
8/3/2019 ReviewCh2-Basic Concept Probability
92/92
6/7/2011
References
1. Alberto Leon-Garcia, Probability and RandomProcesses for Electrical En ineerin Addision-Wesley Publishing, 1994
2. Roy D. Yates, David J. Goodman, Probabilityand Stochastic Processes: A FriendlyIntroduction for Electrical and ComputerEngineering, 2nd, John Wiley & Sons, Inc, 2005
3. Jay L. Devore, Probability and Statistics forEngineering and the Sciences, 3rd edition,Brooks/Cole Publishing Company, USA, 1991.
Chapter 2 : Basic Concepts of Probability Theory
183