Download - Chap 2 Probability and Stochastic Processes
-
7/31/2019 Chap 2 Probability and Stochastic Processes
1/26
Digital Communication
Chap.2
-
7/31/2019 Chap 2 Probability and Stochastic Processes
2/26
Proverbs 9:11
I returned and saw under the sun that
The race is not to the swift,
Nor the battle to the strong,Nor bread to the wise,
Nor riches to men of understanding,
Nor favor to men of skill;But time and chance happen to them all.
-
7/31/2019 Chap 2 Probability and Stochastic Processes
3/26
1.
Let us consider a die, the set of all possible
outcomes is
-
7/31/2019 Chap 2 Probability and Stochastic Processes
4/26
An event is a subset of S, and may
consist of any number of samplepoints.
Example, the event A defined as
A={2,4} consist of the outcomes 2 and 4.
The complement of A denoted by consists of all the sample points in S that
are not in A and, hence, ={1,3,5,6}.
-
7/31/2019 Chap 2 Probability and Stochastic Processes
5/26
Two events are said to be mutually exclusive ifthey have no sample points in common.
That is, if the occurrence of one event exclude
the occurrence of the other. If A is defined as, A={2,4} and the event B is
defined as B={1,3,6} then A and B are mutually
exclusive events. The union(sum) of two events is an event that
consists of all the sample points in the twoevents.
-
7/31/2019 Chap 2 Probability and Stochastic Processes
6/26
If the B is the event defined as
B={1,3,6} and C is the event defined
as C={1,2,3} then, the union of B
and C, denoted by BUC is the event
D=BUC
={1,2,3,6}
Similarly, A U =S(S is the entiresample space or the certain event)
-
7/31/2019 Chap 2 Probability and Stochastic Processes
7/26
If
represents the intersection of the
events B and C, defined by B={1,3,6}and C={1,2,3}, then E={1,3}
CBE
-
7/31/2019 Chap 2 Probability and Stochastic Processes
8/26
When the events are mutually exclusive, the
intersection is the null event, denoted as .
A A
-
7/31/2019 Chap 2 Probability and Stochastic Processes
9/26
Associated with each event A contained in S is
its probability P(A)
The probability of an event A satisfies the
condition P(A)0
The probability of a sample space(certain
event) is P(S)=1
If Ai, i=1,2,3.., are mutually exclusive events ,
Ai and Aj are possibly infinite number of
events
B A
-
7/31/2019 Chap 2 Probability and Stochastic Processes
10/26
ji AA
i i
AP
i
AiP )()(
-
7/31/2019 Chap 2 Probability and Stochastic Processes
11/26
example
A={2,4} and B={1,3,6}
P(AUB)= P(A)+P(B)=1/3+1/2=2/6+3/6=5/6
-
7/31/2019 Chap 2 Probability and Stochastic Processes
12/26
JOINT EVENTS and JOINT PROBABILITY
Inseated of dealing with a single experiment,
let us perform two experiments and consider
their outcomes.
Example: let take two separate tosses of a
single die or a single toss of a two dice.
The sample space S consists of the 36 two-
tuples (i,j), where i,j=1,2,..,6.
-
7/31/2019 Chap 2 Probability and Stochastic Processes
13/26
1,1 2,1 3,1 4,1 5,1 6,1
1,2 2,2 3,2 4,2 5,2 6,2
1,3 2,3 3,3 4,3 5,3 6,3
1,4 2,4 3,4 4,4 5,4 6,4
1,5 2,5 3,5 4,5 5,5 6,5
1,6 2,6 3,6 4,6 5,6 6,6
-
7/31/2019 Chap 2 Probability and Stochastic Processes
14/26
JOINT EVENTS and JOINT
PROBABILITY
If one experiment has the possible outcomes
Ai, i=1,2,,n, and the second experiment has
the possible outcomes Bj, j=1,2,,m, then the
combined experiment has the possible jointoutcomes (Ai,Bj), i=1,2,,n,j=1,2,,m.
Associated with each joint outcome (Ai,Bj) is
the joint probability P(Ai,Bj) which satisfies thecondition
0P(Ai,Bj)1
-
7/31/2019 Chap 2 Probability and Stochastic Processes
15/26
JOINT EVENTS and JOINT
PROBABILITY
Assuming that the outcomes Bj, j=1,2,,m are
mutually exclusive, it follows that (if the union
contains A)
m
j
AiPBjAiP1
)(),(
-
7/31/2019 Chap 2 Probability and Stochastic Processes
16/26
JOINT EVENTS and JOINTPROBABILITY .
Similarly, if the outcomes Ai,
i=1,2,,n are mutually exclusive
then( if the union contains B)
n
i
BjPBjAiP1
)(),(
-
7/31/2019 Chap 2 Probability and Stochastic Processes
17/26
If all the outcomes of the two
experiments are mutuallyexclusive then,
n
i
m
j
BjAi1 1
1),(
JOINT EVENTS and JOINT
PROBABILITY
-
7/31/2019 Chap 2 Probability and Stochastic Processes
18/26
Conditional probability
When a combined experiment in which a joint
event occurs with probability P(A,B). If the
event B has occurred and we wish to
determine the probability of occurrence of theevent A, this is called the conditional
probability of event A given the occurrence of
the event B. It is defined as
)(
)()|(
AP
BAPABP
-
7/31/2019 Chap 2 Probability and Stochastic Processes
19/26
Also the probability of the eventB conditioned on the occurrence
of the event A is
.
)(
)()|(
AP
BAPABP
Conditional probability
Provided P(B)>0, P(A)>0
-
7/31/2019 Chap 2 Probability and Stochastic Processes
20/26
Conditional probability
The above conditional formula may be written
as
Example: Consider the events B and C from
the sample space S={1,2,3,4,5,6},
B={1,3,6}, C={1,2,3}, find P(C\B)? Solution:
)()|()()\()(),( APABPBPBAPBAPBAP (P
3
2
6
36
2
)(
)()\(
BP
CBPBCP
-
7/31/2019 Chap 2 Probability and Stochastic Processes
21/26
Conditional probability
Low of Total Probability: For events A and B, we have,
Generalizesd to any partition of the entire probability
space: if B1, B2, are mutually exclusive events
such that their union covers the entire probability
space(if the union contains A).
][]\[][]\[][][][CCC
BPBAPBPBAPBAPBAPAP
].[]\[][][i
i
ii
i
BPBAPBAPAP
-
7/31/2019 Chap 2 Probability and Stochastic Processes
22/26
Given
Given P[A\B], we can compute P[B\A] as
follows
][]\[][]\[
][]\[
][
][]\[]\[
ccBPBAPBPBAP
BPBAP
AP
BPBAPABP
-
7/31/2019 Chap 2 Probability and Stochastic Processes
23/26
Conditional probability
Bayes Theorem: if Ai, i=1,2,,n, are mutuallyexclusive events such that
and B is an arbitrary event with nonzero
probability then
n
i
SAi
1
n
i
AiPAiBP
AiPAiBPBAiP
BPBAPBAP
1
)()\(
)()|()/(
)()\(),(
-
7/31/2019 Chap 2 Probability and Stochastic Processes
24/26
Conditional Probability: Bayes
Theorem
Ai: transmited signal,
B: Received signal
n
i
AiPAiBP
AiPAiBP
BP
BAiPBAiP
1
)()\(
)()|(
)(
),()\(
-
7/31/2019 Chap 2 Probability and Stochastic Processes
25/26
Statistical Independence
Consider two event A and B and their
conditional Probability P(A\B).
Suppose the occurrence of A does not depend
on the occurrence of B.
P(A|B)=P(A)
When the events A and B satisfy this relation,
they are said to be statistically independent.
)()()()\(),( BPAPBPBAPBAP
-
7/31/2019 Chap 2 Probability and Stochastic Processes
26/26
Statistical Independence
Statistically independence can be extended to
three or more events.
Three statistically independent events A1, A2
and A3 must satisfy the following conditions.
P[A1,A2]=P[A1]P[A2]
P[A1,A3]=P[A1]P[A3]
P[A2,A3]=P[A2]P[A3]
P[A1,A2,A3]=P[A1]P[A2]P[A3]