chap 2 probability and stochastic processes

7/31/2019 Chap 2 Probability and Stochastic Processes

1/26

Digital Communication

Chap.2


2/26

Proverbs 9:11

I returned and saw under the sun that

The race is not to the swift,

Nor the battle to the strong,Nor bread to the wise,

Nor riches to men of understanding,

Nor favor to men of skill;But time and chance happen to them all.


3/26

1.

Let us consider a die, the set of all possible

outcomes is


4/26

An event is a subset of S, and may

consist of any number of samplepoints.

Example, the event A defined as

A={2,4} consist of the outcomes 2 and 4.

The complement of A denoted by consists of all the sample points in S that

are not in A and, hence, ={1,3,5,6}.


5/26

Two events are said to be mutually exclusive ifthey have no sample points in common.

That is, if the occurrence of one event exclude

the occurrence of the other. If A is defined as, A={2,4} and the event B is

defined as B={1,3,6} then A and B are mutually

exclusive events. The union(sum) of two events is an event that

consists of all the sample points in the twoevents.


6/26

If the B is the event defined as

B={1,3,6} and C is the event defined

as C={1,2,3} then, the union of B

and C, denoted by BUC is the event

D=BUC

={1,2,3,6}

Similarly, A U =S(S is the entiresample space or the certain event)


7/26

If

represents the intersection of the

events B and C, defined by B={1,3,6}and C={1,2,3}, then E={1,3}

CBE


8/26

When the events are mutually exclusive, the

intersection is the null event, denoted as .

A A


9/26

Associated with each event A contained in S is

its probability P(A)

The probability of an event A satisfies the

condition P(A)0

The probability of a sample space(certain

event) is P(S)=1

If Ai, i=1,2,3.., are mutually exclusive events ,

Ai and Aj are possibly infinite number of

events

B A


10/26

ji AA

i i

AP

i

AiP )()(


11/26

example

A={2,4} and B={1,3,6}

P(AUB)= P(A)+P(B)=1/3+1/2=2/6+3/6=5/6


12/26

JOINT EVENTS and JOINT PROBABILITY

Inseated of dealing with a single experiment,

let us perform two experiments and consider

their outcomes.

Example: let take two separate tosses of a

single die or a single toss of a two dice.

The sample space S consists of the 36 two-

tuples (i,j), where i,j=1,2,..,6.


13/26

1,1 2,1 3,1 4,1 5,1 6,1

1,2 2,2 3,2 4,2 5,2 6,2

1,3 2,3 3,3 4,3 5,3 6,3

1,4 2,4 3,4 4,4 5,4 6,4

1,5 2,5 3,5 4,5 5,5 6,5

1,6 2,6 3,6 4,6 5,6 6,6


14/26

JOINT EVENTS and JOINT

PROBABILITY

If one experiment has the possible outcomes

Ai, i=1,2,,n, and the second experiment has

the possible outcomes Bj, j=1,2,,m, then the

combined experiment has the possible jointoutcomes (Ai,Bj), i=1,2,,n,j=1,2,,m.

Associated with each joint outcome (Ai,Bj) is

the joint probability P(Ai,Bj) which satisfies thecondition

0P(Ai,Bj)1


15/26


PROBABILITY

Assuming that the outcomes Bj, j=1,2,,m are

mutually exclusive, it follows that (if the union

contains A)

m

j

AiPBjAiP1

)(),(


16/26

JOINT EVENTS and JOINTPROBABILITY .

Similarly, if the outcomes Ai,

i=1,2,,n are mutually exclusive

then( if the union contains B)

n

i

BjPBjAiP1

)(),(


17/26

If all the outcomes of the two

experiments are mutuallyexclusive then,

n

i

m

j

BjAi1 1

1),(


PROBABILITY


18/26

Conditional probability

When a combined experiment in which a joint

event occurs with probability P(A,B). If the

event B has occurred and we wish to

determine the probability of occurrence of theevent A, this is called the conditional

probability of event A given the occurrence of

the event B. It is defined as

)(

)()|(

AP

BAPABP


19/26

Also the probability of the eventB conditioned on the occurrence

of the event A is

.

)(

)()|(

AP

BAPABP


Provided P(B)>0, P(A)>0


20/26


The above conditional formula may be written

as

Example: Consider the events B and C from

the sample space S={1,2,3,4,5,6},

B={1,3,6}, C={1,2,3}, find P(C\B)? Solution:

)()|()()\()(),( APABPBPBAPBAPBAP (P

3

2

6

36

2

)(

)()\(

BP

CBPBCP


21/26


Low of Total Probability: For events A and B, we have,

Generalizesd to any partition of the entire probability

space: if B1, B2, are mutually exclusive events

such that their union covers the entire probability

space(if the union contains A).

][]\[][]\[][][][CCC

BPBAPBPBAPBAPBAPAP

].[]\[][][i

i

ii

i

BPBAPBAPAP


22/26

Given

Given P[A\B], we can compute P[B\A] as

follows

][]\[][]\[

][]\[

][

][]\[]\[

ccBPBAPBPBAP

BPBAP

AP

BPBAPABP


23/26


Bayes Theorem: if Ai, i=1,2,,n, are mutuallyexclusive events such that

and B is an arbitrary event with nonzero

probability then

n

i

SAi

1

n

i

AiPAiBP

AiPAiBPBAiP

BPBAPBAP

1

)()\(

)()|()/(

)()\(),(


24/26

Conditional Probability: Bayes

Theorem

Ai: transmited signal,

B: Received signal

n

i

AiPAiBP

AiPAiBP

BP

BAiPBAiP

1

)()\(

)()|(

)(

),()\(


25/26

Statistical Independence

Consider two event A and B and their

conditional Probability P(A\B).

Suppose the occurrence of A does not depend

on the occurrence of B.

P(A|B)=P(A)

When the events A and B satisfy this relation,

they are said to be statistically independent.

)()()()\(),( BPAPBPBAPBAP


26/26

Statistical Independence

Statistically independence can be extended to

three or more events.

Three statistically independent events A1, A2

and A3 must satisfy the following conditions.

P[A1,A2]=P[A1]P[A2]

P[A1,A3]=P[A1]P[A3]

P[A2,A3]=P[A2]P[A3]

P[A1,A2,A3]=P[A1]P[A2]P[A3]

chap 2 probability and stochastic processes

Documents