review02-random variables (1)

Review of Random Variables

for Communications

EN 2072

Semester 4 May 2011

Prof. Dileeka Dias

Department of Electronic & Telecommunication Engineering

University of Moratuwa

Contents

Introduction Random Variables

Discrete Continuous

Joint Random Variables Discrete Continuous

Functions of Random Variables Single Random Variable

Mean Variance Moments

Two Random variables Correlation and Covariance

Random Variables: Definition

Outcome of a random experiment may be A numerical value (1, 2, 3, 4, 5, 6) Described by a phrase (Heads, Tails)

From a mathematical point of view it is preferable to have a numerical value (real number) assigned to each sample point according to some rule.

If there are m sample points , we assign a real number x( ) to each sample point.

X( ) is the function that maps sample points into real numbers x1, x2, .. xm

X is a random variable which takes on values x1, x2, .. xm.


Example 1 Heads 1

Tails 0

Example 2 1 10

2 20

3 30

4 40

5 50

6 60

X = 1 or 0 , each with probability 1/2

X = 10, 20, 30 .60, each with

probability 1/6

iix 10

"" ,0

"" ,1

Tails

Headsx

i

ii

Random Variables: Discrete

A discrete random variable maps events to values of a countable set (e.g., the integers), with each value in the range having probability greater than or equal to zero.

A discrete random variable is described by a discrete proability density function (probablity mass function)

1)(where

....... ,2 ,1),(

i

iX

iX

xP

mixP


Discrete Probability Density Function (PDF) or Probability Mass Function (PMF) and Discrete Cumulative Distribution Function (CDF)

Discrete PDF: Discrete CDF:

4 ,6

2

2 ,6

3

1 ,6

1

)(

x

x

x

xP iX

4 ,1

42 ,6

4

21 ,6

11 ,0

)()(

x

x

x

x

xXPxFX


Example: The Poisson random variable

A random variable X is said to have a Poisson distribution if

Where is called average or the expected value

...... 3 ,2 ,1 ,0 ,!

)(

kk

ekXP

k

PDF CDF

Example: Telephone calls arriving at a

switch, page view requests to a website are examples of Poisson processes

The probability that there are k incoming calls during the time t and t+ is given by

Where is the average call arrival rate

!

)()()(

k

ektNtNP

k


Random Variables: Continuous

A random variable is called continuous if it can assume all possible values in the possible range of the random variable.

The continuous random variable X is described by the (continuous) probability density function.


It is denoted by , the probability that the random variable X takes the value between x and x+ x where is a very small change in X. )()( xxXxPxfX

)(xfX


If there are two points a and b then the probability that the random variable will take the value between a and b is given by

1)(

dxxf X


The CDF of a continuous random variable is given by:

x

UX duufxF )()(

)()( xFxf Xdxd

X

xexf

x

X ,2

1)(

2

2

2

)(

2

),( 2N

onDistributi Normal Standard )1,0( N

Source: Wikipedia

Mean: Variance: 2


The Gaussian Random Variable


2

2/

0

0

2

)(

2

0

2

)(

2

2

)(

2

22

1

2

1

1

2

1

2

1

2

1

2

1

)()(

2

2

2

2

2

2

2

2

xerf

dte

duedue

due

duufxF

x

t

x uu

x u

x

XX


x

t dtexerf

0

22

2

ut


xQ

xerfxFX

1

22

1

2

1)(

2


x

t dtexerf

0

22

21

2

1 xerfxQ


Example:

Transmitted pulses

Received signal

1 p(t) 0 -p(t)

The received signal is Sampled at the peak point And compared with a threshold of 0 The sample consists of a contribution from the signal and a contribution from the AWGN that corrupts the channel. The sample value can be Ap + n -Ap + n

n is a sample value of the random variable with zero mean


Probability of error

P(e|0) = P(-AP + n) > 0

= P(n > Ap )

P(e|1) = P(AP + n) < 0

= (n< - AP)

P(n > Ap ) P(n< - AP)


For a Gaussian Random variable

)(1)( xXP

xQxFX

nQ

nQnXP 11)(

PP

PP

AQAnPeP

AQAnPeP

)()1|(

)()0|(

Two (Joint) Random Variables: Discrete

If X and Y are two discrete random variables, the conditional probability of xi and yj is given by

)|(| jiYX yxP

1)|()|( || ijXYj

jiYX

i

xyPyxP

)()|()()|(),( || iXijXYjYjiYXjiXY xPxyPyPyxPyxP

Using

Conditional Probability

Joint Probability

Joint Random Variables : Discrete

)(

)|()(

)()|(),(

|

|

jY

jiYX

i

jY

jYjiYX

i

jiXY

i

yP

yxPyP

yPyxPyxP

)()|()()|(),( || iXijXYjYjiYXjiXY xPxyPyPyxPyxP

)(),( iXjiXY

j

xPyxP Similarly Marginal Probabilities


)()|()()|(),( || jXijXYjYjiYXjiXY xPxyPyPyxPyxP

)()(),(

Hence,

)()|(

or )()|(

ift independen are Y and X

|

|

jYiXjiXY

jYijXY

iXjiYX

yPxPyxP

yPxyP

xPyxP


Example: A BSC has error probability p. The probability of transmitting a 1 is a and the probability of transmitting a 0 is 1-a.

Determine the probabilities of receiving a 1 and a 0 at the receiver.

X: RV indicating the input Y: RV indicating the output x1 = 1, x2 = 0 PX (1 ) = a, PX (0 ) = 1-a

pPP

pPP

XYXY

XYXY

1)1|1()0|0(

)0|1()1|0(

||

||


y1 = 1

PY (1 ) = ??

y2 = 0

PY (0 ) = ??

PX (1 ) = a x1 = 1

PX (0 ) = 1-a

x2 = 0

Probability of error

Probability of correct reception

paap

apapPPPPP

j

XYXYY

2)(

)1()1()0()0|1()1()1|1()1(

1for

||

)()|(),()( | iXijXY

i

jiXY

i

jY xPxyPyxPyP

)1(12)(1

)1)(1()0()0|0()1()1|0()0(

2for

||

Y

XYXYY

Ppaap

appaPPPPP

j

Joint CDF of continuous random variables

Joint PDF of continuous random variables

Joint Random Variables: Continuous

1),(

dxdyyxf XY

) and (),( yYxXPyxFXY

),(

),(2

yxFyx

yxf XYXY

dxdyyxfyYyxXxP

x

x

y

y

XY ),(),(

2

1

2

1

2121

Joint Random Variables : Continuous

Conditional Densities for continuous random variables

X and Y are independent if

)(

)()|()|(

Hence,

)()|(),(

)()|(),(

||

|

|

yf

xfxyfyxf

xfxyfyxf

yfyxfyxf

Y

XXYYX

XXYXY

YYXXY

Bayes Rule for Continuous RVs

Conditional PDFs

)()|(

or )()|(

|

|

yfxyf

xfyxf

YXY

XYX

Joint PDFs

)()(),( yfxfyxf YXXY Which means that

Functions of a Random Variable

The Mean (Expected Value)

Example: for a Gaussian RV,

i

ii

X

xPx

dxxxfXXE

)(

or

)(][__

dyedyye

dyeyXE

dxxeXE

yy

y

x

2

2

2

2

2

2

2

2

2

)(

2

)(

2

)(

2

)(

2

2

1

)(2

1][

y Let x

2

1][

Odd function of y 2

DC value of a signal



Let Y = g(X)

Therefore:

Mean Square Value

A function of a RV

i

ii

X

xPxg

dxxfxgYYE

)()(

or

)()(][__

dxxfxXE X )(][22



Let Y = g(X)

Therefore:

A function of a RV

i

ii

X

xPxg

dxxfxgYYE

)()(

or

)()(][__

dxxfxXE X )(][22

Mean Square Value of a signal

Root Mean Square (RMS) value of a signal

][ 2XE


Example

A sinusoidal signal is given by . This is sampled at random time instants. The sampled output is a random variable X.

Find E[X] and E[X2]

The sampling instant is a random variable T. Let be another rv.

tA cos

2,0~ U

q

P (q

1/2

20

Uniform Distribution

cosAX


Example contd.

22

2

1cos][

02

1cos][

cos

22

2

0

222

2

0

AA

dAXE

dAXE

AX

q

q

q

q


Variance

2__

2_

2__

2

)(Deviation Std.

)()( Variance

XXEX

dxxfXxXXEX XX

Average AC power of a signal


Variance

22

2__2

2____2

2____2

2__

2

][][

][

][2][

2

XEXE

XXE

XEXXEXE

XXXXE

XXEX


Moments

The nth moment of X is given by:

The nth central moment of X is given by:

dxxfxXXE Xnnn )(][

__

dxxfxxXXE X

n

n )(])[(___

Functions of two Random Variables

Correlation and Covariance Estimate the nature of dependence between two

random variables

Correlation

If X and Y are independent:

X and Y are said to be orthogonal if

dxdyyxxyfXYER XYXY ),(][

][][)()( YEXEdyyyfdxxxfR YXXY

0][ XYERXY X Y


Correlation and Covariance

Covariance

dxdyyxfYyXxYYXXEC XYXYXY ),(

________

____

________

____

][][][

][][][

YXRYEXEXYE

YXXEYYEXXYE

YYXXEC

XY

XYXY

For Independent RVs

0

][][____

XY

XY

C

YXYEXER

Correlation can be Positive Zero or Negative



Covariance

For Independent RVs

0

][][____

XY

XY

C

YXYEXER

Correlation can be Positive Zero or Negative

____

YXRC XYXY

X and Y are uncorrelated



X and Y are positively correlated

X and Z are negatively correlated X and W are uncorrelated

Scatter Plots

Relationship between Independence and Correlatedness

Correlation Coefficient


11

XY

YX

XYXY

C

review02-random variables (1)

Documents