multiple random variables -...

Multiple Random Variables

Joint Cumulative Distribution

Function

Let X and Y be two random variables. Their joint cumulative

distribution function is FXY

x, y( ) P X x Y y .

0 FXY

x, y( ) 1 , < x < , < y <

FXY

,( ) = FXY

x,( ) = FXY

, y( ) = 0

FXY

,( ) = 1

FXY

x, y( ) does not decrease if either x or y increases or both increase

FXY

, y( ) = FY

y( ) and FXY

x,( ) = FX

x( )

Joint cumulative distribution function for tossing two dice

Joint Cumulative Distribution

Function

Joint Probability Mass Function

Let X and Y be two discrete random variables.

Their joint probability mass function is

PXY

x, y( ) P X = x Y = y .

Their joint sample space is

SXY

= x, y( ) | PXY

x, y( ) > 0{ }.

PXY

x, y( )x S

Xy S

Y

= 1 , P A = PXY

x, y( )x ,y( ) A

PX

x( ) = PXY

x, y( )y S

Y

, PY

y( ) = PXY

x, y( )x S

X

E g x, y( ) = g x, y( )PXY

x, y( )x S

Xy S

Y


Let a random variable X have a PMF

PXY

x, y( ) =

0.8x( ) 0.7

y( )41.17

, 0 x < 5, 4 y < 2

0 , otherwise

Joint Probability Density

Function

fXY

x, y( ) =

2

x yF

XYx, y( )( ) , f

XYx, y( ) 0 , < x < , < y <

fXY

x, y( )dxdy = 1 , FXY

x, y( ) = fXY

,( )d

x

d

y

fX

x( ) = fXY

x, y( )dy and fY

y( ) = fXY

x, y( )dx

P X ,Y( ) R = fXY

x, y( )dxdyR

P x1< X x

2, y

1< Y y

2= f

XYx, y( )dx

x1

x2

dyy

1

y2

E g X ,Y( )( ) = g x, y( )fXY

x, y( )dxdy

The Unit Rectangle Function

rect t( ) =

1 , t < 1 / 2

1 / 2 , t = 1 / 2

0 , t > 1 / 2

= u t +1 / 2( ) u t 1 / 2( )

The product signal g(t)rect(t) can be thought of as the signal g(t)“turned on” at time t = -1/2 and “turned back off” at time t = +1/2.

Let

fXY

x, y( ) =1

wX

wY

rectx X

0

wX

recty Y

0

wY

E X( ) = x fXY

x, y( )dxdy = X0

E Y( ) = y fXY

x, y( )dxdy = Y0

E XY( ) = xy fXY

x, y( )dxdy = X0Y

0Correlation of X and Y

fX

x( ) = fXY

x, y( )dy =1

wX

rectx X

0

wX


Function

For x < X0

wX

/ 2 or y < Y0

wY

/ 2, FXY

x, y( ) = 0

For x > X0

+ wX

/ 2 and y > Y0

+ wY

/ 2, FXY

x, y( ) = 1

For X0

wX

/ 2 < x < X0

+ wX

/ 2 and y > Y0

+ wY

/ 2,

FXY

x, y( ) =1

wX

wY

dudvX

0w

X/2

x

Y0

wY

/2

Y0+w

Y/2

=x X

0w

X/ 2( )

wX

For x > X0

+ wX

/ 2 and Y0

wY

/ 2 < y < Y0

+ wY

/ 2,

FXY

x, y( ) =1

wX

wY

dudvX

0w

X/2

X0+w

X/2

Y0

wY

/2

y

=y Y

0w

Y/ 2( )

wY

For X0

wX

/ 2 < x < X0

+ wX

/ 2 and Y0

wY

/ 2 < y < Y0

+ wY

/ 2,

FXY

x, y( ) =1

wX

wY

dudvX

0w

X/2

x

Y0

wY

/2

y

=x X

0w

X/ 2( )

wX

y Y0

wY

/ 2( )w

Y


Function


Function

Combinations of Two Random

VariablesExample

If the joint pdf of X and Y is fX

x, y( ) = ex u x( )e

y u y( )find the pdf of Z = X / Y . Since X and Y are never negative

Z is never negative.

FZ

z( ) = P Z z( ) = P X / Y z( ) F

Zz( ) = P X zY Y > 0 + P X zY Y < 0

Since Y is never negative

FZ

z( ) = P X zY Y > 0

FZ

z( ) = fXY

x, y( )dxdy

zy

= e xe ydxdy0

zy

0

, z 0

FZ

z( ) = 1 e zy( )e ydxdy0

=e

y z+1( )

z +1e y

0

=z

z +1 , z 0

fZ

z( ) =F

Zz( )

z=

1

z +1( )2

, z 0

0 , z < 0

fZ

z( ) =u z( )

z +1( )2


Variables


Variables

Example

The joint pdf of X and Y is defined as

fXY

x, y( ) =6x , x 0, y 0,x + y 1

0 , otherwise

Define Z = X Y . Find the pdf of Z.

Given the constraints on X and Y , 1 Z 1.

Z = X Y intersects X + Y = 1 at X =1+ Z

2 , Y =

1 Z

2.


Variables

For 0 z 1, FZ

z( ) = 1 6xdxy+ z

1 y

dy0

1 z( )/2

= 1 3x2

y+ z

1 y

dy0

1 z( )/2

FZ

z( ) = 13

41 z( ) 1 z2( ) f

Zz( ) =

3

41 z( ) 1+ 3z( )


Variables

For 1 z 0,

FZ

z( ) = 2 6xdx0

y+ z

dyz

1 z( )/2

= 6 x2

0

y+ z

dyz

1 z( )/2

= 6 y + z( )2

dyz

1 z( )/2

FZ

z( ) =1+ z( )

3

4f

Zz( ) =

3 1+ z( )2

4


Variables


Variables

Conditional Probability FX |A

x( ) =P X x( ) A

P A

Let A = Y y{ }

FX | Y y

x( ) =P X x Y y

P Y y=

FXY

x, y( )F

Yy( )

Let A = y1< Y y

2{ }

FX | y

1<Y y

2

x( ) =F

XYx, y

2( ) FXY

x, y1( )

FY

y2( ) F

Yy

1( )


Function

Let A = Y = y{ }

FX | Y = y

x( ) = limy 0

FXY

x, y + y( ) FXY

x, y( )F

Yy + y( ) F

Yy( )

=y

FXY

x, y( )( )d

dyF

Yy( )( )

FX | Y = y

x( ) =y

FXY

x, y( )( )

fY

y( ) , f

X |Y = yx( ) =

xF

X | Y = yx( )( ) =

fXY

x, y( )f

Yy( )

Similarly, fY |X =x

y( ) =f

XYx, y( )

fX

x( )


Function

In a simplified notation

fX |Y

x( ) =f

XYx, y( )

fY

y( )and f

Y |Xy( ) =

fXY

x, y( )f

Xx( )

Bayes’ Theorem

fX |Y

x( )fY

y( ) = fY |X

y( )fX

x( )Marginal pdf’s from joint or conditional pdf’s

fX

x( ) = fXY

x, y( )dy = fX |Y

x( )fY

y( )dy

fY

y( ) = fXY

x, y( )dx = fY |X

y( )fX

x( )dx


Function

It can be shown that, analogous to pdf, the conditional joint

PMF of X and Y given Y = y is

PX |Y

x | y( ) =P

XYx, y( )

PY

y( )and P

Y |Xy | x( ) =

PXY

x, y( )P

Xx( )

Bayes’ Theorem

PX |Y

x | y( )PY

y( ) = PY |X

y | x( )PX

x( )Marginal PMF’s from joint or conditional PMF’s

PX

x( ) = PXY

x, y( )y S

Y

= PX |Y

x | y( )PY

y( )y S

Y

PY

y( ) = PXY

x, y( )x S

X

= PY |X

y | x( )PX

x( )x S

X


Independent Random Variables

If two continuous random variables X and Y are independent then

fX |Y

x( ) = fX

x( ) =f

XYx, y( )

fY

y( )and f

Y |Xy( ) = f

Yy( ) =

fXY

x, y( )f

Xx( )

.

Therefore fXY

x, y( ) = fX

x( )fY

y( ) and their correlation is the

product of their expected values

E XY( ) = xy fXY

x, y( )dxdy = y fY

y( )dy x fX

x( )dx

E XY( ) = E X( )E Y( )


If two discrete random variables X and Y are independent then

PX |Y

x | y( ) = PX

x( ) =P

XYx, y( )

PY

y( )and P

Y |Xy | x( ) = P

Yy( ) =

PXY

x, y( )P

Xx( )

.

Therefore PXY

x, y( ) = PX

x( )PY

y( ) and their correlation is the

product of their expected values

E XY( ) = xy PXY

x, y( )x S

Xy S

Y

= y PY

y( )y S

Y

x PX

x( )x S

X

E XY( ) = E X( )E Y( )

Covariance

XYE X E X( ) Y E Y( )

*

= x E X( )( ) y* E Y *( )( )fXY

x, y( )dxdy

or = x E X( )( ) y* E Y *( )( )PXY

x, y( )x S

Xy S

Y

XY= E XY *( ) E X( )E Y *( )

If X and Y are independent,

XY= E X( )E Y *( ) E X( )E Y *( ) = 0


Correlation Coefficient

XY= E

X E X( )

X

Y * E Y *( )Y

=x E X( )

X

y* E Y *( )Y

fXY

x, y( )dxdy

or =x E X( )

X

y* E Y *( )Y

PXY

x, y( )x S

Xy S

Y

XY=

E XY *( ) E X( )E Y *( )X Y

= XY

X Y

If X and Y are independent = 0. If they are perfectly positively

correlated = +1 and if they are perfectly negatively correlated


If two random variables are independent, their covariance is

zero.

However, if two random variables have a zero covariance

that does not mean they are necessarily independent.

Independence Zero Covariance

Zero Covariance Independence


In the traditional jargon of random variable analysis, two

“uncorrelated” random variables have a covariance of zero.

Unfortunately, this does not also imply that their correlation is

zero. If their correlation is zero they are said to be orthogonal.

X and Y are "Uncorrelated"XY

= 0

X and Y are "Uncorrelated" E XY( ) = 0


Bivariate Gaussian Random

Variables

fXY

x, y( ) =

exp

x μX

X

22

XYx μ

X( ) y μY( )

X Y

+y μ

Y

Y

2

2 1XY

2( )

2X Y

1XY

2


Variables

Any cross section of a bivariate Gaussian pdf at any value of x or y

is Gaussian. The marginal pdf’s of X and Y can be found using

fX

x( ) = fXY

x, y( )dy

which turns out to be

fX

x( ) =e

x μX( )

2/2

X2

X2

Similarly, fY

y( ) =e

y μY( )

2/2

Y2

Y2


Variables

The conditional pdf of X given Y is

fX |Y

x( ) =

expx μ

X( ) XY X/

Y( ) y μY( )( )

2

2X

2 1XY

2( )

2X

1XY

2

The conditional pdf of Y given X is

fY |X

y( ) =

expy μ

Y( ) XY Y/

X( ) x μX( )( )

2

2Y

2 1XY

2( )

2Y

1XY

2


Variables

multiple random variables -...

Documents