1

Lecture 7

! Conditional Distributions

! Multivariate Distributions

2

Conditional Distributions! Suppose that X and Y have a discrete joint

distribution for which the joint p.f. is f.! For any y such that , the conditional p.f.

of X given that Y=y is:

Check: )(),(

)Pr()Pr()|Pr()|(

2

1

yfyxf

yYyYandxXyYxXyxg

=

===

====

1)()(

1),()(

1)|( 222

1 === åå yfyf

yxfyf

yxgxx

( )2 0f y >

Conditional Distributions Behave Just Like Distributions!

3

! Similarly, for any x such that , theconditional p.f. of Y given that X=x is:

( )1 0f x >

)(),(

)Pr()Pr()|Pr()|(

1

2

xfyxf

xXyYandxXxXyYxyg

=

===

====

4

Example: Education and Monthly Personal Income

! Suppose that education level (X) can take values 1=“belowcollege”, 2=“college”, and 3=“above college”. Supposethat monthly personal income (Y) can take values1=“<2000”, 2=“2000-4999”, 3=“5000-9999”, and4=“>=10000”.

! Suppose that in certain population, the probabilities fordifferent combinations of education level and monthlypersonal income are given by the table below.

Y 1 2 3 4

1 0.2 0.1 0.06 0.04X 2 0.09 0.06 0.1 0.15

3 0.01 0.03 0.08 0.08n What is the conditional p.f. of Y given X=2?

5

! Suppose that in certain population, the probabilities fordifferent combinations of education level and monthlypersonal income are given by the table below.

Y 1 2 3 4

1 0.2 0.1 0.06 0.04X 2 0.09 0.06 0.1 0.15

3 0.01 0.03 0.08 0.08 What is the conditional p.f. of Y given X=2?

21

2 2 2 2

(2, ) (2, )( | 2)(2) 0.4

0.09 9 0.06 3 0.1 1 0.15 3(1| 2) (2 | 2) (3 | 2) (4 | 2)0.4 40 0.4 20 0.4 4 0.4 8

f y f yg yf

g g g g

= =

= = = = = = = =

The conditional probabilities proportional to the 2nd row, but sum up to 1!

Example: Education and Monthly Personal Income

6

Continuous Conditional Distributions! Suppose that X and Y have a continuous joint

distribution. For any y such that , theconditional p.d.f. of X given that Y=y can be defined as

Similarly, for any x such that , the conditionalp.d.f. of Y given that X=x can be defined as

¥<<¥-= xforyfyxfyxg)(),()|(

21

¥<<¥-= yforxfyxfxyg)(),()|(

12

( )2 0f y >

( )1 0f x >

1)|(1 =ò¥

¥-dxyxg

1)|(2 =ò¥

¥-dyxyg

7

8

Example! Suppose the joint p.d.f. of X and Y is:

(1) Please find out .

(2) If x=1/2, then find out .

ïî

ïíì ££=

otherwise

yxforyxyxf0

1421

),(22

)|(2 xyg

3 1Pr |4 2

Y Xæ ö³ =ç ÷è ø

9

Example! Suppose the joint p.d.f. of X and Y is:

For –1<x<0 or 0<x<1, , then

If x=1/2, then

ïî

ïíì ££=

otherwise

yxforyxyxf0

1421

),(22

11)1(821

421)(

1 4221 2

££--== ò xforxxydyxxfx

0)(1 >xf

ïî

ïíì ££-==

otherwise

yxforxy

xfyxfxyg

0

112

)(),()|(

24

12

1 13 324 4

3 1 1 2 7Pr | | 154 2 2 1516

yY X g y dy dyæ ö æ ö³ = = = =ç ÷ ç ÷è ø è øò ò

10

Construction of The Joint Distribution! For any y such that f2(y)>0 and any x,

If f2(y0)=0 for some y0, then we can assume that f(x,y0)=0 for all values of x.

! Thus, for all values of x and y,

Similarly,

)|()(),( 12 yxgyfyxf =

)|()(),( 12 yxgyfyxf =

)|()(),( 21 xygxfyxf =

11

Example

! Suppose that a point X is chosen from auniform distribution on the interval (0,1).After X=x has been observed, a point Y isthen chosen from a uniform distribution onthe interval (x,1). What is the marginal p.d.f.of Y? What is the conditional p.d.f of X givenY=y?

12。

，那么，对于

。

所以，

。

，对于

已知，

ïî

ïíì <<

---

=

<<

ïî

ïíì <<--=

-=

ïî

ïíì <<<-=

ïî

ïíì <<-=

<<îíì <<

=

ò

otherwise

yxforyxyxg

yotherwise

yforydxxyf

otherwise

yxforxyxf

otherwise

yxforxxyg

xotherwise

xforxf

y

0

0)1log()1(

1)|(

100

10)1log(11

)(

0

1011

),(

0

111

)|(

100

101)(

1

02

2

1

13

Independent Random Variables

! Suppose that X and Y have a continuous joint distribution.X and Y are independent

)()(),( 21 yfxfyxf =

0)(..)()|( 211 >"= yftsyforxfyxg

0)(..)()|( 122 >"= xftsxforyfxyg

14

Multivariate Distributions! The joint distribution of more than two

random variables is called a multivariatedistribution.

! Joint d.f. of n random variables X1,…Xn is

! We can use the random vector X=(X1,…,Xn),and let x=(x1,…,xn), then the d.f. for therandom vector becomes F(x), which is definedon n-dimensional space

1 1 1( , , ) Pr( , , )n n nF x x X x X x= £ £L L

nR

15

Discrete Distributions! The random vector X=(X1,…,Xn) can only take

a finite number or an infinite sequence ofdifferent possible values (x1,…,xn) in

! The joint p.f. for any point x=(x1,…,xn) in :

Or simply! For any subset

nRnR

1 1 1( , , ) Pr( , , )n n nf x x X x X x= = =L L)Pr()( xXx ==f

nRAÌ

åÎ

=ÎAfA

xxX )()Pr(

16

Continuous Distributions! There is a nonnegative function f defined on

such that for any subset

The function f is the joint p.d.f. of X=(X1,…,Xn)! The joint p.d.f. can be derived from the joint

d.f. by

at all points (x1,…,xn) where the derivativeexists.

nRnRAÌ

1 1Pr( ) ( , , )n nA

A f x x dx dxÎ = ò òX L L L

11

1

( , , )( , , )n

nn

n

F x xf x xx x

¶=

¶ ¶LLL

17

Marginal Distributions! Marginal p.f. (discrete):

! Marginal p.d.f. (continuous):

å åå

åå=

=

2 4

2

),,(),(

),,()(

13113

111

x xn

x

xn

x

n

n

xxfxxf

xxfxf

!!

!!

nn

n

nn

n

dxdxdxxxfxxf

dxdxxxfxf

!!"#"$%!

!!"#"$%!

421

2

3113

21

1

11

),,(),(

),,()(

-

¥

¥-

¥

¥-

-

¥

¥-

¥

¥-

òò

òò

=

=

18

! Marginal d.f. (discrete or continuous):

1 1 1 1 1 1 2

1

2, ,

13 1 3 1 1 3 3

1 1 2 3 3 4

1

2,4, ,

( ) Pr( ) Pr( , , , )lim ( , , )

( , ) Pr( , )Pr( , , , , )lim ( , , )

j

j

n

nx

j n

n

nx

j n

F x X x X x X XF x x

F x x X x X xX x X X x X XF x x

®¥

=

®¥

=

= £ = £ < ¥ < ¥=

= £ £

= £ < ¥ £ < ¥ < ¥=

L

L

LL

LL

19

Independent Random Variables

! n random variables X1,…,Xn are independent if,for any n sets A1,A2,…,An of real numbers,

1 1 2 2

1 1 2 2

Pr( , , , )Pr( )Pr( ) Pr( )

n n

n n

X A X A X AX A X A X A

Î Î Î

= Î Î Î

LL

1 2 1 1 2 2( , , , ) ( ) ( ) ( )n n nF x x x F x F x F x=L L

1 2 1 1 2 2( , , , ) ( ) ( ) ( )n n nf x x x f x f x f x=L L

20

Random Sample! Given a p.f. or p.d.f. f, we say that n random

variables X1,…,Xn form a random sample fromthis distribution if• these variables are independent;• the marginal p.f. or p.d.f. of each of them is f.

! The joint p.f. or p.d.f. g is specified at allpoints (x1,…,xn) in as:

We say that the variables are independent andidentically distributed (i.i.d). n is called thesample size.

nR

1 2 1 2( , , , ) ( ) ( ) ( )n ng x x x f x f x f x=L L

21

Example: Lifetimes of Light Bulbs

! Suppose the lifetimes of light bulbs produced ina certain factory are distributed according to:

What is the joint p.d.f. for the lifetimes of arandom sample of n light bulbs is drawn fromthe factory’s production?

îíì >

=-

otherwisexforxe

xfx

0,0

)(

22

Note: we will use exp(v) to denote ve

1111

exp , for 0, 1 ( , , ) ( )

0, otherwise

n nn

i i iiin i

i

x x x i , ,n g x x f x ==

=

ìæ ö æ ö- > =ïç ÷ ç ÷= = í è øè øïî

åÕÕK

L

23

Conditional Distributions

! For any values of x2,…,xn such thatf2…n(x2,…,xn)>0, the conditional p.f. or p.d.f. ofX1 given that X2=x2,…,Xn=xn is defined as:

1 21 1 2

2 2

( , , , )( | , , )( , , )

nn

n n

f x x xg x x xf x x

=L

LLL

24

! In general, suppose the random vector X isdivided into a k-dimensional random(sub)vector Y and a (n-k)-dimensional random(sub)vector Z.• The n-dimensional p.f. or p.d.f. of (Y,Z) is f.• The marginal (n-k)-dimensional p.f. or p.d.f. of Z is f2.• Then for any given point such that ,

the conditional k-dimensional p.f. or p.d.f. g1 of Ygiven Z=z is defined as:

knR -Îz 0)(2 >zf

kRforffg Î= yzzyzy)(),()|(

21

1 3 2 4. ., ( , | , , , ) ?nE g g x x x x x =L

211 3

24 2 4

( , , ) ( , )( , , , )

n

n n

f x x for x x Rf x x x

= ÎL

LL

25

Example! Suppose that Z is distributed as:

Given Z=z>0, X1 and X2 are i.i.d, each has a conditional p.d.f. as:

（1) What is the marginal joint p.d.f. of X1

and X2? (2)

îíì >

=-

otherwisezfore

zfz

0,02

)(2

0

îíì >

=-

otherwisexforze

zxgzx

0,0

)|(

(3) What is the conditional p.d.f. of z given X1=x1 and X2=x2 (x1>0 and x2>0)?

26

Solution: The joint conditional p.d.f. of X1 and X2 given Z=z>0 is:

The joint p.d.f. f of Z, X1 and X2 is:

îíì >>

=+-

otherwisexandxforez

zxxgxxz

000

)|,( 21)(2

2112

21

îíì >>>

=

=++-

otherwisexandxzforez

zxxgzfxxzfxxz

000,02

)|,()(),,(

21)2(2

2112021

21

27

The marginal joint p.d.f. of X1 and X2 is:

ïî

ïíì >>

++=

= ò¥

otherwise

xandxforxx

dzxxzfxxf

0

00)2(

4

),,(),(

21321

2102112

28

! Further,

! What is the conditional p.d.f. of z given X1=x1

and X2=x2 (x1>0 and x2>0)?

?)2(

4)4Pr(4

0

4

0 21321

212 =

++=<+ ò ò

-xdxdx

xxXX

ïî

ïíì >++=

=

++-

otherwise

zforezxx

xxfxxzfxxzg

xxz

0

0)2(21

),(),,(),|(

)2(2321

2112

21210

21