lecture 6 - başkent Üniversitesiosezgin/lecture 6.pdf · lecture 6 transformation of random...
TRANSCRIPT
LECTURE 6
TRANSFORMATION OF RANDOM VARIABLES
TRANSFORMATION OF FUNCTION OF A RANDOM VARIABLE
UNIVARIATE TRANSFORMATIONS
TRANSFORMATION OF RANDOM VARIABLES
• If X is an rv with cdf F(x), then Y=g(X) is also an rv.
• If we write y=g(x), the function g(x) defines a mapping from the original sample space of X, S, to a new sample space, , the sample space
of the rv Y.
g(x): S
1-TO-1 TRANSFORMATION OF RANDOM VARIABLES
• Let y=g(x) define a 1-to-1 transformation. That is, the equation y=g(x) can be solved uniquely:
• Ex: Y=X-1 X=Y+1 1-to-1
• Ex: Y=X² X=± sqrt(Y) not 1-to-1
• When transformation is not 1-to-1, find disjoint partitions of S for which
transformation is 1-to-1.
)y(gx 1
If X is a discrete r.v. then S is countable. The sample space for Y=g(X) is ={y:y=g(x),x S}, also
countable. The pmf for Y is
1 1
Y
x g y x g y
f y P Y y P X x f x
Example
• Let X~GEO(p). That is,
• Find the p.m.f. of Y=X-1
• Solution: X=Y+1
• P.m.f. of the number of failures before the first success
• Recall: X~GEO(p) is the p.m.f. of number of Bernoulli trials required to get the first success
,...3,2,1xfor)p1(p)x(f 1x
,...2,1,0yfor)p1(p)1y(f)y(f yXY
Example
• Suppose X~Poisson with
• Let Y=4X X=Y/4
• Then
,...2,1,0!
)(
xx
exp
x
/ 4
( / 4) 0,4,8,...( / 4)!
yep x y y
y
CONTINUOUS RANDOM VARIABLE
• Let X be an rv of the continuous type with pdf f. Let y=g(x) be differentiable for all x and non-zero. Then, Y=g(X) is also an rv of the continuous type with pdf given by
..0
|)(|))(()(
11
wo
yforygdy
dygf
yh
• Let X have the density
1, 0 1
0, otherwise
xf x
Let Y=eX. X=g1 (y)=lnY dx=(1/y)dy.
11. ,0 log 1
1, 1
0, otherwise
h y yy
y eyh y
Example
Example
• Suppose X has an exponential distribution with
• Let Y=4X-2 X=(Y+2)/4
/1, 0, 0x
Xf x e x
X=g1 (y)= (Y+2)/4 dx=(1/4)dy.
( 2) / 41 1| | 2, 0
( ) 4
0 . .
ye yh y
o w
TRANSFORMATION THAT ARE NOT 1-TO-1
• Let y=g(x) if the equation y=g(x) can not be solved uniquely then not one to one transformation should be used.
When transformation is not 1-to-1, find disjoint partitions of S for which transformation is 1-to-1 and use 1-to-1 transformation or use cumulative method.
Example
• Let X be an rv with pmf
Let Y=|X| S ={2, 1,0,1,2} ={0,1,4}
4 1( ) 2, 1,0,1,2
31 2
x
p x x
4(0)
31
10(1) ( 1) (1)
31
17(2) ( 2) (2)
31
y
y x x
y x x
p
p p p
p p p
Example
• Let X be an rv with pmf
1/5, 2
1/ 6, 1
1/5, 0
1/15, 1
11/30, 2
x
x
p x x
x
x
Let Y=X2. S ={2, 1,0,1,2} ={0,1,4} 1/5, 0
( ) 7 /30, 1
17 /30, 4
y
p y y
y
Example • Stores located on a linear city with density f(x)=0.05
-10≤x ≤ 10, 0 otherwise
• Courier incurs a cost of U=16X2 when she delivers to a store located at X (her office is located at 0)
1600080
)()(
160004044
05.005.0)()(
44
416
2/1
4
4
2
uu
du
udFuf
uuuu
dxuUPuF
uX
uuU
uXuXuU
UU
u
uU
• Let X have the density
2 / 21, .
2
xf x e x
Let W=X2. Find the pdf of W.
Example
First step
Find the distribution function of W
G(w) = P[W ≤ w] = P[ X2 ≤ w]
if 0P w X w w
2
21
2
w x
w
e dx
F w F w
where 2
21
2
x
F x f x e
d wd w
F w F wdw dw
Second step
Find the density function of W
g(w) = G'(w).
1 1
2 2 2 21 1 1 1
2 22 2
w w
e w e w
1 1
2 21 1
2 2f w w f w w
1
2 21
if 0.2
w
w e w
TRANSFORMATION OF FUNCTION OF TWO OR MORE RANDOM
VARIABLES
BIVARIATE TRANSFORMATIONS
DISCRETE CASE
• Let X1 and X2 be a bivariate random vector with a
known probability distribution function.
• Consider a new bivariate random vector (U, V)
defined by U=g1(X1, X2) and V=g2(X1, X2) where
g1(X1, X2) and g2(X1, X2) are some functions of X1
and X2 .
• Then, the joint pmf of (U,V) is
VUAxx
XXVU xxfvVuUvuf,21
21
,
21,, ,,Pr,
EXAMPLE • Let X1 and X2 be independent Poisson distribution
random variables with parameters 1 and 2 with joint pmf .
• Find the distribution of Y1=X1+X2. We need to define
new variable Y2=X2. Then Y1=0,1,2,…, Y2=0,1,2,…,Y1 and X2=Y2 and X1=Y1-Y2.
1 2 1 2
1 21 2 1 2
1 2
( , ) 0,1,2,..., 0,1,2,...,! !
x xe
p x x x xx x
1 2 2 1 2
1 21 2 1 2 1
1 2 2
( , ) 0,1,2,..., 0,1,2,...,( )! !
y y ye
p y y y y yy y y
1 2 2 1 1 21 21 1
2 2
1 2 1 1 21 1 20 0
1 1 2 2 1
! ( )( ) ( , )
! ( )! ! !
y y y yy y
y y
y eep y p y y
y y y y y
CONTINUOUS CASE • Let X=(X1, X2, …, Xn) have a continuous joint
distribution for which its joint pdf is f, and consider
the joint pdf of new random variables Y1, Y2,…, Yk
defined as
*
X,,X,XgY
X,,X,XgY
X,,X,XgY
nkk
n
n
21
2122
2111
• If the transformation T is one-to-one and onto, then
there is no problem of determining the inverse
transformation, and we can invert the equation in (*)
and obtain new equations as
**
y,,y,ygx
y,,y,ygx
y,,y,ygx
nknn
k
k
211
211
22
211
11
Assuming that the partial derivatives exist at every point (y1, y2,…,yk=n).
ii y/g 1
• Under these assumptions, we have the following determinant J
called as the Jacobian of the transformation specified by (**). Then, the joint pdf of Y1, Y2,…,Yk can be obtained by using the change of variable technique of multiple variables.
n
nn
n
y
g
y
g
y
g
y
g
detJ1
1
1
11
1
11
• As a result, the new p.d.f. is defined as follows:
otherwise
yyyJgggfyyyg
nnXX
nn
,0
,,,for |,|,,,,,,
21
11
2
1
1,,
211
Example
Let X1 and X2 ~ Exp(1)
Consider the random variables Y1=X1, Y2=X1+X2. Find pdf of Y1 and Y2?
X1=Y1 and X2=Y2-Y1 the Jacobian is
Then
1 2
1 2
( )
, 1 2 1 2( , ) 0, 0x x
x xf x x e x x
1 01
1 1
2
1 2 1 2, 1 2 , 1 2 1 1 2( , ) ( , ) 0y
y y x xf y y f y y y e y y
METHOD OF CONDITIONING
• U=h(X1,X2)
• Find f(u|x2) by transformations (Fixing X2=x2)
• Obtain the joint density of U, X2:
• f(u,x2) = f(u|x2)f(x2)
• Obtain the marginal distribution of U by integrating joint density over X2
222 )()|()( dxxfxufufU
Example • X1~Beta(a2,b2 X2~Beta(a3,b1 Independent
• U=X1X2
• Fix X2=x2 and get f(u|x2)
10))ln(1(18
)ln(1818018)ln(181818
18)()|()(
1011831
)/1)(/(6)()|(),(
01
)/1)(/(6)|(
/1/
103)(10)1(6)(
221
2
2
22
1
2
21
222
2
2
2
2
2
22222
2
2
222
21
2121
2
2
221111
uuuuu
uuuuxuuxdxx
uudxxfxufuf
xux
uux
xxuxuxfxufxuf
xux
xuxuxuf
xdU
dXxUXxXU
xxxfxxxxf
uuuU
Example
• X1, X2 independent Exponential(q)
• f(xi)=q-1e-xi/q xi>0, q>0, i=1,2
• f(x1,x2)= q-2e-(x1+x2)/q x1,x2>0
• U=X1+X2
),2(~01
11)(
11
111
1
11)(
,
/
2
/
2
////
2
/)(
02
/
02
/)(/
0
20
//
021
//
0 0 2
21221
2121
22222
21221
2
qbaq
qqqq
qqq
q
qqqqq
qqqq
qqqq
GammaUuue
eu
eeufuee
dxedxedxee
dxeedxdxeeuUP
XuXuXuXXuU
xuXuXXuU
u
uuu
U
uu
xuxu
xu
xuxu
xuxx
uxx
u xu
M.G.F. Method
• If X1,X2,…,Xn are independent random variables with MGFs Mxi (t), then the MGF of is
n
1i
iXY )t(M)...t(M)t(MnX1XY
Example
Then find the pmf of
Let ~ ,independent
i iX Bin n p
1 2
1
1
1
...1
~ , .
( ) ( )... ( )
( ) ...( )
( )
k
i ki
Y X Xk
nnt t k
n nt k
X Bin n n n p
M t M t M t
pe q pe q
pe q
1
k
ii
Y X
1
1
...1
( ) ( )... ( )
( ) ...( )
( )
Y X Xk
nnt t k
n nt k
M t M t M t
pe q pe q
pe q
Example
ba
bbb
b
ba
aaa
a
,~
)1()1()1(
)()()(
,...,1)1()(
nt)(independe,...,1),(~
11
)...(
1
11
1
11
n
i
i
n
i
i
XX
tXtXXXttY
Y
n
i
i
X
ii
GammaXY
ttt
tMtMeeEeEeEtM
XY
nittM
niGammaX
n
i in
n
nn
i
i
Example
n
i
ii
n
i
ii
n
i
ii
n
i iin
i iinn
nn
nXX
XtaXtaXaXattY
Y
i
n
i
ii
iiX
iii
aaNormalXaY
tata
tata
tata
taMtaMeeEeEeEtM
aXaY
nit
ttM
niNormalX
n
nnnn
i
1
22
11
2
1
22
1
222
1
2
111
1
)...(
1
22
2
,~
2exp
2exp
2exp
)()()(
constants fixed }{
,...,12
exp)(
nt)(independe,...,1),(~
1
1111
ORDER STATISTICS
ORDER STATISTICS
• Let X1, X2,…,Xn be a r.s. of size n from a distribution of continuous type having pdf f(x), a<x<b. Let X(1) be the smallest of Xi, X(2) be the second smallest of Xi,…, and X(n) be the largest of Xi.
1 2 na X X X b
• X(i) is the i-th order statistic.
1 21
1 2
min , , ,
max , , ,
n
nn
X X X X
X X X X
ORDER STATISTICS
• It is often useful to consider ordered random sample.
• Example: suppose a r.s. of five light bulbs is tested and the failure times are observed as (5,11,4,100,17). These will actually be observed in the order of (4,5,11,17,100). Interest might be on the kth smallest ordered observation, e.g. stop the experiment after kth failure. We might also be interested in joint distributions of two or more order statistics or functions of them.
JOINT PDF OF THE OREDER STATISTICS
• If X1, X2,…,Xn is a r.s. of size n from a population with continuous pdf f(x), then the joint pdf of the order statistics X(1), X(2),…,X(n) is
1 2 1 2, , , !
n ng x x x n f x f x f x
The joint pdf of ordered sample is not same as the joint pdf of unordered sample.
)n()1( x...xfor
Order statistics are not independent.
Note: For discrete distributions, we need to take ties into account (two X’s being equal).
Find the joint pdf of the order statistics for the uniform distribution, the standard exponential distribution and normal distribution?
Solution: p.d.f for the uniform is:
( ) 1, 0 1f x x
1 1 1 1( , ,..., ) !, 0 ..., 1,n ng y y y n y y y
Example
Solution: p.d.f for the standard Exponential distribution is:
( ) , 0xf x e x
1
1 1 1 1( , ,..., ) ! , 0 ..., ,
n
i
i
y
n ng y y y n e y y y
Solution: p.d.f for the standard normal distribution is:
2
21
( ) ,2
x
f x e x
2
12
1 1 1 1
!( , ,..., ) , ..., ,
2
ni
i
y
n n
ng y y y e y y y
Example
• Suppose that X1,X2 and X3 represent a random sample of size 3 from population with pdf
• Joint pdf of order statistics Y1,Y2 and Y3?
• Marginal pdf of Y1
( ) 2 0 1f x x x
1 2 3 1 2 3 1 2 3 1 2 3( , , ) 3!(2 )(2 )(2 ) 48 0 1g y y y y y y y y y y y y
1 1
2 2
1 1 1 2 3 2 3 1 1 1
1 2
( ) 48 6 (1 ) 0 1y y
g y y y y dy dy y y y
THE MARGINAL DISTRIBUTIONS FOR THE ODER STATISTICS
Theorem: (p.d.f of the rth order statistics) If X1, X2,…,Xn be a r.s. of size n from a population with continuous pdf f(x), then the p.d.f. of the rth order statistics X(r) is given as:
1 1!( ) ( ) [ ( ) ] [1 ( ) ] ,
( 1)!( )!
r n
r r r r r r
ng y f y F y F y y
r n r
• r-th Order Statistic
y y1 y2 yr-1 yr yr+1 yn … …
P(X<yr) P(X>yr)
f(yr)
# of possible orderings n!/{(r1)!1!(n r)!}
1 1
!( ) ( )
( 1)!( )!
[ ( ) ] [1 ( ) ] ,
r r r
r n
r r r
ng y f y
r n r
F y F y y
Example
• X~Uniform(0,1). A r.s. of size n is taken. Find the p.d.f. of kth order statistic.
• Solution: Let Yk be the kth order statistic.
)1kn,k(Beta~Y
1y0for)y1(y)1kn()k(
)1n(
1)y1(y)!kn()!1k(
!n)y(g
k
kn1k
kn1kkY
Example
• Suppose that X1,X2,…,Xn represent a random sample of size n from population with pdf
• Marginal pdf of Y1 and Yn?
( ) 2 0 1f x x x
2( ) 0 1F x x x
2 1
1 1 1 1 1
2 1
1
( ) 2 (1 ) 0 1
( ) 2 ( ) 0 1
n
n
n n n n
g y ny y y
g y ny y y
Theorem (p.d.f of the largest order statistics): If X1, X2,…,Xn be a r.s. of size n from a population with continuous pdf f(x), then the p.d.f. of the Largest order statistics Y(n) is given as:
1( ) ( )[ ( )] ,n
n n n n ng y n f y F y y
Theorem: (p.d.f of the smallest order statistics) If X1,
X2,…,Xn be a r.s. of size n from a population with continuous pdf f(x), then the p.d.f. of the smallest order statistics X(1) is given as:
1
1 1 1 1( ) ( )[1 ( )] ,n
ng y n f y F y y
Example
Let are an O. S. of sample size n = 6 and the p.d.f. of this sample is
Find:
Solution:
1( ) , 0 2
2f x x
1 2 6...Y Y Y
1 1 6 6( ) , ( ) , ( )r rg y g y g y
1 6 1
6
6!( ) [2 ] , 0 2
( 1)!(6 )!2
r
r r r r rg y y y yr r
5
1 1 1 16
6( ) [ 2 ] , 0 2
2g y y y
5
6 6 6 66
6( ) , 0 2
2g y y y
JOINT P.D.F. OF i-TH AND j-TH ORDER
STATISTIC (FOR i<j)
Theorem:
If X1, X2,…,Xn be a r.s. of size n from a population with continuous pdf f(x), and Y1< Y2<…<Yn are the order statistics of that sample, then the p.d.f. of the two order statistics Yi< Yj , i<j and i,j = 1,2, …,n is given as
1 1!
( , ) [ ( )] ( )[ ( ) ( )] ( )[1 ( )]( 1)!( 1)!( )!
i j i n j
ij i j i i j i j j
ng y y F y f y F y F y f y F y
i j i n j
• Joint p.d.f. of i-th and j-th Order Statistic (for i<j)
y y1 y2 yi-1 yi yi+1 yn … …
P(X<yi) P(yi<X<yj)
f(yi)
# of possible orderings n!/{(i1)!1!(j-i-1)!1!(n j)!}
yj-1 yj yj+1
i-1 items j-i-1 items n-j items 1 item 1 item
P(X>yj)
f(yj)
1 1!( , ) [ ( )] ( )[ ( ) ( )] ( )[1 ( )]
( 1)!( 1)!( )!
i j i n j
ij i j i i j i j j
ng y y F y f y F y F y f y F y
i j i n j
Example
• Suppose that X1,X2,…,Xn represent a random sample of size n from population with pdf
• Find the density of range R=Yn-Y1?
( ) 2 0 1f x x x
2( ) 0 1F x x x
2 2 2
1, 1 1 1 1
!( , ) 2 2 ( ) 0 1
( 2)!
n
n n n n
ng y y y y y y y
n