functions of random variables. method of distribution functions x 1,…,x n ~ f(x 1,…,x n ) u=g(x...
TRANSCRIPT
Functions of Random Variables
Method of Distribution Functions
• X1,…,Xn ~ f(x1,…,xn)
• U=g(X1,…,Xn) – Want to obtain fU(u)
• Find values in (x1,…,xn) space where U=u
• Find region where U≤u
• Obtain FU(u)=P(U≤u) by integrating f(x1,…,xn) over the region where U≤u
• fU(u) = dFU(u)/du
Example – Uniform X• Stores located on a linear city with density
f(x)=0.05 -10 ≤ x ≤ 10, 0 otherwise• Courier incurs a cost of U=16X2 when she delivers to a
store located at X (her office is located at 0)
1600080
)()(
160004044
05.005.0)()(
44
416
2/1
4
4
2
uu
du
udFuf
uuuu
dxuUPuF
uX
uuU
uXuXuU
UU
u
uU
Example – Sum of Exponentials
• X1, X2 independent Exponential()
• f(xi)=-1e-xi/xi>0, >0, i=1,2
• f(x1,x2)= -2e-(x1+x2)/ x1,x2>0
• U=X1+X2
),2(~01
11)(
11
111
1
11)(
,
/2
/2
////
2/)(
02/
02/)(/
0
20
//
021//
0 0 2
21221
2121
22222
21221
2
GammaUuue
eu
eeufuee
dxedxedxee
dxeedxdxeeuUP
XuXuXuXXuU
xuXuXXuU
u
uuuU
uu
xuxuxuxuxu
xuxxuxxu xu
Method of Transformations• X~fX(x)
• U=h(X) is either increasing or decreasing in X
• fU(u) = fX(x)|dx/du| where x=h-1(u)
• Can be extended to functions of more than one random variable:
• U1=h1(X1,X2), U2=h2(X1,X2), X1=h1-1(U1,U2), X2=h2
-1(U1,U2)
22112121
1
2
2
1
2
2
1
1
2
2
1
2
2
1
1
1
),()(||),(),(
||
1duuufufJxxfuuf
dU
dX
dU
dX
dU
dX
dU
dX
dU
dX
dU
dXdU
dX
dU
dX
J
U
Example
• fX(x) = 2x 0≤ x ≤ 1, 0 otherwise
• U=10+500X (increasing in x)
• x=(u-10)/500
• fX(x) = 2x = 2(u-10)/500 = (u-10)/250
• dx/du = d((u-10)/500)/du = 1/500
• fU(u) = [(u-10)/250]|1/500| = (u-10)/125000 10 ≤ u ≤ 510, 0 otherwise
Method of Conditioning
• U=h(X1,X2)
• Find f(u|x2) by transformations (Fixing X2=x2)
• Obtain the joint density of U, X2:
• f(u,x2) = f(u|x2)f(x2)
• Obtain the marginal distribution of U by integrating joint density over X2
222 )()|()( dxxfxufufU
Example (Problem 6.11)• X1~Beta( X2~Beta(Independent
• U=X1X2
• Fix X2=x2 and get f(u|x2)
10))ln(1(18
)ln(1818018)ln(181818
18)()|()(
1011831
)/1)(/(6)()|(),(
01
)/1)(/(6)|(
/1/
103)(10)1(6)(
221
22
22
1
2
21
222
22
22
222222
22
222
21
2121
22221111
uuuuu
uuuuxuuxdxx
uudxxfxufuf
xux
uux
xxuxuxfxufxuf
xux
xuxuxuf
xdU
dXxUXxXU
xxxfxxxxf
uuuU
Problem 6.11
0
1
2
3
4
5
6
7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
u
De
ns
ity
of
U=
X1
X2
f(u)
f(u|x2=.25)
f(u|x2=.5)
f(u|x2=.75)
Method of Moment-Generating Functions
• X,Y are two random variables
• CDF’s: FX(x) and FY(y)
• MGF’s: MX(t) and MY(t) exist and equal for |t|<h,h>0
• Then the CDF’s FX(x) and FY(y) are equal
• Three Properties:– Y=aX+b MY(t)=E(etY)=E(et(aX+b))=ebtE(e(at)X)=ebtMX(at)
– X,Y independent MX+Y(t)=MX(t)MY(t)
– MX1,X2(t1,t2) = E[et1X1+t2X2] =MX1(t1)MX2(t2) if X1,X2 are indep.
Sum of Independent Gammas
,~
)1()1()1(
)()()(
,...,1)1()(
nt)(independe,...,1),(~
11
)...(
1
11
1
11
n
ii
n
ii
XXtXtXXXttY
Y
n
ii
X
ii
GammaXY
ttt
tMtMeeEeEeEtM
XY
nittM
niGammaX
n
i in
n
nn
i
i
Linear Function of Independent Normals
n
iii
n
iii
n
iii
n
i iin
i iinn
nn
nXXXtaXtaXaXattY
Y
i
n
iii
iiX
iii
aaNormalXaY
tata
tata
tata
taMtaMeeEeEeEtM
aXaY
nit
ttM
niNormalX
n
nnnn
i
1
22
11
2
1
22
1
2221
21
11
1)...(
1
22
2
,~
2exp
2exp
2exp
)()()(
constants fixed }{
,...,12
exp)(
nt)(independe,...,1),(~
1
1111
Distribution of Z2 (Z~N(0,1))
2
1
21
0
/1
21
2
2/12/1
2/1
0
21
2/
12/1
0
21
2/
2/12
21
0
2/12/12
2
21
0
2
212/
2/
)2,2/(~t independenmutually ,...,
)2/1(
)(
:Notes
)2,2/1(~
)21()21(22
1
21
2)2/1(
2
1
2
1
2
1
2
12
5.05.02
1Let
0)about (symmetric 2
12
2
1
2
1)(
2
1)()1,0(~
2
2222
2
2
n
n
iin
y
tu
tu
tz
tz
tz
ztz
Z
zZ
nGammaZZZ
dyey
GammaZ
tt
tdueudueudze
duudzuudu
dzuzzu
dzedzedzeetM
zezfNZ
Distributions of and S2 (Normal data) X
data sampled theof sdifference theoffunction a is So
)1(2
)1(2)0)(0(2)1()1(
)1(2
1
2)1(2
1
2)1(2
1
2)1(2
1
)1(2
1)(
)1(2
1
: oftion representa eAlternativ
1 :Variance Sample
0 :Note
,...,111
:Mean Sample
dDistributetly Independen and Normal ),(~,...,
2
22
22
1 11
2
1
2
1 11
2
1
2
1 1
22
1 1
2
1 1
22
2
1
2
2
1111
11
1
21
S
Snn
SnnSnnSnn
nn
XXXXXXnXXnnn
XXXXXXnXXnnn
XXXXXXXXnn
XXXXnn
XXnn
S
S
n
XXS
XXXnXXX
nin
aXaXnn
XX
NIDNIDXX
n
i
n
jji
n
jj
n
ii
n
i
n
jji
n
jj
n
ii
n
i
n
jjiji
n
i
n
jji
n
i
n
jji
n
i i
n
ii
n
ii
n
ii
n
ii
i
n
iii
n
ii
n
i i
n
Independence of and S2 (Normal Data)X
))(exp())(exp(
))(exp())(exp(
)]()(exp[
)()(exp)(),(
}exp{2
20exp)()2,0(~
}2exp{2
22exp)()2,2(~
),(~,
212211
212211
212211
12221121,
2222
212
2222
221
221
21
ttXEttXE
ttXttXE
ttXttXE
XXtXXtEeEttM
tt
tMNXXD
ttt
ttMNXXT
NIDXX
ind
DtTtDT
D
T
Independence of T=X1+X2 and D=X2-X1 for Case of n=2
X
Independence of and S2 (Normal Data) P2X
)()(2
2exp
2
22exp
2
2
2
22exp
2
)22()(exp
2
)()(exp
2
)()(exp
))(exp())(exp(
))(exp())(exp(
21
22
221
2
1
22
221
2
1
2122
2121
22
21
2
2121
221
2
21
221
2
21
212211
212211
tMtMtt
t
ttt
tttttttttttt
tttt
tttt
ttXEttXE
ttXttXE
DT
ind
Independence of T=X1+X2 and D=X2-X1 for Case of n=2
Thus T=X1+X2 and D=X2-X1 are independent Normals and & S2 are independentX
Distribution of S2 (P.1)
2/
1)()1()()1(
2
2
2
2
22
1
2
2
1
2
1
2
2
1
22
2
2
12
2
12
2
1
22
1
21
22
)21()()()(
:tindependen are and Now,
)1(0
1
21
21
11
)2,2/(~
~)1,0(~),(~
2
122
2
2
2
2
22n
XXnSnXnSn
n
ii
n
ii
n
ii
n
iii
n
ii
n
ii
n
i
i
n
n
i
i
ii
ii
tMtMtMtM
SX
XnSnXnXX
XXXXnXX
XXXXXX
XXXXX
nGammaX
ZNX
ZNIDX
n
ii
Distribution of S2 P.2
212
2
2/)1()2/1()2/(2/1
2/
)(
1
)1(
2/1
)(
212
2
2
1
22
1
2
2
2
2/
1)()1()()1(
2
2,2
1~
)1(
)21()21()21(
)21()(
)21()(~
)1,0(~1
,1
~
: :consider Now,
)21()()()(
:tindependen are and Now,
2
2
2
12
2
2
2
2
2
122
2
2
2
2
22
n
nnn
Xn
X
Sn
Xn
X
XX
n
i
n
iXX
n
XXnSnXnSn
nGamma
Sn
ttt
t
M
M
tM
ttMXn
NXn
n
XXZ
nnnNX
Xn
tMtMtMtM
SX
n
ii
n
ii
Summary of Results• X1,…Xn ≡ random sample from N(2)population
• In practice, we observe the sample mean and sample variance (not the population values: , 2)
• We use the sample values (and their distributions) to make inferences about the population values
ons)distributi-F and for t,on presentati
.ppton ngconditioni of method using derivation (See
~)1(
)1()1(
/
/
tindependen are ,
~)1(
1,~
121
2
2
2
212
1
2
2
21
2
22
1
n
n
n
n
ii
n
ii
n
ii
tn
Z
nSn
n
X
nS
Xt
SX
XXSn
n
XXS
nNX
n
XX
Order Statistics• X1,X2,...,Xn Independent Continuous RV’s
• F(x)=P(X≤x) Cumulative Distribution Function• f(x)=dF(x)/dx Probability Density Function
• Order Statistics: X(1) ≤ X(2) ≤ ...≤ X(n) (Continuous can ignore equalities)• X(1) = min(X1,...,Xn)
• X(n) = max(X1,...,Xn)
Order Statistics
)()](1[)](1[
)](1[
])](1[1[)()(
:Minimum of pdf
)](1[1)(1),...,(1
: Minimum of CDF
)()]([)(
)]([)]([)(
)(
:Maximum of pdf
)]([)(),...,(
: Maximum of CDF
11
)1(1
11)1(
)1(
11)(
11)(
)(
xfxFndx
xFdxFn
dx
xFd
dx
xXdPxg
xFxXPxXPxXxXPxXP
X
xfxFndx
xdFxFn
dx
xFd
dx
xXdPxg
xFxXPxXPxXxXPxXP
X
nn
n
nnn
nnn
nn
nnnn
n
Example
• X1,...,X5 ~ iid U(0,1)
(iid=independent and identically distributed)
o.w.0
10)1(5)1)(1(5)( :Minimum
o.w.0
105)1(5)( :Maximum
o.w.0
101)(
11
10
00
)(
44
1
44
xxxxg
xxxxg
xxf
x
xx
x
xF
n
Order Stats - U(0,1) - n=5
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x
f(x)
gn(x)
g1(x)
Distributions of Order Statistics• Consider case with n=4
• X(1) ≤x can be one of the following cases:
• Exactly one less than x• Exactly two are less than x• Exactly three are less than x• All four are less than x
• X(3) ≤x can be one of the following cases:
• Exactly three are less than x• All four are less than x
• Modeled as Binomial, n trials, p=F(x)
Case with n=4
))(1()()(12)()(12)()(12)(
)(3)(4
)()(4)(4
)](1[)]([4
4)](1[)]([
3
4
)](1[1
)](1[)]([4
4)](1[)]([
3
4
)](1[)]([2
4)](1[)]([
1
4
2323
43
443
043)3(
4
043
2231)1(
xFxFxfxfxFxfxFxg
xFxF
xFxFxF
xFxFxFxFxXP
xF
xFxFxFxF
xFxFxFxFxXP
General Case (Sample of size n)
elsewhere0
...)()...(!),...,(
:statisticsorder all ofon distributiJoint
)()()](1[)]()([)]([)!()!1()!1(
!
),(:1
l)multinomia (uses statsorder and ofon distributiJoint
1)()](1[)]([)!()!1(
!)(
111,...,12
11
1
nnnn
jijn
jij
iji
i
jiij
thth
jnjj
xxxfxfnxxg
xfxfxFxFxFxFjniji
n
xxgnji
ji
njxfxFxFjnj
nxg
Example – n=5 – Uniform(0,1)
10)(20),(:5,1
5)1(]1[][!0!4
!5)(:5
)1(20)1(]1[][!1!3
!5)(:4
)1(30)1(]1[][!2!2
!5)(:3
)1(20)1(]1[][!3!1
!5)(:2
)1(5)1(]1[][!4!0
!5)(:1
10)(1)(
513
155115
455155
345144
2235133
325122
415111
xxxxxxgji
xxxxgj
xxxxxgj
xxxxxgj
xxxxxgj
xxxxgj
xxxFxf
Distributions of all Order Stats - n=5 - U(0,1)
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x
f(x)
g1(x)
g2(x)
g3(x)
g4(x)
g5(x)