order statistics and limiting distributions
DESCRIPTION
ORDER STATISTICS AND LIMITING DISTRIBUTIONS. ORDER STATISTICS. Let X 1 , X 2 ,…,X n be a r.s. of size n from a distribution of continuous type having pdf f(x), aTRANSCRIPT
![Page 1: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/1.jpg)
1
ORDER STATISTICS ANDLIMITING DISTRIBUTIONS
![Page 2: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/2.jpg)
2
ORDER STATISTICS
• Let X1, X2,…,Xn be a r.s. of size n from a distribution of continuous type having pdf f(x), a<x<b. Let X(1) be the smallest of Xi, X(2) be the second smallest of Xi,…, and X(n) be the largest of Xi.
1 2 na X X X b
• X(i) is the i-th order statistic.
1 21
1 2
min , , ,
max , , ,
n
nn
X X X X
X X X X
![Page 3: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/3.jpg)
ORDER STATISTICS• It is often useful to consider ordered random
sample. • Example: suppose a r.s. of five light bulbs is tested
and the failure times are observed as (5,11,4,100,17). These will actually be observed in the order of (4,5,11,17,100). Interest might be on the kth smallest ordered observation, e.g. stop the experiment after kth failure. We might also be interested in joint distributions of two or more order statistics or functions of them (e.g. range=max – min)
3
![Page 4: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/4.jpg)
4
ORDER STATISTICS• If X1, X2,…,Xn is a r.s. of size n from a population
with continuous pdf f(x), then the joint pdf of the order statistics X(1), X(2),…,X(n) is
1 2 1 2, , , !
n ng x x x n f x f x f x
The joint pdf of ordered sample is not same as the joint pdf of unordered sample.
)n()1( x...xfor
Order statistics are not independent.
Future reference: For discrete distributions, we need to take ties into account (two X’s being equal). See, Casella and Berger, 1990, pg 231.
![Page 5: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/5.jpg)
Example
• Suppose that X1, X2, X3 is a r.s. from a population with pdf
f(x)=2x for 0<x<1
Find the joint pdf of order statistics and the marginal pdf of the smallest order statistic.
5
![Page 6: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/6.jpg)
6
ORDER STATISTICS
• The Maximum Order Statistic: X(n)
nX nG y P X y
n nX Xg y G yy
![Page 7: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/7.jpg)
7
ORDER STATISTICS
• The Minimum Order Statistic: X(1)
1 1XG y P X y
1 1X Xg y G yy
![Page 8: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/8.jpg)
8
ORDER STATISTICS
• k-th Order Statistic
yy1 y2 yk-1 yk yk+1 yn… …
P(X<yk) P(X>yk)
fX(yk)
# of possible orderingsn!/{(k1)!1!(n k)!}
1!1 ,
1 ! !k
k n k
X X XX y
ng F y f y F y a y b
k n k
![Page 9: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/9.jpg)
Example
• Same example but now using the previous formulas (without taking the integrals): Suppose that X1, X2, X3 is a r.s. from a population with pdf
f(x)=2x for 0<x<1
Find the marginal pdf of the smallest order statistic.
9
![Page 10: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/10.jpg)
Example
• X~Uniform(0,1). A r.s. of size n is taken. Find the p.d.f. of kth order statistic.
• Solution: Let Yk be the kth order statistic.
10
)1kn,k(Beta~Y
1y0for)y1(y)1kn()k(
)1n(
1)y1(y)!kn()!1k(
!n)y(g
k
kn1k
kn1kkY
![Page 11: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/11.jpg)
11
ORDER STATISTICS• Joint p.d.f. of k-th and j-th Order Statistic (for k<j)
yy1 y2 yk-1 yk yk+1 yn… …
P(X<yk) P(yk<X<yj)
fX(yk)
# of possible orderingsn!/{(k1)!1!(j-k-1)!1!(n j)!}
yj-1 yj yj+1
k-1 items j-k-1 items n-j items1 item 1 item
P(X>yj)
fX(yj)
jnjj
1kjkjk
1kkjk)j(X,)k(X )]y(F1)[y(f)]y(F)y(F)[y(f)]y(F[
)!jn()!1kj()!1k(
!n)y,y(g
![Page 12: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/12.jpg)
LIMITING DISTRIBUTION
12
![Page 13: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/13.jpg)
Motivation
• The p.d.f. of a r.v. often depends on the sample size (i.e., n)
• If X1, X2,…, Xn is a sequence of rvs and Yn=u(X1, X2,…, Xn) is a function of them, sometimes it is possible to find the exact distribution of Yn (what we have been doing lately)
• However, sometimes it is only possible to obtain approximate results when n is large limiting distributions.
13
![Page 14: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/14.jpg)
14
CONVERGENCE IN DISTRIBUTION• Consider that X1, X2,…, Xn is a sequence of
rvs and Yn=u(X1, X2,…, Xn) be a function of rvs with cdfs Fn(y) so that for each n=1, 2,…
,n nF y P Y y
lim for all nn
F y F y y
where F(y) is continuous. Then, the sequence Yn is said to converge in distribution to Y.
d
nY Y
![Page 15: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/15.jpg)
15
CONVERGENCE IN DISTRIBUTION
• Theorem: If for every point y at which F(y) is continuous, then Yn is said to have a limiting distribution with cdf F(y). The term “Asymptotic distribution” is sometimes used instead of “limiting distribution”
• Definition of convergence in distribution requires only that limiting function agrees with cdf at its points of continuity.
limnn
F y F y
![Page 16: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/16.jpg)
16
Example
Let X1,…, Xn be a random sample from Unif(0,1).
Find the limiting distribution of the max order statistic, if it exists.
![Page 17: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/17.jpg)
17
Example
Let {Xn} be a sequence of rvs with pmf
1
1, if 2
0, o.w.n
xf x P X x n
Find the limiting distribution of Xn , if it exists.
![Page 18: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/18.jpg)
Example
Let Yn be the nth order statistic of a random sample X1, …, Xn from Unif(0,θ).
18
Find the limiting distribution of Zn=n(θ -Yn), if it exists.
![Page 19: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/19.jpg)
19
Example
Let X1,…, Xn be a random sample from Exp(θ).
Find the limiting distribution of the minorder statistic, if it exists.
![Page 20: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/20.jpg)
Example
• Suppose that X1, X2, …, Xn are iid from Exp(1).
20
Find the limiting distribution of the maxorder statistic, if it exists.
![Page 21: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/21.jpg)
21
LIMITING MOMENT GENERATING FUNCTIONS
• Let rv Yn have an mgf Mn(t) that exists for all n. If
lim ,nnM t M t
then Yn has a limiting distribution which is defined by M(t).
![Page 22: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/22.jpg)
22
Example
~ , with mgf 1nt
XX Bin n p M t pe p
Let =np.
1
lim lim 1
1lim 1
nt
Xn n
ntte
Yn
M t pe p
ee M t
n
The mgf of Poisson()
The limiting distribution of Binomial rv is the Poisson distribution.
![Page 23: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/23.jpg)
23
CONVERGENCE IN PROBABILITY (STOCHASTIC CONVERGENCE)
• A rv Yn convergence in probability to a rv Y if
lim 1nnP Y Y
for every >0.
![Page 24: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/24.jpg)
24
CHEBYSHEV’S INEQUALITY
• Let X be an rv with E(X)= and V(X)=2.
2
21 , 0P X
• The Chebyshev’s Inequality can be used to prove stochastic convergence in many cases.
![Page 25: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/25.jpg)
25
CONVERGENCE IN PROBABILITY (STOCHASTIC CONVERGENCE)
• The Chebyshev’s Inequality proves the convergence in probability if the following three conditions are satisfied.
22. for all .n nV Y n
1. E(Yn)=n where lim .nn
23. lim 0.nn
![Page 26: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/26.jpg)
26
Example
Let X be an rv with E(X)= and V(X)=2<. For a r.s. of size n, is the sample mean. Is
nX
?p
nX
![Page 27: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/27.jpg)
27
WEAK LAW OF LARGE NUMBERS (WLLN)
• Let X1, X2,…,Xn be iid rvs with E(Xi)= and V(Xi)=2<. Define . Then, for every >0,
1
1/n
n ii
X n X
converges in probability to . nX
WLLN states that the sample mean is a good estimate of the population mean. For large n, the sample mean and population mean are close to each other with probability 1. But more to come on the properties of estimators.
![Page 28: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/28.jpg)
28
STRONG LAW OF LARGE NUMBERS
• Let X1, X2,…,Xn be iid rvs with E(Xi)= and V(Xi)=2<. Define . Then, for every >0,
1
1/n
n ii
X n X
lim 1nnP X
that is, converges almost surely to . nX
![Page 29: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/29.jpg)
Relation between convergences
Almost sure convergence is the strongest.
(reverse is generally not true)
29
YYYYYY dn
pn
s.an
![Page 30: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/30.jpg)
30
THE CENTRAL LIMIT THEOREM• Let X1, X2,…,Xn be a sequence of iid rvs
with E(Xi)= and V(Xi)=2<∞. Define
. Then,1
1/n
n ii
X n X
0,1
dnn X
Z N
or
1 0,1 .
n
i di
X nZ N
n
Proof can be found in many books, e.g. Bain and Engelhardt, 1992, page 239
![Page 31: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/31.jpg)
Example
• Let X1, X2,…,Xn be iid rvs from Unif(0,1) and
• Find approximate distribution of Yn.
• Find approximate values of–
– The 90th percentile of Yn.
31
n
1iin XY
40
1
)12(i
iXP
![Page 32: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/32.jpg)
32
SLUTKY’S THEOREM
• If XnX in distribution and Yna, a constant, in probability, then
a) YnXnaX in distribution.
b) Xn+YnX+a in distribution.
![Page 33: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/33.jpg)
33
SOME THEOREMS ON LIMITING DISTRIBUTIONS
• If Xn c>0 in probability, then for any function g(x) continuous at c, g(Xn) g(c) in prob. e.g. .
p
nX c• If Xnc in probability and Ynd in probability, then
• aXn+bYn ac+bd in probability.
• XnYn cd in probability
• 1/Xn 1/c in probability for all c0.
![Page 34: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/34.jpg)
34
Example
1. X~Gamma(, 1). Show that
0,1.d
n
n
n XN
X
![Page 35: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/35.jpg)
35
Example
2. Let X1,…,Xn be r.v.s. each with Gamma(n+1,β). Show that Xn/n converges in probability to a constant c, and find c.
![Page 36: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/36.jpg)
Example
3. Let Xn be “displaced exponential” with p.d.f.
Find the limiting distribution.
36
..0
/)(
)/(
wo
nxifexf
nx
n
![Page 37: ORDER STATISTICS AND LIMITING DISTRIBUTIONS](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bd9550346895da5026a/html5/thumbnails/37.jpg)
Example
4. Let the sample space S={1,2,3,4}. Define the following r.v.s:
Xn(1)=Xn(2)=1,Xn(3)=Xn(4)=0 for n=1,2,… and
X(1)=X(2)=0, X(3)=X(4)=1
a) Does Xn converge in probability to X as n→∞?
b) Does Xn converge in distribution to X as n→∞?
37