binomial random variable approximations, conditional probability density functions and stirling’s...
TRANSCRIPT
Binomial Random Variable Approximations,Conditional Probability Density Functionsand Stirling’s Formula
Let X represent a Binomial r.v ,Then from
for large n. In this context, two approximations are extremely useful.
2
1
2
1
.)(21
k
kk
knkk
kkn qp
k
nkPkXkP
.)()(
) " and between is of sOccurrence ("
2
1
2
1
211 1
21
k
kk
knkk
kkkkkk qp
k
nXPXXXP
kkAP
The Normal Approximation (Demoivre-Laplace Theorem)
Suppose with p held fixed. Then for k in the neighborhood of np, we can approximate
And we have:
where
n npq
.2
1 2/)( 2 npqnpkknk enpq
qpk
n
,2
1
2
1 2/
2/)(
21
22
1
22
1
dyedxenpq
kXkP yx
x
npqnpxk
k
. , 22
11
npq
npkx
npq
npkx
As we know,
If and are within with approximation:
where
We can express this formula in terms of the normalized integral
that has been tabulated extensively.
2
1
2
1
)(21
k
kk
knkk
kkn qp
k
nkPkXkP
1k 2k , , npqnpnpqnp
,2
1
2
1 2/
2/)(
21
22
1
22
1
dyedxenpq
kXkP yx
x
npqnpxk
k
)(2
1)(
0
2/2
xerfdyexerfx y
. , 22
11
npq
npkx
npq
npkx
x erf(x) x erf(x) x erf(x) x erf(x)
0.05
0.10 0.15 0.20 0.25
0.30 0.35 0.40 0.45 0.50
0.55 0.60 0.65 0.70 0.75
0.01994 0.03983 0.05962 0.07926 0.09871 0.11791 0.13683 0.15542 0.17364 0.19146 0.20884 0.22575 0.24215 0.25804 0.27337
0.80 0.85 0.90 0.95 1.00
1.05 1.10 1.15 1.20 1.25
1.30 1.35 1.40 1.45 1.50
0.28814 0.30234 0.31594 0.32894 0.34134 0.35314 0.36433 0.37493 0.38493 0.39435 0.40320 0.41149 0.41924 0.42647 0.43319
1.55
1.60 1.65 1.70 1.75
1.80 1.85 1.90 1.95 2.00
2.05 2.10 2.15 2.20 2.25
0.43943 0.44520 0.45053 0.45543 0.45994 0.46407 0.46784 0.47128 0.47441 0.47726 0.47982 0.48214 0.48422 0.48610 0.48778
2.30
2.35 2.40 2.45 2.50
2.55 2.60 2.65 2.70 2.75
2.80 2.85 2.90 2.95 3.00
0.48928 0.49061 0.49180 0.49286 0.49379 0.49461 0.49534 0.49597 0.49653 0.49702 0.49744 0.49781 0.49813 0.49841 0.49865
A fair coin is tossed 5,000 times.
Find the probability that the number of heads is between 2,475 to 2,525.
We need
Since n is large we can use the normal approximation.
so that and
and
So the approximation is valid for and
Example
Solution
).525,2475,2( XP
,2
1p 500,2np .35npq
,465,2 npqnp
475,21 k .525,22 k
,535,2 npqnp
Here,
Using the table,
Example - continued
2
1
2
.2
1 2/21
x
x
y dyekXkP
.7
5 ,
7
5 22
11
npq
npkx
npq
npkx
516.07
5erf2
|)(|erf)(erf
)(erf)(erf525,2475,2
12
12
xx
xxXP
. 258.0)7.0(erf
The Poisson Approximation
For large n, the Gaussian approximation of a binomial r.v is valid only
if p is fixed, i.e., only if and
What if is small, or if it does not increase with n?
- for example, as such that is a fixed number.
1np .1npq
np
0p ,n np
The Poisson Approximation
Consider random arrivals such as telephone calls over a line.
n: total number of calls in the interval
as we have
Suppose
Δ : a small interval of duration
).,0( T
T n
.Tn
0 T
1 2 n
The Poisson Approximation
p: probability of a single call (during 0 to T) occurring in Δ:
as
Normal approximation is invalid here.
Suppose the interval Δ in the figure:
- (H) “success” : A call inside Δ,
- (T ) “failure” : A call outside Δ
: probability of obtaining k calls (in any order) in an interval of duration Δ ,
.T
p
0p .T
T
Tnp
)(kPn
knkn pp
kkn
nkP
)1(
!)!(
!)(
0 T
1 2 n
The Poisson Approximation
Thus,
.)/1(
)/1(
!
11
21
11
)/1(!
)()1()1()(
k
nk
knk
kn
n
n
kn
k
nn
nnpk
np
n
knnnkP
e
kkP
k
nnppn !
)(lim ,0 ,
the Poisson p.m.f
Suppose
- two million lottery tickets are issued
- with 100 winning tickets among them.
a) If a person purchases 100 tickets, what is the probability of winning?
Example: Winning a Lottery
Solution
.105102
100
ticketsof no. Total
tickets winningof No. 56
pThe probability of buying a winning ticket
X: number of winning tickets
n: number of purchased tickets ,
P: an approximate Poisson distribution with parameter
So, The Probability of winning is:
Winning a Lottery - continued
.005.0105100 5 np
,!
)(k
ekXPk
.005.01)0(1)1( eXPXP
b) How many tickets should one buy to be 95% confident of having a winning ticket?
we need
But or
Thus one needs to buy about 60,000 tickets to be 95% confident of having a winning ticket!
Winning a Lottery - continued
Solution
.95.0)1( XP
.320ln implies 95.01)1( eXP
3105 5 nnp .000,60n
A space craft has 100,000 components
The probability of any one component being defective is
The mission will be in danger if five or more components become defective.
Find the probability of such an event.
n is large and p is small
Poisson Approximation with parameter
Example: Danger in Space Mission
Solution
n
).0(102 5 p
2102000,100 5 np
.052.03
2
3
42211
!1
!1)4(1)5(
2
4
0
24
0
e
ke
keXPXP
k
kk
k
Conditional Probability Density Function
, )( )( xXPxFX
.0)( ,)(
)()|(
BP
BP
BAPBAP
.
)(
)( |)( )|(
BP
BxXPBxXPBxFX
.0
)(
)(
)(
)( )|(
,1)(
)(
)(
)( )|(
BP
P
BP
BXPBF
BP
BP
BP
BXPBF
X
X
Conditional Probability Density Function
Further,
Since for
),|()|(
)(
)( )|)((
12
2121
BxFBxF
BP
BxXxPBxXxP
XX
,12 xx
. )()()( 2112 xXxxXxX
,
)|()|(
dx
BxdFBxf X
X
2
1
21 .)|(|)(x
x X dxBxfBxXxP
x
XX duBufBxF
.)|()|(
Toss a coin and X(T)=0, X(H)=1.
Suppose
Determine
has the following form.
We need for all x.
For so that
and
Example
Solution
}.{HB ).|( BxFX
)(xFX
)|( BxFX
,)( ,0 xXx ,)( BxX
.0)|( BxFX
)(xFX
x
(a)
q1
1
( | )XF x B
x
(b)
1
1
For so that
For and
Example - continued
, )( ,10 TxXx
HTBxX )( .0)|( and BxFX
,)( ,1 xXx
}{ )( BBBxX 1)(
)()|( and
BP
BPBxFX
( | )XF x B
x
1
1
Given suppose Find
We will first determine
For so that
For so that
Example
Solution
),(xFX .)( aXB ).|( Bxf X
).|( BxFX
.
)|(
aXP
aXxXPBxFX
xXaXxXax ,
.
)(
)()|(
aF
xF
aXP
xXPBxF
X
XX
)( , aXaXxXax .1)|( BxFX
Thus
and hence
Example - continued
, ,1
, ,)(
)()|(
ax
axaF
xFBxF
X
X
X
otherwise. ,0
,,)(
)()|()|(
axaF
xfBxF
dx
dBxf
X
X
XX
)|( BxFX
)(xFX
xa
1
(a)
)|( Bxf X
)(xf X
xa(b)
Let B represent the event with
For a given determine and
Example
Solution
.)()(
)( )(
)(
)()(
|)( )|(
aFbF
bXaxXP
bXaP
bXaxXP
BxXPBxF
XX
X
bXa )( .ab ),(xFX )|( BxFX ).|( Bxf X
For we have
and hence
For we have
and hence
For we have
so that
Thus,
Example - continued
,bxa })({ )( )( xXabXaxX
.
)()(
)()(
)()(
)()|(
aFbF
aFxF
aFbF
xXaPBxF
XX
XX
XXX
,bx bXabXaxX )( )( )( .1)|( BxFX
,ax ,)( )( bXaxX
.0)|( BxFX
otherwise.,0
,,)()(
)()|(
bxaaFbF
xfBxf
XX
X
X
)|( Bxf X
)(xf X
xa b
Conditional p.d.f & Bayes’ Theorem
First, we extend the conditional probability results to random variables:
We know that If is a partition of S and B is an arbitrary event, then:
By setting we obtain:
)()|()()|()( 11 nn APABPAPABPBP
)],,,[ 21 nAAAU
Son partition a form ,
)()|()()|()(
)()|()()|()(
Hence
)()|()()|()(
1
11
11
11
n
nn
nn
nn
AA
APAxfAPAxfxf
APAxFAPAxFxF
APAxXPAPAxXPxXP
}{ xXB
Conditional p.d.f & Bayes’ Theorem
Using:
We obtain:
For ,
.)(
)()|()|(
BP
APABPBAP
)(
| )(
|)| AP
xF
AxFAP
xXP
AxXPxXAP
21 )( xXxB
).()(
)|()(
)()(
)|()|(
)(|
|
2
1
2
1
12
12
21
2121
APdxxf
dxAxfAP
xFxF
AxFAxF
APxXxP
AxXxPxXxAP
x
x X
x
x X
XX
XX
Conditional p.d.f & Bayes’ Theorem
Let so that in the limit as
or
we also get
or
,0 , , 21 xxxx ,0
).()(
)|(|)(|lim
0AP
xf
AxfxXAPxXxAP
X
X
.)(
)()|()|(| AP
xfxXAPAxf X
AX
,)()|()|()(
1
dxxfxXAPdxAxfAP XX
dxxfxXAPAP X )()|()(
(Total Probability Theorem)
Bayes’ Theorem (continuous version)
using total probability theorem in
We get the desired result
,)(
)()|()|(| AP
xfxXAPAxf X
AX
.)()|(
)()|()|(|
dxxfxXAP
xfxXAPAxf
X
XAX
probability of obtaining a head in a toss.
For a given coin, a-priori p can possess any value in (0,1).
: A uniform in the absence of any additional information
After tossing the coin n times, k heads are observed.
How can we update this is new information?
Let A= “k heads in n specific tosses”.
Since these tosses result in a specific sequence,
and using Total Probability Theorem we get
Example: Coin Tossing Problem Revisited
Solution
)(HPp
)( pfP
)( pfP
,)|( knkqppPAP
)( pfP
p0 1
.)!1(
!)!()1()()|()(
1
0
1
0
n
kkndpppdppfpPAPAP knk
P
The a-posteriori p.d.f represents the updated information given the event A,
Using
This is a beta distribution.
We can use this a-posteriori p.d.f to make further predictions.
For example, in the light of the above experiment, what can we say about the probability of a head occurring in the next (n+1)th toss?
Example - continued
| ( | )P Af p A
,)(
)()|()|(| AP
xfxXAPAxf X
AX
|
( | ) ( )( | )
( )
( 1)! , 0 1 ( , ).
( )! !
PP A
k n k
P A P p f pf p A
P A
np q p n k
n k k
)|(| Apf AP
p10
Let B= “head occurring in the (n+1)th toss, given that k heads have occurred in n previous tosses”.
Clearly
From Total Probability Theorem,
Using (1) in (2), we get:
Thus, if n =10, and k = 6, then
which is more realistic compare to p = 0.5.
Example - continued
,)|( ppPBP
1
0 .)|()|()( dpApfpPBPBP P
,58.012
7)( BP
1
0
( 1)! 1( ) .
( )! ! 2k n kn k
P B p p q dpn k k n