lecture 19: sums of rv’s, sample mean, laws of...
TRANSCRIPT
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 19: Sums of RVLecture 19: Sums of RV’’s, Sample s, Sample Mean, Laws of Large NumbersMean, Laws of Large Numbers
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 20: Central Limit TheoremLecture 20: Central Limit Theorem
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 21: Hypothesis Testing 1Lecture 21: Hypothesis Testing 1
Confidence Interval
Confidence interval on mean, variance known If: random sample of size n: X1, …, Xn
Xi ~ N(μ, σ2) and X ~ N(μ, σ2/n)
Then the test statistic: n
XZ/2σ
μ−= ~ N(0, 1) by CLT
With a CI, we want some range on μ, P[-Zα/2 ≤ Z ≤ Zα/2] =α,
P[-Zα/2 ≤ n
X/σμ−
≤ Zα/2] = 1- α
→ probability test statistic between 2 points is 1- α
Confidence Interval
P[-Zα/2 n/σ ≤ μ−X ≤ Zα/2 n/σ ] = 1- α → want a range on μ P[- X + (-Zα/2 n/σ )≤ -μ ≤ - X + (Zα/2 n/σ )] = 1- α P[ X + Zα/2 n/σ ≥ μ ≥ X - Zα/2 n/σ ] = 1- α P[ X - Zα/2 n/σ ≤ μ ≤ X + Zα/2 n/σ ] = 1- α
∴a 100(1- α)% CI (2-sided) on μ is: X - Zα/2 n/σ ≤ μ ≤ X + Zα/2 n/σ
or X ± Zα/2 n/σ ∴A CI is a statistic ± (table value) x standard error.
Confidence Interval
CI on mean, variance unknown Up to now, we have known σ. But typically we do not know, so what do we do?
1. If n ≥ 30, we can replace σ in the CI for the mean
with the sample SD, S= ( )2
1
11
N
jj
XN
μ=
−− ∑ .
2. if n < 30, then if X1, …, Xn ~ N(μ, σ2)
the the test statistic t = nS
X/μ−
~ t-distribution with (n-1)
degrees of freedom.
Confidence Interval
Get a CI for μ P [-tα/2, n-1 ≤ t ≤ tα/2, n-1] = 1-α
P [-tα/2, n-1 ≤ nS
X/μ−
≤ tα/2, n-1] = 1-α
∴100(1- α)% CI on μ is: X - (tα/2, n-1 nS / ) ≤ μ ≤ X + (tα/2, n-1 nS / )
or X ± tα/2, n-1 nS /
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 22: Hypothesis Testing IILecture 22: Hypothesis Testing II
Hypothesis Tests
• If the population variance is unknown, use s of the sample to approximate population variance, since under central limit theorem, s = σ when n > 30. Thus solve the problem as before, using s
• With smaller sample sizes, we have a different problem. But it is solved in the same manner. Instead of using the z distribution, we use the t distribution
Hypothesis Tests
• Using t distribution when:• Sample is small (<30)• Parent population is essentially normal
• Population variance (σ) is unknown
• As n decreases, variation within the sample increases, so distribution becomes flatter.
Tests of Hypotheses
•Two types of hypotheses: Null (H0)and alternative (H1)
µCP_1 CP_2
Sample Size Determination
Student’s t-distribution
• Suppose that X1, X2, …, Xn are n random samples from a normal distribution with mean μ and standard deviation s. Then the PDF of
• is given byns
XT/μ−
=
,,]1)/[(
1)2/(]2/)1[()( 2/)1(2 ∞<<∞−
+Γ+Γ
= + tktkk
ktf kπ.,)(
0
1 knumberpositiveanyfordxexk xk −∞
−∫=Γ
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 23: TwoLecture 23: Two--sample Testssample Tests
Inference for a difference in the means
Under the assumption listed earlier, we have the following result.
Test of hypotheses
Confidence Interval
• Verify that the calculated value of the statistic is 6.05.• Since this value is greater than the table value at 5% level, we reject the null hypothesis and conclude at 5% level that there is sufficient evidence to say that Karlsruhe method produces more strength on the average than the Lehigh method.
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 1_9: ReviewLecture 1_9: Review
Review
• Section 2.1• Examples of experiments• Sample Space & Examples
• Discrete• Continuous
• Events & Examples• Certain event• Null event
• Set Operations• Union• Intersection• Complement
Properties of Probability
a. Many times, occurcan S waysof #occurcan A waysof #
)()()(
SNANAP =
Relative frequency of percentage
b. Properties i. For an event A, P(A) = 1 – P(A)’
P(A’) = 1 – P(A)
ii. If 2 events, A & B are mutually exclusive, then P(A ∩ B) = 0.
iii. For 2 events A & B, P(A∪B) = P(A) + P(B) – P(A∩B)
iv. For 3 events A, B, C
P(A∪B∪C) = P(A) + P(B) + P(C) – P(A∩B) – P(A∩C) - P(B∩C) + P(A ∩ B ∩ C)
Probability Theory
Some Probability Laws Commutative Laws: A ∪ B = B ∪ A A ∩ B = B ∩ A Associative Laws: (A ∪ B) ∪ C = A ∪ (B ∪ C) (A ∩ B) ∩ C = A ∩ (B ∩ C) Distributive Laws: A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C) De Morgin’s Laws: (A ∪ B)’ = A’ ∩ B’ (A ∩ B)’ = A’ ∪ B’
Review
• Section 2.3• Counting
• Sampling w/ & w/o replacement• Permutations of n objects
Enumeration or Counting Technologies
Summary
With Replacement Without Replacement Order
relevant (ab ≠ ba)
n1 × n2 ×… nr )!(!rn
nPnr −=
Order irrelevant (ab = ba)
!)!1()!1(
rnrn
−+−
)!(!
!rnr
nrn
C nr −
=⎟⎟⎠
⎞⎜⎜⎝
⎛=
1,1 2, 11,2 2, 2
1, 2 2, 1
Review
• Section 2.4• Conditional probability
• Bayes’ Rule
Review
• Section 2.5• Definition of independence
Independence of events
• If the occurrence of an event A doesn’t alter the probability of the occurrence of other event B, then, event A is independent of event B.
[ ][ ] [ | ][ ]
[ ] [ ] [ ]
P A BP B P B AP A
P A B P A P B
∩= =
∩ =
Independence of events
• If the occurrence of an event A doesn’t alter the probability of the occurrence of other event B, then, event A is independent of event B.
[ ][ ] [ | ][ ]
[ ] [ ] [ ]
P A BP B P B AP A
P A B P A P B
∩= =
∩ =
Review
• Section 2.6• Sequences of independent experiments• binomial probability law• multinomial probability law• geometric probability law• sequences of dependent events
Example1Sequences of independent experiment
• Flip coin 3 times experiment• Each sub-experiment is independent. The
probability of heads is ρ
• Sample Space:{ {H,T } X {H,T} X {H,T} }
• Independent• M=m (B1, B2, B3…Bm)
• P[Bi]=pi ; (Σpi=1)• Repeat N time
Binomial vs. Multinomial• Independent• M=2 (Success/ Failure)
• P[S]=p; P[F]= 1-p• Repeat N time
( ) (1 )
for k=0,1,...nn !k !( )!
k n kn
np k p p
k
nk n k
−⎛ ⎞= −⎜ ⎟⎝ ⎠
⎛ ⎞=⎜ ⎟ −⎝ ⎠
1 21 1 2
1 2
![( ,.., )] ...! !.. !
mkk km m
m
nP k k p p pk k k
=
Sequence of dependence experiments
• The outcome of a given experiment determines which sub-experiment is performance next, or more generally, influences the outcome of next experiment.
• Probability of a certain sequence of outcomeP[ {S0}∩ {S1}∩ {S2} ]=P[{S0}]· P[{S1} | {S0}]· P[{S2} | {S0}∩{S1}]
Only recent outcome determines sub-experiment
= P[{S0}]· P[{S1} | {S0}]· P[{S2} | {S1}]
Markov Chains
Review
• Section 3.1• measurement• sample space
• Section 3.2• Discrete RV & PMF• Examples
Probability mass function
( ) ( ) 0 for all x( ) ( ) 1
( ) [ ] ( )
X
Xx Sx
Xx B
I p xII p x
III P X in B p x∈
∈
≥
=
=
∑
∑
Review
• 3.3• expected value and moments• expected value of functions of RVs• Variance of an RV
Expected value (mean)
• Definition:
• Interpretation• Center of gravity• Average in a large number of repetitions (infinite)
[ ] ( ) ( )X
X k X kx S k
E X x p x x p x∈
= ⋅ = ⋅∑ ∑
Review
• Section 3.4• Conditional PMF• conditional expected value
Conditional probability mass function
• Let X be a discrete random variable with a pmf pX(x). Let C be an event with P[C]>0.
The conditional probability mass function of X : pX(x|C)= P[ X=x | C]
P[{ X=x} ∩C]=-----------------------
P[C]
Conditional expected value • E[X|B]
[ | ] ( | )X
Xx S
E X B x p x B∈
= ⋅∑
Review
• Section 3.5• Important discrete RV’s
• Bernoulli• Binomial• Geometric• Negative binomial• Poisson
• Generation of discrete RV’s
Common Families of Discrete Probability Distribution
Bernoulli B(1,p)
p(x) = f(x) =px(1-p)1-x, x = 0, 1 M(t) = 1 – p + pet μ = p, σ2 = p(1 – p) = pq
Binomial B(n, p) p(x) = f(x) = nxpp
xnxn xnx ...,1,0,)1(
)!(!!
=−−
−
M(t) = (1 – p + pet)n μ = p, σ2 = p(1 – p) = pq
Geometric
p(x) = f(x) = (1-p)x-1p, x = 1, 2, …
M(t) = )1ln(,)1(1
ptep
Pet
t
−−<−−
μ = p1 , σ2 = 22
1pq
pp=
−
Poisson p(x) = f(x) =
!xex λλ −
, x = 0,1, …
M(t) =eλ(et-1) μ =λ, σ2 = λ
Review
• Section 4.1-4.3• CDF• PDF• Expected value of X
Continuous Random Variables
i. Let X be a continuous r.v. The probability distribution or probability density function (p.d.f.) of X is a function f(x) [i.e. p(x)] such that for any 2 numbers a and b with a < b
∫=≤≤b
a
dxxfbxaP )()(
that is, the probability X takes on a value in the interval [a,b] is the area under the graph ot the density function. In order that f(x) be a legitimate p.d.f, it must satisfy 2 conditions:
1. f(x) ≥ 0, for all x
2. 1)( =∫∞
∞−
dxxf
Expected Value for a Continuous R.V.
a. For the discrete r.v. X, E[X] was obtained by summing xp(x) over possible x values. Here we replace summation by integration and the p.m.f by pdf to get a continuous weighted average.
i. Def: The expected value or mean value of a continuous r.v. X with pdf f(x) is:
∫∞
∞−
== dxxxfXEx )(][μ
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 18: ReviewLecture 18: Review
Topics covered• Section 4.4-4.5
• Important continuous RV’s• Uniform• Exponential• Gaussian• Gamma• Rayleigh• Cauchy• laplacian
• Functions of RV’s• Section 4.6-4.7
• Markov inequality• Chebyshev inequality• transform methods
• characteristic function
• Section 5.1-5.2• Two RV’s• Pairs of Discrete RV’s
• Section 5.3-5.4• joint cdf of x and y
• marginal cdf• joint pdf of two continuous rvs
• Section 5.5-5.7• Independence• Joint moments and expected values of functions• conditional probability and conditional expectation
• Section 5.8-5.9• Function of two RV’s• Pairs of jointly Gaussian RV’s
• Section 6.1-6.2• Vector RV’s• Functions of several RV’s
• Section 6.3-6.4• Expected value of vector RV’s• Jointly Gaussian RV’s
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 10: Continuous RV FamiliesLecture 10: Continuous RV Families
The Uniform Distribution
The cumulative distribution is
12)()(
2)1()(][
)(
)()()(
2abXV
abdxab
xdxxxfXE
abax
aba
abx
ax
abxdxxf
dxxfxXPXF
xx
x
x
−=
+=
−==
−−
=−
−−
=−
==
=≤=
∫∫
∫
∫
∞−∞−
∞−
∞−
Exponential Distribution
i. Pdf: ⎩⎨⎧ >≥
=−
otherwisexe
xfx
00&0
)(λλ λ
where λ = rate at which events occur
ii. Correspondingly,
2
0
1)(
1][
0,1)()(
λ
λ
λ λλ
=
=
≥−==≤= −−∫
XV
XE
xedxexXPxF xx
x
iii. An important application of the exponential distribution is to
model the distribution of component lifetime. A reason for its popularity is because of the “memory-less” property of the Exponential Distribution
Normal Distribution
The normal distribution for parameters values μ = 0, and σ2 = 1 is called the standard normal distribution. A r.v. that has a standard distribution is called a standard normal random variable (denoted by Z). The pdf of Z is:
∞<<∞−=−
zezfz
,21)( 2
2
π
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 11: Lecture 11: ChebychevChebychev, Markov, , Markov, TransformTransform
Markov inequality
• if X is any random variable and a > 0, then
Chebyshev inequality• Chebyshev inequality states that in any data
sample or probability distribution, nearly all the values are close to the mean value, and provides a quantitative description of "nearly all" and "close to".
• In particular,• No more than 1/4 of the values are more than 2
standard deviations away from the mean; • No more than 1/9 are more than 3 standard deviations
away; • No more than 1/25 are more than 5 standard deviations
away; • and so on. In general:• No more than 1/k2 of the values are more than k
standard deviations away from the mean.
{ } 2
2
Prk
kX σμ ≤≥−
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 12: Two RVLecture 12: Two RV’’ss
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 13: Joint CDF/PDFLecture 13: Joint CDF/PDF
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 14: Independence/Lecture 14: Independence/CondCond
If E[XY]=0 then X & Y are orthogonal
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 15: Function of 2 RVLecture 15: Function of 2 RV’’ss
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 16: Vector RVs, Functions of Lecture 16: Vector RVs, Functions of several RVsseveral RVs
The Jacobian of the inverse transformation is given by
It can be shown that
We therefore conclude that the joint pdf of V and W can be found using either of the following expressions :
.det),(⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
∂∂
∂∂
∂∂
∂∂
=
wy
vy
wx
vx
wvJ
.),(
1),(yxJ
wvJ =
( )( ) (4.61b)
(4.61a)
wvJwvhwvhf
yxJwvhwvhf
wvf
YX
YXWV
,)),(),,((
,)),(),,((
),(
21,
21,,
=
=
ECE 340Probabilistic Methods in Engineering
M/W 3-4:15
Prof. Vince CalhounProf. Vince Calhoun
Lecture 17: Vector RVLecture 17: Vector RV’’s, Jointly s, Jointly Gaussian RVGaussian RV’’ss