crlb notes
DESCRIPTION
Cramer Rao Lower Bound NotesTRANSCRIPT
![Page 1: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/1.jpg)
Estimation: chapter 2+3
Minimum variance unbiased estimation + the CRLB
Natasha [email protected]://www.ece.uic.edu/~devroye
Spring 2011
• Find a few estimators
Estimation: a first example
• Estimate the DC level, A, of a signal given noisy measurements x[0], x[1], ... x[N-1] where
x[n] are samples of this!
• Compare their performance { • mean?• variance?• pdf?
![Page 2: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/2.jpg)
Estimation: a first example
• Estimators of the DC level, A
Estimation: definitions
![Page 3: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/3.jpg)
Estimation: definitions
How would you pick a ``good’’ estimator?
Vector versions....
Minimum variance unbiased estimation
Why?
So?
![Page 4: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/4.jpg)
Minimum variance unbiased estimation
Give a counter-example! (b1pg.20)
The Cramer-Rao Lower Bound
• the CRLB give a lower bound on the variance of ANY UNBIASED estimator
• does NOT guarantee bound can be obtained
• IF find an estimator whose variance = CRLB then it’s MVUE
• otherwise can use Ch.5 tools (Rao-Blackwell-Lehmann-Scheffe Theorem and Neyman-Fisher Factorization Theorem) to construct a better estimator from any unbiased one - possibly the MVUE if conditions are met
![Page 5: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/5.jpg)
The Cramer-Rao Lower Bound (CRLB)
• Use?
• Intuition?
The Cramer-Rao Lower Bound (CRLB)
![Page 6: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/6.jpg)
CRLB examples
CRLB proof
![Page 7: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/7.jpg)
CRLB T or F
What is I(θ)?
• Why “information”?
• non-negative• additive for independent observations
![Page 8: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/8.jpg)
Vector form of the CRLB
Vector form of the CRLB
![Page 9: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/9.jpg)
Vector form of the CRLB examples
What can we conclude?
CRLB for transformations
![Page 10: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/10.jpg)
Vector CRLB for transformations
Example of vector CRLB with transformation
![Page 11: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/11.jpg)
CRLB for General Gaussian Case
• When observations are Gaussian and one knows the dependence of the mean and covariance matrix on the unknown parameters, we know the closed form of the CRLB (or Fisher information matrix):
CRLB for Gaussians examples
![Page 12: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/12.jpg)
Example: range estimation
!"#$
%&'(
)*+
,)+)+)+ !"
#"
! $$##%#"#&
!
!!
!
"#$$"%#!"!#$
-.&/0%12345607833 "+#)333333/9/:7.939;7.3#&<,($"=
>7?71;73>7@67?219/8333 "+#'( !!)
A7&05.73-1%73B76&C8 !!
!"#$%&'()*$+,-.*
D&/E61%127E
FG1273H&5001&/
#$"
"+#)
#$
"+#''( !!)D4I
J3K%L
&+#)
/%0$,1$!2"3
)'*(*
#$45
678$9$:)('.$6;<&=)<&,($/>,?*.=
Continuous time signal model
!"#$
%&'(
)*+
,)+)+)+ !"
#"
! $$##%#"#&
!
!!
!
"#$$"%#!"!#$
-.&/0%12345607833 "+#)333333/9/:7.939;7.3#&<,($"=
>7?71;73>7@67?219/8333 "+#'( !!)
A7&05.73-1%73B76&C8 !!
!"#$%&'()*$+,-.*
D&/E61%127E
FG1273H&5001&/
#$"
"+#)
#$
"+#''( !!)D4I
J3K%L
&+#)
/%0$,1$!2"3
)'*(*
#$45
678$9$:)('.$6;<&=)<&,($/>,?*.=
Taken from http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522.htm
![Page 13: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/13.jpg)
Example: range estimation
Discrete time signal model
Taken from http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522.htm
!"#$
%&'()*+,-*./+! 0+#"1!"2*3
#4$5+0+#6$!7
89+:;<=*+
>&?22<&@+AB<2*
C&.+"! " !"#
#DD#DE547654 #$%#!$ %$$#$&$' ( !&
)"
#$%&'(&$)%*
&#"1!
!*!
#"! !"1!
+,-&'(&$)%*
"#.! "! " !"
#
''(
'')
*
#++%
#%++%#!
#++
$
#54
#5476
#E54
54
%$+$$#
+$$$$#$&
$$$#
$'
(
(((
(
&
&4$F&(5G ;&2++ @B@HI*.B+2&'()*2+2=&.=<@J+&=+$(
/0123&45678067'1&-9:&,7210;&<'=3;
!"#$
%&'()*+,-*./+! 0+#"1!"2*3
#4$5+0+#6$!7
89+:;<=*+
>&?22<&@+AB<2*
C&.+"! " !"#
#DD#DE547654 #$%#!$ %$$#$&$' ( !&
)"
#$%&'(&$)%*
&#"1!
!*!
#"! !"1!
+,-&'(&$)%*
"#.! "! " !"
#
''(
'')
*
#++%
#%++%#!
#++
$
#54
#5476
#E54
54
%$+$$#
+$$$$#$&
$$$#
$'
(
(((
(
&
&4$F&(5G ;&2++ @B@HI*.B+2&'()*2+2=&.=<@J+&=+$(
/0123&45678067'1&-9:&,7210;&<'=3;
Example: range estimation
Taken from http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522.htm
!"#$
%&'()**+,(-.)/0)10(2345(16-7+.(8&1(-9:/)+(;(<=%>
!!
!!
"
# $#
"%
# "$#
"%
#
"
#
&&'
())*
+
,
,#
&&
'
(
))
*
+
,
,
#
&&'
())*
+
,
"$,#
&&'
())*
+
,
,-
#
?
@
@
#@
@
# @
@
#
?
@
@
ABAB
ABCDE
AFG)1B
!
" "#
!"
"" "#
!"
"" $
$
%
" $
$
$
#
#&
#
#&
"&"&
$
$ $
$
$
..
/
/
.
/
/
./
/
H+7:(9/I()/0(J66*(
/&/KL61&(.61M-
NO*+&9.(2)+P7+7-QQQ R-6()**1&O9M).9&/>(/$'('$ "
$
ST6/(0&(PT)/:6(&8(G)19)U+6-QQ
!"#$%&'()*+")*,#&-!./
![Page 14: crlb notes](https://reader033.vdocuments.site/reader033/viewer/2022051019/55cf92d4550346f57b99f127/html5/thumbnails/14.jpg)
Example: range estimation
Taken from http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522.htm
!"#$
!
"
#
!
"
#
"#
$
%&&
&!
'
$%&
&
&!
'
%&&
&! !!!
!!! "#
$%&
'((
)
"#
$%&
'((
)
"#
$%&
'((
*
+
%
&
%
&
%
&
&
'(
&"
#
'(
&"
'(#')*+,(
,-
-../012.+03412.3+567826.2.0+449 +33,:;<2./02=>267?18,+49
!)!"
! %&&!$%
& '(
. /
. /
!!
!
0
01
0
01
)
+
%&()
%(()(
'
$
$
%(()(
'
$
#
!
!
"
#
!
#!
&
&&
%
&&
'(
'(&
&"
#
'(&
&"
#')*+,(
2
2- @A2AB1:,10
C2D+,.1*+4
D+,.1*+4. /
!
!0
01
0
01)%&()
%(()(*+,!
&
&&
'(
'(&2
!"#$%"&'&()&*"'+,-".
*+,! 6.2EFGH2IJK22(LM'-2?>312:N2EHOFK
/'%0"&1+2$*'2$3%&4/5(&673%289
Example: range estimation
Taken from http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522.htm
!"#$
%&'()*+,-&-*'.-/&*0-*/11'2-*/+*+,-*3456*7(*+,-*.-8/9:
;<&-=#
;>2/1< ?
?!"#
$
%&'( !"#
@7*)-+*+,-*3456*7(*+,-*!"#$%A*B&-*C+1/(&DE*7D*F/1G&H*1-&B8+:*
;<GI"
;>2/1< ?
?
?
!"#%&'(
)(
!"
$*(+%
(*(+%
$( ##
>
?
> $$%
&''(
)
*
*+ 0'+,* ( J*)#
$,"*?
3456*'&*'(2-1&-89*F17F71+'7(/8*+7:
K LM4*N-/&B1-
K 4NL*6O*N-/&B1-
3456*'&*'(2-1&-89*F17F71+'7(/8*+7:
K LM4*N-/&B1-
K 4NL*6O*N-/&B1-
L7*+,-*3456*+-88&*B&A
K 3,77&-*&')(/8*0'+,*8/1)-*%!"#
K P(&B1-*+,/+*LM4*'&*8/1)-
K 6-++-1*7(*M-/1Q9"8/1)-*+/1)-+&
K O,'=,*'&*Q-++-1R**
S T7BQ8-*+1/(&G'++-.*-(-1)9R
S T7BQ8-*4NL*Q/(.0'.+,R
&"#$%'()*+,"*+-#'.&/0'12-#*34
!"#$
%&'()*+,-&-*'.-/&*0-*/11'2-*/+*+,-*3456*7(*+,-*.-8/9:
;<&-=#
;>2/1< ?
?!"#
$
%&'( !"#
@7*)-+*+,-*3456*7(*+,-*!"#$%A*B&-*C+1/(&DE*7D*F/1G&H*1-&B8+:*
;<GI"
;>2/1< ?
?
?
!"#%&'(
)(
!"
$*(+%
(*(+%
$( ##
>
?
> $$%
&''(
)
*
*+ 0'+,* ( J*)#
$,"*?
3456*'&*'(2-1&-89*F17F71+'7(/8*+7:
K LM4*N-/&B1-
K 4NL*6O*N-/&B1-
3456*'&*'(2-1&-89*F17F71+'7(/8*+7:
K LM4*N-/&B1-
K 4NL*6O*N-/&B1-
L7*+,-*3456*+-88&*B&A
K 3,77&-*&')(/8*0'+,*8/1)-*%!"#
K P(&B1-*+,/+*LM4*'&*8/1)-
K 6-++-1*7(*M-/1Q9"8/1)-*+/1)-+&
K O,'=,*'&*Q-++-1R**
S T7BQ8-*+1/(&G'++-.*-(-1)9R
S T7BQ8-*4NL*Q/(.0'.+,R
&"#$%'()*+,"*+-#'.&/0'12-#*34