Estimation: parts of Chapters 10-12

Bayesian Estimation

Natasha Devroye

devroye@ece.uic.edu

http://www.ece.uic.edu/~devroye

Spring 2010

Summary

Bayesian philosophy

Bayesian risk

Bayesian risk

Quadratic error cost

Absolute error cost

Hit-or-miss cost (MAP)

MMSE estimators

MMSE estimators

Example of MMSE

Bayesian estimation on Gaussian priors/noise

This is crucial to know!

Bayesian estimation on Gaussian priors/noise

This is crucial to know!

Bayesian linear model

Bayesian linear model

Examples: MMSE in Gaussian noise

Examples: MMSE in Gaussian noise

Examples: MMSE in Gaussian noise

Properties of MMSE

Maximum a Posteriori (MAP) estimation

Example

MAP properties

Linear MMSE (LMMSE)

LMMSE definition

Geometric LMMSE

Geometric LMMSE

LMMSE example

LMMSE properties

LMMSE properties

Sequential LMMSE

Algebraic approach Geometric approach

Sequential LMMSE: algebraic

Sequential LMMSE: geometric approach

3

!"#$%&"'()**+,"-.$%

/)0“Hit-or-Miss”

Cost Function

! "! "1!1!2/

1!!

|ˆ :Cov. Err.

ˆ :Estimate

!

!

#

#

//34“Squared” Cost Function

(Nonlinear Estimate)

56.$+(

2,%6(

78'-6&,'%

9//347,+-$ Linear Estimate

:',;': E{$},E{1}, CJointly Gaussian 1 and $

(Yields Linear Estimate)

! " ! "% &

1!11!1!!!

11!1

2222/

1122!!

1ˆ

1

:Cov. Err.

ˆ :Estimate

'

'

'#

'(# !! ! " ! "% &

1!11!1!!!

11!1

2222/

1122!!

1ˆ

1

:Cov. Err.

ˆ :

'

'

'#

'(# !!Estimate3"<$=

Bayesian Linear Model

(Yields Linear Estimate)

! " % & % &

% & !!!!!

!!!

>22>>2>22/

>?12>>2>2!!

1

ˆ

1

:Cov. Err.

ˆ :Estimate

'

'

('#

'((#

"

##

"

##!

See http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522_files/EECE%20522%20Notes_28%20Ch_12B.pdf