, 0,1
, ,
N
X Y N real valued
NY = snrX +N
X – arbitrary fixed input distribution holds for scalar and vector signals
2
0
d
d
result known before
Xsnr
I snrsnr
Ä
No
tes
No
tes
d 1mmse
d 2I snr snr
snr
0
cmmse dsnr
I snr cmmse mmsesnr E
Mean of Filtering (causal) MMSE = Smoothing (noncausal) MMSE
‚
Γ uniformly distributed in [0,snr]
Continuous
Continuous
time
time
ˆ ; | ;f Y X Y snr E X Y snr
2ˆ;mse X X E X f Y
|mmse snr mmse X X snr N
| ;
;
| ;( ; ) log
;Y X snr
Y snr
p Y X snrI X Y E
p Y snr
; | ;; | ;Y snr Y X snrp y snr E p y X snr
2| ;
1 1| ; exp
22Y X snrp y x snr y x snr
; ;I X Y I X X snr N
Y = snrX +N
1log 1
2I snr snr ˆ ;
1
snrf Y X Y snr Y
snr
1
1mmse snr
snr
d 1mmse
d 2I snr snr
snr loge[ ]nats
Let N~(0,1) indep. of X.For every distr. pX with EX2<: d 1
mmsed 2
I snr snrsnr
ˆ ; | ;f Y H X Y snr E H X Y snr
2ˆ;mse H X H X E H X f Y
2ˆ ;mmse snr E H X H X Y snr
|mmse snr mmse X snr H X N
| ;
;
| ;( ; ) log
;Y X snr
Y snr
p Y X snrI X Y E
p Y snr
; | ;; | ;Y snr Y X snrp Y snr E p y X snr
2
2| ;
1| ; 2 exp
2
L
Y X snrp y x snr y snr H x
; ;I X Y I X snr H X N
L,KY = snr +H X N
Let N~(0,I) indep. of X.For every distr. pX with EX2<:
d 1; mmse
d 2,
in [ ]
I snrsnr
X snr H X N
I nats
2mmse snr
1 1 1
2 1 2 2
Y X N
Y Y N
1 2, indep. 0,1N N N~
21
2 21 2
1
1snr
snr
1
2
2 22; | |EI X Y Y X E X Y
1 2; ;I X Y I X Y 1 2 2; , ;I X Y Y I X YÏ I snr I snr
Known
for 0 :
1 2 Markov, chainX Y Y
1 2 2; |I X Y Y I snr I mmse sns r rn
1 11
; ,... ; | ,...,,n
in i ni
I X Y Y I X Y Y Y
Chain rule:
1
1 1; |;n
i ii
I X Y YI X Y
1 Ma... , rkov chain:nX Y Y
10
1
li2
;1
min
n
mmse rY nI X s
0
1
2
snrmmss er dI n 1d
2dI snr
smms
nre snr
d 1mmse
d 2I snr snr
snr
2
mmse ˆ ,
ˆ ; | ;
snr E X X
X f Y snr E X Y snr
Y = snrX +N lim ;snr
H X I X Y
20
1 ˆ d2E X X snX rH
2' E , XX X N~ ' '|| lim lim '; ' ;||X X Y Ysn snrrD p p D p I X Y I Xp Y
= =
2
20
2|| | d1
E1
,2
X
XX X XD D mmse X snrX Np X snr
snr
N =
0 0
1 1 d; log d
d dT T XY
XYX Y
I snrT T
pI X Y p
p p
0
2
00
cmmse cmmse ,1
d
1| ; d
T
T
t tt
snr snr t tT
E X E X Y snr tT
0
2
00
mmse mmse ,1
d
1| ; d
T
t tTT
snr snr t tT
E X E X Y snr tT
{Nt} – Gaussian channel; t[0,T]
{Xt} – r.p. lim. power 0
TEXt2dt<
Xab – {Xt} in [a,b]
pX – probab. measure of {Xt} in [0,T]
t t tY = snrX + N
1
2dlog 1 2XI snr snr S
d
2mmse
1X
Xsr
Ssn
nr S
1c1 d
loge2
mms Xsnr Sn
rr
sns
:XX PSD = Sstationary, ~Gaussian,
1dmmse
2dI snr
snrsnr
0
1cmmse mmse d
snrsnr
snr ‚
cmmse2
I snrsnr
snr
2i ini,snr = E X -E X |Y ;snrmmse
2ii-1
ii,snr = E X -E X |Y ;snrmmsep
2i iii,snr = E X -E X |Y ;snrmcm se
{Ni} – seq. of indep. Gaussian i=1,2,…; {Xi} – limited power r.p.
Xn = [X1,… Xn]T
i i iY = snrX + N
1
1mmse ,
2
d;
dn n
n
i
nI X snr X Yr
in
snrs
1 1mmse ,c pmmse ,2 2
; | ii i
snr snri sn I X Y Yr i snr ‚
1. Some direct implications on other results:Equivalency with De Bruijn’s identity which connects the differential entropy h() to the Fisher’s
information matrix J() (connected to the CRLB)
Derivative of the divergence gets an interesting form can be used also in multiuser systems:
2. Duality Information Theory Information Theory Estimation TheoryEstimation Theory:New lower/upper bounds on MMSE and mutual info. I(;)Results from one domain can be applied to the otherResults from one domain can be obtained/proven by the other e.g., linking the filtering and smoothing by a direct expression in
continuous time domain and sandwiching relation in discrete time
1 KY = H X +N, = snr ,…, snrdiag