UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g1 o
f 17
IN35
7:W
IEN
ER
FIL
TE
RIN
GC
ou
rse
bo
ok:
Ch
ap.7
.2St
atis
tica
lDig
ital
Sign
alP
roce
ssin
gan
dm
od
elin
g,M
.Hay
es19
96.
Dav
idG
esb
ert
Sign
alan
dIm
age
Pro
cess
ing
Gro
up
(DSB
)h
ttp
://w
ww
.ifi.u
io.n
o/~
gesb
ert
Mar
ch20
03
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g2 o
f 17
Ou
tlin
e•
Pu
rpo
seo
fWie
ner
filt
erin
g
•E
xam
ple
1:te
mp
ora
lWie
ner
filt
erin
g
•O
pti
mu
mlin
ear
filt
erin
g
•G
eom
etri
cali
nte
rpre
tati
on
•Fi
nd
ing
the
solu
tio
n:T
he
Wie
ner
-Ho
pfe
qu
atio
ns
•A
pp
licat
ion
1:Li
nea
rp
red
icti
on
•A
pp
licat
ion
2:N
ois
eca
nce
ling
•A
pp
licat
ion
3:M
ult
i-an
ten
nas
div
ersi
tyco
mb
inin
g
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g3 o
f 17
Pu
rpo
seo
fWie
ner
filt
erin
gG
oal
:“To
esti
mat
ean
un
know
nra
nd
om
pro
cess
(RP
)fr
om
ase
tof
ob
serv
edR
Ps,
wit
hw
hic
hit
isco
rrel
ated
.”
{d(n
)}d
esir
edra
nd
om
pro
cess
(un
ob
serv
ed)
{x0(n
)}o
bse
rved
ran
do
mp
roce
ss{x
2(n
)}o
bse
rved
ran
do
mp
roce
ss. . .
{xp−
1(n
)}o
bse
rved
ran
do
mp
roce
ss
d(n)
d(n)
^
e(n)
x (
n)x
(n)
x (
n)
2 p−10
filte
rW
erro
r si
gnal
desi
red
sign
al
estim
ated
sig
nal
p ob
serv
atio
ns
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g4 o
f 17
Pu
rpo
seo
fWie
ner
filt
erin
g(I
I)Tw
oso
luti
on
sto
fin
dth
eo
per
ato
r(fi
lter
)W
:
•(1
)o
ne
has
atr
ain
ing
sign
alfo
rd
(n)
and
on
ead
just
sW
tom
inim
ize
the
pow
ero
fe(n
).
•(2
)on
eu
ses
know
led
geo
fco
rrel
atio
nb
etw
een{x
i(n
)}an
dd
(n).
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g5 o
f 17
Exa
mp
le1:
Lin
ear
tem
po
ralfi
lter
ing
⇒W
isa
linea
rF
IRfi
lter
.
⇒{x
i(n
)},i
=0,..p−
1ar
eth
esa
mp
les
ofa
wid
ese
nse
stat
ion
ary
(WSS
)ra
nd
om
sign
al.
xi(n
)=x
(n−i),i
=0..,p−
1
W(z
)=wo
+w
1z−
1+w
1z−
2+..
+wp−
1z−p+
1
. . .d̂
(n)
=w
0x
(n)
+w
1x
(n−
1)+w
2x
(n−
2)+...+
wp−
1x
(n−p
+1)
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g6 o
f 17
Vect
or
Form
ula
tio
n
W=
[w0,w
1,..,wp−
1]T
X(n
)=
[x0(n
),x
1(n
),..,x
p−1(n
)]T
d̂(n
)=W
TX
(n)
wh
ereT
isth
etr
ansp
ose
op
erat
or.
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g7 o
f 17
Op
tim
um
lin
ear
filt
erin
gP
rob
lem
:On
ew
ish
esto
fin
dth
elin
ear
filt
erW
that
min
imiz
esth
eer
ror
bet
wee
nd
(n)
andd̂
(n).
e(n
)=d
(n)−d̂
(n)
J(W
)=E|e
(n)|2
wh
ereE
()is
the
exp
ecta
tio
n.
Fin
dW
osu
chth
atJ
(Wo)
ism
inim
um
.Wo
isth
eo
pti
mu
mli
nea
rfi
lter
inth
eW
ien
erse
nse
.
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g8 o
f 17
Geo
met
rica
lIn
terp
reta
tio
nVe
cto
rsp
ace
rep
rese
nta
tio
n
•W
SSra
nd
om
pro
cess
es({e(n
)},{d
(n)},{x
0(n
)},..,{x
p−1(n
)}ca
nb
evi
ewed
asel
emen
tsin
ave
cto
rsp
ace.
•T
he
scal
arp
rod
uct
bet
wee
ntw
oR
Ps{u
(n)}
and{v
(n)}
isd
efin
edb
yth
eco
rrel
atio
n:<
u,v>
=E
(u(n
)v(n
)∗).
•O
rth
ogo
nal
ity
=d
eco
rrel
atio
n
p=2
x (
n)0x (
n)1d(
n)
e(n)
d(n)
^
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g9 o
f 17
Fin
din
gth
eso
luti
on
Th
eso
luti
onW
ois
give
nb
yth
eW
ien
er-H
op
feq
uat
ion
s.
RxW
o=r dx
wh
ere
(1)
Rx
=E
(X(n
)∗X
(n)T
)(2
)r dx
=E
(d(n
)X(n
)∗)
(3)
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g10
of 1
7
Der
ivat
ion
oft
he
Wie
ner
-Ho
pfe
qu
atio
nD
irec
tmet
ho
du
sin
go
pti
miz
atio
nth
eory
:Th
eex
trem
ao
fth
eco
stfu
nct
ionJ
(W)
are
fou
nd
fro
m(s
eeb
oo
kb
yH
ayes
p.49
):
∂J
∂w∗ 0|W
=Wo
∂J
∂w∗ 1|W
=Wo
. . .∂J
∂w∗ p−
1|W
=Wo
=
0 0 . . . 0
(4
)
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g11
of 1
7
Der
ivat
ion
usi
ng
geo
met
ry{d̂
(n)}
isth
eo
pti
mal
esti
mat
eif
f
•{d̂
(n)}
isth
ep
roje
ctio
no
f{d
(n)}
on
toth
esp
ace
span
ned
by
{x0(n
)},..,{x
p−1(n
)},o
req
uiv
alen
tly:
•{e
(n)}
iso
rth
ogo
nal
toth
esp
ace
span
ned
by{x
0(n
)},..,{x
p−1(n
)}.
<e(n
),x
0(n
)>
=E
(e(n
)x0(n
)∗)
=0
<e(n
),x
1(n
)>
=E
(e(n
)x1(n
)∗)
=0
. . .<e(n
),xp−
1(n
)>
=E
(e(n
)xp−
1(n
)∗)
=0
(5)
p=2
x (
n)0x (
n)1d(
n)
e(n)
d(n)
^
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g12
of 1
7
Exp
ress
ion
for
the
min
imu
mer
ror
Th
em
inim
um
erro
r,o
bta
ined
wit
hW
o,i
sJ
(Wo)
:
J(W
o)
=E|d
(n)|2
+W
H oRxW
o−W
H or dx−rH dxW
o
J(W
o)
=E|d
(n)|2−rH dxW
oW
o
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g13
of 1
7
Ap
pli
cati
on
1:L
inea
rp
red
icti
on
Hay
es’b
oo
k,p.
342
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g14
of 1
7
Ap
pli
cati
on
2:N
ois
eca
nce
lin
gH
ayes
’bo
ok,
p.34
9
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g15
of 1
7
Ap
pli
cati
on
3:M
ult
i-an
ten
na
div
ersi
tyco
mb
inin
g
Weig
htsph
ases
estim
ation
*s
y
1 2
h h1 2w w
=h =h
1 2*
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g16
of 1
7
An
ten
na
div
ersi
tyco
mb
inin
g(I
I)R
ecei
ved
sign
alm
od
el:
X(n
)=Hs(n
)+V
(n)
(6)
wh
ereX
(N)
colle
cts
the
sign
als
rece
ived
onp
ante
nn
as.s
(n)
isth
etr
ansm
itte
dsy
mb
ols
equ
ence
(e.g
.s(n
)=
+/−
1).H
isth
ech
ann
elve
cto
ro
fsiz
ep.V
(n)
isth
ew
hit
en
ois
eve
cto
r.
Pro
ble
m:F
indW
osu
chth
at
E|W
T oX
(n)−s(n
)|2m
inim
um
UN
IVE
RS
ITY
OF
OS
LO
DE
PA
RT
ME
NT
OF
INFO
RM
AT
ICS
D. G
esbe
rt: I
N35
7 S
tati
stic
al S
ign
al P
roce
ssin
g17
of 1
7
An
ten
na
div
ersi
tyco
mb
inin
g(I
II)
An
swer
(Wie
ner
-Ho
pf)
:
RxW
o=r dx
(H∗ H
T+σ
2 vI
)Wo
=H∗
Wo
=(H∗ H
T+σ
2 vI
)−1H∗
wh
ereσ
2 vis
the
vari
ance
ofn
ois
esa
mp
les.
NO
TE
:Th
isre
ceiv
eris
equ
ival
ent
toth
em
inim
um
mea
nsq
uar
eer
ror
(MM
SE)
rece
iver
.