7/29/2019 MIT6_011S10_chap11

1/17

C H A P T E R 11WienerFilteringINTRODUCTIONInthischapterwewillconsidertheuseofLTIsystemsinordertoperformminimummeansquareerror(MMSE)estimationofaWSSrandomprocessofinterest,givenmeasurements of another related process. The measurements are applied to theinput of the LTI system, and the system is designed to produce as its output theMMSEestimateoftheprocessof interest.We first develop the results in discrete time, and for convenience assume (unlessotherwisestated)thattheprocesseswedealwitharezero-mean. Wewillthenshowthatexactlyanalogousresultsapply incontinuoustime, althoughtheirderivationisslightlydifferentincertainparts.OurproblemintheDTcasemaybestated intermsofFigure11.1.Here x[n] is a WSS random process that we have measurements of. We wantto determine the unit sample response or frequency response of the above LTIsystemsuchthatthefilteroutputy

[n]istheminimummeansquareerror(MMSE)

estimateofsometargetprocessy[n]that isjointlyWSSwithx[n]. Definingtheerror

e[n]

as

e[n] =y[n]y[n], (11.1)wewishtocarryoutthefollowingminimization:

min=E{e2[n]}. (11.2)h[ ]

Theresultingfilterh[n]iscalledtheWienerfilterforestimationofy[n]fromx[n].In some contexts it is appropriate or convenient to restrict the filter to be anFIR (finiteduration impulse response) filter of length N, e.g. h[n] =0 except inthe interval 0 n N1. In other contexts the filter impulse response canbe of infinite duration and may either be restricted to be causal or allowed tobe noncausal. In the next section we discuss the FIR and general noncausal IIR

x[n] LTIh[n] y[n]= estimatey[n]= targetprocess

FIGURE11.1 DTLTIfilterforlinearMMSEestimation.cAlanV.OppenheimandGeorgeC.Verghese,2010 195

7/29/2019 MIT6_011S10_chap11

2/17

196 Chapter11 WienerFiltering(infiniteduration impulse response) cases. A later section deals with the moreinvolvedcasewherethefilter isIIRbutrestrictedtobecausal.Ifx[n] =y[n]+v[n]wherey[n]isasignalandv[n]isnoise(bothrandomprocesses),thentheaboveestimationproblemiscalledafilteringproblem. Ify[n] =x[n+n0]with n0 positive, and if h[n] is restricted to be causal, then we have apredictionproblem. Both fit withinthe same general framework, butthe solution under therestrictionthath[n]becausalismoresubtle.

11.1 NONCAUSALDTWIENERFILTERTodeterminetheoptimalchoiceforh[n]in(11.2),wefirstexpandtheerrorcriterionin(11.2):

=E

+

k =

h[k]x[n

k]

y[n]2

. (11.3)The impulse response values that minimize can then be obtained by setting

= 0 for all values of m for which h[m] is not restricted to be zero (or

h[m]otherwiseprespecified):

h[m] =E

2 h[k]x[nk]y[n] x[nm]

ke[n]

= 0. (11.4)

Theabove

equation

implies

that

E{e[n]x[nm]}= 0, or

Rex[m] = 0, forallmforwhichh[m]canbefreelychosen. (11.5)Youmayrecognizetheaboveequation(orconstraint)ontherelationbetweentheinput and the error as the familiarorthogonality principle: for the optimal filter,theerror isorthogonal toall thedata that isused toform theestimate. Underourassumptionofzeromeanx[n], orthogonality isequivalenttouncorrelatedness. Aswewillshowshortly,theorthogonalityprinciplealsoapplies incontinuoustime.Notethat

Rex[m] =E{e[n]x[n

m]}

=E{(y[n]y[n])x[nm]}=R [m]Ryx[m].yx (11.6)

Therefore,analternativewayofstatingtheorthogonalityprinciple(11.5) isthatR

yx[m] =Ryx[m]forallappropriatem . (11.7)

cAlanV.OppenheimandGeorgeC.Verghese,2010

7/29/2019 MIT6_011S10_chap11

3/17

Section11.1 NoncausalDTWienerFilter 197Inotherwords,fortheoptimalsystem,thecrosscorrelationbetweentheinputandoutput of the estimator equals the crosscorrelation between the input and targetoutput.To actually find the impulse response values, observe that since y[n] is obtainedby filtering x[n] through an LTI system with impulse response h[n], the followingrelationshipapplies:

Ryx

[m] =h[m]Rxx[m]. (11.8)Combining this with the alternative statement of the orthogonality condition, wecanwrite

h[m]Rxx[m] =Ryx[m], (11.9)orequivalently,

h[k]Rxx[mk] =Ryx[m] (11.10)k

Equation (11.10) represents a set of linear equations to be solved for the impulseresponsevalues. IfthefilterisFIRoflengthN,thenthereareN equationsintheN unrestricted values of h[n]. For instance, suppose that h[n] is restricted to bezeroexceptforn[0, N1]. Thecondition(11.10)thenyieldsasmanyequationsasunknowns,whichcanbearranged inthefollowingmatrixform,whichyoumayrecognizeastheappropriateformofthenormalequationsforLMMSEestimation,whichweintroduced inChapter8:

Rxx[0] Rxx[1] Rxx[1N] h[0] Ryx[0] Rxx[1] Rxx[0] Rxx[2N] h[1] = Ryx[1] . . . . . . .

. . . .

.

.

. . . .

.

.

Rxx[N

1] Rxx[N

2] Rxx[0] h[N

1] Ryx[N

1]

(11.11)

Theseequationscannowbesolvedfortheimpulseresponsevalues. Becauseoftheparticular structure of these equations, there are efficient methods for solving forthe unknown parameters, but further discussion of these methods is beyond thescopeofourcourse.In the case of an IIR filter, equation (11.10) must hold for an infinite number ofvalues of m and, therefore, cannot simply be solved by the methods used for afinitenumberof linearequations. However, ifh[n] isnotrestrictedtobecausalorFIR, the equation (11.10) must hold for all values of m from to +, so theztransformcanbeappliedtoequation(11.10)toobtain

H(z)Sxx(z) =Syx(z) (11.12)The optimal transfer function, i.e. the transfer functionofthe resulting (Wiener)filter,isthen

H(z) =Syx(z)/Sxx(z) (11.13)If either of the correlation functions involved in this calculation does not possessa ztransform but if both possess Fourier transforms, then the calculation can becarriedout intheFouriertransformdomain.AlanV.OppenheimandGeorgeC.Verghese,2010c

7/29/2019 MIT6_011S10_chap11

4/17

198 Chapter11 WienerFilteringNote the similarity between the above expression for the optimal filter and theexpressionweobtainedinChapters5and7forthegainY X/XX thatmultipliesazeromeanrandomvariableX toproducetheLMMSEestimatorforazeromeanrandom variables Y. In effect, by going to the transform domain or frequencydomain,wehavedecoupledthedesignintoaproblemthatateachfrequencyisassimpleastheonewesolvedintheearlierchapters.Aswewillseeshortly,incontinuoustimetheresultsareexactlythesame:

Ryx

() =Ryx(), (11.14)h()Rxx() =Ryx(), (11.15)

H(s)Sxx(s) =Syx(s), and (11.16)

H(s) =Syx(s)/Sxx(s) (11.17)The meansquareerror corresponding to the optimum filter, i.e. the minimumMSE, can be determined by straightforward computation. We leave you to showthat

Ree[m] =Ryy [m]R [m] =Ryy[m]h[m]Rxy[m] (11.18)yywhere h[m] is the impulse response of the optimal filter. The MMSE is thenjustRee[0]. Itisilluminatingtorewritethisinthefrequencydomain,butdroppingtheargumentej onthepowerspectraS (ej)andfrequencyresponseH(ej)belowtoavoidnotationalclutter:

1 MMSE=Ree[0]= Seed2

1 = (SyyHSxy)d

2

1 SyxSxy=

2Syy

1

Syy Sxx

d1

= Syy1yxyx d. (11.19)2

Thefunctionyx(ej)definedbyyx(e

j) = Syx(ej) (11.20)(ej)Syy(ej)Sxx

evidentlyplaystheroleofafrequencydomaincorrelationcoefficient(comparewithourearlierdefinition ofthe correlationcoefficient between tworandom variables).Thisfunctionissometimesreferredtoasthecoherencefunctionofthetwoprocesses.Again,notethesimilarityofthisexpressiontotheexpressionY Y(12 )thatweY Xobtainedinapreviouslectureforthe(minimum)meansquareerrorafterLMMSEcAlanV.OppenheimandGeorgeC.Verghese,2010

7/29/2019 MIT6_011S10_chap11

5/17

Section11.1 NoncausalDTWienerFilter 199estimationofarandomvariableY usingmeasurementsofarandomvariableX.

EXAMPLE11.1 SignalEstimation inNoise(Filtering)Considerasituationinwhichx[n],thesumofatargetprocessy[n]andnoisev[n],isobserved:

x[n] =y[n] +v[n]. (11.21)We would like to estimate y[n] from our observations of x[n]. Assume that thesignalandnoiseareuncorrelated, i.e. Rvy[m]=0. Then

Rxx[m] =Ryy [m] +Rvv [m], (11.22)Ryx[m] =Ryy [m], (11.23)H(ej) = Syy (ej) . (11.24)

Syy (ej

) +Svv(ej)At values of for which the signal power is much greater than the noise power,H(ej) 1. Where the noise power is much greater than the signal power,H(ej)0. Forexample,when

Syy(ej)=(1 +ej)(1+ej)=2(1+cos) (11.25)andthenoise iswhite,theoptimalfilterwillbea lowpassfilterwitha frequencyresponsethatisappropriatelyshaped,showninFigure11.2. Notethatthefilterin

4

3.5

3

2.5

2

1.5

1

0.5

0

/2 0 /2

S (ej

)yy

H(ej

)

S (ej

)vv

FIGURE11.2 Optimal filter frequency response, H(ej), input signal PSD signal,Syy (e

j),andPSDofwhitenoise,Svv (ej).thiscasemusthavean impulseresponsethatisanevenfunctionoftime,since itsfrequencyresponseisarealandhenceevenfunctionoffrequency.Figure 11.3 shows a simulation example of such a filter in action (though for adifferent Syy (ej). The top plot is the PSD of the signal of interest; the middleplotshowsboththesignals[n]andthemeasuredsignalx[n];andthebottomplotcomparestheestimateofs[n]withs[n] itself.

AlanV.OppenheimandGeorgeC.Verghese,2010c

7/29/2019 MIT6_011S10_chap11

6/17

200 Chapter11 WienerFiltering

FIGURE11.3 Wienerfilteringexample. (FromS.M.Kay,FundamentalsofStatisticalSignalProcessing:EstimationTheory,PrenticeHall,1993. Figures11.9and11.10.)AlanV.OppenheimandGeorgeC.Verghese,2010c

246

810

-10-8-6-4-20

0 5 10 15 20 25 30 35 40 45 50

Data x

Signal y

Sample number, n

(a) Signal and Data

Wiener Filtering Example

2468

10

-10-8-6-4-20

0 5 10 15 20 25 30 35 40 45 50

Sample number, n

(b) Signal and Signal Estimate

Signal estimate y

True signal y

30

25

2015

10

5

0

-5

-10-0.5 -0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5

Syy

Powerspectralde

nsity

(dB)

Power spectral density of AR(1) process

Frequency

Image by MIT OpenCourseWare, adapted from Fundamentals of StatisticalSignal Processing: Estimation Theory, Steven Kay. Prentice Hall, 1993.

7/29/2019 MIT6_011S10_chap11

7/17

Section11.1 NoncausalDTWienerFilter 201

EXAMPLE11.2

Prediction

Supposewewishtopredictthemeasuredprocessn0 stepsahead,so

y[n] =x[n+n0]. (11.26)Then

Ryx[m] =Rxx[m+n0] (11.27)sotheoptimumfilterhassystemfunction

H(z) =zn0 . (11.28)This is of course not surprising: since were allowing the filter to be noncausal,prediction is not a difficult problem! Causal prediction is much more challengingand interesting,andwewillexamine itlater inthischapter.

EXAMPLE11.3 Deblurring(orDeconvolution)v[n]

x[n] G(z) H(z) x[n]r[n] [n]

Known,stablesystem WienerfilterFIGURE11.4 Wienerfilteringofablurredandnoisysignal.

In the Figure 11.4, r[n] is a filtered or blurred version of the signal of interest,x[n],whilev[n]isadditivenoisethatisuncorrelatedwithx[n]. Wewishtodesignafilterthatwilldeblurthenoisymeasuredsignal[n]andproduceanestimateoftheinputsignalx[n]. Notethat intheabsenceoftheadditivenoise,the inversefilter1/G(z) will recover the input exactly. However, this is not a good solution whennoise is present, because the inverse filter accentuates precisely those frequencieswhere the measurement power is small relative to that of the noise. We shallthereforedesignaWienerfiltertoproduceanestimateofthesignalx[n].We have shown that the crosscorrelation between the measured signal, which isthe inputtotheWienerfilter,andtheestimateproducedatitsoutput isequaltothecrosscorrelationbetweenthemeasurementprocessandthetargetprocess. Inthetransformdomain,thestatementofthiscondition is

Sx

(z) =Sx(z) (11.29)or

S(z)H(z) =S (z) =Sx(z). (11.30)xAlanV.OppenheimandGeorgeC.Verghese,2010c

7/29/2019 MIT6_011S10_chap11

8/17

202 Chapter11 WienerFilteringWealsoknowthat

S (z) =Svv (z) +Sxx(z)G(z)G(1/z) (11.31)Sx(z) =Sxr(z) (11.32)

=Sxx(z)G(1/z), (11.33)wherewehave(inthefirstequalityabove)usedthefactthatSvr (z) =G(1/z)Svx(z) =0. Wecannowwrite

Sxx(z)G(1/z)H(z) = . (11.34)

Svv (z) +Sxx(z)G(z)G(1/z)We leave youtocheck thatthis system function assumesreasonable values in thelimitingcaseswherethenoisepowerisverysmall,orverylarge. Itisalsointerestingtoverifythatthesameoverallfilter isobtainedifwefirstfindanMMSEestimater[n] from [n] (as in Example 11.1), and then pass r[n] through the inverse filter1/G(z).

EXAMPLE11.4 De-MultiplicationAmessages[n]istransmittedoveramultiplicativechannel(e.g. afadingchannel)sothatthereceivedsignalr[n] is

r[n] =s[n]f[n]. (11.35)Suppose s[n] and f[n] are zero mean and independent. We wish to estimate s[n]fromr[n]usingaWienerfilter.Again,wehave

Rsr[m] =Rsr[m]=h[m] Rrr [m] . (11.36)

Rss[m]Rff[m]

ButwealsoknowthatRsr[m]=0. Thereforeh[m]=0. Thisexampleemphasizesthat the optimality of a filter satisfying certain constraints and minimizing somecriterion does not necessarily make the filter a good one. The constraints on thefilter and the criterion have to be relevant and appropriate for the intended task.Forinstance,iff[n]wasknowntobei.i.d. and+1or1ateachtime,thensimplysquaringthereceivedsignalr[n]atanytimewouldhaveatleastgivenusthevalueofs2[n],whichwouldseemtobemorevaluableinformationthanwhattheWienerfilterproduces inthiscase.

AlanV.OppenheimandGeorgeC.Verghese,2010c

7/29/2019 MIT6_011S10_chap11

9/17

Section11.2 NoncausalCTWienerFilter 20311.2 NONCAUSALCTWIENERFILTER

InthepreviousdiscussionwederivedandillustratedthediscretetimeWienerfilterfortheFIRandnoncausalIIRcases. Inthissectionwederivethecontinuoustime counterpart of the result for the noncausal IIR Wiener filter. The DT derivationinvolved taking derivatives with respect to a (countable) set of parameters h[m],butintheCTcasetheimpulseresponsethatweseektocomputeisaCTfunctionh(t), so the DT derivation cannot be directly copied. However, you will see thattheresultstakethesameformasintheDTcase;furthermore,thederivationbelowhas anaturalDTcounterpart, whichprovidesanalternate route tothe results intheprecedingsection.OurproblemisagainstatedintermsofFigure11.5.

Estimatorx(t) h(t),H(j) y(t)= estimate

y(t)= targetprocessFIGURE11.5 CTLTIfilterfor linearMMSEestimation.

Let x(t) be a (zeromean) WSS random process that we have measurements of.WewanttodeterminetheimpulseresponseorfrequencyresponseoftheaboveLTIsystemsuchthatthefilteroutputy(t)istheLMMSEestimateofsome(zeromean)targetprocessy(t)that isjointlyWSSwithx(t). Wecanagainwrite

e(t) =y(t)y(t)

min=E{e2(t)}. (11.37)h( )

Assumingthefilterisstable(oratleasthasawelldefinedfrequencyresponse),theprocessy(t) isjointlyWSSwithx(t). Furthermore,

E[y(t+)y(t)]=h()Rxy() =Ryy (), (11.38)Thequantitywewanttominimizecanagainbewrittenas

=E{e2(t)}=Ree(0), (11.39)wheretheerrorautocorrelationfunctionRee()isusingthedefinitionin(11.37)evidentlygivenby

Ree() =Ryy () +R y()R y()Ryy (). (11.40)y y cAlanV.OppenheimandGeorgeC.Verghese,2010

7/29/2019 MIT6_011S10_chap11

10/17

204 Chapter11 WienerFilteringThus

=E{e2(t)}=Ree(0) = 1

See(j)d2

= 1

Syy (j) +S y(j)S y(j)Syy (j)

d2 y y

1

= (Syy +HHSxxHSyxHSxy)d, (11.41)2

wherewehavedroppedtheargumentj fromthePSDs inthe last lineabove fornotationalsimplicity,andhaveusedH todenotethecomplexconjugateofH(j),namelyH(j). Theexpressioninthislastlineisobtainedbyusingthefactthatx(t)andy(t)aretheWSSinputandoutput,respectively,ofafilterwhosefrequencyresponseisH(j). NotealsothatbecauseRyx() =Rxy()wehave

Syx =Syx(j) =Sxy(j) =S . (11.42)xyOur task is now to choose H(j) to minimize the integral in (11.41). We can dothisbyminimizingthe integrand foreach. Thefirstterm inthe integranddoesnotinvolveordependonH,so ineffectweneedtominimize

HHSxxHSyxHSxy =HHSxxHSyxHS . (11.43)yxIfallthequantitiesinthisequationwerereal,thisminimizationwouldbestraight-forward. EvenwithacomplexH andSyx,however,theminimization isnothard.The keytothe minimization is an elementarytechnique referred to ascompletingthe square. For this, we write the quantity in (11.43) in terms of the squaredmagnitude of a term that is linear in H. This leads to the following rewriting of(11.43):

Syx Syx SSyx yx

H

Sxx Sxx

H

Sxx Sxx Sxx . (11.44)

InwritingSxx,wehavemadeuseofthefactthatSxx(j)isrealandnonnegative.WehavealsofeltfreetodividebySxx(j)becauseforanywherethisquantityis0itcanbeshownthatSyx(j)=0also. TheoptimalchoiceofH(j)isthereforearbitrary at such , as evident from (11.43). We thus only need to compute theoptimalH atfrequencieswhereSxx(j)>0.Notice that the second term in parentheses in (11.44) is the complex conjugateof the first term, so the product of these two terms in parentheses is real andnonnegative. Also, the last term does not involve H at all. To cause the termsinparenthesestovanishandtheirproducttotherebybecome0,which isthebestwecando, we evidently must choose as follows (assumingthere arenoadditionalconstraintssuchascausalityontheestimator):

Syx(j)H(j)= (11.45)

Sxx(j)ThisexpressionhasthesameformasintheDTcase. TheformulaforH(j)causesitto inheritthesymmetrypropertiesofSyx(j),soH(j)hasarealpartthat isAlanV.OppenheimandGeorgeC.Verghese,2010c

7/29/2019 MIT6_011S10_chap11

11/17

Section11.3 CausalWienerFiltering 205even in,andan imaginarypartthat isodd in. Its inversetransform isthusarealimpulseresponseh(t),andtheexpression in(11.45) isthefrequencyresponseoftheoptimum(Wiener)filter.Withthechoiceofoptimumfilterfrequencyresponsein(11.45),themeansquare-errorexpressionin(11.41)reduces(justas intheDTcase)to:

1 MMSE=Ree(0)= Seed

2

1 = (SyyHSxy)d

2

= 1

Syy

1SyxSxyd2 Syy Sxx1

= Syy (1)d (11.46)2

wherethefunction(j) isdefinedby

Syx(j)(j)= (11.47)

Syy (j)Sxx(j)andevidentlyplaystheroleofa(complex)frequencybyfrequencycorrelationco-efficient,analogoustothatplayedbythecorrelationcoefficientofrandomvariablesY andX.

11.2.1 OrthogonalityPropertyRearrangingtheequationfortheoptimalWienerfilter,wefind

H Sxx =Syx (11.48)or

Syx =Syx , (11.49)

orequivalentlyR

yx() =Ryx()forall . (11.50)

Again,fortheoptimalsystem,thecrosscorrelationbetweentheinputandoutputoftheestimatorequalsthecrosscorrelationbetweenthe inputandtargetoutput.Yetanotherwaytostatetheaboveresultisviathefollowingorthogonalityproperty:

Rex() =R ()Ryx() = 0forall . (11.51)yxIn

other

words,

for

the

optimal

system,

the

error

is

orthogonal

to

the

data.

11.3 CAUSALWIENERFILTERING

IntheprecedingdiscussionwedevelopedtheWienerfilterwithnorestrictionsonthe filter frequency response H(j). This allowed us to minimize a frequencydomainintegralbychoosingH(j)ateachtominimizetheintegrand. However,cAlanV.OppenheimandGeorgeC.Verghese,2010

7/29/2019 MIT6_011S10_chap11

12/17

206 Chapter11 WienerFilteringifweconstrainthefiltertobecausal,thenthefrequencyresponsecannotbechosenarbitrarilyateachfrequency,sothepreviousapproachneedstobemodified. Itcanbeshownthat foracausalsystemtherealpartofH(j)canbedeterminedfromthe imaginary part, and vice versa, using what is known as a Hilbert transform.ThisshowsthatH(j)isconstrainedinthecausalcase. (Weshallnotneedtodealexplicitly with the particular constraint relating the real and imaginary parts ofH(j),sowewillnotpursuetheHilberttransformconnectionhere.) Thedevelop-mentoftheWienerfilterinthecausalcaseisthereforesubtlerthantheunrestrictedcase,butyouknowenoughnowtobeabletofollowtheargument.Recallourproblem,described intermsofFigure11.6.

Estimatorx(t) h(t),H(j) y

(t)= estimate

y(t)=

target

process

FIGURE11.6 RepresentationofLMMSEestimationusinganLTIsystem.

The inputx(t) isa(zeromean)WSSrandomprocessthatwehavemeasurementsof, and we want to determine the impulse response or frequency response of theaboveLTIsystemsuchthatthefilteroutputy(t)istheLMMSEestimateofsome(zeromean)targetprocessy(t)thatisjointlyWSSwithx(t):

e(t) =y(t)y(t)

min=E{e2(t)}. (11.52)h( )

We shall now require, however, that the filter be causal. This is essential in, forexample,theproblemofprediction,wherey(t) =x(t+t0)witht0 >0.Wehavealreadyseenthatthequantitywewanttominimizecanbewrittenas

1 =E{e2(t)}=Ree(0)= See(j)d

2

= 1

Syy (j) +S (j)S (j)S (j)dy y yy2 y y

1 = (Syy +HHSxxHSyxHSxy)d (11.53)

2

Syx 2 yx= 1

H

Sxx

d+ 1

SyySyxS

d.

2 Sxx 2 Sxx (11.54)

Thelastequalitywastheresultofcompletingthesquareontheintegrandintheprecedingintegral. InthecasewhereH isunrestricted,wecansetthefirstintegralofthe lastequationto0bychoosing

Syx(j)H(j)= (11.55)

Sxx(j)cAlanV.OppenheimandGeorgeC.Verghese,2010

7/29/2019 MIT6_011S10_chap11

13/17

Section11.3 CausalWienerFiltering 207at each frequency. The second integral of the last equation is unaffected by ourchoiceofH,anddeterminestheMMSE.IftheWienerfilterisrequiredtobecausal,thenwehavetodealwiththeintegral

Syx 22

1 HSxx

Sxx d (11.56)

as a whole when we minimize it, because causality imposes constraints on H(j)that prevent it being chosen freely at each . (Because of the Hilbert transformrelationshipmentionedearlier,wecouldforinstancechoosetherealpartofH(j)freely, but then the imaginary part would be totally determined.) We thereforehavetoproceedmorecarefully.Notefirstthattheexpressionweobtainedfortheintegrandin(11.56)bycompletingthesquareisactuallynotquiteasgeneralaswemighthavemadeit. Sincewemayneed

to

use

all

the

flexibility

available

to

us

when

we

tackle

the

constrained

problem,

weshouldexplorehowgenerallywecancompletethesquare. Specifically, insteadof using the real square root Sxx of the PSD Sxx, we could choose a complexsquarerootMxx,definedbytherequirementthat

M or (j) =Mxx(j)Mxx(j), (11.57)Sxx =Mxx xx Sxxandcorrespondinglyrewritethecriterion in(11.56)as

21 HMxx Syx d, (11.58)2 M

xxwhich iseasilyverifiedtobethesamecriterion, althoughwrittendifferently. ThequantityMxx(j)istermedaspectralfactorofSxx(j)oramodelingfilterfortheprocessx. Thereasonforthelatternameisthatpassing(zeromean)unitvariancewhitenoisethroughafilterwithfrequencyresponseMxx(j)willproduceaprocesswiththe PSDSxx(j), sowecanmodel theprocess x as being the resultof suchafilteringoperation. NotethattherealsquarerootSxx(j)weusedearlier isaspecialcaseofaspectralfactor,butothersexist. Infact,multiplyingSxx(j)byanallpassfrequencyresponseA(j)willyieldamodelingfilter:

A(j)Sxx(j) =Mxx(j), A(j)A(j) = 1. (11.59)Conversely, it is easy to show that the frequency response of any modeling filtercanbewrittenastheproductofanallpassfrequencyresponseandSxx(j).It turns out that under fairly mild conditions (which we shall not go into here) aPSDisguaranteedtohaveaspectralfactorthatisthefrequencyresponseofastableandcausal

system,and

whose

inverseis

also

the

frequency

response

of

astableand

causalsystem. (Tosimplifyhowwetalkaboutsuchfactors,weshalladoptanabuseofterminologythatiscommonwhentalkingaboutFouriertransforms,referringtothe factor itselfratherthanthesystemwhose frequencyresponse isthis factorasbeingstableandcausal,withastableandcausal inverse.) For instance,if

2 + 9Sxx(j) = , (11.60)

2 + 4AlanV.OppenheimandGeorgeC.Verghese,2010c

7/29/2019 MIT6_011S10_chap11

14/17

208 Chapter11 WienerFilteringthentherequiredfactor is

j+ 3M

xx(j) = . (11.61)

j+ 2WeshalllimitourselvesentirelytoSxx thathavesuchaspectralfactor,andassumefor the rest of the derivation that the Mxx introduced in the criterion (11.58) issuch a factor. (Keep in mind that wherever we ask for a stable system here, wecanactuallymakedowithasystemwithawelldefinedfrequencyresponse,evenifitsnotBIBOstable,exceptthatourresultsmaythenneedtobeinterpretedmorecarefully.)With these understandings, it is evident that the termHMxx in the integrand in(11.58)iscausal,asitisthecascadeoftwocausalterms. Theotherterm,Syx/M ,xxis generally not causal, but we may separate its causal part out, denoting thetransformofitscausalpartby[Syx/M ]+,andthetransformofitsanticausalpartxxby[Syx/M ] (IntheDTcase,thelatterwouldactuallydenotethetransformofxx .thestrictlyanticausalpart,i.e.,attimes1andearlier;thevalueattime0wouldberetainedwiththecausalpart.)Now consider rewriting (11.58) in the time domain, using Parsevals theorem. IfwedenotetheinversetransformoperationbyI{ },thentheresultisthefollowingrewritingofourcriterion:

2 I{HMxx}I{[Syx/M ]+I{[Syx/M ]} dt (11.62)xx xx

Since the termI{HMxx} is causal (i.e., zero for negative time), the best we cando with it, as far as minimizing this integral is concerned, is to cancel out all of

/M Inotherwords,ourbestchoice isI{[Syx xx]+}.= [Syx/M ]+ , (11.63)HMxx xx

or1 Syx(j)

H(j) = . (11.64)Mxx(j) Mxx(j) +

NotethatthestabilityandcausalityoftheinverseofMxx guaranteethatthislaststeppreservesstabilityandcausality,respectively,ofthesolution.Theexpressionin(11.64)isthesolutionoftheWienerfilteringproblemunderthecausality constraint. It is also evident now that the MMSE is larger than in theunconstrained(noncausal)casebytheamount

2MMSE= 1

Syx

d. (11.65)2 Mxx

EXAMPLE11.5 DTPredictionAlthoughtheprecedingresultsweredeveloped fortheCTcase,exactlyanalogousexpressions with obvious modifications (namely, using the DTFT instead of theAlanV.OppenheimandGeorgeC.Verghese,2010c

7/29/2019 MIT6_011S10_chap11

15/17

Section11.3 CausalWienerFiltering 209CTFT,with integralsfrom to ratherthanto,etc.) applytotheDTcase.Consideraprocessx[n]thatistheresultofpassing(zeromean)whitenoiseofunitvariancethrougha(modeling)filterwithfrequencyresponse

Mxx(ej) =0 +1ej , (11.66)

where both 0 and 1 are assumed nonzero. This filter is stable and causal, andif 1 < 0 then the inverse is stable and causal too. We assume this condition| | | |holds. (If itdoesnt, we canalways findanother modelingfilter forwhich itdoes,bymultiplyingthepresentfilterbyanappropriateallpassfilter.)Supposewewanttodocausalonesteppredictionforthisprocess,soy[n] =x[n+1].ThenRyx[m] =Rxx[m+1],so

Syx =ejSxx =ejMxxM . (11.67)xxThus Syx

= [ejMxx]+ =1 , (11.68)M +xx

andsotheoptimumfilter,accordingto(11.64),hasfrequencyresponseH(ej) = 1 . (11.69)

0 +1ejThe associatedMMSE isevaluatedbytheexpression in(11.65), and turnsouttobe simply 20 (which can be compared with thevalueof20 +12 that would havebeenobtainedifweestimatedx[n+1]byjust itsmeanvalue,namelyzero).

11.3.1 DealingwithNonzeroMeansWehaveso farconsideredthecasewherebothxandy havezeromeans(andthepractical consequence has been that we havent had to worry about their PSDshavingimpulsesattheorigin). Iftheirmeansarenonzero,thenwecandoabetter

jobofestimatingy(t)ifweallowourselvestoadjusttheestimatesproducedbytheLTI system, by adding appropriate constants (to make anaffine estimator). Forthis,wecanfirstconsidertheproblemofestimatingyy fromxx,illustratedinFigure11.7

Estimator y(t)y = estimatex(t)x h(t),H(j)

y(t)y = targetprocessFIGURE11.7 Wienerfilteringwithnonzeromeans.

Denoting the transforms of the covariances Cxx() and Cyx() by Dxx(j) andDyx(j) respectively (these transforms are sometimes referred to as covariancecAlanV.OppenheimandGeorgeC.Verghese,2010

7/29/2019 MIT6_011S10_chap11

16/17

210 Chapter11 WienerFilteringPSDs), the optimal unconstrained Wiener filter for our task will evidently have afrequencyresponsegivenby

Dyx(j)H(j) = . (11.70)

Dxx(j)Wecanthenaddy totheoutputofthisfiltertogetourLMMSEestimateofy(t).

AlanV.OppenheimandGeorgeC.Verghese,2010c

7/29/2019 MIT6_011S10_chap11

17/17

MIT OpenCourseWarehttp://ocw.mit.edu

6.011 Introduction to Communication, Control, and Signal Processing

Spring 2010

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

http://ocw.mit.edu/http://ocw.mit.edu/termshttp://ocw.mit.edu/termshttp://ocw.mit.edu/