agc dsp agc dsp professor a g constantinides©1 adaptive signal processing problem: equalise through...

Professor A G

Constantinides© 1

AGC

DSP

AGC

DSP

Adaptive Signal Processing Problem: Equalise through a FIR filter the

distorting effect of a communication channel that may be changing with time.

If the channel were fixed then a possible solution could be based on the Wiener filter approach

We need to know in such case the correlation matrix of the transmitted signal and the cross correlation vector between the input and desired response.

When the the filter is operating in an unknown environment these required quantities need to be found from the accumulated data.

Professor A G

Constantinides© 2

AGC

DSP

AGC

DSP

Adaptive Signal Processing The problem is particularly acute when

not only the environment is changing but also the data involved are non-stationary

In such cases we need temporally to follow the behaviour of the signals, and adapt the correlation parameters as the environment is changing.

This would essentially produce a temporally adaptive filter.

Professor A G

Constantinides© 3

AGC

DSP

AGC

DSP

Adaptive Signal Processing

A possible framework is:

][nd][ˆ nd]}[{ nx

][new:Filter

Adaptive

Algorithm

Professor A G

Constantinides© 4

AGC

DSP

AGC

DSP

Adaptive Signal Processing Applications are many

Digital Communications Channel Equalisation Adaptive noise cancellation Adaptive echo cancellation System identification Smart antenna systems Blind system equalisation And many, many others

Professor A G

Constantinides© 5

AGC

DSP

AGC

DSP

Applications

Professor A G

Constantinides© 6

AGC

DSP

AGC

DSP


Echo Cancellers in Local Loops

-

+

-

+Rx1 Rx2

Tx1 Rx2

Echo canceller Echo canceller

Adaptive Algorithm Adaptive Algorithm

Hybrid Hybrid

Local Loop

Professor A G

Constantinides© 7

AGC

DSP

AGC

DSP


Adaptive Noise Canceller

Noise

Signal +Noise

-

+

FIR filter

Adaptive Algorithm

PRIMARY SIGNAL

REFERENCE SIGNAL

Professor A G

Constantinides© 8

AGC

DSP

AGC

DSP


System Identification

Unknown System

Signal

-

+

FIR filter

Adaptive Algorithm

Professor A G

Constantinides© 9

AGC

DSP

AGC

DSP


System Equalisation

Unknown System

Signal

-

+

FIR filter

Adaptive Algorithm

Delay

Professor A G

Constantinides© 10

AGC

DSP

AGC

DSP


Adaptive Predictors

Signal

-

+

FIR filter

Adaptive Algorithm

Delay

Professor A G

Constantinides© 11

AGC

DSP

AGC

DSP


Adaptive Arrays

Linear Combiner

Interference

Professor A G

Constantinides© 12

AGC

DSP

AGC

DSP

Adaptive Signal Processing Basic principles: 1) Form an objective function

(performance criterion) 2) Find gradient of objective function

with respect to FIR filter weights 3) There are several different

approaches that can be used at this point

3) Form a differential/difference equation from the gradient.

Professor A G

Constantinides© 13

AGC

DSP

AGC

DSP

Adaptive Signal Processing Let the desired signal be The input signal The output Now form the vectors

So that

][nd][nx

][ny

Tmnxnxnxn ]1[.]1[][][ x

Tmhhh ]1[.]1[]0[ h

hx Tnny ][][

Professor A G

Constantinides© 14

AGC

DSP

AGC

DSP


The form the objective function

where

}][][{)( 2nyndEJ w

Rhhphhpw TTTdJ 2)(

}][][{ TnnE xxR

]}[][{ ndnE xp

Professor A G

Constantinides© 15

AGC

DSP

AGC

DSP


We wish to minimise this function at the instant n

Using Steepest Descent we write

But

][])[(

21

][]1[nnJ

nnhh

hh

Rhphh

22)(

J

Professor A G

Constantinides© 16

AGC

DSP

AGC

DSP

Adaptive Signal Processing So that the “weights update equation”

Since the objective function is quadratic this expression will converge in m steps

The equation is not practical If we knew and a priori we could

find the required solution (Wiener) as

])[(][]1[ nnn Rhphh

pR

pRh 1opt

Professor A G

Constantinides© 17

AGC

DSP

AGC

DSP

Adaptive Signal Processing However these matrices are not known Approximate expressions are obtained

by ignoring the expectations in the earlier complete forms

This is very crude. However, because the update equation accumulates such quantities, progressive we expect the crude form to improve

Tnnn ][][][ˆ xxR ][][][ˆ ndnn xp

Professor A G

Constantinides© 18

AGC

DSP

AGC

DSP

The LMS Algorithm Thus we have

Where the error is

And hence can write

This is sometimes called the stochastic gradient descent

])[][][]([][]1[ nnndnnn T hxxhh

])[][(])[][][(][ nyndnnndne T hx

][][][]1[ nennn xhh

Professor A G

Constantinides© 19

AGC

DSP

AGC

DSP

Convergence

The parameter is the step size, and it should be selected carefully

If too small it takes too long to converge, if too large it can lead to instability

Write the autocorrelation matrix in the eigen factorisation form

ΛQQR T

Professor A G

Constantinides© 20

AGC

DSP

AGC

DSP

Convergence Where is orthogonal and is

diagonal containing the eigenvalues The error in the weights with respect

to their optimal values is given by (using the Wiener solution for

We obtain

Q Λ

])[(][]1[ nnn optoptopt RhRhhhhh

p

][][]1[ nnn hhh Reee

Professor A G

Constantinides© 21

AGC

DSP

AGC

DSP

Convergence Or equivalently

I.e.

Thus we have

Form a new variable

][)1(]1[ nn hh eΛQQe T

][)(

][)1(]1[

n

nn

h

hh

eΛQQQQ

eΛQQQQeT

T

][)1(]1[ nn hh QeΛQe

][][ nn hQev

Professor A G

Constantinides© 22

AGC

DSP

AGC

DSP

Convergence So that

Thus each element of this new variable is dependent on the previous value of it via a scaling constant

The equation will therefore have an exponential form in the time domain, and the largest coefficient in the right hand side will dominate

][)1(]1[ nn vΛv

Professor A G

Constantinides© 23

AGC

DSP

AGC

DSP

Convergence

We require that

Or

In practice we take a much smaller value than this

11 max

max

20

Professor A G

Constantinides© 24

AGC

DSP

AGC

DSP

Estimates Then it can be seen that as the

weight update equation yields And on taking expectations of both

sides of it we have

Or

n

]}[{]}1[{ nEnE hh

])}[][][]([{]}[{]}1[{ nnndnEnEnE T hxxhh

])}[][][][][{(0 nnnndnE T hxxx

Professor A G

Constantinides© 25

AGC

DSP

AGC

DSP

Limiting forms

This indicates that the solution ultimately tends to the Wiener form

I.e. the estimate is unbiased

Professor A G

Constantinides© 26

AGC

DSP

AGC

DSP

Misadjustment The excess mean square error in the

objective function due to gradient noise Assume uncorrelatedness set

Where is the variance of desired response and is zero when uncorrelated.

Then misadjustment is defined as

optT

dJ hp 2min

2d

opth

minmin /))(( JJJJ LMSXS

Professor A G

Constantinides© 27

AGC

DSP

AGC

DSP

Misadjustment

It can be shown that the misadjustment is given by

m

i i

iXS JJ

1min 1

/

Professor A G

Constantinides© 28

AGC

DSP

AGC

DSP

Normalised LMS To make the step size respond to

the signal needs

In this case

And misadjustment is proportional to the step size.

][][][1

2][]1[ 2 nen

nnn x

xhh

10

Professor A G

Constantinides© 29

AGC

DSP

AGC

DSP

Transform based LMS

][nd][ˆ nd]}[{ nx

][new:Filter

Adaptive

Algorithm

Transform

Inverse Transform

Professor A G

Constantinides© 30

AGC

DSP

AGC

DSP

Least Squares Adaptive with

We have the Least Squares solution

However, this is computationally very intensive to implement.

Alternative forms make use of recursive estimates of the matrices involved.

n

i

Tiin1

][][][ xxR

n

indnn

1][][][ xp

][][][ 1 nnn pRh

Professor A G

Constantinides© 31

AGC

DSP

AGC

DSP

Recursive Least Squares Firstly we note that

We now use the Inversion Lemma (or the Sherman-Morrison formula)

Let

][][]1[][ ndnnn xpp

Tnnnn ][][]1[][ xxRR

Professor A G

Constantinides© 32

AGC

DSP

AGC

DSP

Recursive Least Squares (RLS)

Let

Then

The quantity is known as the Kalman gain

][]1[][1][]1[

][ 1

1

nnnnn

n T xRxxR

k

1][][ nn RP

]1[][][]1[][ nnnnn T PxkRP

][nk

Professor A G

Constantinides© 33

AGC

DSP

AGC

DSP

Recursive Least Squares Now use in the computation

of the filter weights

From the earlier expression for updates we have

And hence

][][][ nnn xPk

])[][]1[]([][][][ ndnnnnnn xpPpPh ][nP

]1[]1[][][]1[]1[]1[][ nnnnnnnn T pPxkpPpP

])1[][][]([]1[][ nnndnnn T hxkhh

Professor A G

Constantinides© 34

AGC

DSP

AGC

DSP

Kalman Filters Kalman filter is a sequential estimation

problem normally derived from The Bayes approach The Innovations approach

Essentially they lead to the same equations as RLS, but underlying assumptions are different

Professor A G

Constantinides© 35

AGC

DSP

AGC

DSP

Kalman Filters The problem is normally stated as:

Given a sequence of noisy observations to estimate the sequence of state vectors of a linear system driven by noise.

Standard formulation][][]1[ nnn wAxx ][][][][ nnnn νxCy

Professor A G

Constantinides© 36

AGC

DSP

AGC

DSP

Kalman Filters Kalman filters may be seen as RLS with

the following correspondenceSate space

RLS Sate-Update matrix Sate-noise variance Observation matrix Observations State estimate

][nA

][nx

Tn][x][nC][ny

][nh

}][][{][ TnnEn wwQ I0

][nd

Professor A G

Constantinides© 37

AGC

DSP

AGC

DSP

Cholesky Factorisation In situations where storage and to some

extend computational demand is at a premium one can use the Cholesky factorisation tecchnique for a positive definite matrix

Express , where is lower triangular

There are many techniques for determining the factorisation

TLLR L

agc dsp agc dsp professor a g constantinides©1 adaptive signal processing problem: equalise through...

Documents