introduction to subspace tracking ieee aes dallas chapter meeting richardson, tx feb 28, 2006 scott...

Introduction to Subspace Tracking

IEEE AES Dallas Chapter MeetingRichardson, TX

Feb 28, 2006

Scott BakerL-3 Communications

Greenville, TX [email protected]

Outline

Introduction Time Series Data Model Subspaces & the Subspace Tracking Idea Subspace Tracking Categories A Secular Equation Subspace Tracker A Plane Rotation Subspace Tracker Simulations

Data Model

The time-series model for a sum of complex sinusoids in noise may be written

Q

qne

nqjeqsnx1

)()(

This time series may be placed in the vector form

)()(

)(

)2(

)1(

)( n

nx

x

x

n esωAx

And in a matrix form

)()(2)(1

)1()1(2)1(1

)(nQjenjenje

Qjejeje

ωA

Type?Singular?

Type?

NMNMN xxxX ,,,1 Size?Looks like?

Data Model

Two Typical Goals in Time Series Estimation 1)Parameter estimation

Number of signals Q The frequencies themselves The signal amplitudes

2)Noise reduction Find a cleaner version of x(n)

Typical assumptions Signal model is stationary over a block Noise is pre-whitened as needed N >= M (more snapshots than sensors)

Data Model

The data matrix is often place in form of a correlation matrix Data matrix forms lead to an SVD Correlation matrix forms lead to an EVD

HVΣUX HH VΛVXXR

where

MHH

NHH

IVVVV

IUUUU

What is U?

What is V?

Data Model

The singular value decomposition can be seen in more detail as

HM

H

H

M

NH

v

v

v

0

uuuVΣUX 2

1

2

1

21

N x N

N x M M x M

ui are unitary left singular vectorsvi are unitary right singular vectorsi are singular values

Data Model

The eigenvalue decomposition can be seen in more detail as

M x M

M x MM x Mvi are unitary right eigenvectors

i are eigenvalues

HM

H

H

M

MH

v

v

v

vvvVΛVR 2

1

2

1

21

How is the SVD and EVD related? How are the singular values and eigenvalues related?How are the singular vectors and eigenvectors related?How can we interpret an eigenvalue and an eigenvector?

Data Model

0 10 20 30 40 50 60 70 80 90 100-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 25 30 35 40-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

M = 40;x = sin(2*pi*1 * [1:100]/22); % 1 Hz sinewave, figure; plot(x)X = datamtrx(x, M);R = X'*X;[V, L] = svd(R);figure; plot(V(:,1));hold on; plot(V(:,2))

>> disp(L(1:3,1:3)) 693.4222 0 0 0 531.3492 0 0 0 0.0000

Figure 1 - x Figure 2 – V (1st two)

Box 1

Box 2N=61Period = 22 samples

0 5 10 15 20 25 30 35 40-1

-0.5

0

0.5

Figure 3 – V (last 38)

Why not one?

Subspaces & Tracking

From this example, consider a partition of the EVD

HN

H

N

SN

H

V

VΛ

ΛVVVΛVR S S

Signal Eigenvectors

NoiseEigenvectors

SignalEigenvalues “large”

NoiseEigenvalues “small”Span?

VS and A ?


Extend our previous example, by adding noise

0 10 20 30 40 50 60 70 80 90 100-3

-2

-1

0

1

2

3

M = 40;x = sin(2*pi*1 * [1:100]/22); % 1 Hz sinewave, x = x + randn(1,100);figure; plot(x)X = datamtrx(x, M);R = X'*X;[V, L] = svd(R);figure; plot(V(:,1));hold on; plot(V(:,2))

Box 1

Only onechange

0 5 10 15 20 25 30 35 40-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0 5 10 15 20 25 30 35 400

100

200

300

400

500

600

700

Figure 1 - x Figure 2 – V (1st two) Figure 3 – Eigenvalues

Effect?

Meaning?

Knowledge of the signal eigenvectors VS gives us knowledge of the frequency matrix A

Knowing the span of VS is equivalent to knowing the span of VN

One can show that VS VSH + VNVN

H = I

MUSIC The MUSIC algorithm uses VN to estimate the

frequencies It is possible to show VN

H A = 0

Which the same as VNH [a1,a2,…,aQ] = 0


Recall that the frequency matrix had the form

)()(2)(1

)1()1(2)1(1

)(nQjenjenje

Qjejeje

ωA


So, we know the structure of a column and if we had the qth frequency, we could construct the qth column

We do know that VNH aq = 0 and we know VN from the

decomposition of the data To find q, what can we do?

Yup, good guess! We only need to build a sweep vector a()

for from DC to ½ the sample frequency We can plot the inverse instead so that

peaks are at the Q frequencies S() = 1 / || VN

H a()||

S() is then the frequency spectrum with peaks at the Q frequencies

Way cool!



So how do we do this again? Step 1: Build a data matrix

NMNMN xxxX ,,,1 Step 2: Build a correlation matrix and find its EVD

HH VΛVXXR Step 3: Sort the eigenvalues largest to smallest and partition the

EVD into signal portion & noise portion

NVVV S

S() = 1 / || VNH a()||

Step 4: Compute the MUSIC spectrum & locate Q peaks


randn('state',1);

n=[0:99];

x=exp(i*pi/2*n)+2*exp(i*pi/4*n)+exp(i*pi/3*n)+randn(1,100);

R=corrmtx(x,12,'mod');

pmusic(R,3,'whole')

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8-30

-20

-10

0

10

20

30

40

50

60

Normalized Frequency ( rad/sample)

Pow

er (dB

)

Pseudospectrum Estimate via MUSIC

S()

MUSIC Spectrum

(type ‘help pmusic’)

So are we done yet? In stationary cases, yes In non-stationary cases, no

The Q frequencies are often time-varying Hopping sigals Chirping signals Any kind of frequency modulated signal

It is too expensive to compute an EVD at every instant of time

Instead, use a subspace tracker to track the changes in VS (or equivalently VN) as new data comes in


Let’s slightly modify our data model so that it is recursive


HN

H

N

SN

V

VΛ

ΛVVR S S

HxxRR 1~

The factor is called a fading factor and is slightly less than unity

The effective window size is B = 1 / (1-) The update assumes that we know the old EVD

What type of update is this called?

HN

H

N

SN

V

V

Λ

ΛVVR ~

S~

~

~

~S

~~

And we wish to know the new EVD Do we needthe wholeEVD?

Subspace Tracking Categories

There are about 250 published papers on subspace tracking It sky-rocketed after the invention of the MUSIC algorithm There are several broad categories

Secular equation (e.g., Bunch) Plane rotation (e.g., Businger) Optimization Lanczos Perturbation Power Method & Subspace Iteration Eigenvector computation with a priori eigenvalues Analog or Continuous Time Hybrid Techniques

The Secular Equation Subspace Tracker

Golub discovered the secular equation in 1971

Can be considered the first eigen- based subspace tracker (Businger wrote the first overall ST in 1970)

Reveals a nice relation between original and updated eigenvalues

Seven years later, Bunch, Nielsen and Sorensen introduced the classic secular equation subspace tracker

A keen observation is that the eigenvalues of the new correlation are larger assuming the data has non-zero energy

MM ~

22~

11~

Gene Golub (Stanford)

updated eigenvalue, largest

original eigenvalue, largest


This observation is important because it was noted that the rank-1 modification of the correlation can be written as the rank-1 modification of a diagonal matrix

HxxRR 1~

HH xxVΛVR 1 ~

Stick in the EVD

HHH VVxxVΛVR ]1 [~

Because V is?

HH VzzΛVR ]1 [~

And z is?

What is left in the brackets then?Recall the monotonicity relation


HVDVR ~

HHVQΒVQR ~

HzzΛD 1 where

Is D a diagonal matrix?Note that we can form the EVD

HQΒQD And so the resultant EVD of the updated correlation is

Note we have the form

V~ HVΛVR

~

~

~~

Remember the EVD…

Λ~

related? ~

and of seigenvalue theare how So RD


HzzΛD 1

Answer: They are equal

Hence, to find the updated eigenvalues, we can find the eigenvalue of D.

Recall from basic linear algebra, we can write

bDb

These two expressions can be used to find a formula for the updated eigenvalues

0)det( bDb

Combining these three equations leads to the secular equation


M

i i

iz

1 )(

2)1(1)(

1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6-5

-4

-3

-2

-1

0

1

2

3

4

5

Original eigenvalue (p)

Updated eigenvalue (z)

The secular equation has zeros & poles at the eigenvalue locations

)(

Efficient to curve fithere between poles

EigenvectorComputation(i.i. not needed)IDD

zD

zVDv

ii

i

ii

~

21

1~

A Plane Rot. Subspace Tracker

The first subspace tracker developed in 1970 P. Businger of Bell Labs Used plane rotations Impetus was updating a least squares

problem SVD based solution Businger acknowledged the problem was

posed by G. Dahlquist and related by G. Golub

How might we proceed to update an SVD?


HH

HHH

H

HH

H

H

H

H

VZQI0

0U

VZZVx

ΣQQ

I0

0U

VVx

Σ

I0

0U

x

VΣU

x

XX

VΣUX

~

)1(

)1(

)1(

)1(

~

What is Q? Z?

Is this in SVD form?

What structure does thisfactor have?

Adding a row

Similar trickto what?

The Q and Z matrices are called plane rotation matrices

The specific type of plane rotation used here is called a Given’s rotation


0

* r

b

a

cs

sc

222 |||||| bar

0 cbsa 122 sc

22

2

ba

ac

c

a

bs

Real Given’s Rotation

So how do we use Given’s rotations to diagonalize the matrix? We have to be careful Eliminating one element in a column / row may perturb

another column / row Need to apply Given’s rotations to both rows and columns


***b

a 0 x0

xrx

xQ =

Given’sMatrix

OriginalMatrix Transformed

Matrix

*

****

**

*

Q1=

*

**0 **

*

*

x

*

**

*x

0 0 0 0

xx=

*

**

*

*Q3

0 0 *


Left(Row)Given’sRotationsFirst

What is theresultantform?

A Plane Rot. Subspace Tracker The bidiagonal form is reduced to a diagonal form The process is iterative

Start with column rotations (right) Then a set of row rotations (left) Repeat until the energy is minimal off the main diagonal

HHVZQI0

0UX

~~

So, going back to our SVD update

Q and Z are series of Given’s rotation matrices Q1Q2...QL Z1Z2….ZP

And so our final result then is in the proper SVD form

HVΣUX~

~

~~


Example Simulation

Single signal SNR is 15 dB Sample Rate = 640MHz Linear FM signal

140 to 160 MHz 20MHz / 1000 samples 12.8 GHz / second

= 0.975 (80 samples) TQR + root-MUSIC Spline Interpolation

Polynomial = feature

500 1000 1500 2000 2500 3000 3500 4000 4500 5000

1.42

1.44

1.46

1.48

1.5

1.52

1.54

1.56

1.58

1.6

1.62x 10

8 Freq Track w/ MTH=5, SNR=15, EBS = 40

freq

uenc

y

time sample

100 200 300 400 500 600 700 800 900

1.42

1.44

1.46

1.48

1.5

1.52

1.54

1.56

1.58

1.6

x 108

This is a splinefit to the first ramp.

0 100 200 300 400 500 600 700 800 900 1000

1.4

1.45

1.5

1.55

1.6

x 108 Freq Track w/ MTH=5, SNR=15, EBS = 40

frequency

time sample

0 200 400 600 800 1000 1200 1400 1600 1800 2000

1

1.1

1.2

1.3

1.4

1.5

1.6

x 108 Freq Track w/ MTH=5, SNR=15, EBS = 40

frequency

time sample

Other Examples

introduction to subspace tracking ieee aes dallas chapter meeting richardson, tx feb 28, 2006 scott...

Documents

x randn1

asn x nn x

series data modelsubspaces

frequency matrix aknowing

data modelm

data matrixstep

subspaces trackingknowledge

subspaces trackingso