lecture 13 - eigen-analysis cven 302 july 1, 2002

Lecture 13 - Eigen-analysisLecture 13 - Eigen-analysis

CVEN 302

July 1, 2002

Lecture’s GoalsLecture’s Goals

– Shift Method– Inverse Power Method– Accelerated Power Method– QR Factorization– Householder – Hessenberg Method

Shift methodShift method

It is possible to obtain another eigenvalue from the set equations by using a technique known as shifting the matrix.

xxA Subtract the a vector from each side, thereby changing the maximum eigenvalue

xsxIsxA

Shift methodShift method

The eigenvalue, s, is the maximum value of the matrix A. The matrix is rewritten in a form.

IAB max

Use the Power method to obtain the largest eigenvalue of [B].

Example of Shift MethodExample of Shift Method

Consider the follow matrix A

500

120

010

100

010

001

4

100

120

014

B

Assume an arbitrary vector x0 = { 1 1 1}T


Multiply the matrix by the matrix [A] by {x}

5

1

1

1

1

1

500

120

010

Normalize the result of the product

1

6.0

2.0

5-

5

1

1


1

12.0

04.0

5

5

6.0

2.0

5

6.0

2.0

1

2.0

2.0

500

120

010

Continue with the iteration and the final value is = -5. However, to get the true you need to shift back by:

145max

Inverse Power MethodInverse Power Method

The inverse method is similar to the power method, except that it finds the smallest eigenvalue. Using the following technique.

xxA xAxAA 11

xAx 11

xBx

Inverse Power MethodInverse Power Method

The algorithm is the same as the Power method and the “eigenvector” is not the eigenvector for the smallest eigenvalue. To obtain the smallest eigenvalue from the power method.

1

1

Inverse Power MethodInverse Power MethodThe inverse algorithm use the technique avoids calculating the inverse matrix and uses a LU decomposition to find the {x} vector.

xxA xxUL 1

ExampleExample

512

131

024

A

The matrix is defined as:

82.1

2535.5

9264.4

Matlab ProgramMatlab Program

There are set of programs Power and InversePower.

The InversePower(A, x0,iter,tol) does the inverse method.

Accelerated Power MethodAccelerated Power MethodThe Power method can be accelerated by using the Rayleigh Quotient instead of the largest wk value.

The Rayleigh Quotient is defined as:

11 zA

zz

wz

'

'1

Accelerated Power MethodAccelerated Power MethodThe values of the next z term is defined as:

The Power method is adapted to use the new value.

12

wz

Example of Accelerated Power Example of Accelerated Power MethodMethod

Consider the follow matrix A

100

120

014

A

Assume an arbitrary vector x0 = { 1 1 1}T



1

3

5

1

1

1

100

120

014

333.23

7

1

1

1

111

1

3

5

111

1

4286.0

2857.1

1429.2

1

12

wz



4286.0

1429.2

8571.9

4286.0

2857.1

1429.2

100

120

014

6857.3

429.0

2857.1

143.2

429.0286.1143.2

429.0

143.2

857.9

429.02857.1142.2

2

1163.0

5814.0

6744.2

2

23

wz


1163.0

2791.1

2791.11

1163.0

5814.0

6744.2

100

120

014

1171.43

0282.0

3107.0

7396.2

3

34

wz


0282.0

5931.0

2689.11

0282.0

3107.0

7396.2

100

120

014

0849.44

0069.0

1452.0

7587.2

4

45

wz

And so on ...

QR FactorizationQR Factorization

The technique can be used to find the eigenvalue using a successive iteration using Householder transformation to find an equivalent matrix to [A] having an eigenvalues on the diagonal


Another form of factorization

A = Q*RProduces an orthogonal matrix (“Q”) and a right upper triangular matrix (“R”)

Orthogonal matrix - inverse is transpose

T1 QQ

Why do we care?

We can use Q and R to find eigenvalues

1. Get Q and R (A = Q*R)2. Let A = R*Q3. Diagonal elements of A are eigenvalue approximations 4. Iterate until converged


Note: QR eigenvalue method gives all eigenvalues simultaneously, not just the dominant

In practice, QR factorization on any given matrix requires a number of steps

First transform A into Hessenberg form

xx

xxx

xxxx

xxxxx

xxxxxx

xxxxxxx

xxxxxxxx

xxxxxxxx

M

Hessenberg matrix - upper triangular plus first sub-diagonal

Special properties of Hessenberg matrix make it easier to find Q, R, and eigenvalues

QR Eigenvalue MethodQR Eigenvalue Method


• Construction of QR Factorization

matrixngular upper tria:R

; orthogonal:Q

1 IQQQQ

QRATT


• Use Householder reflections and given rotations to reduce certain elements of a vector to zero.

• Use QR factorization that preserve the eigenvalues.• The eigenvalues of the transformed matrix are much

easier to obtain.

Jordan Canonical FormJordan Canonical Form• Any square matrix is orthogonally similar to a

triangular matrix with the eigenvalues on the diagonal

matrix nn an

1

1

1

J

J

n

1n

2

1

r

1

Similarity TransformationSimilarity Transformation• Transformation of the matrix A of the form H-1AH is known as

similarity transformation.• A real matrix Q is orthogonal if QTQ = I. • If Q is orthogonal, then A and Q -1AQ are said to be orthogonally

similar

• The eigenvalues are preserved under the similarity transformation.xQAxQxAQQQ

xxAQQAx

then ,xAx If

1111

1

Upper Triangular MatrixUpper Triangular Matrix• The diagonal elements Rii of the upper triangular matrix

R are the eigenvalues

0rrrr

r000

rr00

rrr0

rrrr

IAIADet

r000

rr00

rrr0

rrrr

R

nn332211

nn

n333

n22322

n1131211

nn

n333

n22322

n1131211

)())()((

iii r

Householder ReflectorHouseholder Reflector• Householder reflector is a matrix of the form

• It is straightforward to verify that Q is symmetric and orthogonal

1

22

2

www

wwIQT

T

Iwwww4ww4IQQ

Qww2Iww2IQTTTT

TTTT

Householder MatrixHouseholder Matrix• Householder matrix reduces zk+1 ,…,zn to zero

• To achieve the above operation, v must be a linear combination of x and ek

00yyyHxy

xxxxxx

v

v w;ww2IH

k21

n1kk21

2

n1kk1k21k

Tk

xxxxxxexv

001000e

,,,,,,,

....,,,,...,,

Householder TransformationHouseholder Transformation

k

2

2

2nkk

22

21

2k

2

2

2n

2k

22

21

kn1kk21

n1kk21

xxxx)x(xxxv

x2xx)x(xxvv

e x xxαxxxv

xxxxxx

vvv

xv2xx

vv

vv2IHx

00002

12

222 choose weIf 2

2

2

2

2

22

αvxvvv

xvxHx

xxx

xx

vv

xvx

k

k

αxx) sign(xα k2k ofon cancellati avoid to choose

y

x

x

x

x

x

x

x

HxHk

n

k

k

kkk

0

0

1

1

1

1

)()(

Householder matrixHouseholder matrix• Corollary (kth Householder matrix): Let A be an

nxn matrix and x any vector. If k is an integer with 1< k<n-1 we can construct a vector w(k) and matrix H(k) = I - 2w(k)w’(k) so that

Householder matrixHouseholder matrix• Define the value so that

• The vector w is found by

• Choose = sign(xk)g to reduce round-off error

22yx

22

2

221k

22

21

2n

21k

2k

21k

22

21

2

2

2n

21k

2k

2

gyxxx

xx xxxxx

xxx

2

1k2n

21k

2k

2k

2n

21k

2k

2

n1kk

2

2x 2xxxx 2

xxxs

xx)x(00s

1

yx

yxw

1ww

Householder MatricesHouseholder Matrices

n,,1ki ,s/xw

;/sgxsignx/sxw set:4Step

xgg2x2g2s

gx2gx2g

xxxs compute:3 Step

g xsign

xxxg compute:2 Step

0www set:1 Step

ii

kkkk

kk2

2k

22k

2

2n

21k

2k

2

k

2n

21k

2k

1k21

1www ;

s

x,,

s

x,

s

gxsignx,0,,0w

2

2

n1kkk

Example: Householder MatrixExample: Householder Matrix

2/31/32/3

1/32/32/3

2/32/31/3

224

2

2

4

24

1

100

010

001

ww2IH

22424

12231

24

1ww

24)13)(3(2)xg(g2s

3gxsign ;3221xxxg

2

2

1

1:,Ax ;

212

122

121

A

)1(

)1(

1

12222

322

21

Example: Householder MatrixExample: Householder Matrix

1240.09923.00

9923.01240.00

001

6618.07497.00

6618.0

7497.0

0

100

010

001

ww2IH

6618.07497.003

8

3

6510

s

1ww

0294.4)xg(g2s

/3;65g xsign /3;658/3)((1/3)g

7/38/30

2/31/30

1/34/33

212

122

121

2/31/32/3

1/32/32/3

2/32/31/3

AH

)2(

)2(

2

222

)1(

Basic QR FactorizationBasic QR Factorization• [A] = [Q] [R]• [Q] is orthogonal, QTQ = I• [R] is upper triangular• QR factorization using Householder matrices• Q = H(1)H(2)….H(n-1)

end

RHRDefine

zero toreduce R in

n,,1k positions ithw H Find

1n:1k for

AR Define

)1k()k((k)

1)-(k

)k(

)0(

1)-(n

(k)

RR Define

end

QHQ

1:-1:1-nk for

IQ Define

Example: QR FactorizationExample: QR Factorization

AAHH HH QR

RR

HHQ

AHHRHR

'ww2IH

AHRHR

ww2IH

212

122

121

AR

)1()2()2()1(

)2(

)2()1(

)1()2()1()2()2(

)2(

)1()0()1()1(

)1(

)0(

• Similarity transformation B = QTAQ preserve the eigenvalues


2481.07029.06667.0

6202.04134.06667.0

7442.05788.03333.0

1240.09923.00

9923.01240.00

001

2/31/32/3

1/32/32/3

2/32/31/3

HHQ

3721.000

3980.26874.20

3333.03333.13

AHHR

)2()1(

)1()2(

AB

QR = A

Finding Eigenvalues Using Finding Eigenvalues Using QR FactorizationQR Factorization

• Generate a sequence A(m) that are orthogonally similar to A

• Use Householder transformation H-1AH

• the iterates converge to an upper triangular matrix with the eigenvalues on the diagonal

)k()k()1k(

)k()k((k)

QRA

RQA general In

Find all eigenvalues simultaneously!

QR Eigenvalue MethodQR Eigenvalue Method

• QR factorization: A = QR

• Similarity transformation: A(new) = RQ

)3()3()4(

)3()3((3)

)2()2()3(

)2()2((2)

)1()1()2(

)1()1()1(

QRA

RQA

QRA

RQA

QRA

RQA

QAQA

RQA

AQAQR

QRA

)k(T)1k(

)1k(

T1

Example: QR EigenvalueExample: QR Eigenvalue

092302615024810

261527966219290

488410535211112

248107029066670

620204134066670

744205788033330

3721000

39802687420

33330333313

RQA

3721000

39802687420

33330333313

248107029066670

620204134066670

744205788033330

QR

212

122

121

AA

1

0

...

...

...

...

...

...

.

..

..

.

..

..

...

...

...

)(

)(

Example: QR EigenvalueExample: QR Eigenvalue

4178.00038.00017.0

8930.10021.30010.0

3937.18579.14157.2

A

3948.00191.00099.0

9203.19892.20056.0

3930.18691.14056.2

A

5161.01047.00616.0

7694.10527.30310.0

3865.18104.14636.2

A

)4(

)3(

)2(

4142.00000.00000.0

8974.10000.30000.0

3934.18597.14142.2

A

4143.00001.00001.0

8974.10001.30000.0

3934.18596.14143.2

A

4136.00007.00003.0

8982.19996.20002.0

3933.18600.14140.2

A

)7(

)6(

)5(

4142.0 ,0000.3 ,4142.2 321

» A=[1 2 -1; 2 2 -1; 2 -1 2]A = 1 2 -1 2 2 -1 2 -1 2

» [Q,R]=QR_factor(A)Q = -0.3333 -0.5788 -0.7442 -0.6667 -0.4134 0.6202 -0.6667 0.7029 -0.2481R = -3.0000 -1.3333 -0.3333 0.0000 -2.6874 2.3980 0.0000 0.0000 -0.3721

» e=QR_eig(A,6);A = 2.1111 2.0535 1.4884 0.1929 2.7966 -2.2615 0.2481 -0.2615 0.0923

A = 2.4634 1.8104 -1.3865 -0.0310 3.0527 1.7694 0.0616 -0.1047 -0.5161A = 2.4056 1.8691 1.3930 0.0056 2.9892 -1.9203 0.0099 -0.0191 -0.3948A = 2.4157 1.8579 -1.3937 -0.0010 3.0021 1.8930 0.0017 -0.0038 -0.4178A = 2.4140 1.8600 1.3933 0.0002 2.9996 -1.8982 0.0003 -0.0007 -0.4136A = 2.4143 1.8596 -1.3934 0.0000 3.0001 1.8972 0.0001 -0.0001 -0.4143

e = 2.4143 3.0001 -0.4143

MATLAB Example

QR factorization

eigenvalue

Improved QR MethodImproved QR Method

• Using similarity transformation to form an upper Hessenberg Matrix (upper triangular matrix & one nonzero band below diagonal) .

• More efficient to form Hessenberg matrix without explicitly forming the Householder matrices (not given in textbook).

function A = Hessenberg(A)[n,nn] = size(A);for k = 1:n-2 H = Householder(A(:,k),k+1); A = H*A*H;end

» A=[1 2 -1; 2 2 -1; 2 -1 2]A = 1 2 -1 2 2 -1 2 -1 2» [Q,R]=QR_factor_g(A)Q = 0.4472 0.5963 -0.6667 0.8944 -0.2981 0.3333 0 -0.7454 -0.6667R = 2.2361 2.6833 -1.3416 -1.4907 1.3416 -1.7889 -1.3333 0 -1.0000» e=QR_eig_g(A,6);A = 2.1111 -2.4356 0.7071 -0.3143 -0.1111 -2.0000 0 0.0000 3.0000A = 2.4634 2.0523 -0.9939 -0.0690 -0.4634 -1.8741 0.0000 0.0000 3.0000

Improved QR MethodImproved QR Method A = 2.4056 -2.1327 0.9410 -0.0114 -0.4056 -1.9012 0.0000 0.0000 3.0000A = 2.4157 2.1194 -0.9500 -0.0020 -0.4157 -1.8967 0.0000 0.0000 3.0000A = 2.4140 -2.1217 0.9485 -0.0003 -0.4140 -1.8975 0.0000 0.0000 3.0000A = 2.4143 2.1213 -0.9487 -0.0001 -0.4143 -1.8973 0.0000 0.0000 3.0000e = 2.4143 -0.4143 3.0000» eig(A)ans = 2.4142 -0.4142 3.0000

Hessenberg matrix

MATLAB function

eigenvalue

SummarySummary

• Single value eigen-analysis– Power Method– Shifting technique– Inverse Power Method

• QR Factorization– Householder matrix– Hessenberg matrix

HomeworkHomework

• Check the Homework webpage

lecture 13 - eigen-analysis cven 302 july 1, 2002

Documents

inverse method

inverse matrix

qr eigenvalue method

example of shift method

diagonal slide

product slide

dominant slide

maximum eigenvalue slide