krylov subspace methods for eigenvalue problems · krylov subspace methods for eigenvalue problems...

Krylov subspace methods foreigenvalue problems

David S. Watkins

[email protected]

Department of Mathematics

Washington State University

October 16, 2008 – p.

Problem: Linear Elasticity

October 16, 2008 – p.

Problem: Linear ElasticityElastic Deformation(3D, anisotropic, composite materials)

October 16, 2008 – p.


Singularities at cracks, interfaces

October 16, 2008 – p.



Lamé Equations (spherical coordinates)

October 16, 2008 – p.




Separate radial variable.

October 16, 2008 – p.




Separate radial variable.

Get quadratic eigenvalue problem.

(λ2M + λG+K)v = 0

M∗ = M > 0 G∗ = −G K∗ = K < 0

October 16, 2008 – p.

Linear Elasticity, ContinuedDiscretize θ and ϕ variables.

October 16, 2008 – p.

Linear Elasticity, ContinuedDiscretize θ and ϕ variables.(finite element method)

October 16, 2008 – p.


(λ2M + λG+K)v = 0

MT = M > 0 GT = −G KT = K < 0

matrix quadratic eigenvalue problem(large, sparse)

October 16, 2008 – p.


(λ2M + λG+K)v = 0

MT = M > 0 GT = −G KT = K < 0


Find few smallest eigenvalues (and correspondingeigenvectors).

October 16, 2008 – p.


(λ2M + λG+K)v = 0

MT = M > 0 GT = −G KT = K < 0


Find few smallest eigenvalues (and correspondingeigenvectors).

Respect the structure. (symmetric/skew-symmetric)

October 16, 2008 – p.

Hamiltonian Structure

October 16, 2008 – p.

Reduction to First Order

October 16, 2008 – p.

Reduction to First Orderλ2Mv + λGv +Kv = 0

October 16, 2008 – p.


w = λv,

October 16, 2008 – p.


w = λv, Mw = λMv

October 16, 2008 – p.


w = λv, Mw = λMv[

−K 0

0 −M

][

v

w

]

− λ

[

G M

−M 0

][

v

w

]

= 0

October 16, 2008 – p.


w = λv, Mw = λMv[

−K 0

0 −M

][

v

w

]

− λ

[

G M

−M 0

][

v

w

]

= 0

Ax− λBx = 0

October 16, 2008 – p.


w = λv, Mw = λMv[

−K 0

0 −M

][

v

w

]

− λ

[

G M

−M 0

][

v

w

]

= 0

Ax− λBx = 0

symmetric/skew-symmetric

October 16, 2008 – p.

Reduction to Hamiltonian Matrix

October 16, 2008 – p.

Reduction to Hamiltonian MatrixA− λB (symmetric/skew-symmetric)

October 16, 2008 – p.


B = RTJR

(

J =

[

0 I

−I 0

])

sometimes easy, always possible

October 16, 2008 – p.


B = RTJR

(

J =

[

0 I

−I 0

])


A− λRTJR

October 16, 2008 – p.


B = RTJR

(

J =

[

0 I

−I 0

])


A− λRTJR

R−TAR−1 − λJ

October 16, 2008 – p.


B = RTJR

(

J =

[

0 I

−I 0

])


A− λRTJR

R−TAR−1 − λJ

JTR−TAR−1 − λI

October 16, 2008 – p.


B = RTJR

(

J =

[

0 I

−I 0

])


A− λRTJR

R−TAR−1 − λJ

JTR−TAR−1 − λI

H = JTR−TAR−1 is Hamiltonian.

October 16, 2008 – p.

in our case . . .

October 16, 2008 – p.

in our case . . .B =

[

G M

−M 0

]

October 16, 2008 – p.


[

G M

−M 0

]

B = RTJR =

[

I −1

2G

0 M

][

0 I

−I 0

][

I 01

2G M

]

October 16, 2008 – p.


[

G M

−M 0

]

B = RTJR =

[

I −1

2G

0 M

][

0 I

−I 0

][

I 01

2G M

]

H =

[

I 0

−1

2G I

][

0 M−1

−K 0

][

I 0

−1

2G I

]

October 16, 2008 – p.


[

G M

−M 0

]

B = RTJR =

[

I −1

2G

0 M

][

0 I

−I 0

][

I 01

2G M

]

H =

[

I 0

−1

2G I

][

0 M−1

−K 0

][

I 0

−1

2G I

]

Do not compute H explicitly.

October 16, 2008 – p.


[

G M

−M 0

]

B = RTJR =

[

I −1

2G

0 M

][

0 I

−I 0

][

I 01

2G M

]

H =

[

I 0

−1

2G I

][

0 M−1

−K 0

][

I 0

−1

2G I

]

Do not compute H explicitly. (nor M−1)

October 16, 2008 – p.

Working with H

October 16, 2008 – p.

Working with HKrylov subspace methods

October 16, 2008 – p.


just need to apply the operator:

October 16, 2008 – p.


just need to apply the operator: x 7→ Hx

October 16, 2008 – p.



H =

[

I 0

−1

2G I

][

0 M−1

−K 0

][

I 0

−1

2G I

]

October 16, 2008 – p.



H =

[

I 0

−1

2G I

][

0 M−1

−K 0

][

I 0

−1

2G I

]

H−1 =

[

I 01

2G I

][

0 (−K)−1

M 0

][

I 01

2G I

]

October 16, 2008 – p.

Working with H−1

October 16, 2008 – p.

Working with H−1

H−1 =

[

I 01

2G I

][

0 (−K)−1

M 0

][

I 01

2G I

]

October 16, 2008 – p.

Working with H−1

H−1 =

[

I 01

2G I

][

0 (−K)−1

M 0

][

I 01

2G I

]

x 7→ H−1x

October 16, 2008 – p.

Working with H−1

H−1 =

[

I 01

2G I

][

0 (−K)−1

M 0

][

I 01

2G I

]

x 7→ H−1x

Do not compute (−K)−1

October 16, 2008 – p.

Working with H−1

H−1 =

[

I 01

2G I

][

0 (−K)−1

M 0

][

I 01

2G I

]

x 7→ H−1x


Cholesky decomposition:

October 16, 2008 – p.

Working with H−1

H−1 =

[

I 01

2G I

][

0 (−K)−1

M 0

][

I 01

2G I

]

x 7→ H−1x


Cholesky decomposition: (−K) = RTR

To compute w = −K−1v,

October 16, 2008 – p.

Working with H−1

H−1 =

[

I 01

2G I

][

0 (−K)−1

M 0

][

I 01

2G I

]

x 7→ H−1x



To compute w = −K−1v, Solve (−K)w = v.

October 16, 2008 – p.

Working with H−1

H−1 =

[

I 01

2G I

][

0 (−K)−1

M 0

][

I 01

2G I

]

x 7→ H−1x



To compute w = −K−1v, Solve (−K)w = v.

RTRw = v Backsolve!

October 16, 2008 – p.

K =

October 16, 2008 – p. 10

K =

0 20 40 60 80 100 120 140

0

20

40

60

80

100

120

140

nz = 670

October 16, 2008 – p. 10

R =

October 16, 2008 – p. 11

R =

0 20 40 60 80 100 120 140

0

20

40

60

80

100

120

140

nz = 896

October 16, 2008 – p. 11

K−1=

October 16, 2008 – p. 12

K−1=

0 20 40 60 80 100 120 140

0

20

40

60

80

100

120

140

nz = 21316

October 16, 2008 – p. 12

Second Application

October 16, 2008 – p. 13

Second ApplicationNonlinear Optics

October 16, 2008 – p. 13


Schrödinger eigenvalue problem

October 16, 2008 – p. 13



−~2

2m∇2ψ + V ψ = λψ

October 16, 2008 – p. 13



−~2

2m∇2ψ + V ψ = λψ

Solve numerically (finite elements)

October 16, 2008 – p. 13



−~2

2m∇2ψ + V ψ = λψ


Kv = λMv K = KT > 0, M = MT > 0

October 16, 2008 – p. 13



−~2

2m∇2ψ + V ψ = λψ


Kv = λMv K = KT > 0, M = MT > 0

Matrices are large and sparse.

October 16, 2008 – p. 13

Kv = λMv

October 16, 2008 – p. 14

Kv = λMv

Want few smallest eigenvaluesand associated eigenvectors.

October 16, 2008 – p. 14

Kv = λMv


Invert the problem.

October 16, 2008 – p. 14

Kv = λMv


Invert the problem.

K = RTR

October 16, 2008 – p. 14

Kv = λMv


Invert the problem.

K = RTR RTRv = λMv

October 16, 2008 – p. 14

Kv = λMv


Invert the problem.

K = RTR RTRv = λMv

R−TMR−1(Rv) = λ−1(Rv)

October 16, 2008 – p. 14

Kv = λMv


Invert the problem.

K = RTR RTRv = λMv


A = R−TMR−1, AT = A > 0

October 16, 2008 – p. 14

Kv = λMv


Invert the problem.

K = RTR RTRv = λMv


A = R−TMR−1, AT = A > 0

x 7→ Ax

October 16, 2008 – p. 14

Kv = λMv


Invert the problem.

K = RTR RTRv = λMv


A = R−TMR−1, AT = A > 0

x 7→ Ax backsolve

October 16, 2008 – p. 14

Kv = λMv


Invert the problem.

K = RTR RTRv = λMv


A = R−TMR−1, AT = A > 0

x 7→ Ax backsolve

Do not form A explicitly.

October 16, 2008 – p. 14

What our applications have incommon

October 16, 2008 – p. 15


large, sparse matrices

October 16, 2008 – p. 15



use of matrix factorization (Cholesky decomposition)

October 16, 2008 – p. 15




some kind of structure

October 16, 2008 – p. 15




some kind of structure

. . . not enough time to discuss this

October 16, 2008 – p. 15

Classificationof Eigenvalue Problems

October 16, 2008 – p. 16


small

October 16, 2008 – p. 16


small

medium

October 16, 2008 – p. 16


small

medium

large

October 16, 2008 – p. 16

Small Matrices

October 16, 2008 – p. 17

Small Matricesstore conventionally

October 16, 2008 – p. 17


similarity transformations

October 16, 2008 – p. 17



QR algorithm

October 16, 2008 – p. 17



QR algorithm

get all eigenvalues/vectors

October 16, 2008 – p. 17



QR algorithm

get all eigenvalues/vectors

n ≈ 103

October 16, 2008 – p. 17

Medium Matrices

October 16, 2008 – p. 18

Medium Matricesstore as sparse matrix

October 16, 2008 – p. 18


no similarity transformations

October 16, 2008 – p. 18



matrix factorization okay

October 16, 2008 – p. 18



matrix factorization okay

shift and invert

October 16, 2008 – p. 18

0 20 40 60 80 100 120 140

0

20

40

60

80

100

120

140

nz = 670

October 16, 2008 – p. 19

0 20 40 60 80 100 120 140

0

20

40

60

80

100

120

140

nz = 1815

October 16, 2008 – p. 20

Medium Matrices, Continued

October 16, 2008 – p. 21

Medium Matrices, Continuedstore matrix factor as sparse matrix

October 16, 2008 – p. 21


n ≈ 105

October 16, 2008 – p. 21


n ≈ 105

get selected eigenvalues/vectors

October 16, 2008 – p. 21


n ≈ 105


Krylov subspace methods

October 16, 2008 – p. 21


n ≈ 105



Jacobi-Davidson methods

October 16, 2008 – p. 21

Large Matrices

October 16, 2008 – p. 22

Large Matricesstore as sparse matrix

October 16, 2008 – p. 22



October 16, 2008 – p. 22



no shift-and-invert

October 16, 2008 – p. 22



no shift-and-invert

n ≈ 107

October 16, 2008 – p. 22



no shift-and-invert

n ≈ 107


October 16, 2008 – p. 22



no shift-and-invert

n ≈ 107



October 16, 2008 – p. 22



no shift-and-invert

n ≈ 107



Jacobi-Davidson methods

October 16, 2008 – p. 22

Krylov Subspace Methods

October 16, 2008 – p. 23

Krylov Subspace Methodsx 7→ Ax

October 16, 2008 – p. 23


Example: A = R−TMR−1

October 16, 2008 – p. 23



Pick a vector v

October 16, 2008 – p. 23



Pick a vector v

v, Av, A2v, A3v, . . .

October 16, 2008 – p. 23



Pick a vector v

v, Av, A2v, A3v, . . .

Krylov subspace:

Kj(A, v) = span{

v,Av,A2v, . . . , Aj−1v}

October 16, 2008 – p. 23



Pick a vector v

v, Av, A2v, A3v, . . .

Krylov subspace:

Kj(A, v) = span{

v,Av,A2v, . . . , Aj−1v}

Look in here for approximate eigenvectors.

October 16, 2008 – p. 23



Pick a vector v

v, Av, A2v, A3v, . . .

Krylov subspace:

Kj(A, v) = span{

v,Av,A2v, . . . , Aj−1v}

Look in here for approximate eigenvectors.

. . . but need better basis

October 16, 2008 – p. 23

Arnoldi Process

October 16, 2008 – p. 24

Arnoldi Processbuild orthonormal basis v1, v2, v3, . . .

October 16, 2008 – p. 24


jth step: v̂j+1 = Avj −∑j

i=1vihij

October 16, 2008 – p. 24



i=1vihij

hij = 〈Avj , vi〉 (Gram-Schmidt)

October 16, 2008 – p. 24



i=1vihij


Normalization: hj+1,j = ‖ v̂j+1 ‖, vj+1 = v̂j+1/hj+1,j

October 16, 2008 – p. 24



i=1vihij



Collect coefficients hij

October 16, 2008 – p. 24



i=1vihij




H4 =

h11 h12 h13 h14

h21 h22 h23 h24

h32 h33 h34

h43 h44

October 16, 2008 – p. 24



i=1vihij




H4 =

h11 h12 h13 h14

h21 h22 h23 h24

h32 h33 h34

h43 h44

eigenvalues are Ritz values

October 16, 2008 – p. 24

Example: 479 × 479 matrix

−150 −100 −50 0 50 100 150−2000

−1500

−1000

−500

0

500

1000

1500

2000

October 16, 2008 – p. 25


−150 −100 −50 0 50 100 150−2000

−1500

−1000

−500

0

500

1000

1500

2000

October 16, 2008 – p. 26


−150 −100 −50 0 50 100 150−2000

−1500

−1000

−500

0

500

1000

1500

2000

October 16, 2008 – p. 27


−150 −100 −50 0 50 100 150−2000

−1500

−1000

−500

0

500

1000

1500

2000

October 16, 2008 – p. 28

For better accuracy . . .

October 16, 2008 – p. 29


. . . take more steps.

October 16, 2008 – p. 29



but,

October 16, 2008 – p. 29



but, vectors take up space.

October 16, 2008 – p. 29




Alternate plan:

October 16, 2008 – p. 29




Alternate plan:

Start over with a better vector.

October 16, 2008 – p. 29

Implicit Restarts

October 16, 2008 – p. 30

Implicit RestartsTake, say, 30 steps.

October 16, 2008 – p. 30


Get 30 Ritz values.

October 16, 2008 – p. 30


Get 30 Ritz values.

Keep the best ones (e.g. 10) . . .

October 16, 2008 – p. 30


Get 30 Ritz values.


. . . and associated invariant subspace.

October 16, 2008 – p. 30


Get 30 Ritz values.



Restart at step 11.

October 16, 2008 – p. 30


Get 30 Ritz values.



Restart at step 11.

(neat details omitted)

October 16, 2008 – p. 30


Get 30 Ritz values.



Restart at step 11.

(neat details omitted)

Build back up to 30 and then restart again.

October 16, 2008 – p. 30

Example

October 16, 2008 – p. 31

Example460 × 460 Hamiltonian matrix (toy problem)

October 16, 2008 – p. 31


asking for 24 eigenpairs

October 16, 2008 – p. 31



building to dimension 84

October 16, 2008 – p. 31




restarting with 28

October 16, 2008 – p. 31




restarting with 28

Hamiltonian Lanczos process

October 16, 2008 – p. 31

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

October 16, 2008 – p. 32

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

October 16, 2008 – p. 33

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

October 16, 2008 – p. 34

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

October 16, 2008 – p. 35

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

October 16, 2008 – p. 36

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

October 16, 2008 – p. 37

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

October 16, 2008 – p. 38

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

October 16, 2008 – p. 39

−3 −2 −1 0 1 2 3

−3

−2

−1

0

1

2

3

October 16, 2008 – p. 40

Concluding Remarks

October 16, 2008 – p. 41

Concluding RemarksI used these methods to solve my “medium” sizedeigenvalue problems.

October 16, 2008 – p. 41


Simple ideas lead to powerful methods.

October 16, 2008 – p. 41


Simple ideas lead to powerful methods.

Thank you for your attention.

October 16, 2008 – p. 41

krylov subspace methods for eigenvalue problems · krylov subspace methods for eigenvalue problems...

Documents