Algorithms for a large sparse nonlinear eigenvalue problem

Yusaku YamamotoDept. of Computational Science &

EngineeringNagoya University

Outline

Introduction Algorithms

Multivariate Newton’s method Use of the linear eigenvalue of smallest modulus Use of the signed smallest singular value

Numerical experiments Conclusion

Introduction

Nonlinear eigenvalue problem Let A() be an n by n matrix whose elements depend on a

scalar parameter In the nonlinear eigenvalue problem, we seek a value of for which there exists a nonzero vector x such that

and x are called the (nonlinear) eigenvalues and eigenvectors, respectively.

Examples A() = A – I : Linear eigenvalue problem A() = 2 M + C + K : Quadratic eigenalue problem A() = (e –1) A1 + A2 – A3 : General nonlinear eigenvalue

problem

A() x = 0.

Applications of the nonlinear eigenvalue problem

Structural mechanics Decay system: 2 Mx + Cx + Kx = 0

M : mass matrix, C : damping matrix, K : stiffness matrix

Electronic structure calculation Kohn-Sham equation: H() =

H()Kohn-Sham Hamiltonian

Theoretical fluid dynamics Computation of the scaling exponent in turbulent flow

Solution of a quadratic eigenproblem

Transformation to a linear generalized eigenproblem

Quadratic eigenproblem can be transformed into a linear generalized eigenproblem of twice the size.

In general, nonlinear eigenproblem of order k can be transformed into a linear generalized eigenproblem of k times the size.

Efficient algorithms for linear eigenproblems (QR, Krylov subspace method, etc.) can be applied.

2 Mx + Cx + Kx = 0 　 　

y

x

y

x

00

0

M

MC

M

K

Our target problem

Computation of the scaling exponent of a passive scalar field in a turbulent flow Governing equation of the passive scalar

n-point correlation function

Scaling exponent

We are interested in computing n in the case of n = 4.

n(sx1, sx2, …, sxn) = sn n(x1, x2, …, xn)

Turbulent flow

Our target problem (cont’d)

The PDE satisfied by 4

By writing 4= s44, we have

Non-linearity comes from folding the (unbounded) domain of (x1, x2, x3, x4) into a bounded one using the scaling law.

F 4= 0, where Cs

Bss

AsF

22

22

(4(4-1)A + 4B + C) 4 = 0 : quadratic eigenproblem

4(s, s1, s2, …, s5) = 14 4(s/1, s, s2/1, …, s5/1 )

Our target problem (cont’d)

Problem characteristics A() is large (n ~ 105), sparse and nonsymmetric. Dependence of A() on is fully nonlinear (includes

both exponential and polynomial terms in ), but is analytical.

Computation of A() takes a long time. The smallest positive eigenvalue is sought.

Solution of a general nonlinear eigenproblem

Necessary and sufficient condition for

A simple approach (Gat et al., 1997; the case of n = 3) Find the solution of det(A()) = 0 by Newton’s method. Find the eigenvector x as the null space of a constant

matrix A() .

Difficulty Computation of det(A()) requires roughly the same cost as

the LU decomposition of A(). Not efficient when A() is large and sparse.

x 0, A() x = 0 　 　 det(A()) = 0

Equations to be solved

Iteration formula

Approach based on multivariate Newton’s method

Basic idea Regard A() x = 0 as nonlinear simultaneous equations

w.r.t. n+1 variables and x and solve them by Newton’s method.

Since there are only n equations, we add a normalization condition vtx = 1 using some vector v.

0

0

1

)(

xv

xt

A

1

)(

0

)(')(1

1

1

lt

ll

tlll

l

l

l

l AAA

xv

x

v

xxx

Multivariate Newton’s method (cont’d)

Advantages Each iteration consists of solving linear simultaneous

equations. Much cheaper than computing det(A()).

Convergence is quadratic if the initial values and x are sufficiently close to the solution.

Disadvantages The iterates may converge to unwanted eigenpairs or fail

to converge unless both and x are sufficiently good. It is in general difficult to find a good initial value x for the

eigenvector. A'() is necessary in addition to A().

Approaches based on the linear eigenvalue /singular value of smallest modulus

Definition For a fixed , we call a linear eigenvalue of A() if

there exists a nonzero vector y such that A()y = y. For a fixed , we call >0 a linear singular value of

A() if 2 is a linear eigenvalue of A()TA(). Linear eigenvalue / singular value are simply an

eigenvalue / singular value of A() viewed as a constant matrix. and are functions of .

Necessary and sufficient conditions for x 0, A() x = 0 　 　 det(A()) = 0

A() has a zero linear eigenvalue. A() has a zero linear singular value.

Approaches based on the linear eigenvalue /singular value of smallest modulus (cont’d)

A possible approach Let

() = the linear eigenvalue of smallest modulus of A(), () = the smallest linear singular value of A().

Find the solution to () = 0 or () = 0. Find the eigenvector x as the null space of a constant matrix

A().

Advantages Only the initial value foris required. () and () can be computed much more cheaply than

det(A()). Use of the Lanczos, Arnoldi, and Jacobi-Davidson methods

A'() is not necessary if the secant method is used to find .

Approach based on the linear eigenvalue of smallest modulus

Algorithm based on the secant method

Difficulty When A() is nonsymmetric, computing () is

expensive. Though it is much less expensive than computing det(A()).

・ Set two initial values and .

・ Repeat the following until |(l)|

becomes sufficiently small:

・ Find the eigenvector x as the null space of a constant matrix A(l).

()

121

12

1

llll

ll

l

l–1

l–2l

Approach based on the smallest linear singular value

Possible advantages For nonsymmetric matrices, singular values

can be computed much more easily than eigenvalues.

Problems The linear singular value () of A() is defined

as the positive square root of the linear eigenvalue of A()TA().

Hence, () is not smooth at () = 0. The secant method cannot be applied.

Solution Modify the definition of () so that it is smooth

near () = 0. 　　Analytical singular valueSigned smallest singular value

()

l–1 l–2 l

Analytical singular value decomposition

Theorem 1 (Bunse-Gerstner et al., 1991) Let the elements of A() be analytical functions of . Then

there exist orthogonal matrices U’() and V’() and a diagonal matrix ’() = diag(1’(), 2’(), … , n’() ) whose elements are analytical functions of and which satisfy

　 This is called the analytical singular value decomposition of A().

Notes Analytical singular values may be negative. In general,1’() > 2’() > ... > n’() does not hold. Analytical singular values are expensive to compute.

Requires the solution of ODE’s starting from some initial point .

()U’()’()V’()T.

i’()

0

1’()

2’()

3’()

Signed smallest singular value Definition

Let vn and un be the right and left singular vectors of A() corresponding to the smallest linear singular value n . Then we call n = n sgn(vn

Tun) the signed smallest singular value of A().

Theorem 2 Assume that n() is a simple root and |vn()Tun()| 0 in an interval .Then the signed smallest singular value

n() sgn(vn()Tun()) is an analytic function of in this interval.

Proof From the uniqueness of SVD, un() = un’() and vn() = vn’() . Hence,

The right-hand-side is clearly analytical when |vn()Tun()| 0.

.

.

n() = n() sgn(vn()Tun()) = n()vnT()un() / |vn

T()un()|

= vnT()A()vn() / |vn

T()un()| = vn’T()A()vn’() / |vn’T()un’()|.

.

Approach based on the signed smallest singular value

Characteristics of the signed smallest singular value n()

n() = 0 n() = 0

n() is an analytical function of under suitable assumptions.

Easy to compute (requires only n(), vn() and un() )

Algorithm based on the secant method()

l–1

l–2l

・ Set two initial values and .

・ Repeat the following until |(l)|

becomes sufficiently small:

・ Find the eigenvector x as the null space of a constant matrix A(l).

121

12

1

llll

ll

l

Numerical experiments

Test problem Computation of the scaling exponent in turbulent

flow Matrix size is 35,000 and 100,000. Seek the smallest positive (nonlinear) eigenvalue It is known that the eigenvalue is in [0, 4], but the

estimate for the (nonlinear) eigenvector is unknown.

Computational environment Fujitsu PrimePower HPC2500 (16PU)

Algorithm I: Approach based on the linear eigenvalue of smallest modulus

Result for n=35,000

Nonlinear eigenvalue: 2.926654 Computational time ： 35,520 sec. for each value of .

Secant iteration: 4 times

-6.00E+00

-4.00E+00

-2.00E+00

0.00E+00

2.00E+00

4.00E+00

6.00E+00

0 1 2 3 4

()

Algorithm II: Approach based on the signed smallest singular value

Result for n=35,000

Result for n=100,000 Computational time ： 16,200 sec. for each value of .

Could not be computed with Algorithm 1 because the computational time was too long.

- 6.00E- 01

- 4.00E- 01

- 2.00E- 01

0.00E+00

2.00E- 01

4.00E- 01

6.00E- 01

8.00E- 01

1.00E+00

0.00E+00 1.00E+00 2.00E+00 3.00E+00 4.00E+00

() Nonlinear eigenvalue ：2.926654

Computational time:

2,005 sec. for each value of . (1/18 of Algorithm 1)

Secant iteration: 4 times

Conclusion

Summary of this study We proposed an algorithm for large sparse nonlinear

eigenproblem based on the signed smallest positive singular value.

The algorithm proved much faster than the method based on the linear eigenvalue of smallest modulus.

Future work Application of the algorithm to various nonlinear

eigenproblems