krylov 2 filimon

8/12/2019 Krylov 2 Filimon

1/24

Krylov Subspace Iteration Methods

Anastasia Filimon

ETH Zurich

29 May 2008

Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 1 / 24
http://find/


2/24

Set up

The methods represent iterative techniques for solving large linearsystems

Ax=b,

where Ais non-singular nxn-matrix, bis n-vector, n is large.

They are based on projection processes onto Krylov subspaces.

http://find/


3/24

Krylov Subspaces

Krylov subspace generated by an nxn-matrix A, and an n-vector b is

the subspace spanned by the vectors of the Krylov sequence:

Km =spanfb, Ab, A2b, ..., Am1bg.The projection method seeks an approximate solution xm from anane subspace x0+ Km by imposing condition b

Axm

?Lm , Lm is

another subspace of dimension m, x0 is an initial guess to thesolution. In the case of Krylov subspace methods Km = Km(A, r0),r0 =b Ax0 is an n-vector

Km =spanfr0 , Ar0 , A

2r0 , ..., Am1r0

g.

Property 1. Km is the subspace of all vectors in Rn which can be

written asx=p(A)v,

where pis a polynomial of degree not exceeding m 1.Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 3 / 24
http://find/


4/24

Krylov Subspaces

The dierent versions of Krylov subspace methods arise from dierentchoices of the subspace Lm and from the ways in which the system ispreconditioned. Two broad choices for Lm give rise to the best-knowntechniques:

Lm =Km FOMLm =AKm GMRES, MINRES.

http://find/


5/24

Arnoldi Modied Gram-Schmidt Method

This is an algorithm for building an orthogonal basis of the Krylovsubspace Km .

At each step, the algorithm multiplies Arnoldi vector vj byAand thenorthonormalizes the resulting vector wjagainst all previous vjs by a

standard Gram-Schmidt procedure.Anastasia Filimon (ETH Zurich) Krylov Subspace Iteration Methods 29/05/08 5 / 24
http://find/http://goback/


6/24

Arnoldis Method

Proposition 1. Assume the Arnoldi Algorithm does not stop beforethe m-th step. Then the vectors v1 , v2 , ..., vm form an orthonormalbasis of the Krylov subspace

Km =spanfv1 , Av1 , ..., Am1v1g.Proposition 2. Projection method onto the subspace Kjwill beexact when a breakdown occurs at step j.Proposition 3. Denote by Vm the nxmmatrix with column vectors

v1 , ..., vm , byH m , the (m+1)xm Hessenberg matrix whose nonzeroentries hijare dened by Arnoldi Modied Gram-Schmidt Algorithmand by Hm the matrix obtained from H m by deleting its last row.

Then the following relations hold:

AVm =VmHm+ wmeTm

=Vm+1Hm

VTmAVm =Hm

.



7/24

Full Orthogonalization Method (FOM)

Given the initial guess x0 to the original linear system Ax=b,consider the projection method described before, which takesLm =Km(A, r0), where r0 =b Ax0 . Ifv1 = r0kr0k2 in Arnoldismethod and set =kr0k2, then

VTmAVm = Hm from Proposition 3, and

VTmr0 = V

Tm

(v1) = e1 .

As a result, the approximate solution using the above mdimensionalsubspaces is given by

xm =x0+ Vmym , ym =H1m

(e1)

http://find/


8/24


The presented algorithm depends on a parameter m which is thedimension of the Krylov subspace. In practice it is desirable to selectm in a dynamic fashion. This would be possible if the residual norm

of xm is available without compution xm itself.Proposition 4. The residual vector of the approximate solution xmcomputed by the FOM Algorithm is such that

kb

Axm

k2 =hm+1,m eTm ym .

http://find/


9/24


http://find/


10/24

Generalized Minimum Residual Method (GMRES)

The method is a projection method based on taking Lm =AKm ,inwhich Km is the m-th Krylov subspace with v1 =r0/

kr0k2.

Such atechnique minimizes the residual norm over all vectors in x0+ Km .The implementation of an algorithm based on this approach is similarto that of the FOM algorithm.

Any vector x in x0+ Km can be written as x=x0+ Vmy, where y is

an m-vector.Dene

J(y) =kb Axk2 =kb A(x0+ Vmy)k2 ,Using relation from Proposition 3

b Ax = b A(x0+ Vmy) = roAVmy= v1 Vm+1Hmy= Vm+1(e1Hmy).

Since the column-vectors ofVm+1 are orthonormal, then

J(y) =

kb

A(x0+ Vmy)

k2 = e1Hmy2 .



11/24


The GMRES approximation is the unique vector ofx0+ Km whichminimizes J(y), i.e.

xm = x0+ Vmym,

whereym = arg min

y

e1Hmy2The minimizer is inexpensive to compute since it requires the solutionof an (m+1)xm least-squares problem where m is typically small.

http://find/


12/24


http://find/


13/24

Generalized Minimum Residual Method (GMRES).

Practical Implementation issues

A clear diculty with GMRES algorithm is that it does not providethe approximate solution xm explicitly at each step. As a result, it is

not easy to determine when to stop. However, there is a solutionrelated to the way in which the least-squares problem is solved.

In order to solve the least-squares problem mine1Hmy

, it isnatural to transform the Hessenberg matrix into upper triangular form

by using plane rotations.

http://find/


14/24



Proposition 5. Dene the rotation matrices to transform Hm intoupper triangular form

i =

0BBBBBBBBBB@

1...

1ci sisi ci

1...

1

1CCCCCCCCCCA

c2i + s2i =1, si=

hi+1,iq(h

(i1)ii )

2 +h2i+1,i

, ci= h

(i1)iiq

(h(i1)ii )

2 +h2i+1,i

.

http://find/


15/24



Dene the product of matrices i

Qm = mm1 ...1 ,

Rm,

gm = (1, ...,

m+1)T

the resulting matrix and right-hand side

Rm = H(m)m =QmHm ,

gm = Qm(e1) = (1 , ..., m+1)T

.

Denote by Rm the mxm upper triangular matrix obtained from Rm bydeleting its last row and by gm the mdimensional vector obtainedfrom gm by deleting its last component.


( )
http://find/


16/24



Then,1. Vector ym which minimizes

e1Hmy2 is given byym =R

1m gm .

2. The residual vector at step m satises

bAxm =Vm+1(e1Hmy) =Vm+1QTm(m+1em+1) and, as a result,kb Axmk2 =jm+1j .

This was the process for computing the least-squares solution ym .Theprocess must be stopped if the residual normjm+1j is small enough.The last rows ofRm and gm are deleted and the resulting uppertriangular system is solved to obtain ym . Then the approximatesolution xm =x0+ Vmym is computed.


Th S i L Al i h
http://find/


17/24

The Symmetric Lanczos Algorithm

This algorithm can be viewed as a simplication of Arnoldis methodfor the particular case when the matrix is symmetric. When Ais

symmetric, then the Hessenberg matrix Hm becomes symmetrictridiagonal. 0

BBBB@1 22 2 3

...

m1 m1 mm m

1CCCCA


Th C j G di Al i h
http://find/


18/24

The Conjugate Gradient Algorithm

This algorithm is one of the best known iterative techniques forsolving sparse Symmetric Positive Denite linear systems.

Assume we need to minimize the following function

f(x) = 12xTAx xTb

where Ais nxn-matrix positive denite and symmetric, b is n-vector.

The minimum value off(x) isbTA1b/2, achieved by settingx= A

1

b. Therefore, minimizingf(x) and solving Ax=bareequivalent problems ifA is symmetric positive denite.


Th C j G di Al i h
http://find/


19/24


The vector xj+1 can be expressed as .

xj+1 =xj+jpj.

Therefore, the residual vectors must satisfy the recurrence

rj+1 =rjjApj.

To have rjs orthogonal it is necessary that

(rj

jApj, rj) =0

and, as a result,

j = (rj,rj)

(Apj, rj).


Th C j t G di t Al ith
http://find/


20/24


The rst basis vector p1 is the gradient off at x0 , which equals to

Ax0 b . The other vectors in the basis will be conjugate to thegradient . Each nextpk+1 is dened to be in the direction closest tothe gradient rkunder the conjugacy constraint. This direction is givenby the projection ofrkonto the space orthogonal to pk with respectto the inner product induced by A.

pj+1 = rj+1+jpj,

(Apj, rj) = (Apj, pjj1pj1) = (Apj, pj)

j =

(rj,rj)

(Apj, pj),

j=

(rj+1,Apj)

(pj, Apj)

Apj = 1j

(rj+1 rj),j= 1

j

(rj+1,(rj+1 rj))(Apj, pj)

=(rj+1 , rj+1)

(rj, rj)


Th C j t G di t Al ith
http://find/


21/24


The process stops if rj+1 is "suciently small".


Convergence Analysis
http://find/


22/24


One of the main tool used in the analysis of convergence behavior isChebyshev polynomials.

Lemma 1. Let xm be the approximate solution obtained from themth step of the CG algorithm, and let dm =x xm , where x isthe exact solution. Then xm is of the form

xm =x0+ qm(A)r0 ,

where qm is a polynomial of degree m 1 such thatk(IAqm(A))d0kA = min

q2Pm1k(IAq(A))d0kA .

Theorem 2. Let xm be the approximate solution obtained from themth step of the CG algorithm, and x is the exact solution. Then

kx xmkA2"p

k 1pk+1

#mkx x0kA .


http://find/


23/24



http://find/


24/24


http://find/

krylov 2 filimon

Documents