multigrid tutorial

119
A Multigrid Tutorial By William L. Briggs Presented by Van Emden Henson Center for Applied Scientific Computing Lawrence Livermore National Laboratory This work was performed, in part, under the auspices of the United States Department of Energy by University of California Lawrence Livermore National Laboratory under contract number W-7405-Eng-48.

Upload: ninagika

Post on 29-Oct-2015

67 views

Category:

Documents


2 download

DESCRIPTION

Multigrid Tutorial

TRANSCRIPT

Page 1: Multigrid Tutorial

AMultigrid Tutorial

ByWilliam L. Briggs

Presented byVan Emden Henson

Center for Applied Scientific ComputingLawrence Livermore National Laboratory

This work was performed, in part, under the auspices of the United States Department of Energy by Universityof California Lawrence Livermore National Laboratory under contract number W-7405-Eng-48.

Page 2: Multigrid Tutorial

2 of 119

Outline• Model Problems• Basic Iterative Schemes

– Convergence; experiments– Convergence; analysis

• Development of Multigrid– Coarse-grid correction– Nested Iteration– Restriction and Interpolation– Standard cycles: MV, FMG

• Performance– Implementation– storage and computation costs

• Performance, (cont)– Convergence– Higher dimensions– Experiments

• Some Theory– The spectral picture– The subspace picture– The whole picture!

• Complications– Anisotropic operators and grids– Discontinuous or anisotropic

coefficients– Nonlinear Problems: FAS

Page 3: Multigrid Tutorial

3 of 119

Suggested Reading

• Brandt, “Multi-level Adaptive Solutions to Boundary ValueProblems,” Math Comp., 31, 1977, pp 333-390.

• Brandt, “1984 Guide to Multigrid Development, withapplications to computational fluid dynamics.”

• Briggs, “A Multigrid Tutorial,” SIAM publications, 1987.• Briggs, Henson, and McCormick, “A Multigrid Tutorial, 2nd

Edition,” SIAM publications, 2000.• Hackbusch, Multi-Grid Methods and Applications,” 1985.• Hackbusch and Trottenburg, “Multigrid Methods, Springer-

Verlag, 1982”• Stüben and Trottenburg, “Multigrid Methods,” 1987.• Wesseling, “An Introduction to Multigrid Methods,” Wylie,

1992

Page 4: Multigrid Tutorial

4 of 119

Multilevel methods have beendeveloped for...

• Elliptic PDEs• Purely algebraic problems, with no physical grid; for

example, network and geodetic survey problems.• Image reconstruction and tomography• Optimization (e.g., the travelling salesman and long

transportation problems)• Statistical mechanics, Ising spin models.• Quantum chromodynamics.• Quadrature and generalized FFTs.• Integral equations.

Page 5: Multigrid Tutorial

5 of 119

Model Problems

• One-dimensional boundary value problem:

• Grid:

• Let and for

>σ,<<)(=)(σ+)(″− xxfxuxu 010uu =)(=)( 010

Nh =

1

xuv ii )(≈ xff ii )(≈

...,,=,=, Nihixi 10

Ni ...,,= 10

x0 x1 x2 xi xN

x = 0 x = 1

Page 6: Multigrid Tutorial

6 of 119

We use Taylor Series to derivean approximation to u’’(x)

• We approximate the second derivative usingTaylor series:

• Summing and solving,

hOxuh

xuh

xuhxuxu )(+)(′′′+)(″+)(′+)(=)( iiiii +4

32

1 !3!2

hOxuh

xuh

xuhxuxu )(+)(′′′−)(″+)(′−)(=)( iiiii −4

32

1 !3!2

hOh

xuxuxuxu )(+

)(+)(−)(=)(″

iiii

−+ 22

11 2

Page 7: Multigrid Tutorial

7 of 119

We approximate the equationwith a finite difference scheme

• We approximate the BVP

with the finite difference scheme:

>σ,<<)(=)(σ+)(″− xxfxuxu 010uu =)(=)( 010

−...,,==σ+−+−

Nifvh

vvvii

iii +−2

11121

2

vv0 == N 0

Page 8: Multigrid Tutorial

8 of 119

The discrete model problem

• Letting and

we obtain the matrix equation whereis (N-1) x (N-1), symmetric, positive definite, and

A fv =

v ),...,,(= vvv 121T

N −

f ),...,,(= fff 121T

N −

A

ff

fff

f

vv

vvv

h

h

h

h

h

hA

=,

=,

σ+−

−σ+−

−σ+−

−σ+−

−σ+

=

21

121

121

121

12

1

1

2

3

2

1

12

321

2

2

2

2

2

2v

N

NNN

−−−

Page 9: Multigrid Tutorial

9 of 119

Solution Methods• Direct

– Gaussian elimination– Factorization

• Iterative– Jacobi– Gauss-Seidel– Conjugate Gradient, etc.

• Note: This simple model problem can be solvedvery efficiently in several ways. Pretend it can’t,and that it is very hard, because it shares manycharacteristics with some very hard problems.

Page 10: Multigrid Tutorial

10 of 119

A two-dimensional boundary valueproblem

• Consider the problem:

• Where the grid is given:

yyxxu >σ;=,=,=,=,= 010100

Nh

Mh yx ,=,=

11

),(=),( hjhiyx yxji

0 ≤≤ Mi0 ≤≤ Nj

x

y

z

<<,<<,),(=σ+−− yxyxfuuu yyxx 1010

Page 11: Multigrid Tutorial

11 of 119

Discretizing the 2D problem

• Let and . Again, using 2nd

order finite differences to approximate andwe obtain the approximate equation for theunknown , for i=1,2, … M-1 and j=1,2, …, N-1:

• Ordering the unknowns (and also the vector f )lexicographically by y-lines:

yxuv jiji ),(≈ yxff jiji ),(≈u xx u yy

yxu ),( ji

=σ+−+−

+−+−

fvh

vvv

h

vvvjiji

y

jijiji

x

jijiji +,−,,+,−

2

11

2

11 22

MjjMiiv ji =,=,=,=,= 000

v ),...,,,...,...,,,,...,,(= vvvvvvvvv 112111122212112111 −,−,−,−−,,,−,,,T

NNNNNN

Page 12: Multigrid Tutorial

12 of 119

Yields the linear system• We obtain a block-tridiagonal system Av = f :

where Iy is a diagonal matrix with on thediagonal and

=

−−

−−−−

f

fff

v

vvv

AIIAI

IAIIAI

IA

1

3

2

1

1

321

1

2

3

2

1

NNNy

yNy

yy

yy

y

−−−

1

hy2

hhh

hhhh

hhhh

hhh

A

yxx

xyxx

xyxx

xyx

i

σ++−

⋅⋅⋅⋅⋅⋅⋅⋅⋅

−σ++−

−σ++−

−σ++

=

111

1111

1111

111

222

2222

2222

222

Page 13: Multigrid Tutorial

13 of 119

Iterative Methods for LinearSystems

• Consider Au = f where A is NxN and let v be anapproximation to u.

• Two important measures:– The Error: with norms

– The Residual: with

vue ,−=

||=|||| ee ∞ xam i =|||| ee 212 i

Ni =

vAfr −=|||| r ∞ |||| r 2

Page 14: Multigrid Tutorial

14 of 119

Residual correction

• Since and , we can write as

which means that , which is theResidual Equation:

• Residual Correction:

vue ,−= vAfr −=fuA =

fevA =)+(

vAfeA −=

reA =

evu +=

Page 15: Multigrid Tutorial

15 of 119

Relaxation Schemes

• Consider the 1D model problem

• Jacobi Method (simultaneous displacement): Solvethe ith equation for holding other variablesfixed:

==−≤≤=−+− uuNifhuuu Niiii +− 02

11 0112

vi

Nifhvvv idlo

idlo

iwen

i)(

+)(

−)( −≤≤)++(= 11

21 2

11

Page 16: Multigrid Tutorial

16 of 119

In matrix form, the relaxation is

• Let where D is diagonal and L and Uare the strictly lower and upper parts of A.

• Then becomes

• Let , then the iteration is:

fuA =

fuULuD +)+(=fDuULDu +)+(= −− 11

ULDRJ )+(= −1

ULDA )−−(=

=)−−( fuULD

fDvRv −)()( dloJ

wen += 1

Page 17: Multigrid Tutorial

17 of 119

The iteration matrix and theerror

• From the derivation,

• the iteration is

• subtracting,

• or

• hence

fDuULDu +)+(= −− 11

fDuRu += J−1

fDvRv −)()( dloJ

wen += 1

e )()( dloJ

wen = eR

vRuRvu −=− )()( dloJJ

wen

fDvRfDuRvu )+(−+=− −)(−)( dloJJ

wen 11

Page 18: Multigrid Tutorial

18 of 119

Weighted Jacobi Relaxation• Consider the iteration:

• Letting A = D-L-U, the matrix form is:

.• Note that

• It is easy to see that if ,then

fhvvvv idlo

idlo

idlo

iwen

i)(

+)(

−)()( )++(

ω+)ω−(←

21 2

11

fDhvULDIv −)(−)( dlowen ω+)+(ω+)ω−(= 1 121

ω+= fDhvR −)(ω dlo 12

RIRω ]ω+)ω−([= 1 J

uue −≡ )()( xorppatcaxe

eRe )(ω)( dlowen =

Page 19: Multigrid Tutorial

19 of 119

Gauss-Seidel Relaxation (1D)• Solve equation i for ui and update immediately.• Equivalently: set each component of r to zero.• Component form: for set

• Matrix form:

• Let• Then iterate

• Error propagation:

Ni ,−...,,= 121

fhvvv iiii )++(←21

+−2

11

ULDA )−−(=+=)−( fuUuLD

fLDuULDu )−(+)−(= −− 11

ULDRG )−(= −1

fLDvRv −)()( dloG

wen )−(+← 1

eRe )()( dloG

wen ←

Page 20: Multigrid Tutorial

20 of 119

Red-Black Gauss-Seidel• Update the even (red) points

• Update the odd (black) points

fhvvv 22

12122 iiii )++(←21

+−

fhvvv 122

22212 iiii +++ )++(←21

x0 xN

Page 21: Multigrid Tutorial

21 of 1190 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1

- 1

- 0 . 8

- 0 . 6

- 0 . 4

- 0 . 2

0

0 . 2

0 . 4

0 . 6

0 . 8

1

Numerical Experiments• Solve ,• Use Fourier modes as initial iterate, with N =64:

uA = 0 =−+− uuu iii +− 11 02

vk

π

=)(=Nki

v ki nis 1111 −≤≤,−≤≤ NkNicomponent mode

k = 3

k = 1

k = 6

Page 22: Multigrid Tutorial

22 of 119

0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 0 00

0 . 1

0 . 2

0 . 3

0 . 4

0 . 5

0 . 6

0 . 7

0 . 8

0 . 9

1

Error reduction stalls• Weighted Jacobi on 1D problem.• Initial guess:

• Error plotted against iteration number:

Nj

Nj

Nj

v0

π

+

π

+

π

=23

nis6

nisnis31

=ω32

|||| e ∞

Page 23: Multigrid Tutorial

23 of 1190 20 40 6 0 8 0 1 00 1 20

0

0 .1

0 .2

0 .3

0 .4

0 .5

0 .6

0 .7

0 .8

0 .9

1

Convergence rates differ fordifferent error components

• Error, ||e||∞∞∞∞ , in weighted Jacobi on Au = 0 for100 iterations using initial guesses of v1, v3, and v6

k = 1

k = 3

k = 6

Page 24: Multigrid Tutorial

24 of 119

Analysis of stationary iterations

• Let . The exact solution isunchanged by the iteration, i.e., .

• Subtracting, we see that

• Letting e0 be the initial error and ei be the errorafter the ith iteration, we see that after niterations we have

gvRv )()( dlowen +=

guRu +=

eRe )()( dlowen =

eRe )()( nn = 0

Page 25: Multigrid Tutorial

25 of 119

A quick review of eigenvectorsand eigenvalues

• The number is an eigenvalue of a matrix B, and wits associated eigenvector, if Bw = w.

• The eigenvalues and eigenvectors are characteristicsof a given matrix.

• Eigenvectors are linearly independent, and if there isa complete set of N distinct eigenvectors for anNxN matrix, they form a basis; i.e., for any v, thereexist unique ck such that:

λλ

wcv = kkN

k = 1

Page 26: Multigrid Tutorial

26 of 119

“Fundamental theorem ofiteration”

• R is convergent (that is, as ) if andonly if , where

therefore, for any initial vector v(0), we see that if and only if .

• assures the convergence of the iterationgiven by R and is called the convergencefactor for the iteration.

R n→0 n ∞→<)(ρ R 1

|λ|,...,|λ|,|λ|=)(ρ R xam N21

ne )( n ∞→→ sa0 <)(ρ R 1

<)(ρ R 1)(ρ R

Page 27: Multigrid Tutorial

27 of 119

Convergence Rate• How many iterations are needed to reduce the

initial error by ?

• So, we have .

• The convergence rate is given:

01 −d

∼))(ρ(∼||||≤||||||||

RRee −

)(

)( dMMM

001

dM

=gol

101 )(ρ R

noitaretistigid

gol1

goletar ))(ρ(−=)(ρ

= 0101 RR

Page 28: Multigrid Tutorial

28 of 119

Convergence analysis forweighted Jacobi on 1D model

ULDIR −ω )+(ω+)ω−(= 1 1

ω−= ADI −1

IRω

−⋅⋅⋅⋅⋅⋅⋅⋅⋅

−−−−

ω−=

21

121121

12

2

)(λω

−=)(λ ARω 21

For the 1D model problem, he eigenvectors of the weightedJacobi iteration and the eigenvectors of the matrix A arethe same! The eigenvalues are related as well.

Page 29: Multigrid Tutorial

29 of 119

Good exercise: Find theeigenvalues & eigenvectors of A

• Show that the eigenvectors of A are Fouriermodes!

π

=,

π

=)(λ jkk Nkj

wN

kA nis

2nis4 2

,

0 1 0 2 0 3 0 4 0 5 0 6 0

-1

- 0 . 8

- 0 . 6

- 0 . 4

- 0 . 2

0

0 . 2

0 . 4

0 . 6

0 . 8

1

0 1 0 2 0 3 0 4 0 5 0 6 0

- 1

- 0 . 8

- 0 . 6

- 0 . 4

- 0 . 2

0

0 . 2

0 . 4

0 . 6

0 . 8

1

0 1 0 2 0 3 0 4 0 5 0 6 0

-1

- 0 . 8

- 0 . 6

- 0 . 4

- 0 . 2

0

0 . 2

0 . 4

0 . 6

0 . 8

1

0 1 0 2 0 3 0 4 0 5 0 6 0

-1

-0 . 8

-0 . 6

-0 . 4

-0 . 2

0

0 . 2

0 . 4

0 . 6

0 . 8

1

0 1 0 2 0 3 0 4 0 5 0 6 0

-1

-0 . 8

-0 . 6

-0 . 4

-0 . 2

0

0 . 2

0 . 4

0 . 6

0 . 8

1

Page 30: Multigrid Tutorial

30 of 119

Eigenvectors of Rωωωω and A are thesame,the eigenvalues related

• Expand the initial error in terms of theeigenvectors:

• After M iterations,

• The kth mode of the error is reduced by λk at eachiteration

π

ω−=)(λk Nk

Rω 2nis21 2

wce−

=

)(1

1

0 = kkN

k

wcwRceR kMkk

N

kk

Mk

N

k

M−

=

=

)(1

1

1

1

0 λ==

Page 31: Multigrid Tutorial

31 of 119

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

k

Eig

enva

lue

/=ω 31

/=ω 21

/=ω 32=ω 1

N2

Relaxation suppresseseigenmodes unevenly

• Look carefully at

π

ω−=)(λk Nk

Rω 2nis21 2

Note that ifthen for

For ,

10 ≤ω≤<|)(λ| k Rω 1

Nk −,...,,= 121

10 ≤ω≤

π

ω−=λ 21 2

nis21N

π

ω−=2

nis21 2 h

≈)(−= 11 hO 2

Page 32: Multigrid Tutorial

32 of 119

Low frequencies are undamped

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

k

Eig

enva

lue

/=ω 31

/=ω 21

/=ω 32=ω 1

N2

• Notice that no value of will damp out the long(i.e., low frequency) waves.

ω

What value of givesthe best damping ofthe short waves ?

Choose such that

NkN2

≤≤

ω

)(λ−=)(λ NN2

RR ωω

ω

32

Page 33: Multigrid Tutorial

33 of 119

The Smoothing factor

• The smoothing factor is the largest absolute valueamong the eigenvalues in the upper half of thespectrum of the iteration matrix

• For Rω, with , the smoothing factor is ,since

and for .

• But, for long waves .

2rofxamrotcafgnihtooms ≤≤)(λ= k Nk

NR

=ω32

31

=λ=λ NN2

31

<λk 31

NkN2

<<

π−≈λk 32

1 hk 222

Nk

Page 34: Multigrid Tutorial

34 of 119

Convergence of Jacobi on Au=0

• Jacobi method on Au=0 with N=64. Number ofiterations required to reduce to ||e||∞ < .01

• Initial guess :

0 10 20 30 40 50 600

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 600

10

20

30

40

50

60

70

80

90

100

=N

jkvkjπsin

Wavenumber, kWavenumber, k

Unweighted Jacobi Weighted Jacobi

Page 35: Multigrid Tutorial

35 of 119

Weighted Jacobi RelaxationSmooths the Error

• Initial error:

• Error after 35 iteration sweeps:0 0 . 5 1

- 2

- 1

0

1

2

0 0 . 5 1

- 2

- 1

0

1

2

Many relaxationschemes

have the smoothingproperty, where

oscillatorymodes of the error

areeliminated

effectively, butsmooth modes are

dampedvery slowly.

Nj

Nj

Nj

v jk

π

+

π

+

π

=23

nis2161

nis212

nis

Page 36: Multigrid Tutorial

36 of 1190 10 20 30 40 50 60

0

0.2

0.4

0.6

0.8

1

Other relaxations schemes maybe analyzed similarly

• Gauss-Seidel relaxation applied to the 3-pointdifference matrix A (1D model problem):

• Good exercise: Show thatULDRG )−(= −1

0 10 20 30 40 50 60

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

π

=)(λ Gk Nk

R soc 2Nkj

wj

kjk/

,

π

)λ(=2

nis

Page 37: Multigrid Tutorial

37 of 119

Convergence of Gauss-Seidel onAu=0

• Eigenvectors of RG are not the same as those of A.Gauss-Seidel mixes the modes of A.

0 10 20 30 40 50 600

10

20

30

40

50

60

70

80

90

100 Jacobi method on Au=0with N=64. Number ofiterations required toreduce to ||e||∞ < .01

Initial guess : (Modes ofA)

=N

jkvkjπsin

Page 38: Multigrid Tutorial

38 of 119

• Many relaxation schemes have the smoothingproperty, where oscillatory modes of the errorare eliminated effectively, but smooth modesare damped very slowly.

• This might seem like a limitation, but by usingcoarse grids we can use the smoothing property togood advantage.

• Why use coarse grids??

First observation towardmultigrid

x0 xN

x0 xN2

Ωh

Ω2h

Page 39: Multigrid Tutorial

39 of 119

Reason #1 for using coarsegrids: Nested Iteration

• Coarse grids can be used to compute an improvedinitial guess for the fine-grid relaxation. This isadvantageous because:

– Relaxation on the coarse-grid is much cheaper (1/2 asmany points in 1D, 1/4 in 2D, 1/8 in 3D)

– Relaxation on the coarse grid has a marginally betterconvergence rate, for example

instead of 1 )(− hO 241 )(− hO 2

Page 40: Multigrid Tutorial

40 of 119

Idea! Nested Iteration

• …• Relax on Au=f on to obtain initial guess• Relax on Au=f on to obtain initial guess• Relax on Au=f on to obtain … final solution???

• But, what is Au=f on , , … ?

• What if the error still has smooth componentswhen we get to the fine grid ?

Ω4h

Ω2h

Ωh

v2h

vh

Ω2h Ω4h

Ωh

Page 41: Multigrid Tutorial

41 of 119

Reason #2 for using a coarsegrid: smooth error is (relatively)

more oscillatory there!• A smooth function:

• Can be represented by linearinterpolation from a coarser grid:

0 0 . 5 1- 1

- 0 . 5

0

0 . 5

1

0 0 . 5 1- 1

- 0 . 5

0

0 . 5

1

On the coarse grid, the smooth error appears tobe relatively higher in

frequency: in the exampleit is the 4-mode, out ofa possible 16, on the finegrid, 1/4 the way up thespectrum. On the coarse grid, it is the 4-mode outof a possible 8, hence itis 1/2 the way up the

spectrum.

Relaxation will be moreeffective on this mode ifdone on the coarser grid!!

Page 42: Multigrid Tutorial

42 of 119

For k=1,2,…N/2, the kth mode ispreserved on the coarse grid

0 2 4 6 8 1 0 1 2

- 1

- 0 . 8

- 0 . 6

- 0 . 4

- 0 . 2

0

0 . 2

0 . 4

0 . 6

0 . 8

1

0 1 2 3 4 5 6

- 1

- 0 . 8

- 0 . 6

- 0 . 4

- 0 . 2

0

0 . 2

0 . 4

0 . 6

0 . 8

1

wNkj

Nkj

w hjk

hjk ,,

22 =

/

π

=

π

=2

nis2

nis

Also, note that

on the coarse grid

whN/2 → 0

k=4 mode, N=12 grid

k=4 mode, N=6 grid

Page 43: Multigrid Tutorial

43 of 119

For k > N/2, wkh is invisible on

the coarse grid: aliasing!!• For k > N/2, the kth mode on

the fine grid is aliased andappears as the (N-k)th modeon the coarse grid:

0 2 4 6 8 1 0 1 2

- 1

- 0 . 8

- 0 . 6

- 0 . 4

- 0 . 2

0

0 . 2

0 . 4

0 . 6

0 . 8

1

0 1 2 3 4 5 6

- 1

- 0 . 8

- 0 . 6

- 0 . 4

- 0 . 2

0

0 . 2

0 . 4

0 . 6

0 . 8

1

)−(π

−=2

nisN

kNj

π)(

=)(N

kjw

jhk 2

2nis

/

)−(π

−=2

nisN

jkN

)(−= w hkN −

2

k=9 mode, N=12 grid

k=3 mode, N=12 grid

Page 44: Multigrid Tutorial

44 of 119

Second observation towardmultigrid:

• Recall the residual correction idea: Let v be anapproximation to the solution of Au=f, where theresidual r=f-Av. The the error e=u-v satisfiesAe=r.

• After relaxing on Au=f on the fine grid, the errorwill be smooth. On the coarse grid, however, thiserror appears more oscillatory, and relaxation willbe more effective.

• Therefore we go to a coarse grid and relax on theresidual equation Ae=r, with an initial guess of e=0.

Page 45: Multigrid Tutorial

45 of 119

Idea! Coarse-grid correction

• Relax on Au=f on to obtain an approximation• Compute .• Relax on Ae=r on to obtain an approximation

to the error, .• Correct the approximation .

• Clearly, we need methods for the mappings and

Ω2h

Ωh vh

Ω2hΩh

vAfr −= h

e2h

evv hhh +← 2

ΩhΩ2h

Page 46: Multigrid Tutorial

46 of 119

1D Interpolation (Prolongation)

• Mapping from the coarse grid to the fine grid:

• Let , be defined on , . Then

where

vh v2h Ωh Ω2h

vv 22

hi

hi =

vvv 21

212

hi

hi

hi ++ )+(=

21

for 12

0 −≤≤N

i

I 22

hhhh Ω→Ω:

vvI 22

hhhh =

Page 47: Multigrid Tutorial

47 of 119

1D Interpolation (Prolongation)

Ωh

Ω2h

• Values at points on the coarse grid map unchangedto the fine grid

• Values at fine-grid points NOT on the coarse gridare the averages of their coarse-grid neighbors

Page 48: Multigrid Tutorial

48 of 119

The prolongation operator (1D)

• We may regard as a linear operator fromℜ N/2-1 ℜ N-1

• e.g., for N=8,

• has full rank, and thus null space

=

/

//

//

/ 21

12121

12121

121

177

6

5

4

3

2

1

1323

22

21

37

xh

h

h

h

h

h

h

xh

h

h

x v

v

v

v

v

v

v

v

v

v

φ

I2hh

I2hh

Page 49: Multigrid Tutorial

49 of 119

How well does v2h approximate u?

• Imagine that a coarse-grid approximation v2h hasbeen found. How well does it approximate theexact solution u ? Think u error!!

• If u is smooth, a coarse-grid interpolant of v2h

may do very well.

Page 50: Multigrid Tutorial

50 of 119

How well does v2h approximate u?

• Imagine that a coarse-grid approximation v2h hasbeen found. How well does it approximate theexact solution u ? Think u error!!

• If u is oscillatory, a coarse-grid interpolant of v2h

may not work well.

Page 51: Multigrid Tutorial

51 of 119

Moral of this story:

• If u is smooth, a coarse-grid interpolant of v2h

may do very well.

• If u is oscillatory, a coarse-grid interpolant of v2h

may not work well.

• Therefore, nested iteration is most effectivewhen the error is smooth!

Page 52: Multigrid Tutorial

52 of 119

1D Restriction by injection• Mapping from the fine grid to the coarse grid:

• Let , be defined on , . Then

where .

vh v2h Ωh Ω2h

vv hi

hi 22 =

I hhhh

22 Ω→Ω:

vvI hhhh

22 =

Page 53: Multigrid Tutorial

53 of 119

1D Restriction by full-weighting

• Let , be defined on , . Then

where

vh v2h Ωh Ω2h

vvvv hi

hi

hi

hi 122122 )++(= 2

41

+−

vvI hhhh

22 =

Page 54: Multigrid Tutorial

54 of 119

The restriction operator R (1D)

• We may regard as a linear operator fromℜ N-1 ℜ N/2-1

• e.g., for N=8,

• has rank , and thus dim(NS(R))

=

///

//////

412141

412141412141

v

v

v

v

v

v

v

v

v

v

23

22

21

7

6

5

4

3

2

1

h

h

h

h

h

h

h

h

h

h

∼N2

∼N2

I hh2

I hh2

Page 55: Multigrid Tutorial

55 of 119

Prolongation and restriction areoften nicely related

• For the 1D examples, linear interpolation and full-weighting are related by:

• A commonly used, and highly useful, requirement isthat

for c in ℜIcI 22

Thh

hh )(=

I2hh

=

1211

211

21

21 I h

h2

=121

121121

41

Page 56: Multigrid Tutorial

56 of 119

2D Prolongation

vv 222

hji

hji, =

vvv 12

212h

jihji

hji ,+,+ )+(=

21

vvv 12

122h

jihji

hji +,+, )+(=

21

vvvvv 11112

1212h

jih

jih

jihji

hji +,++,,++,+ )+++(=

41 4

121

41

21

121

41

21

41

We denote the operator byusing a “give to” stencil, ] [.Centered over a c-point, ,it shows what fraction of

the c-point’s value iscontributed to neighboring

f-points, .

Page 57: Multigrid Tutorial

57 of 119

2D Restriction (full-weighting)

We denote the operator byusing a “give to” stencil, [ ].Centered over a c-point, ,it shows what fractions of

the neighboring ( ) f-points’value is contributed to the

value at the c-point.

611

81

611

81

41

81

611

81

611

Page 58: Multigrid Tutorial

58 of 119

Now, let’s put all these ideastogether

• Nested Iteration (effective on smooth errormodes)

• Relaxation (effective on oscillatory error modes)

• Residual equation (i.e., residual correction)

• Prolongation and Restriction

Page 59: Multigrid Tutorial

59 of 119

Coarse Grid Correction Scheme

• 1) Relax times on on with

arbitrary initial guess .

• 2) Compute .

• 3) Compute .

• 4) Solve on .

• 5) Correct fine-grid solution .

• 6) Relax times on on with initial

guess .

fuA hhh = Ωh

vh

fuA hhh = Ωh

vh

fvGCv hhh )α,α,,(← 21

α1

α2

vAfr hhhh −=

reA 222 hhh = Ω2h

rIr 22 hhh

h =

eIvv hhh

hh +← 22

Page 60: Multigrid Tutorial

60 of 119

Coarse-grid Correction

Relax on fuA hhh =uAfr hhhh −=Compute

rIr 22 hhh

h =Restrict

Solve reA 222 hhh =rAe 2122 hhh )(= −

Correct

eIe hhh

h ≈ 22

Interpolate

euu hhh +←

Page 61: Multigrid Tutorial

61 of 119

What is ?

• For this scheme to work, we must have , acoarse-grid operator. For the moment, we willsimply assume that is “the coarse-gridversion” of the fine-grid operator .

• We will return to the question of constructinglater.

A 2h

A 2h

A 2h

A 2h

A h

Page 62: Multigrid Tutorial

62 of 119

How do we “solve” the coarse-grid residual equation? Recursion!

uIe hhh

h← 22

uIe 424

2 hhh

h←

uIe 848

4 hhh

h←

fAGu hhh ),(← ν

fAGu 222 hhh ),(← ν

fAGu 444 hhh ),(← ν

fAGu 888 hhh ),(← ν

uAfIf 22 hhhhh

h )−(←

uAfIf 22242

4 hhhhh

h )−(←

uAfIf 44484

8 hhhhh

h )−(←

euu 888 hhh +←

euu 444 hhh +←

euu 222 hhh +←

euu hhh +←

fAe HHH )(= −1

Page 63: Multigrid Tutorial

63 of 119

V-cycle (recursive form)

1) Relax times on , initial arbitrary

2) If is the coarsest grid, go to 4) Else:

3) Correct

4) Relax times on , initial guess

fvVMv hhhh ),(←

α1 fuA hhh = vh

Ωh

vAfIf 22 hhhh

hh )−(←

v2h ← 0

fvVMv 2222 hhhh ),(←

vIvv hhh

hh +← 22

α2 fuA hhh = vh

Page 64: Multigrid Tutorial

64 of 119

Storage Costs: and mustbe stored on each level

In 1-d, each coarse grid has about half the number of points as the finer grid.

In 2-d, each coarse grid has about one- fourth the number of points as the finer grid.

In d-dimensions, each coarse grid has about the number of points as the finer grid.

2− d

21

2222212

NN

d

ddMdddd

−<)+…++++(

−−−−− 32

Total storage cost: less than 2, 4/3, 8/7 the cost of storage on the fine grid for 1, 2, and 3-d problems, respectively.

vh f h

Page 65: Multigrid Tutorial

65 of 119

Computation Costs• Let 1 Work Unit (WU) be the cost of one

relaxation sweep on the fine-grid.• Ignore the cost of restriction and interpolation

(typically about 20% of the total cost).• Consider a V-cycle with 1 pre-Coarse-Grid

correction relaxation sweep and 1 post-Coarse-Grid correction relaxation sweep.

• Cost of V-cycle (in WU):

• Cost is about 4, 8/3, 16/7 WU per V-cycle in 1, 2,and 3 dimensions.

21

2222212

−<)+…++++(

−−−−−

ddMddd 32

Page 66: Multigrid Tutorial

66 of 119

Convergence Analysis

• First, a heuristic argument:– The convergence factor for the oscillatory modes of the

error (e.g., the smoothing factor) is small andindependent of the grid spacing h.

– The multigrid cycling schemes focus the relaxation onthe oscillatory components on each level.

2rofxamrotcafgnihtooms ≤≤)(λ= k Nk

NR

smoothk=1

oscillatoryk=Nk=N/2

Relax on fine gridRelax 1st coarse grid

The overall convergence factor for multigridmethods is small and independent of h!

Page 67: Multigrid Tutorial

67 of 119

Convergence analysis, a littlemore precisely...

• Continuous problem:• Discrete problem:

• Global error: (p=2 for model problem)

• Algebraic error:

• Suppose a tolerance is specified such thatmust satisfy

• Then this is guaranteed if

xuufuA )(=,= iiuvfuA hhhhh ≈,=

uxuE hiii −)(=

hKE ≤p

vue hi

hii −=

ε vh

vu ε≤− h

eEvuuu ε≤+=−+− hhh

Page 68: Multigrid Tutorial

68 of 119

We can satisfy the requirementby imposing two conditions

1) . We use this requirement to determine agrid spacing from

2) , which determines the number ofiterations required.

• If we iterate until on thenwe have converged to the level of truncation.

h ∗E

ε≤

2

Kh

/∗

ε

≤2

1 p

≤2

hKe )(=ε

≤2

∗ pΩh ∗

Page 69: Multigrid Tutorial

69 of 119

Converging to the level oftruncation

• Use a MV scheme with convergence rateindependent of h (fixed and ).

• Assume a d-dimensional problem on an NxNx…xNgrid with .

• The V-cycle must reduce the error fromto

• We can determine θ, the number of V-cyclesrequired to accomplish this.

<γ 1α1 α2

Nh = −1

Oe )(∼ 1

NOhOe )(∼)(∼pp −

Page 70: Multigrid Tutorial

70 of 119

Work needed to converge to thelevel of truncation

• Since θ V-cycles at convergence rate γ arerequired, we see that

implying that .

• Since one V-cycle costs O(1) WU and one WU isO(Nd), we see that the cost of converging to thelevel of truncation using the MV method is

• which is comparable to fast direct methods (FFTbased).

)(∼γ−θ NO p

)(∼θ NO gol

NNO )( d gol

Page 71: Multigrid Tutorial

71 of 119

A numerical example

• Consider the two-dimensional model problem (withσ=0), given by

inside the unit square, with u=0 on the boundary.)−()−(+)−()−(=−− xxyyyxuu yyxx 1611612 222222

• The solution to this problem is

• We examine the effectiveness of MV cycling tosolve this problem on NxN grids [(N-1) x (N-1)interior] for N=16, 32, 64, 128.

yyxxyxu )−()−(=),( 2442

Page 72: Multigrid Tutorial

72 of 119

Numerical Results, MV cycling

Shown are the resultsof 16 V-cycles. Wedisplay, at the end ofeach cycle, the residualnorm, the error norm,and the ratio of thesenorms to their values atthe end of the previouscycle.

N=16, 32, 64, 128

Page 73: Multigrid Tutorial

73 of 119

Look again at nested iteration• Idea: It’s cheaper to solve a problem (i.e., takes

fewer iterations) if the initial guess is good.

• How to get a good initial guess:– Interpolate coarse solution to fine grid.– “Solve” the problem on the coarse grid first.– Use interpolated coarse solution as initial guess on fine

grid.

Now, let’s use the V-cycle as the solver on each grid level! This defines the Full Multigrid (FMG) cycle.

Page 74: Multigrid Tutorial

74 of 119

The Full Multigrid (FMG) cycle

• Initialize

• Solve on coarsest grid

• interpolate initial guess

• perform V-cycle

• interpolate initial guess

• perform V-cycle

ffff Hhhh ,...,,, 42

fAv HHH )(=−1

vIv 424

2 hhh

h←

vIv hhh

h← 22

fvVMv 2222 hhhh ),(←

fvVMv hhhh ),(←

fGMFv hh )(←

Page 75: Multigrid Tutorial

75 of 119

Full Multigrid (FMG)• Restriction• Interpolation• High-order Interpolation

Page 76: Multigrid Tutorial

76 of 119

FMG-cycle (recursive form)

1) Initialize

2) If is the coarsest grid, then solve exactly

Else:

3) Set initial guess

4) , η times

Ωh

ffff Hhhh ,...,,, 42

fGMFv hh )(←

vIv hhh

h← 22

fGMFv 22 hh )(←

fvVMv hhh ),(←

Page 77: Multigrid Tutorial

77 of 119

Cost of an FMG cycle• One V-cycle is performed per level, at a cost of WU per grid (where the WU is for the

size grid involved).• The size of the WU for coarse-grid j is times

the size of the WU on the fine grid (grid 0).• Hence, cost of the FMG(1,1) cycle is less than

• d=1: 8 WU; d=2: 7/2 WU d=3: 5/2 WU

21

1

− −d

2− dj

)−(=)...+++(

21

2221

21

2−

−−− d

ddd 2

2

Page 78: Multigrid Tutorial

78 of 119

How to tell if truncation error isreached with Full Multigrid (FMG)

• If truncation error is reached, for each gridlevel h. The norms of the errors at the “solution”points in the cycle should form a Cauchy sequence and

hOe )(∼ 2

e

e2

hj

hj

∼41

Page 79: Multigrid Tutorial

79 of 119

Cost to achieve convergence totruncation by the FMV method

• Consider using the FMV method, which solves theproblem on to the level of truncation beforegoing to , i.e.,

• We ask that which impliesthat the algebraic error must be reduced byon . Hence, V-cycles are needed, where

thus and computational cost of theFMV method .

Ω2h

Ωh

hKvue 222 phhh )(∼−= 2

ehKe hpph =∼ 2− 2

2−p

Ωh θ1

∼γ−θ1 2

p

)(∼θ1 O 1NO )( d

Page 80: Multigrid Tutorial

80 of 119

A numerical example

• Consider again the two-dimensional model problem(with σ=0), given by

inside the unit square, with u=0 on the boundary.)−()−(+)−()−(=−− xxyyyxuu yyxx 1611612 222222

• We examine the effectiveness of FMG cycling tosolve this problem on NxN grids [(N-1) x (N-1)interior] for N=16, 32, 64, 128.

Page 81: Multigrid Tutorial

81 of 119

FMG results• Shown are results for three FMG cycles, and a

comparison to MV cycle results.

Page 82: Multigrid Tutorial

82 of 119

What is ?• Recall the coarse-grid correction scheme:

– 1) Relax on on to get .

– 2) Compute .

– 4) Solve on .

– 5) Correct fine-grid solution .

• Assume that . Then theresidual equation can be written

for some

• How does act upon ?

fuA hhh = Ωh

reA 222 hhh = Ω2h

eIvv hhh

hh +← 22

A 2h

vAfIf 22 hhhhh

h )−(=

IegnaRe hh

h )(∈ 2

eAr hhh = = uIA hhh

h 22 u 22 hh Ω∈

A h IegnaR )( 2hh

vh

Page 83: Multigrid Tutorial

83 of 119

How does act on ?

• Therefore, the odd rows of are zero (in 1Donly) and . We therefore keep the even rowsof for the residual equations on . Theserows can be picked out by applying restriction!

A h IegnaR )( 2hh

u2h

uI 22

hhh

uIA hhh

h 22

r 12i + = 0IA h

hh

2

rIuIAI hhh

hhh

hhh

222

2 =

IA hh

h2

Ω2h

Page 84: Multigrid Tutorial

84 of 119

Building .

• The residual equation on the coarse grid is:

• Therefore, we identify the coarse-grid operatoras

• Next, we determine how to compute the entries ofthe coarse-grid matrix.

A 2h

rIuIAI hhh

hhh

hhh

222

2 =

A 2h

IAIA 222 h

hhh

hh =

Page 85: Multigrid Tutorial

85 of 119

Computing the ith row of .

• Compute where

A 2h

eA 22 hi

h e Thi2 ),...,,,,...,,(= 001000

ii −1 i +1

0 01 e hi2

0 01 21

21 eI 2

2h

ihh

0 0 −2

1

h2−

2

1

h21

h2 eIA hi

hh

h 22

2

1

h2−4

1

h2 −4

1

h2eIAI h

ihh

hhh

22

2

Page 86: Multigrid Tutorial

86 of 119

• The ith row of is ,

which is the version of .

• Therefore, IF relaxation on leaves only error inthe range of interpolation, then solving

determines the error exactly!

• In general, this will not be the case, but thisargument certainly leads to a plausible representationfor .

The ith row of looks alot like a row of !

A 2h

A h

A 2h 1212

1−−

)( h 2

A hΩ2h

A 2h

Ωh

Page 87: Multigrid Tutorial

87 of 119

The variational properties• The definition for that resulted from the

foregoing line of reasoning is useful for boththeoretical and practical reasons. Together withthe commonly used relationship betweenrestriction and prolongation we have the following“variational properties”:

IAIA 222 h

hhh

hh =

IcI 22

Thh

hh )(=

(Galerkin Condition )

for c in ℜ

A 2h

Page 88: Multigrid Tutorial

88 of 119

Properties of the Grid TransferOperators: Restriction

• Full Weighting: or

ℜ N-1 ℜ N/2-1

• For N=8,

• has rank and a null space withdimension .

I hhhh

22 Ω→Ω:I hh2 :

I hh

73

2 =121

121121

41

×

N1

2−I h

h2 IN )( h

h2

N2

Page 89: Multigrid Tutorial

89 of 119

Spectral properties of .

• How does act on the eigenvectors of ?

• Consider , 1 ≤ k ≤ N-1, 0 ≤ j ≤ N

• Good Exercise: show that

for 1 ≤ k ≤ N/2.

I hh2

I hh2 A h

Nkj

whjk,

π

= nis

π

=)( wN

kwI h

jkjhk

hh

2222

soc ,

≡ wc hjkk ,

2

Page 90: Multigrid Tutorial

90 of 119

Spectral properties of (cont.).

• i.e., [kth mode on ] = ck [ kth mode on ]

: N=8, k=2

: N=4, k=2

I hh2

I hh2 Ωh Ω2h

I hh2

Ωh

Ω2h

Page 91: Multigrid Tutorial

91 of 119

Spectral properties of (cont.).

• Let for , so that

• Then (another good exercise!)

I hh2

kNk −=′ NkN2

<′<2

1 <≤N

k

π

−=)( wN

kwI h

jkjhk

hh

222,′ 2

nis

≡ ws hjkk ,

2

Page 92: Multigrid Tutorial

92 of 119

Spectral properties of (cont.).

• i.e., [(N-k)th mode on ] = -sk [ kth mode on ]

• : N=8, k=6

• : N=4, k=2

I hh2

I hh2 Ωh Ω2h

I hh2

Ω2h

Ωh

Page 93: Multigrid Tutorial

93 of 119

Spectral properties of (cont.).

• Summarizing:

• Complementary modes:

I hh2

wcwI hkk

hk

hh

22 =

wswI hkk

hk

hh

22′ −=

wI hN

hh 22

/ = 0

21 <<

Nk

kNk −=′

wwW hk

hkk ,= naps ′

wWI hkk

hh

22 →

Page 94: Multigrid Tutorial

94 of 119

Null Space of .

• Observe that where i is oddand is the ith unit vector.

• Let .

• While the look oscillatory, they contain all ofthe Fourier modes of , i.e.,

• All the Fourier modes of are needed torepresent the null space of restriction!

I hh2

eAIN

=)( hi

hhh2 naps

ehi

≡η hi

hi eA

A hηi

=η kkN

ki

=

1

1wa ak ≠ 0

A h

0 000 −1 −12 0 0

Page 95: Multigrid Tutorial

95 of 119

Properties of the Grid TransferOperators: Interpolation

• Interpolation: or

: ℜ N/2-1 ℜ N-1

• For N=8,

• has full rank and null space .

I2hh

I 22

hhhh Ω→Ω:

I2hh =

1211

211

21

21

I2hh φ

Page 96: Multigrid Tutorial

96 of 119

Spectral properties of .

• How does act on the eigenvectors of ?

• Consider , 1 ≤ k ≤ N/2-1, 0 ≤ j ≤ N/2

• Good Exercise: show that the modes of areNOT preserved by , but that the space ispreserved:

I2hh

I2hh A 2h

/

π

=)(Nkj

wj

hk2

2nis

A 2h

I2hh W k

wN

kw

Nk

wI 2222

hk

hk

hk

hh

π

π

=2

nis2

soc ′

−= wswc hkk

hkk ′

Page 97: Multigrid Tutorial

97 of 119

Spectral properties of (cont.).

• Interpolation of smooth modes excitesoscillatory modes on .

• Note that if ,

• is second-order interpolation.

I2hh

wswcI2hkk

hkk

hh −= ′

Ω2h

Ωh

Nk

wN

kOw

N

kOwI

2

2

2

22

2hk

hk

hk

hh

+

= 1 ′

≈ whk

I2hh

Page 98: Multigrid Tutorial

98 of 119

The Range of .

• The range of is the span of the columns of

• Let be the ith column of .

• All the Fourier modes of are needed torepresent Range( )

I2hh

I2hh I2

hh

ξi I2hh

:ξi

I2hh

≠,=ξ khkk

N

k

hi

=

1

1bwb 0

A h

Page 99: Multigrid Tutorial

99 of 119

Use all the facts to analyze thecoarse-grid correction scheme1) Relax times on :

2) Compute and restrict residual

3) Solve residual equation

4) Correct fine-grid solution .

• The entire process appears as

• The exact solution satisfies

Ωh vRv hh ← α1

vAfIf 22 hhhhh

h )−(←

fAv 2122 hhh )(=−

vIvv hhh

hh +← 22

vRAfIAIvRv hhhhh

hhh

hh )−()(+← α−α 2122

uRAfIAIuRu hhhhh

hhh

hh )−()(+← α−α 2122

α

Page 100: Multigrid Tutorial

100 of 119

Error propagation of the coarse-grid correction scheme

• Subtracting the previous two expressions, we get

• How does CG act on the modes of ? Assumeconsists of the modes and forand .

• We know how act on and .

eRAIAIIe hhhh

hhh

h )(−← 2122

α−

eGCe hh←

A h

whk′wh

k 21 /≤≤ NkkNk −=′

IAIAR−α ,)(,,, h

hhh

hh

2122

whk wh

k′

Page 101: Multigrid Tutorial

101 of 119

Error propagation of CG• For now, assume no relaxation . Then

is invariant under CG.

where

=α 0wwW h

khkk ,= naps ′

wswswGC hkk

hkk

hk += ′

wcwcwGC hkk

hkk

hk ′′ +=

Nk

ck

π

=2

soc 2N

ksk

π

=2

nis 2

Page 102: Multigrid Tutorial

102 of 119

CG error propagation for k << N

• Consider the case k << N (extremely smooth andoscillatory modes):

• Hence, CG eliminates the smooth modes but doesnot damp the oscillatory modes of the error!

wN

kOw

N

kOw kkk

+

→2

2

2

2

wN

kOw

N

kOw kkk

+

→ 112

2

2

2

Page 103: Multigrid Tutorial

103 of 119

Now consider CG with relaxation

• Next, include relaxation sweeps. Assume thatthe relaxation preserves the modes of(although this is often unnecessary). Letdenote the eigenvalue of associated with .For k < < N/2,

αR A h

λkR wk

wswsw kkkkkkk λ+λ→ ′αα

wcwcw kkkkkkk ′α′

α′′ λ+λ→

Small!

Small!

Page 104: Multigrid Tutorial

104 of 119

The crucial observation:

• Between relaxation and the coarse-grid correction, both smooth andoscillatory components of the errorare effectively damped.

• This is essentially the “spectral”picture of how multigrid works. Weexamine now another viewpoint, the“algebraic” picture of multigrid.

Page 105: Multigrid Tutorial

105 of 119

Recall the variational properties

• All the analysis that follows assumes that thevariational properties hold:

IAIA 222 h

hhh

hh =

IcI 22

Thh

hh )(=

(Galerkin Condition )

for c in ℜ

Page 106: Multigrid Tutorial

106 of 119

Algebraic interpretation ofcoarse-grid correction

• Consider the subspaces that make up andΩh Ω2h

IR )( 2hh IN )( h

h2Ωh

Ω2h IR )( hh2

I2hh I h

h2

For the rest of this talk, ‘R( )’refers to the Range of a linearoperator.

From the fundamental theoremof linear algebra:

orIRIN ))((⊥)( Thh

hh

22

IRIN )(⊥)( hh

hh 22

Page 107: Multigrid Tutorial

107 of 119

Subspace decomposition of .• Since has full rank, we can say, equivalently,

( where means that ).

• Therefore, any can be written as where and .

• Hence,

Ωh

A h

AINIR )(⊥)( 22

hhhA

hh h

yx ⊥A h yxA h =, 0

eh tse hhh +=IRs h

hh )(∈ 2 AINt hh

hh )(∈ 2

)(⊕)(=Ω hhh

hh

h AINIR 22

Page 108: Multigrid Tutorial

108 of 119

Characteristics of the subspaces

• Since for some , weassociate with the smooth components of .But, generally has all Fourier modes in it (recallthe basis vectors for ).

• Similarly, we associate with oscillatorycomponents of , although generally has allFourier modes in it as well. Recall that isspanned by therefore isspanned by the unit vectorsfor odd i, which “look” oscillatory.

qIs hhh

h = 22 q 22 hh Ω∈

sh eh

shI2

hh

t h

eh t h

IN )( hh2

=η ih

i eA AIN )( hhh2

e Thi ),...,,,,...,,(= 001000

Page 109: Multigrid Tutorial

109 of 119

qIAIAIIsGC hhh

hhh

hhh

h )(−= 22

2122

Algebraic analysis of coarse-gridcorrection

• Recall that (without relaxation)

• First note that if then .This follows since for someand therefore

• It follows that , that is, the nullspace of coarse-grid correction is the range ofinterpolation.

AIAIIGC )(−= 2122

hhh

hhh

IRs hh

h )(∈ 2 sGC h = 0qIs hh

hh = 2

2 q 22 hh Ω∈

= 0

IRGCN )(=)( 2hh

A 2h by Galerkin property

Page 110: Multigrid Tutorial

110 of 119

More algebraic analysis ofcoarse-grid correction

• Next, note that if then

• Therefore .

• CG is the identity on

AINt hhh

h )(∈ 2

tAIAIItGC hhhh

hhh

h )(−= 2122

ttGC hh =0

AIN )( hhh2

Page 111: Multigrid Tutorial

111 of 119

How does the algebraic picturefit with the spectral view?

• We may view in two ways:

=

that is,

or as

Ωh

Low frequency modes1 ≤ k ≤ N/2

High frequency modesN/2 < k < N Ωh ⊕

⊕=Ωh HL

)(⊕)(=Ω hhh

hh

h AINIR 22

Page 112: Multigrid Tutorial

112 of 119

Actually, each view is just partof the picture

• The operations we have examined work ondifferent spaces!

• While is mostly oscillatory, it isn’t .and while is mostly smooth, it isn’t .

• Relaxation eliminates error from .

• Coarse-grid correction eliminates error from

H

H

IR )( 2hh L

AIN )( hhh2

IR )( 2hh

Page 113: Multigrid Tutorial

113 of 119

How it actually works (cartoon)

IR )( 2hh

AIN )( hhh2

eh

sh

t h

H

L

Page 114: Multigrid Tutorial

114 of 119

Relaxation eliminates H, butincreases the error in .

IR )( 2hh

AIN )( hhh2

eh

sh

t h

H

L

IR )( 2hh

Page 115: Multigrid Tutorial

115 of 119

CG eliminates butincreases the error in H.

IR )( 2hh

AIN )( hhh2

eh

sh

t h

H

L

IR )( 2hh

Page 116: Multigrid Tutorial

116 of 119

Difficulties:anisotropic operators and grids

• Consider the operator :

• The same phenomenonoccurs for grids with muchlarger spacing in one directionthan the other:

),(=∂∂β−

∂∂

α−2

2

2

2yxf

yu

xu

-100 -50 0 50 1000

0.2

0.4

0.6

0.8

1

1.2 If then the GS-smoothingfactors in the x- and y-directions areshown at right.Note that GS relaxation does notdamp oscillatory components in the x-direction.

βα «

Page 117: Multigrid Tutorial

117 of 119

Difficulties: discontinuous oranisotropic coefficients

• Consider the operator :where

• Solutions: line-relaxation (where whole gridlinesof values are found simultaneously), and/or semi-coarsening (coarsening only in the strongly coupleddirection).

Again, GS-smoothing factors in the x- and y-directionscan be highly variable, and very often, GS relaxation doesnot damp oscillatory components in the one or bothdirections.

)∇),((•∇− uyxD

yxdyxd

yxdyxdyxD

),(),(

),(),(

=),(2212

2111

Page 118: Multigrid Tutorial

118 of 119

For nonlinear problems, use FAS (Full Approximation Scheme)

• Suppose we wish to solve: wherethe operator is non-linear. Then the linearresidual equation does not apply.

• Instead, we write the non-linear residual equation:

• This is transferred to the coarse grid as:

• We solve for and transfer theerror (only!) to the fine grid:

reA =

fuA =)(

ruAeuA =)(−)+(

uAfIeuA 2222 hhhhh

hhh ))(−(=)+(

euw 222 hhh +≡

uIwIuu hhh

hhh

hh )−(+← 222

Page 119: Multigrid Tutorial

119 of 119

Multigrid: increasingly, the righttool!

• Multigrid has been proven on a wide variety ofproblems, especially elliptic PDEs, but has also foundapplication among parabolic & hyperbolic PDEs,integral equations, evolution problems, geodesicproblems, etc.

• With the right setup, multigrid is frequently anoptimal (i.e., O(N)) solver.

• Multigrid is of great interest because it is one of thevery few scalable algorithms, and can be parallelizedreadily and efficiently!