the multigrid method

The Multigrid Method

Nicolas Alt ∙ [email protected]

University of Technology, Munich

JASS 2006


2 / 31JASS 2006

Overview

Model Problems Relaxation Methods Error convergence Multiple grids Performance Theoretical Considerations


3 / 31JASS 2006

Model Problem 1D

Differential Equation in 1D

–u’’(x) + au(x) = f(x)

for 0 < x < 1 , a > 0Boundary: u(0) = u(1) = 0

Partition continuous problem into n subintervals by “sampling” it at the grid points xj = jh, with h = 1/n

Grid Ωh:

u0 u1 u2 u3 u4 u5[ ] Vector u / v

u


4 / 31JASS 2006

Model Problem 1D

Second order finite difference approximation

v0 = vn = 0; with v being the approximate solution to u Written in Matrix-Vector form

Written compactly: Av = f

11 with )(²

2 11 njxfav

h

vvvjj

jjj

1

1

1

1

1

1

)(

)(

²21

1²211²2

²

1

nnn f

f

xf

xf

v

v

ah

ahah

h


5 / 31JASS 2006

Model Problem 2D

(Elliptic) Partial Differential Equation

–uxx – uyy + au = f(x,y)

for 0 < x,y < 1 , a > 0

Boundary: “Frame = 0” Sampled with a two-dimensional grid

(n-1,m-1 interior grid points)

h

y

x

u3,2


6 / 31JASS 2006

Model Problem 2D

Sampling results in difference approximation

Written in Matrix-Vector form

1,1,0

11with 22

00

2

1,1,

2

,1,1

mjivvvv

njfavh

vvv

h

vvv

mjjini

ijijy

jiijji

x

jiijji

1)-(n1)-(m1)-(n1)-(m ismatrix ofDimension matrix identity 1)-(n1)-(n a is I

1)-(n1)-(n isdimension - Problem-1D frommatrix likealmost looks B²

1

²

1

²

1²

1

1

1

1

1

mm f

f

v

v

BIh

Ih

BIh

Ih

B

Tniii

Tniii

fff

vvv

1,1

1,1


7 / 31JASS 2006

Model Problem 2D

Example: System for a=0, n=4, h=1

Again, written compactly: Av = f

3,3

2,3

1,3

3,2

2,2

1,2

3,1

2,1

1,1

3,3

2,3

1,3

3,2

2,2

1,2

3,1

2,1

1,1

010141

010010141

010010141

010

fffffffff

vvvvvvvvv

11

111

1111

11

1

I

B

I


8 / 31JASS 2006

Overview



9 / 31JASS 2006

Relaxation Methods

To solve the PDE, u = A-1f is too complicated Based on an estimated solution v(0) find better

solution v(1) in next step Reduces norm of the error e = u – v Use residual r = f – Av as a measure

Relationship error / residual: Ae = rFor exact solution v = u r = 0

For the following, split matrix A = D – L – UD: diagonal of A; L/U: lower/upper triangular part of A


10 / 31JASS 2006

Relaxation Methods

General approximation: • Try to find a B “close” to A-1, as u – v = A-1r

Jacobi scheme / Simultaneous displacement• jth component of v is calculated using the two

neighbours from previous step

• Solves the PDE locally(compare original problem: –uj-1 + 2uj – uj+1 = h2fj)

fDvRvULD

njfhvvv

J

jjjj

1)0()1(

1J

2)0(1

)0(12

1)1(

)(R :matrixiteration Jacobi

11),(

0)0()1( rvv B


11 / 31JASS 2006

Relaxation Methods

Weighted or damped Jacobi method• Weighting factor 0 < ω < 1

Gauss-Seidel• Like Jacobi, but components updated immediately• Reduces storage requirements

fDvRvRI

fhvvv

njvvv

J

jjjj

jjj

1)0()1(

2)0(1

)0(12

1*

*)0()1(

)1(R :matrixiteration Jacobi Weightedabove) (like)(with

11,)1(

)( :formally

overwrite"" meaning),(2)0(

1)1(12

1)1(

2112

1

jjjj

jjjj

fhvvv

fhvvv


12 / 31JASS 2006

Overview



13 / 31JASS 2006

Error convergence

Simplified problem: Au = 0 v should converge to 0, and e = v

In what way does weighted Jacobi decrease the error? Analyse eigenvectors of iteration matrix

Eigenvectors wk of matrices A and Rω

• Vector wk is also the kth Fourier mode

Eigen values λk of matrix Rω (generally: Rωwk = λkwk)

• For 0 ≤ ω < 1 |λk| < 1, iteration converges

njnkn

jkw jk

0 ,11with ,sin,

11with ,2

sin21)( 2

nkn

kRk


14 / 31JASS 2006

Error convergence

Eigen values Smooth, low-frequency Fourier modes of e: 1 ≤ k ≤ ½n

• |λk| is close to 1 no satisfactory damping Oscillatory, high-frequency modes: ½n ≤ k ≤ n-1

• For the right ω, |λk| is close to 0 good damping

• Optimal damping for ω = ⅔

11with ,2

sin21)( 2

nkn

kRk

0 2 4 6 8-1

-0.5

0

0.5

1

0 2 4 6 8-1

-0.5

0

0.5

1


15 / 31JASS 2006

Error convergence

Damping diagram for the weighted Jacobi method

Oscillatory modes of the error are removed quite well Smooth modes are hardly damped.

11with ,2

sin21)( 2

nkn

kRk


16 / 31JASS 2006

Error convergence

Example code in MATLAB• Grid n = 64• Initial error modes 2 and 16• Solves –u’’(x) = 0

n = 64;% components of AD = 2 * diag(ones(n-1,1),0); U = diag(ones(n-2,1),1); L = diag(ones(n-2,1),-1);% iteration matricesw=2/3; RJ = inv(D) * (L+U); RW = (1-w).*eye(n-1) + w.*RJ;% init f=0, v with modes 2 and 16f = zeros(n-1,1);v = transpose(sin((1:n-1) * 2 * pi / n) + sin((1:n-1) * 16 * pi / n));plot(v); hold on% do 10 iterationsfor i = 1:10 v = RW*v + 0; endplot(v);

0 10 20 30 40 50 60 70-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

initial v

v after 10 iterations

v after 300 iterations


17 / 31JASS 2006

Overview



18 / 31JASS 2006

Multiple Grids

Fundamental idea of multigrid• Make smooth modes look oscillatory! • Smooth mode on Ωh looks oscillatory on grid Ωnh

• A “hierarchy of discretizations” is used to solve the problem of small damping for smooth modes

0 2 4 6 8 10 12-1

-0.5

0

0.5

1

0 1 2 3-1

-0.5

0

0.5

1

Ωh Ω4h


19 / 31JASS 2006

Multiple Grids

Intergrid Transfer coarse → fine: Interpolation• Ω2h → Ωh, “Upsampling”• Linear interpolation is effective

10),( 22

12

21

12

22

nh

jhj

hj

hj

hj

jvvv

vv

h

h

h

hhh

vvvvvvv

vvv

I vv

7

6

5

4

3

2

1

23

2

12

2

1211

211

21

2

1


20 / 31JASS 2006

Multiple Grids

Intergrid Transfer fine → coarse: Restriction • Ωh → Ω2h, “Downsampling”• Simplest method: Injection• Better: Full weighting

• Restriction operator:

• Transfer Operations Ωh ↔ Ω2h sufficient

11),2( 212212412

nhj

hj

hj

hj jvvvv

121121

121

4

12hhI

hhh

h I vv 22


21 / 31JASS 2006

Multiple Grids

Aliasing: Oscillatory modes on Ωh will be represented as smooth modes on Ω2h

A basic two-grid correction scheme• On grid Ωh, relax υ1 times on Ahvh = 0 with initial guess v(0)h

- Restrict fine-grid residual rh to the coarse grid- On grid Ω2h, relax υ2 times on A2he2h = r2h with initial guess

e(0)h = 0- Interpolate the coarse-grid error

• Correct the fine-grid approximation: vh ← vh + eh

• On grid Ωh, relax υ1 times on Ahvh = 0 with initial guess vh


22 / 31JASS 2006

Multiple Grids

Multigrid strategies• Nested iteration: Use coarse grids to generate

improved initial guesses• Coarse grid correction: Approximate the error by

relaxing on the residual equation on a course grid

V-cycle W-cycle FMG scheme


23 / 31JASS 2006

Multiple Grids

The V-Cycle Scheme (Coarse Grid Correction)• V-Cycle(vh, fh)

- Relax υ1 times on Ahvh = 0 with initial guess vh

- If (current grid = coarsest grid) goto last point- Else: f2h = Restrict(fh – Ahvh)- v2h = 0- Call v2h = V-Cycle(v2h, f2h)

- Correct vh += Interpolate(v2h)- Relax υ2 times on Ahvh = 0 with initial guess vh

• Recursive algorithm


24 / 31JASS 2006

Overview



25 / 31JASS 2006

Performance

Storage requirements• Vectors v and f for n = 16 with boundary values

• For d = 1, memory requirement is less then twice that of the fine-grid problem alone

Ωh Ω2h Ω4h Ω8h

17 9 5 3

v

Ωh Ω2h Ω4h Ω8h

17 9 5 3

f

problem ofDimension :dwith ,21

2d

dnceStorageSpa


26 / 31JASS 2006

Performance

Computational costs • 1 work unit (WU): one relaxation sweep on Ωh

• O(WU) = O(N), with N: Total number of grid points• Intergrid transfer is neglected• One relaxation sweep per level (υi = 1)

• 1D problem: Single V-Cycle costs ~4WU,Complete FMG cycle ~8WU

WUCost

WUCost

d

d

2CycleFMG

Cycle-V

)21(

221

2


27 / 31JASS 2006

Performance

Diagnostic Tools• Help to debug your implementation• Methodical Plan for testing modules• Starting Simply with small, simple problems • Exposing Trouble – difficulties might be hidden• Fixed Point Property – relaxation may not change

exact solution• Homogenous Problem: norm of error and residual

should decrease to zero


28 / 31JASS 2006

Overview



29 / 31JASS 2006

Theory

The Impact of Intergrid Transfer and the iterative method may be expressed and proven in a formal way

Two-grid correction TG consists of matrices for Interpolation, Restriction and Relaxation

Spectral picture of multigrid• Relaxation damps oscillator modes• Interpolation & Restriction damp smooth modes

Algebraic picture of multigrid• Decompose Space of the error: Ωh = R N • • L similar to R, H similar to N

)()( 2 TGKerIRangeR hh )()( 22 hh

hhh

h AIKerAINN


30 / 31JASS 2006

Theory

Operations of multigrid, visualized• Plane represents Ωh

• Error eh is successively projected on one of the axes- Relaxations on the fine grid (1)- Two-grid correction (2)- Again, relaxation on the fine grid (3)


31 / 31JASS 2006

End of Presentation

Thanks for your attention Any questions?

the multigrid method

Documents

athe multigrid method

fthe multigrid method

h2fjthe multigrid method

f ibithe multigrid method

multigrid methodnicolas

error e

matrix r

original problem