Multigrid for Nonlinear Problems

Ferien-Akademie 2005, Sarntal, Christoph Scheit

FAS, Newton-MG, Multilevel Nonlinear Method

Outline

Motivation Basic Idea of Multigrid Classical MG-Approaches for nonlinear problems

– Newton-Multigrid– FAS– Properties of both approaches

Multilevel Nonlinear Method Conclusions Bibliography

Motivation

To solve (linear) systems of equations arising from the discretization of nowadays engineering science problems fast and robust solvers are needed

A lot of problems arising from engineering science contain nonlinearities

Example: nonlinear diffusion equation

),())(( yxfuug

Basic Ideas of Multigrid

A lot of relaxations schemes smooth the error. Consider Jacobi-relaxation:

nnnn

nnn

nn

BeeBexx

BezBxzexBx

zBxxfDzUDB

fDUxDxfUxDx

UDA

fAx

11

1

11

111

)(

:,:

Basic Ideas of Multigrid

From eigenvalue analysis (Local Mode analysis) or numerical experiments one can see that only high frequency errors are damped (smoothed) efficiently by most relaxation schemes

The basic idea is now, to smooth the error on a grid, on which the error looks high frequent. On a twice as coarse grid, the same mode appears with the double frequency relative to the number of grid points. Therefore high-frequency errors can be smoothed efficiently on the coarser grid.

Basic Ideas of Multigrid

The residual equation:

Using the residual equation, it is possible to compute the error and update the approximate solution

Bringing it all together, one can use coarser grids to efficiently compute a correction for the actual solution

rAe

xAfxAAx

fAx

~~

Basic Ideas of Multigrid

To use a sequence of Grids, transfer operators are needed:

– Restriction to transport the residual to a coarser grid

– Interpolation (or prolongation) to transport the correction back to the finer grid

Using this ideas, one can construct schemes like the well known V-Cycle

Basic Algorithm

MGM_Basic(x,f,l,ν1,ν2,μ)if (l == 1)

return x = exact_sol(x,f)endx_h = preSmoothing(x,f,ν1);r_h = f-Ax;r_H = Restriction(r_h);for (i = 1; i < μ; i++)

x_H = 0;e_H = MGM_Basic(x_H,r_H,l-1,ν1,ν2,μ)

ende_h = Prolongate(e_H);x_h = x_h + e_hx_h = postSmoothing(x_h,f,ν2)return x_h;

end

Nonlinear Problems

For an equation like our resulting operator N for the discretized equation is itself depending on the solution u

How to modify the algorithm for nonlinear problems?– Important, since for nonlinear problems the Residual Equation does

not hold any more (instead, nonlinear Residual Equation / defect equation):

rxNfxNNx

xxNxNNx

fNx

~~)~(~

fuu ³

Nonlinear Approaches for Multigrid

Newton-Multigrid

-> global linearization FAS (Full Approximation Scheme/Storage)

-> local linearization MNM (Multilevel Nonlinear Method)

-> combine local and global method

Newtons Method

First consider Newtons Method for scalar Problems:

If f(x+s) is a solution, then

)(

)(

)(

)()()(0

xf

xfxx

xf

xfsxfsxf

)(²)()()(

0)(

fsxfsxfsxf

xf

Newtons Method

Newtons Method

How to use Newtons Method for nonlinear equation systems? – Analog to the scalar case:

If F(v+s)=F(u) is a solution

Where s is the error of the current approximation, we get:

)(ξFs²F(v)sF(v)s)F(v

0F(u)T

F(v)F(v)][s)vF(sF(v)0 1TT

F(v)F(v)][vv 1T

Newtons Method

What is grad(F(u))?

-> the Jacobian of F(u)

3

3

2

3

1

3

3

2

2

2

1

2

3

1

2

1

1

1

)(

),,(

),,(

),,(

)(

3213

3212

3211

uf

uf

uf

uf

uf

uf

uf

uf

uf

uuuf

uuuf

uuuf

uJuF

Newtons Method

A concrete example for J(v):

Partial derivation yields:

0)(

0)²²2(³²

²1

31

2

u

fhuuhuuhfux

uiiiii

iF

22

21

2

2

32100

10

01

00132

)(

nuh

uh

h

vJ

Newton-Multigrid

Back to Newtons Method for Multigrid:

Using the nonlinear Residual Equation and the truncated taylor serious yields

The last equation is the linearized equation system and has to be computed instead of the original one using multigrid methods.

rvNuN

evJvFevF

)()(

)()()(

revJ

rvNevJvN

)(

)()()(

evu

Newton-Multigrid - Algorithm

v = init_sol();

r = f-N(u);

while (r < tol)

compute J(v);

e = 0;

for i = 0; i < numV-Cycles; i++

e = MGM_Basic(e,r,l,ν1,ν2,μ)

end

v = v + e;

r = f-N(u);

end

FAS

Newtons-Multigrid doesn‘t use Multigrid ideas to solve the nonlinear equation system, but uses a global linearization and an outer iteration with the basic Multigrid Method embeded as a solver for the linearized equation system

Different from the idea of Newtons Method for Multigrid, FAS treats directly the nonlinear equation system, using a nonlinear smoother for local linearization such as Gauss-Newton relaxation

FAS

Back to the nonlinear Residual Equation (defect correction equation):

We can formulate this equation on the coarse grid by:

Where is the injection operator (instead of full weighted restriction)

NvrNurNvNu

hHh

HhHh

HH vINrIuN ˆHhI

FAS - Algorithm (only V-Cycle)

FAS(x,f,l,ν1,ν2)if(l==1)

return x = exact_sol(x,f);endx_h = preSmoothing(x,f,ν1);f_H = restriction(f – A_h x_h)

+ A_h injection(x_h);x_H = injection(x_h); // initial guess for coarse gridFAS(x_H,f_H,l, ν1,ν2);x_h = x_h + prolongation(x_H – injection(x_h));x_h = postSmoothing(x_h,f,ν2);return x_h;

end

FAS – nonlinear relaxation

Instead of a global linearization FAS uses a nonlinear smoother, which is simply obtained by Newtons Method (for scalar problem):

Consider again the nonlinear equation:

Discretized we obtain:

Using Newtons Method yields the following iteration scheme:

fuxu

3

2

2

0)²²2( 13

12

iiiii fhuuhuuh

)32(

)²²2(222

13

12

i

iiiiiii uhh

fhuuhuuhuu

FAS – implementation hints

Start first with a linear problem; then the FAS-Algorithm must yield the same result as the standard MG-Algorithm (except roundoff errors)

For the nonlinear problem considered here, a standard Gauss-Seidel relaxation works also. In general one has to use a nonlinear smoother like presented above

Since FAS does not approximate the error, but directly improves the current solution on the different grid levels, don‘t forget to inject also the boundary condition (for the error in the standard MG this was not necessary, since for Dirichlet b.c. the b.c. for the error is always zero)

Properties of classical approaches

Newton: Fast convergence, often only

a few newton steps For each newton step, the

linearized equation must be solved accurately

A good initial guess is needed to ensure convergence (small attraction basin)

(slow) backtracking to find a good initial guess

FAS: No global linearization is

needed Convergence even for poor

initial guess, if a good approximation for the nonlinear operator is available (large attraction basin)

Converges slower to the solution than Newton-MG

Multilevel Nonlinear Method (MNM)

While Newtons-MG converges fast, we need a good initial guess

While FAS converges not so fast, it converges even for a poor initial guess if we have a good approximation for the nonlinear operator

Idea: Combine the properties of both algorithms, such that the resulting Method converges fast and even for a poor initial guess -> MNM

Use a robust approximation for the dominating operator -> MNM, Galerkin Coarsening

MNM

Once again back to the nonlinear Residual equation:

Now we want to split this equation into a large linear part and a small nonlinear part. The linear part corresponds to Newton-MG while the nonlinear part corresponds to FAS. The nonlinear part should be small, because in this case it would not be so bad, if the approximation of the nonlinear part is not so good (which was required by FAS)

rNvNu

MNM

To obtain this splitting, we add to the left hand side of the nonlinear Residual Equation J(v)e; e = u-v:

Rearranging the terms yields a linear and a nonlinear term:

Obviously, the linear part is O(e), but what about the nonlinear part?

rv)(uvJvuvJNvNu )())((

))(([...]

],),(,,[))((

vuvJNvNuF

rvuvJNvNuFvuvJ

MNM

Consider a Taylor serious:

Hence the nonlinear part is O(e²) and therefor we obtain a splitting with a large linear but a relatively small nonlinear part

evJvNuNeO

eOevJvNevNuN

)()()()(

)()()()()(2

2

MNM

Back to the complete equation:

There are two methods to bring the operators to the coarser grid: Rediscretization Galerkin Coarsening

We will use rediscretization only for the nonlinear part (though rediscretization might yield a bad approximation in case of a PDE with jumping coefficients, the influence for MNM is only O(e²)). Denote rediscretized operators by a head Â:

rv)(uvJNvNuvuvJ )())((

rv)(uvJvNuNvuvJ )(ˆˆˆ))((

MNM

Now we obtain an iteration by defining:

Substituting this into the original equation and rearranging yields the defect equation for MNM:

n

1n

u:v

u:u

nnnnn

1nn1nn1n

)u(uJ)uJ(uuNr

)u(uJ)uJ(uuN

ˆˆ

ˆˆ

MNM

As one can see, the following operators must be defined on the several levels:

While the first two will be simply rediscretized on each level, the third one is obtained by Galerkin coarsening:

To bring the current approximation to the next coarser level, we will use injection (as for FAS)

)J(u),(uJ,N nnˆˆ

hHn

hTHhn

H )I(uJ][I)(uJ

MNM - Algorithm

MNM(u,N,L,f,l,ν1,ν2)if(l == 0)

solve Nu+Lu=f:return u;

endRelax ν1 times equation

Nu+Lu=f;Compute residual

r=f-(Nu+Lu);Construct linearized operator

K = L+J^(u);Initialize coarse grid solution

u_H = injection(u);Galerkin Coarsening for linearized operator

K_H = galerkinCoarsening(K);

MNM – Algorithm(II)

Compute L for coarse gridL_H = K_H + J^_H(u_H);

Compute RHS for coarse gridf_H = restriction(r) + N_H u_H + L_H u_H;

recursive callMNM(u_H,N_H,L_H,f_H,l-1,ν1,ν2);add correction

u = u + prolongation(u_H - injection(u));postsmoothing

Nu+Lu=f;End

Where L :=the linear correction to N

)(ˆ)( nn uJuJ

MNM – Concrete Example

Consider the equation

For the approximated operators we obtain:

Here B is a scaling to ensure compatibility with the linearized coarse grid operator due to Galerkin coarsening

fCeuxu

2

2

])2([]ˆ[

]2[]ˆ[

12

12

12

12

iu

iiiiHH

iu

iiiiHH

uCeHuuHBuK

uCeHuuHBuN

i

i

MNM - Adaptive

Idea: Use parameters to „controll“ how much of FAS and Newton should be used

Consider the complete coarse grid operator:

Two points of view: The first term is the main term, second and third term

are a nonlinear correction The second term is the main term, while the first and

the third term are a linear correction

HHHH KNKN ˆˆ

MNM - Adaptive

Now we can use a weighting of the operators:

a=1, b=0: Newtons Mehtod a=0, b=1:FAS a=1=b=1, MNM

]1,0[,

ˆ)1(ˆ

ba

baba HHHH KNKN

MNM – Results

2-D diffusion problem

where

)()),,(( xfuuuxg x

0,]|)|(1[

|)|(1||1)(

0,1)(21

2

uu

uukugg

uugg

qp

pp

q

MNM - Results

P/α .5 .75 1

1.5 MNM=0.18

FAS=0.26

MNM=0.26

FAS=0.35

MNM=0.97

FAS=0.64

2 MNM=0.13

FAS=0.23

MNM=0.25

FAS=0.33

MNM=0.38

FAS=0.41

2.5 MNM=0.12

FAS=0.21

MNM=0.25

FAS=0.33

MNM=0.41

FAS=0.42

MNM – Implementation hints

Due to the Galerkin coarsening we have the restriction operator acting on the left hand side as well as on the right hand side hence it cancels out. But the prolongation operator is only on the right hand side, therefor we have to introduce a compatible scaling also for the rediscretized operators.

Since N + L is just an approximation of the fine grid operator (nonlinear), a nonlinear relaxation method is needed, such as for FAS (e.g. Gauss-Newton)

Conclusions(I)

FAS and Newton-MG have both advantages and disadvantages

MNM combines the good properties of both methods, but introduces difficulties due to scaling of the coarse grid approximations for the operators

MNM yields usually fastest convergence factor of all three approaches

Sometimes MNM does not converge, than backtracking can be used, but yields poor convergence

Conclusions(II)

Adaptive MNM can be used instead of MNM with backtracking, yielding a quite good convergence factor

The computational cost per V-Cycle for MNM is more expensive than for FAS or Newtons method, but less than the sum of both

MNM is still a research topic MNM is more complicated to implement

Bibliography

I. Yavneh and G Dardyk, A Multilevel Nonlinear Method, Haifa, 2005

W. L. Briggs, V. E. Henson, and S F. McCormick, A Multigrid Tutorial, SIAM, Philadelphia, second ed., 2000

V. E. Henson, Multigrid for nonlinear problems: an overview, Center for Applied Scientific Computing Lawrence Livermore National Laboratory, 2003