Seediscussions,stats,andauthorprofilesforthispublicationat:https://www.researchgate.net/publication/220690328

AMultigridTutorial,2ndEdition

Book·January2000

Source:DBLP

CITATIONS

44

READS

5,228

3authors:

Someoftheauthorsofthispublicationarealsoworkingontheserelatedprojects:

FOSLS/LL*Viewproject

AdaptiveAlgebraicMultigridMethodsViewproject

WilliamL.Briggs

UniversityofColorado

27PUBLICATIONS2,217CITATIONS

SEEPROFILE

VanEmdenHenson

LawrenceLivermoreNationalLaboratory

43PUBLICATIONS2,237CITATIONS

SEEPROFILE

SteveFMcCormick

UniversityofColoradoBoulder

238PUBLICATIONS6,683CITATIONS

SEEPROFILE

AllcontentfollowingthispagewasuploadedbySteveFMcCormickon12February2015.

Theuserhasrequestedenhancementofthedownloadedfile.

Contents

Preface to the Second Edition ixPreface to the First Edition xi

1 Model Problems 1Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Basic Iterative Methods 7Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3 Elements of Multigrid 31Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

4 Implementation 45Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

5 Some Theory 73Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

6 Nonlinear Problems 95Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

7 Selected Applications 113Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

8 Algebraic Multigrid (AMG) 137Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

9 Multilevel Adaptive Methods 163Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

10 Finite Elements 177Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Bibliography 189

Index 191

vii

Preface to the SecondEdition

Twelve years have passed since the publication of the first edition of A MultigridTutorial. During those years, the field of multigrid and multilevel methods hasexpanded at a tremendous rate, reflecting progress in the development and analysisof algorithms and in the evolution of computing environments. Because of thesechanges, the first edition of the book has become increasingly outdated and theneed for a new edition has become quite apparent.

With the overwhelming growth in the subject, an area in which I have neverdone serious research, I felt remarkably unqualified to attempt a new edition. Re-alizing that I needed some help, I recruited two experts to assist with the project.Steve McCormick (Department of Applied Mathematics, University of Colorado atBoulder) is one of the original researchers in the field of multigrid methods and thereal instigator of the first edition. There could be no better collaborator on thesubject. Van Emden Henson (Center for Applied Scientific Computing, LawrenceLivermore National Laboratory) has specialized in applications of multigrid meth-ods, with a particular emphasis on algebraic multigrid methods. Our collaborationon a previous SIAM monograph made him an obvious choice as a co-author.

With the team in place, we began deliberating on the content of the new edi-tion. It was agreed that the first edition should remain largely intact with littlemore than some necessary updating. Our aim was to add a roughly equal amountof new material that reflects important core developments in the field. A topicthat probably should have been in the first edition comprises Chapter 6: FAS(Full Approximation Scheme), which is used for nonlinear problems. Chapter 7 isa collection of methods for four special situations that arise frequently in solvingboundary value problems: Neumann boundary conditions, anisotropic problems,variable-mesh problems, and variable-coefficient problems. One of the chief moti-vations for writing a second edition was the recent surge of interest in algebraicmultigrid methods, which is the subject of Chapter 8. In Chapter 9, we attemptto explain the complex subject of adaptive grid methods, as it appears in the FAC(Fast Adaptive Composite) Grid Method. Finally, in Chapter 10, we depart fromthe predominantly finite difference approach of the book and show how finite ele-ment formulations arise. This chapter provides a natural closing because it ties aknot in the thread of variational principles that runs through much of the book.

There is no question that the new material in the second half of this edition ismore advanced than that presented in the first edition. However, we have tried tocreate a safe passage between the two halves, to present many motivating examples,

ix

x Preface

and to maintain a tutorial spirit in much of the discourse. While the first half ofthe book remains highly sequential, the order of topics in the second half is largelyarbitrary.

The FAC examples in Chapter 9 were developed by Bobby Philip and Dan Quin-lan, of the Center for Applied Scientific Computing at Lawrence Livermore NationalLaboratory, using AMR++ within the Overture framework. Overture is a parallelobject-oriented framework for the solution of PDEs in complex and moving geome-tries. More information on Overture can be found at http://www.llnl.gov/casc/Overture.

We thank Irad Yavneh for a thorough reading of the book, for his technicalinsight, and for his suggestion that we enlarge Chapter 4. We are also gratefulto John Ruge who gave Chapter 8 a careful reading in light of his considerableknowledge of AMG. Their suggestions led to many improvements in the book.

Deborah Poulson, Lisa Briggeman, Donna Witzleben, Mary Rose Muccie, KellyThomas, Lois Sellers, and Vickie Kearn of the editorial staff at SIAM deservethanks for coaxing us to write a second edition and for supporting the project frombeginning to end. Finally, I am grateful for the willingness of my co-authors tocollaborate on this book. They should be credited with improvements in the bookand held responsible for none of its shortcomings.

Bill BriggsNovember 15, 1999Boulder, Colorado

Preface to the First Edition

Assuming no acquaintance with the subject, this monograph presents the essentialideas that underlie multigrid methods and make them work. It has its origins in atutorial given at the Third Copper Mountain Conference on Multigrid Methods inApril, 1987. The goal of that tutorial was to give participants enough familiaritywith multigrid methods so that they could understand the following talks of theconference. This monograph has been written in the same spirit and with a similarpurpose, although it does allow for a more realistic, self-paced approach.

It should be clear from the outset that this book is meant to provide a basicgrounding in the subject. The discussion is informal, with an emphasis on moti-vation before rigor. The path of the text remains in the lowlands where all of thecentral ideas and arguments lie. Crossroads leading to higher ground and moreexotic topics are clearly marked, but those paths must be followed in the SuggestedReading and the Exercises that follow each chapter. We hope that this approachwill give a good perspective of the entire multigrid landscape.

Although we will frequently refer to the multigrid method, it has become clearthat multigrid is not a single method or even a family of methods. Rather, itis an entire approach to computational problem solving, a collection of ideas andattitudes, referred to by its chief developer Achi Brandt as multilevel methods.

Originally, multigrid methods were developed to solve boundary value problemsposed on spatial domains. Such problems are made discrete by choosing a set of gridpoints in the domain of the problem. The resulting discrete problem is a system ofalgebraic equations associated with the chosen grid points. In this way, a physicalgrid arises very naturally in the formulation of these boundary value problems.

More recently, these same ideas have been applied to a broad spectrum of prob-lems, many of which have no association with any kind of physical grid. The originalmultigrid approach has now been abstracted to problems in which the grids havebeen replaced by more general levels of organization. This wider interpretation ofthe original multigrid ideas has led to powerful new techniques with a remarkablerange of applicability.

Chapter 1 of the monograph presents the model problems to which multigridmethods were first applied. Chapter 2 reviews the classical iterative (relaxation)methods, a firm understanding of which is essential to the development of multigridconcepts. With an appreciation of how the conventional methods work and whythey fail, multigrid methods can be introduced as a natural remedy for restoringand improving the performance of the basic relaxation schemes. Chapters 3 and4 develop the fundamental multigrid cycling schemes and discuss issues of imple-mentation, complexity, and performance. Only in Chapter 5 do we turn to sometheoretical questions. By looking at multigrid from a spectral (Fourier mode) point

xi

xii Preface

of view and from an algebraic (subspace) point of view, it is possible to give anexplanation of why the basic multigrid cycling scheme works so effectively.

Not surprisingly, the body of multigrid literature is vast and continues to grow atan astonishing rate. The Suggested Reading list at the end of this tutorial [see thebibliography in the Second Edition] contains some of the more useful introductions,surveys, and classical papers currently available. This list is hardly exhaustive. Acomplete and cumulative review of the technical literature may be found in theMultigrid Bibliography (see Suggested Reading), which is periodically updated. Itseems unnecessary to include citations in the text of the monograph. The ideaspresented are elementary enough to be found in some form in many of the listedreferences.

Finally, it should be said that this monograph has been written by one whohas only recently worked through the basic ideas of multigrid. A beginner cannothave mastered the subtleties of a subject, but often has a better appreciation ofits difficulties. However, technical advice was frequently necessary. For this, Igreatly appreciate the guidance and numerous suggestions of Steve McCormick,who has mastered the subtleties of multigrid. I am grateful to John Bolstad formaking several valuable suggestions and an index for the second printing. Forthe fourth printing the Suggested Reading section has been enlarged to include sixrecently published books devoted to multigrid and multilevel methods. A genuinelynew development is the creation of mg-net, a bulletin board/newsgroup servicewhich is accessible by sending electronic mail to mgnet@cs.yale.edu. For the realproduction of this monograph, I am grateful for the typing skills of Anne VanLeeuwen and for the editorial assistance of Tricia Manning and Anne-Adele Wightat SIAM.

Chapter 1

Model Problems

Multigrid methods were originally applied to simple boundary value problems thatarise in many physical applications. For simplicity and for historical reasons, theseproblems provide a natural introduction to multigrid methods. As an example,consider the two-point boundary value problem that describes the steady-statetemperature distribution in a long uniform rod. It is given by the second-orderboundary value problem

−u′′(x) + σu(x) = f(x), 0 < x < 1, σ ≥ 0, (1.1)u(0) = u(1) = 0. (1.2)

While this problem can be handled analytically, our present aim is to considernumerical methods. Many such approaches are possible, the simplest of which is afinite difference method (finite element formulations will be considered in Chapter10). The domain of the problem x : 0 ≤ x ≤ 1 is partitioned into n subintervalsby introducing the grid points xj = jh, where h = 1/n is the constant width of thesubintervals. This establishes the grid shown in Fig. 1.1, which we denote Ωh.

At each of the n−1 interior grid points, the original differential equation (1.1) isreplaced by a second-order finite difference approximation. In making this replace-ment, we also introduce vj as an approximation to the exact solution u(xj). Thisapproximate solution may now be represented by a vector v = (v1, . . . , vn−1)T ,whose components satisfy the n − 1 linear equations

−vj−1 + 2vj − vj+1

h2+ σvj = f(xj), 1 ≤ j ≤ n − 1, (1.3)

v0 = vn = 0.

Defining f = (f(x1), . . . , f(xn−1))T = (f1, . . . , fn−1)T , the vector of right-sidevalues, we may also represent this system of linear equations in matrix form as

1h2

2 + σh2 −1−1 2 + σh2 −1

· · ·· · ·

· · −1−1 2 + σh2

v1

····

vn−1

=

f1

····

fn−1

1

2 Chapter 1

Ωh:x0 x1 x2 xj xn−1 xn

x = 0 x = 1

Figure 1.1: One-dimensional grid on the interval 0 ≤ x ≤ 1. The grid spacing ish = 1

n and the jth grid point is xj = jh for 0 ≤ j ≤ n.

y

x

Figure 1.2: Two-dimensional grid on the unit square. The solid dots indicate theunknowns that are related at a typical grid point by the discrete equations (1.5).

or even more compactly as Av = f . The matrix A is (n− 1)× (n− 1), tridiagonal,symmetric, and positive definite.

Analogously, it is possible to formulate a two-dimensional version of this prob-lem. Consider the second-order partial differential equation (PDE)

−uxx − uyy + σu = f(x, y), 0 < x < 1, 0 < y < 1, σ > 0. (1.4)

With σ = 0, this is the Poisson equation; with σ = 0, it is the Helmholtz equation.We consider this equation subject to the condition that u = 0 on the boundary ofthe unit square.

As before, this problem may be cast in a discrete form by defining the gridpoints (xi, yi) = (ihx, jhy), where hx = 1

m and hy = 1n . This two-dimensional grid

is also denoted Ωh and is shown in Fig. 1.2. Replacing the derivatives of (1.4) bysecond-order finite differences leads to the system of linear equations

−vi−1,j + 2vij − vi+1,j

h2x

+−vi,j−1 + 2vij − vi,j+1

h2y

+ σvij = fij ,

(1.5)vi0 = vin = v0j = vmj = 0, 1 ≤ i ≤ m − 1, 1 ≤ j ≤ n − 1.

As before, vij is an approximation to the exact solution u(xi, yj) and fij = f(xi, yj).There are now (m − 1)(n − 1) interior grid points and the same number of

unknowns in the problem. We can choose from many different orderings of theunknowns. For the moment, consider the lexicographic ordering by lines of constanti. The unknowns of the ith row of the grid may be collected in the vector vi =

A Multigrid Tutorial 3

(vi1, . . . , vi,n−1)T for 1 ≤ i ≤ m − 1. Similarly, let fi = (fi1, . . . , fi,n−1)T . Thesystem of equations (1.5) may then be given in block matrix form as

B −aI

−aI B −aI· · ·

· · −aI−aI B

v1

···

vm−1

=

f1···

fm−1

.

This system is symmetric, block tridiagonal, and sparse. It has block dimension(m−1)× (m−1). Each diagonal block, B, is an (n−1)× (n−1) tridiagonal matrixthat looks much like the matrix for the one-dimensional problem. Each off-diagonalblock is a multiple, a = 1

h2x, of the (n − 1) × (n − 1) identity matrix I.

Matrix Properties. The matrices produced by the discretization of self-adjoint boundary value problems have some special properties that are desirablefor many numerical methods. Let A with elements aij be such a matrix. It isgenerally symmetric (A = AT ) and sparse (a large percentage of the elementsare zero). These matrices are also often weakly diagonally dominant, whichmeans that, in magnitude, the diagonal element is at least as large as the sumof the off-diagonal elements in the same row:

n∑j =i

|aij | ≤ |aii| for 1 ≤ i ≤ n.

These matrices are also positive definite, which means that, for all vectors u = 0,we have uT Au > 0. This property is difficult to interpret, but there are severalalternate characterizations. For example, a symmetric positive definite matrixhas real and positive eigenvalues. It can also be shown that if A is symmetric anddiagonally dominant with positive diagonal elements, then A is positive definite.One other matrix property arises in the course of our work: a symmetric positivedefinite matrix with positive entries on the diagonal and nonpositive off-diagonalentries is called an M-matrix.

We occasionally appeal to stencils associated with discrete equations. For theone-dimensional model problem, the stencil representation of the matrix is

A =1h2

(−1 2 + σh2 − 1).

The two-dimensional stencil for hx = hy = h is

A =1h2

−1

−1 4 + σh2 −1−1

.

Stencils are useful for representing operators that interact locally on a grid. How-ever, they must be used with care near boundaries.

The two model linear systems (1.3) and (1.5) provide the testing ground formany of the methods discussed in the following chapters. Before we proceed, how-ever, it is useful to give a brief summary of existing methods for solving suchsystems.

4 Chapter 1

During the past 50 years, a tremendous amount of work was devoted to thenumerical solution of sparse systems of linear equations. Much of this attentionwas given to structured systems such as (1.3) and (1.5) that arise from boundaryvalue problems. Existing methods of solution fall into two large categories: directmethods and iterative (or relaxation) methods. This tutorial is devoted to the lattercategory.

Direct methods, of which Gaussian elimination is the prototype, determine asolution exactly (up to machine precision) in a finite number of arithmetic steps.For systems such as (1.5) that arise from a two-dimensional elliptic equation, veryefficient direct methods have been developed. They are usually based on the fastFourier transform or the method of cyclic reduction. When applied to problems onan n × n grid, these methods require O(n2 log n) arithmetic operations. Becausethey approach the minimum operation count of O(n2) operations, these methods arenearly optimal. However, they are also rather specialized and restricted primarilyto systems that arise from separable self-adjoint boundary value problems.

Relaxation methods, as represented by the Jacobi and Gauss–Seidel iterations,begin with an initial guess at a solution. Their goal is to improve the currentapproximation through a succession of simple updating steps or iterations. The se-quence of approximations that is generated (ideally) converges to the exact solutionof the linear system. Classical relaxation methods are easy to implement and maybe successfully applied to more general linear systems than most direct methods[23, 24, 26].

As we see in the next chapter, relaxation schemes suffer from some disabling lim-itations. Multigrid methods evolved from attempts to overcome these limitations.These attempts have been largely successful: used in a multigrid setting, relaxationmethods are competitive with the fast direct methods when applied to the modelproblems, and they have more generality and a wider range of application.

In Chapters 1–5 of this tutorial, we focus on the two model problems. In Chap-ters 6–10, we extend the basic multigrid methods to treat more general boundaryconditions, operators, and geometries. The basic methods can be applied to manyelliptic and other types of problems without significant modification. Still moreproblems can be treated with more sophisticated multigrid methods.

Finally, the original multigrid ideas have been extended to what are more ap-propriately called multilevel methods. Purely algebraic problems (for example, net-work and structural problems) have led to the development of algebraic multigrid orAMG, which is the subject of Chapter 8. Beyond the boundaries of this book, mul-tilevel methods have been applied to time-dependent problems and problems in im-age processing, control theory, combinatorial optimization (the traveling salesmanproblem), statistical mechanics (the Ising model), and quantum electrodynamics.The list of problems amenable to multilevel methods is long and growing. But firstwe must begin with the basics.

Exercises

1. Derivative (Neumann) boundary conditions. Consider model problem(1.1) subject to the Neumann boundary conditions u′(0) = u′(1) = 0. Findthe system of linear equations that results when second-order finite differencesare used to discretize this problem at the grid points x0, . . . , xn. At the end

A Multigrid Tutorial 5

points, x0 and xn, one of many ways to incorporate the boundary conditionsis to let v1 = v0 and vn−1 = vn. (We return to this problem in Chapter7.) How many equations and how many unknowns are there in this problem?Give the matrix that corresponds to this boundary value problem.

2. Ordering unknowns. Suppose the unknowns of system (1.5) are orderedby lines of constant j (or y). Give the block structure of the resulting matrixand specify the dimensions of the blocks.

3. Periodic boundary conditions. Consider model problem (1.1) subjectto the periodic boundary conditions u(0) = u(1) and u′(0) = u′(1). Find thesystem of linear equations that results when second-order finite differences areused to discretize this problem at the grid points x0, . . . , xn−1. How manyequations and unknowns are there in this problem?

4. Convection terms in two dimensions. A convection term can be addedto the two-dimensional model problem in the form

−ε(uxx + uyy) + aux = f(x).

Using the grid described in the text and second-order central finite differenceapproximations, find the system of linear equations associated with this prob-lem. What conditions must be met by a and ε for the associated matrix tobe diagonally dominant?

5. Three-dimensional problem. Consider the three-dimensional Poisson equa-tion

−uxx − uyy − uzz = f(x, y, z).

Write out the discrete equation obtained by using second-order central finitedifference approximations at the grid point (xi, yj , zk). Assuming that theunknowns are ordered first by lines of constant x, then lines of constant y,describe the block structure of the resulting matrix.

Chapter 2

Basic Iterative Methods

We now consider how model problems (1.3) and (1.5) might be treated using con-ventional iterative or relaxation methods. We first establish the notation for thisand all remaining chapters. Let

Au = f

denote a system of linear equations such as (1.3) or (1.6). We always use u todenote the exact solution of this system and v to denote an approximation to theexact solution, perhaps generated by some iterative method. Bold symbols, such asu and v, represent vectors, while the jth components of these vectors are denotedby uj and vj . In later chapters, we need to associate u and v with a particulargrid, say Ωh. In this case, the notation uh and vh is used.

Suppose that the system Au = f has a unique solution and that v is a computedapproximation to u. There are two important measures of v as an approximationto u. One is the error (or algebraic error) and is given simply by

e = u − v.

The error is also a vector and its magnitude may be measured by any of the standardvector norms. The most commonly used norms for this purpose are the maximum(or infinity) norm and the Euclidean or 2-norm, defined, respectively, by

‖e‖∞ = max1≤j≤n

|ej | and ‖e‖2 =

n∑j=1

e2j

1/2

.

Unfortunately, the error is just as inaccessible as the exact solution itself. How-ever, a computable measure of how well v approximates u is the residual, givenby

r = f − Av.

The residual is simply the amount by which the approximation v fails to satisfythe original problem Au = f . It is also a vector and its size may be measured bythe same norm used for the error. By the uniqueness of the solution, r = 0 if andonly if e = 0. However, it may not be true that when r is small in norm, e is alsosmall in norm.

7

8 Chapter 2

Residuals and Errors. A residual may be defined for any numerical approx-imation and, in many cases, a small residual does not necessarily imply a smallerror. This is certainly true for systems of linear equations, as shown by thefollowing two problems:(

1 −121 −20

) (u1

u2

)=

(−1−19

)and

(1 −13 −1

) (u1

u2

)=

(−11

).

Both systems have the exact solution u = (1, 2)T . Suppose we have computedthe approximation v = (1.95, 3)T . The error in this approximation is e =(−0.95,−1)T , for which ‖e‖2 = 1.379. The norm of the residual in v for thefirst system is ‖r1‖2 = 0.071, while the residual norm for the second system is‖r2‖2 = 1.851. Clearly, the relatively small residual for the first system doesnot reflect the rather large error. See Exercise 18 for an important relationshipbetween error and residual norms.

Remembering that Au = f and using the definitions of r and e, we can derivean extremely important relationship between the error and the residual (Exercise2):

Ae = r.

We call this relationship the residual equation. It says that the error satisfies thesame set of equations as the unknown u when f is replaced by the residual r. Theresidual equation plays a vital role in multigrid methods and it is used repeatedlythroughout this tutorial.

We can now anticipate, in an imprecise way, how the residual equation can beused to great advantage. Suppose that an approximation v has been computedby some method. It is easy to compute the residual r = f − Av. To improve theapproximation v, we might solve the residual equation for e and then compute anew approximation using the definition of the error

u = v + e.

In practice, this method must be applied more carefully than we have indicated.Nevertheless, this idea of residual correction is very important in all that follows.

We now turn to relaxation methods for our first model problem (1.3) with σ = 0.Multiplying that equation by h2 for convenience, the discrete problem becomes

−uj−1 + 2uj − uj+1 = h2fj , 1 ≤ j ≤ n − 1,

u0 = un = 0. (2.1)

One of the simplest schemes is the Jacobi (or simultaneous displacement) method.It is produced by solving the jth equation of (2.1) for the jth unknown and usingthe current approximation for the (j−1)st and (j +1)st unknowns. Applied to thevector of current approximations, this produces an iteration scheme that may bewritten in component form as

v(1)j =

12(v(0)j−1 + v

(0)j+1 + h2fj

), 1 ≤ j ≤ n − 1.

A Multigrid Tutorial 9

To keep the notation as simple as possible, the current approximation (or theinitial guess on the first iteration) is denoted v(0), while the new, updated approx-imation is denoted v(1). In practice, once all of the v(1) components have beencomputed, the procedure is repeated, with v(1) playing the role of v(0). These iter-ation sweeps are continued until (ideally) convergence to the solution is obtained.

It is important to express these relaxation schemes in matrix form, as well ascomponent form. We split the matrix A in the form

A = D − L − U,

where D is the diagonal of A, and −L and −U are the strictly lower and uppertriangular parts of A, respectively. Including the h2 term in the vector f, thenAu = f becomes

(D − L − U)u = f .

Isolating the diagonal terms of A, we have

Du = (L + U)u + f

oru = D−1(L + U)u + D−1f .

Multiplying by D−1 corresponds exactly to solving the jth equation for uj , for1 ≤ j ≤ n − 1. If we define the Jacobi iteration matrix by

RJ = D−1(L + U),

then the Jacobi method appears in matrix form as

v(1) = RJv(0) + D−1f .

There is a simple but important modification that can be made to the Jacobiiteration. As before, we compute the new Jacobi iterates using

v∗j =12(v(0)j−1 + v

(0)j+1 + h2fj

), 1 ≤ j ≤ n − 1.

However, v∗j is now only an intermediate value. The new iterate is given by theweighted average

v(1)j = (1 − ω)v(0)

j + ωv∗j = v

(0)j + ω(v∗j − v

(0)j ), 1 ≤ j ≤ n − 1,

where ω ∈ R is a weighting factor that may be chosen. This generates an entirefamily of iterations called the weighted or damped Jacobi method. Notice that ω = 1yields the original Jacobi iteration.

In matrix form, the weighted Jacobi method is given by (Exercise 3)

v(1) = [(1 − ω)I + ωRJ ]v(0) + ωD−1f .

If we define the weighted Jacobi iteration matrix by

Rω = (1 − ω)I + ωRJ ,

10 Chapter 2

then the method may be expressed as (Exercise 3)

v(1) = Rωv(0) + ωD−1f .

We should note in passing that the weighted Jacobi iteration can also be writtenin the form (Exercise 3)

v(1) = v(0) + ωD−1r(0).

This says that the new approximation is obtained from the current one by addingan appropriate weighting of the residual.

This is just one example of a stationary linear iteration. This term refers to thefact that the update rule is linear in the unknown v and does not change from oneiteration to the next. We can say more about such iterations in general. Recallingthat e = u − v and Ae = r, we have

u − v = A−1r.

Identifying v with the current approximation v(0) and u with the new approxima-tion v(1), an iteration may be formed by taking

v(1) = v(0) + Br(0), (2.2)

where B is an approximation to A−1. If B can be chosen “close” to A−1, then theiteration should be effective.

It is useful to examine this general form of iteration a bit further. Rewritingexpression (2.2), we see that

v(1) = v(0) + Br(0) = v(0) + B(f − Av(0))= (I − BA)v(0) + Bf

≡ Rv(0) + Bf ,

where we have defined the general iteration matrix as R = I − BA. It can also beshown (Exercise 4) that m sweeps of this iteration result in

v(m) = Rmv(0) + C(f),

where C(f) represents a series of operations on f . We return to this general formin Chapter 5.

Before analyzing or implementing these methods, we present a few more ofthe basic iterative schemes. Weighted Jacobi computes all components of the newapproximation before using any of them. This requires 2n storage locations for theapproximation vector. It also means that new information cannot be used as soonas it is available.

The Gauss–Seidel method incorporates a simple change: components of the newapproximation are used as soon as they are computed. This means that componentsof the approximation vector v are overwritten as soon as they are updated. Thissmall change reduces the storage requirement for the approximation vector to onlyn locations. The Gauss–Seidel method is also equivalent to successively settingeach component of the residual vector to zero and solving for the correspondingcomponent of the solution (Exercise 5). When applied to the model problem, thismethod may be expressed in component form as

vj ←− 12(vj−1 + vj+1 + h2fj

), 1 ≤ j ≤ n − 1,

where the arrow notation stands for replacement or overwriting.

A Multigrid Tutorial 11

Once again it is useful to express this method in matrix form. Splitting thematrix A in the form A = D − L − U , we can now write the original system ofequations as

(D − L)u = Uu + f

oru = (D − L)−1Uu + (D − L)−1f .

This representation corresponds to solving the jth equation for uj and using newapproximations for components 1, 2, . . . , j − 1. Defining the Gauss–Seidel iterationmatrix by

RG = (D − L)−1U,

we can express the method as

v ←− RGv + (D − L)−1f .

Finally, we look at one important variation on the Gauss–Seidel iteration. Forweighted Jacobi, the order in which the components of v are updated is immaterial,since components are never overwritten. However, for Gauss–Seidel, the order ofupdating is significant. Instead of sweeping through the components (equivalently,the grid points) in ascending order, we could sweep through the components indescending order or we might alternate between ascending and descending orders.The latter procedure is called the symmetric Gauss–Seidel method.

Another effective alternative is to update all the even components first by theexpression

v2j ←− 12(v2j−1 + v2j+1 + h2f2j

),

and then update all the odd components using

v2j+1 ←− 12(v2j + v2j+2 + h2f2j+1

).

This strategy leads to the red-black Gauss–Seidel method, which is illustrated inFig. 2.1 for both one-dimensional and two-dimensional grids. Notice that the redpoints correspond to even-indexed points in one dimension and to points whoseindex sum is even in two dimensions (assuming that i = 0 and j = 0 correspondsto a boundary). The red points also correspond to what we soon call coarse-gridpoints.

The advantages of red-black over regular Gauss–Seidel are not immediatelyapparent; the issue is often problem-dependent. However, red-black Gauss–Seideldoes have a clear advantage in terms of parallel computation. The red points needonly the black points for their updating and may therefore be updated in any order.This work represents n

2 (or n2

2 in two dimensions) independent tasks that can bedistributed among several independent processors. In a similar way, the black sweepcan also be done by several independent processors. (The Jacobi iteration is alsowell-suited to parallel computation.)

There are many more basic iterative methods. However, we have seen enoughof the essential methods to move ahead toward multigrid. First, it is important togain some understanding of how these basic iterations perform. We proceed bothby analysis and by experimentation.

When studying stationary linear iterations, it is sufficient to work with thehomogeneous linear system Au = 0 and use arbitrary initial guesses to start the

12 Chapter 2

y

x

Figure 2.1: A one-dimensional grid (top) and a two-dimensional grid (bottom),showing the red points () and the black points (•) for red-black relaxation.

relaxation scheme (Exercise 6). One reason for doing this is that the exact solutionis known (u = 0) and the error in an approximation v is simply −v. Therefore, wereturn to the one-dimensional model problem with f = 0. It appears as

−uj−1 + 2uj − uj+1 = 0, 1 ≤ j ≤ n − 1,

u0 = un = 0. (2.3)

We obtain some valuable insight by applying various iterations to this systemof equations with an initial guess consisting of the vectors (or Fourier modes)

vj = sin(

jkπ

n

), 0 ≤ j ≤ n, 1 ≤ k ≤ n − 1.

Recall that j denotes the component (or associated grid point) of the vector v. Theinteger k now makes its first appearance. It is called the wavenumber (or frequency)and it indicates the number of half sine waves that constitute v on the domain ofthe problem. We use vk to designate the entire vector v with wavenumber k.Figure 2.2 illustrates initial guesses v1, v3, and v6. Notice that small values ofk correspond to long, smooth waves, while large values of k correspond to highlyoscillatory waves. We now explore how Fourier modes behave under iteration.

We first apply the weighted Jacobi iteration with ω = 23 to problem (2.3) on

a grid with n = 64 points. Beginning with initial guesses of v1, v3, and v6, theiteration is applied 100 times. Recall that the error is just −v. Figure 2.3(a) showsa plot of the maximum norm of the error versus the iteration number.

For the moment, only the qualitative behavior of the iteration is important. Theerror clearly decreases with each relaxation sweep and the rate of decrease is largerfor the higher wavenumbers. Figures 2.3(b, c) show analogous plots for the regularand red-black Gauss–Seidel iterations, where we see a similar relationship amongthe error, the number of iterations, and the wavenumber. (The complete situationis not quite so simple with red-black Gauss–Seidel, as illustrated in Exercise 20.)

A Multigrid Tutorial 13

Figure 2.2: The modes vj = sin(

jkπn

), 0 ≤ j ≤ n, with wavenumbers k = 1, 3, 6.

The kth mode consists of k2 full sine waves on the interval.

The experiment of Fig. 2.3(a) is presented in a slightly different light in Fig.2.4. In this figure, the log of the maximum norm of the error for the weightedJacobi method is plotted against the iteration number for various wavenumbers.This plot clearly shows a linear decrease in the log of the error norm, indicatingthat the error itself decreases geometrically with each iteration. If we let e(0) be theerror in the initial guess and e(m) be the error in the mth iterate, then we mightexpect to describe the error by a relationship of the form

‖e(m)‖∞ = cmk ‖e(0)‖∞,

where ck is a constant that depends on the wavenumber. We will see that thetheory confirms this conjecture.

In general, most initial guesses (or, equivalently, most right-side vectors f) wouldnot consist of a single mode. Consider a slightly more realistic situation in whichthe initial guess (hence, the error) consists of three modes: a low-frequency wave(k = 1), a medium-frequency wave (k = 6), and a high-frequency wave (k = 32) ona grid with n = 64 points; it is given by

vj =13

[sin

(jπ

n

)+ sin

(6jπ

n

)+ sin

(32jπ

n

)].

Figure 2.5 shows the maximum norm of the error plotted against the number ofiterations. The error decreases rapidly within the first five iterations, after whichit decreases much more slowly. The initial decrease corresponds to the quick elim-ination of the high-frequency modes of the error. The slow decrease is due to thepresence of persistent low-frequency modes. The important observation is that thestandard iterations converge very quickly as long as the error has high-frequencycomponents. However, the slower elimination of the low-frequency componentsdegrades the performance of these methods.

With some experimental evidence in hand, we now turn to a more analyticalapproach. Each of the methods discussed so far may be represented in the form

v(1) = Rv(0) + g,

14 Chapter 2

Figure 2.3: (a) Weighted Jacobi iteration with ω = 23 , (b) regular Gauss–Seidel

iteration, and (c) red-black Gauss–Seidel iteration applied to the one-dimensionalmodel problem with n = 64 points and with initial guesses v1, v3, and v6. Themaximum norm of the error, ‖e‖∞, is plotted against the iteration number for 100iterations.

A Multigrid Tutorial 15

Figure 2.4: Weighted Jacobi iteration with ω = 23 applied to the one-dimensional

model problem with n = 64 points and with initial guesses v1, v3, and v6. The logof ‖e‖∞ is plotted against the iteration number for 100 iterations.

Figure 2.5: Weighted Jacobi method with ω = 23 applied to the one-dimensional

model problem with n = 64 points and an initial guess (v1 + v6 + v32)/3. Themaximum norm of the error, ‖e‖∞, is plotted against the iteration number for 100iterations.

16 Chapter 2

where R is one of the iteration matrices derived earlier. Furthermore, all of thesemethods are designed such that the exact solution, u, is a fixed point of the iteration(Exercise 4). This means that iteration does not change the exact solution:

u = Ru + g.

Subtracting these last two expressions, we find that

e(1) = Re(0).

Repeating this argument, it follows that after m relaxation sweeps, the error inthe mth approximation is given by

e(m) = Rme(0).

Matrix Norms. Matrix norms can be defined in terms of the commonly usedvector norms. Let A be an n×n matrix with elements aij . Consider the vectornorm ‖x‖p defined by

‖x‖p =

(n∑

i=1

|xi|p)1/p

, 1 ≤ p < ∞,

‖x‖∞ = sup1≤i≤n

|xi|.

The matrix norm induced by the vector norm ‖ · ‖p is defined by

‖A‖p = supx=0

‖Ax‖p

‖x‖p.

While not obvious without some computation, this definition leads to the fol-lowing matrix norms induced by the vector norms ‖ · ‖1, ‖ · ‖∞, and ‖ · ‖2:

‖A‖1 = maxj

∑ni=1 |aij | (maximum column sum),

‖A‖∞ = maxi

∑nj=1 |aij | (maximum row sum),

‖A‖2 =√

spectral radius of AT A.Recall that the spectral radius of a matrix is given by

ρ(A) = max |λ(A)|,

where λ(A) denotes the eigenvalues of A. For symmetric matrices, the matrix2-norm is just the spectral radius of A:

‖A‖2 =√

ρ(AT A) =√

ρ(A2) = ρ(A).

If we now choose a particular vector norm and its associated matrix norm, it ispossible to bound the error after m iterations by

‖e(m)‖ ≤ ‖R‖m‖e(0)‖.

A Multigrid Tutorial 17

This leads us to conclude that if ‖R‖ < 1, then the error is forced to zero as theiteration proceeds.

It is shown in many standard texts [9, 20, 24, 26] that

limm→∞

Rm = 0 if and only if ρ(R) < 1.

Therefore, it follows that the iteration associated with the matrix R converges forall initial guesses if and only if ρ(R) < 1.

The spectral radius ρ(R) is also called the asymptotic convergence factor whenit appears in the context of iterative methods. It has some useful interpretations.First, it is roughly the worst factor by which the error is reduced with each relax-ation sweep. By the following argument, it also tells us approximately how manyiterations are required to reduce the error by a factor of 10−d. Let m be the smallestinteger that satisfies

‖e(m)‖‖e(0)‖ ≤ 10−d.

This condition will be approximately satisfied if

[ρ(R)]m ≤ 10−d.

Solving for m, we have

m ≥ − d

log10[ρ(R)].

The quantity − log10(ρ(R)) is called the asymptotic convergence rate. Its reciprocalgives the approximate number of iterations required to reduce the error by onedecimal digit. We see that as ρ(R) approaches 1, the convergence rate decreases.Small values of ρ(R) (that is, ρ(R) positive and near zero) give a high convergencerate.

We have established the importance of the spectral radius of the iteration matrixin analyzing the convergence properties of relaxation methods. Now it is time tocompute some spectral radii. Consider the weighted Jacobi iteration applied to theone-dimensional model problem. Recalling that Rω = (1 − ω)I + ωRJ , we have(Exercise 3)

Rω = I − ω

2

2 −1−1 2 −1

· · ·· · ·

· · −1−1 2

.

Written in this form, it follows that the eigenvalues of Rω and A are related by

λ(Rω) = 1 − ω

2λ(A).

The problem becomes one of finding the eigenvalues of the original matrix A. Thisuseful exercise (Exercise 8) may be done in several different ways. The result is

18 Chapter 2

Interpreting the Spectral Radius. The spectral radius is considered to bean asymptotic measure of convergence because it predicts the worst-case errorreduction over many iterations. It can be shown [9, 20] that, in any vectornorm,

ρ(R) = limm→∞

‖Rm‖1/m.

Therefore, in terms of error reduction, we have

ρ(R) = limm→∞

supe(0)

(‖e(m)‖‖e(0)‖

)1/m

.

However, the spectral radius does not, in general, predict the behavior of theerror norm for a single iteration. For example, consider the matrix

R =(

0 1000 0

).

Clearly, ρ(R) = 0. But if we start with e(0) = (0, 1)T and compute e(1) = Re(0),then the convergence factor is

‖e(1)‖2

‖e(0)‖2= 100.

The next iterate achieves the asymptotic estimate, ρ(R) = 0, because e(2) = 0.A better worst-case estimate of error reduction for one or a few iterations isgiven by the matrix norm ‖R‖2. For the above example, we have ‖R‖2 =100. The discrepancy between the asymptotic convergence factor, ρ(R), andthe worst-case estimate, ‖R‖2, disappears when R is symmetric because thenρ(R) = ‖R‖2.

that the eigenvalues of A are

λk(A) = 4 sin2

(kπ

2n

), 1 ≤ k ≤ n − 1.

Also of interest are the corresponding eigenvectors of A. In all that follows, welet wk,j be the jth component of the kth eigenvector, wk. The eigenvectors of Aare then given by (Exercise 9)

wk,j = sin(

jkπ

n

), 1 ≤ k ≤ n − 1, 0 ≤ j ≤ n.

We see that the eigenvectors of A are simply the Fourier modes discussed earlier.With these results, we find that the eigenvalues of Rω are

λk(Rω) = 1 − 2ω sin2

(kπ

2n

), 1 ≤ k ≤ n − 1,

while the eigenvectors of Rω are the same as the eigenvectors of A (Exercise 10). Itis important to note that if 0 < ω ≤ 1, then |λk(Rω)| < 1 and the weighted Jacobi

A Multigrid Tutorial 19

iteration converges. We return to these convergence properties in more detail aftera small detour.

The eigenvectors of the matrix A are important in much of the following discus-sion. They correspond very closely to the eigenfunctions of the continuous modelproblem. Just as we can expand fairly arbitrary functions using this set of eigenfunc-tions, it is also possible to expand arbitrary vectors in terms of a set of eigenvectors.Let e(0) be the error in an initial guess used in the weighted Jacobi method. Thenit is possible to represent e(0) using the eigenvectors of A in the form

e(0) =n−1∑k=1

ckwk,

where the coefficients ck ∈ R give the “amount” of each mode in the error. Wehave seen that after m sweeps of the iteration, the error is given by

e(m) = Rmω e(0).

Using the eigenvector expansion for e(0), we have

e(m) = Rmω e(0) =

n−1∑k=1

ckRmω wk =

n−1∑k=1

ckλmk (Rω)wk.

The last equality follows because the eigenvectors of A and Rω are the same; there-fore, Rωwk = λk(Rω)wk.

This expansion for e(m) shows that after m iterations, the kth mode of theinitial error has been reduced by a factor of λm

k (Rω). It should also be noted thatthe weighted Jacobi method does not mix modes: when applied to a single mode,the iteration can change the amplitude of that mode, but it cannot convert thatmode into different modes. In other words, the Fourier modes are also eigenvectorsof the iteration matrix. As we will see, this property is not shared by all stationaryiterations.

To develop some familiarity with these Fourier modes, Fig. 2.6 shows them ona grid with n = 12 points. Notice that the kth mode consists of k

2 full sine wavesand has a wavelength of = 24h

k = 2k (the entire interval has length 1). The k = n

2mode has a wavelength of = 4h and the k = n − 1 mode has a wavelength ofalmost = 2h. Waves with wavenumbers greater than n (wavelengths less than 2h)cannot be represented on the grid. In fact (Exercise 12), through the phenomenonof aliasing, a wave with a wavelength less than 2h actually appears on the grid witha wavelength greater than 2h.

At this point, it is important to establish some terminology that is used through-out the remainder of the tutorial. We need some qualitative terms for the variousFourier modes that have been discussed. The modes in the lower half of the spec-trum, with wavenumbers in the range 1 ≤ k < n

2 , are called low-frequency or smoothmodes. The modes in the upper half of the spectrum, with n

2 ≤ k ≤ n − 1, arecalled high-frequency or oscillatory modes.

Having taken this excursion through Fourier modes, we now return to the anal-ysis of the weighted Jacobi method. We established that the eigenvalues of theiteration matrix are given by

λk(Rω) = 1 − 2ω sin2

(kπ

2n

), 1 ≤ k ≤ n − 1.

What choice of ω gives the best iterative scheme?

20 Chapter 2

Figure 2.6: Graphs of the Fourier modes of A on a grid with n = 12 points. Modeswith wavenumbers k = 1, 2, 3, 4, 6, 8, 9 are shown. The wavelength of the kth modeis = 24h

k .

A Multigrid Tutorial 21

Figure 2.7: Eigenvalues of the iteration matrix Rω for ω = 13 , 1

2 , 23 , 1. The

eigenvalues λk = 1− 2ω sin2(

kπ2n

)are plotted as if k were a continuous variable on

the interval 0 ≤ k ≤ n. In fact, 1 ≤ k ≤ n − 1 takes only integer values.

Recall that for 0 < ω ≤ 1, we have |λk(Rω)| < 1. We would like to find thevalue of ω that makes |λk(Rω)| as small as possible for all 1 ≤ k ≤ n − 1. Figure2.7 is a plot of the eigenvalues λk for four different values of ω. Notice that for allvalues of ω satisfying 0 < ω ≤ 1,

λ1 = 1 − 2ω sin2( π

2n

)= 1 − 2ω sin2

(πh

2

)≈ 1 − ωπ2h2

2.

This fact implies that λ1, the eigenvalue associated with the smoothest mode, willalways be close to 1. Therefore, no value of ω will reduce the smooth componentsof the error effectively. Furthermore, the smaller the grid spacing h, the closer λ1 isto 1. Any attempt to improve the accuracy of the solution (by decreasing the gridspacing) will only worsen the convergence of the smooth components of the error.Most basic relaxation schemes share this ironic limitation.

Having accepted the fact that no value of ω damps the smooth componentssatisfactorily, we ask what value of ω provides the best damping of the oscillatorycomponents (those with n

2 ≤ k ≤ n − 1). We could impose this condition byrequiring that

λn/2(Rω) = −λn(Rω).

Solving this equation for ω leads to the optimal value ω = 23 .

We also find (Exercise 13) that with ω = 23 , |λk| < 1

3 for all n2 ≤ k ≤ n − 1.

This says that the oscillatory components are reduced at least by a factor of threewith each relaxation. This damping factor for the oscillatory modes is an important

22 Chapter 2

property of any relaxation scheme and is called the smoothing factor of the scheme.An important property of the basic relaxation scheme that underlies much of thepower of multigrid methods is that the smoothing factor is not only small, but alsoindependent of the grid spacing h.

We now turn to some numerical experiments to illustrate the analytical resultsthat have just been obtained. Once again, the weighted Jacobi method is appliedto the one-dimensional model problem Au = 0 on a grid with n = 64 points. Weuse initial guesses (which are also initial errors) consisting of single modes withwavenumbers 1 ≤ k ≤ n − 1. Figure 2.8 shows how the method performs in termsof different wavenumbers. Specifically, the wavenumber of the initial error is plottedagainst the number of iterations required to reduce the norm of the initial error bya factor of 100. This experiment is done for weighting factors of ω = 1 and ω = 2

3 .With ω = 1, both the high- and low-frequency components of the error are

damped very slowly. Components with wavenumbers near n2 are damped rapidly.

This behavior is consistent with the eigenvalue curves of Fig. 2.7. We see a quitedifferent behavior in Fig. 2.8(b) with ω = 2

3 . Recall that ω = 23 was chosen to

give preferential damping to the oscillatory components. Indeed, the smooth wavesare damped very slowly, while the upper half of the spectrum (k ≥ n

2 ) shows rapidconvergence. Again, this is consistent with Fig. 2.7.

Another perspective on these convergence properties is provided in Figure 2.9.This time the actual approximations are plotted. The weighted Jacobi methodwith ω = 2

3 is applied to the same model problem on a grid with n = 64 points.Figure 2.9(a) shows the error with wavenumber k = 3 after one relaxation sweep(left plot) and after 10 relaxation sweeps (right plot). This smooth component isdamped very slowly. Figure 2.9(b) shows a more oscillatory error (k = 16) after oneand after 10 iterations. The damping is now much more dramatic. Notice also, asmentioned before, that the weighted Jacobi method preserves modes: once a k = 3mode, always a k = 3 mode.

Figure 2.9(c) illustrates the selectivity of the damping property. This experi-ment uses an initial guess consisting of two modes with k = 2 and k = 16. After 10relaxation sweeps, the high-frequency modulation on the long wave has been nearlyeliminated. However, the original smooth component persists.

We have belabored the discussion of the weighted Jacobi method because it iseasy to analyze and because it shares many properties with other basic relaxationschemes. In much less detail, let us look at the Gauss–Seidel iteration. We canshow (Exercise 14) that the Gauss–Seidel iteration matrix for the model problem(matrix A) has eigenvalues

λk(RG) = cos2(

kπ

n

), 1 ≤ k ≤ n − 1.

These eigenvalues, which are plotted in Fig. 2.10, must be interpreted carefully.We see that when k is close to 1 or n, the corresponding eigenvalues are close to 1and convergence is slow. However, the eigenvectors of RG are given by (Exercise14)

wk,j =[cos

(kπ

n

)]j

sin(

jkπ

n

),

where 0 ≤ j ≤ n and 1 ≤ k ≤ n − 1. These eigenvectors do not coincide with theeigenvectors of A. Therefore, λk(RG) gives the convergence rate, not for the kthmode of A, but for the kth eigenvector of RG.

A Multigrid Tutorial 23

Figure 2.8: Weighted Jacobi method with (a) ω = 1 and (b) ω = 23 applied to the

one-dimensional model problem with n = 64 points. The initial guesses consist ofthe modes wk for 1 ≤ k ≤ 63. The graphs show the number of iterations requiredto reduce the norm of the initial error by a factor of 100 for each wk. Note thatfor ω = 2

3 , the damping is strongest for the oscillatory modes (32 ≤ k ≤ 63).

24 Chapter 2

Figure 2.9: Weighted Jacobi method with ω = 23 applied to the one-dimensional

model problem with n = 64 points and with an initial guess consisting of (a) w3,(b) w16, and (c) (w2 + w16)/2. The figures show the approximation after oneiteration (left side) and after 10 iterations (right side).

This distinction is illustrated in Fig. 2.11. As before, the wavenumber k isplotted against the number of iterations required to reduce the norm of the initialerror by a factor of 100. In Fig. 2.11(a), the initial guess (and error) consists ofthe eigenvectors of RG with wavenumbers 1 ≤ k ≤ 63. The graph looks similarto the eigenvalue graph of Fig. 2.10. In Fig. 2.11(b), the initial guess consists ofthe eigenvectors of the original matrix A. The structure of this graph would bemuch more difficult to anticipate analytically. We see that when convergence of theGauss–Seidel method is described in terms of the modes of A, then once again thesmooth modes are damped slowly, while the oscillatory modes show rapid decay.

We have looked in detail at the convergence properties of some basic relax-ation schemes. The experiments we presented reflect the experience of many practi-

A Multigrid Tutorial 25

Figure 2.10: Eigenvalues of the Gauss–Seidel iteration matrix. The eigenvaluesλk = cos2

(kπn

)are plotted as if k were a continuous variable on the interval 0 ≤

k ≤ n.

tioners. These schemes work very well for the first several iterations. Inevitably,however, convergence slows and the entire scheme appears to stall. We have founda simple explanation for this phenomenon: the rapid decrease in error during theearly iterations is due to the efficient elimination of the oscillatory modes of thaterror; but once the oscillatory modes have been removed, the iteration is much lesseffective in reducing the remaining smooth components.

There is also a good physical explanation for why smooth error modes are soresistant to relaxation. Recall from (2.2) that stationary linear iterations can bewritten in the form

v(1) = v(0) + Br(0).

Subtracting this equation from the exact solution u, the error at the next step is

e(1) = e(0) − Br(0).

We see that changes in the error are made with spatially local corrections expressedthrough the residual. If the residual is small relative to the error itself, then changesin the error will be correspondingly small. At least for the model problems we haveposed, smooth error modes have relatively small residuals (Exercise 19), so theerror decreases slowly. Conversely, oscillatory errors tend to have relatively largeresiduals and the corrections to the error with a single relaxation sweep can besignificant.

Many relaxation schemes possess this property of eliminating the oscillatorymodes and leaving the smooth modes. This so-called smoothing property is a seriouslimitation of conventional relaxation methods. However, this limitation can beovercome and the remedy is one of the pathways to multigrid.

In one very brief chapter, we have barely touched upon the wealth of lore andtheory surrounding iterative methods. The subject constitutes a large and impor-tant domain of classical numerical analysis. It is also filled with very elegant math-ematics from both linear algebra and analysis. However, esoteric iterative methodsare not required for the development of multigrid. The most effective multigridtechniques are usually built upon the simple relaxation schemes presented in thischapter. We now use these few basic schemes and develop them into far morepowerful methods.

26 Chapter 2

Figure 2.11: Gauss–Seidel iteration matrix applied to the model problem with n = 64points. The initial guesses consist of (a) the eigenvectors of the iteration matrixRG with wavenumbers 1 ≤ k ≤ 63 and (b) the eigenvectors of A with wavenumbers1 ≤ k ≤ 63. The figure shows the number of iterations required to reduce the normof the initial error by a factor of 100 for each initial guess.

A Multigrid Tutorial 27

Exercises

1. Residual vs. error. Consider the two systems of linear equations given inthe box on residuals and errors in this chapter. Make a sketch showing thepair of lines represented by each system. Mark the exact solution u and theapproximation v. Explain why, even though the error is the same in bothcases, the residual is small in one case and large in the other.

2. Residual equation. Use the definition of the algebraic error and the residualto derive the residual equation Ae = r.

3. Weighted Jacobi iteration.

(a) Starting with the component form of the weighted Jacobi method, showthat it can be written in matrix form as v(1) = [(1 − ω)I + ωRJ ]v(0) +ωD−1f .

(b) Show that the weighted Jacobi method may also be written in the form

v(1) = Rωv(0) + ωD−1f .

(c) Show that the weighted Jacobi iteration may also be expressed in theform

v(1) = v(0) + ωD−1r(0),

where r(0) is the residual associated with the approximation v(0).

(d) Assume that A is the matrix associated with the model problem. Showthat the weighted Jacobi iteration matrix can be expressed as

Rω = I − ω

2A.

4. General stationary linear iteration. It was shown that a general station-ary linear iteration can be expressed in the form

v(1) = (I − BA)v(0) + Bf ≡ Rv(0) + Bf .

(a) Show that m sweeps of the iteration has the form

v(1) = Rmv(0) + C(f).

Find an expression for C(f).

(b) Show that the form of the iteration given above is equivalent to

v(1) = v(0) + Br(0),

where r(0) is the initial residual. Use this form to argue that the exactsolution to the linear system, u, is unchanged by (and is therefore a fixedpoint of) the iteration.

5. Interpreting Gauss–Seidel. Show that the Gauss–Seidel iteration is equiv-alent to successively setting each component of the residual to zero.

6. Zero right side. Argue that in analyzing the error in a stationary linearrelaxation scheme applied to Au = f , it is sufficient to consider Au = 0 witharbitrary initial guesses.

28 Chapter 2

7. Asymptotic convergence rate. Explain why the asymptotic convergencerate,

− log10 ρ(R),

is positive. Which iteration matrix gives a higher asymptotic convergencerate: one with ρ(R) = 0.1 or one with ρ(R) = 0.9? Explain.

8. Eigenvalues of the model problem. Compute the eigenvalues of thematrix A of the one-dimensional model problem. (Hint: Write out a typicalequation of the system Aw = λw with w0 = wn = 0. Notice that vectors ofthe form wj = sin

(jkπn

), 1 ≤ k ≤ n − 1, 0 ≤ j ≤ n, satisfy the boundary

conditions.) How many distinct eigenvalues are there? Compute λ1, λ2, λn−2,λn−1 when n = 32.

9. Eigenvectors of the model problem. Using the results of the previousproblem, find the eigenvectors of the one-dimensional model problem matrixA.

10. Jacobi eigenvalues and eigenvectors. Find the eigenvalues of the weightedJacobi iteration matrix when it is applied to the one-dimensional model prob-lem matrix A. Verify that the eigenvectors of Rω are the same as the eigen-vectors of A.

11. Fourier modes. Consider the interval 0 ≤ x ≤ 1 with grid points xj = jn ,

0 ≤ j ≤ n. Show that the kth Fourier mode wk,j = sin(

jkπn

)has wavelength

= 2k . Which mode has wavelength = 8h? Which mode has wavelength

= 14?

12. Aliasing. On a grid with n − 1 interior points, show that the mode wk,j =

sin(

jkπn

)with n < k < 2n is actually represented as the mode wk′ where

k′ = 2n − k. How is the mode with wavenumber k = 3n2 represented on the

grid? How is the mode with wavelength l = 4h3 represented on the grid? Make

sketches for these two examples.

13. Optimal Jacobi. Show that when the weighted Jacobi method is used withω = 2

3 , the smoothing factor is 13 . Show that if ω is chosen to damp the

smooth modes effectively, then the oscillatory modes are actually amplified.

14. Gauss–Seidel eigenvalues and eigenvectors.

(a) Show that the eigenvalue problem for the Gauss–Seidel iteration matrix,RGw = λw, may be expressed in the form Uw = (D − L)λIw, whereU , L, D are defined in the text.

(b) Write out the equations of this system and note the boundary conditionw0 = wn = 0. Look for solutions of this system of equations of the formwj = µj , where µ ∈ C must be determined. Show that the boundaryconditions can be satisfied only if λ = λk = cos2

(kπn

), 1 ≤ k ≤ n − 1.

(c) Show that the eigenvector associated with λk is wk,j = cos(kπn ) sin

(jkπn

).

15. Richardson iteration.

(a) Recall that for real vectors u, v, the inner product is given by (u,v) =uT v and ‖u‖2

2 = (u,u). Furthermore, if A is symmetric positive definite,

A Multigrid Tutorial 29

then ‖A‖2 = ρ(A), the spectral radius of A. Richardson’s iteration isgiven by

v(1) = v(0) +s

‖A‖2r(0) for 0 < s < 2,

where r(0) = f −Av(0) is the residual. Show that when A has a constantdiagonal, this method reduces to the weighted Jacobi method.

(b) Show that the error after one sweep of Richardson’s method is governedby

‖e(1)‖22 ≤

[1 − s(2 − s)(Ae(0), e(0))

‖A‖2 (e(0), e(0))

]‖e(0)‖2

2.

(c) If the eigenvalues of A are ordered 0 < λ1 < λ2 < · · · < λn andthe smallest eigenvalues correspond to the smooth modes, show thatRichardson’s method has the smoothing property. (Use the fact thatthe eigenvalues are given by the Rayleigh quotients of the eigenvectors,λk = (Awk,wk)/(wk,wk), where wk is the eigenvector associated withλk.)

16. Properties of Gauss–Seidel. Assume A is symmetric, positive definite.

(a) Show that the jth step of a single sweep of the Gauss–Seidel methodapplied to Au = f may be expressed as

vj ← vj +rj

ajj.

(b) Show that the jth step of a single sweep of the Gauss–Seidel method canbe expressed in vector form as

v ← v +(r, ej)

(Aej , ej)ej , 1 ≤ j ≤ n,

where ej is the jth unit vector.

(c) Show that each sweep of Gauss–Seidel decreases the quantity (Ae, e),where e = u − v.

(d) Show that Gauss–Seidel is optimal in the sense that the quantity ‖e −sej‖A is minimized for each 1 ≤ j ≤ n when s = (r, ej)/(Aej , ej), whichis precisely a Gauss–Seidel step.

17. Matrix 2-norm. Show that the matrix 2-norm is given by

‖A‖2 =√

ρ(AT A).

Use the definition of matrix norm and the relations ‖x‖22 = (x,x) and ‖Ax‖2

2 =(Ax, Ax).

18. Error and residual norms. The condition number of a matrix, cond(A) =‖A‖2‖A−1‖2, gives an idea of how well the residual measures the error. Inthe following exercise, use the property of matrix and vector norms that‖Ax‖ ≤ ‖A‖‖x‖.

30 Chapter 2

(a) Begin with the relations Ae = r and A−1f = u. Taking norms andcombining terms, show that

‖r‖2

‖f‖2≤ cond(A)

‖e‖2

‖u‖2.

Knowing that this bound is sharp (that is, equality can be achieved),interpret this inequality in the case that the condition number is large.

(b) Now begin with the relations Au = f and A−1r = e. Taking norms andcombining terms, show that

‖e‖2

‖u‖2≤ cond(A)

‖r‖2

‖f‖2.

Knowing that this bound is sharp (that is, equality can be achieved),interpret this inequality in the case that the condition number is large.

(c) Combine the above bounds to form the following relations:

1cond(A)

‖r‖2

‖f‖2≤ ‖e‖2

‖u‖2≤ cond(A)

‖r‖2

‖f‖2.

19. Residuals of smooth errors. Consider the residual equation, Ae = r, at asingle point, where A is the matrix for the model problem in either one or twodimensions. Show that if e is smooth (for example, nearly constant), then ris small relative to ‖A‖‖e‖. Conversely, show that if e is oscillatory, then r isrelatively large.

20. Numerical experiments. Write a short program that performs weightedJacobi (with variable ω), Gauss–Seidel, and red-black Gauss–Seidel for theone-dimensional model problem. First reproduce the experiments shown inFig. 2.3. Then experiment with initial guesses with different wavenumbers.Describe how each method performs as the wavenumbers increase and ap-proach n.

Chapter 3

Elements of Multigrid

Through analysis and experimentation, we have examined some of the basic iter-ative methods. Our discoveries have formed the beginnings of what we might calla spectral (or Fourier mode) picture of relaxation schemes. As we proceed, moreessential details of this picture will become clear. So far we have established thatmany standard iterative methods possess the smoothing property. This propertymakes these methods very effective at eliminating the high-frequency or oscillatorycomponents of the error, while leaving the low-frequency or smooth components rel-atively unchanged. The immediate issue is whether these methods can be modifiedin some way to make them effective on all error components.

One way to improve a relaxation scheme, at least in its early stages, is to usea good initial guess. A well-known technique for obtaining an improved initialguess is to perform some preliminary iterations on a coarse grid. Relaxation on acoarse grid is less expensive because there are fewer unknowns to be updated. Also,because the convergence factor behaves like 1 − O(h2), the coarse grid will have amarginally improved convergence rate. This line of reasoning at least suggests thatcoarse grids might be worth considering.

With the coarse grid idea in mind, we can think more carefully about its impli-cations. Recall that most basic relaxation schemes suffer in the presence of smoothcomponents of the error. Assume that a particular relaxation scheme has been ap-plied until only smooth error components remain. We now ask what these smoothcomponents look like on a coarser grid. Figure 3.1 shows the answer. A smoothwave with k = 4 on a grid Ωh with n = 12 points has been projected directly tothe grid Ω2h with n = 6 points. On this coarse grid, the original wave still has awavenumber of k = 4. We see that a smooth wave on Ωh looks more oscillatory onΩ2h.

To be more precise, note that the grid points of the coarse grid Ω2h are theeven-numbered grid points of the fine grid Ωh. Consider the kth mode on the finegrid evaluated at the even-numbered grid points. If 1 ≤ k < n

2 , its componentsmay be written as

whk,2j = sin

(2jkπ

n

)= sin

(jkπ

n/2

)= w2h

k,j , 1 ≤ k <n

2.

Notice that superscripts have been used to indicate the grids on which the vectorsare defined. From this identity, we see that the kth mode on Ωh becomes the kth

31

32 Chapter 3

Figure 3.1: Wave with wavenumber k = 4 on Ωh (n = 12 points) projected ontoΩ2h (n = 6 points). The coarse grid “sees” a wave that is more oscillatory on thecoarse grid than on the fine grid.

mode on Ω2h; this fact is easier to understand by noting that there are half as manymodes on Ω2h as there are on Ωh. The important consequence of this fact is thatin passing from the fine grid to the coarse grid, a mode becomes more oscillatory.This is true provided that 1 ≤ k < n

2 . It should be verified that the k = n2 mode

on Ωh becomes the zero vector on Ω2h.As an aside, it is worth mentioning that fine-grid modes with k > n

2 undergoa more curious transformation. Through the phenomenon of aliasing mentionedearlier, the kth mode on Ωh becomes the (n − k)th mode on Ω2h when k > n

2(Exercise 1). In other words, the oscillatory modes of Ωh are misrepresented asrelatively smooth modes on Ω2h.

The important point is that smooth modes on a fine grid look less smooth ona coarse grid. This suggests that when relaxation begins to stall, signaling thepredominance of smooth error modes, it is advisable to move to a coarser grid;there, the smooth error modes appear more oscillatory and relaxation will be moreeffective. The question is: how do we move to a coarser grid and relax on the moreoscillatory error modes?

It is at this point that multigrid begins to come together like a jigsaw puzzle.We must keep all of the related facts in mind. Recall that we have an equationfor the error itself, namely, the residual equation. If v is an approximation to theexact solution u, then the error e = u − v satisfies

Ae = r = f − Av,

which says that we can relax directly on the error by using the residual equation.There is another argument that justifies the use of the residual equation:

Relaxation on the original equation Au = f with an arbitrary initialguess v is equivalent to relaxing on the residual equation Ae = r withthe specific initial guess e = 0.

A Multigrid Tutorial 33

This intimate connection between the original and the residual equations furthermotivates the use of the residual equation (Exercise 2).

We must now gather these loosely connected ideas. We know that many re-laxation schemes possess the smoothing property. This leads us to consider usingcoarser grids during the computation to focus the relaxation on the oscillatorycomponents of the error. In addition, there seems to be good reason to involvethe residual equation in the picture. We now try to give these ideas a little moredefinition by proposing two strategies.

We begin by proposing a strategy that uses coarse grids to obtain better initialguesses.

• Relax on Au = f on a very coarse grid to obtain an initial guess for the nextfiner grid.

···

• Relax on Au = f on Ω4h to obtain an initial guess for Ω2h.

• Relax on Au = f on Ω2h to obtain an initial guess for Ωh.

• Relax on Au = f on Ωh to obtain a final approximation to the solution.

This idea of using coarser grids to generate improved initial guesses is the basisof a strategy called nested iteration. Although the approach is attractive, it alsoleaves some questions. For instance, what does it mean to relax on Au = f on Ω2h?We must somehow define the original problem on the coarser grids. Also, whathappens if, having once reached the fine grid, there are still smooth componentsin the error? We may have obtained some improvement by using the coarse grids,but the final iteration will stall if smooth components still remain. We return tothese questions and find answers that will allow us to use nested iteration in a verypowerful way.

A second strategy incorporates the idea of using the residual equation to relaxon the error. It can be represented by the following procedure:

• Relax on Au = f on Ωh to obtain an approximation vh.

• Compute the residual r = f − Avh.

Relax on the residual equation Ae = r on Ω2h to obtain

an approximation to the error e2h.

• Correct the approximation obtained on Ωh with the error estimate obtainedon Ω2h : vh ← vh + e2h.

This procedure is the basis of what is called the correction scheme. Havingrelaxed on the fine grid until convergence deteriorates, we relax on the residualequation on a coarser grid to obtain an approximation to the error itself. We thenreturn to the fine grid to correct the approximation first obtained there.

There is a rationale for using this correction strategy, but it also leaves somequestions to be answered. For instance, what does it mean to relax on Ae = r onΩ2h? To answer this question, we first need to know how to compute the residual

34 Chapter 3

Figure 3.2: Interpolation of a vector on coarse grid Ω2h to fine grid Ωh.

on Ωh and transfer it to Ω2h. We also need to know how to relax on Ω2h and whatinitial guess should be used. Moreover, how do we transfer the error estimate fromΩ2h back to Ωh? These questions suggest that we need mechanisms for transferringinformation between the grids. We now turn to this important consideration.

In our discussion of intergrid transfers, we consider only the case in which thecoarse grid has twice the grid spacing of the next finest grid. This is a nearlyuniversal practice, because there is usually no advantage in using grid spacingswith ratios other than 2. Think for a moment about the step in the correctionscheme that requires transferring the error approximation e2h from the coarse gridΩ2h to the fine grid Ωh. This is a common procedure in numerical analysis and isgenerally called interpolation or prolongation. Many interpolation methods couldbe used. Fortunately, for most multigrid purposes, the simplest of these is quiteeffective. For this reason, we consider only linear interpolation.

The linear interpolation operator will be denoted Ih2h. It takes coarse-grid vec-

tors and produces fine-grid vectors according to the rule Ih2hv

2h = vh, where

vh2j = v2h

j ,

vh2j+1 =

12

(v2h

j + v2hj+1

), 0 ≤ j ≤ n

2− 1.

Figure 3.2 shows graphically the action of Ih2h. At even-numbered fine-grid points,

the values of the vector are transferred directly from Ω2h to Ωh. At odd-numberedfine-grid points, the value of vh is the average of the adjacent coarse-grid values.

In anticipation of discussions to come, we note that Ih2h is a linear operator from

Rn2 −1 to Rn−1. It has full rank and the trivial null space, N = 0. For the case

n = 8, this operator has the form

Ih2hv

2h =12

121 1

21 1

21

v1

v2

v3

2h

=

v1

v2

v3

v4

v5

v6

v7

h

= vh.

How well does this interpolation process work? First assume that the “real”error (which is not known exactly) is a smooth vector on the fine grid. Assume

A Multigrid Tutorial 35

Figure 3.3: (a) If the exact error on Ωh (indicated by and •) is smooth, aninterpolant of the coarse-grid error e2h (solid line connecting points) should givea good representation of the exact error. (b) If the exact error on Ωh (indicatedby and •) is oscillatory, an interpolant of the coarse-grid error e2h (solid lineconnecting points) may give a poor representation of the exact error.

also that a coarse-grid approximation to the error has been determined on Ω2h

and that this approximation is exact at the coarse-grid points. When this coarse-grid approximation is interpolated to the fine grid, the interpolant is also smooth.Therefore, we expect a relatively good approximation to the fine-grid error, asshown in Fig. 3.3(a). By contrast, if the “real” error is oscillatory, even a very goodcoarse-grid approximation may produce an interpolant that is not very accurate.This situation is shown in Fig. 3.3(b).

Thus, interpolation is most effective when the error is smooth. Because inter-polation is necessary for both nested iteration and the correction scheme, we mayconclude that these two processes are most effective when the error is smooth. Aswe will see shortly, these processes provide a fortunate complement to relaxation,which is most effective when the error is oscillatory.

For two-dimensional problems, the interpolation operator may be defined in asimilar way. If we let Ih

2hv2h = vh, then the components of vh are given by

vh2i,2j = v2h

ij ,

vh2i+1,2j =

12(v2h

ij + v2hi+1,j

),

vh2i,2j+1 =

12(v2h

ij + v2hi,j+1

),

vh2i+1,2j+1 =

14(v2h

ij + v2hi+1,j + v2h

i,j+1 + v2hi+1,j+1

), 0 ≤ i, j ≤ n

2− 1.

The second class of intergrid transfer operations involves moving vectors froma fine grid to a coarse grid. They are generally called restriction operators and aredenoted by I2h

h . The most obvious restriction operator is injection. It is defined byI2hh vh = v2h, where

v2hj = vh

2j .

36 Chapter 3

Figure 3.4: Restriction by full weighting of a fine-grid vector to the coarse grid.

In other words, with injection, the coarse-grid vector simply takes its value directlyfrom the corresponding fine-grid point.

An alternate restriction operator, called full weighting, is defined by I2hh vh =

v2h, where

v2hj =

14(vh2j−1 + 2vh

2j + vh2j+1

), 1 ≤ j ≤ n

2− 1.

As Fig. 3.4 shows, the values of the coarse-grid vector are weighted averages ofvalues at neighboring fine-grid points.

In the discussion that follows, we use full weighting as a restriction operator.However, in some instances, injection may be the better choice. The issue of inter-grid transfers, which is an important part of multigrid theory, is discussed at somelength in Brandt’s guide to multigrid [4].

The full weighting operator is a linear operator from Rn−1 to Rn2 −1. It has a

rank of n2 − 1 (Exercise 4) and a null space of dimension n

2 (Exercise 5). For thecase n = 8, the full weighting operator has the form

I2hh vh =

14

1 2 1

1 2 11 2 1

v1

v2

v3

v4

v5

v6

v7

h

=

v1

v2

v3

2h

= v2h.

One reason for our choice of full weighting as a restriction operator is the importantfact (Exercise 6) that

Ih2h = c(I2h

h )T , c ∈ R.

The fact that the interpolation operator and the full weighting operator are trans-poses of each other up to a constant is called a variational property and will soonbe of importance.

For the sake of completeness, we give the full weighting operator in two dimen-sions. It is just an averaging of the fine-grid nearest neighbors. Letting I2h

h vh = v2h,we have that

v2hij =

116

[vh2i−1,2j−1 + vh

2i−1,2j+1 + vh2i+1,2j−1 + vh

2i+1,2j+1

+ 2(vh2i,2j−1 + vh

2i,2j+1 + vh2i−1,2j + vh

2i+1,2j

)+ 4vh

2i,2j

], 1 ≤ i, j ≤ n

2− 1.

A Multigrid Tutorial 37

We now have a well-defined way to transfer vectors between fine and coarsegrids. Therefore, we can return to the correction scheme and make it precise. Todo this, we define the following two-grid correction scheme.

Two-Grid Correction Scheme

vh ← MG(vh, fh).

• Relax ν1 times on Ahuh = fh on Ωh with initial guess vh.

• Compute the fine-grid residual rh = fh − Ahvh and restrict it to the coarsegrid by r2h = I2h

h rh.

• Solve A2he2h = r2h on Ω2h.

• Interpolate the coarse-grid error to the fine grid by eh = Ih2he

2h and correctthe fine-grid approximation by vh ← vh + eh.

• Relax ν2 times on Ahuh = fh on Ωh with initial guess vh.

This procedure is simply the original correction scheme, now refined by theuse of the intergrid transfer operators. We relax on the fine grid until it ceasesto be worthwhile; in practice, ν1 is often 1, 2, or 3. The residual of the currentapproximation is computed on Ωh and then transferred by a restriction operatorto the coarse grid. As it stands, the procedure calls for the exact solution of theresidual equation on Ω2h, which may not be possible. However, if the coarse-griderror can at least be approximated, it is then interpolated up to the fine grid, whereit is used to correct the fine-grid approximation. This is followed by ν2 additionalfine-grid relaxation sweeps.

Several comments are in order. First, notice that the superscripts h or 2h areessential to indicate the grid on which a particular vector or matrix is defined.Second, all of the quantities in the above procedure are well defined except for A2h.For the moment, we take A2h simply to be the result of discretizing the problemon Ω2h. Finally, the integers ν1 and ν2 are parameters in the scheme that controlthe number of relaxation sweeps before and after visiting the coarse grid. Theyare usually fixed at the start, based on either theoretical considerations or on pastexperimental results.

It is important to appreciate the complementarity at work in the process. Re-laxation on the fine grid eliminates the oscillatory components of the error, leavinga relatively smooth error. Assuming the residual equation can be solved accuratelyon Ω2h, it is still important to transfer the error accurately back to the fine grid.Because the error is smooth, interpolation should work very well and the correctionof the fine-grid solution should be effective.

Numerical example. A numerical example will be helpful. Consider the weightedJacobi method with ω = 2

3 applied to the one-dimensional model problem Au = 0on a grid with n = 64 points. We use an initial guess,

vhj =

12

[sin

(16jπ

n

)+ sin

(40jπ

n

)],

consisting of the k = 16 and k = 40 modes. The following two-grid correctionscheme is used:

38 Chapter 3

• Relax three times on Ahuh = 0 on Ωh with initial guess vh.

• Compute r2h = I2hh rh.

• Relax three times on A2he2h = r2h on Ω2h with initial guess e2h = 0.

• Correct the fine-grid approximation: vh ← vh + Ih2he

2h.

• Relax three times on Ahuh = 0 on Ωh with initial guess vh.

• Compute r2h = I2hh rh.

• Relax three times on A2he2h = r2h on Ω2h with initial guess e2h = 0.

• Correct the fine-grid approximation: vh ← vh + Ih2he

2h.

The results of this calculation are given in Fig. 3.5. The initial guess with its twomodes is shown in the top left figure. In the top right, the approximation vh afterone relaxation sweep is superimposed on the initial guess. Much of the oscillatorycomponent of the initial guess has already been removed, and the 2-norm of theerror has been diminished to 57% of the norm of the initial error. The middle leftplot shows the approximation after three relaxation sweeps on the fine grid, againsuperimposed on the initial guess. The solution (in this case, the error) has becomesmoother and its norm is now 36% of the initial error norm. Further relaxationson the fine grid would provide only a slow improvement at this point. This signalsthat it is time to move to the coarse grid.

The middle right plot shows the fine-grid error after one relaxation sweep on thecoarse-grid residual equation, superimposed on the initial guess. Clearly, we haveachieved another reduction in the error by moving to the coarse grid; the norm ofthe error is now 26% of the initial error norm. This improvement occurs becausethe smooth error components, inherited from the fine grid, appear oscillatory on thecoarse grid and are quickly removed. The error after three coarse-grid relaxationsweeps is shown in the bottom left figure. The norm of the error is now about 8%of its initial value.

The coarse-grid approximation to the error is now used to correct the fine-gridapproximation. After three additional fine-grid relaxations, the 2-norm of the erroris reduced to about 3% of the initial error norm. This result is plotted in thebottom right figure. The residual is once again transferred to the coarse grid andthree coarse-grid relaxations follow. At this point, the 2-norm of the error is about1% of its original value. This experiment demonstrates that relaxation, when doneon two grids and applied to both the original and the residual equation, can bevery powerful.

The two-grid correction scheme, as outlined above, leaves one looming procedu-ral question: what is the best way to solve the coarse-grid problem A2he2h = r2h?The answer may be apparent, particularly to those who think recursively. Thecoarse-grid problem is not much different from the original problem. Therefore, wecan apply the two-grid correction scheme to the residual equation on Ω2h, whichmeans relaxing there and then moving to Ω4h for the correction step. We can re-peat this process on successively coarser grids until a direct solution of the residualequation is possible.

A Multigrid Tutorial 39

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1–1

–0.8

–0.6

–0.4

–0.2

0

0.2

0.4

0.6

0.8

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1–1

–0.8

–0.6

–0.4

–0.2

0

0.2

0.4

0.6

0.8

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1–1

–0.8

–0.6

–0.4

–0.2

0

0.2

0.4

0.6

0.8

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1–1

–0.8

–0.6

–0.4

–0.2

0

0.2

0.4

0.6

0.8

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1–1

–0.8

–0.6

–0.4

–0.2

0

0.2

0.4

0.6

0.8

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1–1

–0.8

–0.6

–0.4

–0.2

0

0.2

0.4

0.6

0.8

1

Figure 3.5: Coarse-grid correction for −u′′ = 0 on a grid with n = 64. Top left:The initial guess, (w16 +w40)/2. Top right: The error after one sweep of weightedJacobi. Middle left: The error after three sweeps of weighted Jacobi. Middle right:The fine-grid error after one sweep of weighted Jacobi on the coarse-grid problem.Bottom left: The fine-grid error after three sweeps of weighted Jacobi on the coarse-grid problem. Bottom right: The fine-grid error after the coarse-grid correction isfollowed by three weighted Jacobi sweeps on the fine grid.

40 Chapter 3

To facilitate the description of this procedure, some economy of notation is de-sirable. The same notation is used for the computer implementation of the resultingalgorithm. We call the right-side vector of the residual equation f2h, rather thanr2h, because it is just another right-side vector. Instead of calling the solution of theresidual equation e2h, we use u2h because it is just a solution vector. We can thenuse v2h to denote approximations to u2h. These changes simplify the notation, butit is still important to remember the meaning of these variables.

One more point needs to be addressed: what initial guess do we use for v2h onthe first visit to Ω2h? Because there is presumably no information available aboutthe solution, u2h, we simply choose v2h = 0. Here then is the two-grid correctionscheme, now imbedded within itself. We assume that there are l > 1 grids withgrid spacings h, 2h, 4h, . . . , Lh = 2l−1h.

V-Cycle Scheme

vh ← V h(vh, fh)

• Relax on Ahuh = fh ν1 times with initial guess vh.• Compute f2h = I2h

h rh.• Relax on A2hu2h = f2h ν1 times with initial guess v2h = 0.• Compute f4h = I4h

2hr2h.• Relax on A4hu4h = f4h ν1 times with initial guess v4h = 0.• Compute f8h = I8h

4hr4h.···

• Solve ALhuLh = fLh.···

• Correct v4h ← v4h + I4h8hv8h.

• Relax on A4hu4h = f4h ν2 times with initial guess v4h.• Correct v2h ← v2h + I2h

4hv4h.• Relax on A2hu2h = f2h ν2 times with initial guess v2h.

• Correct vh ← vh + Ih2hv

2h.• Relax on Ahuh = fh ν2 times with initial guess vh.

The algorithm telescopes down to the coarsest grid, which can consist of one ora few interior grid points, then works its way back to the finest grid. Figure 3.6(a)shows the schedule for the grids in the order in which they are visited. Because ofthe pattern in this diagram, this algorithm is called the V -cycle. It is our first truemultigrid method.

Not surprisingly, the V-cycle has a compact recursive definition, which is givenas follows.

V-Cycle Scheme (Recursive Definition)

vh ← V h(vh, fh).

1. Relax ν1 times on Ahuh = fh with a given initial guess vh.

A Multigrid Tutorial 41

Figure 3.6: Schedule of grids for (a) V-cycle, (b) W-cycle, and (c) FMG scheme,all on four levels.

2. If Ωh = coarsest grid, then go to step 4.

Else

f2h ← I2hh (fh − Ahvh),

v2h ← 0,

v2h ← V 2h(v2h, f2h).

3. Correct vh ← vh + Ih2hv

2h.

4. Relax ν2 times on Ahuh = fh with initial guess vh.

42 Chapter 3

The V-cycle is just one of a family of multigrid cycling schemes. The entirefamily is called the µ-cycle method and is defined recursively by the following.

µ-Cycle Scheme

vh ← Mµh(vh, fh).

1. Relax ν1 times on Ahuh = fh with a given initial guess vh.

2. If Ωh = coarsest grid, then go to step 4.

Else

f2h ← I2hh (fh − Ahvh),

v2h ← 0,

v2h ← Mµ2h(v2h, f2h) µ times.

3. Correct vh ← vh + Ih2hv

2h.

4. Relax ν2 times on Ahuh = fh with initial guess vh.

In practice, only µ = 1 (which gives the V-cycle) and µ = 2 are used. Figure3.6(b) shows the schedule of grids for µ = 2 and the resulting W -cycle. We referto a V-cycle with ν1 relaxation sweeps before the correction step and ν2 relaxationsweeps after the correction step as a V(ν1, ν2)-cycle, with a similar notation forW-cycles.

We originally stated that two ideas would lead to multigrid. So far we havedeveloped only the correction scheme. The nested iteration idea has yet to beexplored. Recall that nested iteration uses coarse grids to obtain improved initialguesses for fine-grid problems. In looking at the V-cycle, we might ask how toobtain an informed initial guess for the first fine-grid relaxation. Nested iterationwould suggest solving a problem on Ω2h. But how can we obtain a good initialguess for the Ω2h problem? Nested iteration sends us to Ω4h. Clearly, we are onanother recursive path that leads to the coarsest grid.

The algorithm that joins nested iteration with the V-cycle is called the fullmultigrid V-cycle (FMG) . Given first in explicit terms, it appears as follows.

Full Multigrid V-Cycle

vh ← FMGh(fh).

Initialize f2h ← I2hh fh, f4h ← I4h

2hf2h, . . ..• Solve or relax on coarsest grid.

···

• v4h ← I4h8hv8h.

• v4h ← V 4h(v4h, f4h) ν0 times.• v2h ← I2h

4hv4h.• v2h ← V 2h(v2h, f2h) ν0 times.

• vh ← Ih2hv

2h.• vh ← V h(vh, fh), ν0 times.

A Multigrid Tutorial 43

We initialize the coarse-grid right sides by transferring fh from the fine grid. An-other option is to use the original right-side function f . The cycling parameter,ν0, sets the number of V-cycles done at each level. It is generally determined bya previous numerical experiment; ν0 = 1 is the most common choice. Expressedrecursively, the algorithm has the following compact form.

Full Multigrid V-Cycle (Recursive Form)

vh ← FMGh(fh).

1. If Ωh = coarsest grid, set vh ← 0 and go to step 3.

Else

f2h ← I2hh (fh),

v2h ← FMG2h(f2h).

2. Correct vh ← Ih2hv

2h.

3. vh ← V h(vh, fh) ν0 times.

Figure 3.6(c) shows the schedule of grids for FMG with ν0 = 1. Each V-cycle is preceded by a coarse-grid V-cycle designed to provide the best initial guesspossible. As we will see, the extra work done in these preliminary V-cycles is notonly inexpensive (Exercise 8), but easily pays for itself.

Full multigrid is the complete knot into which the many threads of the pre-ceding chapters are tied. It is a remarkable synthesis of ideas and techniques thatindividually have been well known and used for a long time. Taken alone, many ofthese ideas have serious defects. Full multigrid is a technique for integrating themso that they can work together in a way that overcomes these limitations. Theresult is a very powerful algorithm.

Exercises

1. Aliasing. Show that the kth mode on a grid Ωh with n − 1 interior pointsappears as the (n − k)th mode on Ω2h when n

2 < k < n.

2. An important equivalence. Consider a stationary, linear method of theform v ← v+B−1(f −Av) applied to the problem Au = f . Use the followingsteps to show that relaxation on Au = f with an arbitrary initial guess isequivalent to relaxation on Ae = r with the zero initial guess:

(a) First consider the problem Au = f with an arbitrary initial guess v = v0.What are the error and residual associated with v0?

(b) Now consider the associated residual equation Ae = r0 = f−Av0. Whatare the error and residual in the initial guess e0 = 0?

(c) Conclude that the problems in (a) and (b) are equivalent.

3. Properties of interpolation. Show that Ih2h based upon linear interpolation

is a linear operator with full rank in one and two dimensions.

4. Properties of restriction. What is the rank of I2hh based on (a) full weight-

ing and (b) injection in one and two dimensions?

44 Chapter 3

5. Null space of full weighting. Show that the null space of the full weightingoperator, N(I2h

h ), has a basis consisting of vectors of the form

(0, 0, . . . ,−1, 2,−1, . . . , 0, 0)T .

By counting these vectors, show that the dimension of N(I2hh ) is n

2 .

6. Variational property.

(a) Let Ih2h and I2h

h be defined as in the text. Show that linear interpolationand full weighting satisfy the variational property Ih

2h = c(I2hh )T by

computing c ∈ R for both one and two dimensions.

(b) The choice of c = 1 found in part (a) is used because full weightingessentially preserves constants. Show that, except at the boundary,I2hh (1h) = 12h (where 1h and 12h are the vectors with entries 1 on

their respective grids).

7. Properties of red-black Gauss–Seidel. Suppose red-black Gauss–Seidelis used with the V-cycle scheme for the one-dimensional model problem.

(a) Does it matter whether the odd unknowns or even unknowns are updatedfirst? Explain.

(b) Show that one sweep of red-black Gauss–Seidel on Ωh leaves the erroreh in the range of interpolation Ih

2h.

(c) Demonstrate that one V-cycle based on red-black Gauss–Seidel and fullweighting is a direct (exact) solver for the one-dimensional model prob-lem.

8. FMG cost. The difference in cost between FMG and a single V-cycle is thecost of all but the last V-cycle on Ωh in the FMG scheme. Estimate the costof these extra V-cycles. Assume that the cost of a V-cycle on grid Ωph isproportional to the number of points in that grid, where p = 2, 4, 8, . . . , n/2.Assume also that ν0 = 1.

Chapter 4

Implementation

The preceding chapter was devoted to the development of several multigrid schemes.We now turn to the practical issues of writing multigrid programs and determiningwhether they work. This will lead us to issues such as data structures, complexity,predictive tools, diagnostic tools, and performance.

Complexity

Writing multigrid programs can be both fun and challenging. The experience ofmany practitioners suggests that such programs should be highly modular. Thisallows them to evolve from simple relaxation programs and makes them much easierto check and debug. Also, the various components of the program (for example,relaxation, interpolation, and restriction subroutines) can be replaced individually.

Choosing a manageable data structure for a multigrid program is essential.Modern programming languages are replete with devices that make data manage-ment easy. For example, in most languages, one can declare a structure that groupstogether all the associated information for each grid level. In a structured language,for instance, a V-cycle could be written along the lines of the following pseudocode:

declare structure:

grid = double Ddim_array f %% the right hand side

double Ddim_array v %% the current approximation

declare Grid: array of structure grid

for j = 0 to coarsest - 1

Grid[j].v <- relax(Grid[j].v, Grid[j].f, num_sweeps_down)

Grid[j+1].f <- restrict(Grid[j].f - apply_operator(Grid[j].v))

endfor

Grid[coarsest].v = direct_solve(Grid[coarsest].v, Grid[coarsest].f)

for j = coarsest-1 to 1

Grid[j].v <- Grid[j].v + interpolate(Grid[j+1].v)

Grid[j].v <- relax(Grid[j].v, Grid[j].f, num_sweeps_down)

endfor

45

46 Chapter 4

The routines relax, restrict, apply operator, interpolate, and direct solve takethe appropriate Ddim arrays, v and f, for the specified grid level and perform theappropriate operations. We do not describe this type of data management in anyfurther detail, as the advances in these languages occur so rapidly that any discus-sion would soon be outdated!

We describe a data structure for a simpler FORTRAN-like language. Multigridcodes “grew up” in such an environment and many people learn to write multigridcodes using MATLAB or a similar prototyping language with more restrictive datastructures. With these languages, there seems to be general agreement that thesolutions and right-side vectors on the various grids should be stored contiguouslyin single arrays. Complicating factors such as irregular domains or local fine-gridpatches might require an exception to this practice. However, single arrays areadvisable for the regular grids discussed in this chapter.

We begin by considering a four-level V-cycle applied to a one-dimensional prob-lem with n = 16 points. A typical data structure is shown in Fig. 4.1. It isinstructive to note how the data structure changes as the V-cycle progresses. Eachgrid needs two arrays: one to hold the current approximations on each grid and oneto hold the right-side vectors on each grid. Because boundary values must also bestored, the coarsest grid involves three grid points (one interior and two boundarypoints). In general, the th coarsest grid involves 2 + 1 points.

Initially, the entire solution array v may be set to zero, which will be the initialguess on each grid. The right-side array f will also be set to zero, except for thevalues on the finest grid, which are known at the outset.

As the V-cycle “descends” into coarser grids, relaxation fills the segment of thesolution array corresponding to each grid. At the same time, the residual vectorsf fill the right-side array corresponding to the next coarsest grid. As the V-cycle“ascends” through finer grids, the right-side array does not change. However, thesolution array is overwritten by additional relaxations on each level. Notice thatwhen a new approximation is computed on one level, the approximation on theprevious level is zeroed out. This provides a zero initial guess on each level in caseanother V-cycle is performed.

We now turn to the important questions of complexity. How much do themultigrid schemes cost in terms of storage and computation? The storage questionis easier to answer. Consider a d-dimensional grid with nd points. (Actually, forDirichlet boundary conditions, there will be (n− 1)d interior points and, as we willsee later, for Neumann boundary conditions, there will be (n+1)d unknown points.)For simplicity, suppose n is a power of 2. We have just seen that two arrays mustbe stored on each level. The finest grid, Ωh, requires 2nd storage locations; Ω2h

requires 2−d times as much storage as Ωh; Ω4h requires 4−d times as much storageas Ωh; in general, Ωph requires p−d times as much storage as Ωh. Adding theseterms and using the sum of the geometric series as an upper bound gives

Storage = 2nd1 + 2−d + 2−2d + · · · + 2−nd

<

2nd

1 − 2−d.

In particular, for a one-dimensional problem (d = 1), the storage requirement isless than twice that of the fine-grid problem alone. For problems in two or moredimensions, the requirement drops to less than 4

3 of the fine-grid problem alone(Exercise 3). Thus, the storage costs of multigrid algorithms decrease relatively asthe dimension of the problem increases.

A Multigrid Tutorial 47

Figure 4.1: Illustration of the course of a four-level (n = 16) V-cycle showingchanges in the data arrays. The v and f arrays hold the solution vectors and right-side vectors, respectively, in the four grids.

We may use similar reasoning to estimate the computational cost of multigridmethods. It is convenient to measure these costs in terms of a work unit (WU),which is the cost of performing one relaxation sweep on the finest grid. It is cus-tomary to neglect the cost of intergrid transfer operations, which typically amountsto 10–20% of the cost of the entire cycle.

First consider a V-cycle with one relaxation sweep on each level (ν1 = ν2 = 1).Each level is visited twice and grid Ωph requires p−d work units. Adding these costsand again using the geometric series for an upper bound gives

V-cycle computation cost

= 21 + 2−d + 2−2d + · · · + 2−nd

<

21 − 2−d

WU.

48 Chapter 4

A single V-cycle costs about 4 WUs for a one-dimensional (d = 1) problem, about83 WUs for d = 2, and 16

7 WUs for d = 3 (Exercise 4).With a slight modification, we can find the computational cost for an FMG cycle.

Assume again that one relaxation sweep is done on each level (ν0 = ν1 = ν2 = 1).As just shown, a full V-cycle beginning from Ωh costs about 2(1 − 2−d)−1 WUs.A V-cycle beginning from Ω2h costs 2−d of a full V-cycle. In general, a V-cyclebeginning from Ωph costs p−d of a full V-cycle. Adding these costs gives us

FMG computation cost

=(

21 − 2−d

) (1 + 2−d + 2−2d + · · · + 2−nd

)<

2(1 − 2−d)2

WU.

An FMG cycle costs 8 WUs for a one-dimensional problem, about 72 WUs for d = 2,

and 52 WU for d = 3 (Exercise 5).

As expected, a single FMG cycle costs more than a single V-cycle, although thediscrepancy is less for higher-dimensional problems. We really need to know howmany V-cycles and FMG cycles are needed to obtain satisfactory results. This begsthe fundamental question: how well do these multigrid cycling schemes work?

Predictive Tools: Local Mode Analysis

The previous section dealt with the practical considerations of implementing multi-grid algorithms. However, it is a common experience to have a multigrid code thatruns, but does not work! Indeed, it can often be puzzling to know what to expectin terms of efficiency and accuracy. The remainder of this chapter presents somepractical tools for determining whether an algorithm is working properly. First, wedeal with tools for predicting the convergence rates that can be expected from thebasic relaxation methods applied to standard problems.

Recall from Chapter 2 that the asymptotic convergence factor of a relaxationscheme is the spectral radius (the largest eigenvalue magnitude) of the correspond-ing iteration matrix. We also defined the smoothing factor as the convergence factorassociated with the oscillatory modes only. Because eigenvalue calculations can bedifficult, this approach to finding convergence factors is limited to fairly simpleiterations applied primarily to model problems.

We now present a more versatile approach for approximating convergence andsmoothing factors called local mode analysis (or normal mode analysis or Fourieranalysis). The goal of this section is rather modest: we show how to apply the basicprocedure to some prototype problems and then point the way to more advancedcalculations. In its full generality, local mode analysis can be applied to generaloperators and to a wide class of relaxation schemes on one or more levels. Withthis generality, local mode analysis is a powerful predictive tool that can be usedto compare multigrid performance with theoretical expectations.

The original proponent of local mode analysis was Achi Brandt, who expressedits significance by saying that

...the main importance of the smoothing factor is that it separates thedesign of the interior relaxation from all other algorithmic questions.Moreover, it sets an ideal figure against which the performance of thefull algorithm can later be judged. [4]

A Multigrid Tutorial 49

Local mode analysis begins with the assumption that relaxation is a local pro-cess: each unknown is updated using information from nearby neighbors. Becauseit is a local process, it is argued that boundaries and boundary conditions can beneglected if we are considering a few relaxation sweeps at interior points. For thisreason, the finite domain of the problem is replaced by an infinite domain.

As before, we are interested in how a particular relaxation scheme acts on theerrors in an approximation. Assume that relaxation is a linear process and denotethe associated matrix by R. Let e(m) denote the algebraic error at the mth step ofrelaxation. Recall (Chapter 2) that the error itself evolves under the action of R:

e(m+1) = Re(m).

The approach of local mode analysis is to assume that the error consists of Fouriermodes and to determine how relaxation acts on those modes. The Fourier modeswe encountered in Chapter 2 have the form wj = sin( jkπ

n ), where the wavenumberk is an integer between 1 and n. This means that the term θ = kπ

n runs roughlyfrom 0 to π. With the new assumption of an infinite domain (no boundaries orboundary conditions to satisfy), the Fourier modes need not be restricted to dis-crete wavenumbers. Instead, we consider modes of the form wj = eιjθ, where thewavenumber θ can take on any value in the interval (−π, π]. (For the remainderof the chapter, we let ι =

√−1 to avoid confusing i with the grid indices.) Notice

that the mode corresponding to a particular θ has a wavelength of 2πh|θ| ; values of

|θ| near zero correspond to low-frequency waves; value of |θ| near π correspond tohigh-frequency waves. The choice of a complex exponential makes computationsmuch easier and accounts for both sine and cosine terms.

An important point should be mentioned here. Local mode analysis is notcompletely rigorous unless the Fourier modes are eigenvectors of the relaxationmatrix, which is not generally the case. However, the analysis is useful for the highfrequency modes of the error, which do tend to resemble the eigenvectors of therelaxation matrix very closely. For this reason, local mode analysis is used for asmoothing analysis of the high frequency modes.

With these ground rules, we are ready to apply the method. We begin withone-dimensional problems and assume that the error at the mth step of relaxationat the jth grid point consists of a single mode of the form

emj = A(m)eιjθ, where − π < θ ≤ π. (4.1)

The goal is to determine how the amplitude of the mode, A(m), changes with eachrelaxation sweep. In each case we consider, the amplitudes at successive steps arerelated by an expression of the form

A(m + 1) = G(θ)A(m).

The function G that describes how the error amplitudes evolve is called the am-plification factor. For convergence of the method, we must have |G(θ)| < 1 forall θ. As we have seen, relaxation is used in multigrid to eliminate the oscillatorymodes of the error. Therefore, the quantity of interest is really the smoothing fac-tor, which is found by restricting the amplification factor, G(θ), to the oscillatorymodes π

2 ≤ |θ| ≤ π. Specifically, we define the smoothing factor as

µ = maxπ2 ≤|θ|≤π

|G(θ)|.

50 Chapter 4

This is the factor by which we can expect the oscillatory modes to be damped (atworst) with each relaxation sweep. With these definitions, it is best to proceed byexample.

Example: One-dimensional problems. Consider the one-dimensional modelproblem

−u′′(x) + c(x)u(x) = f(x).

Letting vj be the approximation to u(xj), we discretize the problem with the usualsecond-order finite-difference approximations and apply weighted Jacobi relaxation.This results in the familiar Jacobi updating step

vm+1j =

ω

2 + h2cj(vm

j−1 + vmj+1 + h2fj) + (1 − ω)vm

j , (4.2)

where cj = c(xj). Knowing that the error, ej = u(xj)− vj , is also governed by thesame weighted Jacobi relaxation, we can write the updating step for the error atthe jth grid point as (Exercise 6)

em+1j =

ω

2 + h2cj(em

j+1 + emj−1) + (1 − ω)em

j . (4.3)

Assume now that the error consists of a mode of the form (4.1) and substitute itinto (4.3). Letting cj = 0 for the moment, we have

A(m + 1)eιjθ =ω

2

A(m) (eι(j+1)θ + eι(j−1)θ)︸ ︷︷ ︸

2eιjθ cos θ

+ (1 − ω)A(m)eιjθ.

As indicated, the Euler formula for cos θ allows for some simplification. Collectingterms now leads to

A(m + 1)eιjθ = A(m)(1 − ω (1 − cos θ)︸ ︷︷ ︸2 sin2(θ/2)

)eιjθ.

Canceling the common term eιjθ and using the indicated trigonometric identity, wecan write the following relationship between successive amplitudes:

A(m + 1) =(

1 − 2ω sin2

(θ

2

))A(m) ≡ G(θ)A(m), where − π < θ ≤ π.

The amplification factor G(θ) = 1− 2ω sin2(θ/2) appears naturally in this calcula-tion and it should look familiar. In this case, we have just reproduced the eigenvaluecalculation for the weighted Jacobi iteration matrix (see Chapter 2); that is, if wemake the substitution θk = πk

n , then G(θk) is just the kth eigenvalue of the Jacobiiteration matrix. As we know, |G(θ)| < 1 provided 0 < ω ≤ 1; with ω = 2

3 , we havethe optimal smoothing factor

µ = G(π

2

)= |G(±π)| =

13.

A similar calculation can be done with Gauss–Seidel. The updating step for theerror at the jth grid point now appears as (Exercise 7)

em+1j =

em+1j−1 + em

j+1

2 + cjh2. (4.4)

A Multigrid Tutorial 51

0 0.5 1 1.5 2 2.5 3 3.50.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 4.2: The amplification factor, |G(θ)|, for the Gauss–Seidel method applied tothe one-dimensional problem −u′′(x) = f(x). The graph is symmetric about θ = 0.The smoothing factor is µ = |G(π

2 )| = 1√5

= 0.45.

Note that because we sweep across the grid from left to right, the previous ((j−1)st)component has already been updated. Again we assume that the errors have theform (4.1) and substitute. Assuming for the moment that cj = 0, we find that theamplitudes are related by (Exercise 7)

A(m + 1) =eιθ

2 − e−ιθA(m) ≡ G(θ)A(m), where − π < θ ≤ π.

To find the smoothing factor from the complex amplification factor, it is easiest toplot |G(θ)|, as shown in Fig. 4.2. A bit of analysis reveals that

µ =∣∣∣G (π

2

)∣∣∣ =1√5

= 0.45.

A subtle point could be made here. The amplification factor, G(θ), gives the(complex) eigenvalues of the Gauss–Seidel iteration matrix, not on a boundeddomain with specified boundary conditions, but on an infinite domain. This cal-culation differs from the eigenvalue calculation of Chapter 2, in which the eigen-values for a bounded domain were found to be real. For this reason, the amplifica-tion factor gives only an estimate of the smoothing factors for a bounded domainproblem.

We can use the above example to illustrate how local mode analysis works witha variable coefficient operator. Suppose that c(x) > 0 on the domain. To avoidworking with a different amplification factor at every grid point, the practice is to

52 Chapter 4

“freeze” the coefficient, c(x), at a representative value c0 = c(ξ), for some ξ in thedomain (often the minimum or maximum value of c(x) on the domain). With theweighted Jacobi iteration, the amplification factor now appears as

G(θ) = 1 − ω

(1 − 2

2 + c0h2cos θ

).

The idea is to find the value of c0, over all possible c(ξ), that gives the worst(most pessimistic) smoothing factor. Occasionally, this calculation can be doneanalytically; more typically, it is done numerically by choosing several differentpossible values of c0. We can rewrite this amplification factor as

G(θ) = G0(θ) −c0ωh2

2cos(θ),

where G0(θ) is the amplification factor for the case that c(x) = 0. In this form, wesee that the effect of the variable coefficient is insignificant unless c0 is comparableto h−2. There is a more general principle at work here: usually the lower orderterms of the operator can be neglected with impunity in local mode analysis.

Local mode analysis can be extended easily to two or more dimensions. In twodimensions, the Fourier modes have the form

e(m)jk = A(m)eι(jθ1+kθ2), (4.5)

where −π < θ1, θ2 ≤ π are the wavenumbers in the x- and y-directions, respectively.Substituting this representation into the error updating step generally leads to anexpression for the change in the amplitudes of the form

A(m + 1) = G(θ1, θ2)A(m).

The amplification factor now depends on two wavenumbers. The smoothing factoris the maximum magnitude of the amplification factor over the oscillatory modes.As we see in Fig. 4.3, the oscillatory modes correspond to π

2 ≤ |θi| ≤ π for eitheri = 1 or i = 2; that is,

µ = maxπ/2≤|θi|≤π

|G(θ1, θ2)|.

Example: Two-dimensional problems. Consider the model problem

uxx + uyy = f(x, y)

on a rectangular domain with a uniform grid in both directions. Applying theweighted Jacobi method, the error satisfies (Exercise 8)

e(m+1)jk =

ω

4

(e(m)j−1,k + e

(m)j+1,k + e

(m)j,k−1 + e

(m)j,k+1

)+ (1 − ω)e(m)

jk . (4.6)

Substituting the Fourier modes (4.5) into the error updating equation, we find that(Exercise 8)

A(m + 1) =[1 − ω

(sin2

(θ1

2

)+ sin2

(θ2

2

))]A(m) ≡ G(θ1, θ2)A(m).

Two views of the amplification factor are given in Fig. 4.4 for the case that ω = 45 .

In the left figure, each curve shows the variation of G over 0 ≤ θ2 ≤ π for fixed

A Multigrid Tutorial 53

(0, 0)

(−π,−π)

(−π, π) (π, π)

(π,−π)

θ1

θ2

Figure 4.3: The oscillatory modes in two dimensions correspond to the wavenumbersπ2 ≤ |θi| < π for either i = 1 or i = 2; this is the region outside the dashed box.

0 0.5 1 1.5

(a)2 2.5 3 3.5

–1

–0.8

–0.6

–0.4

–0.2

0

0.2

0.4

0.6

0.8

1

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1

(b)

Figure 4.4: (a) Amplification factor, G(θ1, θ2), for the weighted Jacobi method ap-plied to the model problem in two dimensions, shown as individual curves of fixedθ1 (θ1 = 0 at the top and θ1 = π at the bottom). (b) Same amplification factorshown as a surface over the region [0, π] × [0, π]. The picture is symmetric aboutboth the θ1- and θ2-axes.

values of θ1; the upper curve corresponds to θ1 = 0 and the lower curve correspondsto θ2 = π. Clearly, the amplification factor decreases in magnitude as the modesbecome more oscillatory. The right figure shows the same amplification factor as a

54 Chapter 4

0

50

1000 10 20 30 40 50 60 70 80 90

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 4.5: Amplification factor, |G(θ1, θ2)|, for the Gauss–Seidel method applied tothe model problem in two dimensions, shown as a surface over the region [−π,−π]×[π, π].

surface over the region [0, π] × [0, π]. (The surface is symmetric about both axes.)Some analysis or experimentation reveals that the best smoothing factor is obtainedwhen ω = 4

5 and it is given by µ = |G(±π,±π)| = 0.6 (Exercise 8). This means thatif we use the Jacobi scheme with ω = 4

5 , then we expect a reduction in the residualnorm by approximately a factor of 0.6 per relaxation sweep. A V(2,1)-cycle, forexample, should have a convergence factor of about 0.63 = 0.216.

A similar calculation can be done for the Gauss–Seidel method applied to themodel problem. The error updating equation is (Exercise 9)

e(m+1)jk =

e(m+1)j−1,k + e

(m)j+1,k + e

(m+1)j,k−1 + e

(m)j,k+1

4. (4.7)

Here we assume that the unknowns have a lexicographic ordering (in order of in-creasing j and k); thus, the unknowns preceding ejk are updated and appear atthe (m + 1)st step. Once again, we substitute the modes given in (4.5). Theamplification factor is most easily expressed in complex form as (Exercise 9)

G(θ1, θ2) =eιθ1 + eιθ2

4 − e−ιθ1 − e−ιθ2.

The magnitude of this function is plotted over the region [−π,−π] × [π, π] in Fig.4.5.

Some computation is required to show that

|G(θ1, θ2)|2 =1 + cos β

9 − 8 cos(α2 ) cos(β

2 ) + cos β,

A Multigrid Tutorial 55

The Discrete L2 Norm. Another norm that is particularly appropriate formeasuring errors in numerical calculations is the discrete L2 norm. If the vectoruh is associated with a d-dimensional domain with uniform grid spacing h, thenits discrete L2 norm is given by

‖uh‖h =

(hd

∑i

(uhi )2

)1/2

,

which is the usual Euclidean vector norm, scaled by a factor (hd) that dependson the geometry of the problem. This scaling factor is introduced to make thediscrete L2 norm an approximation to the continuous L2 norm of a functionu(x), which is given by

‖u‖2 =(∫

Ω

u(x)dx)1/2

.

For example, with d = 1, let u(x) = xm/2, where m > −1 is an integer.Also, let Ω = [0, 1] with grid spacing h = 1

n . Then the associated vector isuh

i = xm/2i = (ih)m/2. The continuous L2 norm is

‖u‖2 =(∫ 1

0

xm/2

)1/2

dx =1√

m + 1,

while the corresponding discrete L2 norm is

‖uh‖h =

(h

n∑i=1

((ih)m/2)2)1/2

h→0=1√

m + 1.

In this case, the discrete L2 norm approaches the continuous L2 norm as h → 0,which occurs only because of the scaling (see Exercise 18).

where α = θ1 + θ2 and β = θ1 − θ2. Restricting the amplification factor to theoscillatory modes, a subtle analysis [25] reveals that the smoothing factor is givenby

µ = G

(π

2, cos−1

(45

))=

12.

These examples illustrate local mode analysis for relatively elementary prob-lems. The same technique can be extended, usually with more computation andanalysis, to anisotropic equations (for example, εuxx +uyy = f) and line relaxation,as discussed in Chapter 7. It can be used for more general operators (for example,convection-diffusion) and for systems of equations. It can also be applied to otherrelaxation methods with different orderings, some of which lead to new compli-cations. For example, red-black relaxation has the property that Fourier modes

56 Chapter 4

become mixed in pairs (in one dimension) or in groups of four (in two dimensions).Thus, the amplification factor is replaced by an amplification matrix. The exten-sion to the coarse-grid correction scheme [10] on two levels requires an analysis ofinterpolation and restriction in a Fourier setting, a subject discussed in the nextchapter.

Diagnostic Tools

As with any numerical code, debugging can be the most difficult part of creatinga successful program. For multigrid, this situation is exacerbated in two ways.First, the interactions between the various multigrid components are very subtle,and it can be difficult to determine which part of a code is defective. Even moreinsidious is the fact that an incorrectly implemented multigrid code can performquite well—sometimes better than other solution methods! It is not uncommon forthe beginning multigrid user to write a code that exhibits convergence factors in the0.2–0.3 range for model problems, while proper tuning would improve the factorsto something more like 0.05. The difficulty is convincing the user that 0.2–0.3 isnot good enough. After all, this kind of performance solves the problem in very fewcycles. But the danger is that performance that is below par for model problemsmight really expose itself as more complexities are introduced. Diagnostic toolscan be used to detect defects in a multigrid code—or increase confidence in theobserved results.

Achi Brandt has said that “the amount of computational work should be pro-portional to the amount of real physical changes in the computed system” and“stalling numerical processes must be wrong.” These statements challenge us todevelop codes that achieve the best possible multigrid performance. The followingshort list of diagnostic tools should help to achieve that goal. A systematic ap-proach to writing multigrid codes is also given by Brandt in the section “Stages inDeveloping Fast Solvers” in his 1984 Guide [4].

Of course, no short list of debugging techniques can begin to cover all contin-gencies. What we provide here is a limited list of tricks and techniques that canbe useful in evaluating a multigrid code; they often tell more about the symptomsof a defective code than the causes. Nevertheless, with increasing experience, theycan guide the user to develop properly tuned codes that achieve multigrid’s fullpotential.

• Methodical Plan. The process of testing and debugging a multigrid code shouldbe planned and executed methodically. The code should be built in a modularway so that each component can be tested and integrated into the evolvingcode with confidence. It is best to test the algebraic solver first (for example,V-cycles); then the discretization can be tested, followed by the FMG solver,if it is to be used. In other words, the initial focus should be on ensuring thatthe basic cycling process solves the discrete system up to expectations. Thissolver can then be used to test discretization accuracy. Poor discretization,especially at boundaries, is often the source of multigrid inefficiency. There-fore, it is important to test the discretization, perhaps with another solver,when multigrid troubles persist. FMG requires an efficient V-cycle or W-cyclesolver and an accurate discretization method. This means that FMG should

A Multigrid Tutorial 57

be implemented and tested after the solver and the discretization componentsare verified (see last item).

• Starting Simply. This recommendation is perhaps the most obvious: it is al-ways best to begin with basic methods applied to small, simple problems.Simpler cases expose troubles more clearly and make it easier to trace thesources. Test problems should consist of either a discrete system with a knownsolution or the simplest form of the desired PDE (usually this means constantcoefficients, no convection, no nonlinearity, and trivial boundary conditions).A good sequence is to test the solver on the coarsest grid that your codeaccepts, then add one finer level to test the two-grid scheme thoroughly. Onecan then proceed progressively and methodically to larger problems. Oncethe simplest cases show expected performance, complexities can be added oneat a time.

• Exposing Trouble. It is critical to start with simple problems so that potentialdifficulties are kept in reserve. At the same time, it is also important to avoidaspects of the problem that mask troubles. For example, reaction terms canproduce strong enough diagonal dominance in the matrix that relaxation byitself is efficient. These terms should be eliminated in the initial tests ifpossible. Similarly, if the matrix arises from implicit time-stepping applied toa time-dependent problem, then a very large or even infinite time step shouldbe taken in the initial tests.

• Fixed Point Property. Relaxation should not alter the exact solution to thelinear system: it should be a fixed point of the iteration. Thus, using theexact solution as an initial guess should yield a zero residual before and af-ter the relaxation process. Furthermore, the coarse-grid problem takes thetransferred residual as its right side, which means that its solution shouldalso be zero. Because the coarse-grid problem uses zero as the initial guess,it is solved exactly, and the correction passed up to the finer grid is also zero.Therefore, neither relaxation nor coarse-grid correction should alter the exactsolution.

This property can be checked by creating a right-side vector correspondingto the known solution (of the linear system) and then using that solution asan initial guess. The relaxation module should be tested first, after which theV-cycle can be tested. If the output from either differs by more than machineprecision from the input, there must be an error in the code.

• Homogeneous Problem. Applying a multigrid V-cycle code to a homogeneousproblem has the advantage that both the norm of the residual and the normof the error are computable and should decrease to zero (up to machine pre-cision) at a steady rate; it may take eight to ten V-cycles for the steady rateto appear. The predictive mode analysis tools described above can be usedto determine the factor by which the residual norm should decrease; it shouldtend to the asymptotic factor predicted by the smoothing analysis.

• Zero Residuals. A useful technique is to multiply the residual by zero just priorto transferring it to the coarse grid. As in the homogeneous problem, thecoarse-grid problem now has a zero right side, so its solution is zero. Because

58 Chapter 4

the initial guess to the coarse-grid problem is zero, it is solved exactly, and thecorrection (also zero) is passed back to the fine grid. The utility of this test isthat the only part of the code now affecting the approximation is relaxationon the fine grid. This means that the sequence of approximations generatedby V(ν1, ν2)-cycles should be identical to the approximations generated usingonly relaxation; this is an easy property to test.

• Residual Printout. A good technique for monitoring performance is to printout the norm of the residual on each level after relaxation on both the descentand ascent legs of the V-cycle. Here again, it is important that the discreteL2 norm be used, because its scaling makes the error norms comparable fromone level to the next. The residuals should behave in a regular fashion: oneach level, the sequence of norms should decline to machine zero at a steadyrate as the cycling continues. The norm on each level should be smallerafter post-relaxation (on the upward leg) than it was after pre-relaxation (onthe downward leg). This ensures that the coarse-grid correction/relaxationtandem is working on each level.

Because most multigrid codes are recursive in nature (although they may notactually use recursive calls), it is important to note that seeing an abnor-mal residual pattern on a given level does not necessarily mean the code issomehow wrong on that level. More frequently, the flaw exists on all levels,because all levels are treated by the same code. Indeed, the most commonculprits are the intergrid transfer operators and boundary treatments. Moni-toring the sequence of residuals, however, can be more helpful in ascertainingthe presence of a problem than simply observing the overall convergence rateon the fine grid.

• Error Graph. Solver trouble that seems impervious to diagnosis can sometimesbe resolved with a picture of the error. Surface plots of the algebraic errorbefore and after relaxation on the fine grid can be extremely informative.Of course, knowledge of the error is required, so solving the homogeneousproblem can be advantageous here. Is the error oscillatory after coarse-gridcorrection? Is it effectively smoothed by relaxation everywhere? Is thereunusual behavior of the error near special features of the domain such asboundaries or interfaces?

• Two-Level Cycles. For any multigrid method to work, it is necessary that thetwo-level scheme (relaxation and exact coarse-grid correction) work. A usefultechnique is to test the two-level scheme. This may be done by replacing therecursive call for V-cycles with a direct or iterative solver on the coarse grid.If an iterative solver is used (many multigrid cycles on the coarse grid mightactually be used here), the coarse-grid problem should be solved very accu-rately, possibly to machine precision. The correction can then be transferredto the fine grid and applied there.

Another useful trick is to use two-level cycling between specific pairs of levels.In particular, if the residual printouts indicate an abnormal residual on aspecific level, it is useful to perform two-level cycling between that level andthe level below it (or between that level and the one above it). This mayisolate exactly where the problem occurs.

A Multigrid Tutorial 59

• Boundaries. One of the most common sources of errors in multigrid programsis the incorrect treatment of boundary data. Generally, interpolation and re-striction stencils must be altered at boundary points and neighbors of bound-ary points. Issues involved here are often very subtle and errors can be difficultto detect. Among the most common indicators is that V-cycles will show con-vergence factors at or near the predicted values for several V-cycles, only tohave convergence slow down and eventually stall with continued cycling. Thereason is that early in the process, errors at the boundaries have little effect,since the boundary is a lower-dimensional feature of the problem. However,as the cycling continues, the errors from the boundaries propagate into theinterior, eventually infecting the entire domain.

One of the most useful techniques in building or debugging a multigrid code isto separate the effects of the boundary entirely. This can be done by replacingthe given boundary conditions by periodic conditions. A periodic boundaryvalue problem should attain convergence factors near those predicted by modeanalysis, because the assumptions underlying the Fourier analysis are moreclosely met. Once it is determined that periodic boundary conditions workproperly, the actual boundary conditions may be applied. If the convergencefactors degrade with continued cycling, then the treatment at the boundariesmay be suspected. A useful technique is to apply extra relaxation sweepsat the boundaries. Often, the extra relaxations will overcome any boundarydifficulties and the overall results will approach the ideal convergence factors.If this does not occur, then a very careful examination of how each piece ofthe code treats the boundaries is in order. Special attention should be paid tothe intergrid transfer operators at and near the boundaries. Just as it is easyto obtain discretizations that are of lower order on the boundary than theinterior, it is also easy to produce intergrid transfers that fail to be consistentin their treatment of boundary data.

• Symmetry. Matrices that arise in the discretization of self-adjoint problemsare often symmetric. Such is the case for most of the matrices featured inthis book. Many coarsening strategies preserve matrix symmetry when it ispresent, as does the so-called Galerkin scheme introduced in the next chapter.Inadvertent loss of symmetry often occurs at boundaries, especially at cornersand other irregularities of the domain. This loss can lead to subtle difficultiesthat are hard to trace. If the fine- and coarse-grid matrices are supposed tobe symmetric, then this should be tested. This is easily done by comparingentries i, j and j, i of the matrix on each grid.

• Compatibility Conditions. We have not yet discussed compatibility condi-tions, but they arise in the context of Neumann boundary conditions (amongother settings), which we discuss in Chapter 7. A compatibility condition isone that must be enforced on the right-side vector to ensure that the problemis solvable. A common source of error is that, while great care may be takento enforce a compatibility condition on the fine grid, it might not be enforcedon the coarse grids. Sometimes, the compatibility condition is enforced au-tomatically in the coarsening process. However, round-off errors may enterand compound themselves as coarsening proceeds, so it may be worthwhileto enforce the condition explicitly on all coarse grids.

60 Chapter 4

• Linear Performance by Nonlinear Codes. We describe the FAS (Full Ap-proximation Scheme) for nonlinear problems in Chapter 6. A useful techniquefor debugging an FAS code is to begin by applying it to a linear problem. FASreduces to standard multigrid in this case, so an FAS code should exhibitstandard performance on these problems. FAS codes should be written so thenonlinearity can be controlled by a parameter: setting the parameter to zeroyields a linear problem, while increasing the parameter value strengthens thenonlinearity. As the nonlinearity increases, the performance of FAS shouldnot, in general, degrade significantly (at least for reasonable values of theparameter).

• Solutions to the PDE. A known solution to the underlying PDE can be veryuseful in assessing whether a multigrid code is working as it should. Thefirst thing to consider is whether the solution computed from the multigridcode looks like a sampled version of the continuous solution. The first cur-sory examination can be qualitative: Does the overall shape of the computedsolution resemble the exact solution? Are the peaks and valleys in the rightplaces?

If the qualitative comparison is good, more quantitative tests can be per-formed. First, the norm of the error (the difference between the sampledcontinuous solution and the approximation) should be monitored as a func-tion of the number of V-cycles. The discrete L2 norm is usually appropriatehere. This norm should behave in very specific ways, depending on the accu-racy of the discretization. If the known solution has no discretization error(for example, if second-order finite differences are used and the known so-lution is a second degree polynomial), then the error norm should be drivensteadily to machine zero with continued V-cycles. Indeed, the rate at which itgoes to zero should be about the same rate at which the norm of the residualdeclines, and it should reflect the predicted asymptotic convergence factor.

On the other hand, if discretization error is present, then we expect the normto stop decreasing after several V-cycles, and it may even grow slightly. Thisindicates (in a correctly working code) that we have reached the level of dis-cretization error (roughly the difference between the continuous and discretesolutions, as discussed in Chapter 5). Here it is useful to solve the sameproblem repeatedly, with the same right side, on a sequence of grids with de-creasing grid spacing h. If the discretization is, for example, O(h2) accuratein the discrete L2 norm, then the error norms should decrease roughly by afactor of four each time h is halved. Naturally, the fit will not be perfect, butif the code is working correctly, the trend should be very apparent.

• FMG Accuracy. The basic components of an effective FMG scheme are anefficient V-cycle (or W-cycle) solver and an accurate discretization. The ideais to assess the discretization error for a given problem using a sequenceof increasingly finer grids. This can be done by choosing a PDE with aknown solution and solving each level in turn, starting from the coarsest.Each level should be solved very accurately, perhaps using many V-cycles,testing the residual norm to be sure it is small. The computed solution canthen be compared on each level to the PDE solution, evaluated at the gridpoints, using the discrete L2 norm. These comparisons yield discretizationerror estimates on each level. FMG can then be tested by comparing its

A Multigrid Tutorial 61

computed approximation on each level to the PDE solution. The ratio ofthese estimates to the discretization error estimates should be close to one asthe mesh size decreases. This signals that FMG is achieving accuracy to thelevel of discretization error. Properly tuning the FMG scheme by choosing theright number of V-cycles, pre-smoothing sweeps, and post-smoothing sweepsmay be required to achieve this property.

Having discussed a variety of practical matters, it is now time to observe theseideas at work in some numerical experiments.

Numerical example. The first experiment deals with the one-dimensional modelproblem Au = 0. The weighted Jacobi method with ω = 2

3 is applied to thisproblem on a grid with n = 64 points. The initial guess consists of two waves withwavenumbers k = 3 and k = 10. For purposes of illustration, we implementeda modified V-cycle algorithm. Called the immediate replacement algorithm, thisversion makes an error correction directly to the fine grid after every coarse-gridrelaxation. In this way, it is possible to see the immediate effect of each coarse-grid correction on the full, fine-grid solution. Although this version is algebraicallyequivalent to the standard algorithm, it is impractical because it involves an inor-dinate number of intergrid transfers. Nevertheless, it is useful for demonstratingalgorithm performance because it allows us to monitor the effects of individualcoarse-grid operations on the error.

Figures 4.6(a, b) show the course of the algorithm by plotting the maximumnorms of the error and the residual after each coarse-grid correction. The algorithmprogresses from left to right in the direction of increasing work units (WUs). Noticethat the data points are spaced nonuniformly due to the different amount of workdone on each level.

Figure 4.6(a) illustrates five full V-cycles with ν1 = ν2 = 1 relaxation sweepon each grid. This computation requires roughly 15 WUs. In these 15 WUs, theinitial error norm is reduced by about three orders of magnitude, giving an averageconvergence rate of about 0.2 decimal digits per WU. Figure 4.6(b) shows the resultof performing V-cycles with ν1 = ν2 = 2 relaxation sweeps on each level. In thiscase, three V-cycles are done in 20 WUs, giving an average convergence rate ofabout 0.15 decimal digits per WU. In this case, it seems more efficient to relaxonly once on each level. For another problem, a different cycling strategy mightbe preferable. It is also possible to relax, say, twice on the “descent” phase of theV-cycle and only once on the “ascent” phase.

The curves of these two figures also show a lot of regular fine structure. Neitherthe error norm nor the residual norm decreases monotonically. The error curvedecreases rapidly on the “descent” to the coarsest grid, while the residual curvedecreases rapidly on the “ascent” to the finest grid. A detailed account of this finestructure should be left to multigrid aficionados!

The previous experiment indicates that the choice of cycling parameters(ν0, ν1, ν2) for multigrid schemes may not be obvious. They are often chosen apriori, based on analysis or prior experimentation, and they remain fixed through-out the course of the algorithm. For certain problems, there are heuristic methodsfor changing the parameters adaptively [4].

62 Chapter 4

Fig

ure

4.6:

(a).

Imm

edia

tere

plac

emen

tve

rsio

nof

the

V-c

ycle

sche

me

appl

ied

toth

eon

e-di

men

sion

alm

odel

prob

lem

with

n=

64po

ints

with

ν 1=

ν 2=

1re

laxa

tion

swee

pon

each

leve

l.T

hem

axim

umno

rms

ofth

eer

ror

and

the

resi

dual

are

plot

ted

agai

nstW

Us

over

the

cour

seof

five

V-c

ycle

s.

A Multigrid Tutorial 63

Fig

ure

4.6,

cont

inue

d:(b

).Im

med

iate

repl

acem

entve

rsio

nof

the

V-c

ycle

sche

me

appl

ied

toth

eon

e-di

men

sion

alm

odel

prob

lem

with

n=

64po

ints

with

ν 1=

ν 2=

2re

laxa

tion

swee

pson

each

leve

l.T

hem

axim

umno

rms

ofth

eer

ror

and

the

resi

dual

are

plot

ted

agai

nstW

Us

over

the

cour

seof

thre

eV

-cyc

les.

64 Chapter 4

We have devoted much attention to the one-dimensional model problem withthe understanding that many of the algorithms, ideas, and results extend directlyto higher dimensions. It is useful to mention a few issues that arise only in higher-dimensional problems. For example, the basic relaxation schemes have many morevariations. In two dimensions, we can relax by points (updating one unknownat a time in one-dimensional problems) or by lines. In line relaxation, an entirerow or column of the grid is updated at once, which generally requires the directsolution of a tridiagonal system for each line. Line relaxation permits additionaloptions, determined by orderings. The lines may be swept forward, backward, orboth (symmetric relaxation). The lines may be colored and relaxed alternatelyin a red-black fashion (often called zebra relaxation). Or the grid may be sweptalternately by rows and then by columns, giving an alternating direction method.Line relaxation will be discussed further in Chapter 7.

These ideas are further generalized in three dimensions. Here we can relax bypoints, lines, or planes, with various choices for ordering, coloring, and direction.There are also ways to incorporate fast direct solvers for highly structured subprob-lems that may be imbedded within a relaxation sweep. Many of these possibilitieshave been analyzed; many more have been implemented. However, there is stillroom for more work and understanding.

Numerical example. We conclude this chapter with an extensive numerical ex-periment in which several multigrid methods are applied to the two-dimensionalproblem

−uxx − uyy = 2[(1 − 6x2)y2(1 − y2) + (1 − 6y2)x2(1 − x2)

]in Ω, (4.8)

u = 0 on ∂Ω,

where Ω is the unit square, (x, y) : 0 < x < 1, 0 < y < 1. Knowing that theanalytical solution to this problem is

u(x, y) = (x2 − x4)(y4 − y2),

errors can be computed.It should be mentioned in passing that the convergence properties of the basic

relaxation methods carry over directly from one to two dimensions when they areapplied to the model problem. Most importantly, weighted Jacobi and Gauss–Seidelretain the property that they smooth high-frequency Fourier modes effectively andleave low-frequency modes relatively unchanged. A guided eigenvalue calculationthat leads to these conclusions is given in Exercise 12.

We first use red-black Gauss–Seidel relaxation in a V-cycle scheme on fine gridswith n = 16, 32, 64, and 128 points in each direction (four separate experiments).Full weighting and linear interpolation are used. We let e be the vector withcomponents u(xi)−vh

i and compute its discrete L2 norm of the error, ‖·‖h. Becausethe error is not available in most problems, a more practical measure, the discreteL2 norm of the residual rh, is also computed.

Table 4.1 shows the residual and error norms after each V-cycle. For each V-cycle, the two columns labeled ratio show the ratios of ‖rh‖h and ‖e‖h betweensuccessive V-cycles. There are several points of interest. First consider the column

A Multigrid Tutorial 65

n = 16 n = 32

V-cycle ‖rh‖h ratio ‖e‖h ratio ‖rh‖h ratio ‖e‖h ratio

0 6.75e+02 5.45e−01 2.60e+03 5.61e−011 4.01e+00 0.01 1.05e−02 0.02 1.97e+01 0.01 1.38e−02 0.022 1.11e−01 0.03 4.10e−04 0.04 5.32e−01 0.03 6.32e−04 0.053 3.96e−03 0.04 1.05e−04 0.26 2.06e−02 0.04 4.41e−05 0.074 1.63e−04 0.04 1.03e−04 0.98∗ 9.79e−04 0.05 2.59e−05 0.595 7.45e−06 0.05 1.03e−04 1.00∗ 5.20e−05 0.05 2.58e−05 1.00∗

6 3.75e−07 0.05 1.03e−04 1.00∗ 2.96e−06 0.06 2.58e−05 1.00∗

7 2.08e−08 0.06 1.03e−04 1.00∗ 1.77e−07 0.06 2.58e−05 1.00∗

8 1.24e−09 0.06 1.03e−04 1.00∗ 1.10e−08 0.06 2.58e−05 1.00∗

9 7.74e−11 0.06 1.03e−04 1.00∗ 7.16e−10 0.06 2.58e−05 1.00∗

10 4.99e−12 0.06 1.03e−04 1.00∗ 4.79e−11 0.07 2.58e−05 1.00∗

11 3.27e−13 0.07 1.03e−04 1.00∗ 3.29e−12 0.07 2.58e−05 1.00∗

12 2.18e−14 0.07 1.03e−04 1.00∗ 2.31e−13 0.07 2.58e−05 1.00∗

13 2.33e−15 0.11 1.03e−04 1.00∗ 1.80e−14 0.08 2.58e−05 1.00∗

14 1.04e−15 0.45 1.03e−04 1.00∗ 6.47e−15 0.36 2.58e−05 1.00∗

15 6.61e−16 0.63 1.03e−04 1.00∗ 5.11e−15 0.79 2.58e−05 1.00∗

n = 64 n = 128

V-cycle ‖rh‖h ratio ‖e‖h ratio ‖rh‖h ratio ‖e‖h ratio

0 1.06e+04 5.72e−01 4.16e+04 5.74e−011 7.56e+01 0.01 1.39e−02 0.02 2.97e+02 0.01 1.39e−02 0.022 2.07e+00 0.03 6.87e−04 0.05 8.25e+00 0.03 6.92e−04 0.053 8.30e−02 0.04 4.21e−05 0.06 3.37e−01 0.04 4.22e−05 0.064 4.10e−03 0.05 7.05e−06 0.17 1.65e−02 0.05 3.28e−06 0.085 2.29e−04 0.06 6.45e−06 0.91∗ 8.99e−04 0.05 1.63e−06 0.506 1.39e−05 0.06 6.44e−06 1.00∗ 5.29e−05 0.06 1.61e−06 0.99∗

7 8.92e−07 0.06 6.44e−06 1.00∗ 3.29e−06 0.06 1.61e−06 1.00∗

8 5.97e−08 0.07 6.44e−06 1.00∗ 2.14e−07 0.06 1.61e−06 1.00∗

9 4.10e−09 0.07 6.44e−06 1.00∗ 1.43e−08 0.07 1.61e−06 1.00∗

10 2.87e−10 0.07 6.44e−06 1.00∗ 9.82e−10 0.07 1.61e−06 1.00∗

11 2.04e−11 0.07 6.44e−06 1.00∗ 6.84e−11 0.07 1.61e−06 1.00∗

12 1.46e−12 0.07 6.44e−06 1.00∗ 4.83e−12 0.07 1.61e−06 1.00∗

13 1.08e−13 0.07 6.44e−06 1.00∗ 3.64e−13 0.08 1.61e−06 1.00∗

14 2.60e−14 0.24 6.44e−06 1.00∗ 1.03e−13 0.28 1.61e−06 1.00∗

15 2.30e−14 0.88 6.44e−06 1.00∗ 9.19e−14 0.89 1.61e−06 1.00∗

Table 4.1: The V(2,1) scheme with red-black Gauss–Seidel applied to a two-dimensional problem on fine grids with n = 16, 32, 64, and 128 points. The discreteL2 norms of the residual and error are shown after each V-cycle. The ratio columnsgive the ratios of residual and error norms of successive V-cycles. The ∗ in the errorratio column indicates that the level of discretization error has been reached.

66 Chapter 4

of error norms. For each of the four grid sizes, the error norms decrease rapidly andthen level off abruptly as the scheme reaches the level of discretization error. Weconfirm this by comparing the final error norms, ‖e‖h, on the four grids (1.03e − 04,2.58e − 05, 6.44e − 06, and 1.61e − 06). These norms decrease by a factor of fouras the resolution doubles, which is consistent with the second-order discretizationwe have used. The residual norms also decrease rapidly for 12 to 14 V-cycles,with the value in the corresponding ratio column reaching a nearly constant value,until the last few cycles. This constant value is a good estimate of the asymptoticconvergence factor of the scheme (approximately 0.07) for this problem. The sharpincrease in the residual norm ratio during the last two cycles reflects the fact thatthe algebraic approximation is already accurate to near machine precision.

In the course of our development, we described several different schemes forrelaxation, restriction, and interpolation. Specifically, we worked with weightedJacobi, Gauss–Seidel, and red-black Gauss–Seidel relaxation schemes; injection andfull weighting restriction operators; and linear interpolation. We now investigatehow various combinations of these schemes perform when used in V-schemes appliedto model problem (4.8).

For completeness, we introduce two more schemes, half-injection and cubic inter-polation. Half-injection, as the name implies, is simply defined in one dimension byv2h

j = 0.5vh2j , with a similar extension to two dimensions. Half-injection is designed

for use on the model problem with red-black relaxation and may be understoodmost easily by considering the one-dimensional case. The idea is that because onesweep of the red-black relaxation produces zero residuals at every other point, full-weighting and half-injection are equivalent. We will see shortly that the schemeindeed works well for this case.

Cubic interpolation is one of the many interpolation schemes that could beapplied and is probably the most commonly used in multigrid after linear interpo-lation. As its name implies, the method interpolates cubic (or lower degree) poly-nomials exactly. It represents a good compromise between the desire for greaterinterpolation accuracy and the increase in computational cost required to achievethe desired accuracy. In one dimension, the basic cubic interpolation operator isgiven by

vh2j = v2h

j ,

vh2j+1 =

116

(−v2h

j−1 + 9v2hj + 9v2h

j+1 − v2hj+2

).

Table 4.2 gives comparative results for many experiments. For each of the Ja-cobi, Gauss–Seidel, and red-black Gauss–Seidel relaxation schemes, we performedsix V-cycles, using all possible combinations of the half-injection, injection, and fullweighting restriction operators with linear or cubic interpolations. In each case, theexperiment was performed using (1,0), (1,1), and (2,1) V-cycles, where (ν1, ν2) indi-cates ν1 pre-correction relaxation sweeps and ν2 post-correction relaxation sweeps.The entries in the table give the average convergence factor for the last five V-cyclesof each experiment. A dash indicates that a particular scheme diverged. The en-tries in the cost line reflect the cost of the method, in terms of operation count,shown as a multiple of the cost of the (1,0) scheme using linear interpolation andinjection. Notice that the cost is independent of the different relaxation schemes,as they all perform the same number of operations.

A Multigrid Tutorial 67

Relaxation Injection Full Weighting Half-Injection(ν1, ν2) Scheme Linear Cubic Linear Cubic Linear Cubic(1,0) Jacobi – – 0.49 0.49 0.55 0.62

GS 0.89 0.66 0.33 0.34 0.38 0.37RBGS – – 0.21 0.23 0.45 0.42Cost 1.00 1.25 1.13 1.39 1.01 1.26

(1,1) Jacobi 0.94 0.56 0.35 0.34 0.54 0.52GS 0.16 0.16 0.14 0.14 0.45 0.43

RBGS – – 0.06 0.05 0.12 0.16Cost 1.49 1.75 1.63 1.88 1.51 1.76

(2,1) Jacobi 0.46 0.31 0.24 0.24 0.46 0.45GS 0.07 0.07 0.08 0.07 0.40 0.39

RBGS – – 0.04 0.03 0.03 0.07Cost 1.99 2.24 2.12 3.37 1.51 1.76

Table 4.2: Average convergence factors over five V-cycles on model problem (4.8)for various combinations of relaxation, restriction, and interpolation operators. Thedashes indicate divergent schemes. The cost line gives the computational cost of aV(ν1, ν2)-cycle scheme using the specified choice of restriction and interpolation, asa multiple of the cost of a (1,0) V-cycle scheme using injection and linear interpo-lation.

A few observations are in order. At least for this problem, cubic interpolationis only noticeably more effective than linear interpolation when injection is usedas the restriction operator. It is also apparent that half-injection is useful onlywith red-black Gauss–Seidel, as expected; even then, several smoothing sweepsare required. Finally, and not surprisingly, the table indicates that you get whatyou pay for: combinations that produce the best convergence factors are also thosewith the higher costs. Parameter selection is largely the art of finding a compromisebetween performance and cost.

The final set of experiments concerns the effectiveness of FMG schemes and,in particular, whether FMG schemes are more or less effective than V-cycles.Table 4.3 describes the performance of three FMG schemes. Square grids withup to n = 2048 points in each direction are used. The FMG(ν1,ν2) scheme uses ν1

relaxation sweeps on the descent phase and ν2 relaxation sweeps on the ascent phaseof each V-cycle. Red-black Gauss–Seidel relaxation is used with full weighting andlinear interpolation. In each case, only one complete FMG cycle is done.

The table shows the discrete L2 norm of the error for each scheme. Evidencethat the FMG code solves the problem to the level of discretization error on eachgrid is that the ratio of the error norms between successive levels is roughly 0.25(for this two-dimensional problem). If the ratio is noticeably greater than 0.25,the solution is probably not accurate to the level of discretization. Based on thisobservation, we suspect that the FMG(1,0) scheme does not solve the problem tothe level of discretization error on any level. This is confirmed when we observe theFMG(1,1) and FMG(2,1) schemes, which do solve to the level of the discretizationerror on all levels. The FMG(2,1) scheme requires more work than the FMG(1,1)with little gain in accuracy; so it appears that FMG(1,1) is the best choice for thisproblem.

68 Chapter 4

FMG(1,0) FMG(1,1) FMG(2,1) FMG(1,1) V(2,1) V(2,1)N ‖e‖h ratio ‖e‖h ratio ‖e‖h ratio WU cycles WU

2 5.86e−03 5.86e−03 5.86e−034 5.37e−03 0.917 2.49e−03 0.424 2.03e−03 0.347 7/2 3 128 2.78e−03 0.518 9.12e−04 0.367 6.68e−04 0.328 7/2 4 1616 1.19e−03 0.427 2.52e−04 0.277 1.72e−04 0.257 7/2 4 1632 4.70e−04 0.395 6.00e−05 0.238 4.00e−05 0.233 7/2 5 2064 1.77e−04 0.377 1.36e−05 0.227 9.36e−06 0.234 7/2 5 20128 6.49e−05 0.366 3.12e−06 0.229 2.26e−06 0.241 7/2 6 24256 2.33e−05 0.359 7.35e−07 0.235 5.56e−07 0.246 7/2 7 28512 8.26e−06 0.354 1.77e−07 0.241 1.38e−07 0.248 7/2 7 281024 2.90e−06 0.352 4.35e−08 0.245 3.44e−08 0.249 7/2 8 322048 1.02e−06 0.351 1.08e−08 0.247 8.59e−09 0.250 7/2 9 36

Table 4.3: Three different FMG schemes applied to the two-dimensional problemon square grids with up to n = 2048 points in each direction. The FMG(ν1,ν2)scheme uses ν1 red-black Gauss–Seidel relaxation sweeps on the descent phase andν2 relaxation sweeps on the ascent phase of each V-cycle. The discrete L2 normof the error and the ratio of errors at each grid level are shown. Solution to thelevel of discretization error is indicated when the ratio column shows a reductionof at least 0.25 in the error norm. For comparison, the V-cycles column shows thenumber of V(2,1)-cycles needed to converge to the level of discretization error, whilethe V-cycle WU column shows the number of work units needed to converge to thelevel of discretization error.

A useful question is whether an FMG(1,1) scheme is more efficient than, say,the V(2,1) scheme in achieving a solution accurate to the level of discretizationerror. We answer this question by performing the V(2,1) method (as in Table 4.1)for all grid sizes from n = 4 through n = 2048 (over 4 million fine-grid points!) andrecording the number of V-cycles required to converge to the level of discretizationerror. These results are presented in the second-to-last column of Table 4.3. It isapparent that the number of cycles required to solve to the level of discretizationerror increases with the problem size.

We can now make some comparisons. Recall our discussion of computationalcosts earlier in the chapter. We determined that a (1,1) V-cycle in d = 2 dimensionscosts about 8

3 WUs (Exercise 3); therefore, a (2,1) V-cycle costs half again as much,or 4 WU. The last column of Table 4.3 shows the costs in WUs of solving to thelevel of discretization error with the V(2,1) scheme on various grids. We also saw(Exercise 4) that the FMG(1,0) scheme, which did not converge to the level ofdiscretization error in this case, requires just under 2 WUs, while the FMG(1,1) andFMG(2,1) schemes, which did achieve the desired accuracy, require approximately72 and 16

3 WUs, respectively; these costs are the same for all grid sizes. Thus, on allof the grids shown in Table 4.3, the FMG(1,1) scheme is significantly less expensivein WUs than the V(2,1) scheme. This confirms the observation that for convergingto the level of discretization error, full multigrid methods are generally preferableto simple V-cycles.

Exercises

Data Structures and Complexity

1. Data structures. Work out the details of the data structures given in Fig.4.1. Assume that for a one-dimensional problem, the finest grid has n − 1 =

A Multigrid Tutorial 69

2L − 1 interior points. Let h = 1n be the grid spacing on Ωh. Let level l

have grid spacing 2l−1h. As suggested in the text, store the approximationsvh,v2h, . . . contiguously in a single array v, with the level L values stored inv1, v2, v3; the level L− 1 values in v4, . . . , v8; etc. Use a similar arrangementfor the right-side values fh, f2h, . . . . How many values are stored on level l,where 1 ≤ l ≤ L? What is the starting index in the v array for the level lvalues, where 1 ≤ l ≤ L?

2. Data structure for two dimensions. Now consider the two-dimensionalmodel problem. The one-dimensional data structure may be retained in themain program. However, the initial index for each grid will now be different.Compute these indices, assuming that on the finest grid Ωh there are (n−1)2

interior points, where n − 1 = 2L − 1.

3. Storage requirements. Verify the statement in the text that for a one-dimensional problem (d = 1), the storage requirement is less than twice thatof the fine-grid problem alone. Show that for problems in two or more dimen-sions, the requirement drops to less than 4

3 of the fine-grid problem alone.

4. V-cycle computation cost. Verify the statement in the text that a singleV-cycle costs about 4 WUs for a one-dimensional (d = 1) problem, about 8

3WUs for d = 2, and 16

7 WUs for d = 3.

5. FMG computation cost. Verify the statement in the text that an FMGcycle costs 8 WUs for a one-dimensional problem; the cost is about 7

2 WUsfor d = 2 and 5

2 WUs for d = 3.

Local Mode Analysis

6. One-dimensional weighted Jacobi.

(a) Verify the Jacobi updating step (4.2).

(b) Show that the error ej = u(xj) − vj satisfies (4.3).

(c) Verify that the amplification factor for the method is given by

G(θ) = 1 − 2ω sin2

(θ

2

).

7. One-dimensional Gauss–Seidel. Verify the error updating step (4.4).Then show that the amplification factor for the method is given by

G(θ) =eiθ

2 − e−iθ.

8. Two-dimensional weighted Jacobi.

(a) Verify the error updating step (4.6).

(b) Show that the amplification factor for the method is given by

G(θ1, θ2) = 1 − ω

(sin2

(θ1

2

)+ sin2

(θ2

2

)).

70 Chapter 4

(c) Show that the optimal smoothing factor is obtained with ω = 45 and

that its value is µ = |G(±π,±π)| = 0.6. Hint: Note that µ(ω) =max |G(θ1, θ2)| is a function of ω. The optimal value of ω is that whichminimizes µ, viewed as a function of ω. The substitutions ξ = sin2( θ1

2 ),η = sin2( θ2

2 ) may be helpful.

9. Two-dimensional Gauss–Seidel.

(a) Verify the error updating step (4.7).(b) Show that the amplification factor for the method is given by

G(θ1, θ2) =eιθ1 + eιθ2

4 − e−ιθ1 − e−ιθ2.

(c) Show that the smoothing factor is given by

µ = G

(π

2, cos−1

(45

))=

12.

10. Nine-point stencil. Consider the nine-point stencil for the operator −uxx−uyy given by

13h2

−1 −1 −1

−1 8 −1−1 −1 −1

.

Find the amplification factors for the weighted Jacobi method and Gauss–Seidel relaxation applied to this system.

11. Anisotropic operator. Consider the five-point stencil for the operator−εuxx − uyy given by

1h2

0 −1 0

−ε 2(1 + ε) −ε0 −1 0

.

Find the amplification factors for weighted Jacobi and Gauss–Seidel appliedto this system. Discuss the effect of the parameter ε in the case that ε << 1.

12. Eigenvalue calculation in two dimensions. Consider the weighted Jacobimethod applied to the model Poisson equation in two dimensions on the unitsquare. Assume a uniform grid of h = 1

n in each direction.

(a) Let vij be the approximation to the solution at the grid point (xi, yj).Write the (i, j)th equation of the corresponding discrete problem, where1 ≤ i, j ≤ n − 1.

(b) Letting A be the matrix of coefficients for the discrete system, write the(i, j)th equation for the eigenvalue problem Av = λv.

(c) Assume an eigenvector solution of the form

vij = sin(

ikπ

n

)sin

(jπ

n

), 1 ≤ k, ≤ n − 1.

Using sine addition rules, simplify this eigenvalue equation, cancel com-mon terms, and show that the eigenvalues are

λk = 4[sin2

(kπ

2n

)+ sin2

(π

2n

)], 1 ≤ k, ≤ n − 1.

A Multigrid Tutorial 71

(d) As in the one-dimensional case, note that the iteration matrix of theweighted Jacobi method is given by Pω = I − ωD−1A, where D corre-sponds to the diagonal terms of A. Find the eigenvalues of Pω.

(e) Using a graphing utility, find a suitable way to present the two-dimensionalset of eigenvalues (either a surface plot or multiple curves). Plot theeigenvalues for ω = 2

3 , 45 , 1, and n = 16.

(f) In each case, discuss the effect of the weighted Jacobi method on low-and high-frequency modes. Be sure to note that modes can have a highfrequencies in one direction and low frequencies in the other direction.

(g) What do you conclude about the optimal value of ω for the two-dimensionalproblem?

Implementation

13. V-Cycle program. Develop a V-cycle program for the one-dimensionalmodel problem. Write a subroutine for each individual component of thealgorithm as follows.

(a) Given an approximation array v, a right-side array f, and a level number1 ≤ l ≤ L, write a subroutine that will carry out ν weighted Jacobisweeps on level l.

(b) Given an array f and a level number 1 ≤ l ≤ L − 1, write a subroutinethat will carry out full weighting between level l and level l + 1.

(c) Given an array v and a level number 2 ≤ l ≤ L, write a subroutine thatwill carry out linear interpolation between level l and level l − 1.

(d) Write a driver program that initializes the data arrays and carries out aV-cycle by calling the three preceding subroutines. The program shouldbe tested on simple problems for which the exact solution is known. Forexample, for fixed k, take f(x) = C sin(kπx) on the interval 0 ≤ x ≤ 1,where C is a constant. Then

u(x) =C

π2k2 + σsin(kπx)

is an exact solution to model problem (1.1). Another subroutine thatcomputes norms of errors and residuals will be useful.

14. Modification of V-cycle code. It is now easy to modify this program andmake comparisons.

(a) Vary ν and vary the number of V-cycles.

(b) Replace the weighted Jacobi subroutine, first by a Gauss–Seidel subrou-tine, then by a red-black Gauss–Seidel subroutine.

(c) Replace the full weighting subroutine by an injection subroutine. Ob-serve that using red-black Gauss–Seidel and injection impairs conver-gence. Try to remedy this by using half-injection. In all cases, determineexperimentally how these changes affect convergence rates and compu-tation time.

(d) Explain the result of using black-red (rather than red-black) Gauss–Seidel with injection.

72 Chapter 4

15. Two-dimensional program. For the two-dimensional problem, proceedagain in a modular way.

(a) Write a subroutine that performs weighted Jacobi on a two-dimensionalgrid. Within the subroutine, it is easiest to refer to v and f as two-dimensional arrays.

(b) Make the appropriate modifications to the one-dimensional code to im-plement bilinear interpolation and full weighting on a two-dimensionalgrid.

(c) Make the (minor) changes required in the main program to create atwo-dimensional V-cycle program. Test this program on problems withknown exact solutions. For example, for fixed k and , take f(x, y) =C sin(kπx) sin(πy) on the unit square (0 ≤ x, y ≤ 1), where C is aconstant. Then

u(x, y) =C

π2k2 + π22 + σsin(kπx) sin(πy)

is an exact soultion to model problem (1.4).

16. FMG programs. Modify the one- and two-dimensional V-cycle programsto carry out the FMG scheme.

17. A convection-diffusion problem. Consider the following convection-diffusionproblem on the unit square Ω = (x, y) : 0 < x < 1, 0 < y < 1:

−ε(uxx + uyy) + aux = A sin(πy)(C2x2 + C1x + C0) on Ω,

u = 0 on ∂Ω,

where ε > 0, A ∈ R, a ∈ R, is an integer, C2 = −ε2π2, C1 = ε2π2 − 2a,and C0 = a+2ε. It has the exact solution u(x, y) = Ax(1−x) sin(πy). Applythe multigrid algorithms discussed in this chapter to this problem. Comparethe algorithms and explore how their performance changes for ε = 0.01, 0.1, 1;a = 0.1, 1, 10; n = 32, 64, 128; l = 1, 3, 16.

18. Discrete L2 norm. Let u(x) = xm/2, where m > −1 is an integer, onΩ = [0, 1], with grid spacing h = 1

n . Let uhi = x

m/2i = (ih)m/2. Show that

the continuous L2 norm is

‖u‖2 =(∫ 1

0

xm/2

)1/2

=1√

m + 1,

while the corresponding discrete L2 norm satisfies

‖uh‖hh→0=

1√m + 1

.

Chapter 5

Some Theory

In the previous chapter, we considered some practical questions concerning theimplementation, cost, and performance of multigrid methods. The arguments andexperiments of that chapter offer good reason to believe that multigrid methods canbe extremely effective. Now we must confront some questions on the theoreticalside. The goal of this chapter is to present multigrid in a more formal settingand offer an explanation of why these methods work so well. In the first part ofthis chapter, we sketch the ideas that underlie the convergence theory of multigrid.In the second section, we present what might be called the subspace picture ofmultigrid. While the terrain in this chapter may seem a bit more rugged than inprevious chapters, the reward is an understanding of why multigrid methods are soremarkably effective.

Variational Properties

We first return to a question left unanswered in previous chapters. In expressing thecoarse-grid problem, we wrote A2hu2h = f2h and said that A2h is the Ω2h versionof the original operator Ah. Our first goal is to define the coarse-grid operator A2h

precisely.The argument that follows assumes we are working with the model problem,

−u′′(x) = f(x), and the corresponding discrete operator Ah. We adopt the notationthat Ωph represents not only the grid with grid spacing ph, but also the spaceof vectors defined on that grid. In keeping with our former notation, we let vh

be a computed approximation to the exact solution uh. For the purpose of thisargument, assume that the error in this approximation, eh = uh − vh, lies entirelyin the range of interpolation, which will be denoted R(Ih

2h). This means that forsome vector u2h ∈ Ω2h, eh = Ih

2hu2h. Therefore, the residual equation on Ωh may

be writtenAheh = AhIh

2hu2h = rh. (5.1)

In this equation, Ah acts on a vector that lies entirely in the range of interpolation.How does Ah act on R(Ih

2h)? Figure 5.1 gives the answer. An arbitrary vectoru2h ∈ Ω2h is shown in Fig. 5.1(a); Ih

2hu2h, which is the interpolation of u2h up

to Ωh, is shown in Fig. 5.1(b); and the effect of Ah acting pointwise on Ih2hu

2h

is shown in Fig. 5.1(c). We see that AhIh2hu

2h is zero at the odd grid points of

73

74 Chapter 5

Figure 5.1: The action of Ah on the range of interpolation R(Ih2h): (a) an arbitrary

vector u2h ∈ Ω2h; (b) the linear interpolant Ih2hu

2h; and (c) AhIh2hu

2h, which iszero at the odd grid points on Ωh.

Ωh. The effect is analogous to taking the second derivative of a piecewise linearfunction.

We may conclude that the odd rows of AhIh2h in (5.1) are zero. On the other

hand, the even rows of (5.1) correspond to the coarse-grid points of Ω2h. Therefore,we can find a coarse-grid form of the residual equation by dropping the odd rowsof (5.1). We do this formally by applying the restriction operator I2h

h to both sidesof (5.1). When this is done, the residual equation becomes

I2hh AhIh

2h︸ ︷︷ ︸A2h

u2h = I2hh rh.

This observation gives us a plausible definition for the coarse-grid operator:A2h = I2h

h AhIh2h. The terms of A2h may be computed explicitly as shown in

Table 5.1. We simply apply I2hh AhIh

2h term by term to the jth unit vector e2hj on

Ω2h. This establishes that the jth column of A2h and also, by symmetry, the jthrow of A2h are given by

1(2h)2

(−1 2 − 1

).

We would get the same result if the original problem were simply discretized onΩ2h using the usual second-order finite differences. Therefore, by this definition,A2h really is the Ω2h version of Ah.

The preceding argument was based on the assumption that the error eh liesentirely in the range of interpolation. This is not the case in general. If it were,then solving the Ω2h residual equation exactly and doing the two-grid correctionwould give the exact solution. Nevertheless, the argument does give a sensible def-inition for A2h. It also leads us to two important properties called the variational

A Multigrid Tutorial 75

j − 1 j j + 1

e2hj 0 1 0

Ih2he

2hj 0 1

2 1 12 0

AhIh2he

2hj − 1

2h2 0 1h2 0 − 1

2h2

I2hh AhIh

2he2hj − 1

4h21

2h2 − 14h2

Table 5.1: Calculation of the ith row of A2h = I2hh AhIh

2h.

properties; they are given by

A2h = I2hh AhIh

2h (Galerkin condition),I2hh = c(Ih

2h)T , c ∈ R.

The first of these, the Galerkin condition, is simply the definition of the coarse-gridoperator. The second property is the relationship satisfied by the interpolation op-erator and the full weighting operator defined in Chapter 3. While these propertiesare not desirable for all applications, they are exhibited by many commonly usedoperators. They also facilitate the analysis of the two-grid correction scheme. Wesee a deeper justification of these properties in Chapter 10.

Toward Convergence Theory

Convergence analysis of multigrid methods is difficult and has occupied researchersfor several decades. We cannot even pretend to address this problem with the rigorand depth it deserves. Instead, we attempt to give heuristic arguments suggestingthat the standard multigrid schemes, when applied to well-behaved problems (forexample, scalar elliptic problems), not only work, but work very effectively. Con-vergence results for these problems have been rigorously proved. For more generalproblems, new results appear at a fairly steady pace. Where analytical results arelacking, a wealth of computational evidence testifies to the general effectiveness ofmultigrid methods. Between analysis and experimentation, the multigrid territoryis slowly being mapped. However, multigrid convergence analysis is still an openarea of computational mathematics.

We begin with a heuristic argument that captures the spirit of rigorous conver-gence proofs. As we have seen, the smoothing rate (the convergence factor for theoscillatory modes) for the standard relaxation schemes is small and independentof the grid spacing h. Recall that the smooth error modes, which remain afterrelaxation on one grid, appear more oscillatory on the coarser grids. Therefore,by moving to successively coarser grids, all of the error components on the originalfine grid eventually appear oscillatory and are reduced by relaxation. It then follows

76 Chapter 5

that the overall convergence factor for a good multigrid scheme should be smalland independent of h.

Now we can be a bit more precise. Denote the original continuous problem (forexample, one of our model boundary value problems) by Au = f . The associateddiscrete problem on the fine grid Ωh is denoted by Ahuh = fh. As before, we letvh be an approximation to uh on Ωh. The discretization error is defined by

Ehi = u(xi) − uh

i , 1 ≤ i ≤ n − 1.

The discretization error measures how well the exact solution of the discrete problemapproximates the exact solution of the original continuous problem. It may bebounded in the discrete L2 norm in the form

‖Eh‖h ≤ Khp, (5.2)

where K is a positive constant and p is a positive integer. For the model problemsin Chapter 1, in which second-order finite differences were used, we have p = 2 (seeExercise 11 for a careful derivation of this fact).

Unfortunately, we can seldom solve the discrete problem exactly. The quantitythat we have been calling the error, eh = uh − vh, will now be called the alge-braic error to avoid confusion with the discretization error. The algebraic error, aswe have seen, measures how well our approximations (generated by relaxation ormultigrid) agree with the exact discrete solution.

The purpose of a typical calculation is to produce approximations vh that agreewith the exact solution of the continuous problem u. Let us specify a tolerance εand an error condition such as

‖u − vh‖h < ε, (5.3)

where u = (u(x1), . . . , u(xn−1))T is the vector of exact solution values sampledat the grid points. This condition can be satisfied if we guarantee that both thediscretization and algebraic errors are small. Suppose, for example, that

‖Eh‖h + ‖eh‖h < ε.

Then, using the triangle inequality,

‖u − vh‖h ≤ ‖u − uh‖h + ‖uh − vh‖h = ‖Eh‖h + ‖eh‖h < ε.

One way to ensure that ‖Eh‖h + ‖eh‖h < ε is to require that ‖Eh‖h < ε2 and

‖eh‖h < ε2 individually. The first condition determines the grid spacing on the

finest grid. Using (5.2), it suggests that we choose

h < h∗ ≡( ε

2K

)1/p

.

The second condition determines how well vh must approximate the exact dis-crete solution uh. If relaxation or multigrid cycles have been performed until thecondition ‖eh‖ < ε

2 is met on grid Ωh, where h < h∗, then we have converged tothe level of discretization error. In summary, the discretization error determinesthe critical grid spacing h∗; so (5.3) will be satisfied provided we converge to thelevel of discretization error on a grid with h < h∗.

A Multigrid Tutorial 77

Consider first a V-cycle scheme applied to a d-dimensional problem with (about)nd unknowns and h = 1

n . We assume (and can generally show rigorously) that withfixed cycling parameters, ν1 and ν2, the V-cycle scheme has a convergence factorbound, γ, that is independent of h. This V-cycle scheme must reduce the algebraicerror from O(1) (the error in the zero initial guess) to O(hp) = O(n−p) (the orderof the discretization error). Therefore, the number of V-cycles required, ν, mustsatisfy γν = O(np) or ν = O(log n). Because the cost of a single V-cycle is O(nd),the cost of converging to the level of discretization error with a V-cycle scheme isO(nd log n). This is comparable to the computational cost of the best fast directsolvers applied to the model problem.

The FMG scheme costs a little more per cycle than the V-cycle scheme. How-ever, a properly designed FMG scheme can be much more effective overall becauseit supplies a very good initial guess to the final V-cycles on Ωh. The key obser-vation in the FMG argument is that before the Ωh problem is even touched, theΩ2h problem has already been solved to the level of discretization error. This isbecause of nested iteration, which is designed to provide a good initial guess forthe next finer grid. Our goal is to determine how much the algebraic error needsto be reduced by V-cycles on Ωh. The argument is brief and worthwhile; however,it requires a new tool.

Energy Norms, Inner Products, and Orthogonality. Energy norms andinner products are useful tools in the analysis of multigrid methods. Theyinvolve a slight modification of the inner product and norms that we havealready encountered. Suppose A is an n×n symmetric positive definite matrix.Define the A-inner product of two vectors u,v ∈ Rn by

(u,v)A ≡ (Au,v),

where (·, ·) is the usual Euclidean inner product on Rn. The A-norm nowfollows in a natural way. Just as ‖u‖ = (u,u)

12 , the A-norm is given by

‖u‖A = (u,u)12A.

The A-norm and A-inner product are sometimes called the energy norm andinner product. We can also use the A-inner product to define a new orthogo-nality relationship. Extending the usual notion of vectors and subspaces, twovectors u and v are A-orthogonal if (u, v)A = 0, and we write u ⊥A v. Wethen say that two subspaces U and V are A-orthogonal if, for all u ∈ U andv ∈ V , we have u ⊥A v. In this case, we write U ⊥A V .

Our FMG argument can be made in any norm, but it is simplest in the Ah-norm. The goal is to show that one properly designed FMG cycle is enough toensure that the final algebraic error on Ωh is below the level of discretization error;that is,

‖eh‖Ah ≤ Khp, (5.4)

where K is a positive constant that depends on the smoothness of the solution andp is a positive integer. The values of K and p also depend on the norm used tomeasure the error, so they will generally be different from the constants in (5.2).

78 Chapter 5

The argument is inductive in nature. If Ωh is the coarsest grid, then FMG isexact and (5.4) is clearly satisfied. Assume now that the Ω2h problem has beensolved to the level of discretization error, so that

‖e2h‖A2h ≤ K(2h)p. (5.5)

We now use (5.5) to prove (5.4).The initial algebraic error on Ωh, before the V-cycles begin, is the difference

between the exact fine-grid solution, uh, and the coarse-grid approximation inter-polated to the fine grid:

eh0 = uh − Ih

2hv2h.

To estimate the size of this initial error, we must account for the error that mightbe introduced by interpolation from Ω2h to Ωh. To do this, we assume that theerror in interpolation has the same order (same value of p) as the discretizationerror and satisfies

‖uh − Ih2hu

2h‖Ah ≤ Kαhp, (5.6)

where α is a positive constant. (This sort of bound can be determined rigorously;in fact, α is typically 1 + 2p.) A bound for the initial error now follows from thetriangle inequality:

‖eh0‖Ah = ‖uh − Ih

2hv2h‖Ah

≤ ‖uh − Ih2hu

2h‖Ah + ‖Ih2hu

2h − Ih2hv

2h‖Ah (triangle inequality)= ‖uh − Ih

2hu2h‖Ah︸ ︷︷ ︸

≤Kαhp

+ ‖u2h − v2h‖A2h︸ ︷︷ ︸≤K(2h)p

(Galerkin conditions).

As indicated, we use (5.5), (5.6), and Exercise 2 to form the following estimate forthe norm of the initial error:

‖eh0‖Ah ≤ Kαhp + K(2h)p = K(α + 2p)hp.

To satisfy (5.4), the algebraic error must be reduced from roughly K(α + 2p)hp toKhp. This means we should use enough V-cycles on Ωh to reduce the algebraicerror by a factor of

β = α + 2p.

This reduction requires ν V-cycles, where γν ≤ β. Because β is O(1), it follows thatν is also O(1). (Typically, β = 5 or 9 and γ ≈ 0.1 for a V(2,1)-cycle, so ν = 1.) Inother words, the preliminary cycling through coarser grids gives such a good initialguess that only O(1) V-cycles are needed on the finest grid. This means that thetotal computational cost of FMG is O(nd), which is optimal.

This discussion is meant to give some feeling for the rigorous arguments that canbe used to establish the convergence properties of the basic multigrid algorithms.These results cannot be pursued much further at this point and must be left to themultigrid literature. Instead, we turn to a different perspective on why multigridworks.

Spectral and Algebraic Pictures

Much of this section is devoted to an analysis of the two-grid correction scheme,which lies at the heart of multigrid. Recall that the V-cycle is just nested ap-plications of the two-grid correction scheme and that the FMG method is just

A Multigrid Tutorial 79

Figure 5.2: The full weighting operator I2hh acting on (a) a smooth mode of Ωh

(k = 2 and n = 8) and (b) an oscillatory mode of Ωh (k = 6 and n = 8). In thefirst case, the result is a multiple of the k = 2 mode on Ω2h. In the second case,the result is a multiple of the n − k = 2 mode on Ω2h.

repeated applications of the V-cycle on various grids. Therefore, an understandingof the two-grid correction scheme is essential for a complete explanation of the basicmultigrid methods.

We begin with a detailed look at the intergrid transfer operators. Considerfirst the full weighting (restriction) operator, I2h

h . Recall that I2hh maps Rn−1 →

Rn2 −1. It has rank n

2 − 1 and null space N(I2hh ) of dimension n

2 . It is importantto understand what we call the spectral properties of I2h

h . In particular, how doesI2hh act upon the modes of the original operator Ah?

Recall that the modes of Ah for the one-dimensional model problem are givenby

whk,j = sin

(jkπ

n

), 1 ≤ k ≤ n − 1, 0 ≤ j ≤ n.

80 Chapter 5

Figure 5.3: A pair of complementary modes on a grid with n = 8 points. The solidline shows the k = 2 mode. The dashed line shows the k′ = n − k = 6 mode.

The full weighting operator may be applied directly to these vectors. The result ofI2hh acting on the smooth modes is (Exercise 4)

I2hh wh

k = cos2(

kπ

2n

)w2h

k , 1 ≤ k ≤ n

2.

This says that I2hh acting on the kth (smooth) mode of Ah produces a constant

times the kth mode of A2h when 1 ≤ k ≤ n2 . This property is illustrated in Fig.

5.2(a). For the oscillatory modes, with n2 < k < n − 1, we have (Exercise 5)

I2hh wh

k′ = − sin2

(kπ

2n

)w2h

k , 1 ≤ k <n

2,

where k′ = n−k. This says that I2hh acting on the (n−k)th mode of Ah produces a

constant multiple of the kth mode of A2h. This property, illustrated in Fig. 5.2(b),is similar to the aliasing phenomenon discussed earlier. The oscillatory modes on Ωh

cannot be represented on Ω2h. As a result, the full weighting operator transformsthese modes into relatively smooth modes on Ω2h.

In summary, we see that both the kth and (n − k)th modes on Ωh become thekth mode on Ω2h under the action of full weighting. We refer to this pair of fine-gridmodes wh

k ,whn−k as complementary modes. Letting Wh

k = spanwhk ,wh

n−k, wehave that

I2hh : Wh

k → spanw2hk .

In passing, it is interesting to note the relationship between two complementarymodes. It may be shown (Exercise 6) that wh

n−k,j = (−1)j+1whk,j . Figure 5.3

illustrates the smooth and oscillatory nature of a pair of complementary modes.As noted earlier, the full weighting operator has a nontrivial null space, N(I2h

h ).It may be verified (Exercise 7) that this subspace is spanned by the vectors nj =Aheh

j , where j is odd and ehj is the jth unit vector on Ωh. As shown in Fig. 5.4,

the basis vectors nj appear oscillatory. However, they do not coincide with theoscillatory modes of Ah. In fact, an expansion of nj in terms of the modes of Ah

requires all of the modes. Thus, the null space of I2hh possesses both smooth and

oscillatory modes of Ah.Having established the necessary properties of the full weighting operator, we

now examine the interpolation operator Ih2h in the same way. Recall that Ih

2h maps

A Multigrid Tutorial 81

Figure 5.4: A typical basis vector of the null space of the full weighting operatorN(I2h

h ).

Figure 5.5: A typical basis vector of the range of interpolation R(Ih2h).

Rn2 −1 → Rn−1 and has full rank. In order to establish the spectral properties of

Ih2h, we ask how Ih

2h acts on the modes of A2h. Letting

w2hk,j = sin

(jkπ

n/2

), 1 ≤ k <

n

2, 0 ≤ j ≤ n

2,

be the Ω2h modes, we can show (Exercise 8) that Ih2h does not preserve these modes.

The calculation shows that

Ih2hw

2hk = ckwh

k − skwhk′ , 1 ≤ k <

n

2, k′ = n − k,

where ck = cos2(

kπ2n

)and sk = sin2

(kπ2n

). We see that Ih

2h acting on the kth modeof Ω2h produces not only the kth mode of Ωh but also the complementary modewh

k′ . This fact exposes the interesting property that interpolation of smooth modeson Ω2h excites (to some degree) oscillatory modes on Ωh. It should be noted thatfor a very smooth mode on Ωh with k n/2,

Ih2hw

2hk =

[1 − O

(k2

n2

)]wh

k + O

(k2

n2

)wh

k′ .

In this case, the result of interpolation is largely the corresponding smooth modeon Ωh with very little contamination from the complementary oscillatory mode.As it has been defined, Ih

2h is a second-order interpolation operator because themagnitude of the spurious oscillatory mode is O( k2

n2 ).We have already anticipated the importance of the range of interpolation, R(Ih

2h).A basis for R(Ih

2h) is given by the columns of Ih2h. While these basis vectors appear

smooth, as Fig. 5.5 shows, they do not coincide with the smooth modes of Ah. Infact, it may be shown that any one of these basis vectors requires all modes of Ah

for a full representation. In other words, the range of interpolation contains bothsmooth and oscillatory modes of Ah.

82 Chapter 5

With this investigation of the intergrid transfer operators, we now return to thetwo-grid correction scheme. We begin with an observation made in Chapter 2 thata stationary linear iteration may be expressed in the form

v(1) = (I − BA)v(0) + Bf = Rv(0) + Bf ,

where B is a specified matrix and R = I − BA is the iteration matrix for themethod. It follows that m sweeps of the iteration can be represented by

v(m) = Rmv(0) + C(f),

where C(f) represents a series of operations on f .We can now turn to the two-grid correction scheme. The steps of this scheme,

with an exact solution on the coarse grid, are given by the following procedure:

• Relax ν times on Ωh with scheme R: vh ← Rνvh + C(f).

• Full weight rh to Ω2h: f2h ← I2hh (fh − Ahvh).

• Solve the residual equation exactly: v2h = (A2h)−1f2h.

• Correct the approximation on Ωh: vh ← vh + Ih2hv

2h.

If we now take this process one step at a time, it may be represented in termsof a single replacement operation:

vh ← Rνvh + C(f) + Ih2h(A2h)−1I2h

h (fh − Ah(Rνvh + C(f))).

The exact solution uh is unchanged by the two-grid correction scheme. Therefore,

uh = Rνuh + C(f) + Ih2h(A2h)−1I2h

h (fh − Ah(Rνuh + C(f))).

By subtracting these last two expressions, we can see how the two-grid correctionoperator, which we now denote TG, acts upon the error, eh = uh − vh. We findthat

eh ←[I − Ih

2h(A2h)−1I2hh Ah

]Rνeh ≡ TGeh. (5.7)

As in Chapter 2, we imagine that the error can be expressed as a linear combi-nation of the modes of Ah. This leads us to ask how TG acts upon the modes ofAh. However, TG consists of R, Ah, (A2h)−1, I2h

h , and Ih2h, and we now know how

each of these operators acts upon the modes of Ah. For the moment, consider thetwo-grid correction scheme TG with no relaxation (ν = 0). Using all of the spectralproperties we have just discovered, it may be shown (Exercise 9) that the coarse-grid correction operator, TG, is invariant on the subspaces Wh

k = spanwhk ,wh

k′;that is,

TGwk = skwk + skwk′ , (5.8)

TGwk′ = ckwk + ckwk′ , 1 ≤ k ≤ n

2, k′ = n − k, (5.9)

where ck = cos2(

πk2n

)and sk = sin2

(πk2n

).

This implies that when TG is applied to a smooth or oscillatory mode, the samemode and its complement result. But it is important to look at the amplitudes of

A Multigrid Tutorial 83

Figure 5.6: Damping factor for the two-grid correction operator TG, without relax-ation, acting on the oscillatory modes with wavenumbers n − k (upper curve) andon the smooth modes with wavenumbers k (lower curve) for 1 ≤ k ≤ n

2 .

the resulting modes. Consider the case of TG acting on very smooth modes andvery oscillatory modes with k n. Then (5.8) and (5.9) become

TGwk = O

(k2

n2

)wk + O

(k2

n2

)wk′ ,

TGwk′ =[1 − O

(k2

n2

)]wk +

[1 − O

(k2

n2

)]wk′ , 1 ≤ k ≤ n

2, k′ = n − k.

TG acting on smooth modes produces smooth and oscillatory modes with verysmall amplitudes. Therefore, the two-grid correction scheme is effective at elim-inating smooth components of the error. However, when TG acts upon highlyoscillatory modes, it produces smooth and oscillatory modes with O(1) ampli-tudes. Therefore, two-grid correction, without relaxation, does not damp oscil-latory modes. Figure 5.6 illustrates this behavior of the two-grid correction schemewith no relaxation by showing the damping factors, ck and sk, for the smooth andoscillatory components of the error.

We now bring relaxation into the picture. Knowing its spectral properties,we can anticipate that relaxation will balance perfectly the action of TG withoutrelaxation. We now include ν steps of a relaxation method R and assume forsimplicity that R does not mix the modes of Ah. Many other relaxation methodscan be analyzed without this assumption. As before, let λk be the eigenvalue of Rassociated with the kth mode wk. Combining all of these observations with (5.7),the action of TG with relaxation is given by (Exercise 10)

TGwk = λνkskwk + λν

kskwk′ , (5.10)

TGwk′ = λνk′ckwk + λν

k′ckwk′ , 1 ≤ k ≤ n

2, k′ = n − k. (5.11)

We know that the smoothing property of relaxation has the strongest effect onthe oscillatory modes. This is reflected in the term λν

k′ , which is small. At thesame time, the two-grid correction scheme alone (without relaxation) eliminates

84 Chapter 5

the smooth modes. This is reflected in the sk terms. Thus, all terms of (5.10) and(5.11) are small, particularly for k n

2 or as ν becomes large. The result is acomplete process in that both smooth and oscillatory modes of the error are welldamped.

We have now completed what we call the spectral picture of multigrid. Byexamining how various operators act on the modes of Ah, we have determinedthe effect of the entire two-grid correction operator on those modes. This analy-sis explains how the two-grid correction process eliminates both the smooth andoscillatory components of the error.

Solvability and the Fundamental Theorem of Linear Algebra. Supposewe have a matrix A ∈ Rm×n. The fundamental theorem of linear algebra statesthat the range (column space) of the matrix, R(A), is equal to the orthogonalcomplement of N (AT ), the null space of AT . Thus, spaces Rm and Rn can beorthogonally decomposed as follows:

Rm = R(A) ⊕N (AT ),Rn = R(AT ) ⊕N (A).

For the equation Ax = f to have a solution, it is necessary that the vector flie in R(A). Thus, an equivalent condition is that f be orthogonal to everyvector in N (AT ). For the equation Ax = f to have a unique solution, it isnecessary that N (A) = 0. Otherwise, if x is a solution and y ∈ N (A), thenA(x + y) = Ax + Ay = f + 0 = f , so the solution x is not unique.

There is another vantage point from which to view the coarse-grid correctionscheme. This perspective will lead to what we will call the algebraic picture ofmultigrid. With both the spectral and the algebraic picture before us, it will bepossible to give a good qualitative explanation of multigrid. Let us now look at thealgebraic structure of the two-grid correction scheme.

The variational properties introduced earlier now become important. Recallthat these properties are given by

A2h = I2hh AhIh

2h (Galerkin property),I2hh = c(Ih

2h)T , c ∈ R.

The two-grid correction scheme involves transformations between the space of fine-grid vectors, Ωh, and the space of coarse-grid vectors, Ω2h. Figure 5.7 diagramsthese two spaces and the action of the full weighting and interpolation operators.

As we have already seen, the range of interpolation, R(Ih2h), and the null space

of full weighting, N(I2hh ), both reside in Ωh and have dimensions of roughly n

2 .From the orthogonality relationships between the subspaces of a linear operator(Fundamental Theorem of Linear Algebra), we know that

N(I2hh ) ⊥ R[(I2h

h )T ].

By the second variational property, it then follows that

N(I2hh ) ⊥ R(Ih

2h).

A Multigrid Tutorial 85

Figure 5.7: Schematic drawing of the space of fine-grid vectors, Ωh; the space ofcoarse-grid vectors, Ω2h; and their subspaces and the intergrid transfer operators,Ih2h and I2h

h .

The fact that the range of interpolation is orthogonal to the null space of fullweighting is significant.

We will now use the notion of A-orthogonality to rewrite the above orthogonalityrelationship. The fact that N(I2h

h ) ⊥ R(Ih2h) means that (qh,uh) = 0 whenever

qh ∈ R(Ih2h) and I2h

h uh = 0. This is equivalent to the condition that (qh, Ahuh) = 0whenever qh ∈ R(Ih

2h) and I2hh Ahuh = 0. This last condition may be written as

N(I2hh Ah) ⊥Ah R(Ih

2h);

that is, the null space of I2hh Ah is Ah-orthogonal to the range of interpolation.

This orthogonality property allows the space Ωh to be decomposed in the form

Ωh = R(Ih2h) ⊕ N(I2h

h Ah).

This means that if eh is a vector in Ωh, then it may always be expressed as

eh = sh + th,

where sh ∈ R(Ih2h) and th ∈ N(I2h

h Ah).It will be helpful to interpret the vectors sh and th. Since sh is an element of

R(Ih2h), it must satisfy sh = Ih

2hq2h, where q2h is some vector of Ω2h. We observed

the smoothing effect of interpolation and noted the smooth appearance of the basisvectors of R(Ih

2h). For this reason, we associate sh with the smooth components ofthe error. We also noted the oscillatory appearance of the basis vectors of N(I2h

h ).For this reason, we associate th with the oscillatory components of the error.

We may now consider the two-grid correction operator in light of these subspaceproperties. The two-grid correction operator without relaxation is

TG = I − Ih2h(A2h)−1I2h

h Ah.

86 Chapter 5

First note that if sh ∈ R(Ih2h), then sh = Ih

2hq2h for some vector q2h in Ω2h. We

then have thatTGsh =

[I − Ih

2h(A2h)−1I2hh Ah

]Ih2hq

2h. (5.12)

However, by the first of the variational properties, I2hh AhIh

2h = A2h. Therefore,TGsh = 0. This gives us the important result that any vector in the range ofinterpolation also lies in the null space of the two-grid correction operator, that is,

N(TG) ⊃ R(Ih2h).

Having seen how TG acts on R(Ih2h), now consider a vector th in N(I2h

h Ah).This case is even simpler. We have

TGth =[I − Ih

2h(A2h)−1I2hh Ah

]th. (5.13)

Because I2hh Ahth = 0, we conclude that TGth = th. This says that TG is the

identity when it acts on N(I2hh Ah). This implies that the dimension of N(TG)

cannot exceed the dimension of R(Ih2h). We therefore have the stronger result

(Exercise 11) thatN(TG) = R(Ih

2h).

This argument gives us the effect of the two-grid correction operator on the twoorthogonal subspaces of Ωh. Let us now put these algebraic results together withthe spectral picture. We have established two independent ways to decompose thespace of fine-grid vectors, Ωh. We have the spectral decomposition

Ωh = L ⊕ H =

Low-frequency modes

1 ≤ k <n

2

⊕

High-frequency modesn

2≤ k < n

and the subspace decomposition

Ωh = R(Ih2h) ⊕ N(I2h

h Ah).

We can now give a schematic illustration of the two-grid correction scheme asit works on an arbitrary error vector. The diagrams in Figs. 5.8–5.11 are ratherunconventional and may require some deliberation. However, they do incorporateall of the spectral and subspace results elaborated on in this chapter.

We first focus on the upper diagrams of each figure. These diagrams portray thespace Ωh as the plane of the page. As we have seen, Ωh may be decomposed in twoways. These two decompositions are represented by two pairs of orthogonal axeslabeled (H, L) for the high-frequency/low-frequency decomposition and (N, R) forthe N(I2h

h Ah)/R(Ih2h) decomposition. The L and R axes are more nearly aligned,

suggesting that the smooth, low-frequency modes are associated with the range ofinterpolation.

Initially, an arbitrary error vector eh in Ωh appears as a point in the planeas shown in Fig. 5.8. This vector has projections on all four axes. Specifically,consider the projections on the R and N axes, which we have called sh and th,respectively. Both sh and th may be further projected onto the L and H axes. Theprojections of sh on the L and H axes are denoted sL and sH , respectively. Theprojections of th on the L and H axes are denoted tL and tH , respectively. Thisgives the picture for the initial error, before any relaxation is done.

A Multigrid Tutorial 87

Figure 5.8: The top diagram shows the space Ωh decomposed along the (low-frequency, high-frequency) axes and along the (R(Ih

2h), N(I2hh Ah)) axes. The initial

error, eh, appears as a point with projections on all four axes. The lower diagramgives a schematic “energy budget” of the initial error. The error energy is dividedbetween R(Ih

2h) and N(I2hh Ah) and further divided between low- and high-frequency

modes.

The upper diagram in Fig. 5.9 shows the effect of several relaxation sweeps onthe fine grid. Assume that enough relaxation sweeps are done to eliminate entirelythe high-frequency components of the error. In the diagram, the resulting error hasno component along the H-axis and the point representing eh is projected downto the L-axis. The diagram also shows the new projections on the R and N axes.Notice that the component of eh in R(Ih

2h) has actually increased.The upper diagram of Fig. 5.10 shows the effect of the rest of the two-grid

correction scheme. Since R(Ih2h) is the null space of TG, the component of eh along

the R-axis vanishes. Therefore, this step is represented by a projection directly ontothe N -axis. This is consistent with our observation that TG acts as the identityon N(I2h

h Ah). This new projection has eliminated a large amount of the initialerror. However, as we proved earlier, the coarse-grid correction operator (without

88 Chapter 5

Figure 5.9: The top diagram shows the effect of several fine-grid relaxation sweeps.Assuming that all oscillatory components are eliminated, the error is projected ontothe low-frequency axis (L). The lower diagram shows the corresponding changes inthe energy budget. Dashed lines indicate the previous energy budget. Shaded areasshow the current budget.

relaxation) does excite oscillatory modes, which can be seen in the diagram as anonzero component in the H direction.

The upper diagram of Fig. 5.11 shows the error after more relaxation sweeps onthe fine grid. Assuming again that relaxation eliminates all high-frequency errormodes, this relaxation sweep is represented as a projection directly onto the L-axis.This further reduces a significant amount of the error.

The pattern should now be evident. By combining relaxation with correctionsfrom the coarse-grid residual equation, we alternately project onto the L and Naxes. In doing this, the error vector is driven toward the origin (zero error) in away that is reminiscent of the convergence of a fixed point iteration.

The lower diagrams of Figs. 5.8–5.11 attempt to illustrate the two-grid correctionscheme in terms of the “energy budget” of the error. Initially, the error has acertain energy or magnitude. Part of this energy resides in R(Ih

2h), which is drawn

A Multigrid Tutorial 89

Figure 5.10: The top diagram shows the error after doing the remainder of thetwo-grid correction. All components in R(Ih

2h) are eliminated as indicated by theprojection onto the N -axis. The lower diagram shows the corresponding changes inthe energy budget.

on the left of the vertical line; the remainder of the energy resides in N(I2hh Ah),

which is drawn on the right of that line. The energy in R(Ih2h) may be divided

further between the low-frequency modes (above the horizontal line) and the high-frequency modes (below the horizontal line). We see that the initial error is fairlyevenly divided between R(Ih

2h) and N(I2hh Ah). As we noted before, most of the

error in R(Ih2h) is associated with the smooth, low-frequency modes; most of the

error in N(I2hh Ah) is associated with the oscillatory, high-frequency modes.

The lower diagram of Fig. 5.9 shows the energy budget after several effectiverelaxation sweeps. The dotted lines indicate the previous budget status. Theshaded regions indicate the budget after relaxation. As described above, relaxationeliminates the high-frequency error components, but at the same time increases thecomponent of the error in R(Ih

2h).

90 Chapter 5

Figure 5.11: The top diagram shows the effect of several more fine-grid relaxationson the error. Once again, all high-frequency components are removed as indicatedby a projection onto the L-axis. The lower diagram shows the corresponding changesin the energy budget.

The lower diagram of Fig. 5.10 shows the energy budget after a correctionstep. As we saw earlier, this step eliminates all of the error in R(Ih

2h). Finally,further relaxation (Fig. 5.11) reduces the remaining oscillatory error modes, but re-introduces error with a small magnitude into R(Ih

2h). A continuation of this cyclewill further reduce the energy of the error; this reduction signifies convergence.

The subspace iteration diagrams, together with the energy budget diagrams,illustrate the way in which relaxation and correction work together. These twoprocesses complement each other remarkably. When they are applied in tandem,the result is an extremely effective algorithm.

It should be remembered that this entire discussion has dealt only with a two-grid scheme. The V-cycle uses this scheme at all levels, ensuring that relaxation is

A Multigrid Tutorial 91

always directed at the oscillatory modes of the current grid. As we have seen, two-grid correction without relaxation takes care of the smooth error components onthe current fine grid. Therefore, by adding relaxation at all levels, all componentsof the error are eventually acted upon and quickly removed.

The efficiency of the two-grid scheme is further amplified by the FMG method.This scheme uses nested V-cycles to compute accurate initial guesses on coarsergrids before relaxing on the finer grids. This ensures that a particular coarse-gridproblem is solved to the level of discretization error before the more expensivefine-grid relaxations are begun. But even in these more elaborate procedures, it isthe combination of relaxation and correction that provides the underlying power.Therefore, the arguments of this chapter, although somewhat qualitative, hold thekey to the remarkable effectiveness of multigrid.

Exercises

1. Energy norm. Assume A is symmetric positive definite. As defined in thischapter, the A-energy inner product and the A-energy norm are given by

(u,v)A = (Au,v) and ‖u‖2A = (u,u)A.

(a) Show that these are acceptable definitions for an inner product and anorm.

(b) Show that ‖r‖2 = ‖e‖A2 .

(c) The error norm ‖e‖2 is generally not computable. Is ‖e‖A computable?Is ‖e‖A2 computable?

2. FMG error analysis. A key step in the FMG error analysis is showing that

‖Ih2hu

2h − Ih2hv

2h‖Ah = ‖u2h − v2h‖A2h .

Use the Galerkin property and the property of inner products that

(Bu,v) = (u, BT v)

to prove this equality for any two coarse-grid vectors.

3. Discretization error. The goal of this exercise is to consider the modelproblem in one dimension and show carefully that the discrete L2 norm ofthe discretization error is bounded by a constant times h2 (see equation (5.2)).Consider the boundary value problem

−u′′(x) = f(x) on (0, 1),u(0) = u(1) = 0.

The discretized form of the problem is

(Ahuh)i = −uhi+1 − 2uh

i + uhi−1

h2= f(xi), 1 ≤ i ≤ n,

uh0 = uh

n+1 = 0.

92 Chapter 5

(a) Let u be the vector consisting of values of the exact solution u sampledat the grid points; that is, ui = u(xi). Expanding u(xi±1) in a Taylorseries about xi, show that

(Ahu)i = −u′′(xi) −h2

24(u(iv)(ξ+) + u(iv)(ξ−)),

where xi−1 < ξ− < xi and xi < ξ+ < xi+1.

(b) Assuming continuity of u(iv) and using the Intermediate Value Theorem,show that the truncation error is given by

τhi ≡ f(xi) − (Ahu)i =

h2

24(u(iv)(ξ+) + u(iv)(ξ−)) =

h2

12f ′′(ξi),

where xi−1 < ξi < xi+1.

(c) Recall that if A is symmetric, then ‖A‖2 = ρ(A), and that if Aw = λw,then A−1w = λ−1w. Use the eigenvalues of Ah and the definition ofmatrix norms to show that ‖(Ah)−1‖h ≤ γ, where γ ≈ π−2. Note thatthis bound is independent of h.

(d) Show that if f ′′ is bounded on [0, 1] and vi = f ′′(ξi), then ‖v‖h isbounded. Is this result true for the (unscaled) Euclidean norm?

(e) Combine the above facts and use properties of matrix norms to concludethat the discrete L2 norm of the discretization error is bounded by aconstant times h2; begin by making the observation that

‖Eh‖h = ‖uh − u‖h = ‖(Ah)−1Ah(uh − u)‖h.

4. Effect of full weighting. Verify that I2hh wh

k = cos2(

kπ2n

)w2h

k , where whk,j =

sin(

jkπn

), 1 ≤ k < n

2 , and I2hh is the full weighting operator.

5. Effect of full weighting. Verify that I2hh wh

k′ = − sin2(

kπ2n

)w2h

k , wherek′ = n− k, 1 ≤ k < n

2 , and I2hh is the full weighting operator. What happens

to whn/2 under full weighting?

6. Complementary modes. Show that the complementary modes whk ,wh

k′on Ωh are related by wh

k′,j = (−1)j+1whk,j , where k′ = n − k.

7. Null space of full weighting. Verify that the vectors nj = Ahehj , where j

is odd and ehj is the jth unit vector on Ωh, form a basis for the null space of

I2hh .

8. Effect of linear interpolation. Show that Ih2hw

2hk = ckwh

k − skwhk′ , where

1 ≤ k < n2 , k′ = n − k, ck = cos2

(kπ2n

), sk = sin2

(kπ2n

), and Ih

2h is the linearinterpolation operator.

9. Two-grid correction scheme. Using the previous results concerning theeffects of Ah, I2h

h , and Ih2h on the fundamental modes, determine the ef-

fect of the coarse-grid correction operator without relaxation, TG = I −Ih2h(A2h)−1I2h

h Ah, on the modes wk and wk′ , where 1 ≤ k ≤ n2 , k′ = n − k.

(Recall that if Aw = λw, then A−1w = λ−1w.)

A Multigrid Tutorial 93

10. Effect of two-grid correction. Let R be an iteration matrix with the sameeigenvectors, wk, as Ah and with eigenvalues λk. Consider the coarse-grid cor-rection operator with ν sweeps of relaxation, TG = [I−Ih

2h(A2h)−1I2hh Ah]Rν .

(a) Determine how TG acts on the modes wk and wk′ , where 1 ≤ k ≤ n2 ,

k′ = n − k.

(b) Find the specific convergence factors for the coarse-grid correction oper-ator that uses one sweep of damped Jacobi with ω = 2

3 .

11. An important equivalence. Prove that the null space of TG is the rangeof interpolation, where TG is the coarse grid correction operator withoutrelaxation. Equation (5.12) of the text can be used to show that N(TG) ⊃R(Ih

2h). Use equation (5.13) and the dimensions of the subspaces to showthat N(TG) = R(Ih

2h).

12. Two-dimensional five-point stencil. When the two-dimensional modelproblem is discretized on a uniform grid with hx = hy = h, the coefficients ateach grid point are given by the five-point stencil

Ah =1h2

−1

−1 4 −1−1

.

What does the stencil for A2h = I2hh AhIh

2h look like if Ih2h is based on bilinear

interpolation and I2hh is based on (a) full weighting? (b) injection?

13. Two-dimensional nine-point stencil. Repeat the previous problem withthe nine-point stencil

Ah =1

3h2

−1 −1 −1

−1 8 −1−1 −1 −1

.

Chapter 6

Nonlinear Problems

The preceding chapters presented multigrid methods applied to linear problems.Because of their iterative nature, multigrid ideas should be effective on nonlinearproblems, where some form of iteration is usually imperative. Just as multigridmethods were developed to improve classical linear relaxation methods, we will seethat they can also guide us in the use of nonlinear methods.

We begin by clarifying the most significant formal difference between linear andnonlinear systems of equations. Consider a system of nonlinear algebraic equations,A(u) = f , where u, f ∈ Rn. (The notation A(u), rather than Au, signifies that theoperator is nonlinear.) Suppose that v is an approximation to the exact solutionu. It is possible to define the error and the residual for this problem, much as wedid earlier: the error is simply e = u − v and the residual is r = f − A(v). If wenow subtract the original equation, A(u) = f , from the definition of the residual,we find that

A(u) − A(v) = r. (6.1)

We are one step from the residual equation that we derived for linear systems inChapter 2. However, because A is nonlinear, even though u − v = e, we cannotconclude that A(u) − A(v) = A(e). This means that we no longer have a simplelinear residual equation, which necessitates changes in the methods that we devisefor nonlinear problems. We must now use (6.1) as the residual equation.

It makes sense to review quickly a classical relaxation method for nonlinearsystems; it will have a part to play in multigrid methods. Working with thesystem of nonlinear algebraic equations A(u) = f , one of the most frequentlyused methods is nonlinear Gauss–Seidel relaxation [16]. Suppose the jth equa-tion of the system can be solved for the jth variable in terms of the other variables(u1, u2, . . . , uj−1, uj+1, . . . , un). We write the result in the abstract form

uj = Mj(u1, u2, . . . , uj−1, uj+1, . . . , un), 1 ≤ j ≤ n.

Just as with linear systems, Gauss–Seidel relaxation takes the form

vj ← Mj(v1, v2, . . . , vj−1, vj+1, . . . , vn), 1 ≤ j ≤ n,

where the current value of each component is used on the right side. For cases wherethis iteration cannot be formed explicitly, we use the same characterization thatwas used for linear Gauss–Seidel: for each j = 1, 2, . . . , n, set the jth component

95

96 Chapter 6

of the residual to zero and solve for vj . Because the residual is r = f − A(v), thischaracterization amounts to solving (A(v))j = fj for vj . Equivalently, as in thelinear case, we can let εj be the jth unit vector, so that the jth step of nonlinearGauss–Seidel amounts to finding an s ∈ R such that

(A(v + sεj))j = fj , 1 ≤ j ≤ n.

This is generally a nonlinear scalar equation in the scalar s, which can be solvedefficiently with one or two steps of (scalar) Newton’s method [16]. When an ap-proximate solution s is found, the jth component is updated by v ← v + sεj .Updating all n components sequentially in this way constitutes one iteration sweepof nonlinear Gauss–Seidel.

Now, to see how residual equation (6.1) can be used as a basis for multigridmethods, let v be the current approximation and replace the exact solution u byv + e. Then residual equation (6.1) becomes

A(v + e) − A(v) = r. (6.2)

Expanding A(v + e) in a Taylor series (in n variables) about v and truncating theseries after two terms, we have the linear system of equations

J(v)e = r, (6.3)

where J = (∂Ai/∂uj) is the n× n Jacobian matrix. Linear system (6.3) representsan approximation to nonlinear system (6.1). It can be solved for e and the currentapproximation v can be updated by v ← v + e. Iteration of this step is Newton’smethod. But how should linear system (6.3) be solved at each step? One highlyrecommended option is to use multigrid. Such a combination of Newton’s methodfor the outer iteration and multigrid for the (linear) inner iteration is called Newton-multigrid.

Newton-multigrid can be effective, but it does not use multigrid ideas to treatthe nonlinearity directly. To do this, we return to the residual equation,

A(v + e) − A(v) = r,

and use it in the familiar two-grid setting. Suppose we have found an approxima-tion, vh, to the original fine-grid problem

Ah(uh) = fh . (6.4)

Proceeding as we did for the linear problem, we now want to use the residualequation on the coarse grid Ω2h to approximate eh, the error in vh. Using theabove argument, the residual equation on the coarse grid appears as

A2h(v2h + e2h) − A2h(v2h) = r2h, (6.5)

where A2h denotes the coarse-grid operator, r2h is the coarse-grid residual, v2h is acoarse-grid approximation to vh, and e2h is a coarse-grid approximation to eh. Oncee2h is computed, the fine-grid approximation can be updated by vh ← vh + Ih

2he2h.We have already encountered the coarse-grid residual r2h. Nothing changes

here; we simply choose it to be the restriction of the fine-grid residual to the coarsegrid:

r2h = I2hh rh = I2h

h (fh − Ah(vh)).

A Multigrid Tutorial 97

Newton’s Method. Perhaps the best known and most important methodfor solving nonlinear equations is Newton’s method, which can be derived asfollows. Suppose we wish to solve the scalar equation F (x) = 0. We expand Fin a Taylor series about an initial guess x:

F (x + s) = F (x) + sF ′(x) +s2

2F ′′(ξ),

where ξ is between x and x + s. If x + s is the solution, then (neglectingthe higher-order terms) the series becomes 0 = F (x) + sF ′(x), from which s =−F (x)/F ′(x). Thus, we can update the initial guess using x ← x−F (x)/F ′(x).Newton’s method results by iterating this process:

x ← x − F (x)F ′(x)

.

Newton’s method for a system of n nonlinear equations is a straightforwardextension of the scalar Newton’s method. We write the system in vector formas

F(x) ≡

f1(x1, x2, . . . , xn)f2(x1, x2, . . . , xn)

...fn(x1, x2, . . . , xn)

=

00...0

.

Expanding in a Taylor series yields

F(x + s) = F(x) + J(x)s + higher-order terms,

where J(x) is the Jacobian matrix

J(x) =

∂f1∂x1

∂f1∂x2

· · · ∂f1∂xn

∂f2∂x1

∂f2∂x2

· · · ∂f2∂xn

......

. . ....

∂fn

∂x1

∂fn

∂x2· · · ∂fn

∂xn

.

Newton’s method results when we replace F(x + s) = 0 by F(x) + J(x)s = 0.Solving for the vector s yields the iteration

x ← x − [J(x)]−1 F(x).

But what about the current approximation v2h? When we move to the coarse grid,it makes sense to restrict the current fine-grid approximation to the coarse grid,often using the same transfer operator used for the residual: v2h = I2h

h vh.Making these substitutions in the residual equation yields

A2h(I2hh vh + e2h︸ ︷︷ ︸

u2h

) = A2h(I2hh vh) + I2h

h (fh − Ah(vh)). (6.6)

The right side of this nonlinear system is known. The goal is to find or approximatethe solution to this system, which we have denoted u2h. The coarse-grid error

98 Chapter 6

approximation, e2h = u2h−I2hh vh, can then be interpolated up to the fine grid and

used to correct the fine-grid approximation vh. This correction step takes the form

vh ← vh + Ih2he

2h or vh ← vh + Ih2h(u2h − I2h

h vh).

The scheme we have just outlined is the most commonly used nonlinear versionof multigrid. It is called the full approximation scheme (FAS) because the coarse-grid problem is solved for the full approximation u2h = I2h

h vh + e2h rather thanthe error e2h. A two-grid version of this scheme is described as follows:

Full Approximation Scheme (FAS)

• Restrict the current approximation and its fine-grid residual to the coarsegrid: r2h = I2h

h (fh − Ah(vh)) and v2h = I2hh vh.

• Solve the coarse-grid problem A2h(u2h) = A2h(v2h) + r2h.

• Compute the coarse-grid approximation to the error: e2h = u2h − v2h.

• Interpolate the error approximation up to the fine grid and correct the currentfine-grid approximation: vh ← vh + Ih

2he2h.

There are several observations to be made about this method. It is worthnoting that if A is a linear operator, then the FAS scheme reduces directly to the(linear) two-grid correction scheme (Exercise 1). Thus, FAS can be viewed as ageneralization of the two-grid correction scheme to nonlinear problems.

Less obvious is the fact that an exact solution of the fine-grid problem is afixed point of the FAS iteration (Exercise 2). This fixed point property, which is adesirable attribute of most iterative methods, means that the process stalls at theexact solution.

A third observation is that the FAS coarse-grid equation can be written as

A2h(u2h) = f2h + τ 2hh ,

where the tau correction τ 2hh is defined by

τ 2hh = A2h(I2h

h vh) − I2hh Ah(vh) .

One of the many consequences of this relationship is that the solution of the coarse-grid FAS equation, u2h, is not the same as the solution of the original coarse-gridequation A2h(u2h) = f2h because τ 2h

h = 0 generally. In fact, as FAS processingadvances, u2h begins to achieve accuracy that compares to that of the solution onthe finest grid, albeit at the resolution of grid 2h. This tau correction relationshipallows us to view FAS as a way to alter the coarse-grid equations so that theirapproximation properties are substantially enhanced.

Here is another important observation. Because the second step in the aboveprocedure involves a nonlinear problem itself, FAS involves an inner and an outeriteration; the outer iteration is the FAS correction scheme, while the inner iter-ation is usually a standard relaxation method such as nonlinear Gauss–Seidel. Atrue multilevel FAS process would be done recursively by approximating solutions to

A Multigrid Tutorial 99

the Ω2h problem using the next coarsest grid, Ω4h. Thus, FAS, like its linearcounterparts, is usually implemented as a V-cycle or W-cycle scheme.

In earlier chapters, we saw the importance of full multigrid (FMG) for obtaininga good initial guess for the fine-grid problem. The convergence of nonlinear itera-tions depends even more critically on a good initial guess. Typically, the better theinitial guess used on the fine grid, the more linear the fine-grid problem appears,and the more effective the fine-grid solver will be. When FMG is used for nonlinearproblems, the interpolant Ih

2hu2h is generally accurate enough to be in the basin of

attraction of the fine-grid solver. Thus, whether we use Newton-multigrid or FASV-cycles on each new level, we can expect one FMG cycle to provide accuracy tothe level of discretization, unless perhaps the nonlinearity is exceptionally strong.These options are investigated in a numerical example later in the chapter.

Example: An algebraic problem. We need to warn the reader that this exampleshould not be taken too seriously: it was devised only as a way to illustrate themechanics of the FAS scheme. The problem does not really need the power ofmultigrid because it can be effectively treated by a good classical relaxation scheme.More importantly, as with most nonlinear problems, the example has subtleties thatmust be addressed before solution techniques should even be considered (Exercise4).

Consider the following nonlinear algebraic system of n equations and n un-knowns uh ∈ Rn:

Ahj (uh) ≡ uh

j uhj+1 = fh

j , 1 ≤ j ≤ n , (6.7)

where the vector fh ∈ Rn is given. (The notation Ahj (uh) means the jth component

of Ah(uh).) We impose the periodic boundary condition uhn+1 = uh

1 to close theproblem. Suppose we have a fine-grid approximation, vh, obtained by relaxation.In passing, note that Gauss–Seidel relaxation is easy to formulate for this problem:it consists of the replacement steps

vhj ←

fhj

vhj+1

, 1 ≤ j ≤ n. (6.8)

To see what the FAS correction looks like, assume that n is a positive even integerand write coarse-grid residual equation (6.5) as

A2hj (v2h + e2h) − A2h

j (v2h) = r2hj , 1 ≤ j ≤ n

2.

In component form, these equations are

(v2hj + e2h

j )(v2hj+1 + e2h

j+1) − v2hj v2h

j+1 = r2hj or

v2hj e2h

j+1 + v2hj+1e

2hj + e2h

j e2hj+1 = r2h

j , 1 ≤ j ≤ n

2.

Notice how the term v2hj v2h

j+1 cancels. Now, e2hj and e2h

j+1 are the unknowns in thejth equation; the coefficients v2h

j , v2hj+1, and r2h

j must come from the fine grid. Wecan obtain the terms v2h

j and v2hj+1 from the fine grid by injection:

v2hj = I2h

h vh2j = vh

2j , v2hj+1 = I2h

h vh2j+2 = vh

2j+2.

100 Chapter 6

Similarly, the coarse-grid residual (using injection) is given by

r2hj = I2h

h rh2j = fh

2j − vh2jv

h2j+1.

The equations that must be solved on the coarse grid for e2hj are therefore given by

vh2je

2hj+1 + vh

2j+2e2hj + e2h

j e2hj+1 = fh

2j − vh2jv

h2j+1, 1 ≤ j ≤ n

2.

This system corresponds to the general FAS equation (6.6). In this example, with aspecific form for the operator Ah, we have simplified the right side of (6.6), resultingin an explicit system for e2h

j . This system would typically be handled by a standardrelaxation method. Note that Gauss–Seidel is again fairly simple because the jthequation is linear in e2h

j , although the system of equations is nonlinear in the vectoreh. In any case, after approximations to e2h

j have been computed, they can thenbe interpolated up to the fine grid to correct vh.

Example: Formulating FAS for a boundary value problem. Consider thetwo-point boundary value problem

−u′′(x) + γu(x)u′(x) = f(x), 0 < x < 1, (6.9)u(0) = u(1) = 0.

The source term f and a constant γ > 0 are given. To set up the FAS solutionto this problem, let Ωh consist of the grid points xj = j

n , for some positive eveninteger n, and let uj = u(xj) and fj = f(xj), for j = 0, 1, . . . , n. Of the manypossible ways to discretize this differential equation, consider the following finitedifference scheme:

Ahj (u) =

−uhj−1 + 2uh

j − uhj+1

h2+ γuh

j

(uh

j+1 − uhj−1

2h

)= fj , 1 ≤ j ≤ n − 1.

Note the use of a centered difference approximation to u′(xj). One consequenceof this choice is that this scalar equation is linear in uh

j even though the vectorequation Ah(uh) = fh is nonlinear in uh. Thus, nonlinear Gauss–Seidel requiresno Newton step and can be stated explicitly as follows:

uhj ← 2

h2fj +(uh

j−1 + uhj+1

)4 + hγ(uh

j+1 − uhj−1)

, 1 ≤ j ≤ n − 1 . (6.10)

To examine the FAS correction step, assume that an approximation vh has beenobtained on the fine grid. Residual equation (6.5), given by

A2hj (v2h + e2h) − A2h

j (v2h) = r2hj ,

appears in component form as

−(v2hj−1 + e2h

j−1) + 2(v2hj + e2h

j ) − (v2hj+1 + e2h

j+1)4h2

+ γ(v2hj + e2h

j )

(v2h

j+1 − v2hj−1

4h+

e2hj+1 − e2h

j−1

4h

)−

−v2hj−1 + 2v2h

j − v2hj+1

4h2

− γv2hj

(v2h

j+1 − v2hj−1

4h

)= r2h

j , 1 ≤ j ≤ n

2− 1 .

A Multigrid Tutorial 101

Canceling terms leaves the coarse-grid equation for the unknowns e2hj :

−e2hj−1 + 2e2h

j − e2hj+1

4h2+ γv2h

j

(e2hj+1 − e2h

j−1

4h

)+ γe2h

j

(v2h

j+1 − v2hj−1

4h

)

+ γe2hj

(e2hj+1 − e2h

j−1

4h

)= I2h

h (fhj − Ah

j (vh))︸ ︷︷ ︸r2h

j

.

As before, the terms v2hj , v2h

j+1, v2hj−1, and r2h

j are obtained by restriction from thefine grid.

This equation is the analogue of FAS equation (6.6), although we have writtenout the terms explicitly. Approximations to the solution, e2h

j , of this equation mustbe computed using a relaxation method such as nonlinear Gauss–Seidel (which canagain be carried out explicitly). These corrections are interpolated up to the finegrid and used to update the fine-grid approximation vh.

Numerical example: FAS for a boundary value problem. To study theperformance of FAS on boundary value problem (6.9), we choose the exact solution

u(x) = ex(x − x2),

which results in the source term

f(x) = (x2 + 3x)ex + γ (x4 − 2x2 + x)e2x.

The problem is discretized on a grid with n = 512 (511 interior points). On thecoarsest grid, consisting of one interior point and two boundary points, the problemis solved exactly by

u1 =f1

8.

In this experiment, we treat the problem using full weighting, linear interpola-tion, and a (2,1) FAS V-cycle based on nonlinear Gauss–Seidel (6.10). For γ = 0,the problem reduces to the one-dimensional Poisson equation, which is the standardmodel problem for multigrid. In this instance, the FAS V-cycle reduces to the stan-dard V-cycle. As γ increases, the nonlinear term γu(x)u′(x) begins to dominatethe problem.

Table 6.1 displays the performance of FAS for various values of γ. Starting withan initial guess of vh = 0, the FAS V-cycles are carried out until the norm of theresidual vector is less than 10−10. Displayed for each choice of γ is the number ofFAS V-cycles required to achieve this tolerance, as well as the average convergencefactor per cycle. For γ ≤ 10, FAS performance is essentially the same as for thelinear case, γ = 0. As γ increases, the performance degrades slowly until, forγ ≥ 40, FAS no longer converges.

We see similar qualitative behavior if we change the exact solution to u(x) =x − x2 so that f(x) = 2 + γ (x − x2)(1 − 2x). FAS performance for this problemis summarized in Table 6.2. Now the method converges for γ < 100, althoughperformance degrades more noticeably as γ approaches γ = 100. For γ ≥ 100, themethod no longer converges. FAS is an effective solver for this problem, even forcases where the nonlinear term is relatively large.

102 Chapter 6

γ0 1 10 25 35 39 ≥ 40

Convergence factor .096 .096 .096 .163 .200 .145 NCNumber of FAS cycles 11 11 11 14 16 13 NC

Table 6.1: FAS performance for the problem −u′′ + γ u u′ = f , with the exactsolution u(x) = ex(x − x2), discretized on a grid with 511 interior points. Shownare average convergence factors and the number of FAS cycles needed to convergefrom a zero initial guess to a residual norm of 10−10. NC indicates that the methoddid not converge.

γ0 1 10 50 70 90 ≥ 100

Convergence factor .096 .096 .094 .148 .432 .728 NCNumber of FAS V-cycles 11 11 11 13 30 79 NC

Table 6.2: FAS performance for the problem −u′′ + γ u u′ = f , with the exactsolution u(x) = x − x2, discretized on a grid with 511 interior points. Shown areaverage convergence factors and the number of FAC cycles needed to converge froma zero initial guess to a residual norm of 10−10. NC indicates that the method didnot converge.

Numerical example: Two-dimensional boundary value problem. We finishthe numerical examples with a study of the performance of FAS and Newton solversapplied to the two-dimensional nonlinear problem

−∆u(x, y) + γ u(x, y) eu(x,y) = f(x, y) in Ω, (6.11)u(x, y) = 0 on ∂Ω, (6.12)

where Ω is the unit square [0, 1] × [0, 1]. For γ = 0, this problem reduces to themodel problem. We discretize this equation on uniform grids in both directions,with grid spacings of h = 1

64 and h = 1128 . Using the usual finite difference operator,

the equation for the unknown ui,j at (xi, yj) = (ih, jh) becomes

4ui,j − ui−1,j − ui+1,j − ui,j−1 − ui,j+1

h2+ γ ui,j eui,j = fi,j , 1 < i, j < n.

(6.13)To enforce the boundary condition, we set

u0,j = uN,j = ui,0 = ui,N = 0

wherever these terms appear in the equations.We consider several approaches to solving this problem. An FAS solver can

be implemented in a straightforward way. We use full weighting and linear inter-polation for the intergrid transfer operators. For the coarse-grid versions of thenonlinear operator, we use discretization (6.13) with the appropriate grid spacing(2h, 4h, . . .). Because the individual component equations of the system are non-linear, the nonlinear Gauss–Seidel iteration uses scalar Newton’s method to solve

A Multigrid Tutorial 103

the (i, j)th equation for ui,j (Exercise 5):

ui,j ← ui,j −h−2(4ui,j − ui−1,j − ui+1,j − ui,j−1 − ui,j+1) + γ ui,j eui,j − fi,j

4h−2 + γ (1 + ui,j) eui,j.

(6.14)

The nonlinear Gauss–Seidel smoother is not intended to solve the system exactly,so we need not solve for ui,j exactly. Although no fixed rule exists to determinehow many Newton steps are required in each nonlinear Gauss–Seidel sweep, a smallnumber usually suffices (we used one Newton step in this example). It should benoted that the number of Newton steps used on the scalar problem can have asignificant impact on both the effectiveness and the cost of the FAS algorithm.

In the previous example, it was simple to solve the equation on the coarsestgrid exactly. This is not the case here, where, on a 3× 3 grid with a single interiorpoint, the equation to be solved for u1,1 is

16u1,1 + γ u1,1 eu1,1 = f1,1.

Because this equation is nonlinear, the “solve” on the coarse grid requires the use ofNewton’s Method. All experiments presented here use a 3 × 3 grid as the coarsestgrid, and in all cases it was determined experimentally that a single Newton stepwas sufficient there.

In this example, we use the source term

f(x, y) = 2((x − x2) + (y − y2)) + γ (x − x2)(y − y2) e(x−x2)(y−y2),

which corresponds to the exact solution

u(x, y) = (x − x2)(y − y2).

The first test examines the effect of FAS for various choices of γ, from the linearcase (γ = 0) through cases in which the nonlinear part of the operator dominates(γ = 10, 000). For each test, the problem is solved on a 127 × 127 interior grid.The problem is deemed solved when the norm of the residual is less than 10−10, atwhich point the average convergence rate is reported. Also reported is the numberof V-cycles required to achieve the desired tolerance. The results appear in Table6.3.

γ0 1 10 100 1000 10000

Convergence factor .136 .135 .124 .098 .072 .039Number of FAS cycles 12 12 11 11 10 8

Table 6.3: FAS performance for the problem −∆u + γ u eu = f , discretized on agrid with 127 × 127 interior points.

One feature of interest is that this problem becomes easier to solve as the non-linear term becomes more dominant. Note that this particular nonlinear term,ueu, involves only the unknown ui,j , and none of its neighboring values. Therefore,as the nonlinear term becomes more dominant, the problem becomes more local

104 Chapter 6

in nature. As this occurs, the smoothing steps, being local solvers, become moreeffective. Indeed, with enough dominance of the nonlinear term, the problem couldbe solved entirely with the nonlinear Gauss–Seidel smoother. This phenomenon isanalogous to increasing diagonal dominance in the linear case. Of course, this typeof behavior would not be expected for other types of nonlinearity.

It is useful to examine how the FAS solver performs compared to Newton’smethod applied to the nonlinear system. The Jacobian matrix for this problem isa block tridiagonal system

J(u) =

J1 BB J2 B

B J3 B. . . . . . . . .

B JN−2 BB JN−1

,

where each of the block matrices is (n− 1)× (n− 1). The off-diagonal blocks B areall − 1

h2 times the identity matrix. The diagonal blocks are tridiagonal, with theconstant value − 1

h2 on the super- and sub-diagonals. The diagonal entries of Jj ,corresponding to the grid locations (xi, yj) = (ih, jh) for fixed j and 1 ≤ i ≤ n− 1,are given by

(Jj)i,i =4h2

+ γ ui,j eui,j .

To perform the Newton iteration, we choose two of the many ways to invert theJacobian matrix. The first is to compute the LU decomposition of J(u) and to useit to solve the system. With a lower triangular matrix L and an upper triangularmatrix U such that LU = J(u), we solve the system J(u)s = F (u), first solvingLy = F (u) and then solving Us = y. Because J(u) is sparse and has a narrowband of nonzero coefficients, the factors L and U are also sparse, narrow-bandedmatrices. Indeed, the banded LU decomposition can be computed quite efficiently;and because the factors L and U are triangular, solving for y and s is also fast.Table 6.4 gives the results of applying Newton’s method, with the band LU solver,to the problem −∆u + γ u eu = f , using a 63 × 63 interior grid.

γ0 1 10 100 1000 10000

Convergence factor 2.6e-13 3.9e-5 7.4e-5 3.2e-4 1.9e-4 1.2e-4Number of Newton iterations 1 3 3 4 4 4

Table 6.4: Performance of Newton’s Method for the problem −∆u + γ u eu = f ,discretized on a 63 × 63 interior grid.

A comparison of Tables 6.3 and 6.4 indicates that convergence of Newton’smethod is much faster than that of FAS. It is well known that Newton’s methodconverges quadratically, so this is not surprising. To make this comparison useful,however, it is important to compare the computational costs of the methods. Wepresent some empirical evidence on this question shortly.

First, however, we consider an alternative method for inverting the Jacobiansystem in the Newton iteration, namely, V-cycles. Certainly in the linear case

A Multigrid Tutorial 105

(γ = 0), we know this is an effective method: both the original and the Jacobianproblems are discrete Poisson equations. As γ increases, the diagonal dominanceof the Jacobian improves, so we should expect even better performance.

If we solve the Jacobian system to the same level of accuracy by V-cycles as wasdone with the LU decomposition, the results should be essentially the same as inTable 6.4. However, it is probably much more efficient to use only a few V-cycles tosolve the Jacobian system approximately. This technique is known as the inexactNewton’s method [8], which we denote Newton-MG.

We now compare the performance of FAS, Newton’s method, and Newton-MGin the following numerical experiment. Consider the same operator as before witha different source term:

−∆u + γueu =((

9π2 + γe(x2−x3) sin(3πy))

(x2 − x3) + 6x − 2)

sin(3πy). (6.15)

With γ = 10, the exact solution is u(x, y) = (x2 − x3) sin(3πy). This solution (asopposed to the previous polynomial solution) results in a nontrivial discretizationerror. The problem is discretized with n = 128 so that the interior grid is 127×127.

No. outer No. innerMethod iterations iterations MegaflopsNewton 3 – 1660.6

Newton-MG 3 20 56.4Newton-MG 4 10 38.5Newton-MG 5 5 25.1Newton-MG 10 2 22.3Newton-MG 19 1 24.6

FAS 11 – 27.1

Table 6.5: Comparison of FAS, Newton, and Newton-multigrid methods for theproblem −∆u + γ u eu = f on a 127 × 127 grid. In all cases, a zero initial guess isused.

Table 6.5 shows the costs of FAS, Newton (with a direct solve), and Newton-MGapplied to (6.15), resulting in the nonlinear system (6.13). The iteration is stoppedwhen the residual norm is less than 10−10. The column labeled outer iterations listshow many V-cycles (for FAS) or Newton steps (for Newton and Newton-MG) arerequired to achieve the desired tolerance. For Newton-MG, we varied the numberof V-cycles used to solve the Jacobian system. The column labeled inner iterationsgives the number of V-cycles used in the approximate inversion of the Jacobian sys-tem. The last column, labeled Megaflops, is the number of millions of floating-pointoperations required to achieve the desired tolerance (as determined by the MAT-LAB flops function). For this example, using these performance measurements, itappears that both the multigrid-based methods are much more efficient than New-ton’s method using a direct solver. Furthermore, Newton-MG compares well toFAS when the number of inner MG iterations is properly tuned.

These results should not be taken too seriously. While the convergence proper-ties may be representative for a fairly broad class of nonlinear problems, the oper-ation counts are likely to vary dramatically with the character of the nonlinearity,details of implementation, computer architecture, and programming language. Itshould also be remembered that we have not accounted for the cost of evaluating the

106 Chapter 6

nonlinear function, which is highly problem-dependent. Therefore, it may be riskyto draw any general conclusions from this single experiment. On the other hand, itdoes seem fairly clear that the multigrid-based methods will outperform Newton’smethod using direct solvers when the problems are large. It is also clear that onlya few V-cycles should be used in the Newton scheme if it is to be competitive withFAS. Finally, in the end, there may be very little difference in the performancebetween the two carefully designed multigrid-based schemes. The choice may de-pend largely on convenience and other available algorithm features (for example,τ -extrapolation in FAS [4]).

There is, of course, one further option we should consider: combining the New-ton and FAS methods with an FMG scheme. The idea is to apply the basic solver(either Newton or FAS) on successively finer grids of the FMG cycle: for each newfine grid, the initial guess is obtained by first solving the nonlinear problem onthe next coarser grid. In the linear case, we showed that convergence to the levelof discretization was achieved in one FMG cycle. Table 6.6 shows the results ofapplying the FMG–FAS combination to (6.15); it suggests that we can hope for thesame kind of performance in the nonlinear case.

The discrete L2 norms are shown for the residual and error (difference betweencomputed and sampled continuous solutions), after one FMG–FAS cycle and eightsubsequent FAS V-cycles on the fine grid. Both the FMG cycle and the FAS V-cycles were performed using nonlinear Gauss–Seidel (2,1) sweeps. Observe that thenorm of the error is reduced to 2.0 × 10−5 by the FMG–FAS cycle alone. FurtherFAS V-cycling does not reduce the error, indicating that it has reached the levelof discretization error. However, subsequent FAS V-cycles do reduce the residualnorm further until it reaches the prescribed tolerance of 10−10. The column labeledMflops in the table gives the cumulative number of floating-point operations aftereach stage of the computation.

Cycle ||rh||h Ratio ||e||h MflopsFMG–FAS 1.07e−2 2.00e−5 3.1

FAS V 1 6.81e−4 0.064 2.44e−5 5.4FAS V 2 5.03e−5 0.074 2.49e−5 7.6FAS V 3 3.89e−6 0.077 2.49e−5 9.9FAS V 4 3.25e−7 0.083 2.49e−5 12.2FAS V 5 2.98e−8 0.092 2.49e−5 14.4FAS V 6 2.94e−9 0.099 2.49e−5 16.7FAS V 7 3.01e−10 0.102 2.49e−5 18.9FAS V 8 3.16e−11 0.105 2.49e−5 21.2

Table 6.6: Performance of the FMG–FAS cycle, followed by eight FAS V-cycles, on−∆u + γueu = f , with γ = 10. The grid size is 127 × 127. Note that one FMG–FAS cycle reduces the error to the level of discretization error, and that subsequentFAS V-cycles further reduce the residual norm quickly to the prescribed toleranceof 10−10.

We show analogous results in Table 6.7 for FMG with a Newton solver appliedto (6.15). Here we again use FMG, applying one step of Newton-MG on eachlevel in the FMG process. Each Newton step starts with an initial guess from thenext coarser grid and uses one (2,1) V-cycle. The discrete L2 norms are shown forthe residual and error after one FMG–Newton-MG cycle followed by subsequent

A Multigrid Tutorial 107

Cycle ||rh||h Ratio ||e||h MflopsFMG–Newton-MG 1.06e−002 2.50e−005 2.4

Newton-MG 1 6.72e−004 0.063 2.49e−005 4.1Newton-MG 2 5.12e−005 0.076 2.49e−005 5.8Newton-MG 3 6.30e−006 0.123 2.49e−005 7.5Newton-MG 4 1.68e−006 0.267 2.49e−005 9.2Newton-MG 5 5.30e−007 0.315 2.49e−005 10.9Newton-MG 6 1.69e−007 0.319 2.49e−005 12.6Newton-MG 7 5.39e−008 0.319 2.49e−005 14.3Newton-MG 8 1.72e−008 0.319 2.49e−005 16.0Newton-MG 9 5.50e−009 0.319 2.49e−005 17.7

Newton-MG 10 1.76e−009 0.319 2.49e−005 19.4Newton-MG 11 5.61e−010 0.319 2.49e−005 21.1Newton-MG 12 1.79e−010 0.319 2.49e−005 22.8Newton-MG 13 5.71e−011 0.319 2.49e−005 24.5

Table 6.7: Performance of the FMG–Newton-MG cycle, followed by 13 Newton-MGsteps, on −∆u + γueu = f , with γ = 10. The grid size is 127× 127. Note that oneFMG–Newton-MG cycle reduces the error to the level of discretization error, andthat subsequent Newton-MG steps on the fine grid further reduce the residual errorto the prescribed tolerance of 10−10.

Newton-MG cycles on the fine grid. The results are very similar to those for FMG–FAS in Table 6.6. Observe that the norm of the actual error is reduced to the levelof discretization error by one FMG–Newton cycle. Subsequent Newton-MG cyclesdo, however, continue to reduce the discrete L2 norm of the residual effectively tobelow the prescribed tolerance.

Both of these methods reduce the error to the level of discretization in one FMGcycle. The flop count indicates that the methods are similar in cost, with the FMG–Newton-MG cycle somewhat less expensive (2.4 Mflops) than the FMG–FAS cycle(3.1 Mflops). However, the individual Newton-MG steps on the fine grid, althoughcheaper, are not quite as effective as FAS V-cycles for reducing the residual norm.Indeed, if the goal is to reduce the residual norm to 10−10, it is somewhat lessexpensive (21.2 vs. 26.2 Mflops) to use FAS than Newton-MG. We remind thereader, however, that these flop counts, like those reported earlier, should not betaken too seriously: comparative measures of efficiency depend critically on specificimplementation details, computing environment, and problem characteristics. Themajor conclusion to be reached is that both methods are very efficient and robust.

One of the goals of FMG is to obtain (efficiently) results that are comparableto the accuracy of the discretization. For nonlinear problems, the nested iterationfeature of FMG improves the likelihood that initial guesses will lie in the basin ofattraction of the chosen iterative method, which accounts for some of its effective-ness. We do not claim that FMG solves all the difficulties that nonlinear problemspresent. Rather, FMG should be considered a potentially powerful tool that canbe used to treat nonlinear problems that arise in practice.

So far we have developed FAS in a fairly mechanical way. We showed howthe development of multigrid for linear problems can be mimicked to produce ascheme that makes sense for nonlinear equations. However, what we have not done

108 Chapter 6

is motivate this development from basic principles in a way that suggests why itworks. We attempt to do so now.

Here is the central question in treating nonlinearities by multigrid: When work-ing on the fine grid with the original equation in the form Ah(vh + eh) = fh, howcan the error for this equation be effectively represented on a coarser grid?

This is analogous to asking: How do you move from the continuous problemto the fine grid; that is, how do you effectively discretize the differential equationA(v + e) = f with a known function v? For one possible answer, we turn to thesecond example above (−u′′ + γuu′ = f) with u replaced by v + e:

−(v + e)′′ + γ(v + e)(v + e)′ = f.

It is important first to think of e and e′ as small quantities so that the dominantpart of this equation is the term −v′′ + γvv′, which must be treated carefully. Toexpose this term in the equation, we expand the products to obtain

−(v′′ + e′′) + γ(vv′ + v′e + ve′ + ee′) = f.

Since −v′′ + γvv′ is known, we simply move it to the right side to obtain thedifferential residual equation

γ(ve′ + v′e + ee′) = f − (−v′′ + γvv′). (6.16)

A natural way to discretize this equation begins by evaluating the right side,

r = f − (−v′′ + γvv′) = f − A(v),

at the grid points; this produces a vector rhj = r(xj) . This way of transferring r

from the differential setting to Ωh is analogous to transferring the fine-grid residualrh to Ω2h by injection: r2h

j = rh2j .

For the left side of residual equation (6.16), we need a sensible scheme forevaluating the coefficients v and v′ on Ωh. For v, the natural choice is again toevaluate it at the grid points: vh

j = v(xj). For v′, we can use a central differenceapproximation at the grid points:

(v′)hj ≈ v(xj+1) − v(xj−1)

2h.

We are now left with the task of representing e and e′ on Ωh. This is naturallydone using the expressions eh

j and

(e′)hj ≈

ehj+1 − eh

j−1

2h,

respectively.Putting all of this together gives us a fine-grid approximation to the differential

residual equation. We write it at grid point x2j of the fine grid as follows:

γvh2j

(eh2j+1 − eh

2j−1

2h

)+ γeh

2j

(vh2j+1 − vh

2j−1

2h

)+ γeh

2j

(eh2j+1 − eh

2j−1

2h

)

= fh2j − Ah

2j(vh)︸ ︷︷ ︸

rh2j

.

A Multigrid Tutorial 109

We have shown how to move from the continuum to the fine grid in a naturalway. How do we make the analogous move from the fine grid to the coarse grid?Terms evaluated at x2j can be restricted directly to the coarse grid; for example,vh2j = v2h

j . The difference approximations can be written as analogous centereddifferences with respect to the coarse-grid points. Making these replacements, wehave the coarse-grid approximation to the fine-grid residual equation:

γv2hj

(e2hj+1 − e2h

j−1

4h

)+ γe2h

j

(v2h

j+1 − v2hj−1

4h

)+ γe2h

j

(e2hj+1 − e2h

j−1

4h

)= (I2h

h rh)j .

This is precisely the FAS equation that we derived in the example above.We just developed FAS as a way to go from the fine to the coarse grid and showed

that it can be viewed as a natural extension of the discretization process that goesfrom the continuum to the fine grid. This exposes a general multigrid coarseningprinciple: What is good for discretization is probably good for coarsening. Thereare notable exceptions to this principle, but the discretization process is always agood place to start for guidance in constructing the coarse-grid correction scheme.

Along these lines, notice that, in moving from the continuum to the fine grid,we could have computed the approximation to v′ by reversing the order of thesteps, that is, by first computing the derivative v′, then evaluating it at the gridpoints: (v′)h

i ≈ v′(xi). When moving from the fine grid to the coarse grid, thischoice means we use the fine-grid difference approximation as a coefficient in thecoarse-grid equation. Its coarse-grid correction formula is

γv2hj

(e2hj+1 − e2h

j−1

4h

)+ γe2h

j

(vh2j+1 − vh

2j−1

2h

)+ γe2h

j

(e2hj+1 − e2h

j−1

4h

)= (I2h

h rh)j .

Note that the only difference over FAS is in the coefficient of the second term.This alternative illustrates that FAS is not necessarily the most direct way to

treat the nonlinearity. In many cases, there are coarsening strategies that aremore compatible with the discretization processes. See [15] for examples that usediscretizations based on projections, as the finite element method does. On theother hand, FAS is advantageous as a general coarsening approach because of itsvery convenient form

A2h(v2h + e2h) − A2h(v2h) = r2h .

Exercises

1. Linear problems. Assume that Ah is actually a linear operator: Ah(uh) =Ahuh. Show that formula (6.5) for the FAS coarse-grid correction reduces tothe formula for the two-grid correction scheme given in Chapter 3. Do thisfirst by showing that the coarse-grid equations are the same, then by showingthat the corrections to the fine-grid approximations are the same.

2. Fixed point property. Assume that relaxation has the fixed point property,namely, that if the exact solution uh of equation (6.4) is used as the initialguess vh, then the result is uh. Assume also that the coarse-grid equationhas a unique solution.

110 Chapter 6

(a) Assume that the coarse-grid equation is solved exactly. Show that FASalso has this fixed point property.

(b) Now show that this fixed point property holds even if the exact solverof the coarse-grid equations is replaced by an appropriate approximatesolver. Assume only that the coarse-grid solver itself exhibits this fixedpoint property.

3. Residual equation. You may have wondered why the FAS scheme is basedon the residual equation and not the original equation Ah(uh) = fh. To seethis more clearly, consider the following coarse-grid approximation to Ah(vh+eh) = fh:

A2h(v2h + e2h) = I2hh fh .

If this is used in place of the coarse-grid equation A2h(v2h+e2h) = A2h(v2h)+r2h, show that FAS generally loses both fixed point properties described inthe previous exercise. Thus, for example, if you start with the exact finegrid solution, you will not get it back after the first iteration. Note also thatthe solution of this coarse-grid equation does not change from one FAS cycleto the next, so that it cannot properly represent smooth components of thefine-grid error, which do change.

4. Example revisited. Here we show that problem (6.7) of the first example issubtle in the sense that it has a solution if and only if the fj satisfy a certaincompatibility condition. For this purpose, assume that fj = 0 for every j.

(a) Remembering that n is an even positive integer, show that if u satisfies(6.7), then

u1 =(

f1f3 · · · fn−1

f2f4 · · · fn

)u1 .

(b) Now show that u1 must be nonzero and that we therefore have thecompatibility condition

f1f3 · · · fn−1 = f2f4 · · · fn .

(c) Assuming that this compatibility condition is satisfied, show that thesolution to (6.7) is determined up to an arbitrary multiplicative constant.

5. Scalar Newton’s methods within nonlinear Gauss–Seidel. Show thatin solving the component equations of (6.11) for ui,j , scalar Newton’s methodtakes the form given in (6.14).

6. Formulating methods of solution. Apply parts (a)–(e) to the followingboundary value problems:

(i) u′′(x) + eu = 0, (ii) u′′(x) + u2 = 1,u(0) = u(1) = 0. u(0) = u(1) = 0.

A Multigrid Tutorial 111

(a) Use finite difference to discretize the problem. Write out a typical equa-tion of the resulting system of equations.

(b) Determine if nonlinear Gauss–Seidel can be formulated explicitly. If so,write out a typical update step. If not, formulate a typical Newton stepthat must be used.

(c) Formulate Newton’s method for the system and identify the linear sys-tem that needs to be solved at each step.

(d) Formulate the solution by FAS, indicating clearly the systems that mustbe solved on Ωh and Ω2h .

Chapter 7

Selected Applications

Our model problem for highlighting basic multigrid concepts has been Poisson’sequation in one and two dimensions. To see how multigrid can be applied to otherproblems, we now consider three variations on our model problem, each, we hope,of fairly general interest: Neumann boundary conditions, anisotropic equations,and variable-coefficient/mesh problems. In each case, we present a typical, butsimplified, problem that we treat with a specially tailored multigrid technique. Themethod we propose is not necessarily the most robust, and we ignore complicatingissues for the sake of clarity. The purpose here is to illustrate the application ofmultigrid to new problems and to gain an entry-level understanding of the broadscope of multigrid techniques.

Neumann Boundary Conditions

Dirichlet boundary conditions specify values of the unknown function on the bound-ary of the domain. In most cases, these so-called essential boundary conditions areeasily treated in the discretization and in the multigrid solver by simply eliminatingthe corresponding unknowns in the equations. On the other hand, Neumann con-ditions involve derivatives of the unknown function. This relatively small changein the boundary conditions can be treated in a variety of ways; it also introduces afew important subtleties.

Consider the following two-point boundary value problem with homogeneousNeumann conditions at both end points (the inhomogeneous case is left for Exercise8):

−u′′(x) = f(x), 0 < x < 1 , (7.1)u′(0) = u′(1) = 0.

With n an odd positive integer, denote the points of a uniform grid on the interval[0, 1] by xj = jh, for 0 ≤ j ≤ n + 1, where h = 1

n+1 . A common way to dis-cretize this problem involves the ghost points x−1 = − 1

h and xn+2 = 1 + 1h . These

points are outside the problem domain and are used only temporarily to produceapproximations at the boundary points.

With this extended grid, we can use central differences for the differential equa-tion at the interior points and boundary points. We also use the ghost points to

113

114 Chapter 7

form central difference approximations for the boundary conditions. These choiceslead to the following set of discrete equations:

−uhj−1 + 2uh

j − uhj+1

h2= fh

j , 0 ≤ j ≤ n + 1 ,

uh1 − uh

−1

2h= 0 ,

uhn+2 − uh

n

2h= 0 .

We could now choose to work with this entire system of equations, which requirescareful treatment of the boundary equations in relaxation and coarsening. Instead,we eliminate the discrete boundary conditions. The two boundary condition equa-tions imply that

uh−1 = uh

1 and uhn+2 = uh

n.

Eliminating the ghost points in the j = 0 and j = n + 1 equations yields thefollowing (n + 2) × (n + 2) system of algebraic equations:

−uhj−1 + 2uh

j − uhj+1

h2= fh

j , 1 ≤ j ≤ n ,

2uh0 − 2uh

1

h2= fh

0 ,

−2uhn + 2uh

n+1

h2= fh

n+1 .

As before, we write this system in the matrix form

Ahuh = fh, (7.2)

where now

Ah =1h2

2 −2−1 2 −1

· · ·· · ·

−1 2 −1−2 2

. (7.3)

Note that system (7.2) involves the boundary unknowns uh0 and uh

n+1 and that Ah

is an (n + 2) × (n + 2) nonsymmetric matrix.The first observation to be made is that two-point boundary value problem (7.1)

is not yet well-posed. First, if this problem has a solution u, it cannot be unique:because the system involves only derivatives of u, the function u+ c must also be asolution for any constant c (Exercise 1). Second and worse yet, we cannot even besure that boundary value problem (7.1) has a solution. If a solution of (7.1) exists,then the source term must satisfy∫ 1

0

f(x) dx = 0 (7.4)

(Exercise 2). This integral compatibility condition is necessary for a solution to exist:if f does not satisfy it, then there can be no solution. Fortunately, the compatibility

A Multigrid Tutorial 115

condition is also sufficient in general. Showing this fact is a little beyond the scopeof this book. It amounts to proving that − ∂2

∂x2 is a well-behaved operator on theappropriate space of functions u that have zero mean:

∫ 1

0u(x) dx = 0.

This reasoning allows us to conclude that if f satisfies compatibility condition(7.4), then the following boundary value problem is well-posed:

−u′′(x) = f(x), 0 < x < 1 ,

u′(0) = u′(1) = 0, (7.5)∫ 1

0

u(x) dx = 0 .

The last condition says that of all the solutions u + c that satisfy the differentialequation and boundary conditions, we choose the one with zero mean.

Note that if u solves (7.5), then u + c solves the same boundary value problemwith

∫ 1

0u(x) dx = c. Thus, the solution of (7.2) with mean value c is easily found

from the zero-mean solution.In a similar way, discrete system (7.2) is not well-posed. Let 1h be the (n + 2)-

vector whose entries are all 1, so the entries of c1h are all c. The matrix Ah has anontrivial null space consisting of the constant vectors c1h (Exercise 3). Thus, ifuh is a specific solution of (7.2), then the general solution is uh + c1h (Exercise 4).

By the Fundamental Theorem of Linear Algebra, (7.2) is solvable only if fh is or-thogonal to N (AT ). It is not difficult to show that the (n+2)-vector ( 1

2 , 1, . . . , 1, 12 )

forms a basis for N (AT ). Thus, the system is solvable only if f is orthogonal toN ((Ah)T ), which implies that

12fh0 +

n∑j=0

fhj +

12fh

n+1 = 0.

As in the continuous problem, we have two issues to handle: solvability, whichrequires that fh be orthogonal to the null space of (Ah)T , and nonuniqueness,which means that two solutions differ by a vector in the null space of Ah. A usefulobservation is that if A is symmetric (A = AT ), then solvability and nonuniquenesscan be dealt with together, because the null spaces of A and AT are identical inthat case.

Fortunately, there is an extremely simple way to symmetrize this problem: wejust divide the first and last equations of the system by 2. This yields a symmetricsystem whose matrix is

Ah =1h2

1 −1−1 2 −1

· · ·· · ·

−1 2 −1−1 1

. (7.6)

Ah and its transpose have as a null space multiples of the constant vector 1h.The right-side vector becomes fh = [f0/2, f1, . . . , fn, fn+1/2]T . Solvability is thenguaranteed by ensuring that fh is orthogonal to the constant vector 1h:

〈fh, 1h〉 =n+1∑j=0

fhj = 0 (7.7)

116 Chapter 7

x0 x1 x2 x3 x4

Ω2h × × × × ×

Ωh

x0 x1 x2 x3 x4 x5 x6 x7 x8

Figure 7.1: For the Neumann boundary condition case, the solution is unknown atthe end points x0 and xn+1, where n is an odd integer. The figure shows Ωh andΩ2h for the case n = 7.

(Exercise 5). This is the discrete analogue of integral compatibility condition (7.4).Hence, the discrete analogue of the well-posed differential system (7.5) is

−uhj−1 + 2uh

j − uhj+1

h2= fh

j , 1 ≤ j ≤ n ,

uh0 − uh

1

h2=

fh0

2, (7.8)

−uhn + uh

n+1

h2=

fhn+1

2,

n+1∑i=0

uhi = 0 ,

or simply

Ahuh = fh , (7.9)〈uh, 1h〉 = 0 . (7.10)

As before, of all solutions to the problem, that last condition selects the solutionwith zero mean. Matrix system (7.9)–(7.10) is well-posed in the sense that it has aunique solution, provided fh satisfies (7.7).

The ingredients of our multigrid algorithm for solving (7.9)–(7.10) are basicallythe same as before, except we now must account for the extra equations associatedwith the unknowns at the boundary points and the zero-mean condition. Figure7.1 shows a fine grid, Ωh, with n + 2 = 9 points and the corresponding coarse gridΩ2h with n+3

2 = 5 points.The approximation vh must include the entries vh

0 and vhn+1, and the coarse-grid

correction v2h must include v2h0 and v2h

(n+1)/2. Relaxation is now performed at allthe fine-grid points, including x0 and xn+1. Relaxation at the end points is basedon the second and third equations of (7.8):

vh0 ← vh

1 + h2fh0 ,

vhn+1 ← vh

n + h2fhn+1.

To account for zero-mean condition (7.10), we now just add the step

vh ← vh − 〈vh, 1h〉〈1h, 1h〉1

h, (7.11)

A Multigrid Tutorial 117

which is just the Gram–Schmidt method applied to orthogonality condition (7.10).(Note that 〈1h, 1h〉 = n + 2.) This global step can be applied before or afterrelaxation on each level. In principle, this step can wait until the very end of themultigrid algorithm since its primary role is to produce an approximation with thecorrect average. From a mathematical point of view, it does not really matter if theintermediate approximation vh has a nonzero mean. However, from a numericalpoint of view, it is probably best to apply (7.11) often enough to prevent a largeconstant from creeping in and swamping accuracy.

The coarsening process is similar to that for the Dirichlet case, but now wemust remember to interpolate corrections to the end point approximations. Theseend points are present on both fine and coarse grids, so the correction process isstraightforward. For example, recalling that the terms v2h

j are errors computed onthe coarse grid, the correction steps are given by

vh0 ← vh

0 + v2h0 , (7.12)

vh1 ← vh

1 +v2h0 + v2h

1

2.

What about restriction of residuals to the coarse grid? In the Dirichlet case, wechose I2h

h to be the transpose of Ih2h scaled by 1

2 :

I2hh =

12

(Ih2h

)T.

We will use the same restriction operator here, but we must determine its form atthe boundary points. Correction step (7.12) shows that interpolation relates v2h

0

to vh0 using the weight 1 and relates v2h

0 to vh1 using the weight 1

2 . Thus, reversingthese relationships (which is what the transpose does) and scaling them by 1

2 , wearrive at the restriction step (Exercise 6)

f2h0 ← 1

2fh0 +

14fh1 .

The coarse-grid matrix we get from the Galerkin condition, A2h = I2hh AhIh

2h,is just the coarse-grid version of Ah defined in (7.3): this follows from Table 5.1for the interior points, and from Table 7.1 for the boundary point x = 0 (x = 1is analogous). One consequence of this relationship is that the multigrid schemepreserves the variational properties we used in the Dirichlet case.

Another subtlety that we need to address is the discrete compatibility condition.If we are given a general source vector fh, we can be sure that the fine-grid problemhas a solution simply by testing the zero-mean condition (7.7). But what do wedo if this test fails? We could simply stop because this problem has no solution.However, we might find it advantageous to solve the nearby problem created bymaking the replacement

fh ← fh − 〈fh, 1h〉〈1h, 1h〉1

h . (7.13)

But what about the coarse-grid problem? The answer is that the coarse-grid equa-tion will be solvable, at least in theory, whenever the fine-grid equation is solvable(Exercise 7). However, to be sure that numerical round-off does not perturb the

118 Chapter 7

0 1

e2h0 1 0

Ih2he

2h0 1 1

2 0

AhIh2he

2h0

12h2 0 − 1

2h2

I2hh AhIh

2he2h0

14h2 − 1

4h2

Table 7.1: Calculation of the first row of A2h = I2hh AhIh

2h at the boundary x = 0.

solvability too much, it is probably best to incorporate a Gram–Schmidt step anal-ogous to (7.13) on the coarse grids:

f2h ← f2h − 〈f2h, 12h〉〈12h, 12h〉1

2h .

Note that 〈12h, 12h〉 = n+32 .

Numerical example. Consider the two-point boundary value problem

−u′′(x) = 2x − 1, 0 < x < 1,

u′(0) = u′(1) = 0.

By integrating the differential equation twice and applying the boundary conditions,we find that the function u(x) = x2

2 − x3

3 + c solves the problem for any constantc. The zero-mean solution corresponds to c = − 1

12 , and we use this function asour exact solution in the numerical experiments. Solutions are approximated on asuccession of grids to illustrate the effectiveness of the algorithm as n increases.

Symmetrization of the problem produces the source vector f , where fi = fi for1 ≤ i ≤ n,

f0 =f0

2= −1

2, and fn+1 =

fn+1

2=

12.

It can be checked that f satisfies discrete compatibility condition (7.7). The restric-tion, interpolation, and coarse-grid operators described above are used with V(2,1)-cycles based on Gauss–Seidel relaxation. For this problem, which is computed indouble precision, we observed that it was enough to apply the Gram–Schmidt pro-cess once at the end of each V-cycle.

Table 7.2 shows the results of this experiment. Each test is stopped when thediscrete L2 norm of the residual drops below 10−10. Listed are the grid sizes,the final residual norm, the average convergence factor, the final error norm (thedifference between the computed solution and the sampled version of the continuoussolution), and the number of V-cycles required to reach the desired tolerance.

A Multigrid Tutorial 119

Grid size Average Numbern ‖rh‖h conv. factor ‖e‖h of cycles31 6.3e−11 0.079 9.7e−05 963 1.9e−11 0.089 2.4e−05 10127 2.6e−11 0.093 5.9e−06 10255 3.7e−11 0.096 1.5e−06 10511 5.7e−11 0.100 3.7e−07 101027 8.6e−11 0.104 9.2e−08 102047 2.1e−11 0.112 2.3e−08 104095 5.2e−11 0.122 5.7e−09 11

Table 7.2: Numerical results for −u′′(x) = f(x), u′(0) = u′(1) = 0. Shown arediscrete L2 norms of the final residual and the final error, average convergencefactors, and number of V(2,1)-cycles required to reduce the residual norm to lessthan 10−10.

It is evident from the table that the method is very robust on this problem inthat the speed of convergence is essentially independent of problem size. Further-more, with the strict tolerance on the residual norm used, the discrete problemsare solved well below the level of discretization error. The final errors represent thediscretization errors themselves: they decrease by almost exactly one-fourth witheach doubling of n, which is evidence of O(h2) behavior.

An interesting subtlety of this problem is that we could have chosen to useone-sided differences for approximating the boundary derivatives:

u′(0) ≈ u1 − u0

hand u′(1) ≈ un+2 − un+1

h.

This choice would have resulted in exactly the same operator matrix, Ah, that weobtained by symmetrizing the discretization based on central differences. However,the source vector would be the original vector, f , rather than the vector f . Usingthis system, and leaving the remainder of the algorithm unchanged, we found ex-perimentally that the speed of convergence and residual errors were approximatelyas we see in Table 7.2, as expected. However, the final errors (that is, the discretiza-tion errors) were much larger and did not decrease nearly as fast with increasingn, because one-sided differences are not as accurate as central difference ((O(h)compared to O(h2)).

Anisotropic Problems

The problems treated thus far have been limited to matrices with constant nonzerooff-diagonal entries, namely, − 1

h2 . Such matrices naturally arise in treating Pois-son’s equation on uniform grids. Our first departure from this situation is to con-sider two-dimensional problems in which the matrices have two different constantsappearing in the off-diagonal terms. Such matrices arise when

• the differential equation has constant, but different, coefficients for the deriva-tives in the coordinate directions, or

120 Chapter 7

• when the discretization has constant but different mesh sizes in each coordi-nate direction.

The model problems for these two cases are, respectively, the differential equa-tion

−uxx − εuyy = f (7.14)

discretized on a uniform grid of mesh size h and Poisson’s equation (ε = 1) dis-cretized on a grid with constant mesh size hx = h in the x-direction and constantmesh size hy = h√

εin the y-direction. To be very distinct from the isotropic case

ε = 1, we assume that 0 < ε << 1.It is interesting that these two anisotropic model problems lead to the same

five-point stencil (Exercise 9):

Ah =1h2

−ε

−1 2 + 2ε −1−ε

. (7.15)

This relationship between problems with variable coefficients and problems withvariable mesh sizes is important; it means that we can think of either example as wedevelop a method to treat anisotropic problems. Note that the weak connection inthe vertical direction in these examples arises, respectively, from a small coefficientof the y-derivative term or a large grid spacing in the y-direction.

This departure from the model (isotropic) problem unfortunately leads to trou-ble with the standard multigrid approach: multigrid convergence factors degradeas ε tends to zero, and even for ε ≈ 0.1, poor performance can be expected.

To understand why multigrid deteriorates, consider (7.14) in the limiting caseε = 0. The matrix then becomes

Ah =1h2

0

−1 2 −10

,

which means that the discrete problem becomes a collection of one-dimensionalPoisson equations in the x-direction, with no connections in the y-direction. Gauss–Seidel or damped Jacobi point relaxation will smooth in the x-direction becauseit does so in one dimension. However, the lack of connections in the y-directionmeans that errors on one x-line have nothing to do with errors on any other: errorsin the y-direction will generally have a random pattern, far from the smoothnessneeded for coarsening to work well.

We can gain further insight into the problems that arise with anisotropy if welook at the eigenvalues of the iteration matrix. Recalling Exercise 12 of Chapter 4,if the weighted Jacobi method with parameter ω is applied to the model Poissonequation in two dimensions on an n×n grid, the eigenvalues of the iteration matrixare given by

λk, = 1 − ω

(sin2

(kπ

2n

)+ sin2

(π

2n

)), 1 ≤ k, ≤ n.

The wavenumbers (frequencies) k and correspond to the x- and y-directions, re-spectively. The same analysis (Exercise 10) applied to the problem (7.14) reveals

A Multigrid Tutorial 121

that the eigenvalues of the Jacobi iteration matrix are

λk, = 1 − 2ω

1 + ε

(sin2

(kπ

2n

)+ ε sin2

(π

2n

)), 1 ≤ k, ≤ n.

Notice that for small ε, the contributions from the wavenumbers in the y-direction(the direction of weak coupling) are insignificant. Thus, there is little variationin the eigenvalues with respect to the y-wavenumbers. The variation in eigenval-ues with respect to the x-wavenumbers is what we expect in the one-dimensionalproblem.

The eigenvalue picture is best given in the two plots of Fig. 7.2, with ε =0.05, n = 16, and ω = 2

3 . The upper plot shows the variation in eigenvalues withrespect to (the y-wavenumber) on lines of constant k; notice that there is littlevariation in the eigenvalues along a single line in this direction. The lower plotshows the variation in eigenvalues with respect to k (the x-wavenumber) on lines ofconstant . This time we see individual curves that look much like the eigenvaluecurves for the one-dimensional problem. In addition, the curves are tightly bunched,meaning that the convergence is much the same along any horizontal line of thegrid. It should be mentioned that local mode analysis, as discussed in Chapter 4,could have been used to reach many of these same conclusions.

These observations suggest two possible strategies for dealing with the anisotropy:

• Because we expect typical multigrid convergence for the one-dimensionalproblems along x-lines, we should “do multigrid” and coarsen the grid inthe x-direction, but not in the y-direction.

• Because the equations are strongly coupled in the x-direction, it will proveadvantageous to solve for entire lines of unknowns in the x-direction all atonce; this is the goal of line or block relaxation.

We now investigate these two strategies in more detail.

Semicoarsening/point relaxation

Because relaxation smooths in the x-direction, it makes sense to coarsen horizon-tally by eliminating every other vertical (y-) line. However, because point relaxationdoes not smooth in the y-direction, we should not coarsen vertically; all horizontallines should be retained on the coarse grid. This semicoarsening process is depictedin Fig. 7.3. This means that when we write Ω2h, we really mean the coarse grid thathas the original grid spacing in the y-direction and twice the original grid spacingin the x-direction.

Interpolation can be done in a one-dimensional way along each horizontal line,giving coarse-grid correction equations of the form

vh2i,j ← vh

2i,j + v2hi,j , vh

2i+1,j ← vh2i+1,j +

v2hi,j + v2h

i+1,j

2.

Semi-coarsening is not as “fast” as full coarsening: going from the fine to the coarsegrid, the number of points is reduced by a factor of about two with semicoarsening,as opposed to the usual factor of about four. This means that W-cycles lose O(n)complexity; however, V-cycle complexity remains O(n) (Exercise 12).

122 Chapter 7

0 2 4 6 8 10 12 14 16–0.4

–0.2

0

0.2

0.4

0.6

0.8

1

l (wavenumber in y direction)

0 2 4 6 8 10 12 14 16–0.4

–0.2

0

0.2

0.4

0.6

0.8

1

k (wavenumber in x direction)

Figure 7.2: The upper plot shows the eigenvalues of the Jacobi iteration matrix, withω = 2

3 , along lines of constant k (wavenumber in the x-direction). The upper curvecorresponds to k = 1 and the lower curve to k = n = 16. The lower plot shows thesame eigenvalues along lines of constant (wavenumber in the y-direction). Thesmall parameter in (7.14) is ε = 0.05.

A Multigrid Tutorial 123

Figure 7.3: The original fine grid (left, solid and dashed lines) is semicoarsened inthe x-direction by deleting the dashed lines (right).

Line relaxation/full coarsening

Line (or block) relaxation can be developed by writing a system of linear equationsin block form. In the case of (7.14), if we order the unknowns along lines of constanty (because strong coupling is in the x-direction), the matrix Ah can be written inblock form as

Ah =

D −cI−cI D −cI· −cI D −cI

· · · −cI−cI D

, (7.16)

where c = εh2 and I is the identity matrix. For this model problem, the diagonal

blocks D are tridiagonal and identical. Each block is associated with an individualhorizontal grid line and has the stencil 1

h2 (−1 2 + 2ε − 1).One sweep of the block Jacobi method consists of solving a tridiagonal system

for each line of constant y. The jth system of the sweep has the form

Dvhj = gh

j ,

where vhj is the jth subvector of vh with entries

(vh

j

)i

= vhi,j , and gh

j is the jthright-side vector with entries(

ghj

)i= fh

i,j +ε

h2

(vh

i,j−1 + vhi,j+1

).

Because D is tridiagonal, these systems can be solved efficiently with some formof Gaussian elimination. The operation count for relaxation remains O(n), and sodoes the cost of either V- or W-cycle solvers. A weighted block Jacobi methodresults from averaging the current approximation and a full Jacobi update using adamping parameter ω.

To see exactly why line relaxation in the direction of strong coupling is effective,we need to look at the convergence properties of the iteration matrix. A brisk

124 Chapter 7

0 2 4 6 8 10 12 14 16–0.2

0

0.2

0.4

0.6

0.8

1

1.2

l (wavenumber in y direction)

0 2 4 6 8 10 12 14 16–0.2

0

0.2

0.4

0.6

0.8

1

1.2

k (wavenumber in x direction)

Figure 7.4: The upper plot shows the eigenvalues of the block Jacobi iteration matrix,with ω = 2

3 , along lines of constant k (wavenumber in the x-direction). The uppercurve corresponds to k = 1 and the flattest curve to k = n = 16. The lowerplot shows the same eigenvalues along lines of constant (wavenumber in the y-direction). The upper curve corresponds to = 1 and the lower curve correspondsto = n = 16. The small parameter in (7.14) is ε = 0.05.calculation (Exercise 11) confirms that the eigenvalues of the block Jacobi iterationmatrix (with damping parameter ω) are given by

λk, = 1 − 2ω

2 sin2(kπ2n ) + ε

(sin2

(kπ

2n

)+ ε sin2

(π

2n

)), 1 ≤ k, ≤ n.

A Multigrid Tutorial 125

As before, k and are the wavenumbers in the x- and y-directions, respectively. Theeigenvalues for the block Jacobi method (with ω = 2

3 and ε = 0.05) are displayedin Fig. 7.4, which is analogous to Fig. 7.2 for the point Jacobi method. The topfigure shows the eigenvalues along lines of constant k; the lower figure shows thesame eigenvalues along lines of constant . The most striking difference betweenthe eigenvalue plots for the two Jacobi methods is the reduction in the size of theeigenvalues: the maximum magnitude of the eigenvalues of the point method is0.98, while the maximum magnitude of the eigenvalues of the block method is 0.81.This reduction in the size of the eigenvalues results in improved convergence. Thisimprovement is typical of block methods that collect additional coefficients in thediagonal blocks. However, for anisotropic problems, the improvement occurs onlyif the coefficients in the direction of strong coupling appear in the diagonal blocks.

Both of these strategies are effective for the anisotropic models. Either onecan be used when there is a weak connection in a coordinate direction that isknown beforehand. Choosing between the two often depends on the nature ofthe application and the computer architecture involved. However, problems inwhich the weak direction is unknown or changing within the domain may not beefficiently handled by either method. For more robustness, we could use pointrelaxation and alternate semicoarsening between the x- and y-directions. However,the increased number of coarse grids that this approach requires is too awkward formost applications. We could also choose to apply full coarsening and alternatingline relaxation that switches between x- and y-line relaxation. This approach canalso be a bit awkward for some applications, particularly as it extends to threedimensions. Another choice is to apply both semi-coarsening and line relaxation.

Semicoarsening/line relaxation

Suppose we want to develop one method that can handle either of the followingstencils:

Ah1 =

1h2

−ε

−1 2 + 2ε −1−ε

or Ah

2 =1h2

−1

−ε 2 + 2ε −ε−1

.

Semi-coarsening in the x-direction could be used to handle Ah1 and y-line relaxation

to handle Ah2 . If we do both, the problem may be viewed as a stack of pencils in

the y-direction as shown in Fig. 7.5. Line relaxation in the y-direction is used tosolve the problem associated with each pencil, and coarsening is done simply bydeleting every other pencil. There is no assumption here about the direction ofweak connections; in fact, this approach applies well to the isotropic case. Again,we lose the O(n) complexity of W-cycles, but V-cycle complexity remains O(n)(Exercise 13).

Numerical example. Many of the above ideas can be illustrated with a fairlysimple example. Consider equation (7.14),

−uxx − εuyy = f,

with homogeneous Dirichlet boundary conditions. The discretized problem hasthe stencil given by (7.15). We apply three different schemes to this problem for

126 Chapter 7

Figure 7.5: It is possible to combine line relaxation and semicoarsening. The origi-nal grid (left) is viewed as a collection of pencils for line relaxation in the y-direction(center) and semi-coarsened in the x-direction (right).

selected values of ε, ranging from large values, so that coupling is dominant in they-direction, to very small values, so that coupling is dominant in the x-direction.Our example uses the right-side function

f(x, y) = 2(y − y2) + 2ε(x − x2),

which produces the exact solution

u(x, y) = (x − x2)(y − y2).

The first approach consists of a standard V(2,1)-cycle, based on Gauss–Seidel re-laxation, full coarsening, full weighting, and linear interpolation. We do not expectthis method to be effective, except when ε ≈ 1. A simple demonstration pinpointsthe difficulty we expect to encounter in this experiment.

Letting ε = 0.001 and using a 16 × 16 grid with n = 16 points, we begin theiteration with a random initial guess. Figure 7.6 shows a surface plot of the errorafter 50 sweeps of Gauss–Seidel. We also show the error along a line of constanty (middle figure) and a line of constant x (bottom figure). Relaxation apparentlysmooths the error nicely in the x-direction, the direction of strong coupling, butleaves highly oscillatory error in the y-direction. While it is easy to envision ap-proximating the smooth error curve on a coarser grid, we could not hope to do sowith the oscillatory error. Therefore, we must keep all the points in the y-directionif we wish to represent the error accurately in that direction.

These observations lead to the second approach, which is semicoarsening. Be-cause smoothing in the y-direction with point Gauss–Seidel is ineffective when εis small, we coarsen only in the x-direction. This means that the coarse and finegrids have the same number of points in the y-direction. Full weighting and linearinterpolation are used, but in a one-dimensional way that involves neighbors inthe x-direction only. Semicoarsening means that we do not use neighbors in they-direction for either restriction or interpolation.

It is important to notice that once the semicoarsening has been performed, thediscrete operator must be altered to fit the new geometry. That is, because thegrid spacing is no longer the same in the two directions, the residual and relaxation

A Multigrid Tutorial 127

0

5

10

15

0

5

10

150

0.02

0.04

0.06

0.08

0.1

0 2 4 6 8 10 12 14 160

0.05

0.1

0.15

0.2

0.25

0 2 4 6 8 10 12 14 160.1

0.15

0.2

0.25

Figure 7.6: Top: Error after 50 sweeps of pointwise Gauss–Seidel on the equation−uxx − εuyy = 0, beginning with a random initial guess. Middle: Error in thestrongly coupled x-direction along a line of constant y. Bottom: Error along a lineof constant x, in the weakly coupled y-direction.

operators must use a stencil like

A =

− ε

h2y

− 1h2

x

(2h2

x

+2ε

h2y

)− 1

h2x

− ε

h2y

,

128 Chapter 7

εScheme 1000 100 10 1 0.1 0.01 0.001 0.0001 0

V(2,1)-cycles 0.95 0.94 0.58 0.13 0.58 0.90 0.95 0.95 0.95Semi-C 0.99 0.99 0.98 0.93 0.71 0.28 0.07 0.07 0.07

Semi-C/line relax 0.04 0.08 0.08 0.08 0.07 0.07 0.07 0.08 0.08

Table 7.3: Convergence factors for the three different schemes for various valuesof ε. The V(2,1) scheme is just standard multigrid, with Gauss–Seidel relaxation,linear interpolation, and full weighting. The semi-C scheme uses semicoarseningin the strongly coupled x-direction. The semi-C/line relax scheme combines semi-coarsening in the x-direction with line relaxation in the y-direction.

where hx and hy are the grid spacings in the x- and y-directions, respectively. Ourearlier discussion suggests that this scheme will work well when the coupling isdominant in the x-direction (small ε), but that it may work poorly otherwise.

The third approach we consider is appropriate when the size of ε is unknown,so that the anisotropy could be in either coordinate direction. Here we use semi-coarsening in the x-direction and line relaxation in the y-direction. Again, weuse full weighting and linear interpolation with neighbors in the x-direction only.Semicoarsening should take care of strong coupling in the x-direction, while linerelaxation should handle strong coupling in the y-direction.

The asymptotic convergence factors for the three approaches and for variouschoices of ε are displayed in Table 7.3. The standard V(2,1)-cycle approach workswell only when ε is very near 1. It converges poorly for values of ε that differfrom 1 by an order of magnitude or more. The semicoarsening approach performsextremely well when ε is very small, but its convergence deteriorates even whenε = 0.01 and 0.1. There is a subtle lesson here. Semicoarsening is effective provided1

h2x

remains larger than εh2

yas hx is doubled. This means that the coarse grids

see much the same anisotropy as the fine grid. However, if 1h2

xand ε

h2y

becomecomparable in size during the x-coarsening process, then convergence rates willdecrease. This effect can be observed in Table 7.3. With n = 16 and ε = 0.1,coarsening in the x-direction quickly produces coupling in the y-direction. Ratherthan continue to semicoarsen in the x-direction, it is best to return to standardcoarsening in this situation.

Finally, the combination of semicoarsening with line relaxation is extremelyrobust, giving excellent convergence results regardless of the degree of anisotropy.

Variable-Mesh Problems

Nonuniform grids are commonly used in practice to accommodate irregularities inthe problem domain or multiple scales in the emerging solution. To see how multi-grid might handle such problems, consider the one-dimensional Poisson equation

−u′′(x) = f(x) , 0 < x < 1 , (7.17)u(0) = u(1) = 0.

A Multigrid Tutorial 129

xh0 xh

1 xh2 xh

3 xh4 xh

5 xh6

x2h0 x2h

1 x2h2 x2h

3

Figure 7.7: With a variable mesh, the coarse grid is defined as the even-numberedgrid points of the fine grid.

With n a positive even integer, define a nonuniform grid on [0 , 1] by the pointsxh

j , 0 ≤ j ≤ n, and mesh sizes hj+ 12≡ xh

j+1 − xhj , 0 ≤ j ≤ n − 1. Using second-

order finite differences, we can discretize (7.17) on this grid. The result is (Exercise14)

−αhj uh

j−1 +(αh

j + βhj

)uh

j − βhj uh

j+1 = fhj , 1 ≤ j ≤ n − 1 , (7.18)

where the coefficients are given by

αhj =

2

hj− 12

(hj− 1

2+ hj+ 1

2

) , βhj =

2

hj+ 12

(hj− 1

2+ hj+ 1

2

) , 1 ≤ j ≤ n − 1 .

(7.19)Note that the operator in (7.18) reduces to the standard stencil 1

h2 (−1 2 − 1)in the uniform grid case, hi+ 1

2= h.

We can solve this discrete system by modifying the standard multigrid schemeto accommodate variable-mesh sizes. The operator Ih

2h is again based on linearinterpolation, but it must now account for the nonuniformity of the grid. Choosingthe coarse grid (Fig. 7.7) to consist of every other fine-grid point, x2h

j ≡ xh2j , 0 ≤

j ≤ n2 , the interpolation process, vh = Ih

2hv2h, is defined by

vh2j = v2h

j , (7.20)

vh2j+1 =

h2j+ 32v2h

j + h2j+ 12v2h

j+1

h2j+ 12

+ h2j+ 32

, 1 ≤ j ≤ n

2− 1.

This formula comes from constructing a linear function in the interval [x2hj , x2h

j+1]that interpolates v2h at the end points (Exercise 15).

Another way to derive (7.20) is to apply operator interpolation. The basic ideais to assume that relaxation does not significantly change the error, so that theresidual must be almost zero. Using (7.18) at xh

2j+1, we have

−αh2j+1e

h2j +

(αh

2j+1 + βh2j+1

)eh2j+1 − βh

2j+1eh2j+2 ≈ 0. (7.21)

Solving for eh2j+1 yields

eh2j+1 ≈

αh2j+1e

h2j + βh

2j+1eh2j+2

αh2j+1 + βh

2j+1

.

This suggests that we define interpolation by the formula

vh2j = v2h

j , (7.22)

vh2j+1 =

αh2j+1v

2hj + βh

2j+1v2hj+1

αh2j+1 + βh

2j+1

, 1 ≤ j ≤ n

2− 1 .

130 Chapter 7

It is important that linear interpolation (7.20) and operator interpolation (7.22)be identical for our one-dimensional model problem (Exercise 16). Note that both(7.20) and (7.22) reduce to a simple average for the uniform grid case.

Full weighting can now be defined by the variational property

I2hh ≡ 1

2(Ih2h

)T

and the coarse-grid operator can be constructed from the Galerkin principle

A2h = I2hh AhIh

2h .

This definition of A2h yields a stencil that is similar, but not identical, to the stencilobtained by using (7.18)–(7.19) on the coarse grid. However, the Galerkin principledoes hold if this system is rescaled by (hj− 1

2+ hj+ 1

2)/2 (Exercise 17).

Variable-mesh problems in two dimensions can be handled in a similar way, butthe first issue in this case is the discretization itself. Forming an accurate discreteapproximation can be tricky on nonuniform grids in two and higher dimensions.Moreover, if the grid is logically rectangular (it can be indexed simply by i, j), thencoarsening is generally straightforward; but unstructured grids can make coarseningseem almost impossible. Instead of treating such cases here, we defer this issue toour discussion of the algebraic multigrid algorithm (Chapter 8), which was designedjust for this purpose.

Variable-Coefficient Problems

Another common practical problem is the solution of variable coefficient differentialequations. Again we consider a simple case to see how multigrid might handle suchproblems. Our model problem is the scalar elliptic equation

−(a(x)u′)′(x) = f(x) , 0 < x < 1 , (7.23)u(0) = u(1) = 0,

where a(x) is a positive function on the interval (0 , 1). A conservative or self-adjointmethod for discretizing (7.23) is developed as follows.

Letting n be a positive even integer, consider the uniform grid defined by thepoints xh

j = jn , 0 ≤ j ≤ n. As usual, suppose that the vector vh approximates u

at these grid points, but that values of the coefficient a are taken at the cell centersxh

j+ 12

= j+ 12

n ; thus, we have ahj+ 1

2= a(xh

j+ 12) for 0 ≤ j ≤ n− 1. The discrete system

that results (Exercise 18) is

1h2

(−ah

j− 12uh

j−1 + (ahj− 1

2+ ah

j+ 12)uh

j − ahj+ 1

2uh

j+1

)= fh

j , 1 ≤ j ≤ n − 1,

uh0 = uh

n = 0. (7.24)

Note that (7.24) becomes the usual discretization in the case a(x) = 1.For slowly varying a(x), we can use a standard multigrid scheme much like the

algorithm used for the model problem. One reasonable approach in this case is touse linear interpolation for Ih

2h, full weighting for I2hh , and the following stencil for

A2h:

A2h =1

(2h)2(−a2h

j− 12

a2hj− 1

2+ a2h

j+ 12

− a2hj+ 1

2

), 1 ≤ j ≤ n

2− 1 , (7.25)

A Multigrid Tutorial 131

where the coefficients are determined by the simple averages

a2hj+ 1

2=

ah2j+ 1

2+ ah

2j+ 32

2, 0 ≤ j ≤ n

2− 1 . (7.26)

This same stencil can be obtained (Exercise 19) from the Galerkin principle

A2h = I2hh AhIh

2h.

The performance of such a standard multigrid scheme will degrade as a(x)begins to vary significantly from cell to cell. One way to understand how thisdegradation happens is to compare (7.24), the uniform mesh discretization of thevariable-coefficient problem, to (7.18)–(7.19), the variable-mesh discretization ofthe constant coefficient Poisson problem. From the perspective of the fine-gridpoint xh

2j+1, the variable coefficient problem can be viewed as a Poisson problemwith variable mesh sizes defined by

h2j+ 12

=γh2j+1

ah2j+ 1

2

, h2j+ 32

=γh2j+1

ah2j+ 3

2

, (7.27)

where

γh2j+1 = h

√√√√ 2ah2j+ 1

2ah2j+ 3

2

ah2j+ 1

2+ ah

2j+ 32

, (7.28)

(Exercise 20). The point that we should get from these rather cumbersome formulasis that the variable-coefficient problem on a uniform mesh can be viewed locally as aconstant-coefficient equation on a nonuniform mesh. This relationship is importantbecause it says that standard multigrid for the variable-coefficient problem corre-sponds to using interpolation based on simple averaging for the nonuniform meshproblem. If the point xh

2j+1 of the variable mesh is close to xh2j , but far from xh

2j+2,then simple averaging cannot accurately represent smooth components (Exercise21).

A simple remedy is to define interpolation for the variable-coefficient problemguided by the variable-mesh case. The easiest way is to use operator interpolation,which gives (Exercise 22)

vh2j = v2h

j , (7.29)

vh2j+1 =

ah2j+ 1

2v2h

j + ah2j+ 3

2v2h

j+1

ah2j+ 1

2+ ah

2j+ 32

, 1 ≤ j ≤ n

2− 1.

Restriction and the coarse-grid operator can then be defined by the variationalrelations

I2hh =

12

(Ih2h

)Tand A2h = I2h

h AhIh2h .

Variable-coefficient problems can be handled in two dimensions by an analogous,but more complicated, operator interpolation scheme. The principle difficulty in ex-tending the one-dimensional approach is that the two-dimensional version of (7.21)

132 Chapter 7

a(x) = 1 + ρ sin(kπx) a(x) = 1 + ρ rand(x)

ρ k = 3 k = 25 k = 50 k = 100 k = 200 k = 400

0 0.085 0.085 0.085 0.085 0.085 0.085 0.0850.25 .084 .098 .098 .094 .093 .083 0.0830.50 .093 .185 .194 .196 .195 .187 0.1730.75 .119 .374 .387 .391 .390 .388 0.3940.85 .142 .497 .510 .514 .514 .526 0.4720.95 .191 .681 .690 .694 .699 .745 0.672

Table 7.4: Average convergence factors for 20 V-cycles of the variable-coefficientmethod. With a(x) = 1 + ρ sin(kπx), there is strong dependence on ρ, but rela-tive insensitivity to all but the smallest value of k. With a(x) = 1 + ρ rand(x),convergence factors depend strongly on the amplitude ρ.

generally cannot be solved for the error corresponding to the fine-grid point in termsof neighboring coarse-grid points alone. For example, for the nine-point stencil

Ahi,j =

13h2

−1 −1 −1

−1 8 −1−1 −1 −1

(7.30)

with a fully coarsened grid (i.e., every other fine-grid line is deleted), the equation atany fine-grid point involves at least four points that do not correspond to coarse-gridpoints. This difficulty can be reduced by considering semicoarsening algorithms,but the equations at fine-grid points still involve other fine-grid points that do notbelong to the coarse grid. Nevertheless, there are effective ways to deal with thisdifficulty that involve collapsing the stencil to eliminate these troublesome fine-gridcouplings, which in turn allow approximate operator interpolation. We leave thisissue to the study of algebraic multigrid in the next chapter.

Numerical example. We illustrate the performance of a basic multigrid methodapplied to two variable-coefficient problems. We solve the one-dimensional prob-lem (7.23) using the discretization in (7.24) and coarse-grid operator (7.25). Thevariable coefficient is

a(x) = 1 + ρ sin(kπx), (7.31)

for various choices of ρ > 0 and positive integers k, and

a(x) = 1 + ρ rand(x), (7.32)

where rand(x) returns a random number between –1 and 1. Noting that a(x) = 1corresponds to the Poisson equation, these functions correspond to perturbationsof the model problem. By increasing the size of the perturbation and the amount oflocal change in the perturbation, we are able to explore the behavior of the methodfor a wide variety of situations.

The grid size in all tests is n = 1024, which we found to be representative ofthe tests done over a wide range of n . We use V(2,1)-cycles based on Gauss–Seidelrelaxation, full weighting, and linear interpolation. The results are displayed inTable 7.4.

For coefficients (7.31), the method depends strongly on the value of ρ, but verylittle on the value of the wavenumber k. For ρ = 0.25, the method works very well,

A Multigrid Tutorial 133

with results akin to the model Poisson problem. However, with increasing ρ, per-formance degrades rapidly until convergence becomes quite poor. This dependenceon ρ is to be expected: for ρ near 1, a(x) has significant variation, from values near0 to values near 1. One might expect that performance would also depend stronglyon the wavenumber k. However, apart from the noticeable jump in convergencefactors between k = 3 and k = 25, this does not appear to be the case, suggestingthat standard multigrid methods are more sensitive to the amplitude of coefficientvariations than to their frequency.

For coefficient (7.32), we drop the regular oscillatory variation in a(x) in favorof random jumps. The various amplitudes, ρ, of the coefficient match those inthe first case. Here, we again see a strong correlation between the amplitude andthe convergence factor. For small amplitudes, the method performs well, but theperformance degrades as ρ nears 1. It is interesting that the method seems relativelyunaffected by the random variations in the coefficient. It performs at least as wellwith the random coefficients as it does with all but the lowest wavenumber case(k = 3) of the smoothly varying coefficients.

These examples serve to highlight the point that a basic multigrid scheme maybe used quite effectively for the variable-coefficient problem, provided the variationin the coefficient function is not too drastic. On the other hand, widely varyingcoefficients require more sophisticated multigrid methods. One approach to suchproblems is to use an algebraic multigrid method, which is the subject of the nextchapter.

Exercises

Neumann Boundary Conditions

1. Differential nonuniqueness. Show that Neumann problem (7.1) does nothave a unique solution by showing that if u solves (7.1), then u + c is also asolution for any constant c. This means that if any particular solution exists,then another can be constructed simply by adding a constant. Now show thatadding a general constant produces all solutions of (7.1). Hint: If u and vsolve (7.1), then (u − v)′′ = 0, so u − v = ax + c for some constants a and c;now apply the boundary conditions.

2. Differential solvability. Show that Neumann problem (7.1) is solvable onlyfor the special source terms f that satisfy integral compatibility condition(7.4). Hint: Assume u solves (7.1), then integrate both sides of the equationand apply the boundary conditions.

3. Discrete null space. Show by inspecting (7.3) that the null space of Ah isthe set c1h.

4. Discrete nonuniqueness. Show that discrete system (7.2)–(7.3) does nothave a unique solution by proving that if uh is a particular solution, thenthe general solution is uh + c1h. Hint: First use Exercise 3 to confirm thatuh + c1h is indeed a solution; then prove that any two solutions must differby a vector in the null space of Ah.

134 Chapter 7

5. Discrete solvability. Show that discrete system (7.9)–(7.10) is solvable onlyfor the special source terms fh that satisfy discrete compatibility condition(7.7). Hint: Use Exercise 3, the fact that the range of a matrix is the or-thogonal complement of the null space of its transpose, and the symmetry ofAh.

6. Restriction at the boundary points. Using the interpolation operator atthe left boundary given in (7.12) and the variational condition I2h

h = 12

(Ih2h

)T ,derive the restriction formula at the left boundary point, f2h

0 ← 12fh

0 + 14fh

1 .

7. Coarse-grid solvability. Consider the coarse-grid equation

A2hu2h = I2hh

(fh − Ahuh

).

(a) Show that interpolation preserves constants; that is, 1h = Ih2h1

2h.

(b) Show that the variational property, Ih2h = c(I2h

h )T , guarantees that thediscrete compatibility condition (7.7) is satisfied (that is, the coarse-gridproblem is solvable).

(c) The variational property is actually not necessary for the discrete com-patibility condition to hold. Show that it is enough for restriction tosatisfy the property that its column sums equal a given constant γ (thismeans that a fine-grid residual is distributed to coarse-grid points withweights that sum to γ).

8. Inhomogeneous Neumann conditions. Consider the two-point boundaryvalue problem

−u′′(x) = f(x), 0 < x < 1 ,

u′(0) = g0 ,

u′(1) = g1 ,

where g0 and g1 are given constants. Modify the essential concepts developedfor (7.1) to accommodate this inhomogeneous case. In particular, show that

(a) compatibility condition (7.4) now involves a nonzero right side;

(b) the right side fh of (7.2) and fh of (7.9) change accordingly to incorpo-rate g0 and g1;

(c) the rest of the development (the uniqueness condition of (7.5) and (7.10),the discrete compatibility condition in (7.7), and the coarse-grid correc-tion process) is essentially unchanged.

Anisotropic Problems

9. Two sources of anisotropy. Show that, after discretization of the two-dimensional Poisson equation (7.14), the small parameter ε > 0 in front ofthe uyy term is equivalent to using a mesh spacing of hx = h in the x-directionand mesh spacing hy = h√

εin the y-direction.

A Multigrid Tutorial 135

10. Eigenvalues for point Jacobi method. Suppose the weighted point Ja-cobi method is applied to the system of equations corresponding to (7.15).Recall that Ah can be expressed as D −L−U , where D represents the diag-onal elements of Ah and U and L represent the respective upper and lowertriangular parts of Ah. Then the weighted Jacobi iteration matrix is given byPJ = I − ωD−1Ah.

(a) Write out a typical equation of the eigenvalue system PJv = λv.

(b) Assume an n × n grid and an eigenvector solution of the form

vij = sin(

ikπ

n

)sin

(jπ

n

), 1 ≤ k, ≤ n − 1.

Using sine addition rules, simplify this eigenvalue equation, cancel com-mon terms, and show that the eigenvalues are given by

λk = 1 − 2ω

1 + ε

(sin2

(kπ

2n

)+ ε sin2

(π

2n

)), 1 ≤ k, ≤ n.

11. Eigenvalues for block Jacobi method. Consider Ah given in (7.16) andwrite it in the form Ah = D − U − L, where D is the block diagonal matrixconsisting of the blocks D on the diagonal and U and L are the respectivelower and upper triangular parts of Ah.

(a) Show that the weighted block Jacobi iteration matrix can be written inthe form

PJ = D−1(U + L) = I − ωD−1Ah.

(b) Write out a typical equation of the eigenvalue system PJv = λv.

(c) Assume an n × n grid and an eigenvector solution of the form

vij = sin(

ikπ

n

)sin

(jπ

n

), 1 ≤ k, ≤ n − 1.

Using sine addition rules, simplify this eigenvalue equation, cancel com-mon terms, and show that the eigenvalues are given by

λk, = 1 − 2ω

2 sin2(

kπ2n

)+ ε

(sin2

(kπ

2n

)+ ε sin2

(π

2n

)), 1 ≤ k, ≤ n.

(d) Compare the magnitudes of the eigenvalues of the block Jacobi methodto those of the point Jacobi method (previous problem).

12. Semicoarsening/point relaxation complexity. Using the techniques ofChapter 4, for an n× n grid, show that with semicoarsening and point relax-ation, the computational cost of a W-cycle is larger than O(n2), but that aV-cycle retains O(n2) complexity.

136 Chapter 7

13. Semicoarsening/line relaxation complexity. Modifying the argument ofthe previous problem, show that with semicoarsening and line relaxation, thecomputational cost of a V-cycle is O(n2).

Variable-Mesh and Variable-Coefficient Problems

14. Variable-mesh discretization. Show that when model problem (7.17)is discretized using second-order finite difference approximations, differenceequations (7.18) and (7.19) result.

15. Linear interpolation for variable meshes. Show that linear interpolationon a nonuniform grid leads to the formula expressed in (7.20). Hint: Constructa linear function in the interval [x2h

j , x2hj+1] that equals v2h at the end points,

then evaluate it at xh2j+1 = x2h

j + h2j+ 12.

16. Operator interpolation for variable meshes. Show that interpolationformulas (7.20) and (7.22) are algebraically equivalent.

17. Galerkin operator for variable meshes. Show that the Galerkin principleapplied to problem (7.18)–(7.19) leads to a similar, but generally differentstencil. Show that the Galerkin principle does hold if (7.18) is multiplied onboth sides by (hj− 1

2+ hj+ 1

2)/2.

18. Variable-coefficient discretization. Verify that the discretization of thevariable-coefficient problem (7.23) results in the difference equation (7.24).

19. Galerkin operator for variable coefficients. Show that the Galerkincoarse-grid operator for (7.24), using linear interpolation and full weighting,is the same as operator (7.25)–(7.26) obtained by simple averages of the fine-grid coefficients.

20. Variable-coefficient versus variable-mesh. Verify formula (7.27)–(7.28)that relates the variable-coefficient problem on a uniform mesh to Poisson’sequation on a nonuniform mesh.

21. Simple averaging for variable meshes. To see that interpolation definedby simple averages can be ineffective for grids with widely varying mesh sizes,consider the three grid points xh

0 = x2h0 = 0, xh

1 = εh, and xh2 = x2h

1 =2h, where h and ε are small positive parameters. Note that the smoothlinear function u(x) = x

2h is 0 at xh0 and 1 at xh

2 . Show that it cannot beapproximated well at xh

1 by the average of these end point values. Obtain anexpression for the error at xh

1 in terms of h and ε and discuss the behavior asε → 0.

22. Interpolation for variable-coefficient problems. Use the operator inter-polation approach (assume the error satisfies the residual equation) to deriveinterpolation formulas (7.29).

Chapter 8

Algebraic Multigrid (AMG)

A natural question arises: Can we apply multigrid techniques when there is nogrid? Suppose we have relationships among the unknowns that are similar to thosein the model problem, but the physical locations of the unknowns are themselvesunknown (or immaterial). Can we hope to apply the tools we have developed? Arelated question is: Can we apply multigrid in the case where grid locations areknown but may be highly unstructured or irregular, making the selection of a coarsegrid problematic? These are the problems that are addressed by a technique knownas algebraic multigrid, or AMG [6]. We describe AMG using many of the conceptsand principles developed in [18]. For a theoretical foundation of general algebraicmethods, of which AMG is a part, see [5].

For any multigrid algorithm, the same fundamental components are required.There must be a sequence of grids, intergrid transfer operators, a relaxation (smooth-ing) operator, coarse-grid versions of the fine-grid operator, and a solver for thecoarsest grid.

Let us begin by deciding what we mean by a grid. Throughout this chapter,we look to standard multigrid (which we refer to as the geometric case) to guideus in defining AMG components. In the geometric case, the unknown variablesui are defined at known spatial locations (grid points) on a fine grid. We thenselect a subset of these locations as a coarse grid. As a consequence, a subset ofthe variables ui is used to represent the solution on the coarse grid. For AMG, byanalogy, we seek a subset of the variables ui to serve as the coarse-grid unknowns.A useful point of view, then, is to identify the grid points with the indices of theunknown quantities. Hence, if the problem to be solved is Au = f and

u =

u1

u2

...un

,

then the fine-grid points are just the indices 1, 2, . . . , n.Having defined the grid points, the connections within the grid are determined

by the undirected adjacency graph of the matrix A. Letting the entries of A beaij , we associate the vertices of the graph with the grid points and draw an edgebetween the ith and jth vertices if either aij = 0 or aji = 0. The connections in

137

138 Chapter 8

4

1

23

56

XXXXXXX

XXXX

XXXXXXXXX

XXXX

A =

Figure 8.1: The nonzero structure of A, where X indicates a nonzero entry, isshown on the left. The resulting undirected adjacency graph appears on the right.

the grid are the edges in the graph; hence, the grid is entirely defined by the matrixA. A simple example of this relationship is given in Fig. 8.1.

Now that we can represent the fine grid, how do we select a coarse grid?With standard multigrid methods, smooth functions are geometrically or physi-cally smooth; they have a low spatial frequency. In these cases, we assume thatrelaxation smooths the error and we select a coarse grid that represents smoothfunctions accurately. We then choose intergrid operators that accurately transfersmooth functions between grids.

With AMG, the approach is different. We first select a relaxation scheme thatallows us to determine the nature of the smooth error. Because we do not haveaccess to a physical grid, the sense of smoothness must be defined algebraically.The next step is to use this sense of smoothness to select coarse grids, which willbe subsets of the unknowns. A related issue is the choice of intergrid transferoperators that allow for effective coarsening. Finally, we select the coarse-gridversions of the operator A, so that coarse-grid correction has the same effect thatit has in geometric multigrid: it must eliminate the error components in the rangeof the interpolation operator.

Algebraic Smoothness

Having chosen a relaxation scheme, the crux of the problem is to determine whatis meant by smooth error. If the problem provides no geometric information (forexample, true grid point locations are unknown), then we cannot simply examinethe Fourier modes of the error. Instead, we must proceed by analogy. In the geo-metric case, the most important property of smooth error is that it is not effectivelyreduced by relaxation. Thus, we now define smooth error loosely to be any errorthat is not reduced effectively by relaxation.

That was simple. Of course, we still need to figure out exactly what this def-inition means. To do this in the simplest case, we focus on weighted point Jacobirelaxation. We also assume that A is a symmetric M-matrix: it is symmetric(AT = A) and positive-definite (uT Au > 0 for all u = 0) and has positive diag-onal entries and nonpositive off-diagonal entries. These properties are shared bymatrices arising from the discretization of many (not all) scalar elliptic differentialequations. These assumptions are not necessary for AMG to work. However, the

A Multigrid Tutorial 139

original theory of AMG was developed for symmetric M-matrices, and if A is farfrom being an M-matrix, it is less likely that standard AMG will be effective insolving the problem.

Recall from Chapter 2 that the weighted point Jacobi method can be expressedas

v ← v + ωD−1(f − Av),

where D is the diagonal of A. As in previous chapters, v is a computed approx-imation to the exact solution u. Remember that the error propagation for thisiteration can be written as

e ←(I − ωD−1A

)e. (8.1)

Weighted Jacobi relaxation, as we know, has the property that after makinggreat progress toward convergence, it stalls, and little improvement is made withsuccessive iterations. At this point, we define the error to be algebraically smooth.It is useful to examine the implications of algebraic smoothness. Because all ofAMG is based on this concept, the effort is worthwhile.

By our definition, algebraic smoothness means that the size of ei+1 is not sig-nificantly less than that of ei. We need to be more specific about the concept ofsize. A natural choice is to measure the error in the A-norm, which is induced bythe A-inner product. As defined in Chapter 5, we have

‖e‖A = (Ae, e)1/2.

Using this norm and (8.1), we see that an algebraically smooth error is characterizedby

‖(I − ωD−1A

)e‖A ≈ ‖e‖A.

When we assume that ω = α‖D−1/2AD−1/2‖−1 for some fixed α ∈ (0, 2) and that‖D−1/2AD−1/2‖ is O(1) (for the model problem, it is bounded by 2), it can beshown (Exercise 1) that

(D−1Ae, Ae) (e, Ae).

Writing this expression in components yields

n∑i=1

r2i

aii

n∑i=1

riei.

This implies that, at least on average, algebraically smooth error e satisfies

|ri| aii|ei|.

We write this condition loosely as

Ae ≈ 0 (8.2)

and read it as meaning that smooth error has relatively small residuals. We willappeal to this condition in the development of the AMG algorithm. While ouranalysis here is for weighted Jacobi, Gauss–Seidel relaxation is more commonly usedfor AMG. A similar, though slightly more complicated analysis can be performedfor Gauss–Seidel relaxation and also leads to condition (8.2) (Exercise 11).

140 Chapter 8

One immediate implication of (8.2) is that ri ≈ 0, so

aiiei ≈ −∑j =i

aijej ; (8.3)

that is, if e is a smooth error, then ei can be approximated well by a weightedaverage of its neighbors. This fact gives us an important foothold in determiningan interpolation operator.

A short digression here should serve to clarify the difference between algebraicand geometric smoothness. We consider a simple example due to Stuben in hisintroduction to AMG [22]. Suppose the problem

−auxx − cuyy + buxy = f(x, y), (8.4)

with homogeneous Dirichlet boundary conditions, is discretized on the unit squareusing a uniform grid and the finite-difference stencils

Dhxx =

1h2

(1 −2 1

), Dh

yy =1h2

1

−21

,

Dhxy =

12h2

−1 1

1 −2 11 −1

. (8.5)

Also suppose that the coefficients a, b, and c are locally constant, but have differentvalues in different quadrants of the domain. Specifically, let the coefficients bedefined in the square domain as shown below:

a = 1c = 1000

b = 0

a = 1c = 1b = 2

a = 1c = 1b = 0

a = 1000c = 1b = 0

Note that this discretization does not produce an M-matrix.Using a zero right side and a random initial guess, the norm of the error es-

sentially stops changing after eight sweeps of Gauss–Seidel. By our definition, thiserror is algebraically smooth. However, it does not appear to be smooth in thegeometric sense (Fig. 8.2). In fact, in three of the four quadrants it is geometricallyquite oscillatory! But because the iteration has stalled, this is precisely the errorAMG must account for in coarse-grid correction. We return to this example later.

Influence and Dependence

Most of AMG rests on two fundamental concepts. We have just discussed the firstconcept, namely, smooth error. The second important concept is that of strongdependence or strong influence. Because of the dominance of the diagonal entry (Ais an M-matrix), we associate the ith equation with the ith unknown; the job ofthe ith equation is to determine the value of ui. Of course, it usually takes all of

A Multigrid Tutorial 141

Figure 8.2: Error that is algebraically smooth, but not geometrically smooth for(8.4).

the equations to determine any given variable precisely. Nevertheless, our first taskis to determine which other variables are most important in the ith equation; thatis, which uj are most important in the ith equation in determining ui?

One answer to this question lies in the following observation: if the coefficient,aij , which multiplies uj in the ith equation, is large relative to the other coefficientsin the ith equation, then a small change in the value of uj has more effect on thevalue of ui than a small change in other variables in the ith equation. Intuitively, itseems logical that a variable whose value is instrumental in determining the valuefor ui would be a good value to use in the interpolation of ui. Hence, such a variable(point) should be a candidate for a coarse-grid point. This observation suggests thefollowing definition.

Definition 1. Given a threshold value 0 < θ ≤ 1, the variable (point)ui strongly depends on the variable (point) uj if

−aij ≥ θ maxk =i

−aik . (8.6)

This says that grid point i strongly depends on grid point j if the coefficient aij iscomparable in magnitude to the largest off-diagonal coefficient in the ith equation.We can state this definition from another perspective.

Definition 2. If the variable ui strongly depends on the variable uj ,then the variable uj strongly influences the variable ui.

With the twin concepts of smooth error and strong influence/dependence inhand, we can return to the task of defining the multigrid components for AMG.As with any multigrid algorithm, we begin by defining a two-grid algorithm, thenproceed to multigrid by recursion. Having defined the relaxation scheme, we haveseveral tasks before us:

142 Chapter 8

• select a coarse grid so that the smooth components can be represented accu-rately;

• define an interpolation operator so that the smooth components can be accu-rately transferred from the coarse grid to the fine grid; and

• define a restriction operator and a coarse-grid version of A using the varia-tional properties.

Defining the Interpolation Operator

Assume for the moment that we have already designated the coarse-grid points.This means that we have a partitioning of the indices 1, 2, . . . , n = C ∪F , wherethe variables (points) corresponding to i ∈ C are the coarse-grid variables. Thesecoarse-grid variables are also fine-grid variables; the indices i ∈ F represent thosevariables that are only fine-grid variables. Next, suppose that ei, i ∈ C, is a set ofvalues on the coarse grid representing a smooth error that must be interpolatedto the fine grid, C ∪ F . What do we know about ei that allows us to build aninterpolation operator that is accurate? With geometric multigrid, we use linearinterpolation between the coarse grid points. With an unstructured, or perhapsnonexistent, grid, the answer is not so obvious.

If a C-point j strongly influences an F -point i, then the value ej contributesheavily to the value of ei in the ith (fine-grid) equation. It seems reasonable thatthe value ej in the coarse-grid equation could therefore be used in an interpolationformula to approximate the fine-grid value ei. This idea can be strengthened bynoting that the following bound must hold for smooth error on average, that is, formost i (Exercise 2):

∑j =i

( |aij |aii

) (ei − ej

ei

)2

1, 1 ≤ i ≤ n. (8.7)

The left side of the inequality is a sum of products of nonnegative terms. Theseproducts must be very small, which means that one or both of the factors in eachproduct must be small. But if ei strongly depends on ej , we know that −aij couldbe comparable to aii. Therefore, for these strongly influencing ej ’s, it must be truethat ei − ej is small; that is, ej ≈ ei. We describe this by saying that smooth errorvaries slowly in the direction of strong connection. Thus, we have a justificationfor the idea that the fine-grid quantity ui can be interpolated from the coarse-gridquantity uj if i strongly depends on j.

For each fine-grid point i, we define Ni, the neighborhood of i, to be the set of allpoints j = i such that aij = 0. These points can be divided into three categories:

• the neighboring coarse-grid points that strongly influence i; this is the coarseinterpolatory set for i, denoted by Ci;

• the neighboring fine-grid points that strongly influence i, denoted by Dsi ; and

• the points that do not strongly influence i, denoted by Dwi ; this set may con-

tain both coarse- and fine-grid points; it is called the set of weakly connectedneighbors.

A Multigrid Tutorial 143

The goal is to define the interpolation operator Ih2h (although physical grids

may not be present, we continue to denote fine-grid quantities by h and coarse-gridquantities by 2h). We require that the ith component of Ih

2he be given by

(Ih2he

)i=

ei if i ∈ C,∑j∈Ci

ωijej if i ∈ F, (8.8)

where the interpolation weights, ωij , must now be determined.Recall that the main characteristic of smooth error is that the residual is small:

r ≈ 0. We can write the ith component of this condition as

aiiei ≈ −∑j∈Ni

aijej .

Splitting the sum into its component sums over the coarse interpolatory set, Ci,the fine-grid points with strong influence, Ds

i , and the weakly connected neighbors,Dw

i , we haveaiiei ≈ −

∑j∈Ci

aijej −∑

j∈Dsi

aijej −∑

j∈Dwi

aijej . (8.9)

To determine the ωij , we need to replace the ej in the second and third sums onthe right side of (8.9) with approximations in terms of ei or ej , where j ∈ Ci.

Consider the third sum over points that are weakly connected to point i. Wedistribute these terms to the diagonal coefficient; that is, we simply replace ej inthe rightmost sum by ei, giving

aii +∑

j∈Dwi

aij

ei ≈ −

∑j∈Ci

aijej −∑

j∈Dsi

aijej . (8.10)

We can justify this distribution in the following way. Suppose we have underesti-mated the dependence, so that ei does depend strongly on the value of the pointsin Dw

i . Then the fact that smooth error varies slowly in the direction of strongdependence means that ei ≈ ej and the distribution to the diagonal makes sense.Alternatively, suppose the value of ei does not depend strongly on the points inDw

i . Then the corresponding value if aij will be small and any error committed inmaking this assignment will be relatively insignificant.

Treating the second sum over Dsi is a bit more complicated because we must

be more careful with these strong connections. We might simply distribute theseterms to the diagonal, and, indeed, this would work nicely for many problems.However, experience has shown that it is better to distribute the terms in Ds

i toCi. Essentially, we want to approximate the ej ’s in this sum with weighted sums ofthe ek for k ∈ Ci. That is, we want to replace each ej , where j is a fine-grid pointthat strongly influences i, with a linear combination of values of ek from the coarseinterpolatory set of the point i. We do this, for each fixed j ∈ Ds

i , by making theapproximation

ej ≈

∑k∈Ci

ajkek∑k∈Ci

ajk

. (8.11)

144 Chapter 8

i-n+1

i+n+1i+n-1

i-ni-n-1

i+n

i+1i-1

i

Figure 8.3: Strong and weak influences on the point i. Coarse-grid points areshown as open circles. C-points with a strong influence on i are indicated withsolid lines, F -points that strongly influence i with dashed lines, and F -points thatweakly influence i with dotted lines.

The numerator is appropriate because the ej are strongly influenced by the ek inproportion to the matrix entries ajk. The denominator is chosen to ensure thatthe approximation interpolates constants exactly. Notice that this approximationrequires that if i and j are any two strongly connected fine-grid points, then theymust have at least one point common to their coarse interpolatory sets Ci and Cj .

If we now substitute (8.11) into (8.10) and engage in a spate of algebra (Exercise3), we find that the interpolation weights are given by

ωij = −

aij +∑

m∈Dsi

aimamj∑

k∈Ci

amk

aii +∑

n∈Dwi

ain

. (8.12)

The calculation of the interpolation weights can be illustrated with a simpleexample. Consider the operator A, defined on a uniform n × n grid, by the stencil

− 12 −2 − 1

2−1 29

4 −1− 1

8 −2 − 18

.

Assume that the partition of the grid into the C- and F -points corresponds to red-black coarsening. For a typical interior fine-grid point i, the four points directlynorth, south, east, and west are the coarse interpolatory set Ci. Using θ = 0.2 asthe dependence threshold, the strong and weak influences on i are shown in Fig.8.3.

The points northwest and northeast of i form the set Dsi (fine-grid points with

strong influence), while the points to the southeast and southwest form the set Dwi

A Multigrid Tutorial 145

(points with weak influence). For this example, (8.9) becomes

294

ei = 2ei+n + 2ei−n + ei−1 + ei+1︸ ︷︷ ︸Ci

+12ui+n−1 +

12ei+n+1︸ ︷︷ ︸

Dsi

+18ei−n−1 +

18ei−n+1︸ ︷︷ ︸

Dwi

.

Substituting ei for the Dwi points, ei−n+1 and ei−n−1, in the rightmost sum yields(

294

− 18− 1

8

)ei = 2ei+n + 2ei−n + ei−1 + ei+1︸ ︷︷ ︸

Ci points

+12ui+n−1 +

12ei+n+1︸ ︷︷ ︸

Dsi

. (8.13)

Point i depends strongly on i + n − 1, which is itself strongly dependent on pointsi − 1 and i + n in Ci. Similarly, i depends strongly on i + n + 1, which in turndepends strongly on i + 1 and i + n from the coarse interpolatory set Ci. Using(8.11), we obtain the approximations

ei+n−1 ≈ −2ei−1 − ei+n

−(2 + 1), ei+n+1 ≈ −2ei+1 − ei+n

−(2 + 1),

which we can substitute into the sum over the strong F -points in (8.13). After allthe algebra is done, (8.13) reduces to the interpolation formula

ei =721

ei+n +621

ei−n +421

ei+1 +421

ei−1.

It is worth noting that, as in many good interpolation formulas, the coefficientsare nonnegative and sum to one. A worthwhile exercise is to show that underappropriate conditions, this must be the case (Exercise 4).

Selecting the Coarse Grid

The preceding discussion of the interpolation operator assumed that we had alreadydesignated points of the coarse grid. We must now turn our attention to this criticaltask. We use the twin concepts of strong influence/dependence and smooth error,just as we did in defining interpolation. As in the geometric problem, we rely onthe fundamental premise that the coarse grid must be one

• on which smooth error can be approximated accurately,

• from which smooth functions can be interpolated accurately, and

• that has substantially fewer points than the fine grid, so that the residualproblem may be solved with relatively little expense.

146 Chapter 8

The basic idea is straightforward. By examining the suitability of each gridpoint to be a point of one of the Ci sets, we make an initial partitioning of the gridpoints into C- and F -points. Then, as the interpolation operator is constructed, wemake adjustments to this partitioning, changing points initially chosen as F -pointsto be C-points in order to ensure that the partitioning conforms to certain heuristicrules.

Before we can describe the coarsening process in detail, we need to make twomore definitions and to introduce these heuristics. Denote by Si the set of pointsthat strongly influence i; that is, the points on which the point i strongly depends.Also denote by ST

i the set of points that strongly depend on the point i. Armed withthese definitions, we describe two heuristic criteria that guide the initial selectionof the C-points:

H-1: For each F -point i, every point j ∈ Si that strongly influences i either shouldbe in the coarse interpolatory set Ci or should strongly depend on at leastone point in Ci.

H-2: The set of coarse points C should be a maximal subset of all points with theproperty that no C-point strongly depends on another C-point.

To motivate heuristic H-1, we examine the approximation (8.11) that was madein developing the interpolation formula. Recall that this approximation applies topoints j ∈ Ds

i that consist of F -points strongly influencing the F -point i. Becauseei depends on these points, their values must be represented in the interpolationformula in order to achieve accurate interpolation. But because they have notbeen chosen as C-points, they are represented in the interpolation formula only bydistributing their values to points in Ci using (8.11). It seems evident that (8.11)will be more accurate if j is strongly dependent on several points in Ci. However,for the approximation to be made at all, j must be strongly dependent on at leastone point in Ci. Heuristic H-1 simply ensures that this occurs.

Heuristic H-2 is designed to strike a balance on the size of the coarse grid.Multigrid efficiency is generally controlled by two properties: convergence factorand number of WUs per cycle. If the coarse grid is a large fraction of the totalpoints, then the interpolation of smooth errors is likely to be very accurate, which,in turn, generally produces better convergence factors. However, relatively largecoarse grids generally mean a prohibitively large amount of work in doing V-cycles.By requiring that no C-point strongly depends on another, H-2 controls the sizeof the coarse grid because C-points tend to be farther apart. By requiring C to bea maximal subset (that is, no other point can be added to C without violating theban on mutual strong dependence), H-2 ensures that C is big enough to producegood convergence factors.

It is not always possible to enforce both H-1 and H-2 (see Exercise 5). Becausethe interpolation formula depends on H-1 being satisfied, we choose to enforce H-1rigorously, while using H-2 as a guide. While this choice may lead to larger coarsegrids than necessary, experience shows that this trade-off between accuracy andexpense is generally worthwhile.

The basic coarse-point selection algorithm proceeds in two passes. We firstmake an initial coloring of the grid points by choosing a preliminary partition intoC- and F -points. The goal in the first pass is to create a set of C-points thathave good approximation properties and also tend to satisfy H-2. Once the initial

A Multigrid Tutorial 147

assignments have been made, we make a second pass, changing initial F -points toC-points as necessary to enforce H-1.

The Coloring Scheme

The first pass begins by assigning to each point i a measure of its potential qualityas a C-point. There are several ways we can make this assessment, but the simplestis to count the number of other points strongly influenced by i. Because those pointsare the members of ST

i , this count, λi, is the cardinality of STi . Once the measures

λi have been determined, we select a point with maximum λi value as the firstpoint in C.

The point we just selected strongly influences several of the other points andshould appear in the interpolation formula for each of them. This implies that thepoints that depend strongly on i should become F -points. We therefore assign allpoints in ST

i to F , which is permissible because we already have a C-point, i, thatstrongly influences them. It is logical to look at other points that strongly influencethese new F -points as potential C-points, because their values could be useful foraccurate interpolations. Therefore, for each new F -point j in ST

i , we increment themeasure, λk, of each unassigned point k that strongly influences j; this would beeach unassigned member of k ∈ Sj .

The process is then repeated. A new unassigned point i is found with maximumλi and it is assigned to C. The unassigned points j ∈ ST

i are then assigned to Fand the measures of the unassigned points in Sj are incremented by 1. This processcontinues until all points have been assigned to C or F .

It is useful to observe that the coarsening determined by this method dependson several factors. Among the most influential is the order in which the grid pointsare scanned when seeking the next point with maximal λ. Because many, if notmost, of the grid points will have the maximal value at the start, any of themcould be selected as the first coarse point. Once the first point is selected, the restproceeds as outlined. Again, any time there is more than one point having themaximal value, there are many possible coarsenings. The heuristics ensure thatwhatever specific coarse grid is obtained, it will have the desired properties: itprovides a good representation of smooth error components, while keeping the sizeof the coarse grid reasonably small.

This coloring algorithm is best illustrated by an example. The upper left draw-ing in Fig. 8.4 shows the graph of the matrix for a nine-point stencil representingthe Laplacian operator on a uniform grid. The operator stencil is

1h2

−1 −1 −1

−1 8 −1−1 −1 −1

. (8.14)

For this example, the dependence threshold is immaterial; for any θ, every con-nection is one of strong dependence. Hence, each point strongly influences, anddepends strongly upon, each of its neighbors. Initially, all interior points have ameasure of λ = 8, all points along the side of the grid have a measure of λ = 5,and the four corner points of the grid have a measure of λ = 3. We also assumethe points are stored in lexicographic order.

148 Chapter 8

Figure 8.4: Sequence of coloring steps for the nine-point Laplacian on a uniformgrid. The upper left diagram is the original grid, the lower right the final coloring.

The remaining drawings in Fig. 8.4 show the coloring of the grid as it evolves.At each step, newly designated C-points are shown in black; newly designated F -points are shown in white with a heavy border; and “undecided” neighbors of thenew F -points whose λ values are updated are shaded in gray. Edges of the graphare removed as the algorithm accounts for the dependencies of the new C- andF -points. The lower right drawing of the figure displays the completed coloring.

A Multigrid Tutorial 149

It is useful to observe that for this example, coarse-grid selection is completeafter this pass. A careful examination of the lower right drawing in the figurereveals that, because of the high connectivity of the graph, both H-1 and H-2 aresatisfied by the C/F coloring produced. In addition, the coarse grid produced isexactly the standard full coarsening that one would use for a geometric multigridmethod for this problem.

It is also instructive to examine the application of this coarsening to the five-point Laplacian stencil on a uniform grid. Once again, the first pass of the processnot only produces a coloring satisfying both heuristics, but one that is very commonin geometric solutions to the problem as well (Exercise 6).

It is not difficult to concoct an example that does not work so well. One exampleis illustrated in Fig. 8.5. Again, the nine-point Laplacian stencil (8.14) is used on ann×n uniform grid, but now we add the periodic boundary conditions ui,j = ui±n,j±n

with n = 7. Note that the initial measure is λ = 8 for all grid points. The thin linesindicate the extent of the grid with the first periodic replication shown outside thelines. At each step, newly designated C-points are shown in black; newly designatedF -points are shown in white with a heavy border; and “undecided” neighbors ofthe new F -points whose λ values are updated are shaded in gray. Edges of thegraph are removed as the algorithm accounts for the dependencies of the new C-and F -points. The final first-pass coloring (lower right of figure) violates H-1, witha large number of F -F dependencies between points not sharing a C-point.

This is an example of a problem for which it is impossible to satisfy both H-1and H-2. Because the coloring algorithm satisfies H-2 and we have determined thatwe must satisfy H-1, a second pass for the coarsening algorithm must be done. Inthis pass, we examine each of the F -points in turn and determine if there are F -Fdependencies with points not depending strongly on a common C-point. If this isthe case, then one of the two F -points is tentatively changed into a C-point, andthe examination moves on to the next F -point. When further F -F dependenciesare encountered, we first attempt to satisfy H-1 using points on the list of tentativepoints. The idea is to satisfy H-1 by converting a minimal number of F -points intoC-points.

Figure 8.6 displays the coarsening produced for the periodic nine-point Lapla-cian after the second pass of the coloring algorithm. The extra C-points are shadedblack with a rim of dark gray. It can be seen that while the coloring now satisfiesH-1, it no longer satisfies H-2. The coarse grid has a few more points, but theinterpolation formula can be built for each F -point. The details of the second-passalgorithm are left as an exercise (Exercise 7).

Two examples highlight some important features of the coarsening algorithm.First, consider the problem

−uxx − uyy = f(x, y), (8.15)

with homogeneous Dirichlet boundary conditions. Suppose (8.15) is discretized byfinite elements using regular quadrilaterals of dimension hx × hy. If hy/hx → 0,then the finite element stencil approaches

−1 −4 −12 8 2

−1 −4 −1

.

While this stencil does not correspond to an M-matrix, the AMG coarsening algo-

150 Chapter 8

Figure 8.5: Sequence of coloring steps for the nine-point Laplacian on a uniformgrid with periodic boundary conditions. The upper left diagram is the original grid,the lower center the final coloring (first coloring pass). The gray circles in the lowerright diagram are points that violate heuristic H-1.

rithm performs quite well on this problem and is very instructive. Note that theproblem is essentially equivalent to the discretization of

−εuxx − uyy = f,

A Multigrid Tutorial 151

Figure 8.6: Second coloring pass for the nine-point Laplacian problem with periodicboundary conditions. The added C-points are shown in the final coloring as blackdots with thick gray outlines.

Figure 8.7: Semi-coarsening produced for a stretched mesh (or anisotropic) problem.Shown on the left is the grid with only the strong dependencies indicated. On theright, the final coarsening is displayed.

where ε is very small. The important feature is that the problem shows strongdependence in the y-direction, and little or no strong dependence in the x-direction.There are several subtle nuances associated with this discretization; our interesthere, however, is in the coarse-grid selection.

The coloring produced for this problem is shown in Fig. 8.7. The algorithmgenerates a semicoarsened grid that is coarsened only in the y-direction. Again,this is precisely the coarse grid that would be used by a geometric semicoarseningmethod for this problem. Perhaps the most important observation is that thegrid has been coarsened only in the direction of strong dependence. This makessense: because smooth error varies slowly in the direction of strong dependence,interpolation can be performed accurately in that direction. AMG coarsening mustexhibit this critical property.

Consider the nine-point Laplacian (Fig. 8.4) and the five-point Laplacian (Exer-cise 6). The coarsening for the five-point operator is called the red-black coarsening.

152 Chapter 8

Figure 8.8: Coarsening of problem (8.4) is shown on the left. Observe how thecoarsening is in the direction of strong dependence in each of the four quadrants ofthe grid (right).

The operator exhibits dependence in both the x- and y-directions, and the red-blackcoarsening is the only pattern that coarsens each grid line in both the x- and y-directions. The nine-point operator exhibits dependence in all directions: x, y, anddiagonal directions. The resulting coarsening, known as full coarsening, coarsens ineach of these directions. It is not possible to coarsen along x, y, and the diagonalgrid line simultaneously; the algorithm instead coarsens every other line in eachdirection, with the intervening lines not represented on the coarse grid. The strongconnectivity of this operator ensures that there are sufficient points available tointerpolate the intervening lines well.

That AMG automatically coarsens in the directions of dependence is madeapparent by the coarsening produced for the problem given in (8.4). The coarseningis shown in Fig. 8.8. Observe that in the upper left quadrant, where a = 1, c = 1000,and b = 0, the strong dependence is in the y-direction, and, in that region, the gridis coarsened primarily in the y-direction. Similarly, in the lower right quadrantwhere the coefficients are a = 1000, c = 1, and b = 0, the strong dependenceis in the x-direction, and the grid is coarsened in the x-direction. In the lowerleft quadrant, the problem is just the normal Laplacian, and the coarsening is thestandard full coarsening we observed for the nine-point Laplacian operator. Finally,in the upper right quadrant, where a = 1, c = 1, and b = 2, the coarsening is in thenorthwest/southeast direction, which is the direction of strong dependence for thestencil given in (8.5).

Coarse-Grid Operators

Recall that although physical grids may not be present, we continue to denotefine-grid quantities by h and coarse-grid quantities by 2h. Once the coarse grid ischosen and the interpolation operator Ih

2h is constructed, the restriction operator

A Multigrid Tutorial 153

I2hh is defined using the usual variational property

I2hh =

(Ih2h

)T.

The coarse-grid operator is constructed using the Galerkin condition

A2h = I2hh AhIh

2h. (8.16)

The reason for defining interpolation and the coarse operator by these variationalprinciple is that the resulting coarse-grid correction is optimal in the Ah-norm(Exercise 8).

Cycling Algorithms

We have now defined all the components necessary to create a two-grid correctionalgorithm for AMG: a relaxation scheme, a set of coarse-grid points C, a coarse-grid operator A2h, and intergrid transfer operators I2h

h and Ih2h. Although we

have discussed weighted Jacobi, Gauss–Seidel relaxation is often preferred. Thetwo-grid correction algorithm appears exactly as it did for geometric multigrid, asshown below.

AMG Two-Grid Correction Cycle

vh ← AMG(vh, fh).

• Relax ν1 times on Ahuh = fh with initial guess vh.

• Compute the fine-grid residual rh = fh − Ahvh and restrict it to the coarsegrid by r2h = I2h

h rh.

• Solve A2he2h = r2h on Ω2h.

• Interpolate the coarse-grid error to the fine grid by eh = Ih2he

2h and correctthe fine-grid approximation by vh ← vh + eh.

• Relax ν2 times on Ahuh = fh with initial guess vh.

Having defined the two-grid correction algorithm, we can define other multigridcycling schemes for AMG guided by geometric multigrid. For example, to create aV-cycle algorithm, we simply replace the direct solution of the coarse-grid problemwith a recursive call to AMG on all grids except the coarsest grid, where we usea direct solver. W-cycles, µ-cycles, and FMG-cycles can also be created by strictanalogy to the geometric multigrid case.

Costs of AMG

How expensive is the AMG algorithm? The geometric case has regular and pre-dictable costs, both in terms of the storage and floating-point operation counts. Bycontrast, there are no predictive cost analyses available for AMG. This is because wedo not know, in advance, the ratio of coarse- to fine-grid points. Furthermore, thisratio is unlikely to remain constant as the grid hierarchy is established. However, we

154 Chapter 8

have two simple tools that provide a posteriori cost estimates. Such estimates areuseful in analyzing the performance of AMG and in building confidence that AMGis an effective tool for many problems. Additionally, they are extremely useful indebugging, code tuning, and algorithm development.

Definitions. Grid complexity is the total number of grid points, on allgrids, divided by the number of grid points on the finest grid. Operatorcomplexity is the total number of nonzero entries, in all matrices Akh,divided by the number of nonzero entries in the fine-grid operator Ah.

Thus, in the geometric case on the model problem, we have grid complexities ofabout 2, 4

3 , and 87 for the one-, two-, and three-dimensional problems, respectively.

Grid complexity gives an accurate measure of the storage required for the right sidesand approximation vectors and can be directly compared to geometric multigrid.

Operator complexity indicates the total storage space required by the operatorsAkh over all grids, which is generally not necessary in the geometric case. But italso has another use. Just as in the geometric case, the work in the solve phaseof AMG is dominated by relaxation and residual computations, which are directlyproportional to the number of nonzero entries in the operator. Hence, the workof a V-cycle turns out to be essentially proportional to the operator complexity.This proportionality is not perfect, but operator complexity is generally considereda good indication of the expense of the AMG V-cycle.

Performance

By its very nature, AMG is designed to apply to a broad range of problems, manyfar removed from the partial differential equations for which geometric multigridwas developed. The performance of AMG will naturally depend in large part onthe peculiarities of the problem to which it is applied. Hence, we cannot hope togive a complete discussion of AMG performance. Accordingly, we close this chapterby discussing two numerical PDE examples.

Numerical example. Because it is helpful to illustrate the similarities and dif-ferences between the geometric and algebraic approaches, the first example is thatof Chapter 4:

−uxx − uyy = 2[(1 − 6x2)y2(1 − y2) + (1 − 6y2)x2(1 − x2)] in Ω,u = 0 on ∂Ω,

(8.17)

where Ω is the unit square. Finite differences are applied on a uniform grid, withn = 16, 32, and 64 grid lines in each coordinate direction. We use a V(1,1)-cyclewith Gauss–Seidel relaxation, sweeping first over the C-points, then over the F -points. This C − F relaxation is the AMG analogue of a red-black Gauss–Seidelsweep. The results of the three experiments are shown in Table 8.1. Note thatwe use the standard (unscaled) Euclidean norm because we do not presume anygeometric structure. In other words, we do not necessarily have the concept ofmesh size. Also, we measure the residual only because the error is not ordinarilyavailable.

In reality, AMG is unnecessary for this problem because it is precisely the casefor which the geometric case was designed; nonetheless, the analysis of the perfor-mance of AMG on this problem is illuminating. We observe first that AMG exhibits

A Multigrid Tutorial 155

V- Ratio of Ratio of Ratio ofcycle N ||r||2 ||r||2 N ||r||2 ||r||2 N ||r||2 ||r||2

0 16 1.73e + 01 – 32 3.49e + 01 – 64 7.01e+01 –1 16 6.29e − 01 0.03 32 2.52e + 00 0.07 64 6.88e + 00 0.102 16 1.44e − 02 0.02 32 9.63e − 02 0.03 64 2.92e − 01 0.043 16 3.56e − 04 0.02 32 3.84e − 03 0.03 64 1.28e − 02 0.044 16 9.52e − 06 0.03 32 1.54e − 04 0.04 64 5.68e − 04 0.045 16 2.55e − 07 0.03 32 6.24e − 06 0.04 64 2.55e − 05 0.046 16 6.87e − 09 0.03 32 2.52e − 07 0.04 64 1.16e − 06 0.057 16 1.85e − 10 0.03 32 1.02e − 08 0.04 64 5.38e − 08 0.058 16 4.95e − 12 0.03 32 4.12e − 10 0.04 64 2.52e − 09 0.059 16 1.37e − 13 0.03 32 1.66e − 11 0.04 64 1.19e − 10 0.0510 16 5.44e − 14 0.46 32 7.29e − 13 0.04 64 6.34e − 12 0.0511 16 4.36e − 14 0.80 32 3.37e − 13 0.46 64 2.58e − 12 0.4112 32 3.17e − 13 0.93 64 2.69e − 12 1.04

Table 8.1: The table shows the results of AMG V-cycles applied to boundary valueproblem (8.17). The 2-norm (Euclidean norm) of the residual after each V-cycle andthe ratio of the residual norms on successive V-cycles are tabulated for n = 16, 32,and 64.

Number Number of Density Average entriesAph of rows nonzeros (% full) per rowAh 4096 20224 0.001 4.9A2h 2048 17922 0.004 8.8A4h 542 4798 0.016 8.9A8h 145 1241 0.059 8.6A16h 38 316 0.219 8.3A32h 12 90 0.625 7.5A64h 5 23 0.920 4.6

Table 8.2: Properties of Aph, p = 2k, k = 0, 1, . . . , 6, for AMG applied to a two-dimensional problem.

the same type of convergence that was observed in Chapter 4 with geometric multi-grid. The residual norm decreases by a relatively constant factor with each V-cycle.This continues until it levels off after about 12 V-cycles near 10−13, where round-offerror is on the order of the residual norm itself. Although we do not show the er-ror, we would find that, as in the geometric case, the level of discretization error isreached after about six V-cycles. The errors would also exhibit the same reductionby a factor of four with each doubling of the resolution, as in the geometric case.

In terms of solver performance, AMG appears to be equivalent to geometricmultigrid for this problem. However, there are other factors to be considered inexamining AMG. These include the time required to do the setup and the storagerequired by the method.

We first examine the storage requirements. For the above experiment with n =64, the operators Aph, where p = 2k on levels k = 0, 1, . . . , 6, have the propertiesshown in Table 8.2. Several observations are in order here. First, the initial coarsegrid, Ω2h, has 2048 points, exactly half the number on the finest grid. This occurs

156 Chapter 8

because the five-point Laplacian operator yields the red-black coarsening describedearlier (and in Exercise 6). However, each succeeding coarse grid has approximatelyone-fourth the number of points as the next finer grid. This may be understood byobserving that the average number of nonzeros per row, which is 4.9 on the fine gridwith the five-point operator (boundary points account for the average being below5), increases to 8.8 on Ω2h and remains above 8 for the next few grids. Evidently,the Galerkin coarse grid operators have become, effectively, nine-point Laplacianoperators! The nine-point Laplacian operator yields full coarsening, in which eachsucceeding grid has one-fourth the number of points as the next finer grid. It isilluminating (Exercise 9) to examine the interpolation stencils and the formation ofthe coarse-grid operators to discover why the five-point operator becomes a nine-point operator on the coarse grid.

Summing the number of rows of all operators and dividing by the number ofrows in the fine grid matrix (4096) shows that the grid complexity for this problemis 1.68. Thus, we know that storage of the vector of unknowns and the right sidesrequire 1.68 times the space required for the fine-grid quantities. By contrast,the geometric approach has a grid complexity of about 4

3 . The difference can beexplained by the fact that the first coarse grid produced by AMG is the red-blackgrid, while the geometric approach does full coarsening on all grids.

Summing the number of nonzeros in all operators and dividing by the numberof nonzeros in the fine-grid operator shows that the operator complexity is 2.205.Thus, the matrices on all levels require just over twice the storage of the originaloperator Ah. In many cases, operators do not need to be stored in the geometricapproach, so there is no explicit operator complexity to be considered. On theother hand, operator complexity also reflects the cost of one relaxation sweep onall grids, so a V(1,1)-cycle of AMG on this problem costs about 4.4 WUs (2.2 WUson the descent and the ascent). This figure should be compared to 8

3 WUs in thegeometric case (neglecting the cost of intergrid transfers, as before).

Turning to the cost of performing the setup phase, we noted earlier that itis difficult to predict or even account for the amount of arithmetic involved. Wecan, however, make an observation about the “wall clock time” taken by the setupphase. In the n = 64 example, the setup phase required 0.27 seconds, while thesolution phase required 0.038 seconds per V-cycle. The setup phase, then, requiredapproximately the same amount of time as seven V-cycles.

These experiments show that AMG performs quite well on a model problem, butthat it is more expensive in terms of storage, computation, and time than geometricmultigrid. However, AMG is not intended for use on nicely structured problems; itis intended for problems on which geometric multigrid cannot be applied. We closeour discussion with such an example.

Numerical example. Consider the problem

−∇ · (a(x, y)∇u) = f(x, y) (8.18)

in a domain Ω with u = 0 on ∂Ω. Specifically, let Ω = ([0, 10] × [0, 5.5])/B, whereB is the disk of radius 1

2 centered on (1.25, 2.75) shown in Fig. 8.9. The problem isdiscretized on the unstructured triangulation shown in Fig. 8.10. The discontinuous

A Multigrid Tutorial 157

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

Figure 8.9: The computational domain for (8.18).

Figure 8.10: The discretization mesh for (8.18).

diffusion coefficient is given by

a(x, y) =

0 if√

(x − 1.25)2 + (y − 2.75)2 < 12 (interior of hole)

1 if 12 ≤

√(x − 1.25)2 + (y − 2.75)2 < 2

1000 if 2 ≤√

(x − 1.25)2 + (y − 2.75)2 < 6

1 else.

If (x, y) is between the circles, centered about the point (1.25, 2.75), of radius 2

158 Chapter 8

Number Number of Density Average entriesLevel of rows nonzeros (% full) per row

0 4192 28832 0.002 6.91 2237 29617 0.006 13.22 867 14953 0.020 17.23 369 7345 0.054 19.94 152 3144 0.136 20.75 69 1129 0.237 16.46 30 322 0.358 10.77 20 156 0.390 7.88 18 125 0.383 6.99 3 9 1.000 3

Table 8.3: Statistics on coarsening for AMG applied to (8.18).

Ratio ofV-cycle ||r||2 ||r||2

0 1.00e + 00 –1 4.84e − 02 0.052 7.59e − 03 0.163 1.72e − 03 0.234 4.60e − 04 0.275 1.32e − 04 0.296 3.95e − 05 0.307 1.19e − 05 0.308 3.66e − 06 0.319 1.12e − 06 0.3110 3.43e − 07 0.3111 1.05e − 07 0.3112 3.22e − 08 0.31

Table 8.4: Convergence of AMG V-cycles applied to (8.18) in terms of the 2-norm(Euclidean norm) of the residual.

and 6, then the diffusion coefficient is three orders of magnitude larger than it iselsewhere.

This problem has features that would challenge a geometric multigrid algorithm.First, and most important, it is discretized on an unstructured grid. It is not at allobvious how to generate a sequence of coarse grids for this problem. In addition,the discontinuous coefficient causes real difficulty. Since the jump follows circularpatterns and is not grid aligned, it is not possible to overcome this problem eitherby semicoarsening or by line relaxation.

To understand how AMG performs on this problem, we first examine the coars-ening statistics in Table 8.3. One interesting phenomenon is that the relative den-sity of the operators (percentage of nonzeros) increases as the grids become coarser.This increasing density can be seen in the last column, where the average numberof entries per row increases through several coarse levels. Fortunately, the operatorcomplexity is not adversely affected: a short calculation shows that the operatorcomplexity is 2.97, while the grid complexity is 1.89. It is also interesting to notethat the first coarse-grid operator has more actual nonzero coefficients than doesthe fine-grid operator. This is relatively common for AMG on unstructured grids.

A Multigrid Tutorial 159

The convergence of the method is summarized in Table 8.4. Using the standard2-norm, we see that after 11 V-cycles, the residual norm reached 10−7 and theprocess attained an asymptotic convergence factor of 0.31 per V-cycle. Clearly,AMG does not converge as rapidly for this problem as for the previous modelproblem, where we saw a convergence factor of about 0.1. Nevertheless, in lightof the unstructured grid and the strong, non-aligned jump discontinuities in thecoefficients, the convergence properties are quite acceptable.

Exercises

1. Smooth error. Under the assumptions that ω = ‖D−1/2AD−1/2‖−1 and‖D−1/2AD−1/2‖ is O(1), show that an implication of the smooth error con-dition, ‖

(I − ωD−1A

)e‖A ≈ ‖e‖A, is that

(D−1Ae, Ae) (e, Ae).

Show that this property can be expressed as

||r||D−1 ||e||A.

2. Smooth error. Show that for a symmetric M-matrix A, smooth error variesslowly in the direction of strong influence as follows. Assume ‖r‖D−1 ‖e‖A.

(a) Show that ||e||A << ||e||D. Hint: ||e||2A = (Ae, e) = (D−1/2Ae, D1/2e) ≤||r||D−1 ||e||D.

(b) Show that if (as is often true)∑

j =i |aij | ≈ aii, then∑i

∑j =i

|aij |(ei − ej)2 <<∑

i

aiie2i .

Hint: Show that (Ae, e) = 12

∑ij(−aij)(ei − ej)2 +

∑ij aije

2i .

(c) Conclude that (8.7) must hold on average, that is, for most i.

3. Deriving interpolation weights. Carry out the calculation that leads toexpression (8.12) for the interpolation weights.

4. Properties of interpolation weights. Let A be an M-matrix whose rowssum to zero for interior points, and let the ith row represent an interior point.Show that substituting ej = ei in the sum over Ds

i in (8.9) and in the sumover Dw

i in (8.11) produces interpolation coefficients ωij that are nonnegativeand sum to unity.

5. Enforcing the heuristics. Construct a simple one-dimensional example inwhich the two heuristics H-1 and H-2 cannot be simultaneously enforced.Hint: Use periodic boundary conditions and odd n.

6. Coarse-grid selection. Show that the first pass of the coarsening algorithm(the initial coloring phase) applied to the five-point Laplacian stencil

1h2

−1

−1 4 −1−1

produces standard red-black coarsening (the C-points are the red squares ona checkerboard; the F -points are the black squares).

160 Chapter 8

7. Second coloring pass. The central idea behind the second-pass coloringalgorithm is to test each F -point, i, in turn, to ensure that each point in Ds

i

depends strongly on at least one point in Ci. When an F -point, i, is found todepend strongly on another F -point, j, that does not depend strongly on apoint in Ci, then j is made (tentatively) into a C-point; testing of the pointsin Ds

i then begins again. If all those points now depend strongly on pointsin Ci, then j is put permanently in C. However, if some other point in Ds

i isfound that does not depend strongly on a point in Ci, then i itself is placed inC and j is removed from the tentative C-point list. The process is repeatedfor the next F -point and continues until all F -points have been treated.

Apply this second-pass algorithm to the nine-point Laplacian operator withperiodic boundaries (Figure 8.5) and determine the final coarsening. Observethat the final coarsening depends on the order in which the F -points areexamined.

8. Using variational properties. Show that using the variational propertiesto define interpolation and the coarse-grid operator is optimal in the followingway. The two-grid correction scheme, given in the text, corrects the fine-gridapproximation vh by a coarse-grid interpolant Ih

2hv2h that gives the least

error in the sense that

‖eh − Ih2hv

2h‖Ah = minw2h

‖eh − Ih2hw

2h‖Ah ,

where eh = uh − vh is the error in vh.

9. Coarsening the five-point operator. Examine the five-point Laplacianoperator on a uniform two-dimensional grid, and determine why the Galerkinprocess alters it to the nine-point operator on the coarse grid. Hint: This canbe done symbolically by examining which entries of the stencil will be usedto interpolate the various points and then by symbolically carrying out theGalerkin multiplication A2h = I2h

h AhIh2h.

10. Smooth error implications. Show that if A = D−L−U is a symmetric M-matrix and Q = D+L (Gauss–Seidel) or Q = D (Jacobi), then the quantities(Q−1Ae, r) and (e, r) are nonnegative.

11. Smooth error. Show that if A = D − L − U is a symmetric M-matrix andGauss–Seidel produces errors that satisfy∥∥(

I − (D + L)−1A)e∥∥

A≈ ‖e‖A,

then||r||D−1 ||e||A.

Hint: Show first that ‖(D + L)−1Ae‖A ‖e‖A. Then show and use the factthat (D + LT )−1A(D + L)−1 ≤ D−1.

12. V-cycle costs. Let the grid complexity be denoted σΩ and the operatorcomplexity be denoted σA. In addition, define the following quantities: κA,the average number of nonzero entries per row over all levels; κI , the averagenumber of interpolation points per F -point; nA

m, the number of nonzero entriesin Am; and nC

m and nFm, the number of C- and F -points, respectively, on grid

Ωm.

A Multigrid Tutorial 161

(a) Show that the number of floating-point operations on level m for onerelaxation sweep, residual transfer, and interpolation are 2nA

m, 2nAm +

2κInFm, and nC

m + 2κInFm, respectively.

(b) Noting that ∑m

nFm ≈ n,

show that the total flop count for a V(ν1, ν2)-cycle is given approximatelyby

n(2(ν + 1)κAσΩ + 4κI + σΩ − 1),

where ν = ν1 + ν2.

Chapter 9

Multilevel Adaptive Methods

Numerical problems often exhibit special features in small local regions that requireresolution and accuracy well beyond what is required in the rest of the domain. Innumerical weather modeling, isolated phenomena such as tornados or storm frontsmay demand substantially enhanced accuracy. The numerical simulation of theflight of an aircraft may require especially accurate approximations around thefuselage or wings. In these and many other cases, it is wasteful to let local accuracyrequirements dictate the global discretization and solution process. The goal ofthis chapter is to understand how to treat local demands in a multilevel contextwithout overburdening the overall computation.

There is a wide variety of methods that effectively treat local demands. Herewe consider the so-called fast adaptive composite grid method (FAC) [14]. Its dis-tinctive features are that it always works with uniform grids and subgrids, and itis in tune with the variational theme that appears throughout this book. We beginwith a one-dimensional example.

Consider the two-point boundary value problem

−u′′(x) = f(x), 0 < x < 1 ,

u(0) = u(1) = 0.

To keep matters simple, assume that a local fine grid with mesh size h = 18 is needed

on the interval (12 , 1) to resolve some special feature near x = 3

4 , but that a meshsize of 2h = 1

4 is deemed adequate for the rest of the domain. This need mightarise, for example, in the presence of a source term f that is nonzero only nearx = 3

4 (Fig. 9.1).Imagine first that we solve the problem on a global grid, Ωh, with mesh size

h = 18 , using a two-grid method based on linear interpolation, full weighting, and the

variational properties as described in Chapter 5. We write the discrete problem as

Ahuh = fh . (9.1)

In component form, we have

−uhi−1 + 2uh

i − uhi+1

h2= fh

i , 1 ≤ i ≤ n ,

uh0 = uh

n+1 = 0,

where n = 7 for this case.

163

164 Chapter 9

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.5

1

1.5

Figure 9.1: Graph of a sharply peaked impulse function centered at x = 34 .

This simple uniform grid involves unnecessary computation: the accuracy re-quired locally at x = 3

4 dictates the global mesh size h = 18 , forcing us to solve the

problem with too many grid points. To reduce this waste, we can start by elimi-nating relaxation (the most expensive multigrid process) at fine-grid points whereit is not needed, that is, in [0, 1

2 ]. However, we can do better. As we will see, it ispossible to reduce costs further by selectively eliminating interpolation, restriction,and the computation of residuals.

Consider Fig. 9.2, which shows the fine grid Ωh with n = 7 interior points andthe underlying coarse grid Ω2h with n−1

2 = 3 interior points. Regular multigridrelaxes on all fine-grid points, denoted by ∗, •, and , then performs coarse-gridcorrection based on the points denoted by ×.

With this notation, let us see how we might eliminate points of Ωh in [0, 12 ).

Restricting relaxation to the local fine grid in (12 , 1) means that the Ωh residuals

change at ∗ and • points, because the approximation vh changes at ∗ points. How-ever, fine-grid residuals do not change at points between successive coarse-gridsolves. Similarly, the coarse-grid residual does not change at × = 1

4 because it isobtained by restriction from points.

For the moment, suppose that we continue to store vh at points, just forconvenience. We must be sure that the algorithm starts with the correct rightsides, f2h

i , on the coarse grid, particularly at the point × = 14 . We begin with a

zero initial guess, vh = 0, so the residual at the coarse-grid point × = 14 is just the

restriction of the fine-grid source term fhi . With this observation, we can give our

first scheme:

• Initialize vh = 0 and f2h1 ≡

(I2hh fh

)1

= fh1 +2fh

2 +fh3

4 .

• Relax on vh on the local fine grid xh5 , xh

6 , xh7 = 5

8 , 34 , 7

8.

A Multigrid Tutorial 165

Ω2h x2h0 x2h

1 x2h2 x2h

3 x2h4

Ω2h × × ×

x = 0 18

14

38

12

58

34

78 1

Ωh • ∗ ∗ ∗

Ωh xh0 xh

1 xh2 xh

3 xh4 xh

5 xh6 xh

7 xh8

Figure 9.2: Local fine-grid points on Ωh are denoted by ∗ and the interface pointis denoted by (•). Computation at the remaining fine-grid points () is eliminated.The coarse-grid points (×) comprise Ω2h, while represents boundary points.

• Compute fine-grid residual rh = fh −Ahvh and transfer it to the coarse grid:

f2h2 ← rh

3 + 2rh4 + rh

5

4, f2h

3 ← rh5 + 2rh

6 + rh7

4.

• Compute an approximation v2h to the coarse-grid residual equation A2hu2h =f2h.

• Update the residual at × = 14 for use in later cycles: f2h

1 ← f2h1 −−v2h

0 +2v2h1 −v2h

2(2h)2 .

• Interpolate the correction and update the approximation: vh ← vh + Ih2hv

2h.

At this point, reducing the number of computations further becomes a littletricky. It might seem at first glance that the fine-grid approximation, vh, is neverused outside of [12 , 1], but this is not quite true: the residual transfer in the thirdstep involves the residuals rh at • = 1

2 and = 38 , which in turn involve vh at

= 14 and = 3

8 . Thus, we need to store vh at these two border points just outsidethe local fine grid. On the other hand, we do not need to store vh

1 at = 18 .

This observation suggests introducing a new variable w2h1 that accumulates the

coarse-grid solution at × = 14 , which underlies the fine-grid point = 1

4 . We canidentify w2h

1 = vh2 , allowing us to keep track of the solution at this border point.

The solution at the other border point, = 38 , will be found by interpolation. The

improved algorithm now appears as follows:

• Initialize vh = 0, w2h1 = 0, and f2h

1 ≡(I2hh fh

)1

= fh1 +2fh

2 +fh3

4 .

• Relax on vh on the local fine grid xh5 , xh

6 , xh7 = 5

8 , 34 , 7

8.

• Compute fine-grid residual rh = fh −Ahvh on the local fine grid, its interfacepoint xh

4 = 12 , and the border point xh

3 = 38 , and transfer it to the coarse grid:

f2h2 ← rh

3 + 2rh4 + rh

5

4, f2h

3 ← rh5 + 2rh

6 + rh7

4.

166 Chapter 9

• Compute an approximation v2h to the coarse-grid residual equation A2hu2h =f2h.

• Update the residual at × = 14 for use in later cycles: f2h

1 ← f2h1 −−v2h

0 +2v2h1 −v2h

2(2h)2 .

• Update the coarse-grid approximation at × = 14 : w2h

1 ← w2h1 + v2h

1 .

• Interpolate the correction and update the approximation: vh ← vh + Ih2hv

2h

everywhere except at = 18 , 1

4 .

Note that we do not interpolate to = 14 , because the solution at this point is held

in w2h1 , nor to = 1

8 , because a fine-grid solution is not needed at this point.We have devoted an absurd amount of effort here to avoiding computation at

the single point = 18 . However, it is important to imagine situations in which the

local fine grid comprises a very small fraction of the domain. In such situations,points like = 1

8 predominate and the savings can be significant. As we see shortly,the savings can be even greater for two-dimensional problems.

We can take one more step and effectively remove the border points from thecomputation. Consider the residual f2h

2 at the interface point • = 12 ; it is obtained

by full weighting the fine-grid residuals rh3 , rh

4 , and rh5 . Because w2h

1 accumulatesthe solution at = 1

4 , we have vh2 = w2h

1 ; and because the solution at = 38

is determined by interpolation, we have vh3 = (w2h

1 + vh4 )/2. Thus, the fine-grid

residuals that contribute to f2h2 are computed as follows:

rh3 = fh

3 − −vh2 + 2vh

3 − vh4

h2

= fh3 − −w2h

1 + w2h1 + vh

4 − vh4

h2

= fh3 , (9.2)

rh4 = fh

4 − −(w2h1 + vh

4 )/2 + 2vh4 − vh

5

h2

= fh4 − − 1

2w2h1 + 3

2vh4 − vh

5

h2, (9.3)

rh5 = fh

5 − −vh4 + 2vh

5 − vh6

h2. (9.4)

Writing g2h2 ≡ 1

4

(fh3 + 2fh

4 + fh5

), a bit of algebra yields the residual f2h

2 at theinterface point:

f2h2 ≡ 1

4(rh3 + 2rh

4 + rh5

)(9.5)

= g2h2 − −w2h

1 + 2vh4 − vh

6

(2h)2.

Note that the stencil on the right side of (9.5) corresponds to the usual Ω2h stencilfor the residual. This is a result of the variational properties at work. Yet, even inmore general cases, it is possible to compute the result of transferring the fine-gridresidual to the interface point(s). In turn, this allows us to eliminate the borderpoints. These simplifications lead to our final algorithm.

A Multigrid Tutorial 167

Fast Adaptive Composite Grid Method (FAC)

• Initialize vh = 0, w2h1 = 0, and f2h

1 ≡(I2hh fh

)1

= fh1 +2fh

2 +fh3

4 .

• Relax on vh on the local fine grid xh5 , xh

6 , xh7 = 5

8 , 34 , 7

8.

• Compute the right sides for the local coarse grid: f2h2 ← g2h

2 − −w2h1 +2vh

4 −vh6

(2h)2 ,

f2h3 ← rh

5 +2rh6 +rh

74 .

• Compute an approximation v2h to the coarse-grid residual equation A2hu2h =f2h.

• Update the residual at × = 14 for use in later cycles: f2h

1 ← f2h1 −−v2h

0 +2v2h1 −v2h

2(2h)2 .

• Update the coarse-grid approximation at × = 14 : w2h

1 ← w2h1 + v2h

1 .

• Interpolate the correction and update the approximation: vh ← vh + Ih2hv

2h

at the local fine grid and the interface points 12 , 5

8 , 34 , 7

8.

Summary of FAC Terms

• The local fine grid is the finest grid used for computation; it covers onlythat part of the domain where additional resolution is needed. FAC avoidscomputation on a global fine grid.

• The global coarse grid, Ω2h, is the finest grid used for computation thatcovers the entire domain. The notation suggests a mesh refinement factorof two, but larger factors are permitted.

• Interface points are the boundary points of the local fine grid.

• Border points are fine-grid points that lie outside the local fine grid. Theyare used temporarily to develop special interface stencils.

• Boundary points are the usual points of the domain where boundary con-ditions apply.

• Slave points, which appear only in two and higher dimensions, are inter-face points that do not correspond to coarse-grid points.

• The composite grid is the combination of the fine- and coarse-grid pointson which the discrete solution is ultimately determined.

It is important to see FAC from a different perspective. We developed thescheme by eliminating the points from the fine grid and producing the top twouniform grids (Ωh and Ω2h) shown in Fig. 9.3. The question we need to ask nowis: At what points are we actually approximating the solution? When we use FAC,the solution is approximated at the composite grid (Ωc in Fig. 9.3) consisting of

168 Chapter 9

Ωh • ∗ ∗ ∗

Ω2h × × ×

Composite grid Ωc × • ∗ ∗ ∗

vc0 vc

1 vc2 vc

3 vc4 vc

5 vc6

Figure 9.3: FAC is done entirely on uniform grids, such as the top two grids shownabove. However, the solution is in effect determined on the nonuniform compositegrid shown at the bottom.

• local fine-grid points (∗),

• interface points (•), and

• coarse-grid points that do not lie under the local fine-grid or interface points(× points in (0, 1

2 )).

FAC operates only on uniform grids, but it effectively solves problems on com-positive grids. It is fully consistent with multigrid principles because, in the vari-ational case, FAC is equivalent to solving the problem by multigrid using globalgrids, but local relaxation. This property distinguishes FAC from other adaptivemethods.

While we never need to construct the composite grid, it serves as an importantconceptual tool for understanding the refinement algorithm that we just developed.For example, because FAC is a method for solving the problem on the compositegrid, we should be able to derive the associated composite-grid equations. This willallow us to see what algebraic problem FAC actually solves.

To derive these composite-grid equations, first consider the composite-grid ap-proximation vc that FAC produces (we include the boundary points vc

0 and vc6 for

completeness):

vc0 = 0 , vc

1 = w2h1 , vc

2 = vh4 , vc

3 = vh5 , vc

4 = vh6 , vc

5 = vh7 , vc

6 = 0 .

We also need to define the source vector f c. Leaving the definition of fc2 aside for

the moment, we have

fc1 =

fh1 + 2fh

2 + fh3

4, fc

3 = fh5 , fc

4 = fh6 , fc

5 = fh7 .

Now imagine that the FAC scheme has converged, by which we mean that vc

does not change from one cycle to the next. Inspecting the algorithm shows thatthe stencils in the uniform regions (× and points) are the usual ones with theappropriate mesh sizes:

−vc0 + 2vc

1 − vc2

(2h)2= fc

1 ,

−vci−1 + 2vc

i − vci+1

h2= fc

i , 3 ≤ i ≤ 5 . (9.6)

A Multigrid Tutorial 169

The difficult stencil is the one at the interface point • = 12 . Consider (9.5) and

remember we are assuming that FAC has converged, so relaxation does not changevh5 . This means that rh

5 must be zero. Using this observation and substituting (9.2)and (9.3) into (9.5) gives us

f2h2 =

14

(rh3 + 2rh

4

)=

14

(fh3 + 2fh

4

)− (−w2h

1 + 3vh4 − 2vh

5 )/(2h)2 .

We must have f2h2 = 0, or else the FAC coarse-grid step would compute a nonzero

correction to vc and we could not have converged. We thus conclude that the stencilat • = 1

2 is−w2h

1 + 3vh4 − 2vh

5

(2h)2=

14

(fh3 + 2fh

4

).

In composite-grid terms, with fc2 ≡ 1

4

(fh3 + 2fh

4

), we have

−vc1 + 3vc

2 − 2vc3

(2h)2= fc

2 . (9.7)

One of the main concerns in the design of any numerical algorithm is howto measure the error. This is a complex issue that involves various error sources(discretization error, algebraic error, and floating-point errors), the choice of norms,and various ways to approximate the error. Most of these issue are beyond the scopeof this book, but a few brief comments are in order:

• Estimating discretization error is of particular interest in adaptive methodsbecause it can be used to decide where to refine. There are many conven-tional approaches that involve solving local problems or estimating higherderivatives of the emerging solution. However, the presence of several levelsof discretization enables multilevel methods to use extrapolation: you cancompare the approximations on two or more levels to predict the error onthe finest level. For this purpose, it is convenient to have the full approxima-tions available on each level, so it can be useful to rewrite the FAC correctionscheme that we have presented as an FAS scheme (Exercise 1).

• The algebraic error can also be estimated in several ways, but a natural es-timate is to apply a norm to the residual of the composite-grid equations.Remember, the composite grid consists of the uniform fine grid in the refine-ment region, so any point within this region has a standard fine-grid equa-tion. Similarly, points outside the refinement region have standard coarse-gridequations. But the interface has special equations (9.7) that require specialresiduals. Putting this together for our example gives

rc1 =

14

(fh1 + 2fh

2 + fh3

)− 2w2h

1 − vh4

(2h)2, standard coarse-grid residual;

rc2 =

14

(fh3 + 2fh

4

)− −w2h

1 + 3vh4 − 2vh

5

(2h)2, interface residual from (9.7);

rci = fh

i − −vhi+1 + 2vh

i+2 − vhi+3

h2, 3 ≤ i ≤ 5, standard fine-grid residual.

170 Chapter 9

• Scaling the error norms is particularly important in adaptive methods becauseof the presence of different levels of resolution. The proper scales depend onthe discretization and other factors. For our example, you can think of theresidual rc as a source term that is constant in a neighborhood of each node.Thus, rc

1 represents the value in the interval (14 − h , 1

4 + h), rc2 represents the

value in the interval (12 −h , 1

2 + h2 ), and rc

i represents the value in the interval( i+2

8 − h2 , i+2

8 + h2 ) for i = 3, 4, 5. (This does not cover the interval [0, 1], but

we can simply assume that the source is zero in [0, 116 ) and (15

16 , 1].) Using thepremise that the residual represents a piecewise constant source term, we canobtain the norm of rc by taking the integral of the square of the source termdefined on (0, 1), which yields

‖rc‖ =

√√√√2h(rc1)2 +

32h(rc

2)2 + h

5∑3

(rci )2 .

Two-Dimensional Problems

With some modification, the above ideas can be extended to two-dimensional prob-lems. Figure 9.4 shows a square domain in the plane with a local fine grid in thenortheast corner. There are many points in this two-dimensional grid that are anal-ogous to = 1

8 , at which fine-grid computations can be neglected. For this reason,

• • • ••

•

•

•

∗ ∗ ∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗ ∗ ∗

∗∗∗∗∗∗∗

∗∗∗∗∗∗∗

∗∗∗∗∗∗∗

∗∗∗∗∗∗∗

∗∗∗∗∗∗∗

∗∗∗∗∗∗∗

∗∗∗∗∗∗∗

Figure 9.4: Composite grid on a two-dimensional domain. The coarse-grid pointsare at the intersection of the solid horizontal and vertical lines. The local fine gridconsists of ∗ points. Interface points coinciding with coarse-grid points are markedby •. The new slave points () are interface points that do not overlay a fine-gridpoint. Boundary points have been omitted.

A Multigrid Tutorial 171

the FAC scheme shows substantial savings over standard multigrid on a global finegrid.

A new type of point arises in two-dimensional problems. Denoted by in Fig.9.4, the slave points are interface points that do not correspond to a coarse-gridpoint. These points are needed to provide a complete set of boundary points forthe local fine grid. They are determined simply by averaging the two neighboringinterface point values. For example, in Fig. 9.4, a typical slave point on a horizontalinterface would be determined by

vhi+1,j =

vhi,j + vh

i+2,j

2.

The one-dimensional FAC method can be generalized in all respects to twodimensions. Special restriction formulas at the interface points corresponding to(9.5) can be used to eliminate border points. Similarly, with some additional work,it is possible to find the coarse-grid and interface equations as was done in onedimension (Exercise 4). However, our goal is not to expose these details (which canbe found in [14]). Rather, we will only outline the flow of the two-dimensional FACmethod in fairly qualitative terms.

Fast Adaptive Composite Grid Method (FAC)in Two Dimensions

• Initialize vh = 0. Initialize w2h = 0 and f2h = I2hh fh at coarse-grid points

outside the local fine grid.

• Relax on vh on the local fine grid.

• Compute the right sides for the local coarse grid using special stencils atthe interface points and the usual residual transfer f2h = I2h

h fh at pointsunderlying the local fine grid.

• Compute an approximation v2h to the coarse-grid residual equation A2hu2h =f2h.

• Update the coarse-grid residual for use in later cycles: f2h ← f2h − A2hv2h

at points outside the local fine grid.

• Update the coarse-grid approximation: w2h ← w2h+v2h at coarse-grid pointsoutside the local fine grid.

• Interpolate the correction and update the approximation: vh ← vh + Ih2hv

2h

at the local fine grid and interface points. Interpolate to the slave points fromtheir interface neighbors.

We have presented FAC as a two-grid scheme. Like other multigrid methods,it is done recursively in practice: in the fourth step of the above algorithm, thesolution of the global coarse-grid problem is done using further applications of thebasic two-grid scheme. This procedure leads to V-cycles defined on a sequence ofgrids, possibly involving several global coarse grids together with several telescopingfiner grids.

172 Chapter 9

An FMG–FAC scheme is more subtle. One could apply a straightforward FMGcycling scheme to the uniform grids that are used in the V-cycle process. However,this would be too costly when there are many levels with few points per level(Exercise 3). A more efficient approach is to use a sequence of increasingly coarsecomposite grids as the basis for FMG, a strategy that takes us beyond the scope ofthis book.

The fundamental basis for FAC is the composite grid. In fact, probably themost effective way to apply FAC to a given problem is to start by developing ac-curate composite-grid equations. The FAC algorithm is then a natural extensionof multigrid, where the finest level is the composite grid, relaxation is simply re-stricted to the local fine grid, and coarsening is based on restricting the compositegrid residual to the global coarse grid.

A popular alternative to FAC is the multilevel adaptive technique, or MLAT[3]. Although these two approaches can lead to similar algorithms in the end, thebasis for their development is very different. While FAC is based on the compositegrid, MLAT comes from generalizing the full approximation scheme (FAS). A littleinspection of FAS, as developed in Chapter 6, shows that the finest grid need not beglobal: one can relax only on a local fine grid and correct the coarse-grid equationsby FAS only at points that lie under it. MLAT can thus be developed from a global-grid FAS code, and it often proves effective for handling adaptive refinement.

Numerical example. To illustrate FAC performance in a simple setting, FAC isapplied to Poisson’s equation with homogeneous Dirichlet boundary conditions onthe unit square in the plane. The exact two-spike solution, as shown in Fig. 9.5,has values of magnitude 1 but opposite sign at (1

4 , 14 ) and (3

4 , 34 ), falls off steeply

in the neighborhood of these points, and is zero at the boundary. In the tests, weused composite grids constructed from a global grid of mesh size h and refinement

Figure 9.5: Two-spike solution of the composite grid problem based on a 63 × 63interior global grid, with mesh size h = 1

64 , and 51×51 patches of mesh size h2 = 1

128centered at ( 1

4 , 14 ) and ( 3

4 , 34 ).

A Multigrid Tutorial 173

Figure 9.6: Aerial view of a composite grid based on a 31 × 31 interior global gridof mesh size h = 1

32 and 27×27 patches of mesh size h2 = 1

64 centered at ( 14 , 1

4 ) and( 34 , 3

4 ).

patches of size h2 in the subregions [ 1

32 , 1532 ] × [ 1

32 , 1532 ] and [1732 , 31

32 ] × [ 1732 , 3132 ]. Tests

were done for h = 132 , 1

64 , 1128 , with the case h = 1

64 shown in Fig. 9.6.Table 9.1 contains the results of applying a V(1,0)-cycle to each of the three

composite grids. The convergence factor was obtained by applying many cyclesto the homogeneous problem (with zero right side), starting with a random initialguess, and observing the worst-case composite-grid residual reduction factor. Note

Global h Convergence factor Discrete L2 norm of discretization error1/32 0.362 2.34e − 21/64 0.367 5.742 − 31/128 0.365 1.43e − 3

Table 9.1: Estimates for asymptotic V(1,0)-cycle convergence factors and discreteL2 norm of the discretization errors. FAC was applied using composite grids withthree different global grids of mesh size h, each with two local patches of meshsize h

2 .

174 Chapter 9

Figure 9.7: Side view of a global 15× 15 grid with two coarser supporting grids andthree successively finer local grids.

the apparent h-independence of these convergence factors. These factors are quiteacceptable considering the fact that V(1,0)-cycles were used. Also shown are thediscretization error estimates, obtained by applying several V(1,0)-cycles to thetwo-spike problem and measuring the difference, in the discrete L2 norm, betweenthe computed and the exact solutions, evaluated on the composite grid. Note theapparent O(h2) behavior of the discretization error.

Another illustration of the grids that comprise the composite grids is given inFig. 9.7. Here we show a side view of a global grid of mesh size h = 1

8 and localgrids of mesh size h

8 = 164 in the subregions [18 , 3

8 ] × [ 18 , 38 ] and [58 , 7

8 ] × [ 58 , 78 ]. Note

that the effective refinement factor (ratio of mesh sizes of the finest patch and finestglobal grid) is eight. Large refinement factors are permissible with FAC becauselocal grids with factors of two are included between the global and finest local grids.For similar reasons, the finest global grid is supported by a full sequence of coarserglobal grids, as shown.

Exercises

1. FAS version of FAC. The FAC algorithm we have described is a correctionscheme in which the coarse-grid problem is the residual equation. RewriteFAC for the one-dimensional model problem as an FAS scheme in which thefull approximation equations are solved on the coarse grid.

2. Higher mesh refinement factors. We developed the FAC scheme assum-ing a mesh refinement factor of two: the global coarse-grid mesh size is 2h,while the local fine-grid mesh size is h. Rewrite FAC for solving the one-dimensional model problem when the coarse-grid mesh size is 4h = 1

4 and thefine-grid mesh size is h = 1

16 . Assume that the local fine grid covers ( 12 , 1) as

A Multigrid Tutorial 175

before. (For full efficiency, FAC should include the intermediate grid of meshsize 2h that covers ( 1

2 , 1), but the two-grid scheme suffices here.)

3. Cycling cost. Show that V-cycles maintain O(n) complexity for locallyrefined grids in the sense that the cost of an FAC V-cycle is proportionalto the number, n, of composite-grid points. For concreteness, consider asequence of q grids on the unit square in the plane, constructed by startingfrom a global m × m grid and refining successive grids by a factor of two inthe northeast quadrant (so that each grid is m×m). Now show that W-cyclesdo not maintain O(n) complexity. Similarly, show that FMG based on theseuniform grids also does not maintain O(n) complexity. Finally, show thatFMG based on coarsened composite grids does maintain O(n) complexity.(Coarse composite grids are formed by coarsening all grids in the sequence,so the first coarse composite grid is formed from q grids of size m

2 × m2 .)

4. Two-dimensional interface source terms. Deriving the Ω2h and Ωc inter-face equations is somewhat complicated in two dimensions. Take the first stepin this direction by forming the source term for Ω2h and Ωc at the interfacecorner point in Fig. 9.4.

5. Symmetry. Show that the composite-grid matrix Ac is nonsymmetric byshowing that the coefficient in (9.6) that connects vc

2 to vc3 is not equal to

the corresponding coefficient in (9.7). Show, however, that symmetry can beretained by rescaling: the matrix DcAc is symmetric, where Dc is the 5 × 5diagonal matrix with entries dc

1 = dc2 = 2h and dc

3 = dc4 = dc

5 = h.

6. Neumann problem. Rederive FAC applied to Neumann problem (7.1) ofChapter 7. Show that the principles of Neuman boundary conditions apply byshowing that the null space and range of the rescaled composite-grid matrix,DcAc of Exercise 5, are equal and consist of scalar multiples of the constantvector 1.

Chapter 10

Finite Elements

Until now, our development of multigrid methods has relied on finite differences.However, there are several other discretization methods, all of which have theirplace in computation. We now turn to the finite element method. This approachmay seem more abstract and less direct at the outset, but its use of variationalproperties provides a powerful framework that is well suited to developing andanalyzing multigrid methods. It is impossible to cover finite elements completelyin this short chapter, so we focus instead on surveying the critical ideas, leavingthe important details to the exercises and further study.

Said simply, finite difference methods replace the problem domain by a grid andproduce a vector whose components are approximations to the solution at the gridpoints. On the other hand, finite element methods partition the problem domaininto subregions and produce a simple function in each subregion that approximatesthe solution. Finite element methods can be applied to two major categories ofproblems: differential equations and functional minimization. When the differentialequation is defined in terms of a self-adjoint linear operator and the functionalis defined appropriately, these two categories coincide. To exploit this fortunateduality between differential equations and functional minimization, we focus on theself-adjoint case.

Let Ω be a suitably nice bounded domain in the plane with boundary ∂Ω. Con-sider the following Poisson equation with homogeneous Dirichlet boundary condi-tions:

Lu ≡ −uxx − uyy = f in Ω,

u = 0 on ∂Ω.

We first need to introduce some new tools and notation. Most of our time willbe spent in the space L2(Ω) of square integrable functions on Ω (functions thatsatisfy

∫Ω

u2 dΩ < ∞) equipped with the L2(Ω) inner product (·, ·) defined by

(u, v) =∫

Ω

uv dΩ.

Because we will be taking partial derivatives, it is not enough for the functions weconsider just to be L2(Ω) integrable. We need to know that they are differentiable.We will be vague about this for the moment and just assume that the functions

177

178 Chapter 10

reside in a subspace H ⊂ L2(Ω) that contains suitably differentiable functions. Wealso assume that the functions in H vanish on ∂Ω so that they satisfy the boundaryconditions.

There are two important properties of the operator L that we need (Exercise2):

• L is self-adjoint in the sense that (Lu, v) = (u, Lv) for any u, v ∈ H; and

• L is positive in the sense that (Lu, u) > 0 for any nonzero u ∈ H.

It is worth noting that if L is a matrix, u and v are vectors, and (·, ·) is the usualEuclidean inner product, then self-adjoint and positive correspond to symmetricand positive definite, respectively (Exercise 3).

With these properties in hand, we can demonstrate the crucial duality mentionedearlier: solving the differential equation Lu = f is formally equivalent to minimizingthe functional

F (u) ≡ 12(Lu, u) − (f, u)

over u ∈ H. The functional F is quadratic in u and it has instructive scalar andmatrix analogues (Exercises 4 and 5). The minimization of F is often written inthe compact notation

u = argminv∈H F (v), (10.1)

which means “find the argument that minimizes F over all functions in H.” Thislink between the differential equation and the minimization problem lies at theheart of the finite element formulation. It is so important that we should study itfor a moment to understand it.

Suppose u is a candidate function for minimizing F and v = 0 is any otherfunction in H. Consider the value of F at the point u + v. Using the linearity of Land the bilinearity of the inner product, we see that

F (u + v) =12(L(u + v), u + v) − (f, u + v)

=12

((Lu, u) + (Lu, v) + (u, Lv) + (Lv, v)) − (f, u) − (f, v)

= F (u) +12

((Lu, v) + (u, Lv) + (Lv, v)) − 〈f, v〉 .

Because L is self-adjoint, we know that (u, Lv) = (Lu, v). This allows us tosimplify the above expression further:

F (u + v) = F (u) + (Lu, v) − (f, v) +12(Lv, v)

= F (u) + (Lu − f, v) +12(Lv, v)︸ ︷︷ ︸

positive

.

The last term in this expression is positive because L is a positive operator. There-fore, we see that if Lu = f , then F (u + v) ≥ F (u) for all v ∈ H, which meansthat u minimizes F over H. Conversely, if u minimizes F , then it follows that(Lu − f, v) = 0 for all v ∈ H, which means that

(Lu, v) = (f, v) for all v ∈ H . (10.2)

A Multigrid Tutorial 179

Ω

i − 1 i i + 1

j − 1

j

j + 1

Figure 10.1: A domain Ω showing the four elements surrounding grid point (i, j).

This condition, which is useful for implementing finite elements, is essentially an-other way of saying that Lu = f . Thus, we have shown that the differential equationand minimization problem are essentially equivalent.

The finite element discretization of the problem can now be described. Supposethat the domain is the unit square, Ω = (0, 1)× (0, 1), on which we place a uniform(n + 1) × (n + 1) grid, Ωh, with mesh size h = 1

n . As before, let (xi, yj) bethe grid point with coordinates (ih, jh). However, we now focus not on the gridpoints, but on the square regions surrounding each grid point; these small regionsare called elements (Fig. 10.1). Notice that there are four elements surroundingeach grid point. The sets of four elements corresponding to neighboring grid pointsoverlap in one or two elements. (Everything that follows can also be carried out onnonuniform grids.)

We begin with a common choice of approximating functions. Let Hh be thesubspace of H consisting of piecewise bilinear functions uh: each uh ∈ Hh is zeroon ∂Ω, continuous on Ω, and bilinear within each element. This means that atypical function in Hh has the form uh(x, y) = axy + bx + cy + d on each element,as depicted in Fig. 10.2. The level h discretization of (10.1) is

uh = argminvh∈Hh F (vh). (10.3)

We have seen that the minimization problem on H is equivalent to solving (10.2)for u ∈ H. This equivalence is also true on Hh. Thus, the discrete minimizationproblem (10.3) is equivalent to finding uh ∈ Hh so that

(Luh, vh) = (f, vh) for all vh ∈ Hh. (10.4)

180 Chapter 10

0

5

10

15

20

0

5

10

15

20–0.05

0

0.05

0.1

0.15

Figure 10.2: A typical function in the space Hh used to approximate the solutionto the problem. It is zero on ∂Ω, continuous on Ω, and bilinear within each of thenine elements pictured.

Finite Element Terminology. The terminology associated with finite ele-ment methods is not entirely standard and can get confusing. The discretizationof the minimization problem is generally called the Rayleigh–Ritz formulation.The finite element method also has a Galerkin form, which starts not from theminimization problem (10.1), but from the equation Lu = f . As with Rayleigh–Ritz, the idea behind the Galerkin form is to approximate the solution u ∈ H byuh ∈ Hh. However, the Galerkin approach requires the residual Luh − f to beorthogonal to every vh ∈ Hh. This condition also leads to (10.4). Thus, when Lis a positive, self-adjoint, linear operator, and F is quadratic, the Rayleigh–Ritzand Galerkin formulations are essentially the same.

The practical solution of (10.4) requires several additional considerations. First,we need to choose a local basis for Hh consisting of functions, each of which isnonzero on its own particular patch of four elements. Specifically, let εh

i,j(x, y)denote the piecewise bilinear function in Hh that is centered on an interior gridpoint (xi, yj); it has the value 1 at (xi, yj) and is zero at all other grid points (Fig.10.3). Any uh ∈ Hh can then be expanded as

uh(x, y) =n−1∑i,j=1

uhi,jε

hi,j(x, y). (10.5)

The nodal value uhi,j gives the value of uh at (xi, yj); for this reason, (10.5) is called

a nodal basis expansion.

A Multigrid Tutorial 181

Figure 10.3: A typical basis function εhi,j(x, y) for the space Hh. It is nonzero, only

on the patch of four elements that have (xi, yj) as a node.

The next step is to substitute expansion (10.5) into expression (10.4). However,this task leads to a paradox: the operator L is second order and the approximatingfunction uh is linear in x and y, so Luh = 0 in each element! How can Luh be zerowhen we know generally that (Luh, uh) > 0? The answer is that Luh is not zero onΩ. In fact, Luh does not even make sense on Ω because uh is not suitably smooth:the first partials of uh are piecewise smooth and therefore square integrable, buttheir discontinuities at the element boundaries prevent second derivatives frombeing square integrable. This means that we really cannot substitute (10.5) into(10.4). Fortunately, there is a way out of this dilemma: we just rewrite (10.4) in aform that requires fewer derivatives. This is done by applying the Gauss divergencetheorem (the analogue of integrating by parts) to (Lu, v), where u, v ∈ H, and usingthe fact that u and v vanish on ∂Ω (Exercise 1):

(Lu, v) =∫

Ω

(−uxx − uyy)v dΩ

=∫

Ω

(uxvx + uyvy) dΩ

= (∇u,∇v) .

This maneuver replaces the second-order derivatives in the problem by first-orderderivatives. Making this replacement in (10.4), we are led to the weak form

(∇uh,∇vh) = (f, vh) for all vh ∈ Hh. (10.6)

Strictly speaking, this new problem is more general or weaker than the differentialequation: a solution may exist, but it may not be twice differentiable, which the

182 Chapter 10

classical solution of Lu = f requires. We stay with the weak form for the remainderof the chapter, so we are now free to be specific about what suitably smooth means:H ⊂ L2(Ω) is assumed to consist of functions that vanish on ∂Ω and that have firstpartials in L2(Ω).

With the weak form of the problem, we can now substitute the expansion (10.5)into (10.6). Furthermore, we choose the so-called trial functions vh to be the basisfunctions εi,j . The result of this substitution is a linear system whose unknownsare the nodal values uh

i,j . The matrix coefficients in this system are inner productsof the form (∇εh

i,j ,∇εhk,) and the right-side values are (f, εh

i,j). Note that withour chosen bilinear basis functions, only neighboring basis functions overlap; thus,the inner products (∇εh

i,j ,∇εhk,) are zero unless k = i or i ± 1 and = j or j ± 1.

Evaluating the nine inner products associated with the (i, j) patch results in a rowof the stiffness matrix (Exercise 6) given by the stencil

Ahi,j =

13

−1 −1 −1

−1 8 −1−1 −1 −1

. (10.7)

This is the just the nine-point finite difference stencil scaled by 13 (instead of the

usual 13h2 ).

The inner products (f, εhi,j) involve the integrals

(f, εhi,j) =

∫ xi+1

xi−1

∫ yi+1

yi−1

fεhi,j dxdy,

which are generally approximated numerically. The simplest numerical integrationscheme amounts to replacing the function f by its value f(xi, yj) (Exercise 7):

(f, εhi,j) =

∫ xi+1

xi−1

∫ yi+1

yi−1

fεhi,j dxdy

≈ f(xi, yj)∫ xi+1

xi−1

∫ yi+1

yi−1

εhi,j dxdy

=h2

4f(xi, yj). (10.8)

When all rows of stiffness matrix (10.7) and corresponding right sides (10.8) areassembled, the result is the matrix equation

Ahuh = fh. (10.9)

Here we have introduced the source vector (fh)i,j =(

h2

4 f(xi, yj))

and the solution

vector (uh)i,j =(uh

i,j

)∈ Ωh.

This discretization shows that finite elements and finite differences boil down tosimilar matrix problems. But it is important to keep in mind that the methodologyused to obtain them is quite different. Practical implementation of a multigridscheme for finite elements can be done easily if we are guided by—and remainfaithful to—the basic principle of functional minimization. For example, it mayseem convenient to rescale (10.9) by dividing both sides by h2, but this wouldchange the relationship between the various grids used in the multigrid scheme,and it could easily introduce conceptual errors into the overall process.

A Multigrid Tutorial 183

We develop the multigrid scheme in the abstract by focusing on the functionsuh that solve (10.3) and (10.4), rather than their nodal values uh. First considerrelaxation, whose goal is to provide an inexpensive method for eliminating oscilla-tory errors in the approximation, vh. We can do this by making local changes ofthe form

vh ← vh − sεhi,j , (10.10)

where s ∈ R is a suitably chosen step size. But how should s be chosen? Our planof being faithful to the minimization principle gives us the answer: choose the beststep size in the sense that it minimizes the functional over all possible choices. Themathematical statement is

s = argmint∈R F (vh − tεhi,j). (10.11)

This gives us the following relaxation scheme, which amounts to using (10.10) and(10.11) to sweep over the grid points:

For each i, j = 1, 2, ..., n − 1:

compute s = argmint∈R F (vh − tεhi,j) and make the replacement vh ← vh − sεh

i,j .

For our special functional F , this coordinate relaxation scheme is none other thanGauss–Seidel applied to (10.7) (Exercise 8).

The coarse-grid correction process is also easy to formulate in the abstract. Wedefine the coarse-grid space H2h ⊂ Hh as the set of piecewise bilinear functionsassociated with the standard coarse grid Ω2h formed by deleting the odd-numberedlines of Ωh. The goal is to change the approximation vh by a function v2h ∈ H2h

that approximates the presumably smooth error. The form of this correction isvh ← vh + v2h. But how should v2h be chosen? Again, being faithful to the mini-mization principle gives us the answer: choose the best coarse-grid correction in thesense that it minimizes the functional over all possible choices. The mathematicalstatement is

v2h = argminw2h∈H2h F (vh + w2h). (10.12)

The coarse-grid correction scheme is

Compute v2h = argminw2h∈H2h F (vh + w2h) and set vh ← vh + v2h.

This correction step, together with the coordinate relaxation scheme defined above,constitutes the core of the multigrid method. We can design practical algorithmsbased on this core just as we did in Chapter 3.

Our development of the finite element coarse-grid correction step appears sim-pler than it was for finite differences, because it comes naturally from the mini-mization principle. However, this simplicity is somewhat deceptive because it muststill be expressed in terms of nodal vectors, and this requires intergrid transfer andcoarse-grid operators. We will see that there are no choices that need to be madehere: these operators are determined by the spaces and bases that we have alreadyselected.

184 Chapter 10

2i 2i + 1 2(i + 1)

x

Figure 10.4: Cross-section of the grid along a line of constant y showing the coarse-grid points (2i, 2j) and (2(i+1), 2j) and the intermediate fine-grid point (2i+1, 2j).The coarse-grid element is linear in x between the two coarse-grid points.

Consider first the process of adding an Ω2h function, v2h ∈ H2h, to an Ωh

function. This process requires that we take the Ω2h nodal representation for v2h,

v2h(x, y) =

n2 −1∑

i,j=1

v2hi,jε

2hi,j(x, y) ,

and convert it to a Ωh nodal representation,

v2h(x, y) =n∑

i,j=1

vhi,jε

hi,j(x, y) .

In other words, we must find the coefficients vhi,j that allow us to write v2h as a

function in Hh. We know that we can do this because H2h is a subset of Hh, sov2h must be in Hh. What we are looking for here is really just the interpolationoperator Ih

2h that allows us to determine vh = Ih2hv

2h.Consider the grid Ω2h node with indices (i, j) that is located at the point

(i(2h), j(2h)). This point is also the grid Ωh node ((2i)h, (2j)h) with indices (2i, 2j).Remembering that the coefficients in the nodal representations are just the functionvalues at the nodes, we have

vh2i,2j = v2h((2i)h, (2j)h) = v2h(i(2h), j(2h)) = v2h

i,j .

This says that vh2i,2j = v2h

i,j .Consider now the fine-grid node at point ((2i + 1)h, (2j)h) (see Fig. 10.4). This

node is halfway between nodes ((2i)h, (2j)h) and (2(i+1)h, (2j)h) along a horizontalgrid line. This line segment is part of a single coarse-grid element in which v2h isbilinear and y is constant; thus, v2h must be linear in x along this line. This leadsus to conclude that

vh2i+1,2j =

12

(v2h

i,j + v2hi+1,j

).

For similar reasons, we have

vh2i,2j+1 =

12

(v2h

i,j + v2hi,j+1

)

A Multigrid Tutorial 185

andvh2i+1,2j =

14

(v2h

i,j + v2hi+1,j + v2h

i,j+1 + v2hi+1,j+1

).

As you can see, we have just reinvented the interpolation operator Ih2h introduced

in Chapter 3.The next task is to determine the coarse-grid version of the operator Ah, which

will eventually be called A2h. It is most easily discovered by working with the nodalvectors and translating the minimization principle into matrix terms. As we haveseen, discrete minimization principle (10.3) is equivalent to discrete equation (10.4),which in turn is equivalent to matrix equation (10.9). Similar reasoning shows thatmatrix equation (10.9) is equivalent to the matrix minimization principle

uh = argminvh∈Ωh Fh(vh) , (10.13)

whereFh(vh) ≡ 1

2(Ahvh,vh) − (fh,vh)

(Exercise 5). Furthermore, similar reasoning also shows that coarse-grid correctionproblem (10.12) is equivalent to the matrix problem

v2h = argminw2h∈Ω2h Fh(vh + Ih2hw

2h).

We use this form of the coarse-grid correction problem to carry out an informativecalculation. Using the properties of inner products, we have (read this one with apencil and paper!)

Fh(vh + Ih2hw

2h)

=12

(Ah

(vh + Ih

2hw2h

),vh + Ih

2hw2h

)− (fh,vh + Ih

2hw2h)

=12(Ahvh,vh) +

12(AhIh

2hw2h,vh) +

12(vh, AhIh

2hw2h)

+12(AhIh

2hw2h, Ih

2hw2h) − (fh,vh) − (fh, Ih

2hw2h)

=12(Ahvh,vh) − (fh,vh)

+12(AhIh

2hw2h, Ih

2hw2h) + (vh, AhIh

2hw2h) − (fh, Ih

2hw2h)

= Fh(vh) +12

(

Ih2h

)TAhIh

2h︸ ︷︷ ︸A2h

w2h,w2h

−

(

Ih2h

)T︸ ︷︷ ︸I2h

h

(fh − Ahvh

),w2h

.

Notice that the first underbraced item,(Ih2h

)TAhIh

2h, is a matrix that operateson a coarse-grid vector. Furthermore, the role it plays in the inner product isanalogous to the role played by Ah in the functional Fh. This suggests that we letA2h =

(Ih2h

)TAhIh

2h, the coarse-grid version of Ah.

The second underbraced item,(Ih2h

)T , must take a fine-grid vector into a coarse-grid vector; that is, it plays the role of a restriction operator. It therefore makessense to let I2h

h =(Ih2h

)T .

186 Chapter 10

With these operators identified, we may conclude the calculation. Letting rh =fh − Ahvh and f2h =

(I2hh

)rh, we have

Fh(vh + Ih2hw

2h) = Fh(vh) +12

(A2hw2h,w2h

)−

((I2hh

)rh,w2h

)= Fh(vh) + F 2h(w2h) .

We should stand back and see what we have done. First, because Fh(vh) isindependent of w2h, the minimization of Fh(vh + Ih

2hw2h) is equivalent to the

minimization of F 2h(w2h) over vectors w2h ∈ Ω2h; this is just the coarse-gridcorrection scheme introduced in Chapter 3.

Equally important, we have seen the variational properties of Chapter 5 emergerather naturally through the minimization principle. They bear repeating:

A2h = I2hh AhIh

2h (Galerkin property),

I2hh =

(Ih2h

)T.

Thus, the finite element coarse-grid correction scheme amounts to choosing bi-linear interpolation, its transpose for restriction, and the Galerkin property fordetermining the coarse-grid operator. Except for scaling differences, we have rein-vented the transfer operators, coarse-grid operators, and the coarse-grid correctionscheme developed in the previous chapters. It is important to keep in mind thatfor positive self-adjoint problems, we now have the option of developing multigridsolvers directly from the differential equations, as we did before, or using the min-imization principle, as we did here. The differential equations may provide moreflexibility simply because you do not have to adhere to optimality, but the choiceto follow the minimization principle may give guidance and assurance that is lessforthcoming from the differential equations.

Exercises

1. Weak form. For functions u, v that are sufficiently smooth on Ω and vanishon ∂Ω, show that (Lu, v) = (∇u,∇v). Hint: First apply the divergencetheorem to the product v∇ · u, where u is a vector-valued function and v isa scalar. Then remember that Lu = −∆u = −∇ · (∇u).

2. L is self-adjoint and positive. Let u, v be sufficiently smooth functionsthat vanish on ∂Ω. Show that the Poisson operator L = − ∂2

∂x2 − ∂2

∂y2 is self-adjoint in the sense that (Lu, v) = (u, Lv). Hint: Use Exercise 1. Showthat L is positive in the sense that (Lu, u) > 0 for u = 0. Hint: Show that(Lu, u) = 0 implies u is constant.

3. Self-adjoint and positive for matrices. Show that if L is a matrix, u andv are vectors, and (·, ·) is the usual vector dot product, then self-adjoint andpositive correspond to symmetric and positive definite, respectively.

4. Scalar analogue. Show that if L > 0 and f are scalars and (·, ·) is just scalarmultiplication, then (Lu, u) is a quadratic function. Furthermore, 1

2 (Lu, u)−(f, u) is minimized by the solution of Lu = f , which is u = f/L.

A Multigrid Tutorial 187

5. Matrix analogue. Suppose that L is a symmetric positive-definite n × nmatrix, u and f are n-vectors, and (·, ·) is the usual vector dot product.Show that 1

2 (Lu, u) − (f, u) is minimized by the solution of Lu = f , which isu = L−1f .

6. Stiffness matrices. For fixed indices i and j, show that the inner products(∇εh

i,j ,∇εhk,) for k = i or i±1 and = j or j±1 give the 3×3 stencil (10.7).

7. Right-side inner products. Show that∫ xi+1

xi−1

∫ yi+1

yi−1

εhi,jf dxdy ≈ h2

4f(xi, yj),

where the region of integration is a single element.

8. Gauss–Seidel. Show that minimization problem (10.11) results in Gauss–Seidel applied to (10.9).

9. Variational property. Verify each of the steps in the simplification ofFh(vh + Ih

2hw2h) that led to the variational properties.

Bibliography

The ultimate resource for multigrid methods is the MG-Net website, located athttp://www.mgnet.org. The site is maintained by Craig Douglas and sponsoredby Yale University, CERFACS, the University of Kentucky, and the NSF. Thesite features an extensive bibliography (over 3200 citations), links to free soft-ware (at least 22 packages at last count), information on conferences, and issues ofthe Multigrid Newsletter (back to 1991). To subscribe to the newsletter, write tomgnet@cs.yale.edu.

The Copper Mountain Conferences on Multigrid Methods have been held bi-ennially since 1983. There have also been six European multigrid conferences.Proceedings of recent conferences are available via MG-Net.

[1] R.E. Bank, PLTMG: A Software Package for Solving Elliptic Partial Differ-ential Equations. Users’ Guide 7.0, vol. 15, Frontiers in Applied Mathematics,SIAM, Philadelphia, 1994.

[2] J.H. Bramble, Multigrid Methods, vol. 294, Pitman Research Notes in Math-ematical Sciences, Longman Scientific and Technical, Essex, England, 1993.

[3] A. Brandt, Multi-level adaptive solutions to boundary value problems, Math.Comput., 31, 1977, pp. 333–390.

[4] A. Brandt, Multigrid Techniques: 1984 Guide with Applications to FluidDynamics, GMD-Studien Nr. 85, Gesellschaft fur Mathematik und Datenver-arbeitung, St. Augustin, Bonn, 1984.

[5] A. Brandt, Algebraic multigrid theory: The symmetric case, Appl. Math.Comput., 19, 1986, pp. 23–56.

[6] A. Brandt, S.F. McCormick, J. Ruge, Algebraic multigrid (AMG) forsparse matrix equations, in Sparsity and its Applications, D.J. Evans, ed.,Cambridge University Press, Cambridge, UK, 1984, pp. 257–284.

[7] J.W. Demmel, Applied Numerical Linear Algebra, SIAM, Philadelphia, 1997.

[8] J.E. Dennis and R.B. Schnabel, Numerical Methods for Unconstrained Op-timization and Nonlinear Equations, vol. 16, Classics in Applied Mathematics,SIAM, Philadelphia, 1996.

[9] G. Golub and C. Van Loan, Matrix Computations, Johns Hopkins Univer-sity Press, Baltimore, MD, 1996.

189

190 Bibliography

[10] W. Hackbusch, Multigrid Methods and Applications, vol. 4, ComputationalMathematics, Springer-Verlag, Berlin, 1985.

[11] W. Hackbusch and U. Trottenberg, Multigrid Methods, vol. 960, LectureNotes in Mathematics, Springer-Verlag, Berlin, 1982.

[12] D. Jespersen, Multigrid methods for partial differential equations, in Studiesin Numerical Analysis, vol. 24, Studies of Mathematics, MAA, WashingtonD.C., 1984, pp. 270–318.

[13] S.F. McCormick, ed., Multigrid Methods, vol. 3, Frontiers in Applied Math-ematics, SIAM, Philadelphia, 1987.

[14] S.F. McCormick, Multilevel Adaptive Methods for Partial Differential Equa-tions, vol. 6, Frontiers in Applied Mathematics, SIAM, Philadelphia, 1989.

[15] S.F. McCormick, Multilevel Projection Methods for Partial DifferentialEquations , vol. 62, CBMS-NSF Regional Conference Series in Applied Math-ematics, SIAM, Philadelphia, 1992.

[16] J.M. Ortega and W.C. Rheinboldt, Iterative Solution of Nonlinear Equa-tions in Several Variables, Academic Press, San Diego, 1970. Reprinted as vol.30, Classics in Applied Mathematics, SIAM, Philadelphia, 2000.

[17] U. Rude, Mathematical and Computational Techniques for Multilevel AdaptiveMethods, vol. 13, Frontiers in Applied Mathematics, SIAM, Philadelphia, 1993.

[18] J. W. Ruge and K. Stuben, Algebraic multigrid, in Multigrid Methods, vol.3, Frontiers in Applied Mathematics, S. F. McCormick, ed., SIAM, Philadel-phia, 1987, pp. 73–130.

[19] V. V. Shaidurov, Multigrid Methods for Finite Elements, vol. 318, Mathe-matics and Its Applications, Kluwer, Dordrecht, 1995.

[20] G.W. Stewart, Introduction to Matrix Computations, Academic Press, NewYork, 1973.

[21] G. Strang and G. Fix, An Analysis of the Finite Element Method, Prentice-Hall, Englewood Cliffs, NJ, 1973.

[22] K. Stuben, Algebraic Multigrid (AMG): An Introduction with Applications,GMD-Studien Nr. 53, Gesellschaft fur Mathematik und Datenveranbeitung,St. Augustin, Bonn, 1999.

[23] L.N. Trefethen and D. Bau, Numerical Linear Algebra, SIAM, Philadel-phia, 1997.

[24] R.S. Varga, Matrix Iterative Analysis, Prentice-Hall, Englewood Cliffs, NJ,1962.

[25] P. Wesseling, An Introduction to Multigrid Methods, John Wiley and Sons,Chichester, 1992.

[26] D. Young, Iterative Solution of Large Linear Systems, Academic Press, NewYork, 1971.

Index

A-inner product, 77, 139A-norm, 77, 139A-orthogonal, 77

algebraic error, 7, 76algebraic multigrid, 4, 137aliasing, 19, 32, 80alternating direction method, 64AMG costs, 153AMG two-grid correction cycle, 153amplification factor, 49anisotropic problems, 120asymptotic convergence factor, 48, 66

bilinear functions, 179block Jacobi method, 123border points, 165, 167boundary points, 167boundary value problems, 1

coarse-grid correction, 84, 87, 93coarse interpolatory set, 142coloring, 146compatibility condition, 114complementary modes, 80, 92composite grid, 167condition number, 29conservative method, 130convection-diffusion equation, 5, 72convergence factor, 17, 66, 75, 93convergence rate, 17, 71coordinate relaxation, 183correction scheme, 33cubic interpolation, 66cyclic reduction, 4

damped Jacobi method, 9, 22, 28, 93data structure, 46, 69diagonally dominant, 3direct methods, 4

discrete L2 norm, 55discretization error, 77

eigenvectors, 18, 28elements, 179energy inner product, 77energy norm, 77, 91essential boundary conditions, 113

Fast adaptive composite grid method(FAC), 76, 163, 167

fast Fourier transform, 4finite difference method, 1finite element method, 177FMG, 78, 91FMG cycle, 42, 48, 67, 72, 77Fourier analysis, 48Fourier modes, 12, 28, 31full approximation scheme (FAS), 98full coarsening, 152full multigrid (FMG), 42, 43, 48, 68,

72, 75, 78full multigrid V-cycle, 42, 48, 67, 72,

77, 91full weighting operator, 36, 43, 44, 71,

72, 79, 80, 84, 92, 93

Galerkin, 180Galerkin condition, 75, 153Gauss divergence theorem, 181Gauss–Seidel method, 4, 10, 11, 22,

28, 29, 71Gaussian elimination, 4ghost points, 113global coarse grid, 167global grid, 163Gram–Schmidt method, 117grid complexity, 154

half-injection, 66, 71

191

192 Index

Helmholz equation, 2high-frequency modes, 19, 86

immediate replacement algorithm, 61inexact Newton’s method, 105injection, 35, 43, 71inner product, 28, 77, 91interface points, 167interpolation, 34, 36, 43, 71, 72, 80,

81, 92, 143interpolation weights, 144iteration matrix, 9, 11, 16, 19, 28iterative method, 4

Jacobi method, 4, 8Jacobian, 96

Klauss Stuben, 140

level of discretization error, 60, 67, 76,91

lexicographic ordering, 2line relaxation, 64line relaxation/full coarsening, 123linear stationary iteration, 43local fine grid, 163, 167local mode analysis, 48low-frequency modes, 19, 86

µ-cycle, 42M-matrix, 3, 138matrix norm, 16multilevel methods, 4

neighborhood, 142nested iteration, 33Neumann boundary conditions, 4, 113Newton’s method, 96, 97Newton-multigrid, 96nodal basis expansion, 180nonlinear Gauss–Seidel, 95norm, 7, 64normal mode analysis, 48null space, 34, 36, 79, 80, 85–87, 92,

93

operator complexity, 154operator interpolation, 129orthogonality, 77, 86oscillatory modes, 19, 21, 80, 81, 83

periodic boundary conditions, 5Poisson equation, 2positive definite, 2, 3, 28, 91positive operator, 178prolongation, 34

quadratic functional, 178

range of interpolation, 73, 81, 84–86,93

rank, 34, 36, 79rate of convergence, 17Rayleigh–Ritz, 180red-black coarsening, 144, 152red-black Gauss–Seidel, 11, 44, 64, 71relaxation method, 4, 7, 87residual, 7, 32, 33, 74residual equation, 8, 73, 74restriction operator, 35, 36, 79Richardson iteration, 28, 29

self-adjoint, 130, 177, 178semi-coarsening/line relaxation, 125semi-coarsening/point relaxation, 121slave points, 167smooth modes, 19, 21, 80, 83smoothing factor, 22, 48, 49smoothing property, 25, 83smoothing rate, 75spectral radius, 16, 29square integrable functions, 177stationary linear iteration, 10, 11, 43stiffness matrix, 182storage, 46strong dependence, 140strong influence, 140symmetric Gauss–Seidel, 11symmetric positive definite, 2, 28, 91

tau correction, 98Taylor series, 96two-grid correction, 37, 38, 78, 82, 91two-point boundary value problem, 1

undirected adjacency graph, 137

V-cycle, 40, 42, 43, 46, 47, 61, 64, 71,72, 77, 90, 91

variable-coefficient problems, 130variable-mesh problems, 128

Index 193

variational property, 36, 75, 84, 153,166, 177, 186

W-cycle, 42wavenumber, 12weak form, 181weakly connected neighbors, 142weighted Jacobi method, 9, 11, 12, 19,

22, 28, 29, 37, 71, 138work units, 47, 61, 68

zebra relaxation, 64

View publication statsView publication stats