Applied Math 225: lecture 11

I Last time:I The fast Fourier transformI Spectral methods

I Today:I Final project organiztionI Domain decomposition

Final project

I Final project worth 35% of grade

I Due on Thursday May 16th at 11:59 PM on Canvas, alongwith associated code

I In general, it should be completed in teams of one, two, orthree people.

I n ≥ 4 person projects will be allowed with instructorpermission. I expect to see a plan for how the work will bedivided.

I If you’re looking for teammates, try posting to Piazza ortalking to the teaching staff

Very rough length guidelines

Team members Pages

1 92 143 18n b9.5n0.6c

The precise length of write-up is not important. Scientific contentis more important.

AM 225: final project topic

I Find an application area of interest and apply methods fromthe course to it.

I Project must involve some coding. No purely theoreticalprojects allowed. Any programming language can be used.

I It is encouraged to take problems directly from research,within reason. It should be an aspect of a project that iscarried out for this course, as opposed to something alreadyongoing.

AM 225: project proposal

By April 5th at 5 PM, each team should arrange a half-hourmeeting with Chris or Nick to discuss a project idea and direction.

Four points automatically awarded for doing this.

Nothing written is necessary—only the meeting is required.However, feel free to bring documents, papers, or other resourcesto the meeting.

Total grade for project: 60 points. A detailed breakdown is postedon the website.

Domain decomposition

There are a wide variety of different numerical approaches fornumerical linear algebra, each with its own strengths. For example:

I BLAS/LAPACK: optimized for dense linear algebra, and basedupon direct solution algorithms such as LU, QR, Cholesky, etc.

I Krylov methods: well-suited to arbitrary sparse matrix algebra.

I Fast Fourier transform: efficient for high-accuracy problemson structured grids.

I Multigrid: very efficient for sparse linear systems arising inphysical PDE problems.

In practice, we often encounter problems that are composed ofparts that are suited to different methods, or are too large to fit ona single processor. Domain decomposition allows us to split up alarge linear system into components. Additionally, the componentsmay be computed in parallel.

Case I: Non-overlapping grids

Consider solving the Poisson equation with zero Dirichlet boundarycoefficients on the domain shown below.

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19

20 21 22

23 24 25

26

27

The domain can be decomposed into (1) a square grid with N = 4,(2) a square grid with N = 3, and (3) two connecting gridpoints.

We know how to solve the Poisson equation on the two squaredomains using the FFT.

Case I: Non-overlapping grids

Let the solution vector be v = (v1, v2, v3) ∈ R27 be broken up intothe three sets of gridpoints. Let f ∈ R27 be the correspondingsource term.

Write the matrix equation as Av = h2f where h is the grid spacing.With the second-order finite difference stencils, A has the form

A =

A11 0 A13

0 A22 A23

AT13 AT

23 A33

.

Key observation: By construction the A12 term is absent and thereis no direct coupling between domain 1 and domain 2.

Schur complement

Performing a block LDU decomposition yields

A =

I 0 00 I 0

AT13A−111 AT

23A−122 I

I 0 00 I 00 0 S

A11 0 A13

0 A22 A23

0 0 I

where

S = A33 − AT13A−111 A13 − AT

23A−122 A23

is defined as the Schur complement. The inverse of A is

A−1=

A−111 0 −A−1

11 A13

0 A−122 −A−1

22 A23

0 0 I

I 0 00 I 00 0 S−1

I 0 00 I 0

−AT13A

−111 −AT

23A−122 I

.

Schur complement

In this form, solving the system Av = h2f is broken down intosimpler components:

I Performing A−111 and A−122 : we can use our previous solvers tocompute these operations.

I Multiplying by A13 and A23: this is simple to do, especiallysince these matrices are sparse.

I Performing the inverse S−1: this is the most challenging part.However, S is smaller than the original matrix since it onlyinvolves the connecting gridpoints.

Options for inverting the Schur complement

Option 1: Compute S exactly. This can be done by performing thematrix products Sek for unit vectors ek in domain 3. Each matrixproduct requires one solve of A−111 and A−122 . Since S is an SPDmatrix, it can be solved efficiently via Cholesky factorization.

Option 2: Use a Krylov subspace method such as conjugategradient. Since we can efficiently multiply by S , this is attractive.In addition, S generally turns out to be better conditioned than theoriginal matrix.

Generalization to more subdomains

Consider k > 2 non-overlapping subdomains, with the boundarygridpoints indexed as k + 1. The matrix has the form

A =

A1,1 0 A1,k+1

. . ....

0 Ak,k Ak,k+1

AT1,k+1 · · · AT

k,k+1 Ak+1,k+1

and the Schur complement is

S = Ak+1,k+1 −k∑

j=1

ATj ,k+1A

−1j ,j Aj ,k+1.

Note that many steps (e.g. the inversions A−1j ,j ) can be computedin parallel.

Case II: Overlapping domains

Consider solving the Poisson equation with zero Dirichlet boundaryconditions on the domain shown below.

1 2 3 4

5

9

10 19

23

25 26 27

12

14

13

15

16 17

24

6 7 8

10

11 18

21 2220

This domain consists of two overlapping squares with N = 4. Itcan be broken down into (B) points in the first square but not thesecond, (C ) points common to both squares, and (D) points in thesecond square but not the first.

Case II: Overlapping domains

The matrix has the form

A =

AB,B AB,C 0AC ,B AC ,C AC ,D

0 AD,C AD,D

.

The representation can be amalgamated in two ways:

A =

(ABC ,BC ABC ,D

AD,BC AD,D

)=

(AB,B AB,CD

ACD,B ACD,CD

).

Here the BC and CD suffixes correspond to the combinedgridpoints in those domains. Similarly the solution vector can bedecomposed as

v =

(vBCvD

)=

(vBvCD

).

Iterative Schwarz methodsFor overlapping domains, we introduce two different iterativeapproaches. Define b = h2f to the be source term and considersolving Av = b.

Given a solution vi , the additive Schwarz method proceeds asfollows to obtain a better answer vi+1:

1. Calculate r = b − Avi .

2. Calculate w = A−1BC ,BC rBC .

3. Calculate x = A−1CD,CDrCD .

4. Define the new solution1 as

vi+1 = vi +

wB

(wC + xC )/2xD

.

1Note that this is a bit different to the Applied Numerical Linear Algebratextbook. In the C domain, Demmel takes a sum instead of an average. I havefound an average leads to better convergence.

Comments on the additive Schwarz method

Note that steps 2 and 3 can be calculated efficiently using the fastsolvers on the square grids. Steps 2 and 3 can also be done inparallel.

In step 3 of the additive Schwarz method, we use the originalresidual r from step 1, even though we have new information fromdoing the solve on the BC in step 2. This suggests a modifiedapproach.

The multiplicative Schwarz method

The multiplicative Schwarz method recomputes the residual beforedoing the solve on CD.2 This leads to the following iteration:

1. rBC = (b − Avi )BC

2. vi+ 12

= vi + A−1BC ,BC rBC

3. rCD = (b − Avi+ 12)CD

4. vi+1 = vi+ 12

+ A−1CD,CDrCD

This typically gives better performance than the additive Schwarzmethod. However, the ability to parallelize in a straightforwardmanner is lost, since the second solve A−1CD,CD incorporatesinformation from the first.

2This is similar to the logic behind moving from the Jacobi method to theGauss–Seidel method. See the AM205 notes.

Finite element methods

I The finite element methodis a framework fordiscretizing and solvingPDEs, especially ellipticPDEs

I Solution is represented as asum of simple functions(elements)

I Widely used in science andindustry, such as for solidmechanics, and thermalanalysis.

Wheel rim analysis in COMSOL

Multiphysics R©. The rim is covered in

a triangular mesh, and simple

functions are specified on these

triangles, from which the solution is

constructed.

Comparison with finite-difference methods

Finite element methods

I make it easier to deal with complex boundary conditions,

I provide more mathematical guarantees about convergence.

Finite difference methods

I are simpler to implement, and sometimes more efficient forthe same level of accuracy,

I are easier to apply to a wider range of equations(e.g. hyperbolic equations).

These are just broad generalizations though—both methods have alarge body of literature, with many extensions.

Frequently, the two approaches lead to similar3 numericalimplementations.

3And in some cases identical.

Book reference

We will make use of the following books:

I Claes Johnson. Numerical Solution of Partial DifferentialEquations by the Finite Element Method. Dover, 2009.4

I Thomas J. R. Hughes. The Finite Element Method: LinearStatic and Dynamic Finite Element Analysis. Dover, 2000.5

I Dietrich Braess, Finite elements: Theory, fast solvers, andapplications in solid mechanics, Cambridge University Press,2007.6

4A good general introduction.5Comprehensive, with a particular emphasis on solid mechanics.6A more technically rigorous treatment of the subject.

Overview

The main idea is to formulate an elliptic PDE as a variationalminimization problem over a suitably-defined function space.

To obtain a numerical method, we approximate this function spaceby a finite-dimensional subspace, the finite-element space.

There are some subtleties to choosing the correct function space touse. We begin by considering a specific example that motivates theneed for a careful treatment.

The need for a careful treatment

Consider a domain Ω that is an open subset of Rn. We can alsodefine the closure7 Ω and the boundary ∂Ω.8

The most natural function spaces to use are C k(Ω), the space ofall functions on Ω that are differentiable k times.

However, even for some simple cases, these spaces can posetheoretical difficulties, such as a loss of regularity.

7This is found by adding all limits of sequences in Ω to it. For example ifΩ = (0, 1), the sequences xn = 1/n and yn = 1− 1/n converge to 0 and 1,respectively. Thus Ω = [0, 1].

8The boundary is technically defined as ∂Ω = Ω \ Ω. Thus ∂Ω = 0, 1 inthis example.

The need for a careful treatment

Consider the domain with reentrant corner

Ω = (x , y) ∈ R2 : x2 + y2 < 1 and (x < 0 or y > 0).

Identify z = x + iy ∈ C with (x , y). The function w(z) = z2/3 isanalytic in Ω, and so its imaginary part u(z) = Imw(z) is aharmonic function that satisfies

∇2u = 0

on Ω with boundary conditions

u(e iϕ) = sin2ϕ

3for 0 ≤ ϕ ≤ 3π

2,

u = 0 elsewhere on ∂Ω.

Note that w ′(z) = 23z−1/3 and thus even the first derivatives of u

are not bounded as z → 0.

The example function u(x , y)

−1−0.6−0.2

0.20.6

1 −1−0.6

−0.20.2

0.61

0

1/3

2/3

1

x

y

u(x , y)

Note how the gradient of u increases without limit near(x , y) = (0, 0). This function is not in C 1(Ω), the space ofdifferentiable functions on Ω.

The need for a careful treatment

The example described on the previous slides is physicallyreasonable—u(x , y) could represent the shape that a soap filmwould take when bounded by a wire in the shape of ∂Ω.

As mentioned, the finite-element method is based on formulatingminimization problems over a suitably chosen function space. Eventhough C 1(Ω) appears a natural choice, this example highlightsthe theoretical difficulties of using it.

We now move onto a model problem, but we bear this issue inmind going forward.

One-dimensional model problem

We now consider a model problem, (D),

−u′′(x) = f (x) for 0 < x < 1,

u(0) = u(1) = 0,

where f is a continuous function. Integrating this equation twiceshows that there is a unique solution.

This problem can be used to model several different physicalscenarios.

Physical analogsScenario 1: Consider an elastic cord under tension between 0 to 1.Let u(x) represent a small downward displacement, and f (x)represent an applied downward force density.

(x)

u(x)

f

Scenario 2: Let u(x) be the temperature in a bar whose ends arekept at a fixed temperature of zero.9 Let f (x) be an applied heatsource along the bar.

(x)

u(x)

f9u could be expressed as the temperature relative to some baseline.

Alternative formulations

We now consider two alternative formulations of the modelproblem. First, introduce an inner product on two functions v andw as

(v ,w) =

∫ 1

0v(x)w(x)dx .

Introduce the function space

V =

v ∈ C ([0, 1]) :

v ′ is piecewise continuous and boundedon [0, 1] and v(0) = v(1) = 0

and the linear functional

F (v) =1

2(v ′, v ′)− (f , v).

Alternative formulations

Formulating the problem as a minimization yields

(M) Find u ∈ V such that F (u) ≤ F (v) for all v ∈ V .

Formulating the problem in variational form yields

(V) Find u ∈ V such that (u′, v ′) = (f , v) for all v ∈ V .