tutorial 2 introduction to numerical schemes - technion · pdf file236861 numerical geometry...

236861 Numerical Geometry of Images

Tutorial 2

Introduction to numerical schemes

Guy Rosman

c©2012

Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes

Classifying PDEs

Looking at the PDEAuxx + 2Buxy + Cuyy + Dux + Euy + Fu + .. = 0,and its discriminant, B2 − AC , the equation is said to be:

I Parabolic if B2 − AC = 0 - for example, the heat equation,ut = uxx

I Hyperbolic if B2 − AC > 0 - for example, the second orderwave equation, utt = uxx

I Elliptic if B2 − AC < 0 - for example, Laplace equation,uxx + uyy = 0


Classifying PDEs (Cont.)

This can be generalized to second order equations in N dimensionsvia the positive definiteness of the second order operator involved:

(∇)T(

A BB C

)(∇) u + {lower order terms} = 0


Heat equation

The simple parabolic PDE

∂u

∂t= K

∂2u

∂2x

with the initial values

u(0, x) = u0(x)

and some boundary conditions is called the (one-dimensional) heatequation.


I This equation describes the thermal energy transport in a 1Drod, where u(t, x) describes the temperature at a point x attime t and K denotes the thermal conductivity constant.

I In 2D it describes the effect of defocusing on an image inoptics

I The solution is given by convolution of the initial data with aGaussian kernel:

u(x , t) = G (x , t) ∗ u0(x) =∫RG (x̄ , t)u0(x − x̄)dx̄


Boundary conditions

Dirichlet:

u(t, 0) = a u(t, 1) = b

Neumann:

ux(t, 0) = a ux(t, 1) = b

Reflective - Neumann with a = b = 0.Mixed boundary conditions:

u(t, 0) = a, ux(t, 1) = b.

Periodic:

u(t, 0−) = u(t, 1+)


Discretization of the heat equation

Replace the continuous system of coordinates (t, x) by a discretegrid (n,m) = (n∆t,m∆x), and the continuous function u(t, x) bya discrete version unm = u(n∆t,m∆x).

Figure: The grid


Finite differences

1. Replace the first-order time derivative ut(t, x) by a forwardfinite difference in time

D+t unm =

un+1m − unm

∆t≈ ut(t).

2. Replace the second-order space derivative uxx(t, x) by acentral difference in space

D0xxu

nm =

unm+1 − 2unm + unm−1

(∆x)2≈ uxx(x).


Finite differences’ motivation

From the Taylor expansion,

u(t +∆t) = u(t) + ut(t)∆t +1

2utt(t)(∆t)2 + ...

we obtain

ut(t) =u(t +∆t)− u(t)

∆t− 1

2utt(t)∆t + ...

=u(t +∆t)− u(t)

∆t+ O(∆t).

This first-order approximation of the first derivative is called theforward finite-difference approximation and is denoted by D+

t unm.


Finite differencesForward difference

D+t unm =

un+1m − unm

∆t≈ ut(t) + O(∆t).

In the same manner, backward difference can be defined

D−t unm =

unm − un−1m

∆t≈ ut(t) + O(∆t).

To approximate a second-order derivative, use the centraldifference approximation

D0xxu

nm =


(∆x)2≈ uxx(x) + O((∆x)2).

Note: We can also write

D0xxu

nm = D−

x D+x unm.


Finite differences

Proving the order of convergence – by writing a fourth order Taylorseries and canceling out terms,


(∆x)2=

unm + (ux)nm ∆x + 1

2 (uxx)nm (∆x)2

+ 13! (uxxx)

nm (∆x)3 + 1

4! (uxxxx)nm (∆x)4

−2unm+unm − (ux)

nm ∆x + 1

2 (uxx)nm (∆x)2

− 13! (uxxx)

nm (∆x)3 + 1


(∆x)2=

12 (uxx)

nm (∆x)2 + 1

4! (uxxxx)nm (∆x)4+

12 (uxx)

nm (∆x)2 + 1


(∆x)2


Finite differences

= uxx +1

12(uxxxx)

nm (∆x)2

= uxx +O((∆x)2

)


The discrete heat equation

The continuous equation ut(t, x) = Kuxx(t, x) is replaced by

un+1m − unm

∆t= K


(∆x)2

or

un+1m = unm +

∆t

(∆x)2K(unm+1 − 2unm + unm−1

),

This scheme is explicit (called also Euler scheme) and very simpleto implement.


I We denote the continuous solution by U = u(t, x) and thediscrete solution by u = unm.

I We denote the continuous PDE operator L = ∂t − K∂xx .

I We denote the discrete operator L∆t,∆x = D+t − KD0

xx .


Stability and Convergence

1. Convergence: the discrete solution converges to thecontinuous solution

lim∆t,∆x→0

‖u − U‖ .

in some norm.

2. Consistency: the discrete difference operator converges to thecontinuous

lim∆t,∆x→0

‖L(u)− L∆t,∆x(u) ‖ = 0

for every bounded u.


Stability

3. Numerical stability: noise from initial conditions, numericalerrors, etc. is not amplified.The numerical scheme is un+1 = Nun

Formally, stability means

∀N > 0 ∃C (N) > 0 s.t. ∀n ≤ N ‖N n‖ ≤ C (N).

If C is a constant for all N, this is known as unconditionalstability.

Lax’s equivalence theorem: Given a well-posed initial valueproblem and a finite-difference approximation to it that satisfiesthe consistency condition, stability is the necessary and sufficientcondition for convergence.

consistency: stability ⇔ convergence


The CFL condition for the wave equation

I The (Courant-Friedrichs-Lewy (CFL) condition) is a necessarycondition for advection problems limiting the maximum timestep allowed in the difference equations.

I Specifically, this condition states that the domain ofdependence of the differential equation must be included inthe domain of dependence of the difference equation.

I Note: This does not apply to equations where informationflows “instantly” such as the heat equation – but the termCFL condition is often still used to denote stability limitationson step size in these cases.


A condition for stability of the discrete heat equation

Denote r = K ∆t(∆x)2

. The finite difference scheme is

un+1m = unm + r

(unm+1 − 2unm + unm−1

)= (1− 2r)unm + runm+1 + runm−1

If 1− 2r > 0, we can write:

un+1m ≤ (1− 2r)max

munm + r max

munm + r max

munm

Then un+1m ≤ max

munm(1− 2r + 2r) = max

munm.

(maximum principle property)


Observation: Maximum principle implies stability of the scheme.

Lemma(Maximum principle)If K ∆t

(∆x)2≤ 1/2, then

minm

u0m ≤ unm ≤ maxm

u0m.


0 10 20 30 40 50 60 70−8

−6

−4

−2

0

2

4

6

8Initial data

0 10 20 30 40 50 60 70−5

−4

−3

−2

−1

0

1

2

3

4

5 Solution to 1−D heat equation

dt=0.0008

0 10 20 30 40 50 60 70−5

−4

−3

−2

−1

0

1

2

3

4

5Solurion to 1−D heat eq

dt=0.004

0 10 20 30 40 50 60 70−4

−3

−2

−1

0

1

2

3

4x 10

70

dt=0.006

Figure: Computation of the discrete heat equation with Neumannboundary conditions. (h = 0.1) Top left: Initial data. Top right: stablesolution of the heat equation conditions ∆t = 0.0008. Bottom left:stable solution of the heat equation for ∆t = 0.004. Bottom right:unstable solution (∆t = 0.006).


The 2D heat equation. Numerical scheme.

ut = K (uxx + uyy )

un+1m,p = unm,p + K

( ∆t

(∆x)2(unm+1,p − 2unm,p + unm−1,p)+

+∆t

(∆y)2(unm,p+1 − 2unm,p + unm,p−1)

).

A stabilty condition suggested by Courant, Friedrichs and Lewy:

K( ∆t

(∆x)2+

∆t

(∆y)2

)≤ 1

2.

If ∆x = ∆y , then the condition is:

K∆t

(∆x)2≤ 1

4.


Stability condition by Fourier analysis.

Another way to check the stability of a numerical scheme is byFourier analysis. Looking again at the heat equation:

ut = uxx

using the linearity of the equation we assume a solution of theform unm = λne iξm∆x , where for stability we would like ‖λ‖ < 1.For the usual explicit scheme, we obtain:

un+1m − unm

∆t=


(∆x)2−→

λn(λ− 1)e iξm∆x =∆t

(∆x)2λn(e iξ∆x + e−iξ∆x − 2)e iξm∆x −→

λ = 1 +∆t

(∆x)2(e iξ∆x + e−iξ∆x − 2)


Stability condition by Fourier analysis.

finally arriving at

‖λ‖ = ‖1 + ∆t

(∆x)2(2cos(ξ∆x)− 2)‖ ≤? 1.

Again we see that the condition ∆t(∆x)2

< 12 guarantees stability.

The same method extends to higher dimensions.In the case of a finite time interval T , we can settle for a weakerform of stability, by demanding merely

‖λ‖ < 1 + K̃T

.This condition is known as the von-Neumann condition. Theanalysis is often referred to as von-Neumann analysis.


The 2D heat equation. Numerical example

Figure: Application of the heat eq. on a gray-scale image.


Implicit finite difference schemes for the heat equation

un+1m − unm

∆t= K

un+1m+1 − 2un+1

m + un+1m−1

(∆x)2

I Main advantage of implicit schemes: are unconditionallystable (i.e. stable for all time-steps and thus we can take largestep times).(I −∆tA)Un+1 = Un.

I Requires solving a linear system of equations. The matrix(I −∆tA) is tridiagonal and can to be inverted using theThomas algorithm (we will describe it in detail later on).

I Note: Assuring stability does not prevent the accuracy fromdeteriorating..


tutorial 2 introduction to numerical schemes - technion · pdf file236861 numerical geometry...

Documents