math ee draft

8/10/2019 Math EE Draft

1/23

Introduction

Heraclitus, a famous Greek philosopher, has said, Everything changes, and nothing stands still.

Indeed, rates of change are essential to many principles and laws describing the behavior of the real

world. Differential equations, containing derivatives, encapsulate these changes in mathematical terms.

They are useful because through humans observations of the natural world, rates of change are easily

discovered. Their broad applicability in helping to model processes in almost every scientific field,

whether it be the motion of fluids or the growth of economies, motivates the study of their solutions as

an important field in applied mathematics. The investigation of differential equations is, for me, an

important endeavor as someone interested in physics because differential equations and their solutions

are cornerstones of the mathematical models that inform physics.

By solutions of differential equations I mean the set of functions that satisfy the differential

equation, often only on a specific interval. Because differential equations involve derivatives of functions,

integrating is required to solve these equations. It is not always feasible or possible to integrate and

resolve functions analytically into an expression involving only a finite number of operations. In fact, a

very small subset of differential equations is solved explicitly. Therefore, for most differential equations

with real-world applications, one needs to approximate these solutions numerically. The famous

mathematician Leonhard Euler published one such method to approximate these solutions, which led to a

variety of other techniques to be developed. Among them are Runge-Kutta methods and multistep

methods. Today, these methods are coupled with the power of computers to help elucidate and modify

mathematical models of a vast array of behaviors and processes that are important to advancements in

knowledge.


2/23

However, because these solutions are only approximated,1it is important that error analysis is an

integral part of solving these equations numerically. Mathematicians are concerned with the size and

analytic form of errors. Each numerical method, depending on their complexity, has a different error term

and thus, quantifying and analytically representing these errors are important problems in mathematics.

For this essay, I will only be investigating first-order differential equations (what the hell are they???) This

leads to the research question:

How can we use Taylors Theorem to analyze and quantify truncation errors when using Runge-

Kutta methods to approximate first-order differential equations?

When one mentions error in mathematics one means the difference between the approximate

solution and the exact solution.2In this essay, I am primarily concerned with truncation errors, or errors

due to an inability to calculate exact formulas due to the infinite number of terms. I must reduce the

truncation by performing more calculations at the expense of more complexity and time required. What I

am essentially doing is studying how the remainder of Taylor polynomials diminishes as I progressively

use more terms in the infinite Taylor series in the quest for more accurate results.

Due to a very powerful theorem called Taylors Theorem, it is possible to analyze truncation error

quantitatively. I can even provide an upper bound for the errors. In this essay I will connect how Taylors

Theorem provides a general framework to provide an analytical formula for the truncation errors of

different methods of approximating the solutions to a differential equation with illustrations of specific

methods for approximating the solutions, of which there are two broad types: the Runge-Kutta methods

and the multistep methods. [MY THESIS IS: COMPARE AND CONTRAST PLEASE!!!]

1http://homepage.math.uiowa.edu/~atkinson/NA_Overview.pdf

2http://www.math.unl.edu/~gledder1/Math447/EulerError


3/23

Definitions of Errors

Suppose I have a function that varies with time: where is unknown. I know thevalue of

at time

:

and the rate of change of

with respect to time. In other words I know

the differential equation of form:

I am interested in the local and global truncation errors of different methods of approximating

differential equations. In other words I want to find how closely the approximation approaches thereal value

. I will do so by starting out from what I know,

and project into the future

using the rate of change. Similar to the differential formula , if I let then Ican find by using the recursive formula:

Thus, if I know , I can use this formula to find , then , ,. The function

, which depends on the values of

and the slope at that point, is called the

increment function. It is an approximation of the change between and . It can be as simple as thefirst derivative (which yields Euler method) or much more complicated, yielding different methods.

The increment function is only an approximation of the real change. The local truncation error is

defined to be the error in the approximation applied over one single step. Thus, if we know , thenwe can approximate and calculate the local truncation error as a function of step-size ():

The global truncation error is defined to be the error in the approximation over all steps. If

we know and want to approximate , then I can calculate the global truncation error as a functionof step-size (


4/23


5/23


6/23

Theorem. Let the real valued function be k+1 times differentiable on an open intervalwithcontinuous from the interval between and . Then we have

where

() for some value such that Proof.We shall prove the theorem using a type of reverse mathematical induction. What we have

to prove is that for all natural numbers

we have:

To prove this inductively, we first have to consider the basis case. Our basis case for this proof shall

be looking at the

-th derivatives relationship to its derivative, the

-th derivative. Knowing this,

we can then work backwards towards the first derivative, and finally, the function. In other words, I am

considering the case . Since the function is hypothesized to be times differentiable and thatits -th derivative is continuous on the interval we can apply the mean value theorem toonthis interval:

for some

This agrees with the hypothesis of the theorem.

When we integrate both sides of this function from to using the dummy variable , we have


7/23

Since the antiderivative of the -th derivative is simply the -th derivative, we use thesecond part of the fundamental theorem of calculus, which states that to find the definite integral fromto

we subtract the values of the antiderivatives at

, the upper bound, and

, the lower bound.

Calculating all this, in addition to some rearranging, yields:

Thus we have provided the basis case and the intuition for our proof. In order to prove the

inductive step, I am assuming that the hypothesis is true for and I have to prove it is true for .Thus we have to prove:

I will do this by considering the equality for the case of derivatives, which I have assumed true,and taking the definite integral from to , substituting the variable for the dummy variable :

which, indeed, yields

because


8/23

where is any constant, which in this example are the derivatives ofat .We have shown that if the equation is true for case , then we can show that the equation will be

true for case , although the way up to Equipped with Taylors formula, we shall turn our attention to describe a variety of different

numerical methods that are used to approximate solutions to differential equations and express their

error analytically.

The most famous example of Taylors theorem is the Mean Value Theorem, which states that if a

functionis differentiable then for every two points and in the domain of, there exists a point at which the derivative is equal to the slope of the secant line between the pointsand , which we can write as:

For this particular example, let the two points be and . By the mean value theorem we have:

for some which can be rearranged into

We can see that this fits into the Taylors polynomial of degree 1 for , with the remainder term

in keeping with Taylors Theorem:


9/23

This gives us a bound for the value of the error for the Taylors first-degree polynomial

approximation tobecause if we find , the maximum value forwhere , then: Taylors theorem, which gives us the analytic form of the error, along with Big -O notation, which

gives us the rate of error growth compared to other functions, allow us to analyze methods for their

errors.

While it will not be shown here, Taylors formulaalso generalizes to partial differential equations in

many dimensions.

Eulers Formula

Eulers methoduses information about the first derivative and the initial value and assumes that

the function acts linearly locally to project the value of the function into the future. Suppose I have a

function , with initial condition and I want to approximate the solution to thedifferential equation

Eulers method has the recursive formula:5

where is the approximation to the equation at time and is the step size .

5


10/23

Geometrically, the problem is to approximate the shape of a continuous curve using polygonal lines.

By taking short steps into the future, determining the how the curve will change at that point using the

slope at that point, we can hopefully create a polygonal curve that does not deviate too far from the

curve we want to approximate.

To derive Eulers formula using Taylors Theorem, I first assume that the true soluti on has twocontinuous derivatives and is twice-differentiable on an open interval that I am approximate the solution

on. Then, with Taylors Theorem, I have

where

Since I know the initial condition , I can derive the following from Taylors Theorem where

Using the value of , I can use the recursive formula to find the value of and so on.Eulers method simply ignores the

term, which means that the local truncation error is:

( )

This demonstrates that when , the step size approaches 0, the local truncation error isproportional to . If I can determine the maximum value that reaches in the interval to be , then I can bound

Due to the ease of Eulers method, one trade-off

is that the local truncation error is proportional to a


11/23

lower power of the step size, meaning that the error is greater for small .The global truncation error of Eulers method tells us the error accumulated by the method when I

approximate

through

steps, starting with

. This is, intuitively, the product of

, the number of

steps and the local truncation error of one single step. If we set the step size to be then thenumber of steps is

. Therefore, the global truncation error can be bounded to be:

We then, can expect the global truncation error to be proportional to . This shows that Eulers

method is first order.

Figure 16.

Midpoint Method

If I know the derivative of the function at a point, then the function is locally linear, increasing or

decreasing on some interval around that point. Let us choose the step size

, to approximate

the function such that the function is locally linear in the interval . This means that thederivative approaches as approaches . Eulers method only uses the left-handestimate of the derivative, its value at time to project the value of at time . If I can use theestimate of the derivative at time , at the center of the interval, then the prediction will have ahigher order error term.

The midpoint method is given by the recursive formula:

[ ]6Lecture 48. Numerical ODEs Single Variable Calculus Coursera


12/23

Intuitively, I first compute using Eulers formula, then I take the midpoint of the segmentjoining the points and . I then find the derivative of the function at that midpoint,and use that slope instead in my projection to find a more accurate value of

.

Figure 2.7

I know that the error of the

midpoint method will be less than

that of Eulers method, becausewe

are sacrificing computational

simplicity for accuracy.

To find the error, I first find the Taylor series around the midpoint of the interval * + which gives, according to Taylors theorem

Thus, the exact value of the function at according to the Taylor series is:7Lecture 48. Numerical ODEs Single Variable Calculus Coursera


13/23

[

]

The exact value of the function at according to the Taylor series is

[

]

Finding I have [ ]

[ ] We can see that the final step approximates the midpoint method, but we omit the error term.

However, since is an exact value of the differential equation solution I want to approximate atthe midpoint, and I only know the approximations by approximating using Eulers method andthen finding the midpoint of the segment, I have to find the error of * +. fromthe exact value * +. To do this, I find the value of from the Taylor seriesexpansion from point which I know exactly due to the given conditions of the initial valueproblem:


14/23

()

( )

(

)

For simplicity, I will now denote as . I can compute to be

()

The partial differential equations version of Taylors Theorem applies here. We want to

approximate

*

+from the exact value

*

+. The function changes in

only one variable, so we can use Taylors theorem to state that8

8http://web.mit.edu/10.10/www/Study_Guide/DiffEq.html


15/23

[ ] [ ] [ ] ,[ ]- [ ] {

}

[ ] Therefore, when I multiply by to both side, I have [ ] [ ]

[ ]

To find the local truncation error, I subtract the exact value of the function with the approximated

value

[ ] [ ]

I have established that the local truncation error of the midpoint method is in . The globaltruncation error of the midpoint method is calculated in a similar manner to Eulers method. The global

truncation error will be


16/23

Since the midpoint method has the global truncation error in , it is a second-order method.

We have found a way to provide the intuition for and quantify the errors of different methods to

approximate a numerical solution using Taylor series. Eulers method and the Midpoint method are called

Runge-Kutta methods, which generalize them to any order possible. By continually predicting a solution

at a future time, then finding the midpoint and correcting the slope by finding the slope at an interior

point, these methods are also called predictor-corrector methods. We are essentially projecting the

behavior of the function at a future time, then correcting the behavior by exploiting the continuity of the

function, making sure that as we approach the function, we are getting closer to the true function.

Linear Multistep Methods

Eulers9method and the midpoint method are called one-step methods, because they only take into

account the value from the previous step, discarding any values more than one step before. In fact, any

Runge-Kutta method has multiple stages for each step, but they discard all previous information.

Multistep methods take into account the function values at previous steps, thereby improving the

efficiency. However, the disadvantage is that one must know the values of the function we want to

approximate at many equal intervals.

I want to approximate solution at time of

Integrating both sides I have

( )

9http://www.academia.edu/200568/Linear_multistep_numerical_methods_for_ordinary_differential_equations
http://www.academia.edu/200568/Linear_multistep_numerical_methods_for_ordinary_differential_equationshttp://www.academia.edu/200568/Linear_multistep_numerical_methods_for_ordinary_differential_equationshttp://www.academia.edu/200568/Linear_multistep_numerical_methods_for_ordinary_differential_equationshttp://www.academia.edu/200568/Linear_multistep_numerical_methods_for_ordinary_differential_equations


17/23

( ) The linear multistep method approximates the integral by interpolating the slope, or passing a

curve between the known values of at times and projecting the value attime . A linear multistep method is a linear sum of the approximated functions values and theslopesvalues at previous points. We can write the general linear multistep method equation withsteps,whose values and I know as10:

To find the error, I find the Taylor expansion of the terms around . I only want a linear sum ofand its derivatives at that point so I will combine every other numbers as constants. ( ) ( )

( ) ( )

Therefore every single terms in this sum consists of like terms, which means that when we combine them,

we have according to Taylors Theorem

10

ftp://130.149.13.120/pub/numerik/baerwolf/ode_nonstiff_I.pdf
ftp://130.149.13.120/pub/numerik/baerwolf/ode_nonstiff_I.pdfftp://130.149.13.120/pub/numerik/baerwolf/ode_nonstiff_I.pdfftp://130.149.13.120/pub/numerik/baerwolf/ode_nonstiff_I.pdfftp://130.149.13.120/pub/numerik/baerwolf/ode_nonstiff_I.pdf


18/23

Therefore the local truncation error is,

A linear multistep method has order when subtracted, all the coefficients of the terms where .

Two-Step Adams-Bashforth Method

A simple linear multistep method is called the Adams-Bashforth method, developed by the

mathematician John Couch Adams to solve a problem proposed by F. Bashforth dealing with the motion

of liquid through narrow spaces without gravity.

The two-step Adams-Bashforth method assumes that I know the value of and its slope at twopoints of distance and respectively from my desired value. Then, I can approximate theslope of with the function

( ) We can check that the approximation of the slope gives the desired valuesat time

and

at time

.

Thus, the two-step Adam-Bashforth Method can be derived using the general principle of linear

multistep methods.


19/23

( )

[ ]

I can verify the two-step Adam Bashforth method has error in because


20/23

[ ]

[ ]

The global truncation error is

because

An ExampleConsider the differential equation

Let the step size . I can make a table to show the approximations made by each method


21/23

Eulers method

Midpoint Method

[ ]Adams-Bashforth method

[ ]

0 -0.5 -0.5 -0.5

0.5 -0.875 -0.876953 -0.876953

1.0 -1.25781 -1.26847

1.5 -1.66214 -1.68285

2.0 -2.09421 -2.12069

2.5 -2.55154 -2.57804

3.0 -3.0271 -3.04982

3.5 -3.51392 -3.53155

4.0 -4.00706 -4.01988

4.5 -4.50355 -4.51249

5.0 -5.00178 -5.00783

Real value0 -0.5

0.5 -0.877541

1.0 -1.26894

1.5 -1.68243

2.0 -2.1192

2.5 -2.57586

3.0 -3.04743

3.5 -3.52931

4.0 -4.01799

4.5 -4.51099

5.0 -5.00669


22/23

This illustrates the fact that the midpoint method is more accurate than Eulers method

and is roughly as accurate as the two-step Adam-Bashforth method.

ConclusionThere is a vast range of methods one can use to approximate solutions to differential equations. The

more complex the calculations, i.e. the higher order the method, the more accurate of a solution one

obtains. To arrive at the perfect numerical method to approximate the differential equation, the

numerical analyst has to balance the need for accuracy with the computational power he has at his

disposal.


23/23

[Example and percent error]

math ee draft

Documents