one implementation variant of the finite difference method for solving odes/daes

Computers and Chemical Engineering 28 (2004) 303–309

One implementation variant of the finite differencemethod for solving ODEs/DAEs

B. Wua, R.E. Whitea,∗a Department of Chemical Engineering, University of South Carolina, Columbia, SC 29208, USA

Received 11 December 2001; received in revised form 16 June 2003; accepted 4 August 2003

Abstract

An easy-to-use variant of the finite difference method (FDM), the polynomial finite difference method (PFDM), for the numerical solution ofordinary differential equations (ODEs) and differential-algebraic equations (DAEs), is presented. Compared to the traditional implementationof the FDM, the PFDM approach has two major advantages: straightforward implementation, and easily adjustable accuracy order. Someexamples are presented to compare the numerical solutions of the PFDM to those of other popular ODEs/DAEs methods. These examplesshow that the PFDM also has the following good features: feasible for ODEs/DAEs in the implicit form, capable of self-starting in high orders,and applicable to stiff problems.© 2003 Elsevier Ltd. All rights reserved.

Keywords: The finite difference method; Polynomial; Integration methods; Ordinary differential equations; Differential-algebraic equations; Backwarddifferentiation formulae

1. Introduction

Ordinary differential equations (ODEs) and differential-algebraic equations (DAEs) are commonly encounteredin mathematical modeling of physical systems. Many ap-proaches for the numerical solution of ODEs/DAEs, e.g. theEuler method, the Runge-Kutta method, and the backwarddifferentiation formulae (BDF) method, have been widelyused and extensively studied (Rice & Do, 1995; Holland &Liapis, 1983; Ascher & Petzold, 1998; Brenan, Campbell &Petzold, 1989; Hairer & Wanner, 1991). Generally speaking,high-order explicit methods are always chosen for ODEsproblems, due to their good solving efficiency. However,for DAEs and stiff ODEs, implicit methods must be usedto guarantee the stability of numerical solutions. The finitedifference method (FDM) has been widely used in solvingpartial differential equations and boundary value problems(Rice & Do, 1995; Holland & Liapis, 1983). It can also beapplied to initial value problems (Nguyen & White, 1987;Kimble & White, 1990), as demonstrated by the followingprocedure. A system of ODEs/DAEs can be written in the

∗ Corresponding author. Tel.:+1-803-777-3270;fax: +1-803-777-9597.

E-mail address: [email protected] (R.E. White).

implicit form

f (t, y(t), y′(t)) = 0 (1a)

with the initial condition

y(t0) = y0 (1b)

wheref is a vector ofm functions,y ∈ Rm a vector ofmdependent variables, andt time or a time-like independentvariable. The application of the FDM normally replaces thetime derivativey′(t) with finite difference formulae on someuniform node points in thet coordinate. For Eq. (1), applyingthe three-point finite difference formulae onn + 1 evenlyspaced node points from starting timet0 to ending timetfgives

f

(t, y(t0),

−3y(t0) + 4y(t0 + �t) − y(t0 + 2�t)

2�t

)= 0

(2a)

f

(t, y(t0 + i�t),

y(t0 + (i + 1)�t) − y(t0 + (i − 1)�t)

2�t

)= 0 (2b)

for 1 ≤ i ≤ n − 1

f

(t, y(tf ),

3y(tf ) − 4y(tf − �t) + y(tf − 2�t)

2�t

)= 0 (2c)

0098-1354/$ – see front matter © 2003 Elsevier Ltd. All rights reserved.doi:10.1016/j.compchemeng.2003.06.002

304 B. Wu, R.E. White / Computers and Chemical Engineering 28 (2004) 303–309

where

�t = tf − t0

n(3)

Since there aren + 1 vectors of equations andn vectorsof unknown dependent variables inEq. (2), one vector ofequations inEq. (2) is redundant. Normally,Eq. (2a) isomitted, then solvingEqs. (2b) and (2c)will give the de-sired values for dependent variablesy(t0 + i�t) (1 ≤ i ≤n). For a large number of nodes, sinceEqs. (2b) and (2c)have a sparse structure, special band algorithms are usuallyused to improve the solving efficiency (Nguyen & White,1987). It is obvious that the above solving procedure ofthe FDM is more involved in implementation (discretiza-tion over many nodes and using different equations for dif-ferent nodes) than those of many other integration meth-ods. In addition, to improve the solution accuracy ofEq. (1)with the FDM, which is onlyO(�t)2 in Eq. (2), totally dif-ferent sets of high-order finite difference formulae need tobe used (Kimble & White, 1990). Due to the above disad-vantages, the FDM is usually not preferred for initial valueproblems.

2. The polynomial implementation variantof the FDM

From a Taylor series expansion, we have the expressionfor the dependent variabley(t) in the interval [t0,t0 + h],whenh = �t

y(t) = y(t0) + y′(t0)(t − t0)

+ y′′(t0)2!

(t − t0)2 + · · · + y(n)(t0)

n!(t − t0)

n

+ y(n+1)(t0+, θ(t − t0))

(n + 1)!(t − t0)

n+1 (4)

where θ represents a value between 0 and 1 rearrangingEq. (4)gives an approximation polynomial fory(t)

y(t) = y0 + x1(t − t0) + x2(t − t0)2 + · · · + xn(t − t0)

n

(5a)

where

x1 = y′(t0)· · ·xn = y(n)(t0)

n!

(5b)

with the truncation error is

τ = y(n+1)(t0 + θ(t − t0))

(n + 1)!(t − t0)

n+1 = O(hn+1) (5c)

An approximation for the time derivatives of the de-pendent variables can be easily obtained by differentiatingEq. (5a)

y′(t) = x1 + 2x2(t − t0) + · · · + nxn(t − t0)n−1 (6)

SubstitutingEqs. (5a) and (6)into Eq. (1a)yields

f (t, y0 + x1(t − t0) + x2(t − t0)2 + · · · + xn(t − t0)

n,

x1 + 2x2(t − t0) + · · · + nxn(t − t0)n−1) = 0 (7)

SinceEq. (7) is valid for any time point inside [t0,t0 + h]andx1, x2, . . . , xn could be treated as unknown variables,Eq. (7)can be evaluated onn different node points to providethe necessary equations to solve forx1, x2, . . . , xn. For anintegration steph, if we divide it inton+1 node points witha uniform grid sizeh/n, and by applyingEq. (7)on the lastn nodes, we get

f

t0 + ih

n, y0 + x1

(ih

n

)

+ x2

(ih

n

)2

+ · · · + xn

(ih

n

)n

,

x1 + 2x2

(ih

n

)+ · · · + nxn

(ih

n

)n−1

= 0 (8)

for 1 ≤ i ≤ n

Eq. (8)can be solved forx1, x2, . . . , xn through a nonlin-ear equation solver. Then the dependent variabley(t) at theend of the integration step as well as at any point inside theintegration step can be easily obtained withEq. (5a). At thefirst look, the above polynomial approach has no relation tothe FDM, however, it is actually equivalent to the FDM onminimum node points. For example, the third order polyno-mial satisfies

y

(t0 + h

3

)= y0 + x1

(h

3

)+ x2

(h

3

)2

+ x3

(h

3

)3

(9b)

y

(t0 + 2h

3

)= y0 + x1

(2h

3

)+ x2

(2h

3

)2

+ x3

(2h

3

)3

(9c)

y(t0 + h) = y0 + x1(h) + x2(h)2 + x3(h)3 (9c)

Solving the above linear equations forx1, x2, andx3 gives

x1 = y0 − 6y(t0 + (h/3)) + 3y(t0 + (2h/3)) + 2y(t0 + h)

2h(10a)

x2 = 9y(t0 + (h/3)) − 18y(t0 + (2h/3)) + 9y(t0 + h)

2h2

(10b)

x3 =−9y0 + 27y(t0 + (h/3)) − 27y(t0 + (2h/3))

+ 9y(t0 + h)

2h3(10c)

B. Wu, R.E. White / Computers and Chemical Engineering 28 (2004) 303–309 305

Then the time derivatives of the dependent variables are

y′(

t0 + h

3

)= x1 + 2x2

h

3+ 3x3

(h

3

)2

= −y(t0 + h) + 6y(t0 + (2h/3)) − 3y(t0 + (h/3)) − y0

2h(11a)

y′(

t0 + 2h

3

)= x1 + 2x2

2h

3+ 3x3

(2h

3

)2

= 2y(t0 + h) + 3y(t0 + (2h/3)) − 6y(t0 + (h/3)) + y0

2h(11b)

y′(t0 + h) = x1 + 2x2h + 3x3h2

=11y(t0 + h) − 18y(t0 + (2h/3))

+ 9y(t0 + (h/3)) − 2y0

2h(11c)

Eqs. (11a–c)are the same formulae as those of the thirdorder FDM. Due to the equivalence between the above poly-nomial approach and the FDM, it is natural to treat the abovepolynomial-based method as a variant of the FDM and thusis called the polynomial finite difference method (PFDM)subsequently for simplicity. It is worth noting that there is nosuch equivalence between polynomials and the finite differ-ence formulae if more-than-minimum node points are usedfor the FDM. Since the PFDM approximates both dependentvariables and their time derivatives in the same polynomialform, it can be used for DAEs as well as ODEs. It can alsobe directly utilized to solve higher order ODEs/DAEs in theform

f (t, y(t), y′(t), y′′(t), . . . , yn(t)) = 0 (12)

which are always cast into first order form if solved withother methods. Sincen, the order of the PFDM, can be largerthan one, the PFDM has the good feature of self-starting withhigh orders; and thus it could be used as the starter methodfor multi-step methods, e.g. the Adams–Moulton method.

The PFDM applies a polynomial on the unknown pointsin front of the current starting point to determine the un-known parametersx1, x2 . . . xn in the integration process. Ifsome past solution points are also used in the determinationof x1, x2 . . . xn, effective integration schemes can also beobtained. For example, if only one unknown point in frontof the current starting point and some past solution pointsare used, the BDF method can be derived, as shown belowfor the third order case.

Assuming one future point, the current starting point, andtwo past points are used; the grid size is uniformlyh; andstarting time istn (to be consistent with BDF conventions),applying the third order polynomial on those node pointsgives

y(tn+1) = yn + x1h + x2h2 + x3h

3 (13a)

y(tn) = yn (13b)

y(tn−1) = yn + x1(−h) + x2(−h)2 + x3(−h)3 (13c)

y(tn−2) = yn + x1(−2h) + x2(−2h)2 + x3(−2h)3 (13d)

The time derivative at the future unknown point can be de-termined by usingEq. (13)to be

y′(tn+1) = x1 + 2x2h + 3x3h2

= −2yn−2 + 9yn−1 − 18yn + 11yn+1

6h(14)

where for simplicity the notations for dependent variablesare written as

yi = y(ti) (15)

SubstitutingEq. (14)into Eq. (1a)and evaluating the ob-tained equation at the unknown pointt = tn+1 gives thethird order BDF corrector formula

f

(t, yn+1,

−2yn−2 + 9yn−1 − 18yn + 11yn+1

6h

)= 0

(16)

In fact, just comparing the finite difference formula withthe corrector formula of the BDF method (Brenan et al.,1989) will show directly their correspondence (the differ-ence of time interval used could cause some difference inconstants). Among the implicit schemes derivable from theFDM, the BDF method has been thoroughly researched andwidely used. However, several popular solvers based onthe BDF method, e.g. LSODI and DASSL (Brenan et al.,1989), must start from low order methods with small stepsizes, which causes low starting efficiency. It is desirable thathigh-order schemes, e.g. the implicit Runge-Kutta methodor the PFDM, be used to provide these solvers with goodstaring efficiency. Since the PFDM is closely related to theBDF method, it has the potential to be seamlessly integratedwith existing BDF solvers. For each integration step, the nthorder PFDM requires solvingm × n nonlinear equations,however, the same order BDF method only requires solv-ing m nonlinear equations by utilizing the previous solutionpoints. Thus, high-order PFDMs might not be as efficientas the same order BDF methods after the starting stage. Asimple way to take advantage of both methods is to use thePFDM in the initial stage to provide accurate starting pointsfor high-order BDF methods, which needs to be investigatedin the future.

The stability of the PFDM can be analyzed by solving thefollowing ODE

y′(t) = λy (17a)

with

y(t0) = y0 (17b)

whereλ < 0 andy0 = 0.


Fig. 1. Regions of absolute stability for the PFDM of orderp, p = 1, 2, 3, 4(stable outside the ovals). Re and Im are, respectively, the real coordinateand imaginary coordinate of the complex plane.

Applying the second order PFDM to Eq. (17) yields thesolution at theith step

yi =[

4 + λh

4 − 3λh + (λh)2

]i

y0 (18)

Let

z = λh (19)

then the region of absolute stability of the second order FDMis

Ra ={z :

∣∣∣∣ 4 + z

4 − 3z + z2

∣∣∣∣ < 1

}(20)

Similar analysis can be conducted on the PFDM with otherorders. The implicit Euler (the first order PFDM) methodis well known to be absolute stable (A-stable). As shownin Fig. 1, the second order PFDM is also A-stable; andthe stability region of the PFDM gradually decreases withthe increase of the order. However, only small instabilityregions in the left half plane are shown for the PFDM withlow orders. Comparing the stability regions of the PFDM tothose of the BDF method (Ascher & Petzold, 1998) showsthat, for the same order, the PFDM is more stable than theBDF method.

Polynomials can be efficiently evaluated in computer pro-grams, i.e. for the following polynomial

p = anxn + an−1x

n−1 + · · · + a0 (21)

Table 1The approximate solutions forExample 1using the PFDM

Analytical solution First order PFDM (implicit Euler) Second order PFDM Fourth order PFDM Eighth order PFDM

h = 0.5 0.60653066 0.66666667 0.60869565 0.60653485 0.60653066Relative error (%) – 9.915 0.357 0.001 0.000

h = 5.0 0.67379470E−2 0.16666667 −0.22727273 E−1 0.45423380E−2 0.67334746E−2Relative error (%) – 2373.553 437.303 32.586 0.066

Horner’s method can used to minimize the multiplicationoperations to improve the computation efficiency (Baase,1988), i.e.

p = (· · · ((anx + an−1)x + an−2)x + · · · + a1)x + a0 (22)

With a good solver for nonlinear equations, it is straight-forward to solveEq. (8). A Fortran subroutine, PFDMS(PFDM solver), built by the authors with a nonlinear equa-tion solver HYBRD in Minpack (this numerical package isavailable fromwww.netlib.org), only takes about one hun-dred lines of FORTRAN code (available free upon request).

Some example problems are provided subsequently todemonstrate the features of the PFDM.

Example 1.

y′(t) = −y (23a)

with

y(0) = 1 (23b)

which has an analytical solution

y(t) = e−t (24)

The numerical solutions forExample 1with the PFDMare listed inTable 1. It is obvious that good accuracies can beachieved with high-order PFDMs, even at large step sizes.

When solving DAEs, the approximations for the timederivatives of algebraic variables are not involved. One ex-ample of DAEs problem is given below

Example 2.

y′1(t) = −y1 + y2 (25a)

−12y1 + y2 − 2 = 0 (25b)

with

y1(0) = 1

2(25c)

y2(0) = 9

4(25d)

which has the analytical solutions

y1(t) = 4 − (4 − y1(0)) e−t/2 (26a)

y2(t) = 4 + (4 − y2(0)) e−t/2 (26b)

http://www.netlib.org


Table 2The approximate solutions forExample 2using the BDF (Gear’s) method, the PFDM, and the semi-implicit Runge-Kutta method

Analyticalsolution (y, y2)

Second order Gear’sBDF method

Second orderPFDM

Third order semi-implicitRK (Michelsen)

Third orderPFDM

h = 0.5 1.274197 1.281250 1.272727 1.274434 1.2741542.637099 2.640625 2.636363 2.637099 2.637077

Relative error (%) – 0.554 0.115 0.019 0.003– 0.134 0.028 0.000 0.001

h = 0.5 for 10 steps 3.712703 3.729600 3.711149 3.712949 3.7126573.856352 3.864799 3.855575 3.856474 3.856329

Relative error (%) – 0.455 0.0419 0.007 0.001– 0.219 0.020 0.003 0.001

The numerical solutions forExample 2with the PFDMand two other methods, Gear’s BDF method and a thirdorder semi-implicit Runge-Kutta method (Holland & Liapis,1983; Gear, 1971; Michelsen, 1977), are given inTable 2.The results show that, for the same order, the PFDM alwayshas better accuracy than the other two methods.

The stability of PFDM on stiff problems is demonstratedwith the following ODEs.

Example 3.

y′1(t) = −500.5y1 + 499.5y2 (27a)

y′2(t) = 499.5y1 − 500.5y2 (27b)

with

y1(0) = 2 (27c)

y2(0) = 1 (27d)

which has the analytical solutions

y1(t) = 1.5 e−t + 0.5 e−1000t (28a)

y2(t) = 1.5 e−t − 0.5 e−1000t (28b)

The stability of the explicit Euler method requires the stepsize (Rice & Do, 1995)

h ≤ 2

|λmax| (29)

Table 3The approximate solutions forExample 3using the Euler method and the PFDM

Analyticalsolution (y1, y2)

Explicit Eulermethod

Implicit Euler method(first order PFDM)

Second orderPFDM

Third orderPFDM

h = 0.001 for 10 steps 1.485097 1.362674 1.485571 1.485102 1.4850981.485052 1.362674 1.484594 1.485047 1.485051

Relative error (%) – 8.243 0.032 0.0003 0.0001– 8.241 0.031 0.0003 0.0001

h = 1.0 for 10 steps 0.680999E−4 0.867156E27 0.146484E−2 0.824905E−4 0.70073E−40.680999E−4 −0.867156E27 0.146484E−2 0.824905E−4 0.700737E−4

Relative error (%) – 1.273E34 2.051E3 21.132 2.898– 1.273E34 2.051E3 21.132 2.898

where

λmax = 1000 (30)

for this example. The step size needs to be less than0.002 to maintain the stability of the explicit Eulermethod. However, as shown from the results inTable 3,the stability is not a problem for the PFDM at muchlarger step sizes. Though the implicit Euler method isalso stable, the accuracy of high-order PFDMs is muchbetter.

For partial differential equations (PDEs) or partialdifferential-algebraic equations (PDAEs), using the methodof lines (MOL), the numerical solution can be achievedby the integration of the time domain conducted overthe discretisized space domain. High-order PFDMs inthe time domain can be combined with high-order fi-nite difference formulae in the space domain to ob-tain a good accuracy of the finial results. The follow-ing parabolic PDE is used to compare the PFDM to theCrank–Nicolson method for integrating over the time do-main.

Example 4.

∂θ

∂t= ∂2θ

∂x2(31a)

with the initial condition

θ(x, t)|t=0 = sin(πx) (31b)


Table 4The approximate solutions forExample 4using the Crank–Nicolson method and the PFDM

Internal nodes (nx = 11) Analyticalsolution (t = 1.0)

Crank–Nicolson Second orderPFDM

Fourth Order PFDMh = 0.05 for 20 steps

h = 0.0005 for2000 steps

h = 0.05 for20 steps

h = 0.0005 for2000 steps

h = 0.05 for20 steps

θ|x=0.1 0.159833E−4 0.156231E−4 −0.397567E24 0.159862E−4 0.171255E−4 0.159882E−4θ|x=0.2 0.304021E−4 0.297152E−4 0.342219E24 0.304103E−4 0.325775E−4 0.304139E−4θ|x=0.3 0.418449E−4 0.408987E−4 −0.321638E−4 0.418575E−4 0.448405E−4 0.418625E−4θ|x=0.4 0.491917E−4 0.480789E−4 0.308261E24 0.492071E−4 0.527139E−4 0.492130E−4θ|x=0.5 0.517232E−4 0.505530E−4 −0.303779E24 0.517396E−4 0.554268E−4 0.517459E−4θ|x=0.6 0.491917E−4 0.480789E−4 0.308261E24 0.492071E−4 0.527139E−4 0.492130E−4θ|x=0.7 0.418449E−4 0.408987E−4 −0.321638E24 0.418575E−4 0.448405E−4 0.418625E−4θ|x=0.8 0.304021E−4 0.297152E−4 0.342219E24 0.304103E−4 0.325775E−4 0.304139E−4θ|x=0.9 0.159833E−4 0.156231E−4 −0.397567E24 0.159862E−4 0.171255E−4 0.159882E−4Average relative error (%) – 2.26 0.117822E31 0.027 7.16 0.039

and the boundary conditions

θ(x, t)|x=0 = 0 (31c)

θ(x, t)|x=1 = 0 (31d)

The analytical solution of Eq. (31) is given by

θ(x, t) = sin(πx)exp(−π2t) (32)

The Crank–Nicolson method gives the following integra-tion scheme in the time domain

θi|t=tn+1 = θi|t=tn + h

2

(∂2θi

∂x2

∣∣∣∣t=tn+1

+ ∂2θi

∂x2

∣∣∣∣t=tn

)(33)

for 1 ≤ i ≤ nx − 1while the PFDM directly integrates thefollowing ODEs

dθi

dt= ∂2θi

∂x2for 1 ≤ i ≤ nx − 1 (34)

In Eqs. (33) and (34), the spatial derivatives on the righthand side are approximated by five point finite differencesas given below fornx uniform grid points

∂2θ2

∂x2= 10θ1 − 15θ2 − 4θ3 + 14θ4 − 6θ5 + θ6

12(�x)2+ O(�x)4

(35a)

∂2θi

∂x2= −θi−2 + 16θi−1 − 30θi + 16θi+1 − θi+2

12(�x)2+ O(�x)4

(35b)

for 3 ≤ i ≤ nx − 2

∂2θnx−1

∂x2

= 10θnx−15θnx−1−4θnx−2 + 14θnx−3 − 6θnx−4 + θnx−5

12(�x)2

+ O(�x)4 (35c)

The Crank–Nicolson method only provides an accuracyof O(h2) in the time domain. Since high-order finite dif-ference formulae are used in the space domain. The solu-tion accuracy will be limited by the integration time stepsize for the Crank–Nicolson approach. However, the accu-racy limitation over the time domain is avoided with thehigh-order PFDM approach. As shown inTable 4, better ac-curacy can be achieved with the second order PFDM thanwith the Crank–Nicolson method when the same integratingstep size is used. With one hundred times more steps, the ac-curacy achieved by the Crank–Nicolson approach is still twoorders of magnitude lower than that with the fourth orderPFDM.

3. Conclusions

For initial value problems, the FDM has not been usedwidely due to its disadvantage of implementation difficulty.To get around this problem, an easy-to-use variant of theFDM, the PFDM, is proposed. Since the PFDM uses thesame polynomial form approximation for both the depen-dent variables and their time derivatives, it is much morestraightforward to utilize than the traditional FDM for inte-grating ODEs/DAEs. The relationship between the PFDMand the BDF method and the stability of the PFDM, hasbeen investigated in this work. The features of PFDM aredemonstrated through some example problems.

References

Ascher, U. M., Petzold, L. R., 1998. Computer Methods for OrdinaryDifferential Equations and Differential–Algebraic Equations, SIAM,Philadelphia, PA.

Baase, S., 1988. Computer Algorithms, 2nd edition, Addison-WesleyPublishing Company, Reading, MA.

Brenan, K. E., Campbell, S. L., Petzold, L. R., 1989. Numerical So-lution of Initial-Value Problems in Differential–Algebraic Equations,North-Holland, New York.


Gear, C. W. (1971). Simultaneous numerical solution of differential-algebraic equations.IEEE Trans. Circuit Theory, 18, 89.

Hairer, E., Wanner, G., 1991. Solving Ordinary Differential Equations II– Stiff and Differential-Algebraic Problems, Springer, Berlin.

Holland, C. D., Liapis, A. I., 1983. Computer Methods for SolvingDynamic Separation Problems, McGraw-Hill, New York.

Kimble, M. C., & White, R. E. (1990). A five-point finite differencemethod for solving parabolic partial differential equations.Comput.Chem. Engng., 14, 921.

Michelsen, M. L. (1977). Application of the semi-implicit Runge-Kuttamethods for integration of ordinary and partial differential equations.Chem. Eng. J., 14, 107.

Nguyen, T. V., & White, R. E. (1987). A finite difference procedurefor solving coupled nonlinear elliptic partial differential equations.Comput. Chem. Engng., 11, 543.

Rice, R. G., Do, D. D., 1995. Applied Mathematics and Mod-eling for Chemical Engineers, John Wiley & Sons, Inc., NewYork.

one implementation variant of the finite difference method for solving odes/daes

Documents