a automatic reformulation of odes to systems of first order equations

A

Automatic reformulation of ODEs to systems of first order equations

A. BIRKISSON, University of Oxford

Most numerical ODE solvers require problems to be written as systems of first order differential equations.This normally requires the user to rewrite higher order differential equations as coupled first order systems.Here, we introduce the treeVar class, written in object-oriented Matlab, which is capable of algorithmicallyreformulating higher order ODEs to equivalent systems of first order equations. This allows users to specifyproblems using a more natural syntax, and saves them from having to manually derive the first orderreformulation. The technique works by using operator overloading to build up syntax trees of expressionsas mathematical programs are evaluated. It then applies a set of rules to the resulting trees to obtainthe first order reformulation, which is returned as another program. This technique has connections withoperator overloaded algorithmic/automatic differentiation. We present how treeVar has been incorporatedin Chebfun, greatly improving the ODE capabilities of the system.

CCS Concepts: •Computing methodologies → Representation of mathematical functions;•Mathematics of computing → Mathematical software; Differential equations; Automatic differentiation;

General Terms: Algorithms, Design

Additional Key Words and Phrases: Analysis of mathematical functions, automatic differentiation, auto-matic reformulation, initial value problems, object-oriented programming, operator overloading, ordinarydifferential equations, syntax trees

1. INTRODUCTIONEvery day, thousands of mathematicians, scientists and engineers use numericalsolvers to solve initial-value problems (IVPs) of ordinary differential equations (ODEs).Compared with boundary-value problems (BVPs), where conditions are imposed on thesolution at both endpoints of the space or time interval, IVPs have a special structure.This special structure is that all information passes in one direction over the interval,from the initial time t = t0 to the final time t = tf .

The standard way to take advantage of this local structure of IVPs is to use time-stepping methods, such as Runge-Kutta or Adams-Bashforth formulas [Stoer and Bu-lirsch 2002]. Most numerical solvers based on such formulas only accept first orderdifferential equations, that is, problems on the form

y′ = f(t,y), t0 ≤ t ≤ tf , (1)

with given initial value(s) y(t0) = y0. Therefore, solving higher order differential equa-tions requires writing them as an equivalent system of first order differential equa-tions. This is for example the case with all the ODE solvers in Matlab, such as ode45(a Runge-Kutta method) and ode113 (an Adams-Bashforth method) [Shampine andReichelt 1997].

This work is supported by The MathWorks, Inc. and by the European Research Council under the EuropeanUnion’s Seventh Framework Programme (FP7/2007-2013)/ERC grant agreement 291068.Author’s addresses: Mathematical Institute, University of Oxford, Andrew Wiles Building, Woodstock Road,Oxford, England OX2 6GG. E-mail: [email protected] to make digital or hard copies of all or part of this work for personal or classroom use is grantedwithout fee provided that copies are not made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page. Copyrights for components of this work ownedby others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or repub-lish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Requestpermissions from [email protected].© YYYY ACM. 0098-3500/YYYY/01-ARTA $15.00DOI: http://dx.doi.org/10.1145/0000000.0000000

ACM Transactions on Mathematical Software, Vol. V, No. N, Article A, Publication date: January YYYY.

A:2 Asgeir Birkisson

Even in the case of boundary-value problems for ordinary differential equations,many numerical solvers also require problems to be written on a “standard” form sim-ilar to (1), namely

y′ = f(t,y), a ≤ t ≤ b, (2)g(y(a), y(b)) = 0. (3)

In [Ascher and Russell 1981], Ascher and Russell list techniques for how various typesof BVPs, such as those with nonseparated boundary conditions, problems with condi-tions imposed at an unknown boundary point and problems with integral terms can beconverted to the standard form before being passed to numerical solvers.

While it is possible to rewrite any ODE as a system of first order differential equa-tions, this usually requires the user to do so manually. Additionally, it is not aestheti-cally attractive. As an example, consider the well known van der Pol oscillator, whichoriginally arose in the study of nonlinear electrical circuits [van der Pol 1926; Strogatz1994]. The van der Pol oscillator is affected by nonlinear damping, and is governed bythe second order differential equation

u′′ + µ(u2 − 1)u′ + u = 0, u(t0) = u0, u′(t0) = u′0. (4)

Here, u = u(t) and µ ≥ 0 is a scalar parameter that controls the strength of the damp-ing. It feels more natural to solve the problem numerically as the second order ODE(4), rather than as its first order reformulation,

u′1 = u2,

u′2 = µ(1− u21)u2 − u1,u1(t0) = u0, u2(t0) = u′0.

(5)

Here, we introduce the treeVar class (or simply treeVar), written in object-orientedMatlab. It enables automatic conversion of ODEs posed as higher order equations, suchas (4), to systems of first order equations, such as (5). This allows users to pose prob-lems in their natural form; they can then be solved using standard ODE solvers afterthe reformulation. The technique works by applying operator overloading to build upsyntax trees of mathematical expressions as functions are evaluated. It then analysesthe resulting trees and applies a set of rules to them to obtain the first order reformu-lation. treeVar works for both scalar and coupled quasilinear differential equations.

This paper is structured as follows. In Section 2, we describe the implementation oftreeVar and list the algorithms it employs for carrying out the first order reformu-lation. The technique applied by treeVar is related to operator overloaded algorith-mic/automatic differentiation, which obtains derivatives of mathematical functions byinjecting statements into the evaluation trace of a program, defined in Section 2.

In Section 3, we give examples on how the automatic reformulation of ODEs can beused to solve problems with the solvers readily available in Matlab. In Section 4, wediscuss how treeVar has been incorporated in Chebfun [Driscoll et al. 2014], greatlyimproving its IVP capabilities by enabling the system to avoid using global methods(as opposed to time-stepping methods) for solving IVPs. Finally, we list our conclusionsand potential further avenues of investigation in Section 5.

The presentation in this paper is based on Version 1.0 of treeVar, available onGitHub [Birkisson 2016]. treeVar was introduced in Version 5.1 of Chebfun (with sometweaks for Chebfun specific applications).

1.1. Alternatives to a first order reformulationBefore continuing, we mention that there exist numerical methods for solving ODEsthat don’t require problems to be written on their first order form. These are particu-


AUTOMATIC REFORMULATION OF ODES A:3

larly important for differential equations on the formy′′ = f(t, y),

that is, where the right-hand side does not depend on y′. Such problems often arise incelestial mechanics. Two classes of methods especially fit for such problems are oftenreferred to as Nystrom and Stormer methods [Hairer et al. 2008].

The simplest type of a Stormer method, where for a timestep size ∆t, the solution atthe n+ 1st timestep is computed via

yn+1 = −yn−1 + 2yn + (∆t)2f(tn, yn),

was applied in [Verlet 1967] for integrating the equation of motion of a system of parti-cles interacting through a Lennard-Jones potential. Subsequently, in the field of molec-ular dynamics, the method became known as Verlet integration.

2. THE TREEVAR CLASS AND CONVERSION OF ODES TO FIRST ORDER SYSTEMS2.1. The evaluation trace and AD backgroundIn various applications in scientific computing, derivatives play a central role. Givena function f(x), the two best known methods for computing derivatives are finite dif-ference schemes and symbolic differentiation. Both these methods suffer from draw-backs; finite difference schemes suffer both from round-off errors and truncation er-rors (which usually dominate the round-off errors), while symbolic differentiation canexperience a combinatorial explosion of computation time and memory requirements[Griewank and Walther 2008; Neidinger 2010].

Automatic, or algorithmic, differentiation (AD) aims to introduce techniques to com-pute accurate derivatives in a fast way. The derivatives only suffer from round-offerrors from the evaluation of functions, not truncation errors. Like finite differenceschemes, AD works entirely at the numerical level. However, AD uses informationabout the functions it is differentiating to avoid truncation errors.

A key concept in AD is that of the evaluation trace. The evaluation trace is essen-tially a sequence of elementary calculations taken to evaluate a program with inputvariables u1, . . . , un, to obtain output variables y1, . . . , ym. In the mathematical context,a program often means a function or an operator. The evaluation trace is defined byGriewank and Walther as follows:

An evaluation trace is basically a record of a particular run of a particularprogram, with particular specified values for the input variables, showing thesequence of floating point values calculated by a (slightly idealized) processorand the operators that computed them. [Griewank and Walther 2008, p. 4]

Each part of the sequence is associated with what is known in the AD literature asan intermediate variable. The final output of the program is obtained by performingseries of computations, using elementary operators such as cos, log, +, ×, with theintermediate variables. All the elementary operators are assumed to be either unaryor binary.

At the start of a computation, we initialise the intermediate variables vk, for k ≤ 0,to be the input variables to the program. We denote the kth elementary operator in thesequence by ϕk, and its result by the intermediate variable vk, for k > 0. For example,if ϕk is a binary operator, we have vk = ϕk(vi, vj), where vi and vj are intermediatevariables, with i, j < k.

As an example of an evaluation trace, let y be the function of two variables given bythe formula

y = (u1 + u2) sin(u22) + 4(u1 + u22). (6)



Assignment Value of vkv−1 = u1 2.0000v0 = u2 0.5000v1 = v−1 + v0 2.0000 + 0.5000 = 2.5000v2 = v20 0.50002 = 0.2500v3 = sin(v2) sin(0.2500) = 0.2474v4 = v1v3 2.5000× 0.2474 = 0.6185v5 = v−1 + v2 2.0000 + 0.2500 = 2.2500v6 = 4v5 4.0000× 2.2500 = 9.0000v7 = v4 + v6 0.6185 + 9.0000 = 9.6185y = v7 9.6185

Table I: An evaluation trace for y = (u1 + u2) sin(u22) + 4(u1 + u22), with u1 = 2, u2 = 0.5.

Suppose that we want to obtain the value of y with u1 = 2 and u2 = 0.5. The evaluationtrace for one implementation of that computation is shown in Table I.

It is by working with derivative information of the intermediate variables in theevaluation trace that AD can yield derivatives of the output variables y1, . . . , ym withrespect to the input variables u1, . . . , um. A popular technique for converting the origi-nal program to a program which can return derivatives is that of operator overloading.This technique works in programming languages which support operator overloadingin object-oriented programming, such as C++ and Matlab. The idea is to introduce anew class, which in addition to storing the values of variables, also stores the valuesof derivatives. Then, as operations are carried out, not only are the variable valuesupdated, but also the derivatives, according to the chain rule of calculus. We refer to[Griewank and Walther 2008] for how to incorporate the chain rule into computations,and further discussion of AD.

2.2. Obtaining syntax trees by operator overloadingRather than introducing a new class that in combination with operator overloadingcomputes derivatives, we introduce a new class for obtaining syntax trees of mathe-matical expressions via operator overloading. We call this class treeVar. Evaluatingfunctions with treeVar arguments yields their corresponding syntax trees, which isthe first step for deriving the first order reformulation of differential equations.

The fields of treeVar objects are listed in Table II. For every property of the object,we define a function with the same name that returns the corresponding value. Forexample, if u k is a treeVar object representing the intermediate variable uk in theevaluation trace, then

height(uk)

is equal to the height field of u k.A crucial element of our algorithm for first order reformulation is propagating in-

formation about dependencies of variables. This is achieved with the ID field of thetreeVar object, which is a Boolean vector. Boolean vectors are elements of Bn, thespace of vectors with n elements that are either 0 or 1. We represent members of Bn

as row vectors. By rj , we mean the jth element of the vector r ∈ Bn. To every interme-diate variable ui in the evaluation trace, we associate the Boolean vector r(i). A valuer(i)j = 1 indicates that the intermediate variable ui depends on the jth input variable,

while r(i)j = 0 indicates that ui does not depend on the jth input variable.



method The method leading to the construction of the variable.numArgs The number of arguments to the methodcenter The center node of the syntax tree (the argument to a univariate

method).left The left node of the syntax tree (the first argument to a bivariate

method).right The right node of the syntax tree (the second argument to a bi-

variate method).diffOrder The differential order of the treeVar, which represents how

many times the base variables have been differentiated whenwe arrive at the current treeVar. Note that diffOrder is vectorvalued, with the ith element equal to the highest derivative ofthe ith input variable that appears in the syntax tree.

height The height of the syntax tree, i.e., the number of operations be-tween the base variables(s) and the current variable.

ID A Boolean vector, whose ith element is equal to 1 if the treeVarvariable was constructed from the ith base variable, 0 otherwise.

hasTerms A Boolean that indicates whether a treeVar is constructed froma sequence of computations that include multiple terms with de-pendent variables.

Table II: Fields of the treeVar object.

We use the following standard Boolean algebra notation for vectors r, s, t ∈ Bn:

∨ Logical OR. If t = r ∨ s, then ti = 1 iff ri = 1 or si = 1 (or both).

Before we evaluate the program defining the differential operator, we seed base vari-ables that correspond to each unknown function in the differential equations. The basevariables correspond to intermediate variables in the evaluation trace with index ≤ 0,and the seeding operation involves setting one element of the ID vector of the treeVarobject equal to 1. For example, for the Lorenz equations

x′ = 10(y − x) y′ = x(28− z)− y z′ = xy − 83z (7)

where we initialise the intermediate variables as

u−2 = x, u−1 = y, u0 = z,

we have

id(u−2) = [1 0 0], id(u−1) = [0 1 0] id(u0) = [0 0 1]. (8)

For differential equations where the independent (time) variable appears explicitly,the corresponding base variable gets seeded with an all-zero ID vector.

After seeding the base variables, we now carry out the computations in the evalua-tion trace. The result of evaluating each elementary operator (with either one or twotreeVar arguments) will be another treeVar variable, which gets further propagatedin the trace. The values of the properties of the treeVar object returned depend on theinput variables and the type of the operator. In the Lorenz example above, we arriveat

id(x′) = [1 1 0], id(y′) = [1 1 1], id(z′) = [1 1 1], (9)

indicating that x′ depends on x and y, while y′ and z′ depend on all of x, y and z, as isclear from (7).



method(vk) = cos

numargs(vk) = 1

center(vk) = vi

id(vk) = id(vi)

height(vk) = height(vi) + 1

difforder(vk) = difforder(vi)

hasterms(vk) = hasterms(vi)

Table III: Output rules for the elementary operation vk = cos(vi), where vi and vk areintermediate treeVar variables.

method(vk) = +

numargs(vk) = 2

left(vk) = vi

right(vk) = vj

id(vk) = id(vi) ∨ id(vj)

height(vk) = max(height(vi),height(vj)) + 1

difforder(vk) = max(difforder(vi),difforder(vj)) (elementwise)

hasterms(vk) = hasterms(vi) ∨ hasterms(vj)

Table IV: Output rules for the elementary operation vk = vi + vj , where vi, vj and vkare intermediate treeVar variables.

To give an idea of the rules that treeVar applies, we show the rules governing theoutput from the intermediate operators cos and + (which are prototypical for unaryand binary operators respectively) in Tables III and IV.

At the end of the evaluation, we will have obtained a syntax tree that can be eitherprinted or plotted. For example, let the expression

f = x′′ + ey + cos(2t), (10)

where t is the independent time variable of the problem and x = x(t), y = y(t) be given.Its syntax tree can be obtained and plotted (cf. Figure 1) as follows.

>> t = treeVar([0 0]);>> x = treeVar([1 0]);>> y = treeVar([0 1])y =treeVar with properties:diffOrder: [0 0]hasTerms: 0height: 0ID: [0 1]method: ’constr’numArgs: 0



u1 2.00

diff

u2

exp

plus

2.00 t

times

cos

plus

Fig. 1: A syntax tree of the expression f = x′′ + ey + cos(2t).

left: []center: []right: []

>> f = diff(x,2) + exp(y) + cos(2*t)f =treeVar with properties:diffOrder: [2 0]hasTerms: 1height: 3ID: [1 1]method: ’plus’numArgs: 2left: [1x1 treeVar]center: []right: [1x1 treeVar]

>> plot(f)

Comparing the output from sequence above with Table II, we make the following ob-servations:

— Since y is a base variable, its ID field is identical to what it was seeded with. In thesyntax tree for f, both x and y appear, hence its ID field is [1 1].

— The highest order derivative of x that appears in the syntax tree of f is 2, while noderivative of y is in the syntax tree of f. Thus, the diffOrder field of f is [2 0].

— y is a base variable, so its method is denoted to be the treeVar constructor, and ithas no children in the left, center or right fields. The method at the top of thesyntax tree for f is +, and the treeVar arguments of the + method are stored in theleft and right fields of f.

2.3. Converting ODEs to a first order systemWe now describe how the syntax trees obtained by evaluating programs with treeVararguments are used to convert a system of quasilinear ODEs and a vector of right-hand side values to a system of coupled first order ODEs. The essential algorithm ispresented in Algorithm 2.1. It is implemented as the treeVar method toFirstOrder.The main algorithm makes use of some accompanying algorithms for working withsyntax trees. These are:

— expandTree (implemented as the treeVar method expandTree), which traversessyntax trees recursively to expand factors of input trees where highest order deriva-tives appear.



— splitTree (implemented as the treeVar method splitTree), which traverses syntaxtrees recursively to split them into two trees. One of the trees contains the termswhere the highest order derivatives of the input variables appear (this tree is usedfor determining the coefficient multiplying the highest order derivatives). The othertree contains the remaining terms.

— toInfix (implemented as the treeVar method tree2infix), which recursively tra-verses syntax trees stored in treeVar variables and converts them to infix form ofthe mathematical expressions they describe, using the method, left, center andright properties of treeVar variables.

All the treeVar implementations of the algorithms are available and extensively doc-umented in the code repository [Birkisson 2016]. In Figure 2, we show the results ofthe five main steps of Algorithm 2.1, namely, evaluating expressions, expanding thesyntax tree, splitting the syntax tree, incorporating the right-hand side of the differ-ential equation, and converting trees back to infix form. The differential equation weconsider is

5t(u′′ + uu′) = 2, (11)

and the command for calling the treeVar implementation of Algorithm 2.1 is

funOut = treeVar.toFirstOrder(@(t,u) 5*t.*(diff(u,2)+u.*diff(u)), 2)

While the computer generated output funOut, shown in Figure 2e, isn’t particularlyhuman readable, it is perfectly acceptable as an input for the Matlab ODE solvers. Wedon’t expect end users to ever have to work directly with the output from toFirstOrder,instead, it can be passed straight to the Matlab solvers.

3. EXAMPLESHere, we show examples of how the treeVar class can be used in Matlab for convertinga higher order differential equation to a first order system, returning an anonymousfunction that we can evaluate to confirm it gives the expected results. We then passthe output from the first order reformulation to the Matlab ODE solvers to solve aninitial value problem.

Let the differential equations of interest be the equations governing linearly cou-pled anharmonic oscillators, which can exhibit chaotic dynamics for certain parameterchoices [Steeb et al. 1987]. The equations are:

x′′ +Ax+ ax3 + cy2 = 0,

y′′ +Ay + ay3 + cx2 = 0,(12)

with first order reformulationu′1 = u2,

u′2 = −(A+ au21)u1 − cu23,u′3 = u4,

u′4 = −(A+ au23)u3 − cu21.

(13)

We begin by creating an anonymous function that defines the left-hand sides of thedifferential equations (12) in Matlab, which gets passed to the toFirstOrder method oftreeVar. Note that the diff method is used to represent differentiation. We also passto toFirstOrder a vector containing the right-hand sides of the differential equations.The output is an another anonymous function that corresponds to the first order re-formulation (13). We can evaluate that function at given values of t and u1, . . . , u4 and



5.00 t

times

u 2.00

diff

u

u 1.00

diff

times

plus

times

(a) Original syntax tree of the expression 5t(u′′ + uu′)

5.00 t

times

u 2.00

diff

times

5.00 t

times

u

u 1.00

diff

times

times

plus

(b) Expanded syntax tree

5.00 t

times

u 2.00

diff

times

5.00 t

times

u

u 1.00

diff

times

times

(c) After splitting syntax tree into two trees, one where highest order derivative appears,and another containing the remaining terms.

2.00

5.00 t

times

u

u 1.00

diff

times

times

minus

(d) Incorporating the right-hand side of the differential equationfunOut =

@(t,u)[u(2);minus(eq1_var1,times(times(eq1_var2,t),times(u(1),u(2))))./coeffs{1};]

(e) Infix form of the syntax tree above, which can be passed to Matlab’s ODEsolvers.

Fig. 2: The results of the five main steps of the first order conversion algorithm, Algo-rithm 2.1, for the differential equation 5t(u′′ + uu′) = 2.



Algorithm 2.1 First order reformulation of ODEs.Input: A function f the specifies a system of quasilinear ordinary differential equa-

tions of arbitrary order. A vector r with entries for the right-hand sides of thedifferential equations.

Output: A function g that corresponds to the first order reformulation of f and r.1: m← Number of dependent variables in the problem2: n← Number of differential equations in the problem3: t← treeVar with id(t) = [0 . . . 0] ∈ Bm . Indep. var. of the problem4: for j = 1, . . . ,m do . Seed base variables5: uj ← treeVar with id(uj) = [0 . . . 0 1 0 . . . 0] ∈ Bm (jth entry)6: end forF will be an n-array of treeVar variables, where the ith variable contains the syntaxtree of the ith differential equation.

7: F ← f(t, u1, . . . , um) . Evaluate f with treeVar arguments8: D ← [0 . . . 0] ∈ Nm . For storing order of highest deriv. of each var.9: for k = 1, . . . , n do . Determine highest derivatives of each input var.

10: D ← max(D, difforder(Fk)) (elementwise)11: end for

Compute a vector that contains the indices of dependent variables in the reformu-lated problem. For example, if converting a system where u′′, v′′′ and w′, are thehighest order derivatives appearing, I = [1 3 6], since when the problem is writtenon its first order form, u = y1, v = y3 and w = y6.

12: Let I ∈ Nm, with I1 = 1, Ij = Ij−1 +Dj−1 for j ≥ 2.13: for k = 1, . . . , n do . Convert each differential equation to first order14: T ← Fk . Work with the kth differential equation15: T ← expandTree(T, D) . Expand factors of syntax tree16: T ← splitTree(T, D) . Split tree into derivative and non-derivative parts

Create a new treeVar to account for the rhs of the original differential equation17: S ← treeVar, with method(S) = −, left(S) = rk and right(S) = T18: gk ← toInfix(S, D, I) . Convert syntax tree to infix form19: end for20: return g

compare with the result of evaluating a manually constructed first order reformulationto confirm we obtained correct results. The code is given as follows:

A = 0.3; a = 1; c = 1; % Specify coefficients% Define the second order coupled system of ODEsodeFun = @(t,x,y) [diff(x,2) + A*x + a*x.^3 + c*y.^2; ...

diff(y,2) + A*y + a*y.^3 + c*x.^2];rhs = [0; 0]; % RHS of ODEs% Convert algorithmically to a first order systemanonFun = treeVar.toFirstOrder(odeFun, rhs);% The manually derived first order reformulationcorrectFun = @(t,u) [u(2); -(A+a*u(1).^2)*u(1) - c*u(3).^2;

u(4); -(A+a*u(3).^2)*u(3) - c*u(1).^2];% Arbitrary values for t and u_1, ..., u_4 to compare at:t = 1; evalPt = [2.4 2.3 5.2 1.2];norm(anonFun(t, evalPt) - correctFun(t, evalPt))ans =

2.842170943040401e-14



0 25 50 75 100

-2.25

-1.5

-0.75

0

0.75

1.5

x

y

(a)

0 20 40 60 80 10010−15

10−10

10−5

100

(x1(t)−

x2(t))

2+

(y1(t)−

y2(t))

2

(b)

Fig. 3: (a) Solution of the linearly coupled anharmonic oscillator (12), with initial con-ditions (14). (b) Separation of two pairs of trajectories (x1, y1) and (x2, y2), arising fromslightly different initial conditions, as t grows.

We note that the difference between evaluating the automatically converted andthe manually constructed anonymous functions is close to but not exactly 0. This isdue to Matlab computing the answer by slightly different sequences of floating pointoperations in the two cases.

To solve the ODE (12), we can now pass the anonymous function obtained above tothe ode113 solver along with time interval and the initial conditions1, which we taketo be

x(0) = 0.7, x′(0) = 0, y(0) = 0.8, y′(0) = 0. (14)

The treeVar method solveIVP takes care of all the steps required to compute a solutionof an ODE, given anonymous functions specifying the differential equations and initialconditions, a vector for the right-hand sides of the equations and a timespan on whichthe solution is to be computed. For this problem, we call it as follows:

icFun = @(x,y) [x - .7; diff(x); y - .8; diff(y)];rhs = [0; 0];odeDom = [0 100];[t, xy] = treeVar.solveIVP(odeFun, icFun, rhs, odeDom)

We plot the solutions x and y in Figure 3a. Figure 3b shows how changing one of theinitial conditions from x(0) = 0.7 to x(0) = 0.7001 results in different trajectories thatseparate from the original ones as t grows, confirming that the system has a positiveLyaponov coefficient.

4. CHEBFUN AND TREEVAR

The chebop class of Chebfun has the capability of solving nonlinear BVPs of ODEsin the continuous setting, implementing Newton’s method in function space (whichis often referred to as Newton-Kantorovich iteration). This is achieved using spectralmethods in combination with automatic differentiation [Birkisson and Driscoll 2012].However, such global methods are often ineffective for IVPs as they make no use of

1We have also implemented a method sortConditions, which, given initial conditions specified for variablesof the original problem, determines which conditions should be imposed to each variable in the reformulatedproblem. The chebop solver makes use of that method, as will be evident in the next section.



the special structure of IVPs — poor initial guesses can lead to the Newton iterationto fail, even when it is damped.

The remedy for IVPs is to use timestepping methods for computing a high accuracysolution to the problem, then convert the solution to a chebfun. This is done by theoverloaded ‘backslash’ (\) method of chebop. The default algorithm for solving an IVPwith chebop is as follows:

(1) Call the treeVar class for automatically reformulating the problem.(2) Solve the problem using the ode113 solver of Matlab with a strict tolerance.(3) Sample the solution at finer and finer Chebyshev grids until the Chebyshev coeffi-

cients of the solution have decayed sufficiently.

As an example of how IVPs can now be easily solved with chebop, here, we solve thevan der Pol Equation (4) on the interval t ∈ [0, 100], with coefficient µ = 10 and initialconditions u(0) = 1, u′(0) = 0.

% Define a chebop on the time interval [0, 100]:N = chebop(@(t,u) diff(u, 2) - 10*(1-u^2)*diff(u) + u, [0 100]);% Impose initial conditions u(0) = 1, u’(0) = 0.N.lbc = [1; 0];% Call overloaded \ for solving:u = N\0;

The solution is a chebfun object of length 14277, that is, a polynomial of degree 14276.In Figure 4a, we plot the solution, and in Figure 4b, we plot its Chebyshev coefficients,showing that we have resolved the solution to a high precision. Since the solution isa chebfun, we can carry out all the normal chebfun operations on it, such as findingall its local maxima (plotted as red stars in Figure 4a) and comparing the distancebetween them to determine the period of the oscillator.

[maxval, maxpos] = max(u, ’local’);diff(maxpos(2:end-1))ans =19.07836956700565919.07836956708635519.07836956703874219.078369567054025

Another example we consider is the Rossler equations, a dynamical system thatonly has a single nonlinear term, yet still exhibits chaotic behaviour [Rossler 1976].The equations are

u′ = −v − w, v′ = u+ 0.2v, w′ = 0.2 + w(u− 5.7) (15)

and we arbitrarily pick the initial conditions

u(0) = −4, v(0) = 0, w(0) = 0.

The code for setting up and solving the problem for t ∈ [0, 100] is as follows:

% Assign the differential equation to a chebop on our selected domainN = chebop(@(t,u,v,w) [diff(u) + v + w;

diff(v) - u - 0.2*v; diff(w) - 0.2 - w*(u - 5.7)], [0 100]);% Assign boundary conditions to the chebop.N.lbc = @(u, v, w) [u+4; v; w];



0 25 50 75 100

-2

-1

0

1

2

(a)

0 5000 10000 15000

Degree of Chebyshev polynomial

10−20

10−15

10−10

10−5

100

Magnitudeofcoeffi

cient

(b)

Fig. 4: (a) Solution of the van der Pol ODE (5) with initial conditions u(0) = 1, u′(0) = 0.Local maxima are plotted as red stars. (b) Chebyshev coefficients of the computed so-lution.

-5

10

0

5

5 15

10

w

0

15

10

v

20

5

u

-5

25

0-10 -5

-15 -10

Fig. 5: The Rossler attractor for Equations (15).

% Solve and plot solution compomentents against each other[u, v, w] = N\0; plot3(u, v, w)

The presence of a strange attractor is evident from a 3D plot of the solution compo-nents, shown in Figure 5.

As a final example of what treeVar has enabled for the chebop class, we mention thequiver method for plotting phase plane portraits of second order scalar ODEs (firstorder coupled systems of two functions are also supported). Given a chebop, the methodobtains the first order reformulation of its differential equation, which is then usedto compute a vector field that is passed to the built-in quiver method of Matlab for



-2 0 2 4 6 8 10 12 14 16

θ

-2

-1

0

1

2

3

4

5

θ

Fig. 6: Phase portrait for the damped nonlinear pendulum (16), showing trajectoriesfor different initial velocities.

plotting. The following code plots a phase portrait for the damped nonlinear pendulum:

θ + 14 θ + sin(θ) = 0. (16)

It then computes trajectories for different initial velocities and overlays them on theplot, shown in Figure 6. Notice that while trajectories with high enough initial veloc-ities are able to swing around a few times, eventually, they all are brought to rest atthe pendulum’s stable equilibrium.

% Define a chebopN = chebop(@(t,theta) diff(theta,2)+diff(theta)/4+sin(theta),[0 50]);% Call quiver, specifying upper and lower limits of axes:quiver(N, [-2 16 -2 5.5])% Overlay trajectories for different initial velocitieshold onfor init = 0:0.5:5

N.lbc = [0; init];theta = N\0;plot(theta, diff(theta))

endhold off

5. CONCLUSION AND FURTHER WORKWe have shown how the introduction of the treeVar class enables us to algorithmicallyconvert a higher order differential equation to a system of first order equations. Thisallows us to call standard ODE solvers such as Matlab’s ode45 and ode113 on problemsthat are described more closely to their natural mathematical form. In particular, forChebfun, it allowed the chebop class to apply time-stepping algorithms to IVPs, ratherthan global spectral methods. This led to a much more robust solver for IVPs, avoid-ing issues with diverging Newton’s method for poor initial guesses. It also offers easyplotting of phase portraits for ODEs. A significant part of the demonstrations in the



forthcoming ODE textbook [Trefethen et al. 2017] were enabled by capabilities treeVarintroduced.

We believe that the structural analysis of mathematical problems, such as imple-mented here with treeVar, has many other possible applications in scientific comput-ing software, especially for offering users a convenient syntax while applying powerfulalgorithms under the hood. We have already started investigating applications for op-timal control problems, as well as for splitting PDE operators into linear and nonlin-ear parts for exponential integrators [Cox and Matthews 2002; Kassam and Trefethen2005].

ACKNOWLEDGMENTS

The author would like to thank Nick Trefethen for the many comments, suggestions and revisions whileworking on this article.

REFERENCESU. Ascher and R. D. Russell. 1981. Reformulation of boundary value problems into “standard” form. SIAM

Rev. 23, 2 (1981), 238–254. DOI:http://dx.doi.org/10.1137/1023039A. Birkisson. 2016. The treeVar class for automatic reformulation of ODEs to first order systems. https:

//github.com/asgeirbirkis/treeVar. (2016).A. Birkisson and T. A. Driscoll. 2012. Automatic Frechet differentiation for the numerical solu-

tion of boundary-value problems. ACM Trans. Math. Softw. 38, 4, Article 26 (2012), 29 pages.DOI:http://dx.doi.org/10.1145/2331130.2331134

S. M. Cox and P. C. Matthews. 2002. Exponential time differencing for stiff systems. J. Comput. Phys. 176, 2(March 2002), 430–455. DOI:http://dx.doi.org/10.1006/jcph.2002.6995

T. A. Driscoll, N. Hale, and L. N. Trefethen (Eds.). 2014. Chebfun Guide. Pafnuty Publications, Oxford.http://www.chebfun.org/docs/guide/

A. Griewank and A. Walther. 2008. Evaluating Derivatives: Principles and Techniques of Algorithmic Differ-entiation (2nd ed.). SIAM, Philadelphia.

E. Hairer, S. P. Nørsett, and G. Wanner. 2008. Solving Ordinary Differential Equations I: Nonstiff Problems(2nd revised ed.). Springer Berlin Heidelberg.

A.-K. Kassam and L. N. Trefethen. 2005. Fourth-order time-stepping for stiff PDEs. SIAM J. Sci. Comput.26, 4 (April 2005), 1214–1233. DOI:http://dx.doi.org/10.1137/S1064827502410633

R. D. Neidinger. 2010. Introduction to automatic differentiation and MATLAB object-oriented programming.SIAM Rev. 52, 3 (2010), 545–563. DOI:http://dx.doi.org/10.1137/080743627

O. E. Rossler. 1976. An equation for continuous chaos. Physics Letters A 57, 5 (1976), 397 – 398.DOI:http://dx.doi.org/10.1016/0375-9601(76)90101-8

L. F. Shampine and M. W. Reichelt. 1997. The MATLAB ODE Suite. SIAM J. Sci. Comput. 18, 1 (Jan. 1997),1–22. DOI:http://dx.doi.org/10.1137/S1064827594276424

W.-H. Steeb, J. A. Louw, and C. M. Villet. 1987. Linearly coupled anharmonic oscillators and integrability.Aust. J. Phys. 40, 5 (Oct. 1987), 587–592. http://dl.acm.org/citation.cfm?id=43973.43974

J. Stoer and R. Bulirsch. 2002. Introduction to Numerical Analysis. Springer.S. H. Strogatz. 1994. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and

Engineering. Westview Press.L. N. Trefethen, A. Birkisson, and T. A. Driscoll. 2017. Exploring ODEs. (2017). In preparation.B. van der Pol. 1926. On “relaxation-oscillations”. The London, Edinburgh, and Dublin Philosophical Mag-

azine and Journal of Science 2, 11 (1926), 978–992. DOI:http://dx.doi.org/10.1080/14786442608564127L. Verlet. 1967. Computer “experiments” on classical fluids. I. Thermodynamical prop-

erties of Lennard-Jones molecules. Phys. Rev. 159 (Jul 1967), 98–103. Issue 1.DOI:http://dx.doi.org/10.1103/PhysRev.159.98


http://dx.doi.org/10.1137/1023039

https://github.com/asgeirbirkis/treeVar

https://github.com/asgeirbirkis/treeVar

http://dx.doi.org/10.1145/2331130.2331134

http://dx.doi.org/10.1006/jcph.2002.6995

http://www.chebfun.org/docs/guide/

http://dx.doi.org/10.1137/S1064827502410633

http://dx.doi.org/10.1137/080743627

http://dx.doi.org/10.1016/0375-9601(76)90101-8

http://dx.doi.org/10.1137/S1064827594276424

http://dl.acm.org/citation.cfm?id=43973.43974

http://dx.doi.org/10.1080/14786442608564127

http://dx.doi.org/10.1103/PhysRev.159.98

a automatic reformulation of odes to systems of first order equations

Documents