Mathematical Programmingwith MATLAB

Monday 20.09.2010

HA13: Differential Equations

• Differential Equation Solvers• Function Handles and Anonymous Functions• First Order Differential Equation• Second Order Differential Equations• Initial Value Problems• Boundary Value Problems

Solving Differential Equations Numerically

• MATLAB includes a number of functions that solve ordinary differential equations of the form

• Higher order differential equations and systems of differential equations must be reformulated into a system of first order expressions.

( , )dy

f t ydt

Differential Equation Solvers• Not every differential equation can be solved using

the same technique. • All of the differential equation solvers in MATLAB

have the same format. • This makes it easy to try different techniques, by just

changing the function name.• MATLAB includes a large family of differential

equation solvers.• A good first try is usually the ode23 or ode45 solver

function, which uses a Runge-Kutta technique.

• A minimum of three inputs is required - function handle to a function that describes the first

order differential equation or system of differential equations in terms of t and y

- the time span of interest - an initial condition for each equation in the system.

• If you do not specify the result array, [t,y] , the functions create a plot of the results

[t,y] = odesolver(function_handle, [initial_time, final_time], [initial_cond_array])

Family of Solver Functions

ODE solver function

Type of problems likely to be solved with this technique

Numerical Solution Method

Comments

ode45 non-stiff differential equations Runge-Kutta Best choice for a first guess technique, if you don’t know much about the function. Uses an explicit Runge-Kutta (4,5) formula, called the Dormand-Prince pair

ode23 non-stiff differential equations Runge-Kutta This techniques uses an explicit Runge-Kutta (2,3) pair of Bogacki and Shampine. If the function is “mildly stiff” this may be a better approach

ode113 non-stiff differential equations Adams Unlike ode45 and ode23 which are single step solvers, this technique is a multistep solver.

ode15s stiff differential equation and differential algebraic equations

NDFs (BDFs) Uses numerical differentiation formulas or backward differentiation formulas. It is difficult to predict which technique will work best for a stiff differential equation

ode23s stiff differential equations Rosenbrock modified second order Rosenbock formulation

ode23t moderately stiff differential equations and differential algebraic equations

Trapazoid rule Useful if you need a solution without numerical damping

ode23tb stiff differential equations TR-BDF2 This solver uses an implicit Runge-Kutta formula with the trapazoid rule and a second order backward differentiation formula.

ode15i fully implicit differential equations

BDFs .

Function Handles

• A function handle is a nickname (alias name) for a function.

• The function handle can refer to – a standard MATLAB function, stored as an M-file – an anonymous MATLAB function.

• Anonymous functions can be defined in an M-file program, or in the command window.

2*dy

tdt

This function corresponds to:

You can use an anonymous function as input to an ode solver function

Because we did not assign a name to the ode function output, a graph is created.

See next slide.

-1 -0.5 0 0.5 10

0.2

0.4

0.6

0.8

1Solution to dy/dt = 2 * t

time

y

This figure was generated automatically by the ode45 function. The title and labels were added in the usual way.

Solving Differential Equations Numerically

Systems of differential equations• If you want to specify a system of equations, it is

probably easier to use a function M-file. • The output of the function must be a column vector

of first derivative values. • For example:

dyx

dtdx

ydt

This function M-file represents a system of equations

1 2

2 1

y y

y y

dyx

dtdx

ydt

Systems of differential equations: Example

• We can now use an M-file function as input to an ODE solver

• The handle for an existing M-file is @m_file_name

ode45(@another_fun, [-1,1], [1,1])

Systems of differential equations: Example … continue

-1 -0.5 0 0.5 1-2

-1

0

1

2Solution to dy/dt=x and dx/dt= -y

time

x an

d y

y

x

This system of equations was solved using ode45.

The title, labels and legend were added in the usual way.

Systems of differential equations: Example … continue

1st Order Differential Equations

• The initial value problem of the 1st order differential equation is of the form

y'(t) = f(t), y(0) = a• It can be solved in MATLAB using functions ode23 or ode45.• Example: RL- circuit (a resistor and an inductor) L I’ + R I = E, where E=12, R=2, L=0.5.

RL Circuit

• The function file is diff_i.m

function di = diff_i(t,i) % The differential equation of the RL circuit

E=12; % voltage R=2; % resistance L=0.5; % inductanse

di = E/L - (R/L)*i;

RL Circuit: Current as Function of Time

% RL circuit: % calculate the current as a function of time

t0 = 0; % solve current for values 0...5 s tl = 5;

i0 = 0; % initial condition at time t0

[t, i] = ode23('diff_i', t0, tl, x0); plot(t,i,'-b')

axis([0 2 0 7]) title('RL Circuit') xlabel(’Time,s') ylabel(’Current,A’)

2nd Order Differential Equation• The initial value problem of the 2nd order

differential equation of the form y''(t) = f(t), y(0) = a, y'(0) = b have to be transformed to a system of 1st order differential equations. • Let us call the original variable y1= y • Introduce a new variable y2 = y1'• After substituting to the original equation we get the system of following equations y1´ = y2 y2' = y1'' = f(t) y1(0) = a y2(0) = y1’(0) = b

Simple Harmonic Motion• A typical example of a system that undergoes simple

harmonic motion is an idealized spring–mass system, in which is a mass is attached to a spring.• The differential equation is m x’’ + b x’ + k x = mg, where k=10, b=2, g=9.81, m=2• We have to solve the system of two equations:

x1 = x x2 = x1’ x1’ = x2 x2’ = - (k/m) x1 – (b/m) x2 + g

Solution in MATLAB

• The function string.m contains the definition of the differential equations

function xe=string(t,x) k=10; b=2; g=9.81; m=2; A=[0 1; -k/m -b/m]; b=[0,g]; xe=A*x+b';

Solution in MATLAB … continue

• On the command window we call the function ode23

>> t0=0; tfin=10; % the start and end times of calculation >> x0=[0;0]; % initial values >> [t,x]=ode23(’string',t0,tfin,x0);

>> plot(t,x(:,1)) % plot the position vector x = x_1 >> plot(t,x(:,2)) % plot the velocity vector x’ = x_2

Component Form of Equations

• Instead of the matrix form the system of differential equations can be written in the function string.m

alternatively also in the component form:

function xe=string(t,x) k=10; b=2; g=9.81; m=2; xe(1) = x(2); xe(2) = (-k/m)*x(1) + (-b/m)*x(2) + g;

MATLAB dsolve (from Symbolic Toolbox)

dy = y

dt

To pin down the constant C1 we need a boundary condition.

To find C1, we need a boundary condition that tells us some value of y(t): e.g. y(1)=2.

Symbolic Toolbox is an optional toolbox that allows symbolic computations.It must be installed separately on your computer.

Another differential equation example

Makes sense since derivative of t2 is 2t

and 2t = 2 t2/t

What function y(t) has as its derivative 2y/t ?

HA14: Other Numerical Techniques

• Interpolate between data points, using either linear or cubic spline models.• Perform numerical differentiations• Perform numerical integrations• Solve differential equations numerically• Root Finding and Minimization

Interpolation: Summary

• MATLAB includes an interpolation technique called interp1.

• Its is the most commonly used function for linear interpolations.

• A cubic spline technique is also available.

0 1 2 3 4 5 60

2

4

6

8

10

12A Data Plot

x-axis

y-ax

is

-1 0 1 2 3 4 5 6

0

2

4

6

8

10

12

14

16Measured Data

x-axis

y-ax

is

Interpolated Point

Interpolated Points

Linear Interpolation – Connect the points with a straight line to find y

Linear InterpolationAssume the function between two points is a straight line.

Linear Interpolation: MATLAB Code

• interp1 is the MATLAB function for linear interpolation.

• First define an array of x and the y values.• Now define a new x array, that includes the x values

for which you want to find y values new_y=interp1(x,y, x_new)

Interpolation: Function interp1

1-D Syntaxynew = interp1(xorig,yorig,xnew,’method’)

Where ynew - new y value(s)xorig, - original x valuesyorig - original y valuesxnew, - new x value(s) where we want the new y’method’ - interpolation technique

Where ‘method’ can be:• 'nearest‘ - nearest

neighbor interpolation• 'linear‘ (default)• 'spline‘ (cubic spline

interpolation• 'cubic‘ (piecewise

cubic)

• Interpolation is the technique of creating new data points within existing points

• MATLAB has built-in interpolation functions for both 1-D and 2-D interpolation.

-1 0 1 2 3 4 5 6

0

2

4

6

8

10

12

14

16Measured Data

x-axis

y-ax

is

Other Interpolation Techniques• Cubic Spline - A cubic spline creates a smooth curve, using a third degree polynomial• MATLAB includes other interpolation techniques

including– Nearest Neighbor– Cubic– Two dimensional interpolation: interp2– Three dimensional interpolation: interp3

We can get an improved estimate by using the spline interpolation technique

-1 0 1 2 3 4 5 6

0

2

4

6

8

10

12

14

16Cubic Spline Interpolation

x-axis

y-ax

is

Cubic Spline Interpolation. The data points on the smooth curve were calculated. The data points on the straight line segments were measured. Note that every measured point also falls on the curved line.

Interpolation: DemoPerform the following: >> x = 0:12;

>> y = cos(x);

>> xi = linspace(0,12,50);

>> yi = interp1(x,y,xi);

>> yi2 = interp1(x,y,xi,’spline’);

>> plot(x,y,‘bo',xi,yi,’r-’,xi,yi2,’g--’)

• Create a vector from 0 to 12

• Evaluate cosine at every x value.

• Create a more fine spacing

• Create new y values at the finer x spacing.

• Repeat the interpolation but with a cubic spline technique.

• Plot the three datasets – the original, the default interpolation and the cubic spline interpolation

0 2 4 6 8 10 12-1

-0.5

0

0.5

1

1.5

Note the different results from the different

interpolation methods

Data Interpolation: Examplex = 0:.5:4*pi;y = sin(x) + sin(2*x)/2;plot(x,y,'bo-') % shows linear

% Now assume have only the points, but not the functionxx = 0:.1:4pi % finer samplingyy = interp1(x,y,xx);hold onplot(xx,yy,'r*');% zoom in: looks the same: interp1 does linear fit by default

yyy = interp1(x,y,xx,'spline');plot(xx,yyy,'b+');

ytrue = sin(xx) + sin(2*xx)/2;plot(xx,ytrue,'g');

% better, is it more accurate?

Numerical Derivatives

• The diff function finds the difference between adjacent elements in an array

• It can be used to approximate a derivative.

Function diff

• Numerical differentiation uses the diff function.• The diff function is easy to understand, even if you

have not taken calculus.• It just calculates the difference between the points in

an array.• If we know how y changes with x, we could create a

set of ordered pairs for any number of x values. • The more values of x and y, the better the

approximation of the slope.

-1 0 1 2 3 4 5 60

2

4

6

8

10

12

14

16Sample Data

x-axis

y-a

xis

2 1

2 1

y yslope

x x

3 2

3 2

y yslope

x x

4 3

4 3

y yslope

x x

5 4

5 4

y yslope

x x

6 5

6 5

y yslope

x x

The derivative of a data set can be approximated by finding the slope of a straight line connecting each data point

Function diff: Example

Numerical Derivative– Finite differences can be used to compute a

numerical derivative.

– MATLAB provides a built-in function for calculating numerical derivative using the forward difference method. ii

ii

xx

yyforward

x

y

dx

dy

1

1 :

It is easy to create your own difference formula by taking

advantage of MATLAB’s vector

processing.

Numerical Derivative: Function diff

Syntaxynew = diff(y,order)

Where:y – data to differentiateorder – order of the derivativeynew – forward differenced data

If you have a function that you want to differentiate, first evaluate the function using a small Dx, then put the resulting data through diff

To compute a derivativeyou need to use diff on both the y and x data

DyDx

The slope is an approximation of the derivative – in this case based on data measurements

Numerical Derivative: DemoPerform the following: >> xi = 0:0.25:12; >> yi= xi.*2 + xi; >> yi_diff1 = diff(yi,1); >> yi_diff2 = diff(yi,2); >> xi_diff1 = diff(xi,1); >> xi_diff2 = diff(xi,2); >> plot(yi,’b-’); >> hold, plot(yi_diff1./xi_diff1,’r--’); >> plot(yi_diff2./xi_diff2,’g:’);

• Create a vector from 0 to 12• Evaluate your function• Calculate the first y difference• Calculate the second y difference• Repeat for the x’s

• Plot your three different lines, making sure to divide by the Dx’s.

0 5 10 15 20 25 30 35 40 45 500

5

10

15

20

f(x)

f'(x)

f''(x)

Numerical Integration

• MATLAB handles numerical integration with two different quadrature functionso quad uses adaptive Simpson quadratureo quadl uses adaptive Lobatto quadrature

• Both functions require the user to enter a function in the first field. – called out explicitly as a character string – defined in an M-file– anonymous function.

• The last two fields in the function define the limits of integration

0 2 4 60

5

10

15

An integral can be approximated by the area under a curve

y-ax

is

x-axis0 2 4 6

0

5

10

15A Trapazoid rule approximation

y-ax

isx-axis

The area under a curve can be approximated using the trapezoid rule.

An integral is often thought of as the area under a curve

Numerical Integration: Geometrical Derivation

Numerical Integration– You integrate to either determine the area under

the curve, or as part of a differential equation.– There are times when you cannot integrate a

function analytically.– The trapezoidal method is a common technique for

computing area under the curve.– If noise in your data is a problem, fit the data to a

function or smooth your data with averaging.

1-n

1i1

1

2 iiii xx

yy

Numerical Integration … continue

Syntaxarea = trapz(x,y)

Where x – vector of x datay - vector of y data

• MATLAB provides several built-in functions for computing area under the curve

– trapz function calculates area for data sets

– quad function calculates area for functions

area = quad(‘fn’,lo,hi)

Where fn – the function to evaluatelo – lower boundhi – upper bound

trapz function:

>> x = 0.001:0.1,15;

>> func = x.^3 ./ (exp(x) - 1);

>> area = trapz(x,func)

area =

6.4926

quad function:

>> area = quad('x.^3 ./ (exp(x) - 1)',001,15)

area = 6.4927

Calculating an Integral: Demo

Normally you would use the trapz function if you did not

have a function and only data

1)(

3

xe

xxf Integrate the following function:

Rounded answer: 6.4939

Data Problems

• Problem: Noisy Data– Do not really want to exactly fit to the points– Real question: what is the "best fit"

• 'polyfit(x,y,n)' – minimizes "squared error"– n is "degree" of the polynomial– returns coefficients– polyval(coeffs,X) evals at vector X using those coeffs

• Splines go through sparse data (knot points); curve fitting approximates data (handles noise)

Real data is typically ugly

Interpolation vs Extrapolation

• Interpolation is MUCH better behaved – has at least two points to help.

• Extrapolation is often really ugly – Check different order of polyfit using values of 6 and -1

Numerical Derivative: Smoothing– Noise in the data will impact the

calculation and requires work-arounds.

• Fit the data to a function, thendifferentiate the function analytically or with numerical methods.

• Filter (smooth) the data, then differentiate the smoothed data.

– Filtering can change calculated derivative values.

– Watch out for a shift in your data when using moving averages.

– Use the “Less is More” approach.

-14

-12

-10

-8

-6

-4

-2

0

0 10 20 30 40

Time

Pre

ss

ure

-150

-100

-50

0

50

100

150

0 10 20 30 40

Time

dP

/dT

?

-14

-12

-10

-8

-6

-4

-2

0

0 10 20 30 40

Time

Pre

ssu

re

Original Data 30 point moving average

Data Averaging (Smoothing)• Data averaging can be accomplished through convn and filter.

– These functions use a mathematical technique called convolution.

– In convolution, you have a data set, and what is termed a kernel. The weighting of the average depends on the values used in the kernel.

– The kernel is “slid” along the data set, elements are multiplied together, the results are summed and a new data set is built from the results.

Kernel values should sum to 1 or else you will change the mean

level of your data!

Unequal kernel values will give a weighted average

Data Averaging (Smoothing)

Syntax (for our case we’ll just deal with vectors)

ysmooth = convn( yorig , kernel , 'shape‘ )Where:

yorig – row or column data to smoothkernel – row or column vector of filter weightsshape – ‘same’ returns the same length as the

original data set.

ysmooth = filter( kernel, 1, yorig)Where:

yorig – row or column data to smoothkernel – row or column vector of filter weights

convn with the same option does not cause a data shift, whereas filter will shift the data by the number of points used in the smoothing.

Perform the following:Load the decaying sine data file from above (save to your desktop, then double-click it in the Current Directory window, or use the load command)

>> kernel = ones(5,1)./5;

>> C = convn(ysine,kernel,'same');

>> D = filter(kernel,1,ysine);

>> plot(xsine,ysine,'.',xsine,C,xsine,D)

>> legend('data','5 pt avg - convn','5 pt avg - filter')

Data Averaging (Smoothing): Demo

• Create a [5x1] vector where each value = 0.2 and the sum = 1

• Moving avg with convn• Moving avg with filter

• Plot data and build a legend

0 5 10 15-1

-0.5

0

0.5

1

1.5

data

5 pt avg - convn

5 pt avg - filter

Notice how filter shifted the data

Root and Minima Locating

Syntax

yzero = fzero(‘fcn’,xguess)

Where fcn – function to searchxguess – guess at the zero

• MATLAB provides built-in functionality finding the zeros and minima of a function.

ymin = fminsearch(‘fcn’,xguess)

Where

fcn – function to search

xguess – guess at the minimum

fzero function:

>> x = linspace(-5,5,100);

>> y = x.^2-4.*x-8;

>> plot(x,y)

>> zero = fzero('x.^2-4.*x-8',0)

zero =

-1.4641

fminsearch function:

>> minx = fminsearch('x.^2-4.*x-8',0)

minx = 2

Root and Minima Locating: Demo

84)( 2 xxxf Find the root & minimum of this function:

-5 0 5-20

-10

0

10

20

30

40

(-1.4641,0)

(2,-12)

Roots of Function: Example

• Example: find roots of the function f(x)=sin(x) >> x0 = 3.1 ; % starting point >> tol = 0.01; % desired accuracy >> trace = 1; % show intermediate results >> z1 = fzero('sin(x)', x0) >> z1 = fzero('sin(x)', x0, tol) >> z1 = fzero('sin(x)', x0, tol, trace)

System of Non-linear Equations

• First write the definition for the system of the equations in the file eqns.m

function eq=eqns(p); %EQNS defines the left hand side of equations % The system of the nonlinear equations to be solved is % x^2 + y^2 - 3xy = 0 % x^2 + y = 0

x = p(1); y = p(2); eq = zeros(2,1); eq(1) = x^3 + y^3 - 3*x*y; eq(2) = x^2 + y;

• Let us try to find a solution on different intervals: >> fsolve('eqns', [-5, 5]) >> fsolve('eqns', [-2, 2])

Symbolic Derivatives

Symbolic Toolbox is an optional toolbox that allows symbolic computations in MATLAB.It must be installed separately on your computer.

Example: Partial Derivative of Gaussian

Symbolic Integration

Definite Integral: Example

Symbolic Bounds of Integration

Improper Integrals (of one type)