numerical analysis, lecture 1: introductionbutler.cc.tut.fi/~piche/numa/lecture0102.pdf ·...

Numerical Analysis, lecture 1: Introduction

• Course info

• What is numerical analysis? (textbook §1.1-2)

function example2[t,Y] = ode45(@rocket,[0 40],[0 0]);plot(t,Y(:,1),'linewidth',3)xlabel('t'), ylabel('h','rot',0), box off function dy = rocket(t,y)v = y(2); m = max(180-10*t,0);M = 120+m;dy = [v ((5000+10*v)*(t<=18)-0.1*v*abs(v))/M - 9.81];

0 5 10 15 20 25 30 35 400

500

1000

1500

2000

2500

3000

t

h

Numerical Analysis, lecture 1, slide ! 2

Operational info for course MAT-31102/31107

Lecturer

Lectures (in Finnish)

Textbook (read it!)

Tutorials

Bring your laptop with Matlab or Octave to the tutorials

Lars Eldén, Linde Wittmeyer-Koch, Hans Bruun Nielsen: Introduction to Numerical Computation, Analysis and Matlab Illustrations, Studentlitteratur, 2004.

43!

prof. Robert Pichéautomation science and [email protected]

http://www.tut.fi/~piche/numa

see POP (schedule) & home page (exercises & model quiz questions & solutions)

see POP (schedule) & home page (slides)

Numerical Analysis, lecture 1, slide !

Home page

3

http://www.tut.fi/~piche/numa

MAT-31102 / 31107 Numeerinen analyysi / Numerical analysis

Perehdytään tieteellisen ja teknillisen laskennan perusmenetelmien teoriaaan ja käyttöön. Theory and practical application of essential numerical methods for scientific and engineering problem solving.

Opettajat

luennoitsija/lecturer Robert Piché, assari/teaching assistant Jari Niemi

Sisältö / Contents

• Johdanto, Virheanalyysi / Intro, Error analysis [slides, harjoitus/exercises, solutions]!• Tietokonearitmetiikka / Computer Arithmetic [slides, harjoitus/exercises, solutions]!• Epälineaarisen yhtälön ratkaiseminen / Solving nonlinear equations [slides,!harjoitus/exercises, solutions bisection illinois newtonraphson secant ]• Funktioiden interpolointi! / Function interpolation [slides, harjoitus/exercises, solutions, neville, osculate]• Numeerinen integrointi / Numerical integration [slides,!harjoitus/exercises, solutions]• Funktioiden approksimointi / Function approximation [slides, harjoitus/exercises, solutions, lspolyfit]• Tavallisten differentiaaliyhtälöiden alkuarvotehtävät / ODE initial value problems [slides, harjoitus/exercises, solutions]

Oppikirja / Textbook

Lars Eldén, Linde Wittmeyer-Koch, Hans Bruun Nielsen: Introduction to Numerical Computation, Analysis and Matlab Illustrations, Studentlitteratur, 2004. [errata]

Pikakokeet / Quizzes

Harjoitustunnilla annetaan pikakokeet, josta saa bonuspisteitä. The weekly tutorial sessions include quizzes that give you bonus points.

Tentti / Exam

Tavallinen tai graafinen/ohjelmoitava laskin ja yksi A4 kaksipuolinen sivu käsinkirjoitetut muistiinpanot sallittu. MallikysymyksiäOrdinary or graphing/programmable calculator and one A4 two-sided page of handwritten notes allowed. Model exam


Exercise sets & PC labs

4

theory problems

problems from the textbook

Matlab/Octave questions (bring your laptop!)

modelling problems

answers! (complete solutions are posted after one week)

Numerical Analysis, Exercises for lectures 1–2 [§1.1–2, 2.1–3]1 a. Determine the absolute error and the relative error of 106/39 as an approximation of

the natural logarithm base e.b. How many correct decimals does x̄ = 0.9951 have as an approximation of x =

0.9949? How many significant digits?c. Some books say that x̄ has t correct decimals if x̄ rounded to t decimals is equal

to x rounded to t decimals. By this definition, how many correct decimals doesx̄ = 0.9951 have as an approximation of x = 0.9949?

2 How accurately do we need to know π in order to compute√

π with 4 correct decimals?[textbook page 37 exercise E1]

3 Derive the error propagation formula ∆ff � α1

∆x1x1

+ α2∆x2x2

+ α3∆x3x3

for the functionf(x1, x2, x3) = xα1

1 xα22 xα3

3 . [page 38 exercise E3]

4 Reformulate or approximate the following Matlab/Octave expressions to avoid cancellationfor x � 0, and compute their values for the specified x value. [page 38 exercise E5]

a. exp(x)-exp(-x), x = 10−5

b. 1-cos(x), x = 10−5

c. 1/(sqrt(1+x^2)-sqrt(1-x^2)), x = 10−2

5 For any x0 > −1, the sequence recursively defined by the formula

xn+1 = 2n+1��

1 + 2−nxn − 1�

converges to ln(1 + x0 + 14x2

0). However, computing the sequence for x0 = 4 in Mat-lab/Octave gives completely incorrect results. Try it:

x=4; for n=1:100, x=2^(n+1)*(sqrt(1+x/2^n)-1); end; x

Explain the problem, and fix it.

6 A gas exerts a pressure of 0.892 atm (all decimals correct) in a 5.00 L container (all decimalscorrect) at 15± 1 degrees Celsius. How much gas is there? (Hint: look up the ideal gaslaw.)

7 The following code computes the zeros of the quadratic polynomial ax2 + bx + c using thewell-known formula:

d = sqrt(b^2-4*a*c);r1 = -(b-d)/(2*a);r2 = -(b+d)/(2*a);

Try it with the polynomial x2 − 109x + 1, which has 2 positive real zeros. Which zerois not computed accurately, and why? Try again, but with the last two lines replacedby:

r1 = -(b+sign(b)*d)/(2*a);r2 = c/(a*r1);

Answers

1. (a) ∆x � −3.3 · 10−4, ∆xx � −1.2 · 10−4. (b) 3 correct decimals, 3 significant digits.

(c) 1 and 3 but not 2

2. |π̄ − π| ≤ 1.8 · 10−4

4. (a) 2.00000000003333e-05 (b) 4.99999999995833e-11 (c) 9.99999998750000e+03

5. use xn+1 = 2xn/(√

1 + 2−nxn + 1)

6. 0.1886 ± .001 mol


Miniquizzes give bonus points

5

quiz during exercise period

Numerical Analysis 1 — Quiz questions for lectures 7-8

Find the elements of the first column of the following Romberg table for� 1

0 e−x2dx.

Answer:h T1(h) T2(h) T3(h)

1 0.683940.5 0.73137 · · ·0.25 0.74298 · · · · · ·

Solution: We can determine the elements of the first column of the Romberg tablewith the trapezoidal rule for a corresponding step length. With the step length h = 1we need values of function f(x) = e−x2

at f(0) = f0 = 1.0000 and f(1) = f1 = 0.3679.Now

T1(1) = h

�f0

2+

f1

2

�= 1 ·

�1.0000

2+

0.3679

2

�= 0.68394

The recursive formula T (h) = 12T (2h)+h(f1 + f3 + f5 · · ·+ fm−1) presented on page 172

gives us the other elements.

T1(0.5) = 0.5 · T1(1) + h(f(0.5)) = 0.5 · 0.68395 + 0.5 · 0.77880 = 0.73137

T1(0.25) = 0.5 · T1(0.5) + h(f(0.25) + f(0.75))

= 0.5 · 0.73137 + 0.25 · (0.93941 + 0.56978) = 0.74298

Compute the integral

� 0.1

0

cos x

x13

dx using the approximation cosx ≈ 1− x2

2

Answer:

� 0.1

0

cos x

x13

dx ≈ 0.3228

Solution:

� 0.1

0

cos x

x13

dx ≈� 0.1

0

1− x2

2

x13

dx =

� 0.1

0

�x−

13 − 1

2x

53

�dx =

� 0.1

0

3

2x

23 − 3

16x

83 ≈ 0.3228

Using Simpson’s rule with the step length h = 14 find an approximation for

� 1

0 e−x2dx.

Answer:� 1

0 e−x2dx ≈ 0.7468

Solution:

S(h) = S

�1

4

�=

h

3(f(0) + 4f(0.25) + 2f(0.5) + 4f(0.75) + f(1))

=1

12(1.0000 + 4 · 0.9394 + 2 · 0.7788 + 4 · 0.5698 + 0.3679) = 0.7468

1

Numerical Analysis 1 — Quiz questions for lectures 7-8

Find the elements of the first column of the following Romberg table for� 1

0 e−x2dx.

Answer:h T1(h) T2(h) T3(h)

1 0.683940.5 0.73137 · · ·0.25 0.74298 · · · · · ·

Solution: We can determine the elements of the first column of the Romberg tablewith the trapezoidal rule for a corresponding step length. With the step length h = 1we need values of function f(x) = e−x2

at f(0) = f0 = 1.0000 and f(1) = f1 = 0.3679.Now

T1(1) = h

�f0

2+

f1

2

�= 1 ·

�1.0000

2+

0.3679

2

�= 0.68394

The recursive formula T (h) = 12T (2h)+h(f1 + f3 + f5 · · ·+ fm−1) presented on page 172

gives us the other elements.

T1(0.5) = 0.5 · T1(1) + h(f(0.5)) = 0.5 · 0.68395 + 0.5 · 0.77880 = 0.73137

T1(0.25) = 0.5 · T1(0.5) + h(f(0.25) + f(0.75))

= 0.5 · 0.73137 + 0.25 · (0.93941 + 0.56978) = 0.74298

Compute the integral

� 0.1

0

cos x

x13

dx using the approximation cosx ≈ 1− x2

2

Answer:

� 0.1

0

cos x

x13

dx ≈ 0.3228

Solution:

� 0.1

0

cos x

x13

dx ≈� 0.1

0

1− x2

2

x13

dx =

� 0.1

0

�x−

13 − 1

2x

53

�dx =

� 0.1

0

3

2x

23 − 3

16x

83 ≈ 0.3228

Using Simpson’s rule with the step length h = 14 find an approximation for

� 1

0 e−x2dx.

Answer:� 1

0 e−x2dx ≈ 0.7468

Solution:

S(h) = S

�1

4

�=

h

3(f(0) + 4f(0.25) + 2f(0.5) + 4f(0.75) + f(1))

=1

12(1.0000 + 4 · 0.9394 + 2 · 0.7788 + 4 · 0.5698 + 0.3679) = 0.7468

1

• study the model questions beforehand!• closed-book quiz, one question per week• grading: zero or one• total bonus 0-6 points (exam is 30 points)


What previous students say about this course

6

Luennoitsijalle palautetta: vaikka opiskelijat ovatkin huonoja nauramaan luennoilla, älä anna tämän hämätä sinua: heitit luennoilla hyvää läppää, jatka samaan malliin! Tämän takia ajattelin hiukan positiivisemmin "aikaista" kouluunlähtöä.

Harjoitukset tukivat hyvin luentoja ja asiat aukesi oikeastaan harkkojen myötä vasta kunnolla. Hyödyllinen kurssi, olisi kannattanut jo aikaisemmin käydä.

Opetus oli hyvää mutta aihe niin kuivaa että kurkku on vieläkin kipeänä

Ihan hyvä kokonaisuus. Enemmän laskuesimerkkejä luennoilla, jos mahdollista.

Tässä kurssissa oli mielestäni järjestelyt ja muut systeemit kohdallaan.

Palautettahan ei pitäisi antaa, enne kun näkee arvosanansa, että voi sitten sopivalla kitkeryydellä asennoitua tämän palautteen kirjottamiseen.

do y! have any questions?


Numerical simulations are important in engineering & natural sciences…


“The objective of numerical analysis is to construct and analyse methods to solve practical computational problems”

• computational problems

- 2 small examples

• NA topics

• issues

• history


a computational problem is solved in 4 stages

• collect data and formulate a mathematical model

get(x) x > 0? disp(x)

x = -(x+1)^2

• find an algorithm (method)

• compute

5

1

25

10= 20

0

efficiencyratio

00

• look at results


A small example: how high does a thrown ball go? (page 2)

Model & data

Algorithm

Computation & Result>> v0 = 25; >> g = 9.81; >> h = (v0^2)/2/g

h =

31.8552

12mv0

2 = mgh, g = 9.81m ! s"2 , v0 = 25m ! s"1

h = v02

2g


Another example:how high does a rocket go? (p. 2)

Model & data

Algorithm

Computation & Resultfunction example2[t,Y] = ode45(@rocket,[0 40],[0 0]);plot(t,Y(:,1)) function dy = rocket(t,y)v = y(2); m = (180-10*t)*(t<=18);M = 120+m;Mp =-10*(t<=18);T = 5000*(t<=18);d = 0.1*v*abs(v);dy = [ v (T-9.81*M-d-Mp*v)/M ];

Use an adaptive Runge-Kutta method to solve the ODE IVP

!v =T " Mg " d " !M v

M, !h = v, v(0) = 0, h(0) = 0

0 5 10 15 20 25 30 35 400

500

1000

1500

2000

2500

3000

t

h

fuel mass m = 180 !10t, thrust T = 5000 (0 " t " 18), rocket mass M = 120 + m, drag d = 0.1v v , Newton's law (Mv #) = T ! Mg ! d


This course will introduce most of the main topics of numerical analysis

• understand floating point computation• solve nonlinear equations• approximate functions• compute integrals• solve ordinary differential equations

NA topics are also covered in other courses: • matrix analysis MAT-31090/31096

• optimization MAT-41120

• intro tech. comp. with Matlab MAT-45700/45706


A numerical analyst studiesthe mathematical properties of algorithms (p.6)

• how many operations does the algorithm need?

• how fast does the iteration converge?

• how accurate is the approximation?

Solving Ax = b with Gaussian elimination requires 13 n

3 +O(n2 ) multiplications.

!f (x) =f (x + h) " f (x " h)

2h+O(h2 )

(quadratic convergence)In Newton-Raphson method, limn!"

xn+1 # x$

xn # x$ 2 = K


We will frequently use Taylor series

14

f (x) = 1n!f (n)(c)(x ! c)n

n=0

N

"SN

! "### $###+

1(N +1)!

f (N+1)(#)(x ! c)N+1

RN! "##### $#####

Taylor polynomial and remainder

truncation error

standard functions

ex = 1+ x + 12!x2 +

13!x3 +!

sin x = x ! 13!x3 +

15!x5 !!

cos x = 1! 12!x2 +

14!x4 !!

ln(1+ x) = x ! 12x2 +

13x3 !!

(1+ x)p = 1+ px + p(p !1)2!

x2 +p(p !1)(p ! 2)

3!x3 +!


what happened, what’s next

Computational problems are solved in 4 stages: build a mathematical model, devise an algorithm, compute, and look at results.

Numerical analysis is concerned mainly with stage 2, i.e. inventing and analysing numerical algorithms.

Core topics: solving equations, approximating functions, computing integrals, and solving differential equations.

Typical questions: how accurate is it? how fast does it converge? how many operations does it require?

Next lecture: error analysis (§2.1-3)

Numerical Analysis, lecture 2: Error Analysis(textbook sections 2.1–3)

• sources of error

• error characterizations

• sensitivity & cancellation

• the right way to solve

!f (x )"x !f

!x

ax2 + bx + c = 0


There are many kinds of “error” (p. 9-10)

!

"

###

$

###

this course

human error bugs and blunders

model error simplifications and idealizations

data error measurements of physical quantities

truncation error approximations of infinite sequences, integrals, limits, etc.

rounding error rounding of input data & in arithmetic operations


There are many ways to describe the errorof an approximation of a real number (p. 11, 13)

absolute & relativea : exact, a : approximateabsolute error of a : a ! a (=:"a)relative error of a :"a a (provided a # 0)

correct decimalsa has t correct decimals if !a " 0.5 #10$t

bounding interval a ![a " #,a + #] is written as a = a ± #

for example:

a = 200 = 14.14213562!

a = 14.14 = 1.414 !101

"a = # 0.00213562!

"aa

= #0.0001510!

"a $ 0.005

a = 14.14 ± 0.005

a has 2 correct decimals

a has 4 significant digits

significant digits

a = d0.d1d2!!10E (d0!0)

has s significant digits if "a # 0.5 $101+E%s


These examples illustrate the various ways of describing error

a = ! = 3.14159265!

a =355113

"a =

"aa

=

"a # 0.5 $10

a = 355113

± 0.5 $10

a has correct decimals

a has significant digits

a = 0.001234 ± 0.5 !10"5

a has 5 correct decimals and 3 significant digits

a = 56.789 ± 0.5 !10"3

a has 3 correct decimals and 5 significant digits

a = 210000 ± 5000a has 2 significant digits

a = 1.789 ± 0.005

a has correct decimals and significant digits

p. 13:

2.667.10-7

8.49 .10-8

-6

-6

6

7 2 3


Rounding increases approximation error (p. 11-12)

a Matlab function to round to t decimals>> Round = @(x,t) round(x*10^t)/10^t;>> Round( [1.23767, 0.774501, 6.3225, 6.3235], 3)

ans =

1.238 0.775 6.323 6.324

“round to nearest even” would give 6.322 !

rounding error boundsif a = (a rounded to t decimals) then a = a ± 0.5 !10"t

if b = b ± # and b = (b rounded to t decimals) then b = b ± (# + 0.5 !10"t )

example (p. 12)

rounding b = 11.2376 ± 0.1 to 1 decimal gives b = 11.2 ± 0.15

proof: b ! b = b ! b + b ! b " b ! b + b ! b


In addition, absolute errors accumulate;in multiplication, relative errors (p. 13-16)

Addition!(x1 + x2 ) " !x1 + !x2

!(x1 " x2 ) # !x1 + !x2

Subtraction

Examplex1 = 123.4 ± 0.05 = 123.4 ! (1± 0.0004052)x2 = 122.1± 0.05 = 122.1 ! (1± 0.0004095)

x1 " x2 = 1.3 ± 0.1

x1x2

= 1.010647 ! (1± 0.0008147)

= 1.010647 ± 0.00082Division

!(x1 x2 )x1 x2

<!!x1x1

+!x2x2

Multiplication!(x1x2 )x1x2

"!x1x1

+!x2x2

+!x1x1

#!x2x2

$ !x1x1

+!x2x2

!(x1 + x2 ) = (x1 + x2 ) " (x1 + x2 )

= (x1 " x1) + (x2 " x2 )

# x1 " x1 + x2 " x2

proof


Which solar collector is most efficient? (p.14, 17)

! = K QTdIEfficiency:

Collector 1! = 0.76, "Q

Q= 1.5%, "Td

Td= 1%, "I

I= 3.6%

"!!

# "!!

<$"QQ

+"TdTd

+"II

= 6.1%

! = 0.76 % (1± 0.061) = 0.76 ± 0.046 & 0.714 ' ! ' 0.806

Collector 2! = 0.70, "Q

Q= 0.5%, "Td

Td= 1%, "I

I= 2%

"!!

# "!!

<$"QQ

+"TdTd

+"II

= 3.5%

! = 0.70 % (1± 0.035) = 0.70 ± 0.025 & 0.675 ' ! ' 0.725


!f (x )"x !f

!x

We can approximate the error that ispropagated by a univariate function… (p. 14-15)

Formula:

!f = f (x ) " f (x) = #f ($)!x for some $ %[x, x ]Proof using Mean Value Theorem:

f (x) = f (x ! "x) = f (x ) ! #f (x )"x + 12 ##f ($)("x)2

Proof using Taylor’s formula:

a = 2.05 ± 0.01, f (a) = a = ?Example (p.15)

!f " #f (x )!x

!f (a) = 12 a

, "f <!0.01

2 2.05# 0.0035

$ a = 2.05 ± 0.0035, i.e. a %[1.42828,1.435329]

verify: a ! 2.04, 2.06"# $% = [1.4282856857,1.4352700094]


…and by a multivariate function (p. 17-18)

!f " #f

#x1(x ) $ !x1 +

#f#x2

(x ) $ !x2 +!

Example (p.18)

x1 = 0.75 ± 0.01, x2 = 0.413 ± 0.003, y = sin(x12x2 ) = ?

y = sin(x12x2 ) = 0.230229

!y <! 2x1x2 cos(x12x2 ) " !x1 + x12 cos(x1

2x2 ) " !x2 # 0.0077

$ y = 0.23 ± 0.008

∆ f (x1,x2, . . . ,xn) = ?


Cancellation is the loss of significant digitsdue to subtraction of nearly equal values (p. 19-20)

Example (slide 7)

x1 = 123.4 ± 0.05, x2 = 122.1± 0.05 (4 significant digits)x1 ! x2 = 1.3 ± 0.1 (1 significant digit)

loga ! logb = log ab

To avoid cancellation, reformulate using identities

1+ x ! 1! x = 1+ x ! 1! x( ) " 1+ x + 1! x1+ x + 1! x

=2x

1+ x + 1! x

or approximate using a Taylor polynomial1− cosx

x≈ x

2− x3

24


There is a wrong way and a right way to compute quadratic roots

Example (p.19)

(3 significant digits)

(5 significant digits)

Solve x2 !18x +1 = 0

x1 = 9 + 80 " 9 + 8.9443 = 17.9443

x2 = 9 ! 80 " 9 ! 8.9443 = 0.0557

x2 =1

9 + 80" 1

17.9443 " 0.0557280

Vieta's formula is obtained by substituting x = 0 into ax2 + bx + c = a(x ! x1)(x ! x2 )

The roots of ax2 + bx + c = 0 are x1,2 =

!b ± b2 ! 4ac2a

" cancellation when 4ac ! b2

A better formula is: x1 =! b + sign(b) " b2 ! 4ac( )

2a, x2 =

cax1


what happened, what’s next

• Sensitivity analysis studies how error intervals propagate in calculations.

• Rounding adds an absolute error.

• Absolute error accumulates in addition & subtraction, relative error in multiplication & division

• Taylor formula for error propagation

• Cancellation can sometimes be avoided by changing the formula.

Next lecture: computer arithmetic (§2.4-8)

numerical analysis, lecture 1: introductionbutler.cc.tut.fi/~piche/numa/lecture0102.pdf ·...

Documents