Computational Methods forEngineering Applications I

Spring semester 2017

Prof. J. H. Walther, Prof. P. KoumoutsakosETH Zentrum, CLT F 12CH-8092 Zürich

ExamIssued: August 25, 2017, 15:00

Short Questions

Question 1:

a) FALSE - it is of order N-1 (you leave out one point)

b) TRUE

c) FALSE - of order N-1, not highest

Question 2: B-Splines

a) M B-splines are cosntructed from a knot vector withM+d+1 entries. We have 4 B-splinesand a knot vector with 8 entries. Therefore, d = 8− 4− 1 = 3.

b) The depicted B-splines are "clamped". The end knots (0,1) where the splines are clampedobtain the value B=1. Plus, the end knots are repeated in the knot vector.

Question 3: true/false

a) TRUE

b) FALSE - you don’t have to recalculate coefficients

c) FALSE - it also has to be orthogonal to y2

d) TRUE

Question 4: true/false

a) FALSE - knots must be ordered

b) TRUE - M + d+ 1 ≤ 5 and M ≥ 1 so at most, d = 3.

c) FALSE - M + d + 1 ≤ 5, so we get maximum M for d = 0 (M = 4). But because ofrepeated knots, only 2 basis functions can be obtained.

Question 5: Rate of convergence

1

a) We know that the error of the composite trapezoidal rule behaves as εTRPZ ∼ O(h2)and the error of the composite Simpson’s rule behaves as εSMP ∼ O(h4), where h is thesubinterval size. With h′ = h/2 the error decrease for the Simpson’s rule is larger than thatfor the trapezoidal rule. Therefore, from the graph, the steeper line in the logarithmic scalecorresponds to the Simpson’s rule (dashed line = Simpson’s). The black continuous linecorresponds to the trapezoidal rule.

b) If we use N2 = 10N1, we can calculate the change in the subinterval size as:

h2 =b− aN2

=b− a10N1

=h110

The error of the trapezoidal rule will decrease as O(h2), so:

e2 = O(h22) = O(h21100

)

The error will be 100 time less than with the original number of points.

Question 6: Order of accuracy

a) The numerical method approximates a quantity of interest depending on a parameter h(might denote e.g. the interval size of a numerical integration method). As h → 0, theerror e between the numerical result and the quantity of interest decreases. We can writethe error as a polynomial in h. The exponent of h in the lowest-order term is called orderof accuracy.It is important because it tells how fast the error of a numerical method goes to zero as wedecrease the interval size/mesh size h. In practice, we can only use finitely small h, so thatthe error will not vanish. The order of accuracy tells us how much the error will decrease ifwe decrease h by a certain factor (e.g. of 2). We can use this to compare different methods.One can find it either by Taylor expansion, or by applying the numerical method to anexample function and observing how the error decreases when h is decreased.

b) Not only the accuracy counts, but also how small the error is - there is a constant in front ofthe power in h (monomial in h). If we know that we cannot make h smaller than a certainthreshold, then we should choose the method that has the smallest error for this h. Thiscould also be the method with lower order of accuracy. If we can make h arbitrarily small,then at some point the method with higher order accuracy will always have the smaller error.(In practice, one might also take into consideration how easy the method is to implement.)

Question 7: Least Squares

a) Any point on the regression line will do.To show that α and β remain the same, consider that α and β minimize the sum of squarederrors (SSR) of the fit for N points. For a new point the error is 0. Hence α and β alsominimize SSR for N + 1 points.

b) Choose one point that is not on the line and the second point y-mirrored against the line(not simply mirrored!!). Third point can be anywhere on the line but not between the firstand the second points.

2

Question 8: Newton’s Method

a) Starting from (i), it is possible to find the root of the function (0,0), function is smoothand differentiable towards 0. If you start from (ii), you will fall into the asymptote, solutionwill diverge from actual root.

b) Either point you start from, impossible to find the root (0,0), because derivative is notdefined. The method will diverge

c) Either point you start from (i, ii, iii) you will be able to find one of the three roots of thefunction

d) Newton’s method will not work, function has no roots!

Question 9: Numerical integration

a) Which integration method(s) would you use in the following situations? Give a short justi-fication.

1. I know that the function is very smooth. Function evaluations are costly. Gauss in-tegration because it is most efficient and good for smooth functions (but for smoothfunctions, all methods will work well)

2. I want to implement integration quickly for a proof of concept. High accuracy is notso important at this stage. Composite Trapez/Simpson because they are most simpleto implement

3. The function has a non-continuous first derivative at a few known locations. anymethod on the subintervals

4. The function has a non-continuous first derivative at a few unknown locations. Adaptiveintegration because it is able to refine around the kinks

5. Function evaluations are cheap and I want to achieve a certain error threshold. Rombergor adaptive integration because they can be easily used to achieve a certain errorthreshold. The error must be estimated by Richardson error estimation.

6. I want to integrate a function in N > 1 variables. Monte Carlo integration because theother methods become very inefficient in higher dimensions.

b) A polynomial of order 11 has 12 parameters. In N-C the weights are degrees of freedom,thus we need 12 weights. In Gauss the weights AND abscissas are degrees of freedom, thuswe need 6 each.

3

Numerical Problems

Question 10: Cross Validationy = b:

r1 = 1/2, r2 = 1/2, r3 = 1

r21 = 1/4, r22 = 1/4, r23 = 1

Cross validation error is a sum of squares of the residues: 3/2.

y = ax+ b:r1 = 1, r2 = 1/2, r3 = 1

r21 = 1, r22 = 1/4, r23 = 1

Cross validation error is 9/4. Model y = b gives a smaller cross validation error.

Question 11: Newton’s Method

a) Consider the function f ∈ C1(R). We expand around x0 up to first order:

f(x0 + dx) = f(x0) + f ′(x0)dx+O(dx2).

We want to solve for dx = x1 − x0 such that f(x1) = 0. We neglect the second orderterms. Therefore, the above equation becomes

0 = f(x1) = f(x0 + dx) = f(x0) + f ′(x0)dx,

which gives

dx = − f(x0)

f ′(x0),

so we obtain the Newton iteration:

x1 = x0 −f(x0)

f ′(x0).

b) The solution of f(x) = 0 is given by x̄ = −b/a.We now perform Newton on f from x0 ∈ R:

x1 = x0 −f(x0)

f ′(x0)= x0 −

ax0 + b

a= − b

a= x̄.

This result is valid for any starting point x0, so Newton’s method takes exactly one iterationto converge.

c) Solving cos(x) = ex − 1 is equivalent to solve f(x) = 0 with f(x) = cos(x) + 1 − ex.Therefore, one iteration of Newton gives

x1 = x0 −f(x0)

f ′(x0)= x0 +

cos(x0) + 1− ex0sin(x0) + ex0

.

4

Question 12: Order of accuracy and Richardson extrapolationThe derivative of a function f can be approximated by by the so-called central finite differencemethod:

f ′(x) ≈ f(x+ h)− f(x− h)

2h.

a) Show that this method has order of accuracy 2. Assume that the function f is infinitelyoften differentiable.

f(x+ h) = f(x) + hf ′(x) +h2

2f ′′(x) +

h3

6f ′′′(x) + . . .

f(x− h) = f(x)− hf ′(x) +h2

2f ′′(x)− h3

6f ′′′(x) + . . .

Using these two equations, we can compute

f(x+ h)− f(x− h)

2h=

2hf ′(x) + 2h3

6f ′′′(x) + . . .

2h

= f ′(x) +h2

6f ′′′(x) + . . .

This means the method has order of accuracy 2.

b) Derive the formula for the first step of Richardson extrapolation.Above we derived

G0(h) = f ′(x) +h2

6f ′′′(x) + . . .

G0(h/2) = f ′(x) +h2

24f ′′′(x) + . . .

It follows that

G1(h) =4

3(G0(h/2)− 1

4G0(h)).

c) From the above it follows that the derived method has order of accuracy 3.

Question 13: Ellipsea) The narrower histogram (dashed line) corresponds to the rectangular domain. ‘Better’

sampling spaces (i.e, smaller domains) leads to smaller variance.

b) The error must scale as 1/√M , which indicates that the correct curves are the ones with

the open symbols. As we already deduced in part (a), the rectangular domain should havesmaller absolute error. Hence open circles = circular domain, and open triangles (smallerabsolute error) = rectangular domain.

Question 14: Gauss Quadrature

5

Question 13: Ellipse (8 pnts)(4 pts per answer)

a) The narrower histogram (dashed line) corresponds to the rectangular domain. ‘Better’

sampling spaces (i.e, smaller domains) leads to smaller variance.

b) The error must scale as 1/pM , which indicates that the correct curves are the ones with

the open symbols. As we already deduced in part (a), the rectangular domain should have

smaller absolute error. Hence open circles = circular domain, and open triangles (smaller

absolute error) = rectangular domain.

Question 14: Lagrange Interpolation (10 pnts)a) (6 pnts)

Since we know that the relation is of 2nd order, we can apply Lagrange interpolation for

N=3 data points. We select to interpolate using t1 = 4, t2 = 5, t3 = 10:

L1(t) =(t� 5)(t� 10)

(4� 5)(4� 10)=

t2� 15t+ 50

6

(1 pt)

L2(t) =(t� 4)(t� 10)

(5� 4)(5� 10)=

t2� 14t+ 40

�5

(1 pt)

L3(t) =(t� 4)(t� 5)

(10� 4)(10� 5)=

t2� 9t+ 20

30

(1 pt)

x(t) =3X

i=1

yiLi(t)

= 28 ·t2� 15t+ 50

6� 40 ·

t2� 14t+ 40

�5+ 130 ·

t2� 9t+ 20

30= t

2 + 3t

(1)

(2 pts)

So, using the resulting quadratic polynomial, we calculate:

x(8) = 88 [m]

b) (3 pnts) As long as there is no noise in the data, the interpolating polynomial will always

be the same as above (1 pt).

With N = 5 five points he will get Lagrange basis polynomials of order N � 1 = 4. With

N = 3 the order of the basis polynomials is 2 and with N = 4 the order of the lagrangian

basis polynomials is 3.(1 pt)

When constructing the interpolating function, higher order terms of the Lagrangian basis

polynomials will cancel each other out, always resulting in the same quadratic polynomial.

(1 pt)

5

c) (1 pt)

If we used a quadratic function, we would use LSQ to fit the coefficients, according to our

data set. Since there is no noise in the data set, the resulting fitting polynomial will be the

same as above (subquestion a) .

Question 15: Gauss Quadrature (10 pnts)a) For interval [�1, 1] (1 pnt):

IG = f

�

r1

3

!+ f

r1

3

!.

For interval [a, b] (1 pnt):

IG =b� a

2

g

�

r1

3

!+ g

r1

3

!!,

where g(x) = f�a+ b�a

2 (x+ 1)�.

b) Consider the general polynomial of degree 3: p(x) = ax3 + bx

2 + cx+ d. Exact solution (2

pnts):1Z

�1

p(x)dx =2

3b+ 2d,

Gauss Solution (2 pnts):

IG(p) = p

�

r1

3

!+ p

r1

3

!

=2

3b+ 2d.

Therefore using Gauss gives the same result for any polynomial of degree 3.

c) Consider the special case: p(x) = x4. Exact solution (2 pnts):

1Z

�1

p(x)dx =2

5,

Gauss result (2 pnts):

IG(p) =

�

r1

3

!4

=2

9,

therefore IG(p) 6=1R

�1

p(x)dx.

6

a) For interval [−1, 1]:

IG = f

(−√

1

3

)+ f

(√1

3

).

For interval [a, b]:

IG =b− a

2

(g

(−√

1

3

)+ g

(√1

3

)),

where g(x) = f(a+ b−a

2(x+ 1)

).

b) Consider the general polynomial of degree 3: p(x) = ax3 + bx2 + cx+ d. Exact solution:

1∫−1

p(x)dx =2

3b+ 2d,

Gauss Solution:

IG(p) = p

(−√

1

3

)+ p

(√1

3

)=

2

3b+ 2d.

Therefore using Gauss gives the same result for any polynomial of degree 3.

c) Consider the special case: p(x) = x4. Exact solution:

1∫−1

p(x)dx =2

5,

Gauss result:

IG(p) =

(−√

1

3

)4

=2

9,

therefore IG(p) 6=1∫−1p(x)dx.

Question 15: CircleDenote by G(n) the perimeter of n-sided polygon:

G(n) = 2n sinπ

n

Richardson extrapolation is

R(n) =4G(2n)−G(n)

3

6

Values are

G(4) = 4√

2 ≈ 5.66,

G(8) = 8

√2−√

2 = 16 sin(π

8

)≈ 6.12,

R(4) ≈ 6.28

Given that circumference of a circle is 2π ≈ 6.28 we can compute the errors:

E(4) ≈ 0.00489

Question 16: Gram-Schmidt orthonormalizationa) Φ is not an orthonormal set of basis functions, as φ2 and φ3 are not orthogonal to each

other:

〈φ2, φ3〉 =

∫ π

−πx · sin(x) dx

= (−xcos(x))|π−π +

∫ π

−πcos(x) dx

= −2πcos(π) + 0

= 2π 6= 0

b) We follow the Gram-Schmidt orthonormalization process to get the orthonormalized basisset φ̂1, φ̂2, φ̂3 :

φ̂1(x) =1

(∫ π−π 11 dx)

1/2=

1√2π

φ̃2(x) = φ2 − φ̂1

∫ π

−πφ̂1φ2 dx

= x− 1√2π

∫ π

−π

1√2πx dx

= x− 1

2π

[x2

2

]π−π

= x

φ̂2(x) =x

(∫ π−π x ·x dx)

1/2=

x√[x3

3

]π−π

=

√3

2π3x

φ̃3(x) = φ3 − φ̂1

∫ π

−πφ̂1φ3 dx− φ̂2

∫ π

−πφ̂2φ3 dx

= sin(x)− 1√2π

∫ π

−π

1√2πsin(x) dx−

√3

2π3x

∫ π

−π

√3

2π3x · sin(x) dx

= sin(x) +1

2π[cos(x)]π−π −

3

2π3x

∫ π

−πx · sin(x) dx

= sin(x)− 3

π2x

7

φ̂3(x) =sin(x)− 3

π2x

(∫ π−π(sin(x)− 3

π2x) · (sin(x)− 3π2x) dx)

1/2

=sin(x)− 3

π2x√π − 6

π

,where we computed:

∫ π

−π(sin(x)− 3

π2x) · (sin(x)− 3

π2x) dx =

∫ π

−π(sin2(x)− 6

π2x · sin(x) +

9

π4x2) dx

=

∫ π

−π

1− cos(2x)

2dx− 6

π22π +

[3

π4x3]π−π

= π − 6

π

Question 17: See scanned solution

8

Question 18: Cubic splinesa) From g = f − S, we can write

U [g] = U [f − S]

=

b∫a

[f ′′(x)− S ′′(x)]2dx

=

b∫a

[f ′′(x)]2

+ [S ′′(x)]2 − 2f ′′(x)S ′′(x)dx

= U [f ] + U [S]− 2

b∫a

f ′′(x)S ′′(x)dx,

so we obtain the desired result

U [f ] = U [S] + U [g]− 2

b∫a

S ′′(x)g′′(x)dx.

b) By integrating by parts, one obtains

b∫a

g′′(x)S ′′(x)dx = [g′(x)S ′′(x)]ba −

b∫a

g′(x)S ′′′(x)dx

= −b∫

a

g′(x)S ′′′(x)dx,

where we used the natural boundary conditions on the cubic spline, i.e., S ′′(a) = 0 andS ′′(b) = 0.

c) Let us denote the N data points as x1 = a, x2, . . . , xN = b. We can split the integral intothe N − 1 subintervals:

b∫a

g′(x)S ′′′(x)dx =N−1∑i=1

xi+1∫xi

g′(x)S ′′′(x)dx

=N−1∑i=1

Ci

xi+1∫xi

g′(x)dx

=N−1∑i=1

Ci [g(x)]xi+1

xi

=N−1∑i=1

Ci (g(xi+1)− g(xi))

= 0,

11

where we used the fact that S are third order polynomial on each intervals so their thirdderivatives are constants Ci ∈ R. The last equality is due to the fact that S and f interpolatethe data, so g(xi) = f(xi)− S(xi) = 0.

d) From questions (b) and (c), we easily obtain that

b∫a

S ′′(x)g′′(x)dx = 0 ,

which, combined with (a), gives

U [f ] = U [S] + U [g] .

As the energy U is always positive, we obtain that U [S] ≤ U [f ] for any interpolatingfunction f ∈ C2([a, b]), which concludes the proof.

Question 19: See scanned solution

12

Question 20: See scanned solution

15

Pseudocode

Question 21: See scanned solution

18

Algorithm 1 Gram-Schmidt Orthonormalization for vectorsInput:

x1, . . . , xK , {set of linearly independent vectors of length N}Output:

u1, . . . , uK , {orthonormal set of vectors of length N}//Helper function: scalar product of two vectorsfunction scalarproduct(v1, v2, N) //two vectors and their length

tmp← 0for i← 1, . . . , N do

tmp← tmp+ v1[i] ∗ v2[i]end forreturn tmp

end function

Steps:

//First vector: only normalizationnorm← sqrt(scalarproduct(x1, x1, N))for i← 1, . . . , N do

u1[i]← x1[i]/normend for

//All other vectors: orthogonalization and normalizationfor j ← 2, . . . , K do

uj ← xj

//orthogonalizefor prev ← 1, . . . , j − 1 do

sc← scalarproduct(uprev, xj, N)for l← 1, . . . , N do

uj[l]← uj[l]− sc ∗ uprev[l] //projectionend for

end for//normalizenorm← sqrt(scalarproduct(uj, uj, N))for i← 1, . . . , N do

uj[i]← uj[i]/normend for

end for

1

Question 22: Monte Carlo Samplinga) We choose to sample for our integration from an orthogonal domain, containing both circles.

Therefore, with (x1, y1) = (0,0) and R1=5 and (x2, y2) = (12,0) and R2=10:

• x should be in the interval [-5,22]: We have to transform the random values taken byrandom() to this interval. Sampling in the x-direction will be:x = random() ∗ 27− 5

• y should be in the interval [-10,10]: We transform sampling in the y-direction:y = random() ∗ 20− 10

Algorithm 1 Monte Carlo integration for overlapping areaInput:

circle center x2circle radii R1, R2

number of Monte Carlo samples MOutput:

overlap area of 2 circles

Steps:R1 = 5R2 = 10x2 = 12M = 10000

function overlapMC(x2, R1, R2,M)double area_rectangle= (R1 + x2 +R2) ∗ 2. ∗R2

int pts_inside = 0for int i:=0; i < M; i++ do

double xi = (x2 +R2 +R1) ∗ random()−R1

double yi = 2 ∗R2 ∗ random()−R2

if (xi ∗ xi + yi ∗ yi ≤ R1 ∗R1) and ((xi − x2) ∗ (xi − x2) + yi ∗ yi ≤ R2 ∗R2) thenpts_inside ++

end ifend forreturn (double) pts_inside / M * area_rectangle

end function

b) In general, if you want to compute the MC error, you have to keep both function evaluationsAND function evaluations squared. In the specific case, the function is the unit function, ifthe random point is inside the overlapping area.

c) For M2 = 10M1 we know that MC error behaves as: εMC ∼√|V ar|M

. Therefore, εMC−2 =εMC−1√

10

d) If we use 2-times the sampling space, and we sample from a uniform distribution, then thevariance will be 22 times larger. Therefore, using the same relation as above, the MC errorwill be 2 times larger.

20

Good luck!

21