Part 3: A Very Basic Introduction to Calculus

Measuring & Modeling the World in Motion

“And one and all they had a longing to get away from this painfulness,this ceremony which had reminded them of things they could not bear tothink about – to get away quickly and go about their business and forget.”John GalsworthyThe Man of Property (from The Forsyte Saga)

1 / 78

Capturing changing behaviour

So far ...

We have mainly viewed “measurement” as a rather static notion.e.g. a fixed

length (L) of a line with given properties (L2 = 2, L3 = 20, etc)size of population (eg people living in a town)amount paid to one individual in a month.

The notion of function is a starting point for going beyond this staticworld view.

2 / 78

What do functions do?

Most simplified description

A function, f , is just a way of describing what should happen to anumber (x) when the function is applied to it.

Meaning?

Functions f “take” numbers (x) and “return” numbers (y).

f : Takes→ Returns

The relation between x and what f produces from x : y = f (x);

The domain from which x is taken: Takes = N or Takes = R, etc

The range in which y is given – ie Returns = N or Returns = R, etc

Domain and range do not have to be the same: we can haveTakes = R and Returns = N (we will use Takes = Returns = R).

3 / 78

A few examples

We use the notation f : D→ R, instead of “takes” & “returns”

round : R→ Z: round(x) is the integer “closest” to x .

square : R→ R: square(x) = x2

recip : R→ R: recip(x) = 1/x

pow2 : R→ R: pow2(x) = 2x

polyex : R→ R: polyex(x) = 3x3 + 5x2 − x + 59

Combining functions

We may use the output of a function to supply the input of another(or the same function).

This is called composition. If f and g are functions: f (g(x)) denotesthe process “apply f to the result of applying g to its input x”.Some texts use the notation (f ◦ g)(x).

4 / 78

Some important examples

Trigonometric Functions: sin & cos

(Natural) Logarithms & Exponential

5 / 78

Trigonometric Functions – A brief overview

Angles: radians and degrees

rsinθ

r cos

θ

π /2/2π

θ

θπ/2 − r

π /2

A circle has 360◦ ≡ 2π radians.The values of sin θ and cos θ are the lengths of the lines shown(when the circle has radius r = 1)We return to these functions when we look at Complex Numbers.

6 / 78

An important basic class of function:Lines and linear functions

Benefits

a. So-called “Linear functions” allow us to consider movement in“straight” lines.

b. There is a simple rule that allows to find the “equation” of the(unique) line connecting two points: {< x1, y1 >,< x2, y2 >}.

c. This rule tells us immediately how “steep” the line is: that is to sayits gradient.

d. Computation using lines underpin the original development ofdifferential calculus by Isaac Newton in the 17th century.

7 / 78

Some Examples

y = x : “Output” value identical to input value.

y = x + 20: Output value adds 20 to the input.

y = 2x : Output value multiplies input by 2.

y = 2.5x + 7: multiply by 2.5 and add 7.

8 / 78

Linear Functions in the “real” (ie actual) world

We find many examples in practice of “linear functions” (although notnamed as such):

Casino games such as roulette: a given stake (x) may return:

2x : (x is staked on red and red results: wins x+ original stake)37x : (x is staked on a single number which is the outcome: wins36x+ original stake).

Horse-racing “odds”: “prices” are offered in terms such as 7− 1,100− 30, etc corresponding to the linear functions 8x and (130/30)x .[NB the amount staked is included here].

Driving speeds: “50 miles per hour” describes the distance travelledin x hours as the linear function 50x .

9 / 78

The unique line joining two points in 2-dimensions

Given two points – < x1, y1 > and < x2, y2 >

We mentioned that there is a unique line connecting these points.

That is: a unique (linear) function f (x) that given any x reports thevalue of y so that the point < x , y > is on this line.

How do we find out what this function is?

S. Find the “steepness” of the line: how much y should increaserelative to increase in x .

O. Find what point should be on the line when x = 0 (the offset)

L. Let the steepness be m and offset be c . The line function is then:

y = mx + c

10 / 78

Finding the gradient

Steepness is just another term for “gradient”

We’re trying to find “by how much” y should increase whenever xincreases by some amount.

We know that < x1, y1 > and < x2, y2 > are points on the line.

So we know that: the increase from x1 to x2 should produce anincrease from y1 to y2.

In other words the “gradient” is simply,

How much y should change

(Relative to) how much x has changed

Hence the value we’re looking for is:

m =y2 − y1x2 − x1

11 / 78

Finding the offset

What do we know already?

We know < 0, c > is on the line (but don’t know what c is).

We know < x1, y1 > is on the line (we started from there).

We know the line has gradient m (we’ve just found this out)

This gradient does not change: it’s a line.

So the gradient going from < 0, c > to < x1, y1 > isexactly the same asthe gradient going from < x1, y1 > to < x2, y2 >.

In other words, c must be such that (when x = 0)

m =y2 − y1x2 − x1

=y1 − c

x1 − 0

In other words, c = y1 − mx1.

12 / 78

Diagrammatic view

(x1,y1)

(x2,y2)

y2−y1

x2−x1

C

xy= + Cy2−y1

x2−x1

x

y

If we know m and a single point < p, q > on the line

y = q + m(x − p)

13 / 78

Lines, Curves, and Limits – or“I thought Calculus was meant for ‘difficult’ functions”

Lines can only do so much

There are some computations where the functions involved are not“easily” viewed in terms of lines.

For example,

a. A car starts from place 0, with initial speed 20 kmph. It increases itsspeed by 10 kmph every second. What speed will it be travelling atafter 5 minutes? How far will it have gone in this time?

b. A business has sold 3000 units and starting from an initial turnover of200 units per year expects to increase its sales each year by a furtherconstantly increasing rate of 30 units per year each year. How manyyears will the company take to sell 20, 000 units?

c. The tortoise has a 100metre start on Achilles. If in each secondAchilles reduces the distance left to catchup by half, how many secondswill he take to overtake the tortoise?

These are questions about “rates of increase”: the answers will not befound by “linear functions”.

14 / 78

Some example Curves

What’s happening here?

and here?

15 / 78

Linear Approximation – going back to Lines

The origins of differential calculus

Two ideas are fundamental to this:

The idea of “limit”: how things behave as some value is“approached”.

The notion of “linear approximation”: finding the line “bestdescribing” a function.

Using a line to describe a “curve”

We start with some function: F (x) – such as the two on the previousslide.

We choose two places, x = a and x = b and find the values F (a) andF (b).

Use the line connecting < a,F (a) > to < b,F (b) > to “reasonabout” F (x).

We know how to find the unique line joining two points.

16 / 78

An example

The first case from above

That’s a bit rubbish isn’t it?

The line joining < −10,F (−10) > to < 5,F (5) > doesn’t seem to tell usmuch about F (x).Certainly doesn’t suggest that F (x) = x2 (the actual function drawn).

17 / 78

Only because we’re not being careful

What are we actually trying to find out?

What we’re seeking is some information about how F (x) behavesas a “line” when x = 5

In particular what can we discover about the line “touching”< 5, 25 > or, in general, the line “touching” < x , x2 >.

The line between < −10, 100 > and < 5, 25 > tells us something.

But we can do better.

Moving closer to the point of interest

We started by using the line between < −10, 100 > and < 5, 25 >.What if we used < −5, 25 > instead?Then < −2, 4 > then < 0, 0 >, < 2, 4 >, . . .

What happens?

The “line” “looks” like a “better fit” because ofthe change in its gradient.

18 / 78

Tabular form of how the line changes

Approximating point Line function to < 5, 25 > Gradient

< −10, 100 > y = −5x + 50 −5

< −5, 25 > y = 25 0

< −2, 4 > y = 3x + 10 3

< 0, 0 > y = 5x 5

< 2, 4 > y = 7x − 10 7

< 3, 9 > y = 8x − 15 8

19 / 78

What do these look like?

20 / 78

In summary

The “closer” < x , x2 > is to < 5, 25 > the “better” the quality ofinformation the line between the two points provides about thegradient of the “line touching” the curve at < 5, 25 >.

But we cannot use the line from < 5, 25 > to < 5, 25 > itself.(Easy question: Why not?)

Key question

How do we see what’s happening at the point of interest?

For the example under consideration:

We want to know what’s going on at the point < 5, 25 >.

We look at the line between < 5, 25 > and < x , x2 > with x getting“close to” 5:That is, set x = 5 + d and let d become “very very small”.[In general for a point < x , x2 > we look at points< x + d , (x + d)2 >]

21 / 78

We want to find a gradient

Which gradient?

The gradient of the line joining < 5, 25 > to < 5 + d , (5 + d)2 >.[In general, the gradient of the line between< x , x2 > and < x + d , (x + d)2 >]

We know how to do this, it’s

(5 + d)2 − 25

5 + d − 5=

(5 + d)2 − 25

d

a bit of rearranging

(5 + d)2 − 25

d=

25 + 10d + d2 − 25

d= 10 + d

22 / 78

Two critical observations

The “cancellation” of d is “valid” because d 6= 0.

So we now know:The “nearer d gets to 0 the nearer the gradient (of the “linetouching” < 5, 25 >) gets to 10”.

A note on terminology

We have referred to the “gradient of the line touching a point on a curve”.In formal usage, where the habit of obscuring simple language through theemployment of Latin phraseology is second nature, the expression“gradient of the tangent at a point . . .” is often used.There is no good reason for doing this (outside rigorous formal settingsand the subject matter of paintings by Titian, Correggio, Bronzino etc.):the Latin word tangent translates to the English word touching, cf theinfluence on words such as “tango”.

23 / 78

What about points other than < 5, 25 >?

We can proceed in the same way for < x , x2 >

Consider the gradient of a line between< x + d , (x + d)2 > and < x , x2 >.

See how this gradient behaves as d gets very small.

This gradient is

(x + d)2 − x2

x + d − x=

x2 + 2xd + d2 − x2

d=

2xd + d2

d= 2x + d

In other words. . .

The gradient of the line touching the point < x , x2 > on the curve y = x2

approaches the value 2x .

24 / 78

Review – What just happened?

The ideas developed to show that

“The gradient of the line touching the point < x , x2 > on thecurve f (x) = x2 is ‘best described by’ the value of the functionf (x) = 2x”

are, usually, “formally” expressed as

limd→0

(x + d)2 − x2

d= 2x

25 / 78

There are a lot of strange symbols here

What is this in English?

The jargon “limd→0” just captures our notion of choosing a point< (x + d), (x + d)2 > “closer and closer” to the point < x , x2 >, ie“letting d become as close to (without reaching) 0.

Limits

This is referred to as a “limit”. There are some extremely subtle andnon-intuitive aspects involved in a proper treatment of limits. We do not,however, need to deal with these in this module.

26 / 78

Differentiating functions: the (first) derivative

Derivatives

For a function (where we use f : R→ R) the notion of

“the function for the gradient of the line touching the point < x , f (x) >”

is referred to as

The (first) derivative of the function f (x).

Various notations exist for this: we use f ′(x) or dydx

The general form of f ′(x)

The method we used to show f ′(x) = 2x for f (x) = x2 extends toarbritrary f :

f ′(x) = limd→0

f (x + d) − f (x)

d

27 / 78

A few warnings

Some complications deliberately omitted

The definition

f ′(x) = limd→0

f (x + d) − f (x)

d

is only “valid” for “well-behaved” f .

Functions may be well-behaved for some values of x but not others.

One example of such a function is f (x) = 1/x when x = 0.

In general, for our purposes we can ignore these subtleties.

So I have to go through this process every time to find f ′(x)?

There is no need to use “first principles” methods to find f ′(x).

Why not?

Because the derivatives have already been found: why repeat work?

28 / 78

8 Simple Rules for Finding A Derivative

Function form h(x) Rule name First Derivative h′(x)

1 Constant 0

f (x) + g(x) Sum f ′(x) + g ′(x)

f (x)g(x) Product f ′(x)g(x) + f (x)g ′(x)

f (x)

g(x)Quotient

f ′(x)g(x)− f (x)g ′(x)

g(x)g(x)

f (g(x)) Composition (Chain Rule) f ′(g(x))g ′(x)

xn Polynomial nxn−1

sin x (or cos x) Trigonometric cos x (or − sin x)

exp x (or log x) Exponential (or Logarithm) exp x (or 1/x))

29 / 78

What these rules mean

Using these rules and (known) function definitions allows us to findderivatives “fairly easily”.

For example, the trigonometric function tan x . Since

tan x =sin x

cos x

we can use the Quotient Rule.

Similarly, for arbitrary log functions, eg log2 we can use

log2 x =log x

log 2(Natural logarithms)

It is important to understand how to use these rules.

Understanding is not the same as memorizing : you do not need to beable to “parrot” them.

30 / 78

Summary of this Part of the module so far

What we’ve seen already

Differential calculus is concerned (among other things) with reasoningabout the behaviour of functions.

An important aspect being functions describing “the gradient of theline touching (the graph of) a function at the point < x , f (x) >”.

This function is called the “first derivative of f (x)” denoted f ′(x).

In the next lecture we shall see that these ideas offer very powerfulcomputational tools for optimization problems, ie those concernedwith minimizing or maximizing quantities.

31 / 78

Part 3 continued

What derivatives tell us about functions

32 / 78

Recap of earlier material

What do we already know?

The “first” derivative f ′(x) of a function f (x) describes the gradient(or steepness) of “the line touching the point < x , f (x) > for (thegraph or plot of) the function f (x)”.

So f ′(x) is itself a function.

We also have a set of rules that, in principle, allow us to find f ′(x) for“most” functions we are likely to meet.

In this section we consider the following

What further information can be gleaned from this gradient?

Since there’s a “first” derivative does this mean there are also second,third etc derivatives?

If so, do these give useful computational data?

33 / 78

Some example Function Graphs and Selected Lines

and a bit more complicated

34 / 78

The gradient in the first examples

The function f (x) = x2 with derivative f ′(x) = 2x

The diagram shows the line touching x2 for two distinct points:

< −5, 25 > and < 5, 25 >

The first of these lines has gradient f ′(−5) = 2× (−5) = −10.

The second has gradient f ′(5) = 2× 5 = 10.

The first line “slopes downwards”, ie the corresponding line functiony = −10x − 25 gets “smaller” as x gets “larger”.

The second line “slopes upwards”, ie the corresponding line functiony = 10x − 25 gets “larger” as x gets “larger”.

As “x increases from −5” (while the gradient is < 0) x2 decreases:−4 > −5 but 16 < 25.

As “x increases from 5” (gradient being > 0) x2 increases:6 > 5 and 36 > 25.

This is NOT a coincidence.

35 / 78

What about the second example?

The function in this is f (x) = 2x3 − 2x2 − x − 1

Once x gets large enough, f (x) always increases.

But there is an interval (roughly when −0.5 ≤ x ≤ 1) when itsbehaviour is less clear.

The line shown is that touching the point < −5,−296 >.

This gradient function is f ′(x) = 6x2 − 4x − 1 and f ′(−5) = 169.

The line function is thus y = 169x + 549.

The gradient is greater than 0, so this function is increasing at thispoint.

36 / 78

What if the gradient of the touching line is 0?

Like this?

or this

(the function f (x) 2x3 − 2x2 − x − 1 when −0.5 ≤ x ≤ 1 from earlier)37 / 78

Turning Points, 2nd derivatives and maxima and minima

Turning points

The line with gradient m = 0 touches the curve f (x) = x2 at< 0, 0 >.

When x < 0 all lines touching < x , x2 > have negative gradients(slope downwards).

When x > 0 all lines touching < x , x2 > have positive gradients(slope upwards).

When x = 0 (so that m = 0) the curve is at a “turning” point. Itsvalue is (clear from diagram) a minimum.

Not all cases are this clear

What if we can’t “see” the shape of the curve?

Can we still tell what happens to its function?

Yes: we look at the second derivative: f ′′(x) the derivative of f ′(x).

38 / 78

f (x) = x2 revisited: The Second Derivative Test

f ′(x) = 2x ; f ′′(x) = 2

The second derivative of x2 is a constant which is positive, ie > 0.This tells us that 0 (the value of f ′(x) = 2x where the line touching< x , x2 > is horizontal) is a (local) minimum for the function.That is: every value of x , (with x 6= 0) is such that f (x) > f (0).In fact (for this function) x = 0 is also a global minimum.

Turning points for f (x) = 2x3 − 2x2 − x − 1 when −0.5 ≤ x ≤ 1

1. Find f ′(x): f ′(x) = 6x2 − 4x − 1 (recall Table of Rules).

2. When is f ′(x) = 0? (the values of x for zero gradient at < x , f (x) >).

3. f ′(x) = 0 for two values of x : (Recall Part 2 of the module)x = (1/3) +

√5/18 ∼ 0.86 and x = (1/3)−

√5/18 ∼ − 0.19.

4. Find f ′′(x): f ′′(x) = 12x − 4.

5. f ′′((1/3) +√

5/18) > 6.3: positive so a local minimum;f ′′((1/3)−

√5/18) < − 6.3: negative so a local maximum

39 / 78

Confirming this is so

f ((1/3) +√

5/18) ∼ − 2.067 ; f ((1/3)−√

5/18) ∼ − 0.896

40 / 78

In total

To analyse turning points and properties for f (x)

Compute f ′(x) the form of the first derivative.

Find the value(s) of x for which f ′(x) = 0.

These give the turning points of f (x) and where (local) minima andmaxima occur.

Compute f ′′(x) the second derivative.

For each turning point, t, compute the value f ′′(t):

If f ′′(t) < 0 then f (t) is a local maximum.If f ′′(t) > 0 then f (t) is a local minimum.If f ′′(t) = 0 then we cannot make any conclusions.

41 / 78

Optimization

Background

So-called Optimization Problems arise throughout Computer Science.The general form of such problems can be summarised as:

We have a collection of resources, R, which we use to achieve somegoal, G .

Reaching this goal produces a reward or “benefit”, b.

When we use resources, however, there is a cost, c .

The problem isHow to use resources so as to

minimize the cost or maximize the benefit?

42 / 78

Some examples

The League of Legends PC Game

In this game players have a Health score (H) and available armour score(A). So R is a combination of these two. Depending on these players havean effective health (E ) (the benefit element) calculated by

E =H(100 + A)

100

Additional Health and Armour units may be purchased at a given cost of2.5 tokens (for Health) and 18 tokens for armour.How should an initial allocation of T tokens be used to maximize E ?

43 / 78

Another example

Medication effectiveness

Typically drugs such as caffeine are absorbed into the bloodstream over aperiod of time (given a single dose), during which some of the drug will bebe eliminated.Using α for the absorption rate; β for elimination rate, the concentrationfor an initial dosage D after time t, is known to be given by the function,

c(t) =D

1− β/α( exp(−βt)− exp(−αt) )

How long does it take for the concentration to be maximal?

This example is from a recent (2009) clinical study:K.-Y. Seng, C.-Y. Fun, Y.-L. Law, W.-M. Lim, W. Fan, and C.-L. Lim.,Population pharmacokinetics of caffeine in healthy male adults usingmixed-effects models.Jnl. of Clinical Pharmacy and Therapeutics, 34:103–114, February 2009.

44 / 78

How does calculus help solve these? - League of Legends

What are we trying to maximise?

We want to spend (some part) x of the allocation of T tokens over H andA so as to maximise E .If x units of Health are purchased then (T − 2.5x)/18 are available forarmour.It is sufficient to consider “how much should be spent on H”.In summary we want to choose x to maximize

E (x) = x +x(T − 2.5x)

1800= x +

xT

1800− 2.5x2

1800

That is to say,

E (x) =

(1800 + T

1800

)x − x2

720

45 / 78

What does this look like?

For a starting number of tokens T = 3600

Now,

E ′(x) =1800 + T

1800− x

360

so that E ′(x) = 0 when x = (360)(1800 + T )/(1800), ie

x = 360 + (T/5)

46 / 78

The medication example

Although the time function is more involved (but still within the Table ofRules given) we have a similar approach.

c(t) =D

1− β/α( exp(−βt)− exp(−αt) )

To find c ′(t), notice that D, α and β are constants. For exp(−ct) usethe Chain Rule: f (g(t)) with f (x) = exp(x) and g(x) = −cx .We get,

c ′(t) =D

1− β/α( α exp(−αt) − β exp(−βt) )

Rearranging things,

c ′(t) =D

1− β/α

(α exp(βt) − β exp(αt)

exp((α + β)t)

)47 / 78

Continuing example

This isn’t as intimidating as it appears: exp(x) is always positive

To find when c ′(t) = 0 we only need to consider

α exp(βt) − β exp(αt)

so that we choose t (the time to reach maximum concentration) withα exp(βt) = β exp(αt) which gives c ′(t) = 0 when

t =logα − log β

α− β

(with slightly different notation you will find this verified on page 108(second column) of the paper referenced).

48 / 78

More than one variable?

Problem in “real” cases

The techniques just outlined are fine when when only have a functionof a single variable to optimize.

In many real applications, however, functions with many variables,< x1, x2, . . . , xn > arise.

Optimization methods must find a setting for all n variables in orderto maximize or minimize such functions.

49 / 78

A small example

Suppose we have a surface,

Figure : The function z = −(3x2 + 2y2 + xy) between −2 ≤ x , y ≤ 2

and wish to find the best setting for x and y?

50 / 78

Partial Derivatives - brief summary

OS1. For a fixed value of x (α say) a line describes a single variablefunction of y :

z = f (α, y) = −(3α2 + 2y2 + αy).

OS2. From (OS1), for any α we can find the first derivative of f (α, y), sothere is some βα that optimizes f (α, βα).

OS3. We can do the same with any fixed value of y to get a single variableof function of x :

z = f (x , β) = −(3x2 + 2β2 + βx).

OS4. So we see there is some choice (α, β) for (x , y) which maximizes

z = f (α, β) = −(3α2 + 2β2 + αβ).

51 / 78

How do we find (α, β) to maximizef (x , y) = −(3x2 + 2y 2 + xy)?

(almost) the same way as before

a. Find f ′(α, y) and write the expression for “optimizing”, y .

b. Find f ′(x , β) and write the expression for “optimizing”, x .

c. Solve the simultaneous equations so that the values for < x , y >optimize both f (α, y) and f (x , β).

The function described in (a) is called the partial derivative of f w.r.t.y and denoted by fy (or ∂f /∂y).

∂f

∂x= −6x − y

∂f

∂y= −4y − x

52 / 78

Then what?

Need to solvefx = −6x − y = 0fy = −4y − x = 0

Only solution is x = y = 0.

How do we know if (0, 0) is maximal or minimal or something else?

We need to use a more involved form of the “second derivative test”.

fx (∂f /∂x) is a function of both x and y .

fy (∂f /∂y) is a function of both x and y .

So we can define: fxx , fxy , fyx , fyy : the second partial derivatives.

53 / 78

2nd order partial derivatives

fxx =∂(−6x − y)

∂x= − 6

fxy =∂(−6x − y)

∂y= − 1

fyx =∂(−4y − x)

∂x= − 1

fyy =∂(−4y − x)

∂y= − 4

54 / 78

The 2nd Order derivative test

a. ConstructD(x , y) = fxx fyy − f 2

xy

b. If D(α, β) ≤ 0: cannot deduce anything about f (α, β).

c. If D(α, β) > 0 and fxx(α, β) > 0: minimum

d. If D(α, β) > 0 and fxx(α, β) < 0: maximum

e. If D(α, β) > 0 and fxx(α, β) = 0: ??

Note

fxy and fyx are always identical.

We could also use fyy (α, β) > 0, etc, in (c)–(e): if fxx(α, β) > 0 andfyy (α, β) ≤ 0 then D(x , y) ≤ 0 and the test could not be applied.

55 / 78

3 or more variables

Most of these concepts carry over to 3 or more variables.If f : Rm → R with arguments x =< x1, . . . , xm > we have m (first order)partial derivatives {

∂f

∂x1,∂f

∂x2, . . . ,

∂f

∂xm

}Finding critical points of f (x) hinges on finding solutions for the system ofm equations {

∂f

∂xi

}(x) = 0

Analysis of critical points in higher dimensions uses an analogue of the“second derivative” style methods

56 / 78

The Hessian Matrix

D(x , y) (the 2-variable case) can be presented in terms of the determinantof a 2× 2-matrix. Write the 4 2nd order partial derivatives of f (x , y) as

H2 =

(fxx fxyfyx fyy

); det H2 = fxx fyy − fxy fyx = D(x , y)

For arbitrary m ×m matrices. Hm = [ fxixj ], i.e.

Hm =

fx1x1 fx1x2 · · · fx1xm−1 fx1xmfx2x1 fx2x2 · · · fx2xm−1 fx2xm

......

. . ....

...fxm−1x1 fxm−1x2 · · · fxm−1xm−1 fxm−1xm

fxmx1 fxmx2 · · · fxmxm−1 fxmxm

The matrix Hm is called the Hessian and is important to advanced imageprocessing and computer vision.

57 / 78

Summary

What differential calculus brings to Computer Science

By studying gradients of functions we gain information about thefunction’s properties.

This allows us to consider details about when a function switchesfrom increasing to decreasing (and vice-versa).

In consequence to reason about maximal and minimal values.

These thereby provide powerful tools for optimization problems whicharise in numerous areas of Computer Science from Algorithms,Machine Learning, and Robotics.

In the next lectures we look at the other main branch of calculus:Integral Calculus

58 / 78

Part 3 continued further

Integral Calculus – Antiderivatives & Area

59 / 78

In this lecture

We first examine some issues raised by the fact that both f (x) andf ′(x) are functions.

Introduce the notion of the antiderivative, F (x), of a function.

Illustrate that the antiderivative of f (x) is directly linked with thenotion of “measuring the area covered between any two differentvalues (a and b) by the graph of f (x).”

Present an analogue of the Table of Rules for Derivatives, presentedearlier, for finding antiderivatives.

60 / 78

Some implications of f ′(x) being a function

What we’ve seen already

A function, f (x) is simply a “rule” that tells us what (Real) numbersto report for any given Real input (x).

For “well-behaved” functions (not matter how intricate these seem)we find a lot of information about the function’s behaviour from itsfirst derivative. For example,

The function describing how steep the line touching < x , f (x) > is.As a result, whether f (x) is increasing at < x , f (x) > or decreasing oris “standing still”, ie the line has gradient 0.Similarly, we can determine if “standing still” points are local minimaor local maxima.This last item provides a very powerful tool in solving

optimization problems.

61 / 78

Since f ′(x) is a function, how does f (x) relate to f ′(x)?

What do we mean by this question?

We’ve seen that if f (x) = x2 then f ′(x) = 2x .

Suppose F (x) is a function for which F ′(x) = x2. What can we sayabout F (x) in this case?

The Antiderivative of a function

For a function, f (x) its antiderivative is any function, F (x) with theproperty F ′(x) = f (x).

Caveats

The antiderivative may not “exist”.

The antiderivative is not unique. (Qn: Why not?)

If we wish to specify a unique F (x) then we must know its output forone input value.

62 / 78

The role of the antiderivative in measuring areas

Considering, for f (x), the nature of functions F (x) for whichF ′(x) = f (x) may, at first sight, appear to be little more thanformalistic introspection of no relevance to “real” problems.

In fact antiderivatives have extremely important applications as ameans of solving “difficult” measurement problems.

Problems whose nature, while arguably solvable by other means, aremuch much more easily described using antiderivatives.

What is the area of the shaded regions?

63 / 78

The area between x = 6 and x = 14 for f (x) = x2

64 / 78

Light blue: “too big”; Orange “too small”; BUT

As the “rectangle” base gets smaller

Upper Lower8f (14) 8f (6)4(f (14) + f (10)) 4(f (6) + f (10))2(f (14) + f (12) + f (10) + f (8)) 2(f (6) + f (8) + f (10) + f (12))

Base width Upper Estimate Lower Estimate Difference

8 1568 288 1280

4 1184 544 640

2 1008 688 320

So as the rectangle base gets smallerThe gap between “upper” and “lower” gets smaller too.

65 / 78

Limits make a return

Suppose we make h smaller and smaller?

We expect the “upper value” (U(h)) to get “closer and closer” to the“lower value”.This upper value is just a sum of rectangle areas: all have base length hand for each k between 1 and N = (b − a)/h there is a rectangle withheight f (a + kh). In other words,

U(h) =N∑

k=1

hf (a + kh)

What happens when h gets close to (without reaching) 0?

Using the earlier jargon from “first derivative”, (a = 6, b = 14, f (x) = x2)

U(h) = limh→0

8/h∑k=1

h ∗ (6 + kh)2 =

8/h∑k=1

(36h + 2kh2 + h3)

66 / 78

The general case: arbitrary f (x), a and b

U(h) = limh→0

N∑k=1

hf (a + kh)

And now we’re stuck.

Or are we? How antiderivatives help

Suppose we know that F (x) and f (x) (the function we’re working with)are linked by the property: F ′(x) = f (x), ie that F (x) is theantiderivative of f .Recalling the “formal” definition of “first derivative” from earlier, thismeans that:

f (x) = limh→0

F (x + h) − F (x)

h

So what?

Let’s use the same h and see what happens (remember N = (b − a)/h).

67 / 78

A bit of algebraic passage-work: this happens (N = b−ah )

U(h) = limh→0

N∑k=1

hf (a + kh)

= limh→0

N∑k=1

h

(F (a + kh + h) − F (a + kh)

h

)

= limh→0

N∑k=1

F (a + kh + h) − F (a + kh)

= limh→0

N∑k=1

F (a + (k + 1)h) − F (a + kh)

(∗∗) = limh→0

F (a + (N + 1)h) − F (a + h)

= limh→0

F (a + (b − a)h/h + h) − F (a + h)

= limh→0

F (a + b − a + h) − F (a + h) = F (b)− F (a)

68 / 78

The importance of antiderivatives

The hand-waving flummery on the previous slide is NOT a proof.It does, however, provide a rationale for the following:

In order to find the area covered by the function f (x) betweenthe values x = a and x = b it is enough to know theantiderivative of f (x).If F (x) is the antiderivative of f (x), ie F ′(x) = f (x) then thearea sought is

F (b) − F (a)

69 / 78

A bit more notation: definite and indefinite integrals

It is a little wearisome constantly to repeat the phrase

“antiderivative of f (x)”.

The standard notation used instead is the so-called “integral sign” (∫

) andthe expression “F (x) is the antiderivative of f (x)” written as

F (x) =

∫f (x) dx

This is called an indefinite integral since we do not have enoughinformation about F (x) uniquely to define its behaviour.For the “area” applications when a and b are supplied, we use

F (b)− F (a) =

∫ b

af (x)dx

This is a definite integral: we have two points (a and b) to use.

70 / 78

The two previous examples: x2 and 2x3 − 2x2 − x − 1

It is not too difficult to verify that,∫x2dx =

x3

3∫(2x3 − 2x2 − x − 1)dx =

2x4

4− 2x3

3− x2

2− x

[Exercise: Show that the derivatives of the expressions in the thirdcolumn match the functions in the first.]As a result the “area covered by x2 between x = 6 and x = 14” is∫ 14

6x2dx =

143

3− 63

3∼ 842.667

71 / 78

Similarly, for the function 2x3 − 2x2 − x − 1

∫ 0.8

−0.4(2x3 − 2x2 − x − 1)dx

is(0.2048− 1.024

3− 0.32− 0.8

)−(

0.0128 +0.128

3− 0.8 + 0.4

)∼ −0.912

[Qn: Why does this “area” seem to be less than 0?][Hint: Look at the values of this function in the diagram given.]

72 / 78

“Negative” areas

If we want to interpret∫ ba f (x)dx as a physical area in the usual sense (ie

always positive) we can do the following:

If f (x) ≤ 0 for every a ≤ x ≤ b (look at graph to see this) take the

area as −∫ ba f (x)dx .

If f (x) ≤ 0 for every a ≤ x ≤ c and f (x) ≥ 0 for every c ≤ x ≤ bcompute the area using∫ b

cf (x)dx −

∫ c

af (x)dx

In general we can split the range into:“positive” sections (those sections [a, b] where f (x) ≥ 0) and“negative” sections (those sections [c , d ] f (x) ≤ 0) then apply

∑Positive sections [a,b]

∫ b

af (x)dx −

∑Negative sections [c,d ]

∫ d

cf (x)dx

73 / 78

Some basic integration rules

Function form h(x) Rule name

∫h(x)dx

cf (x) ScalarProduct c

∫f (x)dx

f (x) + g(x) Sum

∫f (x)dx +

∫g(x)dx

xn Polynomialxn+1

n + 1(n 6= −1)

x−1 Reciprocal log xsin x (or cos x) Trigonometric − cos x (or sin x)

exp x Exponential exp xlog x (Natural) Logarithm x log x − x

There is no direct equivalent of the product, quotient or chain rules (butthere are methods to deal with these).

74 / 78

Some not so basic integration rules

Exploiting the “Product Rule”: Integration by parts

The Product Rule to find the derivative of f (x)g(x) gives this as

f ′(x)g(x) + f (x)g ′(x)

It follows, therefore that∫(f ′(x)g(x) + f (x)g ′(x))dx = f (x)g(x)

and so, using the Integration Sum Rule from the previous slide:∫f ′(x)g(x)dx = f (x)g(x) −

∫f (x)g ′(x)dx

This can be very useful if the function, h(x) can be “easily” described as aproduct of the form f ′(x)g(x).

75 / 78

An example

h(x) = x exp(x)

Letting f (x) = exp(x) and g(x) = x we get f ′(x) = exp(x) and g ′(x) = 1:hence h(x) = f ′(x)g(x)∫

h(x)dx =

∫f ′(x)g(x) = f (x)g(x) −

∫f (x)g ′(x)dx

So that,∫h(x)dx = x exp(x) −

∫exp(x)dx = x exp(x)− exp(x)

The main problem with “Integration by Parts” is it may require somecreativity to find the most suitable “rewrite” of h(x) as f ′(x)g(x).

76 / 78

Another not so simple Rule: The Substitution Rule

This is an “analogue” of the “Composition (or Chain) Rule”.We know that the derivative of f (g(x)), is f ′(g(x))g ′(x). So ifh(x) = f ′(g(x))g ′(x) for some choice of f (x) and g(x), then∫

h(x)dx = f (g(x)).

An example: h(x) = x cos(x2)

Choosing f (x) = sin(x) and g(x) = x2,

f ′(g(x))g ′(x) = cos(x2)(2x) = 2h(x)

And so∫h(x)dx =

1

2

∫f ′(g(x))g ′(x)dx = f (g(x)) =

sin(x2)

2

As with Integration by Parts, some ingenuity is often needed to applySubstitution effectively.

77 / 78

Summary of Differential & Integral Calculus

Calculus provides powerful computational methods for reasoningabout functions.

The information given by the derivative of f (x) can be applied todetermine when f is increasing, decreasing or “turning”.

We can use such information to model and solve non-trivialoptimization problems.

Integral calculus and the notion of antiderivative offers a usefultechnique for non-trivial “area calculations”.

78 / 78