Download - Calculus IB: Lecture 21
Calculus IB: Lecture 21
Luo Luo
Department of Mathematics, HKUST
http://luoluo.people.ust.hk/
Luo Luo (HKUST) MATH 1013 1 / 30
Outline
1 The Proof of Fundamental Theorem of Calculus
2 Integrability and Properties of Definite Integral
3 Taylor Series
Luo Luo (HKUST) MATH 1013 2 / 30
Outline
1 The Proof of Fundamental Theorem of Calculus
2 Integrability and Properties of Definite Integral
3 Taylor Series
Luo Luo (HKUST) MATH 1013 3 / 30
Fundamental Theorem of Calculus
Theorem (Fundamental Theorem of Calculus)
Let f be a continuous function on the closed interval [a, b]. If F (x) is anantiderivative of f , i.e., F ′(x) = f (x), then
∫
b
a
f (x)dx = F (b)− F (a),
which is often denoted as F (x)∣
∣
b
aor
[
F (x)]b
a.
In other words, whenever you can find∫
f (x)dx = F (x) + C ,
it is just one step further to find the corresponding definite integral:∫
b
a
f (x)dx = F (b)− F (a).
Luo Luo (HKUST) MATH 1013 3 / 30
The Proof of Fundamental Theorem of Calculus
Partition the interval [a, b] with n subintervals of equal lengthb − a
n.
By the mean value theorem Theorem, there exists x∗ in [xi−1, xi ] such that
F (xi )− F (xi−1) = F ′(x∗i )(xi − xi−1) = f (x∗i ) ·b − a
n
Then, we take the Riemann sum of the function f on [a, b]:
n∑
i=1
f (x∗i ) ·b − a
n=
n∑
i=1
(F (xi )− F (xi−1)) = F (xn)−F (x0) = F (b)−F (a).
Thus by the definition of definite integral, we have
∫
b
a
f (x)dx = limn→∞
n∑
i=1
f (x∗i ) ·b − a
n= F (b)− F (a).
Luo Luo (HKUST) MATH 1013 4 / 30
Example of Fundamental Theorem of Calculus
Example
Find the definite integral
∫ 2
−1
1
x2dx
Since
(
−1
x
)
′
=1
x2, we use fundamental theorem of calculus to obatin
∫ 2
−1
1
x2dx =
(
−1
x
)
∣
∣
∣
∣
∣
2
−1
= −1
2− 1 = −3
2
Luo Luo (HKUST) MATH 1013 5 / 30
Example of Fundamental Theorem of Calculus
-4 -3 -2 -1 0 1 2 3 4
0
5
10
15
20
The definite integral
∫ 2
−1
1
x2dx can NOT be negative!!!
Luo Luo (HKUST) MATH 1013 6 / 30
Outline
1 The Proof of Fundamental Theorem of Calculus
2 Integrability and Properties of Definite Integral
3 Taylor Series
Luo Luo (HKUST) MATH 1013 7 / 30
Fundamental Theorem of Calculus
Theorem (Fundamental Theorem of Calculus)
Let f be a continuous function on the closed interval [a, b]. If F (x) is anantiderivative of f , i.e., F ′(x) = f (x), then
∫
b
a
f (x)dx = F (b)− F (a),
which is often denoted as F (x)∣
∣
b
aor
[
F (x)]b
a.
The function f (x) =1
x2is NOT continuous at 0 and
∫ 2
−1
1
x2dx 6= −3
2.
In fact, if one uses Riemann sum to approximate this integral, the limit ofthe Riemann sum is ∞.
Luo Luo (HKUST) MATH 1013 7 / 30
Riemann Sums and Integrability
The definite integral of a continuous function f (x) on an interval [a, b] can bedefined by using subintervals of equal length
∆x =b − a
n;
i.e., with subdivision points a = x0 < x1 < x2 < · · · < xi < · · · xn = b, wherexi = x0 + i∆x , and ci in [xi−1, xi ].
If the limit of Riemann sum
limn→∞
n∑
i=1
f (ci ) ·∆
exists on real numbers, we say the function f is Riemann integrable if the limit ofthe Riemann sum exists and has a unique limit L. The limit is called the definiteintegral of f from a to b, denoted by
∫
b
a
f (x)dx = limn→∞
n∑
i=1
f (ci ) ·∆ = L
Luo Luo (HKUST) MATH 1013 8 / 30
Integrable Functions
Theorem (sufficient condition)
If f is continuous on on [a, b], then f must be (Riemann) integrable.
Theorem (sufficient condition+)
If f is continuous over [a, b] or bounded on [a, b] with a finite number ofdiscontinuous points, then f is integrable on [a, b].
Theorem (necessary condition)
If f is (Riemann) integrable on [a, b], then f must be bounded on [a, b].
Luo Luo (HKUST) MATH 1013 9 / 30
Integrable Functions
Example
The function
f (x) =
{
x2 0 ≤ x < 1
2 1 ≤ x ≤ 2
is integrable om [0, 2].
0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
Luo Luo (HKUST) MATH 1013 10 / 30
Integrable Functions
The definite integral of
f (x) =
{
x2 0 ≤ x < 1
2 1 ≤ x ≤ 2
on [0, 2] is
∫ 2
0f (x)dx = lim
t→1−
∫
t
0f (x)dx +
∫ 2
1f (x)dx
Luo Luo (HKUST) MATH 1013 11 / 30
Non-Integrable Functions
Theorem (necessary condition)
If f is (Riemann) integrable on [a, b], then f must be bounded on [a, b].
Example
The function f (x) =1
x2is unbounded on [−1, 2].
Even if M > 0 is sufficient large, we have xM =1√
M + 1in [−1, 2]
such that f (xM) = M + 1 > M.
Luo Luo (HKUST) MATH 1013 12 / 30
Non-Integrable Functions
Theorem (sufficient condition+)
If f is continuous over [a, b] or bounded on [a, b] with a finite number ofdiscontinuous points, then f is integrable on [a, b].
Example (Dirichlet Function)
Dirichlet function is defined as follows
D(x) =
{
1 if x is a rational number,
0 if x is not a rational number.
Dirichlet function is nowhere continuous (in other words, there are infinitenumber of discontinuous points) and not (Riemann) integrable on any[a, b] when a < b.
Luo Luo (HKUST) MATH 1013 13 / 30
Non-Integrable Functions
Consider the behavior of Dirichlet function
D(x) =
{
1 if x is a rational number,
0 if x is a irrational number.
on interval [a, b], where a < b.
Let a = 0 and b = 1. given rational number 0.123, we can constructinfinite irrational numbers
0.123x1x2x3 . . . xn . . .
Since each xi can be select from 0, 1, . . . , 9, we can think
#rational numbers
#irrational numbers≈ lim
n→∞
(
1
10
)n
= 0
Intuitively, rationals number in [0, 1] is much less than irrational number.
Luo Luo (HKUST) MATH 1013 14 / 30
Non-Integrable Functions
In other words, D(x) = 0 almost everywhere. If we think the area of asegment is 0, then it is reasonable that the “area” of “graph” under D(x)on interval [a, b] is also 0.
In fact, we can define other types of integration (not Riemann integration)to characterize the area under the graph of a function.
Luo Luo (HKUST) MATH 1013 15 / 30
Non-Integrable Functions
For example, in the view of Lebesgue integration, Dirichlet function isintegrable on [a, b] and
∫
b
a
D(x)dx = 0.
However, the expression
∫
b
a
D(x)dx
is undefined by Riemann integration.
Luo Luo (HKUST) MATH 1013 16 / 30
Integrable Functions
In homework and exam of MATH 1013, “integrable” always refers
to Riemann integrable.
Luo Luo (HKUST) MATH 1013 17 / 30
Some Properties of Integrable Functions
Let f and g are integrable on closed interval [a, b], then
1 for any constants A, B , we have
∫
b
a
[Af (x) + Bg(x)]dx = A
∫
b
a
f (x)dx + B
∫
b
a
f (x)dx
2 for any constants a ≤ b ≤ c , we have
∫
c
a
f (x)dx =
∫
b
a
f (x)dx +
∫
c
b
f (x)dx
3 if f (x) ≥ g(x) on [a, b], then
∫
b
a
f (x)dx ≥∫
b
a
g(x)dx
4 if a > b, we define
∫
a
b
f (x)dx = −∫
b
a
f (x)dx in conventional.
Luo Luo (HKUST) MATH 1013 18 / 30
Outline
1 The Proof of Fundamental Theorem of Calculus
2 Integrability and Properties of Definite Integral
3 Taylor Series
Luo Luo (HKUST) MATH 1013 19 / 30
Integral Sandwiches for sin x and cos x
Consider that cos x ≤ 1 and sin x < x . Then for any x ≥ 0,
cos x ≤ 1 =⇒∫
x
0cos tdt ≤
∫
x
01dt ⇐⇒ sin t
∣
∣
∣
∣
x
0
≤ t
∣
∣
∣
∣
x
0
sin x ≤ x =⇒∫
x
0sin tdt ≤
∫
x
0tdt ⇐⇒ − cos t
∣
∣
∣
∣
x
0
≤ x2
2
1− x2
2≤ cos x =⇒
∫
x
0
(
1− t2
2
)
dt ≤∫
x
0cos tdt = sin x
x − x3
3!≤ sin x =⇒
∫
x
0
(
t − t3
3!
)
dt ≤∫
x
0sin tdt = − cos x + 1
· · · · · · =⇒ · · · · · ·
Exclamation mark “!” means factorial, that is, k! = 1 · 2 · · · · k for anypositive integer k . We define 0! = 1 in conventional.
Luo Luo (HKUST) MATH 1013 19 / 30
Integral Sandwiches for sin x and cos x
Repeating such procedures, we have (show that by induction)
x − x3
3!+
x5
5!− · · · − x4n−1
(4n − 1)!≤ sin x ≤ x − x3
3!+
x5
5!− · · ·+ x4n+1
(4n + 1)!
1− x2
2!+
x4
4!− · · · − x2n−2
(2n − 2)!≤ cos x ≤ 1− x2
2!+
x4
4!− · · ·+ x2n+2
(2n + 2)!
We can approximate sin x and cos x by polynomial functions
sin x ≈ x − x3
3!+
x5
5!− · · ·+ x4n+1
(4n + 1)!
cos x ≈ 1− x2
2!+
x4
4!− · · ·+ x2n+2
(2n + 2)!
Luo Luo (HKUST) MATH 1013 20 / 30
Integral Sandwiches for sin x and cos x
We can use the polynomial
p(x) = x − x3
3!+
x5
5!− x7
7!
to estimate the function value of sin x , then we have
p(x) ≤ sin x ≤ x − x3
3!+
x5
5!− x7
7!+
x9
9!=⇒ sin x − p(x) ≤ x9
9!
If we restrict x on[
0,π
4
]
, the above inequalities implies
0 ≤ sin x − p(x) ≤ x9
9!≤
(π4 )9
9!= 3.13× 10−7
Luo Luo (HKUST) MATH 1013 21 / 30
Integral Sandwiches for sin x and cos x
-2 -3 /2 - - /2 0 /2 3 /2 2
-3
-2
-1
0
1
2
3
Luo Luo (HKUST) MATH 1013 22 / 30
Integral Sandwiches for sin x and cos x
We can use the polynomial
q(x) = 1− x2
2!+
x4
4!− x6
6!
to estimate the function value of cos x , then we have
q(x) ≤ cos x ≤ 1− x2
2!+
x4
4!− x6
6!+
x8
8!=⇒ 0 ≤ cos x − q(x) ≤ x8
8!
If we restrict x on[
0,π
4
]
, the above inequalities implies
0 ≤ cos x − q(x) ≤ x8
8!≤
(π4 )8
8!= 3.59× 10−6
Luo Luo (HKUST) MATH 1013 23 / 30
Integral Sandwiches for sin x and cos x
-2 -3 /2 - - /2 0 /2 3 /2 2
-3
-2
-1
0
1
2
3
Luo Luo (HKUST) MATH 1013 24 / 30
Taylor Series and Linear Approximation
More general, for f (x) that is infinitely differentiable, we can approximateit by Taylor series
f (x) ≈ f (a) +f ′(a)
1!(x − a) +
f ′′(a)
2!(x − a)2 +
f ′′′(a)
3!(x − a)3 + · · ·
In above examples, we take a = 0 and f (x) be sin x/cos x .
We can also think this strategy is an extension of linear approximation
f (x) ≈ f (a) + f ′(a)(x − a)
Luo Luo (HKUST) MATH 1013 25 / 30
Taylor Series and Optimization
Let a = xk , we have
f (x) ≈ f (xk) + f ′(xk)(x − xk) +f ′′(xk)
2(x − xk)
2 +f ′′′(xk)
3!(x − xk)
3 + · · ·
The iteration of gradient descent is
xk+1 =xk −1
Lf ′(xk)
= argminx
[
f (xk) + f ′(xk)(x − xk) +L
2(x − xk)
2
]
Recall that we suppose f ′′(x) ≤ L for positive L in the analysis of convexoptimization, which means the update is optimizing and upper bound offirst three terms in Taylor series.
Luo Luo (HKUST) MATH 1013 26 / 30
Taylor Series and Optimization
It is easy to check
xk+1 =xk −1
Lf ′(xk)
= argminx
[
f (xk) + f ′(xk)(x − xk) +L
2(x − xk)
2.
]
We define
g(x) = f (xk) + f ′(xk)(x − xk) +L
2(x − xk)
2,
then g ′′(x) = L > 0 and g ′(x) = f ′(xk) + L(x − xk).
Hence, g(x) is convex and xk+1 is its unique critical point (minimizer).
Luo Luo (HKUST) MATH 1013 27 / 30
Taylor Series and Optimization
What happens if we directly minimize the first three terms?
f (x) ≈ f (xk) + f ′(xk)(x − xk) +f ′′(xk)
2(x − xk)
2 +f ′′′(xk)
3!(x − xk)
3 + · · ·
We define h(x) = f (xk) + f ′(xk)(x − xk) +f ′′(x)
2(x − xk)
2, then
h′′(x) = f ′′(x) and h′(x) = f ′(xk) + f ′′(xk)(x − xk).
Hence, if f (x) is strictly-convex then h′′(x) = f ′′(xk) > 0 is convex andxk+1 is its unique critical point (minimizer) of h(x) is
xk+1 = xk −f ′(xk)
f ′′(xk),
which leads to Newton’s Method!
Luo Luo (HKUST) MATH 1013 28 / 30
Taylor Series and Optimization
In theoretical, we can also optimize
l(x) = f (xk) + f ′(xk)(x − xk) +f ′′(xk)
2(x − xk)
2 +f ′′′(xk)
3!(x − xk)
3
to establish an optimization algorithm, but solving such sub-problem ismore complicated and difficult to be extended to high-dimensional case.
Luo Luo (HKUST) MATH 1013 29 / 30
Taylor Series and Optimization
Similar to linear approximation, the approximation
f (x) ≈ f (a) +f ′(a)
1!(x − a) +
f ′′(a)
2!(x − a)2 + · · ·+ f (n)(a)
n!(x − a)n
has high accuracy when x is close to a.
Intuitively, if x is far away from a, we require a larger n to increase n! andcontrol the magnitude of
f (n)(a)
n!(x − a)n.
Luo Luo (HKUST) MATH 1013 30 / 30