18 examples of calculus of variations and optimal …kawski/classes/apm582/16... · 18 examples of...

18 EXAMPLES OF CALCULUS OF VARIATIONSAND OPTIMAL CONTROL PROBLEMS

H. J. Sussmann — November 1, 2000

Here is a list of examples of calculus of variations and/or optimal control problems. Some are easy, othershard. Three of them are still unsolved. Some can be solved directly by elementary arguments, others cannotbe solved unless one uses the machinery of the calculus of variations or optimal control. Some statementsmay look a little bit too technical at this point, but will become clearer later in the course. In all the problems,when we talk about “all paths” we mean “all appropriately smooth paths.” The precise meaning will dependon the problem, and usually this question—of choosing the correct space of functions or paths in which towork—is regarded nowadays as part of the problem. On the other hand, the founders of the subject—peoplelike the Bernoulli brothers, Newton, Leibniz, Euler, Lagrange, to name a few—were perfectly happy talkingabout “all paths,” without ever being troubled by the fact that not every function has a derivative and notevery function can be integrated. For the purposes of this particular discussion, you can follow our illustriouspredecessors and just forget about the choice of the space of paths.

1. (The oldest of all calculus of variations problems; at least 3000 years old, for n = 2 or n = 3, of course.)Of all paths ξ : [0, 1] �→ IRn such that ξ(0) = A, ξ(1) = B, find the paths or paths that minimize theintegral

J(ξ) =∫ 1

0

‖ξ(t)‖ dt ,

where ‖ · ‖ is Euclidean norm, that is, ‖(x1, . . . , xn)‖ =√∑n

i=1 x2i , and A, B are two given points in

IRn.

2. (The rope-stretching problem.) Given the length of a string, find the configurations that maximizethe distance between the endpoints. (NOTE: this is easily proved to be equivalent to Problem 1. Inancient Egypt and India the art of rope-stretching was practiced in order to produce segments and othergeometrical figures, showing knowledge of the fact that a segment solves the rope-stretching problem. InEgypt, according to Democritus, geometric constructions were carried out with the help of specializedworkers—whom Democritus describes as experts in “composing lines,” and calls by the Greek word“harpenodaptai,” which means “rope-stretchers”—by means of pegs and cords. Their skills were usedto build altars, temples and pyramids, where it was deemed necessary to produce certain geometricshapes that would obey very precise specifications, such as being made of perfectly straight segmentsor perfect circles. These demands often originated with the Gods, who insisted that an altar or templebe built so as to meet very strict requirements, failing which their wrath would be unleashed uponthe builders and their communities. Rope-stretching is a good analog device for making long segmentsprecisely because the solution of the rope-stretching problem is a segment. This is one of the oldestexamples I know of mathematics applied to engineering.)

3. (A more modern version of Problem 1.) Same question as in Problem 1, but ‖ · ‖ is now a general normon IRn, not necessarily Euclidean or smooth or strictly convex. (A good example to keep in mind is the“�1 norm” given by ‖x‖ = |x1| + . . . + |xn|.)

4. (The isoperimetric problem, also known as “Dido’s” problem, inspired by the story told by Virgil (70-19 BCE) in the Aeneid about the foundation of Carthage, ca. 850 BCE; more than 2100 years old;essentially solved by Zenodorus, ca. 200-140 BCE, who showed that, (i) among all polygons with agiven perimeter P and given number N of sides, the regular polgyon has the largest area, and (ii) acircle had a larger area than any regular polygon of the same perimeter.) Of all simple closed rectifiablecurves ξ : [0, 1] �→ IR2 of a given length L, find the ones that maximize the enclosed area.

1

5. (The light reflection problem, mathematically trivial but historically important; about 1900 years old.)Of all paths in the closed upper half plane {(x, y) ∈ IR2 : y ≥ 0} that go from a given point A to anotherpoint B and go through at least one point in the x axis, find the ones of minimum length. (NOTE:Hero of Alexandria—believed to have been born in 10 CE and to have died in 75 CE—pointed out inhis Catoptrics that when a light ray emitted by an object is reflected by a mirror it follows a path fromthe object to the eye which is the shortest of all possible such paths. This is the first example in historyof an explanation of a physical phenomenon on the basis of a “curve minimization principle.”)

6. (The light refraction problem, easily doable today as a Freshman Calculus exercise, but important forpeople such as Leibniz when they were first able to solve it; about 350 years old. In his 1684 NovaMethodus paper, Leibniz wrote, referring to his solution of the light refraction and other related problems,that “Other very learned men have sought in many devious ways what someone versed in this calculuscan accomplish in these lines as by magic. . . . This is the beginning of much more sublime Geometry,pertaining to even the most difficult and most beautiful problems of applied mathematics, which withoutour differential calculus or something similar noone could attack with any such ease.”) Assuming thatthe speed of light is equal to a constant c+ when y > 0 and to a different constant c− when y < 0, findthe light rays (i. e., minimum time paths) from a given point A in the open upper half plane to anothergiven point B in the open lower half plane.

7. (The catenary; solved incorrectly by Galileo in the 1630s; solved correctly by Johann Bernoulli in 1691.)Find the shape of a chain (or rope) of uniform mass density and given length L, held fixed at itsendpoints, and otherwise hanging freely. Mathematically, this becomes: given two points A, B in thetwo-dimensional plane IR2, and a length L (which should be ≥ dist(A, B)), then, among all the paths[0, 1] � t �→ ξ(t) = (x(t), y(t)) such that ξ(0) = A, ξ(1) = B, and

∫ 1

0

√x(t)2 + y(t)2 dt = L, find the one

that minimizes the integral

J(ξ) =∫ 1

0

y(t)√

x(t)2 + y(t)2dt .

(Note: J(ξ) is equal to L times the ordinate of the center of mass of the hanging chain. So the problemamounts to finding the path whose center of mass is as low as possible.)

8. (The brachistochrone; solved incorrectly by Galileo in 1638; solved correctly by Johann Bernoulli andothers in 1696-7.) Suppose we drop a particle at a point A so that it will fall freely along a curve ξgoing from A to another point B. For given A, B, how should ξ be chosen so that the falling time isminimized?The above is, roughly, Johann Bernoulli’s formulation of the problem. A standard translation intomathematics, found in most textbooks, is as follows. The energy E is constant, and E = K + P , whereK is kinetic energy and P is potential energy. Since K = 1

2(x2 + y2) (assuming the particle mass to be

equal to 1) and P = gy, we have12(x2 + y2) + gy = E .

If we write A = (a, α), B = (b, β), and assume further that our curve is actually given by y = ϕ(x),a ≤ x ≤ b, we have

12

((dx

dt

)2

+(dy

dt

)2)

+ gy = E ,

so (dx

dt

)2

+(dy

dt

)2

= 2E − 2gy ,

(dx)2 + (dy)2 = (2E − 2gy)(dt)2 ,(1 +

(dy

dx

)2)(dx)2 = (2E − 2gy)(dt)2 ,

dt =

√1 +

(dydx

)2

√2E − 2gy

dx ,

2

and then our problem is to minimize the integral

J(ϕ) =∫ b

a

√1 + ϕ′(x)2√

2E − 2gϕ(x)dx ,

among “all functions” ϕ : [a, b] → IR such that ϕ(a) = α, ϕ(b) = β, and gϕ(x) ≤ E for all x.If we factor out the

√2 and just omit it, take g = 1 (which is possible by a suitable choice of units) and

relabel E − ϕ as our new ϕ, then the integral becomes

J(ϕ) =∫ b

a

√1 + ϕ′(x)2√

ϕ(x)dx ,

and the constraint on ϕ is just ϕ(x) ≥ 0 for all x.Later in the course we will discuss whether the above formulation is a satisfactory translation intomathematics of Johann Bernoulli’s original statement. It will be argued that it is a very imperfecttranslation, and that a much better one is possible and yields nicer results.

9. (The “reflected brachistochrone.”) Assuming that the speed of light c(x, y) is equal to√|y|, find the

light rays (i. e., minimum time paths) from a given point A to another given point B ∈ IR2.

10. (One-dimensional soft landing.) For the system

x = y , y = u , |u| ≤ 1 ,

given a starting position x− and velocity y−, find the path of minimum time that ends at x = 0, y = 0.

11. (n-dimensional soft landing.) Same as in Problem 10, but now x, y, u are n-dimensional vectors, andthe constraint on u is ‖u‖ ≤ 1, ‖ · ‖ being Euclidean norm.

12. (Fuller’s problem; a very important problem, famous because of the surprising nature of the solutions.)For the system

x = y , y = u , |u| ≤ 1 ,

given a starting position x− and velocity y−, find the path ending at (0, 0) and defined on an interval[0, T ] for some T > 0 (to be found) that minimizes the integral

∫ T

0x(t)2 dt.

13. Given a ∈ IR, b ∈ IR, L > 0, find, of all functions ξ : [0, L] �→ IR such that ξ(0) = a, ξ(L) = b, and|ξ(t)‖ ≤ 1 for all t, the ones that minimize the integral

J(ξ) =∫ L

0

ξ(t)2 dt .

14. (The “Markov-Dubins problem,” going back to A. A. Markov, 1887. Solved by L. Dubins in 1957 forn = 2, and by H. Sussmann in 1992 for n = 3. D. Mittenhuber later proved that the n > 3 case reducesto the n = 3 case.) Of “all paths” ξ in IRn that are parametrized by arc-length (that is, ‖ξ(t)‖ = 1 forall t), satisfy the curvature bound ‖ξ(t)‖ ≤ 1, and go from a given initial point x− and initial directionv− to a given terminal point x+ and terminal direction v+, find the ones of minimum length.

15. (The “Markov-Dubins-Reeds-Shepp problem,” a variant of Problem 9, solved by J. Reeds and L. Sheppin 1990 for n = 2, and by Sussmann in 1992 for n = 3. Mittenhuber later proved that the n > 3 casereduces to the n = 3 case.) Of “all paths” ξ in IRn that satisfy ξ(t) = ε(t)v(t), ε(t) ∈ {−1, 1}, ε(·)measurable, v(·) Lipschitz, ‖v(t)‖ ≤ 1, that go from a given initial point x− and initial direction v− toa given terminal point x+ and terminal direction v+, find the ones of minimum length. (NOTE: thinkof a “vehicle” that is moving in such a way that at each time t it points to some direction v(t)—sov(t) is a unit vector—and the motion takes place with speed 1, while at the same time the vehicle canturn—that is, v(t) can change—but with a bound ‖v(t)‖ on how fast this can happen. Then the Dubinscase corresponds to a vehicle that is not allowed to back up—that is, ε = 1—whereas the Reed-Sheppcase corresponds to allowing backups, that is ε = ±1.)

3

16. (The “Markov-Dubins problem with angular acceleration control,” a variant of Problem 9. Still open,although some partial—and rather counterintuitive—results are known.) For the system

x = cos θ , y = sin θ , θ = ω , ω = u , ‖u‖ ≤ 1 , (1)

of “all paths” t �→ (x(t), y(t), θ(t), ω(t)) ∈ IR × IR × S1 × IR that go from a given initial condition(x−, y−, θ−, ω−) to a given terminal condition (x+, y+, θ+, ω+), find the ones of minimum length.

17. (General regularity of optimal controls. Still open, although some answers are known, and the answersthat are known are somewhat surprising, as will be explained later in the course.) For two smooth (i. e.,C∞) vector fields f and g in IRn, and two points A, B in IRn, let P (f, g, A, B) be the following minimumtime problem: find a T ≥ 0 and a measurable function [0, T ] � t �→ u(t) ∈ [−1, 1] such that the solution[0, T ] � t �→ ξ(t) ∈ IRn of the O.D.E. initial value problem

ξ(t) = f(ξ(t)) + u(t)g(ξ(t)) , ξ(0) = A ,

exists on [0, T ], is such that ξ(T ) = B, and T is as small as possible.Which of the following statements are true?

a. Whenever f , g are C∞ vector fields and P (f, g, A, B) has a unique solution (T, u(·)), then u(·) hasat most finitely many points of discontinuity.

b. Whenever f , g are real-analytic vector fields and P (f, g, A, B) has a unique solution (T, u(·)), thenu(·) has at most finitely many points of discontinuity.

c. Whenever f , g are real-analytic vector fields and P (f, g, A, B) has a unique solution (T, u(·)), thenu(·) has at most finitely many points of non-analyticity.

d. Whenever f , g are C∞ vector fields and P (f, g, A, B) has a unique solution (T, u(·)), then u(·) hasat most countably many points of discontinuity.

e. Whenever f , g are real-analytic vector fields and P (f, g, A, B) has a unique solution (T, u(·)), thenu(·) has at most countably many points of discontinuity.

f. Whenever f , g are real-analytic vector fields and P (f, g, A, B) has a unique solution (T, u(·)), thenu(·) has at most countably many points of non-analyticity.

g. Whenever f , g are C∞ vector fields and P (f, g, A, B) has a unique solution (T, u(·)), then u(·) hasat most a set of measure zero of points of discontinuity.

h. Whenever f , g are real-analytic vector fields and P (f, g, A, B) has a unique solution (T, u(·)), thenu(·) has at most a set of measure zero of points of discontinuity.

i. Whenever f , g are real-analytic vector fields and P (f, g, A, B) has a unique solution (T, u(·)), thenu(·) has at most a set of measure zero of points of non-analyticity.

j. Whenever f , g are C∞ vector fields and P (f, g, A, B) has a unique solution (T, u(·)), then u(·) iscontinuous on some dense open set of its interval of definition.

k. Whenever f , g are real-analytic vector fields and P (f, g, A, B) has a unique solution (T, u(·)), thenu(·) is continuous on some dense open set of its interval of definition.

�. Whenever f , g are real-analytic vector fields and P (f, g, A, B) has a unique solution (T, u(·)), thenu(·) is real-analytic on some dense open set of its interval of definition.

18. (General regularity of subriemannian minimizers. Still open, although several erroneous proofs of anaffirmative answer have appeared in the literature, e. g. one by R. Strichartz in J. Diff. Geometry24, 1983, pp. 221-263, later retracted in the same journal, 30, 1989, pp. 595-596.) A subriemannianmetric is like a Riemannian metric except that the metric is only defined (and smooth) on a smoothsubbundle E of the tangent bundle, and only curves ξ whose direction ξ(t) at each time t belongs tothe corresponding space E(ξ(t)) are permitted. Are all length-minimizers smooth (if parametrized byarc-length)?

4

THE EULER-LAGRANGE EQUATIONAND ITS INVARIANCE PROPERTIES

Hector J. Sussmann

November 8, 2000

1. THE EULER-LAGRANGE EQUATION

1.1. Standard calculus of variations problems. The calculus of variationsdeals mainly with optimization problems involving a function

(x, x) �→ L(x, x, t)

of “position, velocity, and time.” Nowadays, this function is usually called theLagrangian. In contemporary mathematics, the “position”—or “configuration,” or“state”1—x is usually taken to be a vector2, x = (x1, x2, . . . , xn) in n-dimensionalEuclidean space R

n, and the velocity x = (x1, x2, . . . , xn) is then also a vector inR

n.

Remark 1.1.1. In the earliest work, n was usually one or two or three. Con-sidering higher dimensions is natural in today’s mathematics, but the idea thatone might as well write (x1, x2, . . . , xn) rather than x, or (x, y), or (x, y, z), and“do everything in n-dimensions, with n completely arbitrary,” was by no meansnatural to the founders of the subject and took a long time to evolve. Lagrange,for example, devotes a lot of space in his Mecanique Analytique to persuading thereader that a system of N point particles in 3-dimensional space can be thought ofas a single point evolving in a space of 3N coordinates. ♦

Remark 1.1.2. One can also consider problems in which x is a vector in aninfinite-dimensional space, and a lot of the current activity deals with such ques-tions. However, here we will only discuss finite-dimensional problems. ♦

The objective of a typical calculus of variations problem is to find a curve t �→ ξ(t)in x-space that minimizes the integral

I(ξ) =∫ b

a

L(ξ(t), ξ(t), t)dt

of the Lagrangian, among all curves3

1We will point out below that one should really distinguish between the config-uration q of a physical system and a coordinate representation in terms of an n-tuplex = (x1, x2, . . . , xn) of numbers, but at this point we will ignore the distinction

2We follow the modern convention of using superscripts for “contravariant indices,”i.e., components of vectors, and subscripts for “covariant indices,” i.e., components ofcovectors.

3In twentieth century mathematics, “all curves” means, of course, all curves in somesuitably chosen function space, such as that of all absolutely continuous curves, or that

2 H. J. SUSSMANN

For example, one could specify a time interval [a, b], an initial point x, and aterminal point x, and seek to minimize I in the class of all curves ξ : [a, b] → R

n

sucht that ξ(a) = x and ξ(b) = x. This is a “standard calculus of variationsproblem”:

THE PROBLEM P(L, a, b, x, x)

Given L, a, b, x, x ,

minimize I =∫ b

aL(ξ(t), ξ(t), t)dt ,

subject to ξ(a) = x and ξ(b) = x .

(1.1)

Alternatively, one may think that the use of the symbol x as a variable is a littlebit confusing, and prefer to avoid using it. In order to make it transparently clearthat L is “a function of three independent variables”—namely, the configurationpoint x, the velocity vector u, and the time t—one would use a letter such as urather than x as the name for the velocity variable. This leads to regarding theLagrangian as a function

Ω × Rn × [a, b] � (x, u, t) �→ L(x, u, t)(1.2)

and rewriting (1.1) in the equivalent form

THE PROBLEM P(L, a, b, x, x)

Given L, a, b, x, x ,

minimize I =∫ b

aL(ξ(t), η(t), t)dt ,

subject to ξ(a) = x , ξ(b) = x ,

and ξ(t) = η(t) for a ≤ t ≤ b .

(1.3)

The vector variable x is the state, or configuration, of the problem (1.1) or (1.3).The set Ω of all possible values of x is the configuration space, or state space, of theproblem. For the time being, we will require Ω to be an open subset of a Euclideanspace R

n. Later on, we will allow Ω to be a more general differentiable manifold,for example a spherical surface.

So we will stipulate that

• the configuration space Ω is an open subset of Rn;

• the variable u—the “velocity”—takes values in Rn;

• L is a real-valued function on Ω × Rn × [a, b].

of all Lipschitz curves. The choice of function space can sometimes be crucial, as will beseen when we discuss the “Lavrentiev phenomenon” later on.

EULER-LAGRANGE AND COORDINATE INVARIANCE 3

Remark 1.1.3. A minimization problem of the form (1.1)—or (1.3)—is a point-to-point calculus of variations problem, because the endpoint constraint on thetrajectory ξ(·) specifies that ξ(·) has to start at a given point x and end at anothergiven point x. One can of course consider, both for the calculus of variations and foroptimal control, more general set-to-set problems, in which the endpoint constraintsare

ξ(a) ∈ S1 and ξ(b) ∈ S2 ,(1.4)

where S1, S2 are two given subsets of Ω. Even more generally, one can consider“mixed” endpoint constraints of the form

(ξ(a), ξ(b)) ∈ S ,(1.5)

where S is a given subset of Ω×Ω. (For example, the constraint that ξ be a closedcurve—i. e., that ξ(b) = ξ(a)—would be expressed by taking S = {(x, x) : x ∈ Ω}.)Such more general problems have been studied since the very beginning of thecalculus of variations, and give rise to interesting questions about transversalityconditions. At this point, we will only deal with calculus of variations (and optimalcontrol) point-to-point problems. This means that, for the time being, we are leavingout the whole area of transversality conditions. ♦

1.2. The Euler-Lagrange equation. We now look for necessary conditionsfor a given curve t �→ ξ∗(t) to be a solution of the problem (1.1).

So we will be dealing with a problem P(L, a, b, x, x) as before, together with agiven curve ξ∗. The curve ξ∗ will be referred to, in this kind of discussion, as “thereference curve,” meaning “the curve that we are examining in order to decide if itis a solution of the minimization problem.”

The general condition known today as the “Euler-Lagrange equation” was de-rived by Leonhard Euler4 (1707-1793) and Joseph-Louis Lagrange5 (1736-1813).

4Euler entered the University of Basel in 1720 at the age of 13. There he studied themethods developed by the Bernoullis for the study of isoperimetric problems, but soonbecame convinced that there was a need for a general theory, that would go beyond thesolutions of specific problems. He began to develop his approach in 1731, and by 1740he had achieved his goal and come up with a general theory. He published his results in1744, in a book entitled Methodus inveniendi maximi minime propriate gaudentes sive solutioproblematis isoperimetrici latissimo sensu accepti (A method for discovering curved lineshaving a maximum or minimum property or the solution of the isoperimetric problemtaken in its widest sense). There he gave a general procedure for writing down whatbecame known as Euler’s equation. The words “calculus of variations” do not yet appear inthe Methodus inveniendi. They were coined by Euler later, in 1760, as a name for Lagrange’smethod that Euler had decided to adopt in lieu of his own discretization approach, andappeared in print in 1766.

5Lagrange was a 19-year-old living in Turin when, on 12 August 1755, he wrote toEuler a brief letter to which was attached an appendix containing mathematical details ofa new idea. He introduced a new method that could turn out the necessary condition ofEuler, and more, almost automatically. After seeing Lagrange’s work, Euler dropped hisown method, espoused that of Lagrange, and renamed the subject the calculus of variations.In the summary to his first paper using variations, Euler says “Even though the authorof this had meditated a long time and had revealed to friends his desire yet the glory offirst discovery was reserved to the very penetrating geometer of Turin LA GRANGE, whohaving used analysis alone, has clearly attained the same solution which the author haddeduced by geometrical considerations.”

4 H. J. SUSSMANN

The Euler-Lagrange equation says6 that the identity

ddt

(∂L∂x

)= ∂L

∂x(1.6)

must be satisfied along the curve ξ∗.

Equation (1.6) makes perfect sense and is a necessary condition for optimalityfor a vector-valued variable x as well as for a scalar one. It can be written as asystem:

d

dt

(∂L

∂xi

)=

∂L

∂xi, i = 1, . . . , n .(1.7)

Alternatively, we can regard Equation (1.6) as a vector identity, in which

x = (x1, . . . , xn)

denotes an n-dimensional vector, and the expressions∂L

∂x,

∂L

∂x

stand for the n-tuples( ∂L

∂x1, . . . ,

∂L

∂xn

),

( ∂L

∂x1, . . . ,

∂L

∂xn

).

A modern mathematician might be troubled by the use of x both as an “independentvariable” and as a function of time evaluated along a trajectory, and might preferto write (1.6) as

ddt

[∂L∂u

(ξ∗(t), ξ∗(t), t

)]= ∂L

∂x

(ξ∗(t), ξ∗(t), t

), i=1, . . . , n ,(1.8)

writing the Lagrangian as in (1.2). This makes it clear that, to compute the left-hand side of (1.6), we must

1. first evaluate ∂L∂x “treating x as an independent variable,”

2. then plug in the curve ξ∗(t) and its velocity ξ∗(t) in the slots for x and x,3. finally, differentiate with respect to t.

1.3. The theorem. One possible precise statement of the theorem about theEuler-Lagrange equation is as follows:

Theorem 1.3.1. Assume n is a positive integer, Ω is an open subset of Rn, a, b

are real numbers such that a < b, L is a real-valued function on Ω×Rn×[a, b], and x,

x are given points in Ω. Assume that L is a function of class C1. Let ξ∗ : [a, b] �→ Ωbe a curve of class C1 which is a solution of the minimization problem P(L, a, b, x, x)in the space of all curves of class C1. Then the Euler-Lagrange equation (1.8) holdsfor all t ∈ [a, b].

6We use the notations that are common today, not Lagrange’s. The symbol ∂ forpartial derivative was first used by Legendre in 1786.


This is far from being the best possible result. One can prove Theorem 1.3.1 un-der much weaker hypotheses, but in that case one has to be more careful about thestatement. We will discuss such improvements later, and we will postpone the proofuntil we have become more familiar with the result and with its generalizations.

Notice that, under the hypotheses of Theorem 1.3.1,

1. the curve t �→ (ξ∗(t), ξ∗(t), t) is continuous,2. the functions

(x, u, t) �→ ∂L

∂x(x, u, t)

and

(x, u, t) �→ ∂L

∂u(x, u, t)

are continuous,

Therefore the conclusion of Theorem 1.3.1 says in fact that the function

t �→ ∂L

∂u(ξ∗(t), ξ∗(t), t) ,(1.9)

which is only known a priori to be continuous, is in fact differentiable everywhere,and its derivative is the function

t �→ ∂L

∂x(ξ∗(t), ξ∗(t), t) .(1.10)

Hence the continuous function (1.9) is in fact continuously differentiable.It should be immediately clear from the statement of Theorem 1.3.1 the tech-

nical conditions can be weakened. For example:

(A) The statement of Theorem 1.3.1 contains absolutely no reference to anyderivatives of L with respect to time. So it ought to be possible to re-lax the hypothesis that L is of class C1 and assume instead that L is justdifferentiable with respect to x and u, maybe with the x- and u-derivativescontinuous as functions of (x, u, t).

(B) Suppose we only assume that the curve ξ∗ is Lipschitz, and that it is asolution of P(L, a, b, x, x) in the space of all Lipschitz curves. Then we wouldof course no longer know a priori that the functions (1.9) and (1.10) arecontinuous, but we would know that they are measurable and bounded. It isnot unreasonable to expect that in this case the conclusion of Theorem 1.3.1might still follow, in the sense that the function (1.9) is the indefiniteintegral of the function (1.10) (that is, equivalently, the function (1.9)—which is only known to be bounded and measurable—is in fact absolutelycontinuous, and its derivative is equal almost everywhere to the function(1.10).

(C) Suppose we only assume that the curve ξ∗ is absolutely continuous, and thatit is a solution of P(L, a, b, x, x) in the space of all absolutely continuouscurves. Then we would of course no longer know a priori that the functions(1.9) and (1.10) are continuous or bounded, but we would know that theyare measurable. Once again, it may not appear unreasonable to expect thatin this case the conclusion of Theorem 1.3.1 might still follow, in the sensethat the function (1.9) is the indefinite integral of the function(1.10) (that is, equivalently, the function (1.9)—which is only known to be

6 H. J. SUSSMANN

and measurable—is in fact absolutely continuous, and its derivative is equalalmost everywhere to the function (1.10).

It will turn out that the conjectures suggested by (A) and (B) are true while, onthe other hand, the one suggested by (C) is false, due to the so-called “Lavrentievphenomenon.”

Moreover, we will see later that Theorem 1.3.1 can be extended even further,in rather surprising ways. For example, no differentiability of L with respect to thevelocity u is needed. This should of course look strange to you at this point, becauseif we do not assume that L is differentiable with respect to u then it is not clear atall what the meaning of (1.8) might possibly be. It will turn out, however, that ifone interprets things properly, one can get rid of the requirement on differentiabilitywith respect to the velocity u.

On the other hand, it will turn out, also rather surprisingly, that the require-ment on differentiability with respect to x can be weakened a bit but not much.This may seem strange because, if you look at (1.8), you may get the impressionthat more regularity is required for the dependence of L on u than for the depen-dence of L on x since, after all, in (1.8) we are only differentiating L with respectto x, whereas L is being differentiated with respect to u and then the u-derivativeis differentiated again. Yet we will see later that some kind of differentiability withrespect to x is crucial no matter what we do, whereas the differentiability withrespect to u can be completely avoided if one does things right.

1.4. The momentum. The Euler-Lagrange equation contains the “vector”

π(t)def= ∂L

∂x(ξ∗(t), ξ∗(t), t) ,(1.11)

known as the momentum along the trajectory ξ∗(·). So the components πi of themomentum are given by

πi(t)def= ∂L

∂ui (ξ∗(t), ξ∗(t), t) , for i = 1, . . . , n .(1.12)

Naturally, π depends on t, so it is not a “vector” but a field of “vectors” along thetrajectory ξ∗, that is, a map t �→ π(t) that assigns to each time t in the intervalwhere ξ∗ is defined a “vector” π(t). (In modern differential-geometric terminology,π really is a field of covectors along ξ∗(·). As we will see later, π shows up incontrol theory playing an even more important role, and is often referred to bycontrol theorists as the “adjoint vector.” This is a somewhat unfortunate choiceof terminology, because the π(t) are really covectors rather than vectors, but it isstandard and we will follow it.)

2. COORDINATE INVARIANCE

2.1. Coordinate charts. Now the time has finally come to clarify the distinc-tion between “configurations” and “vectors x in R

n.” It is actually quite simple: ifwe are trying to describe the behavior of a physical system (for example, the motionof a point particle), then this motion will take place in the “configuration space”Q of the system (for example, in three-dimensional physical space for the case of apoint particle). Now, a point of Q is not a tuple of numbers. We may use tuplesof numbers to represent points of Q, but these tuples are not the points themselvesbut, rather, representations of the points. Choosing a way to represent points qof Q in a configuration space Q as n-tuples x = (x1, . . . , xn) is called choosing ann-dimensional coordinate chart on Q. Moreover, this coordinate chart may only beappropriate to represent some, but not necessarily all the points of Q. This leadsus to the following

Definition 2.1.1. Let Q be a set, and let n be a nonnegative integer. Ann-dimensional coordinate chart on Q is a bijective map

Dx � q �→ x(q) = (x1(q), . . . , xn(q)) ∈ Rx

from some subset Dx (called the domain of the chart x) onto an open subset Rx ofR

n (called the range of x). ♦

It is important to realize that for a physical system there will always be manydifferent charts. Some of them will be more “natural” than others, as we shallsoon see, but even among the “natural” ones there will be no such thing as acanonical choice of the chart. For example, even if we want to study the motionof a single particle in three-dimensional physical space, the possible configurationsof the particle are the positions, that is, points in space, but to represent thesepositions as triples of numbers we have to choose an origin and three axes (if we arelooking for “Cartesian” coordinates), and this can obviously be done in many ways.Moreover, there are also many other, non-Cartesian, ways to represent points, e. g.by means of cylindrical or spherical coordinates.

2.2. Represention of velocity vectors (tangent vectors). Suppose a phys-ical system has configuration space Q, and suppose the system is evolving duringa time interval I ⊆ R in such a way that it describes a curve

I � t �→ ξ(t) ∈ Q .

Suppose x is an n-dimensional chart of our system. Assume. moreover, that thecurve ξ is entirely contained in the domain Dx of a chart

x : Dx �→ Rx ⊆ Rn .

Then our curve ξ admits a coordinate representation ξx relative to the chart x,given by

ξx(t) = x(ξ(t)) for t ∈ I .

8 H. J. SUSSMANN

This coordinate representation ξx is now a curve in Rn, so we may talk, for example,

about the curve being “differentiable.”What should we mean by the statement that ξ itself (and not just the coordinate

representation ξx!!!) is “differentiable” at a particular time t ∈ I ? One obviousanswer is to say that its coordinate representation is differentiable at t. This has thedrawback that, as we said before, there are going to be lots of different coordinatecharts for Q, which will give rise to different coordinate representations of our curveξ. But let us postpone the discussion of this difficulty till later, and, temporarily,give the following

Definition 2.2.1. Let Q be a set, and let I � t �→ ξ(t) ∈ Q be a Q-valued mapdefined on an interval I ⊆ R. Let x be an n-dimensional chart on Q. Assume thatξ(t) ∈ Dx for all t ∈ I. Let t ∈ I. We say that ξ is differentiable at time t relativeto the chart x if the coordinate representation ξx is differentiable at t. ♦

Now we would like to talk about the “velocity” ξ(t) (that is, the derivative ofξ at time t). Later we will give a precise definition of this object, but even withouthaving done this, it should be reasonable to expect that, whatever ξ(t) turns outto be, it will be represented, relative to the chart x, by the vector ξx(t), which is aperfectly well defined n-tuple of numbers, i. e., a member of R

n.We can thus give the following

Definition 2.2.2. Let Q, I, ξ, x, t be as in Definition 2.2.1, and assume thatξ is differentiable at t relative to the chart x. Then the x-representation of ξ(t) isthe vector (ξ(t))x ∈ R

n given by

(ξ(t))x ∈ Rn =

˙︷︸︸︷ξx (t) . ♦

Notice that we have not yet defined what the “velocity vector” ξ(t) is. We haveonly said what its representation (ξ(t))x relative to a particular chart x is. Later,we will assign a precise meaning to the expression ξ(t), as a “tangent vector.”

2.3. Invariance of the Euler-Lagrange equation. We have now come tothe main point of this discussion: the important discovery, made by Lagrange, thatthe Euler-Lagrange equation is invariant under arbitrary changes of coordinates.

Let us first show what this means by doing a straightforward calculation. Sup-pose we have a curve

I � t �→ ξ∗(t) ∈ Q

and two different coordinate charts

x : Dx �→ Rx ⊆ Rn , X : DX �→ RX ⊆ R

n .

Suppose our curve ξ∗ is contained in both domains, that is,

ξ∗(t) ∈ Dx ∩ DX for all t ∈ I .

Suppose our two coordinate charts are C2 related, in the following sense:

Definition 2.3.1. Let Q be a set, and let x, X, be two n-dimensional chartson Q. Let k be a nonnegative integer. We say that x and X are Ck related if

1. the images x(Dx ∩ DX), X(Dx ∩ DX) of the “overlap set” Dx ∩ DX underthe coordinate maps x, X, are open in Rx, RX, respectively;


2. the “change of coordinates” maps

Φx,X : x(Dx ∩DX) �→ X(Dx ∩ DX) ,

ΦX,x : X(Dx ∩DX) �→ x(Dx ∩ DX) ,

defined by the conditions

Φx,X(x) = X whenever x = x(q) , X = X(q) for some q ∈ Dx ∩ DX ,

ΦX,x(X) = x whenever x = x(q) , X = X(q) for some q ∈ Dx ∩ DX ,

are of class Ck. ♦

(The map Φx,X is the “coordinate change from x to X,” that is, the map thatassigns, to each point x ∈ Rx which is the x-coordinate representation of someq ∈ Dx ∩ DX, the point X that represents the same q in the X-chart. It is clearthat Φx,X is well defined, because, if x ∈ Rx is the x-coordinate representation ofsome q ∈ Dx ∩ DX, then this q is unique, because x is injective, and X(q) exists,because q ∈ DX. Naturally, similar remarks apply to the map ΦX,x.)

Now suppose L is a “Lagrangian on Q.” The precise meaning of this is that Lis a function of points q ∈ Q, “velocity vectors” (i. e., tangent vectors) q, and time,that is, L is a function

Q|Q× [a, b] � (q, q, t) �→ L(q, q, t) ∈ R ,

defined at points (q, q) in the “configuration-velocity space” Q|Q and times t in someinterval [a, b]. More precisely, what we are now temporarily calling “configuration-velocity space” will later be called “the tangent bundle of configuration space,”and the mysterious “velocity vectors” will be interpreted as “tangent vectors.”Since we have not yet defined the notion of a tangent vector, we shall for the timebeing assume that L—whatever it may be—is specified by giving its representation(x, x, t) �→ Lx(x, x, t), for each coordinate chart x, by means of a function Lx.

In addition, we shall assume that “the values of Lx(x, x, t) and LX(X, X, t) arethe same whenever x, X are coordinate representations of the same point q, and x,X are coordinate representations of the same ‘velocity’ q relative to two charts x,X.” First of all, we have to decide what this means. Clearly, what it should meanis that

Lx(x(q, q), t) = LX(X(q, q), t) whenever (q, q) ∈ Dx ∩ DX ,(2.1)

where x(q, q), X(q, q), are the coordinate representations of a “configuration-velocitypoint” (q, q) with respect to the charts x, X. Equivalently, (2.1) should say that

Lx(x, t) = LX(X, t) wheneverx = x(q, q) and X = X(q, q) for some ∈ Dx ∩DX .

(2.2)

The first thing we have to do is figure out what (2.2) means directly in terms of thefunctions Lx and LX, without going through the intermediate point (q, q). Clearly,the vectors x, X are coordinate representations of a position and velocity, so theyshould belong to R

2n and be given as pairs (x, x), (X, X), where x, X represent q

and x, X represent q. Moreover, as long as q ∈ Dx ∩ DX, we already know how toexpress X in terms of x, using the map Φx,X, and there ought to be a similar wayto “express x in terms of X.” (As we will soon see, this is not 100% accurate. Wewill express x in terms of X and X, not just in terms of X alone.) Let us find theformula for this. Using Φ for Φx,X, and Ψ for ΦX,x, we have

10 H. J. SUSSMANN

Xi = Φi(x1, . . . , xn)xj = Ψj(X1, . . . , Xn) ,

for i = 1, . . . , n, j = 1, . . . , n. Then we can express X as a function Φ(x, x), and x

as a function Ψ(X, X), by differentiating. The result is

Xi = Φi(x1, . . . , xn, x1, . . . , xn) =n∑

j=1

∂Φi

∂xj(x1, . . . , xn)xj ,(2.3)

xj = Ψj(X1, . . . , Xn, X1, . . . , Xn) =n∑

i=1

∂Ψj

∂Xi(X1, . . . , Xn)Xi ,(2.4)

for i = 1, . . . , n, j = 1, . . . , n. So now we are able to write the complete formulaerelating x = (x, x) and X = (X, X):

Xi = Φi(x1, . . . , xn) ,(2.5)

Xi = Φi(x1, . . . , xn, x1, . . . , xn) ,(2.6)xj = Ψj(X1, . . . , Xn) ,(2.7)

xj = Ψj(X1, . . . , Xn, X1, . . . , Xn) ,(2.8)

for i = 1, . . . , n, j = 1, . . . , n, where the functions Φi, Ψj are defined by (2.3) and(2.4). (Notice that, as we announced before, we cannot express X just in terms ofx; the formulas actually give X as a function of x and x.)

The identities (2.3) and (2.4) are the transformation formulas for velocity vec-tors. They tell us how to find the components of such a vector in one coordinatesystem if we know the components in another coordinate system.

Let us now compute the partial derivatives of the functions given by (2.5), (2.6),(2.7), (2.8). We have

∂Xi

∂xj=

∂Φi

∂xj(x1, . . . , xn) ,

∂Xi

∂xj=

n∑k=1

∂2Φi

∂xj∂xk(x1, . . . , xn)xk ,

∂Xi

∂xj=

∂Φi

∂xj(x1, . . . , xn) ,

∂xj

∂Xi=

∂Ψj

∂Xi(X1, . . . , Xn) ,

∂xj

∂Xi=

n∑k=1

∂2Ψj

∂Xi∂Xk(X1, . . . , Xn)Xk ,

∂xj

∂Xi=

∂Ψj

∂Xi(X1, . . . , Xn) .

Now let us study how the components of the momentum are represented in our twocharts x, X. Let us use p, P to denote the two representations, so

p = (p1, . . . , pn) =( ∂L

∂x1, . . . ,

∂L

∂xn

)


and

P = (P1, . . . , Pn) =( ∂L

∂X1, . . . ,

∂L

∂Xn

).

Repeated use of the Chain Rule gives

∂L

∂Xi=

n∑k=1

∂L

∂xk· ∂xk

∂Xi+

n∑k=1

∂L

∂xk· ∂xk

∂Xi

=n∑

k=1

∂L

∂xk· ∂Ψk

∂Xi+

n∑k=1

n∑�=1

∂L

∂xk· ∂2ψk

∂Xi∂X�X� ,

Pi =∂L

∂Xi

=n∑

j=1

∂L

∂xj· ∂xj

∂Xi+

n∑j=1

∂L

∂xj· ∂xj

∂Xi

=n∑

j=1

∂L

∂xj· ∂xj

∂Xi

=n∑

j=1

∂L

∂xj· ∂Ψj

∂Xi

=n∑

j=1

pj ·∂Ψj

∂Xi.

So we got the formula

Pi =n∑

j=1

pj ·∂Ψj

∂Xi,(2.9)

and an identical argument also yields the identity

pj =n∑

i=1

Pi ·∂Φi

∂xj.(2.10)

Let us look at these transformation formulae together with the identities we derivedearlier for velocity vectors:

Pi =n∑

j=1

∂Ψj

∂Xipj ,(2.11)

pj =n∑

i=1

∂Φi

∂xjPi ,(2.12)

Xi =n∑

j=1

∂Φi

∂xjxj ,(2.13)

xj =n∑

i=1

∂Ψj

∂XiXi .(2.14)

Notice that the transformation formulae for the momentum are different from theformulae for the velocities. The formula giving you X in terms of x is the sameas that giving you p in terms of P . In other ways, the momentum and the velocity

12 H. J. SUSSMANN

transform in opposite ways. The same rule that enables you to go from x to X willtake you from P to p , not from p to P !!

For historical reasons, objects such as the momentum are called covariant vec-tors, or, simply, covectors. Objects such as the velocity, which transform in theopposite way, are called contravariant vectors, or, simply, vectors, or ordinary vec-tors, or tangent vectors.

Remark 2.3.2. You may have noticed that we used superscripts for the com-ponents of velocity vectors, and subscripts for the components of momentum “vec-tors.” This is common practice in differential geometry and physics, and there arevery good reasons for it, which will be discussed later. ♦

We now return to the issue of the invariance of the Euler-Lagrange equation.The equation says that a certain “vector” is equal to zero. This “Euler-Lagrangevector” is the one having components

ei = ai − bi

relative to the x chart, where

ai =d

dt

(∂L

∂xi

),

bi =∂L

∂xi.

In the X system, the ““Euler-Lagrange vector” will have components

Ei = Ai −Bi

where

Ai =d

dt

(∂L

∂Xi

),

Bi =∂L

∂Xi.

We now have to find the transformation formula relating the Ei to the ei. For thispurpose, we first study the transformation of the ai to the Ai and of the bi to theBi. We have

Ai =d

dt

(∂L

∂Xi

)

=n∑

k=1

d

dt

(∂L

∂xk

)· ∂Ψk

∂Xi+

n∑k=1

∂L

∂xk· d

dt

(∂Ψk

∂Xi

)

=n∑

k=1

d

dt

(∂L

∂xk

)· ∂Ψk

∂Xi+

n∑k=1

n∑�=1

∂L

∂xk· ∂2Ψk

∂Xi∂X�X�

=n∑

k=1

ak · ∂Ψk

∂Xi+

n∑k=1

n∑�=1

pk · ∂2Ψk

∂Xi∂X�X� ,


Bi =∂L

∂Xi

=n∑

k=1

∂L

∂xk· ∂Ψk

∂Xi+

n∑k=1

n∑�=1

∂L

∂xk· ∂2ψk

∂Xi∂X�X�

=n∑

k=1

bk · ∂Ψk

∂Xi+

n∑k=1

n∑�=1

pk · ∂2ψk

∂Xi∂X�X� ,

and then

Ei = Ai − Bi

=n∑

k=1

ak · ∂Ψk

∂Xi+

n∑k=1

n∑�=1

pk · ∂2Ψk

∂Xi∂X�X�

−n∑

k=1

bk · ∂Ψk

∂Xi−

n∑k=1

n∑�=1

pk · ∂2Ψk

∂Xi∂X�X�

=n∑

k=1

ak · ∂Ψk

∂Xi−

n∑k=1

bk · ∂Ψk

∂Xi

=n∑

k=1

(ak − bk) · ∂Ψk

∂Xi

=n∑

k=1

ek · ∂Ψk

∂Xi.

So we have derive the formula

Ei =n∑

k=1

ek · ∂Ψk

∂Xi,(2.15)

from which it follows that if all the the ei vanish then all the Ei vanish as well. Anidentical argument works in the other direction as well, so we have in fact shownthat

e1 = e2 = . . . = en = 0 ⇐⇒ E1 = E2 = . . . = En = 0 .(2.16)

In other words

The Euler-Lagrange equationholds in the x chart if andonly if it holds in the X chart.

This is the desired invariance statement.

2.4. A comment on vectors, covectors, and tensors. Let us compare thetransformation formulae for the velocity, the momentum, the “Euler-Lagrange vec-tor,” and other objects we have introduced, such as the two parts whose difference

14 H. J. SUSSMANN

is the Euler-Lagrange vector. The formulas say that

Xi =n∑

k=1

∂Φi

∂xkxk ,(2.17)

xj =n∑

k=1

∂Ψj

∂XkXk ,(2.18)

Pi =n∑

k=1

∂Ψk

∂Xipk ,(2.19)

pj =n∑

k=1

∂Φk

∂xjPk ,(2.20)

Ei =n∑

k=1

∂Ψk

∂Xiek ,(2.21)

ej =n∑

k=1

∂Φk

∂xjEk .(2.22)

Ai =n∑

k=1

ak · ∂Ψk

∂Xi+

n∑k=1

n∑�=1

pk · ∂2Ψk

∂Xi∂X�X� ,(2.23)

aj =n∑

k=1

Ak · ∂Φk

∂xi+

n∑k=1

n∑�=1

Pk · ∂2Φk

∂xi∂x�x� ,(2.24)

Bi =n∑

k=1

bk · ∂Ψk

∂Xi+

n∑k=1

n∑�=1

pk · ∂2ψk

∂Xj∂X�X� . ,(2.25)

bj =n∑

k=1

Bk · ∂Φk

∂xj+

n∑k=1

n∑�=1

Pk · ∂2Φk

∂Xi∂x�x� .(2.26)

These formulas show that:1. The transformation law for the es and Es is the same as that for the ps and

P s. In other words: the Euler-Lagrange “vector” is really a covector, likethe momentum.

2. The transformation laws for the xs and Xs are “opposite” from those forthe ps and P s. This why velocities are called contravariant vectors.

3. In all three cases (that is, in the transformation laws for the es and Es,the ps and P s, and the xs and Xs), the laws are such that if the objectunder consideration vanishes in one coordinate chart then it vanishes in allof them. That is,

(T) the vanishing of the object is an invariant property, not depending onthe choice of a coordinate chart.

Later, we will define other quantities, called tensors, which are more generalthan vectors and covectors, but still have Property (T). Property (T) willturn out to be a manifestation of the tensor nature of the velocity and themomentum.

4. The transformation laws for the as and As and the bs and Bs are quitedifferent. In particular, Property (T) does not hold. So the two parts whosedifference is the Euler-Lagrange covector are not themselves covectors, andare not even tensors of any kind.

HAMILTON’S EQUATIONS

AND WEIERSTRASS’

SIDE CONDITIONHector J. Sussmann

November 15, 2000

CONTENTS

1. THE TWO FORMS OH HAMILTON’S SYSTEM OF EQUATIONS 11.1 The Legendre condition 21.2 Hamilton’s missed opportunity 31.3 The “control Hamiltonian” versus Hamilton’s Hamiltonian 6

2. WEIERSTRASS MISSES ANOTHER OPPORTUNITY 72.1 The Weierstrass side condition 82.2 A Hamiltonian rewriting of the Weierstrass condition 112.3 What Weierstrass missed 122.4 A toy problem 13

1. THE TWO FORMS OH HAMILTON’S SYSTEM OF EQUATIONS

We will now discuss an interesting story of “missed opportunities,” by showinghow Hamilton in the 1830s, Weierstrass in the 1870s, and Caratheodory1 in the1930s, had all the knowledge that was needed to make a dramatic discovery andget close to what we now call the “maximum principle” of optimal control, butfailed to make it for a fairly trivial reason. Specifically, it was merely the factthat they chose one particular formalism rather than another that on the surfacelooked nearly identical, that led them astray and prevented them from making thediscovery.

We will argue that

1This handout just deals with Hamilton and Weierstrass. The discussion of Caratheodory’s

attempt to rewrite the Weierstrass condition in Hamiltonian form will be given next week.

2 H. J. SUSSMANN

The evolution of the calculus of variations reached a criticalfork in the road when Hamilton’s equations were introduced.The way the equations were actually written was only one oftwo possible formulations, and may not have been the best one.If Hamilton’s equations had been written in a slightly differ-ent way—using a Hamiltonian that is a function of all threesets of variables (positions, velocities and momenta) ratherthan a function of the positions and momenta only—thensome important discoveries that were not made until the 1950swould almost certainly have been made much sooner. Weier-strass and Caratheodory were misled by the fact that only the“wrong” form of the equations was available to them.

Before we get to Hamilton, Weierstrass, and Carathedory, it will be cobvenient toanalyze the second-order necessary condition due to Legendre.

1.1. The Legendre condition. Recall that the Euler-Lagrange system is

given by

ddt

(∂L∂x

)= ∂L

∂x(1.1)

or

ddt

[∂L∂u

(ξ∗(t), ξ∗(t), t

)]= ∂L

∂x

(ξ∗(t), ξ∗(t), t

),(1.2)

or

ddt

[∂L∂ui

(ξ∗(t), ξ∗(t), t

)]= ∂L

∂xi

(ξ∗(t), ξ∗(t), t

), i = 1, . . . , n .(1.3)

This necssary condition for an optimum can be derived, as we shall see later,by looking at the “first variation” δI of the cost functional I, and requiring thatδI = 0. So in fact (1.1) is a necessary condition for the stationarity of I.

The next natural step was to look at the second variation of I, and this was donein 1786 by Adrien-Marie Legendre (1752-1833), who found an additional necessarycondition for a minimum. His condition, derived for the scalar case, is

∂2L∂x2

(ξ∗(t), ξ∗(t), t

)≥ 0 (i.e., ∂2L

∂u2

(ξ∗(t), ξ∗(t), t

)≥ 0) .(1.4)

With an appropriate reinterpretation, Legendre’s condition (1.4) is also necessaryin the vector case: all we have to do is read (1.4) as asserting that the Hessianmatrix

{∂2L

∂ui∂uj

(ξ∗(t), ξ∗(t), t

)}1≤i,j≤n

has to be nonnegative definite, that is, that

∑ni,j=1 αiαj ∂2L

∂ui∂uj

(ξ∗(t), ξ∗(t), t

)≥0 for all α=(α1, . . . , αn)∈R

n.(1.5)

HAMILTON AND WEIERSTRASS 3

1.2. Hamilton’s missed opportunity. The work of William Rowan Hamil-ton (1805-1865), published in 1834 and 1835, produced a far-reaching reformulationof the Euler-Lagrange equations. But we will see that this reformulation could havebeen even more powerful if the new equations had been written in a “slightly dif-ferent” form. To explain this, we will have to write and analyze in detail bothversions, the one that Hamilton used, and the one that he could have used butdid not, so that we can then compare them and show that what looks like a smalldifference has in fact enormous implications.

In a sense, the issue at stake will seem rather trivial, just a matter of rewrit-ing the Euler-Lagrange system in a different formalism. However, sometimes for-malisms can make a tremendous difference. To understand what happened andwhat could have happened but did not, let us first see if we can make sense of thetwo necessary conditions for a minimum that have been presented so far, namely,the Euler-Lagrange equation (1.1) and the Legendre condition (1.4), and look fora single simple property that would imply both conditions.

The Legendre condition is clearly the second-order necessary condition for aminimum of a function, namely, the function R

n � u �→ L(ξ∗(t), u, t). But (1.1)does not look at all like the first-order condition for a minimum of that samefunction. It is natural to ask whether there might be a way to relate the twoconditions. Is it possible that both could be expressed as necessary conditions fora minimum of one and the same function? The answer is “yes,” and understandinghow this is done leads straight to optimal control theory, the maximum principle,and far-reaching generalizations of the classical theory. But before we get therelet us tell the story of how Hamilton almost got there himself but missed, andWeierstrass got even closer but missed as well.

Let us look at another way of writing (1.1). Suppose a curve t �→ ξ∗(t) isa solution of (1.1). Define a function (x, u, p, t) �→ H(x, u, p, t) of three vectorvariables x, u, p in R

n, and of t ∈ R, by letting

H(x, u, p, t) = 〈p, u〉 − L(x, u, t) .(1.6)

Then define the “momentum” p by

π(t) =∂L

∂u

(ξ∗(t), ξ∗(t), t

).(1.7)

It is then clear that∂H

∂p= u ,

so along our curve ξ∗:

dξ∗dt

(t) =∂H

∂p

(ξ∗(t), ξ∗(t), π(t), t

).(1.8)

Also,∂H

∂x= −∂L

∂x.

Therefore (1.2), with π(t) defined by (1.7), says that

dπ

dt(t) = −∂H

∂x

(ξ∗(t), ξ∗(t), π(t), t

).(1.9)

4 H. J. SUSSMANN

Finally,∂H

∂u= p − ∂L

∂u,(1.10)

so (1.7) says:∂H

∂u

(ξ∗(t), ξ∗(t), π(t), t

)= 0 .(1.11)

So we have shown that

The system of equations (1.8), (1.9), (1.11), usually written moreconcisely as

dxdt

= ∂H∂p

, dpdt

= −∂H∂x

, ∂H∂u

= 0 ,(1.12)

is exactly equivalent to (1.1), provided that H is defined as in (1.6).

More precisely,

Theorem 1.2.1. Assume n is a positive integer, Ω is an open subset of Rn, a,

b are real numbers such that a < b, L is a real-valued function on Ω × Rn × [a, b],

and x, x are given points in Ω. Assume that L is a function of class C1. Letξ∗ : [a, b] �→ Ω be a curve of class C1. Then the Euler-Lagrange equation (1.2)holds for all t ∈ [a, b] if and only if the “control Hamilton equations” (1.8), (1.9),(1.11) are satisfied for some choice of the function t �→ π(t). Moreover, in that casethe momentum function π is necessarily given by (1.7). ♦

We will call the function H the control Hamiltonian, and refer to (1.12) as thecontrol Hamiltonian form of the Euler-Lagrange equations. In our view, Formula(1.6) is the definition that Hamilton should have given for the Hamiltonian, andEquations (1.12) are “Hamilton’s equations as he should have written them.”

What Hamilton actually wrote was (in our notation, not his)dx

dt=

∂H∂p

,dp

dt= −∂H

∂x,(1.13)

where (x, p, t) �→ H(x, p, t) is a function of x, p and t alone, defined by the formula

H(x, p, t) = 〈p, x〉 − L(x, x, t) ,(1.14)

which resembles (1.6) but is not at all the same. The difference is that in Hamilton’sdefinition x is supposed to be treated not as an independent variable, but as afunction of x, p, t, defined implicitly by the equation

p =∂L

∂x(x, x, t) .(1.15)

It is easy to prove the following:

If the map (x, x, t) �→ (x, p, t) defined by (1.15) can be inverted,i.e., if we can “solve (1.15) for x as a function of x, p, t,” then (1.13)is equivalent to (1.12).

More precisely,


Theorem 1.2.2. Assume n is a positive integer, Ω is an open subset of Rn, a,

b are real numbers such that a < b, L is a real-valued function on Ω × Rn × [a, b],

and x, x are given points in Ω. Assume that L is a function of class C1. Letξ∗ : [a, b] �→ Ω be a curve of class C1. Let π be the function given by (1.7). Assumethat for some δ > 0 there exists a map

Uδ � (x, p, t) �→ v(x, p, t) ,

of class C1, defined on the set

Uδ = {(x, p, t) : a ≤ t ≤ b , ‖x− ξ∗(t)‖ < δ , ‖p − π(t)‖ < δ} ,

such that

p =∂L

∂u(x, u, t) ⇐⇒ u = v(x, p, t) whenever (x, p, t) ∈ Uδ , u ∈ R

n .(1.16)

Define

H(x, p, t) = 〈p, v(x, p, t)〉 − L(x, v(x, p, t), t) .(1.17)

Then the Euler-Lagrange equation (1.2) holds for all t ∈ [a, b] if and only if the“Hamilton equations”

dx

dt=

∂H∂p

,dp

dt= −∂H

∂x,(1.18)

are satisfied along (ξ∗, π), that is, if and only if

ξ∗(t) =∂H∂p

(ξ∗(t), π(t), t) ,

π(t) = −∂H∂x

(ξ∗(t), π(t), t) ,

for all t ∈ [a, b]. ♦

Proof. If (x, p, t) ∈ Uδ , then it is clear that

H(x, p, t) = H(x, v(x, p, t), p, t) .

So the chain rule tells us that∂H∂x

=∂H

∂x+

∂H

∂u· ∂v

∂x,(1.19)

that is,

∂H∂x

(x, p, t) =∂H

∂x(x, v(x, p, t), p, t)+

∂H

∂u(x, v(x, p, t), p, t) · ∂v

∂x(x, p, t) .(1.20)

Now, (1.10) implies

∂H

∂u(x, v(x, p, t), p, t) = p − ∂L

∂u(x, v(x, p, t), p, t) = p − p = 0 .(1.21)

Therefore∂H∂x

(x, p, t) =∂H

∂x(x, v(x, p, t), p, t) ,(1.22)

and, similarly,

∂H∂p

(x, p, t) =∂H

∂p(x, v(x, p, t), p, t) .(1.23)

6 H. J. SUSSMANN

If ξ∗ is a solution of Euler-Lagrange and π is defined by (1.7), then (1.7) and (1.16)imply ξ∗(t) = v(ξ∗(t), π(t), t), and Theorem 1.2.1 tells us that

ξ∗(t) =∂H

∂p(ξ∗(t), ξ∗(t), π(t), t) ,

π(t) = −∂H

∂x(ξ∗(t), ξ∗(t), π(t), t) ,

so

ξ∗(t) =∂H

∂p(ξ∗(t), ξ∗(t), π(t), t)

=∂H

∂p(ξ∗(t), v(ξ∗(t), π(t), t), π(t), t)

=∂H∂p

(ξ∗(t), π(t), t) ,

π(t) = −∂H

∂x(ξ∗(t), ξ∗(t), π(t), t)

= −∂H

∂x(ξ∗(t), v(ξ∗(t), π(t), t), π(t), t)

= −∂H∂x

(ξ∗(t), π(t), t) ,

showing that (1.18) is satisfied. The converse is also easily proved.

1.3. The “control Hamiltonian” versus Hamilton’s Hamiltonian. Itshould be clear from the above discussion that the Hamiltonian reformulation ofthe Euler-Lagrange equations in terms of the “control Hamiltonian” is at leastas natural as the classical version (1.13)-(1.14)-(1.15), and perhaps even simpler.Moreover, the control formulation has at least one obvious advantage, namely,

(A1) the control version of the Hamilton equations is exactly equiv-alent to the Euler-Lagrange system under completely general condi-tions, whereas the classical version (1.13)-(1.14)-(1.15) only makessense when the transformation (1.15) can be inverted, at least locally,to solve for x as a function of x, p, t.

We now show that (A1) is not the only advantage of the control view over the classi-cal one. To see this, we must take another look at Legendre’s condition (1.4). SinceH(x, u, p, t) is equal to −L(x, u, t) plus a linear function of u, (1.4) is completelyequivalent to

∂2H

∂u2

(ξ∗(t), ξ∗(t), π(t), t

)≤ 0 , i.e.,

∂2H

∂x2

(ξ∗(t), ξ∗(t), π(t), t

)≤ 0 .(1.24)

Now let us write (1.24) alongside the third equation of (1.12):

∂H∂u

= 0 and∂2H∂u2 ≤ 0 ,(1.25)

and stare at the result for a few seconds.These equations unmistakably suggest something! They show that


the Euler-Lagrange and Legendre conditions have to do with criti-cal points—with respect to the velocity variable u—of a function H.Precisely, Euler-Lagrange says that the gradient ∇uH of this func-tion has to vanish, and then Legendre adds to this the extra require-ment that the second u-derivative of H—i.e., the Hessian matrix{ ∂2H

∂ui∂uj}1≤i,j≤n—must be nonpositive definite.

The message should be clear: what must be happening here is that H has amaximum as a function of u.

We state this formally as a conjecture.

CONJECTURE M: Besides (1.12) (or the equivalent form (1.1)), anadditional necessary condition for optimality of an arc ξ∗ should bethat the function R

n � u �→ H(ξ∗(t), u, π(t), t) ∈ R have a max-imum at ξ∗(t) for each t. That is, “the velocity that is actuallyused must maximize the Hamiltonian—for fixed time, position andmomentum—among all possible values of the velocity vector.”

We contend that

1. Conjecture M is an extremely natural consequence of rewriting Hamilton’sequations “as Hamilton should have done it,” in terms of we are calling the“control Hamiltonian.”

2. It is reasonable to guess that, if Hamilton had actually written “Hamilton’sequations” in this alternative way, then he himself, or some other 19th cen-tury mathematician, would have written (1.25) and been led by it to theconjecture.

3. On the other hand, it is only by using the Hamiltonian of (1.6), as op-posed to Hamilton’s own form of the Hamiltonian, that one can see that theEuler-Lagrange and Legendre condition really are first- and second-orderconditions about critical points of the same function. This function can-not be L itself, because the first order conditions do not say that ∂L

∂u= 0.

Nor can it be Hamilton’s Hamiltonian H, which isn’t even a function of u.Only the use of the “control” Hamiltonian leads naturally to Conjecture M.

It will turn out that

Conjecture M is true. Moreover, once its truth is knownthen vast generalizations are possible.

2. WEIERSTRASS MISSES ANOTHER OPPORTUNITY

The contributions of Karl Weierstrass (1815-1897) to the calculus of variations arecontained in the notes of his three series of lectures on the topic, given in 1875,1879 and 1882, and not formally published as books or journal articles.

8 H. J. SUSSMANN

2.1. The Weierstrass side condition. Weierstrass considered the problemof minimizing an integral I(ξ) of the form

I(ξ) =∫ b

a

L(ξ(s), ξ(s)) ds ,

for Lagrangians L such that

(W) L(x, x) is positively homogeneous with respect to the velocity x (that is,L(x, α x) = αL(x, x) for all x, x and all α ≥ 0) and does not depend ontime. ♦

As will become clear from Remark 2.1.1, we have a good reason for using s ratherthan t as the “time” variable in the expression for I.

Remark 2.1.1. In a sense, one can always assume (W) “without loss of gen-erality.” Indeed, suppose we are interested in a completely arbitrary problemP(L, a, b, x, x) of the standard form considered earlier, namely,⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

given L, a, b, x, x ,

minimize I =∫ b

a L(ξ(t), ξ(t), t)dt ,

subject to ξ(a) = x and ξ(b) = x .

(2.1)

Then we can always define a new function Λ(x, t, u, τ) = τL(x, u/τ, t), and thinkof t as a new x variable, say x0, and of τ as , dx0

ds, were s is a new time variable, or

“pseudotime,” not to be confused with the true time variable t. Then, using xnew

for (x, x0)—so that xnew = (x, t)—we have∫L(x, x, t)dt =

∫L(x,

dx

dt, t)dt

=∫

L

(x,

(dt

ds

)−1 dx

ds, t

)dt

dsds

=∫

Λ

(xnew,

dxnew

ds

)ds ,

so the minimization problem for Λ is equivalent to (2.1). Moreover, if α>0, then

Λ(x, t, αu, α τ)=αΛ(x, t, u, τ) ,

i. e.,Λ(xnew, α xnew)=αΛ(xnew, xnew) ,

where we are now using the dot to denote differentiation with respect to s ratherthan t. So the new Lagrangian Λ is positively homogeneous with respect to thenew velocity variable xnew and does not depend on the new time s. ♦

So when Weierstrass imposed Condition (W) on his Lagrangians, he was doingso “without loss of generality.” However, “without loss of generality” is a danger-ous phrase, and does not at all entail “without loss of insight.” We shall arguebelow that this restriction, in conjunction with the dominant view that Hamilton’sequations had to be written in the form (1.13), may have served to conceal fromWeierstrass the true meaning and the far-reaching implications of the new conditionhe discovered.


Weierstrass introduced what is now known as

THE WEIERSTRASS EXCESS FUNCTION,

E(x, u, u) = L(x, u)− ∂L

∂u(x, u) · u .(2.2)

This function depends on three sets of independent variables, namely, x, u and u.That is, E is a function of position and two velocity vectors.

He then proved his side condition:

(SC) For a curve s �→ ξ∗(s) to be a solution of the minimizationproblem, the function E has to be ≥ 0 when evaluated forx = ξ∗(s), u = ξ∗(s), and a completely arbitrary u.

Notice that Lagrangians with Property (W) satisfy the identity

L(x, u) =∂L

∂u(x, u) · u .(2.3)

Therefore Weierstrass could equally well have written his excess function as

E(x, u, u) = L(x, u)− ∂L

∂u(x, u) · u −

(L(x, u)− ∂L

∂u(x, u) · u

),(2.4)

and he could even have generalized this formula to the time-dependent case, bywriting

E(x, u, u, t)=L(x, u, t)−∂L

∂u(x, u, t)·u−

(L(x, u, t)−∂L

∂u(x, u, t)·u

).(2.5)

Remark 2.1.2. Formulae (2.4) and (2.5) have a simple geometric interpetation.Assume, for simplicity, that L does not depend on t. For a given point x, let Gx

be the graph ofy = L(x, u) def= Lx(u)

as a function of u. Fix a value u of u, and let

y = L(x, u) ,

so the pointP = (u, y)

lies on Gx. Let Γx,u be the tangent hyperplane to Gx at P . Then the equation ofΓx,u is

y = y +∂L

∂u(x, u) · (u − u) ,

that isy = L(x, u) +

∂L

∂u(x, u) · (u − u) .

10 H. J. SUSSMANN

Given any u, the ordinates ygraph(u), yplane(u) of the points of Gx, Γx,u, that lieon the line {u} × R are given by

ygraph(u) = L(x, u) , yplane(u) = L(x, u) +∂L

∂u(x, u) · (u − u) .

ThereforeE(x, u, u) = ygraph(u) − yplane(u) .

In other words (cf. Figure 1):

The number E(x, u, u) tells us by how much the actual value of thefunction y = Lx(u) at a particular u is larger than the value at uof the linearization (i.e., tangent approximation) of this functionat the point (u, L(x, u)).

y

u

ε(x (s),x (s),u)* *

.-u=x (s)

*y=L(x (s),u)

*y=L(x (s),u)

*

.

- --

P

Figure 1. The Weierstrass side condition


Therefore the Weierstrass side condition says that

(SC) If a curve t �→ ξ∗(t) is a solution of the minimization prob-lem, then for every t, if we draw the tangent hyperplane tothe graph of the function u �→ L(ξ∗(t), u, t) at the point ofthis graph corresponding to u = ξ∗(t), then the graph mustlie entirely above the hyperplane. ♦

2.2. A Hamiltonian rewriting of the Weierstrass condition. Using

π(t) =∂L

∂u(ξ∗(t), ξ∗(t), t) ,

as in (1.7), we see that

E(ξ∗(t), ξ∗(t), u, t)(2.6)

=(L(ξ∗(t), u, t)− 〈π(t), u〉

)−

(L(ξ∗(t), ξ∗(t), t) − 〈π(t), ξ∗(t)〉

),

which the reader will immediately recognize as saying that

E(ξ∗(t), ξ∗(t), u, t) = H(ξ∗(t), ξ∗(t), π(t), t) − H(ξ∗(t), u, π(t), t) ,(2.7)

where H is our “control Hamiltonian.” So Weierstrass’ condition, expressed interms of the control Hamiltonian, simply says that

(MAX) Along an optimal curve t �→ ξ∗(t), if we define π(t) via (1.7),then for every t the value u = ξ∗(t) must maximize the (con-trol) Hamiltonian H(ξ∗(t), u, π(t), t) as a function of u.

Moreover, (MAX) can be considerably simplified. Indeed,

The requirement that π(t) be defined via (1.7) is now redundant.

To see this, observe that, if H(ξ∗(t), u, π(t), t), regarded as a function of u, has amaximum at u = ξ∗(t), then ∂H

∂u (ξ∗(t), ξ∗(t), π(t), t) has to vanish, so π(t) must begiven by (1.7). Moreover, the vanishing of ∂H

∂u (ξ∗(t), ξ∗(t), π(t), t) is also one of theconditions of (1.12). So we can just drop (1.7) altogether, and state (1.12) and(MAX) together:

(NCO) If a curve t �→ ξ∗(t) is a solution of the minimization problem(2.1), then there has to exist a function t �→ π(t) such that thefollowing three conditions hold for all t:

ξ∗(t) =∂H

∂p(ξ∗(t), ξ∗(t), π(t), t) ,(2.8)

π(t) = −∂H

∂x(ξ∗(t), ξ∗(t), π(t), t) ,(2.9)

H(ξ∗(t), ξ∗(t), π(t), t) = maxu

H(ξ∗(t), u, π(t), t) .(2.10)

12 H. J. SUSSMANN

As a version of the necessary conditions for optimality, (NCO) encapsulates in onesingle statement the combined power of the Euler-Lagrange necessary conditionsand the Weierstrass side condition as well, of course, as the Legendre condition,which obviously follows from (NCO). Notice the elegance and economy of languageachieved by this unified statement: there is no need to bring in an extra entitycalled the “excess function.” Nor does one need to include a formula specifyinghow π(t) is defined, since (2.10) does this automatically. So the addition of thenew Weierstrass condition to the three equations of (1.12) results in a new set ofthree, rather than four, conditions, a set much “simpler than the sum of its parts.”Notice moreover that (2.10) is exactly Conjecture M. So we can surmise at thispoint that (NCO), as stated, could probably have been discovered soon after thework of Hamilton—since it is strongly suggested by (1.25)—and almost certainlyby Weierstrass, if only Hamilton’s equations had been written in the form (1.6),(1.12).

So we can now add two new items to our list of advantages of the “controlformulation” of Hamilton’s equations over the classical one:

(A2) Using the control Hamiltonian, it would have been an obvious next step towrite Legendre’s condition in “Hamiltonian form,” as in (1.25), and this wouldhave led immediately to the formulation of Conjecture M, a proof of which wouldthen have been found soon after.

(A3) In terms of the control Hamiltonian, the Weierstrass side condition has amuch simpler statement, not requiring the introduction of an “excess function,” andcan be combined with the Hamilton equations into an elegant unified formulation(NCO) of the necessary conditions for optimality.

2.3. What Weierstrass missed. This is by no means the end of our story.There is much more to the new formulation (NCO) than just elegance and sim-plicity. If you compare (NCO) with all the other necessary conditions that we hadwritten earlier, a remarkable new fact becomes apparent. Quite amazingly, thederivatives with respect to the u variable are completely gone. All the earlier equa-tions involved u-derivatives of L or of H, and even if we use the classical version(1.13) of Hamilton’s equations—which involves no functions of u and therefore nou-derivatives—the fact remains that in order to get to (1.13) we first have to solve(1.15), which does involve a u-derivative.

Now, if our necessary conditions for optimality can be stated without any ref-erences to u-derivatives, it is reasonable to guess that the u-derivatives of L are notneeded. Also, the minimization that occurs in (2.10) makes sense over any subsetU of R

n, so there is no longer any reason to insist that the range of values of u bethe whole space. This leads us to

CONJECTURE M2: (NCO) should still be a necessary condition for optimalityeven for problems where the derivative x is restricted to belong to some subsetU of R

n, and L(x, u, t) is not required to be differentiable or even continuouswith respect to u. ♦

Naturally, once u is restricted to a subset U of Rn, and L(x, u, t) is not required

to be differentiable with respect to u, the maximization condition (2.10) no longerimplies (1.7). So we see that (2.10) is more fundamental than (1.7).


Notice also that there has been a subtle change in the formulation of the neces-sary conditions for optimality. In the classical conditions of the calculus of variations(including Weierstrass’ own version of the side condition), the covector π(t)—the“momentum”—is defined by (1.7), and the necessary conditions are equations hap-pening along the trajectory ξ∗. These equations can be formulated somewhat moresimply using π(t), but π(t) can also be completely eliminated, since one can alwayssubstitute for it its value given by (1.7). The new version of the necessary condi-tions for an optimum given by Conjecture M2 is of a different form: rather thanstate that certain equations hold along ξ∗, the conditions are now an existentialstatement, asserting the existence of a vector-valued function t �→ π(t) with certainproperties. The elimination of π(t), by expressing π(t) in terms of ξ∗(t), ξ∗(t) andt, is only possible when U = R

n and L is differentiable with respect to u, in whichcase (1.7) follows from (2.10). Otherwise, π(t) is a “new,” “independent” object.

Since Weierstrass was working under assumptions that imply (1.7), he missedthe chance to see that his condition was really about π(t), not about ∂L

∂u evaluatedalong ξ∗, and that, stated this way, his condition was valid for much more generalproblems, for which π(t) no longer equals ∂L

∂u (ξ∗(t), ξ∗(t), t).

2.4. A toy problem. To test Conjecture M2, let us look at a toy problemwhere the answer is reasonably easy to guess directly, and see whether using (NCO)gives the same result.

Example 2.4.1. Let T be a fixed positive number. Suppose that we want tofind a real-valued (Lipschitz) function t �→ x(t) on the interval [0, T ] that satisfies

x(0) = 1 , x(T ) = 1 , and |x(t)| ≤ 1 for all t ∈ [0, T ] ,(2.11)

and minimizes the integral

I =∫ T

0

x(t)2dt

among all the functions satisfying the constraints (2.11). This looks exactly like acalculus of variations problem of the classical type—with a Lagrangian L given by

L(x, x, t) = x2 ,

except that the derivative x is required to satisfy an “inequality constraint” |x| ≤ 1.If we apply the classical Euler-Lagrange equations formally, we get

∂L

∂x= 0 ,

since L does not depend on x, so ∂L∂x = 0, and then

∂L

∂x=

d

dt

(∂L

∂x

)= 0 .

But∂L

∂x= 2x .

So x(t) ≡ 0, contradicting the endpoint constraints x(0) = 1, x(T ) = 1. So there isno way at all to satisfy Euler-Lagrange together with the boundary conditions.

On the other hand, we can apply (NCO) formally, making the sensible guessthat in this case the maximization with respect to u should be made over the setU = [−1, 1] of permissible values of u. Then (2.8) and (2.9) become the equations

14 H. J. SUSSMANN

x = u, p = 2x. This suggests, among other things, that p as a function of t shouldbe of class C1, since p has to be continuous. In addition, (2.10) says that u = 1 whenp > 0, and u = −1 when p < 0. (When p = 0 all values of u ∈ U are solutions.)So when p > 0 we have p = 2, and when p < 0 we have p = −2. This shows thatp(·) is a convex function of t when p(t) > 0 and a concave function when p(t) < 0.Now, a continuous function on [0, T ] which is convex wherever it is > 0 and concavewherever it is < 0 can only be of one of the 13 types P , N , Z, PZ, PN , ZP , ZN ,NP , NZ, PZP , PZN , NZP or NZN , where “Z” means “identically zero,” “Pmeans “> 0,” “N” means “< 0,” and, for example, a symbol such as PZN means“a positive piece followed by a zero piece and then by a negative piece.” (This iseasy to prove. For example, ZPZ is impossible because the P part would haveto vanish at two points and be convex and > 0 in between, which clearly cannothappen. The reader should analyze all possibilities and verify that only our 13cases can arise.) Clearly, P corresponds to u = 1, and N to u = −1. As for Z, itcorresponds to p(t) ≡ 0, which yields x(t) ≡ 0 (since p = 2x) and then u(t) ≡ 0(since x = u). So Z corresponds to x = u = 0. (Notice that (2.10) does not directlydetermine the value of u when p = 0, since in this case every value of u satisfies(2.10), but by suitably combining the three conditions of (NCO) we have still beenable to find u for p = 0.) With this in mind, it is clear that the only way to satisfythe endpoint conditions x(0) = 1, x(T ) = 1, is with a PN or an NP or an NZPcurve. Of these three possibilities, PN is obviously not optimal, so we are left withNP and NZP , and then a direct analysis shows that the minimum is NP if T ≤ 2and NZP if T > 2. This result can of course be proved directly, and corresponds tothe obvious intuition that to minimize

∫x2 one should try to move from the initial

value x = 1 towards x = 0 as fast as possible, stay at x = 0 as long as as possible,and then move back towards x = 1, choosing the departure time from x = 0 so asto arrive to x = 1 exactly at time T .

So (NCO) gives us the correct solution whereas the Euler-Lagrange equation isnot satisfied by this solution. ♦

THE PONTRYAGINMAXIMUM PRINCIPLE

Hector J. Sussmann — November 29, 2000

CONTENTS

1. CARATHEODORY’S MISSED OPPORTUNITY 1

2. THE MAXIMUM PRINCIPLE 22.1 A classical version for fixed time interval problems 32.2 A classical version for variable time interval problems 52.3 An example 7

The purpose of this note is to present a precise statement of a technicallysimple version of the Pontryagin maximum principle. But before we getthere we will add one more example to our list of “missed opportunities,” byshowing how Caratheodory failed to discover the Hamiltonian maximizationcondition because of his insistence on using the “wrong” Hamiltonian.

1. Caratheodory’s missed opportunity

Constantin Caratheodory (1873-1950) published his work on the calculus ofvariations in a 1935 book, cf. [2]. His discussion of the excess function,and of how Weierstrass’ condition can be expressed in Hamiltonian form, isworth quoting in detail1, because it shows that the author understood theimportance of expressing the Weierstrass condition in Hamiltonian form, butwas unable to find its simple Hamiltonian meaning because he was using aninappropriate Hamiltonian formalism.

Caratheodory uses xi for the position variables (that is, our qi) andyi for the momenta (that is, our pi). He works in the neighborhood of a“regular line element” e = (t, xi, xi), where the “regularity” assumption isthe requirement that the Hessian matrix {Lxi,xj}1≤i,j≤n be positive definiteat e. This condition guarantees that, near e, one can express everything in“canonical coordinates” (t, xi, yi), i.e., that the map (t, x, x) → (t, x, y) isinvertible, if we let yi = χi(t, x, x), where χi(t, x, x) = ∂L

∂xi(t, x, x).

He then expresses the Weierstrass condition, which involves the “lineelement (t, xi, xi)” together with a nearby “line element” (t, xi, x

′i), in terms

of the corresponding canonical coordinates (t, xi, yi) and (t, xi, y′i). He uses

H to represent what he calls “the Hamiltonian” (that is, our classical Hamil-tonian), given by the formula H =

∑ni=1 yixi − L(t, x, x), where xi =

ϕi(t, x, y), and ϕ is the inverse function of χ (that is, ϕ(t, x, χ(t, x, u)) = u

1We follow [2], pages 210-212.

2 H. J. SUSSMANN

and χ(t, x, ϕ(t, x, y)) = y). In addition, he uses H ′ to represent the Hamil-tonian evaluated at the other point (t, x, y′) in phase space, that is, H ′ =∑n

i=1 y′ix′i − L(t, x, x′), where x′

i = ϕi(t, x, y′), y′i = χi(t, x, x′).With these notations, the Weierstrass excess function becomes

E(t, x, x, x′) = L(t, x, x′) − L(t, x, x) − ∂L

∂x(t, x, x) · (x′ − x)

= L(t, x, x′) − L(t, x, x) − y · (x′ − x)

= y · x − L(t, x, x) −(y · x′ − L(t, x, x′)

)= y · x − L(t, x, x) −

(y′ · x′ − L(t, x, x′)

)− (y − y′) · x′

= H − H ′ − (y − y′) · x′ .

Since

∂H ′

∂y′j=

∂

∂y′j

( n∑i=1

y′ix′i − L(t, x, x′)

)

= x′j +

n∑i=1

y′i∂ϕi

∂y′j(t, x, y′) − ∂

∂y′jBig(L(t, x, ϕ(t, x, y′))

)

= x′j +

n∑i=1

y′i∂ϕi

∂y′j(t, x, y′) −

n∑i=1

∂L

∂x′i

(t, x, x′)∂ϕi

∂y′j(t, x, x′)

= x′j +

n∑i=1

y′i∂ϕi

∂y′j(t, x, y′) −

n∑i=1

y′iϕi

∂y′j(t, x, x′)

= x′j ,

Caratheodory concluded that “the equation

E(t, x, x, x′) = L′ − L − Lxj(x′j − xj)

= H − H ′ − H ′y′j

(yj − y′j)

therefore holds, by which the E function is represented in the desired form.”

2. The maximum principle

The first rigorous statement and proof of the maximum principle appears inthe book [6]. This “classical” version was then improved by other authors,e.g. L. Cesari [3], M. Hestenes [4], E. B. Lee and L. Markus [5], and L. D.Berkovitz [1].

Here we shall present a couple of simple versions of the result, undertechnical assumptions that will be considerably weakened later.

THE MAXIMUM PRINCIPLE 3

2.1. A classical version for fixed time interval problems. Webegin by quoting a relatively simple version.

We assume that the following conditions are satisfied:

C1. n, m are nonnegative integers;C2. Q is an open subset of R

n;C3. U is a closed subset of R

m;C4. a, b are real numbers such that a ≤ b;C5. Q×U×[a, b] � (x, u, t) �→ f(x, u, t) = (f1(x, u, t), . . . , fn(x, u, t) ∈ R

n

and Q × U × [a, b] � (x, u, t) �→ L(x, u, t) ∈ R are continuous maps;C6. for each (u, t) ∈ U × [a, b] the maps

Q � x �→ f(x, u, t) =(f1(x, u, t), . . . , fn(x, u, t)

)∈ R

n

andQ � x �→ L(x, u, t) ∈ R

are continuously differentiable, and their partial derivatives with re-spect to the x coordinates are continuous functions of (x, u, t);

C7. x, x are given points of Q;C8. TCP[a,b](Q, U, f) (the set of all “trajectory-control pairs defined on

[a, b] for the data Q, U, f”) is the set of all pairs (ξ, η) such that:a. [a, b] � t �→ η(t) ∈ U is a measurable bounded map,b. [a, b] � t �→ ξ(t) ∈ Q is an absolutely continuous map,c. ξ(t) = f(ξ(t), η(t), t) for almost every t ∈ [a, b];.

C9. TCP[a,b](Q, U, f) � (ξ, η) �→ J(ξ, η) ∈ R is the functional given by

J(ξ, η) def=∫ b

a

L(ξ(t), η(t), t) dt ;

C10. γ∗ = (ξ∗, η∗) (the “reference TCP”) is such thata. γ∗ ∈ TCP[a,b](Q, U, f),b. ξ∗(a) = x and ξ∗(b) = x,c. J(ξ∗, η∗) ≤ J(ξ, η) for all (ξ, η) ∈ TCP[a,b](Q, U, f) such that

ξ(a) = x and ξ(b) = x.

Theorem 2.1.1. Assume that the data n, m, Q, U , a, b, f , L, x, xsatisfy conditions C1-C7, TCP[a,b](Q, U, f) and J are defined by C8-C9,and γ∗ = (ξ∗, η∗) satisfies C10. Define the Hamiltonian H to be the function

Q × U × Rn × R × [a, b] � (x, u, p, p0, t) �→ H(x, u, p, p0, t) ∈ R

given by

H(x, u, p, p0, t)def= 〈p, f(x, u, t)〉− p0L(x, u, t) .

Then there exists a pair (π, π0) such that

E1. [a, b] � t �→ π(t) ∈ Rn is an absolutely continuous map;

E2. π0 ∈ R and π0 ≥ 0;E3. (π(t), π0) = (0, 0) for every t ∈ [a, b];

4 H. J. SUSSMANN

E4. the “adjoint equation” holds, that is,

π(t) = −∂H

∂x(ξ∗(t), η∗(t), π(t), π0, t)

for almost every t ∈ [a, b];E5. the “Hamiltonian maximization condition” holds, that is,

H(ξ∗(t), η∗(t), π(t), π0, t) = max{

H(ξ∗(t), u, π(t), π0, t) : u ∈ U}

for almost every t ∈ [a, b].

Remark 2.1.2. It is clear that∂H

∂p(x, u, p, p0, t) = f(x, u, t) .

Therefore the adjoint equation, together with the equation of C8-c, say thatthe pair (ξ∗, π) is a solution of Hamilton’s equations

ξ∗(t) =∂H

∂p(ξ∗(t), η∗(t), π(t), π0, t) ,(2.1)

π(t) = −∂H

∂x(ξ∗(t), η∗(t), π(t), π0, t) .(2.2) ♦

Remark 2.1.3. The map π is often referred to in the control theoryliterature as the “adjoint vector.” Its proper status from the differential-geometric point of view is that of a field of covectors along ξ∗. ♦

Remark 2.1.4. The number π0 is the “abnormal multiplier.” ♦Remark 2.1.5. A trajectory-control pair for which there exists a pair

(π, π0) that satisfies E1-E5 is called an extremal. Therefore the MaximumPrinciple says that a necessary condition for a TCP to be a solution ofthe problem of minimizing the functional J(ξ, η) subject to the conditions(ξ, η) ∈ TCP[a,b](Q, U, f) and the endpoint conditions C10-b is that (ξ, η)be an extremal. ♦

Remark 2.1.6. An extremal is normal if the pair (π, π0) can be chosenso that π0 = 0, and abnormal if the pair (π, π0) can be chosen so that π0 = 0,The pair (π, π0) need not be unique, so in particular it can happen that anextremal is both normal and abnormal. ♦

Remark 2.1.7. Property E3 is referred to as the “nontriviality condi-tion.” ♦

Remark 2.1.8. The adjoint equation says that

π(t) = −π(t) · ∂f

∂x(ξ∗(t), η∗(t), t) + π0

∂L

∂x(ξ∗(t), η∗(t), t) .

In particular, if (π(t), π0) = (0, 0) for one value τ of t, then π0 = 0, so theequation reduces to

π(t) = −π(t) · ∂f

∂x(ξ∗(t), η∗(t), t) ,


which is a linear homogeneous time-varying ordinary differential equationfor π. Therefore the fact that π(τ) = 0 implies that π(t) = 0 for all t, so(π(t), π0) = (0, 0) for all t. Hence the nontriviality condition could equallywell have been stated by saying “for some t” rather than “for all t.” ♦

Remark 2.1.9. If we let

h(t) = H(ξ∗(t), η∗(t), π(t), π0, t) ,

then, formally, the derivative of h is given by

h(t) =∂H

∂x(Γ(t)) · ξ∗(t)+

∂H

∂u(Γ(t)) · η∗(t)+

∂H

∂p(Γ(t)) · π(t)+

∂H

∂t(Γ(t))

=∂H

∂u(Γ(t)) · η∗(t) +

∂H

∂t(Γ(t)) ,

where we have written Γ(t) = (ξ∗(t), η∗(t), π(t), π0, t) and used the fact thatξ∗ = ∂H

∂p and π = −∂H∂x by Hamilton’s equations. If we disregard the fact that

η∗(t) need not exist (because η∗ is not assumed to be diferentiable or evencontinuous), and in addition we pretend that “∂H

∂u (ξ∗(t), η∗(t), π(t), π0, t)must vanish because the function u �→ H(ξ∗(t), u, π(t), π0, t) has a maxi-mum at u = η∗(t),” then we end up with

h(t) =∂H

∂t(ξ∗(t), η∗(t), π(t), π0, t) .

This leads to the conjecture—that we are very far from having proved—thatif the dynamical law f and the Lagrangian L do not depend on t then thefunction [a, b] � t �→ H(ξ∗(t), η∗(t), π(t), π0, t) is constant. We will see laterthat this can be made precise and proved rigorously.

2.2. A classical version for variable time interval problems. Weassume that the following conditions are satisfied:



m;C4. the maps Q × U � (x, u) �→ f(x, u) = (f1(x, u), . . . , fn(x, u)) ∈ R

n

and Q × U � (x, u) �→ L(x, u) ∈ R are continuous;C5. for each u ∈ U the maps

Q � x �→ f(x, u) =(f1(x, u), . . . , fn(x, u)

)∈ R

n

and Q � x �→ L(x, u) ∈ R

are continuously differentiable, and their partial derivatives with re-spect to the x coordinates are continuous functions of (x, u);

C6. x, x are given points of Q;C7. TCP (Q, U, f) (the set of all “trajectory-control pairs for the data

Q, U, f”) is the set given by

TCP (Q, U, f) def=⋃{

TCP[a,b](Q, U, f) : a, b ∈ R, a ≤ b}

,

6 H. J. SUSSMANN

where, for a, b ∈ R such that a ≤ b, TCP[a,b](Q, U, f) is the set of allpairs (ξ, η) such that:

a. [a, b] � t �→ η(t) ∈ U is a measurable bounded map;b. [a, b] � t �→ ξ(t) ∈ Q is an absolutely continuous map;c. ξ(t) = f(ξ(t), η(t)) for almost every t ∈ [a, b].

C8. TCP (Q, U, f) � (ξ, η) �→ J(ξ, η) ∈ R is the functional given by

J(ξ, η)def=∫ b

aL(ξ(t), η(t)) dt for (ξ, η)∈TCP[a,b](Q, U, f), a, b∈R, a ≤ b.

C9. γ∗ = (ξ∗, η∗) (the “reference TCP”) are a∗, b∗ are such thata. a∗ ∈ R, b∗ ∈ R, and γ∗ ∈ TCP[a∗ ,b∗](Q, U, f);b. ξ∗(a∗) = x and ξ∗(b∗) = x;c. J(ξ∗, η∗) ≤ J(ξ, η) for all a, b ∈ R such that a ≤ b and all

(ξ, η) ∈ TCP[a,b](Q, U, f) such that ξ(a) = x and ξ(b) = x.

Theorem 2.2.1. Assume that the data n, m, Q, U , f , L, x, x satisfyconditions C1-C6, TCP (Q, U, f) and J are defined by C7-C8, and a∗, b∗,γ∗ = (ξ∗, η∗) satisfy C9. Define the Hamiltonian H to be the function

Q × U × Rn × R � (x, u, p, p0) �→ H(x, u, p, p0) ∈ R

given by

H(x, u, p, p0)def= 〈p, f(x, u)〉 − p0L(x, u) .


E1. [a∗, b∗] � t �→ π(t) ∈ Rn is an absolutely continuous map;

E2. π0 ∈ R and π0 ≥ 0;E3. (π(t), π0) = (0, 0) for every t ∈ [a∗, b∗];E4. the “adjoint equation” holds, that is,

π(t) = −∂H

∂x(ξ∗(t), η∗(t), π(t), π0)

for almost every t ∈ [a∗, b∗];E5. the “Hamiltonian maximization condition with zero value” holds, that

is,

0 = H(ξ∗(t), η∗(t), π(t), π0) = max{H(ξ∗(t), u, π(t), π0) : u ∈ U

}for almost every t ∈ [a, b].

Remark 2.2.2. If we fix a∗ and b∗, and consider only TCPs (ξ, η) be-longing to TCP[a∗,b∗](Q, U, f), then γ∗ is obviously a solution of the mini-mization problem considered in Theorem 2.1.1. So all the conditions givenby that theorem should be satisfied here. And, indeed, this is what hap-pens, because the only difference between the conclusion of Theorem 2.1.1and that of Theorem 2.2.1 is that in Theorem 2.2.1 we obtain the additionalcondition that the value of the Hamiltonian must vanish. ♦


Remark 2.2.3. We already pointed out in Remark 2.1.9 that, at leastformally, the conclusions of Theorem 2.1.1 already imply that the Hamilton-ian function t �→ H(ξ∗(t), η∗(t), π(t), π0) is constant, since in our problem fand L do not depend on t. So the only novelty of Theorem 2.2.1 is that theconstant now turns out to be equal to zero. ♦

2.3. An example. We consider the “one-dimensional soft landing”problem in which it is desired to find, for a given initial point (α, β) ∈ R

2,a trajectory-control pair (ξ, η) of the system

x = y , y = u , −1 ≤ u ≤ 1 ,

such that ξ goes from (α, β) to (0, 0) in minimum time.This is, of course, a problem of the kind discussed in Theorem 2.2.1, so

we shall solve it by applying the theorem.In this case, the configuration space Q is R

2, the control set U is thecompact interval [−1, 1], the dynamical law f is given by

f(x, y, u) = (y, u) ,

and the Lagrangian is identically equal to 1.The Hamiltonian H is given by

H(x, y, u, px, py, p0) = pxy + pyu − p0 ,

where we are using px, py to denote the two components of the momentumvariable p.

Assume that (ξ∗, η∗) is a solution of our minimum time problem, andthat (ξ∗, η∗) ∈ TCP[a∗ ,b∗](Q, U, f).

Then Theorem 2.2.1 tells us that there exists a pair (π, π0) satisfying allthe conditions of the conclusion. Write π(t) = (πx(t), πy(t)). The adjointequation then implies

πx(t) = 0 ,

πy(t) = −πx(t) .

Therefore the function πx is constant. (Notice that we are using the factthat πx is absolutely continuous!) Let A ∈ R be such that πx(t) = A for allt ∈ [a∗, b∗]. Then there must exist a constant B such that

πy(t) = B − At for t ∈ [a∗, b∗] .

Now, if A and B were both equal to zero, the function πy would vanishidentically and then the Hamiltonian maximization condition would say thatthe function

[−1, 1] � u �→ 0is maximized by taking u = η∗(t), a fact that would give us no informationwhatsoever about η∗.

Fortunately, the conditions of Theorem 2.2.1 imply that A and B cannotboth vanish. To see this, observe that if A = B = 0 then it follows that

8 H. J. SUSSMANN

πx(t) ≡ πy(t) ≡ 0. But then the nontriviality condition tells us that π0 = 0.So the value H(ξ∗(t), η∗(t), π(t), π0) would be equal to −π0, which is notequal to zero. This contradicts the fact that, for our time-varying problem,the Hamiltonian is supposed to vanish.

Now that we know that A and B cannot both vanish, there are twopossibilities. Either A = 0 or A = 0.

Suppose first that A = 0. Then B = 0, and Hamiltonian maximizationtells us that [−1, 1] � u �→ Bu is maximized by u = η∗(t). If B > 0, thisimplies that η∗(t) = 1 for all t. If B < 0 then it follows that η∗(t) = −1 forall t.

Now assume that A = 0. Then t �→ B−Atdef= ϕ(t) is a nonconstant linear

function of t. Therefore ϕ vanishes at most once on the interval [a∗, b∗]. Ifϕ never vanishes on [a∗, b∗], or vanishes at one of the endpoints, then η∗(t)is either always equal to 1 (if ϕ(t) > 0) or always equal to −1 (if ϕ(t) < 0).If ϕ(τ) = 0 for some τ ∈ ]a∗, b∗[, then η∗(t) will be equal to 1 for t < τ andto −1 for t > τ (if ϕ changes sign at τ from positive to negative) or η∗(t)will be equal to −1 for t < τ and to 1 for t > τ (if ϕ changes sign at τ fromnegative to positive).

So we have proved that η∗ is of one of the following four types:

a. constantly equal to 1,b. constantly equal to −1,c. constantly equal to −1 for t < τ and to 1 for t > τ for some τ ∈ ]a∗, b∗[,d. constantly equal to 1 for t < τ and to −1 for t > τ for some τ ∈ ]a∗, b∗[.

For a problem where the control set U is a compact convex subset of Rm, a

control η∗ that takes values in the set of extreme points of U is said to be abang-bang control. In our case, we have proved that all optimal controls arebang-bang and either constant of piecewise constant with one switching.

Notice that the switchings of the optimal control η∗ are determined bythe function ϕ, which in this example happens to be πy. That is why thisfunction is called the switching function for this problem.

Notice also that we have made essential use of all the conditions givenby Theorem 2.2.1. In particular, the nontriviality condition was crucial, andthe fact that the value of the Hamiltonian is equal to zero was also decisive.

References[1] Berkovitz, L. D., Optimal Control Theory. Springer-Verlag, New York, 1974.

[2] Caratheodory, C., Calculus of Variations and Partial Differential Equations of the FirstOrder. Part II: Calculus of Variations. Eng. transl. by R. B. Dean, 2nd edition, Holden

Day, Inc., San Francisco, 1967. (Originally published as Variationsrechnung und PartielleDifferentialgleichungen erster Ordnung. B. G. Teubner, Berlin, 1935.)

[3] Cesari, L., Optimization—Theory and Applications. Springer-Verlag, New York, 1983.[4] Hestenes, M. R., Calculus of Variations and Optimal Control Theory. Wiley, 1966.

[5] Lee, E. B., and L. Markus, Foundations of Optimal Control Theory. Wiley, New York, 1967.[6] Pontryagin, L. S., V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mischenko, The Mathe-

matical Theory of Optimal Processes. Wiley, New York, 1962.

LIE BRACKETS AND THEMAXIMUM PRINCIPLE

Hector J. Sussmann — December 6, 2000

CONTENTS

1. COORDINATE INVARIANCE, LIE BRACKETS, AND THE MAXIMUMPRINCIPLE 1

1.1 Covariance, invariance, and coordinate democracy 11.2 Differentiable manifolds 21.3 Lie brackets 51.4 Control and Lie brackets 81.5 Using Lie brackets to understand optimal trajectories 91.6 Implementing Lagrange’s covariance idea in optimal control 10

1. COORDINATE INVARIANCE, LIE BRACKETS, AND THEMAXIMUM PRINCIPLE

In this note we show how the concept of a Lie bracket plays an essential role inthe coordinate-free formulation of the Maximum Principle. We will explain howthe important discovery by Lagrange of the invariance of the Euler-Lagrangesystem under arbitrary coordinate changes has an optimal control analgue,and why this analogue must involve Lie brackets.

1.1. Covariance, invariance, and coordinate democracy. While Euler’swork was full of “geometric” arguments, and contained many pictures, Lagrangeused no pictures. (Newton’s 1687 Principia had 250 figures. Lagrange proudlystated in the preface of his 1788 Mecanique Analytique that “no diagrams will befound in this work.”)

Lagrange was very proud of having freed mechanics from geometry by doingeverything analytically. Little did he know that one of his discoveries anticipatedRiemann’s revolutionary creation of differential geometry.

Lagrange allowed the configuration of a collection of N particles to be rep-resented in terms of completely general systems of 3N coordinates. This was adecisive step in the process that would free geometry from the constraint of havingto deal with 3-dimensional Euclidean space. “To appreciate how all was new in thiswork [the Mecanique Analytique], let us point out that, when Lagrange described amotion in terms of a system of Cartesian coordinates in space, this was such a non-trivial step that he devoted two pages to trying to persuade the reader” (Lochak,[6], p. 49).

He then showed that the Euler-Lagrange system has the same form in “curvilin-ear” or “generalized” coordinates, so that the system is invariant under arbitrary,

2 H. J. SUSSMANN

nonlinear coordinate changes. This was the first example of what would later be-come the cornerstone of differential geometry, namely, the principle of “coordinatedemocracy,” according to which all coordinate systems, “rectangular” as well as“curvilinear,” have to be treated equally.

Lagrange saw the significance of this invariance property:

It is perhaps one of the principal advantages of our method that it ex-presses the equations of every problem in the most simple form relativeto each set of variables and that it enables us to see beforehand whichvariables one should use in order to facilitate the integration as much aspossible. [Quoted from [9], p. 33.]

Remark 1.1.1. For a striking illustration of what Lagrange had in mind here,the reader should think of how much simpler it is to derive the equations of motionof a body under an inverse-square attracting force if one first proves that the motionmust take place in a plane, and then does the whole calculation in polar coordinates,using the invariance of the Euler-Lagrange equations. ♦

In 1854, sixty-six years after the publication of Lagrange’s treatise, Riemann gavehis famous lecture, sketching the brilliant ideas that would become the basis ofmodern differential geometry. It turns out that one of these ideas was preciselythat of invariance under general coordinate changes, whose importance Lagrangehad presciently anticipated.

1.2. Differentiable manifolds.

Definition 1.2.1. Let Q be a set and let k be a nonnegative integer. An atlasof class Ck on Q is a set A of coordinate charts on Q such that:

1. Any two charts x, X ∈ A are Ck-related.2.

⋃{Dx : x ∈ A

}= Q.

If n is a nonnegative integer such that all the charts belonging to A are n-dimensionalthen A is said to be an n-dimensional atlas on Q. ♦

Definition 1.2.2. Let k be a nonnegative integer. A differentiable manifold ofclass Ck is a pair Q = (Q0,A) such that Q0 is a set and A is an atlas of class Ck

on Q0.If Q = (Q0,A) is a differentiable manifold of class Ck, we will refer to A as the

structural atlas, or the differentiable structure of Q. The set Q0 is the set of pointsof Q. Often we will use the same notation Q to refer to the set Q0, so we shall talkabout “a point q ∈ Q” rather than “a point q ∈ Q0.” ♦

If k is a nonnegative integer, Q = (Q0,A) is a differentiable manifold of classCk, and q ∈ Q, we use A(q) to denote the set of all the charts x ∈ A such thatq ∈ Dx. It is easy to show that all the charts x ∈ A(q) have the dimension. (Thisfollows from a simple rank argument if k > 0, and from the theorem on invarianceof domains if k = 0.) This common dimension of all the charts x ∈ A(q) is calledthe dimension of Q at q.

A manifold Q such that the dimension of Q at q is the same for all q ∈ Q issaid to be of pure dimension. In that case the integer n such that the dimension

LIE BRACKETS AND THE MAXIMUM PRINCIPLE 3

of Q at q equals n for all q ∈ Q is called the dimension of Q, and Q is said to ben-dimensional.

Remark 1.2.3. If Q = (Q0,A) is a differentiable manifold, then one can definea topology on Q by declaring a subset S of Q to be open if x(S ∩ Dx) is open inRx for every chart x ∈ A.

One can then show easily that every connected component of a manifold is amanifold of pure dimension. ♦

Definition 1.2.4. Let k be a strictly positive integer, and let Q = (Q0,A) bea differentiable manifold of class Ck. Let q ∈ Q and let n be the dimension of Q atq. A tangent vector for Q at q is a map A(q) � x �→ vx ∈ R

n that assigns to eachchart x ∈ A(q) a column vector

vx =

⎡⎢⎢⎢⎣

vx,1

vx,2

...vx,n

⎤⎥⎥⎥⎦ ∈ R

n ,

in such a way that, if x,X are any two charts in A(q), then

vX =∂Φx,X

∂x(x) · vx(1.1)

or, equivalently,

vX,i =n∑

j=1

∂Φx,X,i

∂xj(x)vx,j for i = 1, . . . , n ,(1.2)

where x = x(q) and

Φx,X =

⎡⎢⎢⎢⎣

Φx,X,1

Φx,X,2

...Φx,X,n

⎤⎥⎥⎥⎦ ∈ R

n

is the change of coordinates map from x to X. ♦

Equations (1.1), (1.2) are the transformation formulas for vectors.We use TqQ to denote the set of all tangent vectors for Q at q, and refer to TqQ

as the tangent space of Q at q.To define a tangent vector v ∈ TqQ, it suffices to specify its component repre-

sentation vx relative to some coordinate chart x belonging to the structural atlasof Q, because once this is done the representation vX relative to any other chart Xis determined by (1.1).

Definition 1.2.5. Let k be a strictly positive integer, and let Q = (Q0,A) bea differentiable manifold of class Ck. Let q ∈ Q and let n be the dimension of Qat q. A cotangent vector—or covector—for Q at q is a map A(q) � x �→ wx ∈ R

n

that assigns to each chart x ∈ A(q) a row vector

wx =

⎡⎢⎢⎢⎣

wx1

wx2...

wxn

⎤⎥⎥⎥⎦ ∈ Rn ,

4 H. J. SUSSMANN

in such a way that, if x,X are any two charts in A(q), then

wx = wX · ∂Φx,X

∂x(x)(1.3)

or, equivalently,

wxj =

n∑i=1

wXi

∂Φx,X,i

∂xj(x) for j = 1, . . . , n ,(1.4)

where x = x(q) and

Φx,X =

⎡⎢⎢⎢⎣

Φx,X,1

Φx,X,2

...Φx,X,n

⎤⎥⎥⎥⎦ ∈ R

n

is the change of coordinates map from x to X. ♦

Equations (1.3), (1.4), are the transformation formulas for covectors.We use T ∗

q Q to denote the set of all covectors of Q at q, and refer to T ∗q Q as

the cotangent space of Q at q.To define a tangent vector w ∈ T ∗

q Q, it suffices to specify its component repre-sentation wx relative to some coordinate chart x belonging to the structural atlasof Q, because once this is done the representation wX relative to any other chartX is determined by (1.3).

Proposition 1.2.6. Let k be a strictly positive integer, and let Q = (Q0,A)be a differentiable manifold of class Ck. Let q ∈ Q, and let v ∈ TqQ, w ∈ T ∗

q Q.Define the inner product

w · v def= wx · vx ,

where x is any chart belonging to A(q). Then w · v does not depend on the choiceof x.

Proof. Let x, X be two charts belonging to A(q). Then

wX · vX =

(wx ·

(∂Φx,X

∂x(x)

)−1)

·(

∂Φx,X

∂x(x)vx

)

= wx · vx ,

using (1.1), (1.3), and the fact that the square matrix ∂Φx,X

∂x(x) is invertible. (The

invertibility of ∂Φx,X

∂x (x) follows from (1.1), because (1.1) implies that

vX =∂Φx,X

∂x(x)vxvx

=∂Φx,X

∂x(x)

∂ΦX,x

∂x(x)vX

for every v ∈ TqQ, so

v =∂Φx,X

∂x(x)

∂ΦX,x

∂x(x)v

for every v ∈ Rn. This implies that

∂Φx,X

∂x(x)

∂ΦX,x

∂x(x) = identity matrix ,


and the conclusion follows.)

It follows from Proposition 1.2.6 that the cotangent space T ∗q Q can be regarded

in a natural way as the dual of the tangent space TqQ.

1.3. Lie brackets. What do we need to make the Maximum Principle com-pletely coordinate-free? To answer this question we recall that a control systemis, basically, a family of vector fields, so it is natural to ask oneself what are the“natural” coordinate-free operations among vector fields.

An obvious example of such an operation is addition. It turns out that, if fand g are two vector fields, then there is another natural covariant object derivedfrom them, namely, the Lie bracket [f, g]. This is defined in coordinates by theformula

[f, g](x) =∂g

∂x(x) · f(x) − ∂f

∂x(x) · g(x) .(1.5)

Here we are writing f and g as columns of functions, that is,

f(x) =

⎡⎢⎢⎢⎣

f1(x)f2(x)

...fn(x)

⎤⎥⎥⎥⎦ , g(x) =

⎡⎢⎢⎢⎣

g1(x)g2(x)

...gn(x)

⎤⎥⎥⎥⎦ .(1.6)

In addition, if h is a vector field, written as a column of functions as above, then∂h∂x is the Jacobian matrix of h, which is a square matrix of functions:

∂f

∂x=

⎡⎢⎢⎢⎢⎣

∂f1

∂x1 · · · ∂f1

∂xn

∂f2

∂x1 · · · ∂f2

∂xn

.... . .

...∂fn

∂x1 · · · ∂fn

∂xn

⎤⎥⎥⎥⎥⎦ .(1.7)

It follows that, if f , g are vector fields, and we let h = [f, g], then the componentshi of h are given by

hi =n∑

j=1

∂gi

∂xjfj −

n∑j=1

∂f i

∂xjgj .

Remark 1.3.1. If f, g are vector fields, then [f, g] is a vector field. The precisemeaning of this is that “[f, g] transforms like a vector field under a change ofcoordinates.” To prove this, let us recall the way vector fields transform. Supposeh is a vector field, given with respect to a coordinate chart x : Dx �→ R

n byfunctions hx,1, . . . , hx,n. Suppose X is another chart, and h is given in this newchart by functions hX,1, . . . , hX,n. Suppose

Φx,X : x(Dx ∩ DX) �→ X(Dx ∩ DX)

andΦX,x : X(Dx ∩ DX) �→ x(Dx ∩ DX)

be the coordinate change maps.

6 H. J. SUSSMANN

The fact that h is a vector field says that, at each point q ∈ Dx ∩ DX, if wetake a curve ξ such that ξ(0) = q, whose tangent vector at time 0 is h(q), and letξx, ξX, be the coordinate representations, so that

ξx(t) = x(ξ(t)) and ξX(t) = X(ξ(t)) ,

then, if we let x = x(q), X = X(q), we have

hX(X) =d

dt

∣∣∣t=0

(ξX(t)

)=

d

dt

∣∣∣t=0

(X(ξ(t))

)=

d

dt

∣∣∣t=0

(Φx,X

(x(ξ(t))

))=

d

dt

∣∣∣t=0

(Φx,X

(ξx(t)

))=

∂Φx,X

∂x(x) · hx(x) ,

so that the components hX,i, hx,j are related by

hX,i(X) =n∑

j=1

∂Φi

∂xj(x)hx,j(x)

=n∑

j=1

∂Φi

∂xj(Ψ(X))hx,j (Ψ(X))

for i = 1, . . . , n, if we write

Φ = Φx,X , Ψ = ΦX,x .

Now suppose f , g are two vector fields, and q is a point belonging to Dx ∩ DX.Then

fX,i(X) =n∑

j=1

∂Φi

∂xj(Ψ(X))fx,j (Ψ(X)) =

n∑j=1

(∂Φi

∂xjfx,j

)(Ψ(X)) ,

so

∂fX,i

∂Xk(X) =

n∑j=1

n∑�=1

(∂2Φi

∂xj∂x�fx,j +

∂Φi

∂xj

∂fx,j

∂x�

)(Ψ(X)) · ∂Ψ�

∂xk(X) .

Similarly,

∂gX,i

∂Xk(X) =

n∑j=1

n∑�=1

(∂2Φi

∂xj∂x�gx,j +

∂Φi

∂xj

∂gx,j

∂x�

)(Ψ(X)) · ∂Ψ�

∂xk(X) .


Thereforen∑

k=1

∂gX,i

∂XkfX,k(X)

=n∑

k=1

n∑j=1

n∑�=1

(∂2Φi

∂xj∂x�gx,j +

∂Φi

∂xj

∂gx,j

∂x�

)(Ψ(X)) · ∂Ψ�

∂xk(X) · fX,k(X)

=n∑

j=1

n∑�=1

(∂2Φi

∂xj∂x�gx,j +

∂Φi

∂xj

∂gx,j

∂x�

)(Ψ(X)) ·

(n∑

k=1

∂Ψ�

∂xk(X) · fX,k(X)

)

=n∑

j=1

n∑�=1

(∂2Φi

∂xj∂x�gx,j +

∂Φi

∂xj

∂gx,j

∂x�

)(x) · fx,�(x)

=n∑

j=1

n∑�=1

(∂2Φi

∂xj∂x�gx,jfx,�

)(x) +

n∑j=1

n∑�=1

(∂Φi

∂xj

∂gx,j

∂x�fx,�

)(x) .

Similarly,n∑

k=1

∂fX,i

∂XkgX,k(X)

=n∑

j=1

n∑�=1

(∂2Φi

∂xj∂x�fx,jgx,�

)(x) +

n∑j=1

n∑�=1

(∂Φi

∂xj

∂fx,j

∂x�gx,�

)(x)

=n∑

j=1

n∑�=1

(∂2Φi

∂x�∂xjfx,�gx,j

)(x) +

n∑j=1

n∑�=1

(∂Φi

∂x�

∂fx,�

∂x�gx,j

)(x)

=n∑

j=1

n∑�=1

(∂2Φi

∂xj∂x�fx,�gx,j

)(x) +

n∑j=1

n∑�=1

(∂Φi

∂x�

∂fx,�

∂x�gx,j

)(x) ,

if we assume that x and X are C2-related, so that

∂2Φi

∂x�∂xj=

∂2Φi

∂xj∂x�.

Then

hX,i(X) =n∑

k=1

∂gX,i

∂XkfX,k(X)

n∑k=1

−∂fX,i

∂XkgX,k(X)

=n∑

j=1

n∑�=1

(∂Φi

∂xj

∂gx,j

∂x�fx,�

)(x) −

n∑j=1

n∑�=1

(∂Φi

∂x�

∂fx,�

∂x�gx,j

)(x)

=n∑

j=1

∂Φi

∂xj(x)

(n∑

�=1

∂gx,j

∂x�(x)fx,�(x) − ∂fx,�

∂x�(x)gx,j(x)

)(x)

=n∑

j=1

∂Φi

∂xj(x)hx,j(x) ,

which is the desired transformation formula. ♦

The fact that [f, g] is a vector field suggests that it should have an intrinsic,coordinate-free characterization. In other words: rather than define what [f, g]

8 H. J. SUSSMANN

means using coordinates, and then prove that when we change coordi-nates [f, g] transforms as it should, it ought to be possible to say directlywhat [f, g] without invoking coordinates at all.

This is indeed true. The precise coordinate-free characterization of theLie bracket is:

[f, g](q) = limt→0

δ(t) − q

t2,(1.8)

where

δ(t) = qetfetge−tfe−tg .(1.9)

(Here t �→ qeth denotes, for a vector field h, the integral curve of h that goes throughq when t = 0.)

1.4. Control and Lie brackets. Lie brackets play a crucial role in optimalcontrol theory in many ways and, more generally, they are the key objects in thetheory of nonlinear finite-dimensional control. They play a central role in nonlinearcontrollability, feedback linearization, and system equivalence.

Just to give a simple illustration having nothing to with optimality, let usconsider a control theory problem totally analogous to that of the characterizationof flatness for a metric. In control theory, “linear control systems”—i.e., systemsof the form q = Aq + Bu, evolving in R

n, with u taking values in Rm, and A, B

matrices of the appropriate sizes—play a crucial role, comparable in importanceto that of the flat metrics within the class of general Riemannian metrics. So itis important to know when a system that looks nonlinear because it is presentedin “curvilinear coordinates,” is in fact linear, in the sense that one can make itlinear by changing coordinates. The answer turns out to be a criterion involvingLie brackets.

Precisely, suppose we are studying control systems

q = f(q) + u1g1(q) + . . . + umgm(q) , q ∈ Rn ,

with C∞ vector fields f , g1, . . . , gm, and a point q in Rn.

Is there a way to know if such a system is “a linear system in disguise,” thatis, a system obtained from a linear system by changing coordinates?

For example, look at the two systems

x1 = x2 + 2x2x3 ,

x2 = x3 − x1x2 + x32 ,(1.10)

x3 = u + x22 + x1x3 − x2

1x2 + 2x1x32 ,

x1 = u1 ,

x2 = u2 ,(1.11)x3 = u1x2 − u2x1 .

Which one is “more nonlinear”? We will see later that (1.10), which superficiallylooks very nonlinear, is in fact a linear system in disguise, whereas (1.11), whosenonlinearity appears to be much milder, is in fact highly nonlinear. And the answerwill be obtained by studying the Lie bracket structures associated to both systems.It will turn out that a necessary condition for the system to be equivalent near q to


a linear system is that all the iterated brackets of members of the set {f, g1, . . . , gm}involving two gi’s should vanish near q. Under an extra technical condition (“strongaccessibility,” that is, the requirement that the Lie brackets adk

f (gi)(q), k ≥ 0,i = 1, . . . , m span the whole space) this condition is sufficient as well.

So Lie brackets help us organize our knowledge of nonlinear systems in terms oftheir deep structural properties. This way of classifying systems is as superior to thenaıve approach based on a superficial counting of the number of nonlinearities asthe animal taxonomy of modern zoology is to the classifications found in medievalbestiaries. Exactly as a dolphin is structurally closer to a zebra than to a shark—in spite of the very visible ways, such as shape and habitat, in which a dolphinresembles a shark more than a zebra—System (1.10) is much closer to a linearsystem than System (1.11). Lie brackets enable us to discover this, and then onecan find abundant confirmation by looking at the properties of both systems. (Forexample, (1.10) is globally stabilizable by a smooth state feedback, but (1.11) isnot even locally stabilizable near any point.)

1.5. Using Lie brackets to understand optimal trajectories. For opti-mal control problems, the effort to understand the properties of optimal trajectoriesin terms of reasonable invariants has begun only recently, and most of the work re-mains to be done.

Just to give the flavor of some of these results, we concentrate on one problemthat has attracted a lot of attention in recent years, namely, that of minimumtime control. Suppose, for example, that we are looking at minimum time controlof systems in R

n of the form x = f(x) + ug(x), with |u| ≤ 1, where f and gare smooth vector fields. Here the control space U is the interval [−1, 1]. TheHamiltonian is H = ϕ + uψ − p0, where ϕ = 〈p, f(x)〉 and ψ = 〈p, g(x)〉. Then uwill equal 1 when ψ > 0, and −1 when ψ < 0. A naıve genericity argument maysuggest the wrong conclusion that ψ = 0 is a “rare” event, which will happen atmost at isolated points, so that in fact the optimal trajectories will be “bang-bang,”i.e., such that u is piecewise constant with values 1 and −1 and finitely many jumps.This, however, is quite wrong.

Example 1.5.1. Let Q = R2, and assume that q=(x, y)∈Q evolves according

to x = 1−y2, y = u, |u| ≤ 1, so f , g are the vector fields (x, y) �→ (1 − y2, 0),(x, y) �→ (0, 1), respectively. (Equivalently, f = (1−y2) ∂

∂x and g = ∂∂y .) Suppose

we want a trajectory that goes from (0, 0) to (1, 0) in minimum time. It is obviousthat the solution is given by u(t) ≡ 0, y(t) ≡ 0, x(t) ≡ t, 0 ≤ t ≤ 1, and theoptimal time is 1. So the optimal control is “singular”—i.e., takes values in theopen interval ]−1, 1[ —and therefore drastically fails to be bang-bang. And one caneasily verify that this example is “stable under perturbations” in any reasonablesense of the word. So it is not true that optimal controls are always, or evengenerically, bang-bang. ♦

In view of our previous remarks, it is to be expected that the Lie brackets ofthe vector fields f and g will play a crucial role in determining the properties of asolution. Not surprisingly, a close analysis of the problem, applying (MP), showsthat this is indeed so.

10 H. J. SUSSMANN

For example, the “singular” control of Example 1.5.1 is closely related to aproperty of the Lie brackets of the vector fields f and g. Precisely, the singulartrajectory exists and is contained in the x axis because [f, g] vanishes on the x axis.

Since the computation of Example 1.5.1, as presented, does not explicitly dis-play any Lie brackets, we will now make the Lie brackets stand out by looking,more generally, at systems of the form x = f(x) + ug(x), with |u| ≤ 1, withoutspecializing to a particular pair of vector fields f , g.

If one computes the derivative of ψ along an extremal γ, the result is ρ evaluatedalong γ, where ρ = 〈p, [f, g]〉. So, if n happens to be 2, and g and [f, g] arelinearly independent at each point, then ψ and ψ cannot vanish simultaneously, forotherwise p would be orthogonal to g(x) and [f, g](x), so p would vanish, whichis a contradiction. (Recall that the Maximum Principle for variable time-intervalproblems has the extra condition that H must vanish. Since H = 〈p, f(x)+u g(x)〉−p0, it is clear that p = 0 implies p0 = 0, contradicting nontriviality.) Now, if afunction and its derivative do not vanish simultaneously, it follows that the zeros ofthe function are isolated. In our case, this implies that the optimal controls mustbe bang-bang. So we have proved a simple theorem on the structure of optimaltrajectories: for a minimum time problem in R

2 arising from a control systemx = f(x) + ug(x), with |u| ≤ 1, if g and [f, g] are linearly independent at eachpoint, then all optimal trajectories are bang-bang. The reader can verify that inExample 1.5.1 the Lie bracket [f, g] vanishes along the x axis, so the trajectory ofthat example is contained in the “singular set” of points where g and [f, g] fail tobe linearly independent, in perfect agreement with our theorem.

1.6. Implementing Lagrange’s covariance idea in optimal control.Our elementary arguments about Lie brackets and minimum time control provide agood illustration of how Lagrange’s principle of “coordinate democracy” applies inthe optimal control setting, and why a truly intrinsic formulation of the maximumprinciple, on manifolds, is needed.

To understand optimal trajectories, one needs to study certain functions, suchas the “switching function” ψ. We can think of ψ as a “generalized component”of the (co)vector p, in the sense that the ordinary components pi of p are theinner products of p with the coordinate vector fields ∂

∂qi , and ψ is the product ofp with the vector field g. The adjoint equation gives us an explicit formula forthe functions pi. A truly “democratic” version of the maximum principle shouldtreat all generalized components equally, that is, should give us in one swoop thederivative of every function 〈p, X(q)〉, for every vector field X.

With such a version one could, in each case, work with the generalized compo-nents of p that are most natural for the problem under study, such as, for example,the “switching function” 〈p, g(q)〉 for a system q = f(q) + ug(q). This would be,we believe, true in spirit to Lagrange’s idea that it is good to have a method that“expresses the equations of every problem in the most simple form rel-ative to each set of variables and ... enables us to see beforehand whichvariables one should use in order to facilitate the integration as muchas possible.”

An intrinsic version, treating all “generalized components” (sometimes knownas “momentum functions”) equally, can be formulated in a number of ways, e.g.


using Poisson brackets. For a general version of the maximum principle on man-ifolds, using generalized components as above (and also for an alternative versioninvolving connections along curves), we refer the reader to [8]. The crucial point ofthe version given in [8] is—considering, for simplicity, the case of the minimum timeproblem for a control system of the form q = f(q, u)—that the adjoint equation isrephrased as the following statement (the “intrinsic adjoint equation”):

(IAE) for every smooth vector field X, the derivative with respect to t of thefunction

t �→ 〈π(t), X(ξ∗(t))〉is the function

t �→ 〈π(t), [fη∗(t), X](ξ∗(t))〉where we define

fu(q) = f(q, u) .

The point of using (IAE) is that it treats the inner product of with X—i.e., “theX component of π(t)”—equally for all vector fields X, thus enabling us to followLagrange’s prescription (cf. Page 2) that one should express “the equations of everyproblem in the most simple form relative to each set of variables and ... enables us tosee beforehand which variables one should use in order to facilitate the integrationas much as possible,” by working in each case with the functions of the form 〈p, X〉that are most suited to the problem.

1.7. Why the language of Lie brackets is natural for optimal control.The preceding discussion, based on rather trivial examples, is meant to illustratewhat we have in mind when we talk about “understanding the qualitative propertiesof trajectories” and why we say that the appropriate language for discussing thisquestion and formulating conditions is that of Lie brackets of vector fields.

The reader should think of the following analogy. When one tries to solve asimple 2 × 2 system of linear equations such as 2x + 3y = 5, 6x − 2y = 4, onemay fail to notice that the number 2.(−2) − 3.6 is important. If, however, one“does it with letters” (as a Calculus student would say), i.e., sets out to solve ageneral system a11x + a12y = b1, a21x + a22y = b2, then one discovers immediatelythat the number a11a22 − a12a21 determines the properties of the solutions, so itbecomes natural to gives this number a name such as “determinant,” and to look fora generalization of the concept to higher dimensions. Similarly, when one appliesthe maximum principle to study a particular problem such as the one of Example1.5.1, one is in fact computing a Lie bracket, although this fact may go unnoticed.But when one “does it with letters,” e.g. by working with a general problem ofthe form x = f(x) + ug(x), leaving f and g unspecified, then the Lie bracket [f, g]stands out immediately as the object that “determines” what happens, exactly asthe expression a11a22 − a12a21 did in the linear algebra example.

It is a general fact that, when we apply the maximum principle to minimumtime problems, we get conclusions of the form “if such and such thing is true of theLie brackets of the vector fields involved, then the optimal trajectories have suchand such properties.” This can be ascertained in all kinds of examples, providedthat one makes sure that the computations are always done “with letters.” But theoccurrence of the brackets is due to a profound a priori reason, namely, Statement(INV) of the previous section.

12 H. J. SUSSMANN

In recent years, ever since the advent of “differential geometric control theory,”much has been done to determine, in a systematic way, how structural propertiesof a system, embodied in their Lie bracket relations, relate to properties of theoptimal trajectories. For example, it has been known for a long time that for linearminimum time optimal control the “bang-bang property” holds. From a generalnonlinear perspective, one can show that the bang-bang property holds when certainLie brackets Bi can be expressed as linear combinations of some other brackets.Linear systems just happen to be those for which the Bi vanish. This providesan explanation of the linear bang-bang theorem from the nonlinear perspective, aswell as a generalization.

References

[1] Berkovitz, L. D., Optimal Control Theory. Springer-Verlag, New York, 1974.

[2] Caratheodory, C., Calculus of Variations and Partial Differential Equations of the FirstOrder. Part II: Calculus of Variations. Eng. transl. by R. B. Dean, 2nd edition, Holden

Day, Inc., San Francisco, 1967. (Originally published as Variationsrechnung und Partielle

Differentialgleichungen erster Ordnung. B. G. Teubner, Berlin, 1935.)[3] Cesari, L., Optimization—Theory and Applications. Springer-Verlag, New York, 1983.

[4] Hestenes, M. R., Calculus of Variations and Optimal Control Theory. Wiley, 1966.[5] Lee, E. B., and L. Markus, Foundations of Optimal Control Theory. Wiley, New York, 1967.

[6] Lochak, G., La Geometrisation de la Physique. Flammarion, Paris, 1994.[7] Pontryagin, L. S., V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mischenko, The Mathe-

matical Theory of Optimal Processes. Wiley, New York, 1962.[8] Sussmann, H. J., An introduction to the coordinate-free maximum principle. In Geometry of

Feedback and Optimal Control, B. Jakubczyk and W. Respondek Eds., Marcel Dekker, NewYork, 1997, pp. 463–557.

[9] Yourgrau, W., and S. Mandelstam, Variational Principles in Dynamics and Quantum Theory.W. B. Saunders & Co., Philadelphia, 1968.

A CAR �following E� Nelson� ��

x

y

θφ

X � udDRIV E�X � usSTEER�X ,

X � �x, y, φ, θ† ,

DRIV E � �cos�θ � φ, sin�θ � φ, sin�θ, † ,

STEER � � , , ,�† .

��

STEER� θ � �

DRIVE� x � cos�θ � φ� y � sin�θ � φ�φ � sin�θ� θ � �

�STEER, DRIV E� � WRIGGLE

�WRIGGLE, DRIV E� � SLIDE

SLIDE � �− sinφ, cosφ, , † ,

WRIGGLE � SLIDE�ROTATE �when θ � ,

ROTATE � � , ,�, .

• The e�ect of WRIGGLE is attained by doing�

STEER� DRIVE� �STEER� �DRIVE

• The e�ect of SLIDE is attained by doing�

WRIGGLE� DRIVE� �WRIGGLE� �DRIVE

��

STEER

DRIVE

ROTATE

SLIDE

��

[ ]= WRIGGLESTEER DRIVE,

STEER

DRIVE

STEER

DRIVE

WRIGGLE

WRIGGLE

=SLIDE+ROTATEon θ=0

��

[ ]=WRIGGLE DRIVE,

DRIVE

DRIVE

WRIGGLE

SLIDE

WRIGGLE

��

TRANSVERSALITY,

SET SEPARATION,

AND THE PROOF OF

THE MAXIMUM PRINCIPLE

Hector J. Sussmann — January 3, 2001

CONTENTS

1. THE TRANSVERSALITY CONDITIONS 21.1 Cones 21.2 Boltyanskii tangent cones to a set at a point 21.3 The maximum principle with transversality conditions for fixed time

interval problems 31.4 The maximum principle with transversality conditions for variable

time interval problems 41.5 An example 51.6 A remark on more general endpoint conditions 7

2. THE SEPARATION THEOREM 72.1 The maximum principle as a set separation condition for fixed time

interval problems 72.2 The maximum principle as a set separation condition for variable

time interval problems 9

3. THE PROOF OF THE MAXIMUM PRINCIPLE 103.1 The variational equation 113.2 Packets of needle variations 113.3 The endpoint map 123.4 The main technical lemma 12

2 H. J. SUSSMANN

1. THE TRANSVERSALITY CONDITIONS

We now move one step further, and present two even more general versions ofthe Pontryagin maximum principle. The new versions are still rather simpleas far as technical hypotheses go, but contain a new ingredient, namely,the “transversality condition.” This shows up as an additional necessarycondition for optimality for a problem in which the terminal constraint,instead of being of the form “ξ(b) = x” considered so far, is of the moregeneral form

ξ(b) ∈ S ,

where S is some given subset of Q.

1.1. Cones. If X is a real linear space, a cone in X is a subset C of Xsuch that

1. C �= ∅,2. C is a union of rays (that is, whenever r ∈ R, r ≥ 0, and c ∈ C, it

follows that r · c ∈ C).

If C is a cone in X then necessarily 0 ∈ C. In general, a cone need not beconvex. A cone C is convex if and only if it is closed under addition, thatis, c1 + c2 ∈ C whenever c1, c2 ∈ C.

If X is endowed with a topology, then we can talk about closed cones.We will only be interested in cones in finite-dimensional real linear spacesX . In that case, X has a canonical topology, so the concept of a closed coneis well defined.

The (algebraic) polar of a cone C in a real linear space X is the set

C⊥ def= {w ∈ X† : 〈w, c〉 ≤ 0 for all c ∈ C} ,

where X† denotes the (algebraic) dual of X , that is, the set of all linearfunctionals w : X → R. (When X is a topological linear space one wants toconsider the topological dual, consisting of the continuous linear functionals,and the corresponding topological polar of a cone. But for us, since we areonly interested in the finite-dimensional case, algebraic duals and polars aresufficient.)

It is easy to see that the polar of any cone C in X is a convex cone.Moreover, if X is finite-dimensional and C is a cone in X , then C⊥ is closedin X†, and C⊥⊥—-which is a subset of X††, a space which is canonicallyidentified with X—is the closed convex hull of C. In particular, C⊥⊥ = Cif and only if C is closed and convex.

1.2. Boltyanskii tangent cones to a set at a point. Let S be asubset of a finite-dimensional real linear space X , and let x be a point of S.A Boltyanskii tangent cone (or “approximating cone”) to S at x is a closed


convex cone C in X such that there exist a neighborhood U of 0 in X anda continuous map ϕ : U ∩ C → S having the property that

ϕ(v) = x + v + o(‖v‖) as v → 0, v ∈ C .

1.3. The maximum principle with transversality conditions forfixed time interval problems. We assume that the following conditionsare satisfied:



m;C4. a, b are real numbers such that a ≤ b;C5. Q×U×[a, b] (x, u, t) → f(x, u, t) = (f1(x, u, t), . . . , fn(x, u, t))∈R

n

and Q × U × [a, b] (x, u, t) → L(x, u, t) ∈ R are continuous maps;C6. for each (u, t) ∈ U × [a, b] the maps

Q x → f(x, u, t) =(f1(x, u, t), . . . , fn(x, u, t)

)∈ R

n

andQ x → L(x, u, t) ∈ R

are continuously differentiable, and their partial derivatives with re-spect to the x coordinates are continuous functions of (x, u, t);

C7. x is a given point of Q, and S is a subset of Q;C8. TCP[a,b](Q, U, f) (the set of all “trajectory-control pairs defined on

[a, b] for the data Q, U, f”) is the set of all pairs (ξ, η) such that:a. [a, b] t → η(t) ∈ U is a measurable bounded map,b. [a, b] t → ξ(t) ∈ Q is an absolutely continuous map,c. ξ(t) = f(ξ(t), η(t), t) for almost every t ∈ [a, b];.

C9. TCP[a,b](Q, U, f) (ξ, η) → J(ξ, η) ∈ R is the functional given by

J(ξ, η) def=∫ b

a

L(ξ(t), η(t), t) dt ;

C10. γ∗ = (ξ∗, η∗) (the “reference TCP”) is such thata. γ∗ ∈ TCP[a,b](Q, U, f),b. ξ∗(a) = x and ξ∗(b) ∈ S,c. J(ξ∗, η∗) ≤ J(ξ, η) for all (ξ, η) ∈ TCP[a,b](Q, U, f) such that

ξ(a) = x and ξ(b) ∈ S,C11. C is a Boltyanskii tangent cone to S at the point x = ξ∗(b).

Theorem 1.3.1. Assume that the data n, m, Q, U , a, b, f , L, x, S,satisfy conditions C1-C7, TCP[a,b](Q, U, f) and J are defined by C8-C9,γ∗ = (ξ∗, η∗) satisfies C10, x is defined by C11 and C satisfies C11. Definethe Hamiltonian H to be the function

Q × U × Rn × R × [a, b] (x, u, p, p0, t) → H(x, u, p, p0, t) ∈ R

4 H. J. SUSSMANN

given by

H(x, u, p, p0, t)def= 〈p, f(x, u, t)〉− p0L(x, u, t) .


E1. [a, b] t → π(t) ∈ Rn is an absolutely continuous map;

E2. π0 ∈ R and π0 ≥ 0;E3. (π(t), π0) �= (0, 0) for every t ∈ [a, b];E4. the “adjoint equation” holds, that is,

π(t) = −∂H

∂x(ξ∗(t), η∗(t), π(t), π0, t)

for almost every t ∈ [a, b];E5. the “Hamiltonian maximization condition” holds, that is,

H(ξ∗(t), η∗(t), π(t), π0, t) = max{

H(ξ∗(t), u, π(t), π0, t) : u ∈ U}

for almost every t ∈ [a, b],E6. the “transversality condition” holds, that is,

−π(b) ∈ C⊥ .

1.4. The maximum principle with transversality conditions forvariable time interval problems. We assume that the following condi-tions are satisfied:



m;C4. the maps Q × U (x, u) → f(x, u) = (f1(x, u), . . . , fn(x, u)) ∈ R

n

and Q × U (x, u) → L(x, u) ∈ R are continuous;C5. for each u ∈ U the maps

Q x → f(x, u) =(f1(x, u), . . . , fn(x, u)

)∈ R

n

and Q x → L(x, u) ∈ R

are continuously differentiable, and their partial derivatives with re-spect to the x coordinates are continuous functions of (x, u);

C6. x is a given point and S is a given subset of Q;C7. TCP (Q, U, f) (the set of all “trajectory-control pairs for the data



TCP[a,b](Q, U, f) : a, b ∈ R, a ≤ b}

,


a. [a, b] t → η(t) ∈ U is a measurable bounded map;b. [a, b] t → ξ(t) ∈ Q is an absolutely continuous map;c. ξ(t) = f(ξ(t), η(t)) for almost every t ∈ [a, b].


C8. TCP (Q, U, f) (ξ, η) → J(ξ, η) ∈ R is the functional given by

J(ξ, η)def=∫ b

aL(ξ(t), η(t)) dt for (ξ, η)∈TCP[a,b](Q, U, f), a, b∈R, a ≤ b.

C9. γ∗ = (ξ∗, η∗) (the “reference TCP”) and a∗, b∗ are such thata. a∗ ∈ R, b∗ ∈ R, and γ∗ ∈ TCP[a∗ ,b∗](Q, U, f);b. ξ∗(a∗) = x and ξ∗(b∗) ∈ S;c. J(ξ∗, η∗) ≤ J(ξ, η) for all a, b ∈ R such that a ≤ b and all

(ξ, η) ∈ TCP[a,b](Q, U, f) such that ξ(a) = x and ξ(b) ∈ S.C10. C is a Boltyanskii tangent cone to S at the point x = ξ∗(b∗).

Theorem 1.4.1. Assume that the data n, m, Q, U , f , L, x, S satisfyconditions C1-C6, TCP (Q, U, f) and J are defined by C7-C8, and a∗, b∗,γ∗ = (ξ∗, η∗), x, C, satisfy C9. Define the Hamiltonian H to be the function

Q × U × Rn × R (x, u, p, p0) → H(x, u, p, p0) ∈ R

given by

H(x, u, p, p0)def= 〈p, f(x, u)〉 − p0L(x, u) .


E1. [a∗, b∗] t → π(t) ∈ Rn is an absolutely continuous map;

E2. π0 ∈ R and π0 ≥ 0;E3. (π(t), π0) �= (0, 0) for every t ∈ [a∗, b∗];E4. the “adjoint equation” holds, that is,

π(t) = −∂H

∂x(ξ∗(t), η∗(t), π(t), π0)


is,

0 = H(ξ∗(t), η∗(t), π(t), π0) = max{H(ξ∗(t), u, π(t), π0) : u ∈ U

}for almost every t ∈ [a, b],

E6. the “transversality condition” holds, that is,

−π(b∗) ∈ C⊥ .

1.5. An example. We consider the “one-dimensional landing prob-lem” in which it is desired to find, for a given initial point (α, β) ∈ R

2, atrajectory-control pair (ξ, η) of the system

x = y , y = u , −1 ≤ u ≤ 1 ,

such that ξ goes from (α, β) to some point in the y axis in minimum time.This is exactly the same as the “soft landing problem” discussed before,except that now we are not asking the landing to be soft, that is, our terminalcondition is that the terminal position x be zero, but we are not imposingany condition on the terminal velocity.

6 H. J. SUSSMANN

The configuration space Q for our problem is R2, the control set U is

the compact interval [−1, 1], the dynamical law f is given by

f(x, y, u) =[

yu

],

and the Lagrangian is identically equal to 1. The terminal constraint isξ(b) ∈ S, where

S = {(x, y) ∈ R2 : x = 0} .

The Hamiltonian H is given by

H(x, y, u, px, py, p0) = pxy + pyu − p0 ,

where we are using px, py to denote the two components of the momentumvariable p.

Assume that (ξ∗, η∗) is a solution of our minimum time problem, andthat (ξ∗, η∗) ∈ TCP[a∗ ,b∗](Q, U, f).

Then Theorem 1.4.1 tells us that there exists a pair (π, π0) satisfying allthe conditions of the conclusion. Write π(t) = (πx(t), πy(t)). The adjointequation then implies

πx(t) = 0 ,

πy(t) = −πx(t) .

Therefore the function πx is constant. Let A ∈ R be such that πx(t) = Afor all t ∈ [a∗, b∗]. Then there must exist a constant B such that

πy(t) = B − At for t ∈ [a∗, b∗] .

Now, if A and B were both equal to zero, the function πy would vanishidentically and then the Hamiltonian maximization condition would say thatthe function

[−1, 1] u → 0is maximized by taking u = η∗(t), a fact that would give us no informationwhatsoever about η∗.

Fortunately, the conditions of Theorem 1.4.1 imply that A and B cannotboth vanish. To see this, observe that if A = B = 0 then it follows thatπx(t) ≡ πy(t) ≡ 0. But then the nontriviality condition tells us that π0 �= 0.So the value H(ξ∗(t), η∗(t), π(t), π0) would be equal to −π0, which is notequal to zero. This contradicts the fact that, for our time-varying problem,the Hamiltonian is supposed to vanish.

It is clear that the set S itself is a Boltyanskii tangent cone to S atthe terminal point ξ∗(b∗). Therefore the transversality condition says thatπ(b∗) ∈ S⊥, i.e., that

πy(b∗) = 0 .

So R t → πy(t) = B −At is a linear function which is not identically zero(because A and B do not both vanish) but vanishes at the endpoint b∗ ofthe interval [a∗, b∗]. Therefore πy(t) never vanishes on [a∗, b∗[. It follows


that the optimal control η∗ is either constantly equal to 1 or constantly equalto −1. We have thus proved that all optimal controls are bang-bang andconstant.

1.6. A remark on more general endpoint conditions. Supposewe wanted to consider endpoint conditions of the form

(ξ(a), ξ(b)) ∈ S ,

where S is a subset of Q×Q. (For example, we could require that ξ(a) ∈ S1

and ξ(b) ∈ S2, in which case we would take S = S1×S2. Or we could require“periodic endpoint conditions” ξ(a) = ξ(b), in which case we would take Sto be the diagonal of Q, that is, the set {(x, x) : x ∈ Q}.)

These problems can easily be reduced to the ones considered in §1.3 and§1.4, by means of the following trick: work with a new system in Q × Q,with state evolution equations x = f(x, u, t), x′ = 0. (This has the effect ofkeeping track of the initial state so that it is still there as part of the stateat any later time.) One needs an aditional transformation because now wedo not want to have to initialize the state at a prespecified point (x, x′) ofQ×Q. For this reason, we add an extra piece of dynamics prior to time a∗,by means of the dynamical law x = v, x′ = v, on the time interval [a∗−1, a∗],where v is a new control with values in R

n. This has the effect of allowingus to fix a point (x, x′) of Q × Q to be the initial condition at time a∗ − 1,while letting the state at time a∗ be of the form (x, x) with x arbitrary.

2. THE SEPARATION THEOREM

We now proceed to “geometrize” the problem by completely eliminating theoptimization aspect, and working instead with a question about separationof two sets.

Precisely, let us say that two subsets S1, S2 of a set A are separated ata point x ∈ S1 ∩ S2 if

S1 ∩ S2 = {x} ,

that is, if S1 ∩ S2 contains no points other than x.If A is a topological space, we say that S1 and S2 are locally separated

at x if there exists a neighborhood U of x such that

U ∩ S1 ∩ S2 = {x} .

2.1. The maximum principle as a set separation condition forfixed time interval problems. We assume that the following conditionsare satisfied:



m;C4. a, b are real numbers such that a ≤ b;

8 H. J. SUSSMANN

C5. Q×U×[a, b] (x, u, t) → f(x, u, t) = (f1(x, u, t), . . . , fn(x, u, t))∈Rn

is a continuous map;C6. for each (u, t) ∈ U × [a, b] the map

Q x → f(x, u, t) =(f1(x, u, t), . . . , fn(x, u, t)

)∈ R

n

is continuously differentiable, and its partial derivatives with respectto the x coordinates are continuous functions of (x, u, t);

C7. x is a given point of Q, and S is a subset of Q;C8. TCP[a,b](Q, U, f) (the set of all “trajectory-control pairs defined on

[a, b] for the data Q, U, f”) is the set of all pairs (ξ, η) such that:a. [a, b] t → η(t) ∈ U is a measurable bounded map,b. [a, b] t → ξ(t) ∈ Q is an absolutely continuous map,c. ξ(t) = f(ξ(t), η(t), t) for almost every t ∈ [a, b];.

C9. R[a,b](Q, U, f ; x) (the “reachable set from x for the data Q, U, f overthe interval [a, b]”) is defined by

R[a,b](Q, U, f ; x) def={y : (∃(ξ, η) ∈ TCP[a,b](Q, U, f))(ξ(a) = x ∧ ξ(b) = y)

}.

C10. γ∗ = (ξ∗, η∗) (the “reference TCP”) is such thata. γ∗ ∈ TCP[a,b](Q, U, f),b. ξ∗(a) = x and ξ∗(b) ∈ S,c. R[a,b](Q, U, f ; x) and S are blocally separated at the point x =

ξ∗(b);C11. C is a Boltyanskii tangent cone to S at x.

Theorem 2.1.1. Assume that the data n, m, Q, U , a, b, f , x, S, satisfyconditions C1-C7, TCP[a,b](Q, U, f) and R[a,b](Q, U, f ; x) are defined by C8-C9, γ∗ = (ξ∗, η∗) satisfies C10, x is defined by C10, and C satisfies C11.

Assume, moreover, that C is not a linear subspace of Rn.

Define the Hamiltonian H to be the function

Q × U × Rn × [a, b] (x, u, p, t) → H(x, u, p, t) ∈ R

given by

H(x, u, p, t) def= 〈p, f(x, u, t)〉 .

Then there exists a π such that

E1. [a, b] t → π(t) ∈ Rn is an absolutely continuous map;

E2. π(t) �= 0 for every t ∈ [a, b];E3. the “adjoint equation” holds, that is,

π(t) = −∂H

∂x(ξ∗(t), η∗(t), π(t), t)

for almost every t ∈ [a, b];


E4. the “Hamiltonian maximization condition” holds, that is,

H(ξ∗(t), η∗(t), π(t), t) = max{

H(ξ∗(t), u, π(t), t) : u ∈ U}

for almost every t ∈ [a, b],E5 the “transversality condition” holds, that is,

−π(b) ∈ C⊥ .

2.2. The maximum principle as a set separation condition forvariable time interval problems. We assume that the following condi-tions are satisfied:



m;C4. the map Q × U (x, u) → f(x, u) = (f1(x, u), . . . , fn(x, u)) ∈ R

n iscontinuous;

C5. for each u ∈ U the map

Q x → f(x, u) =(f1(x, u), . . . , fn(x, u)

)∈ R

n

is continuously differentiable, and its partial derivatives with respectto the x coordinates are continuous functions of (x, u);

C6. x is a given point and S is a given subset of Q;C7. TCP (Q, U, f) (the set of all “trajectory-control pairs for the data



TCP[a,b](Q, U, f) : a, b ∈ R, a ≤ b}

,


a. [a, b] t → η(t) ∈ U is a measurable bounded map;b. [a, b] t → ξ(t) ∈ Q is an absolutely continuous map;c. ξ(t) = f(ξ(t), η(t)) for almost every t ∈ [a, b].

C8. R(Q, U, f ;x) (the “reachable set from x for the data Q, U, f”) is de-fined by

R(Q, U, f ;x) def=⋃

−∞<a≤b<+∞R[a,b](Q, U, f ; x)

where

R[a,b](Q, U, f ; x) def={y : (∃(ξ, η) ∈ TCP[a,b](Q, U, f))(ξ(a) = x ∧ ξ(b) = y)

}.

C9. γ∗ = (ξ∗, η∗) (the “reference TCP”) and a∗, b∗ are such thata. a∗ ∈ R, b∗ ∈ R, and γ∗ ∈ TCP[a∗ ,b∗](Q, U, f);b. ξ∗(a∗) = x and ξ∗(b∗) ∈ S;c. R(Q, U, f ; x) and S are locally separated at the point x = ξ∗(b∗).

10 H. J. SUSSMANN

C10. C is a Boltyanskii tangent cone to S at x.

Theorem 2.2.1. Assume that the data n, m, Q, U , f , x, S satisfyconditions C1-C6, TCP (Q, U, f) and R(Q, U, f ; x) are defined by C7-C8,and a∗, b∗, γ∗ = (ξ∗, η∗), x, C, satisfy C9 and C10.

Assume, moreover, that C is not a linear subspace of Rn.

Define the Hamiltonian H to be the function

Q × U × Rn (x, u, p) → H(x, u, p) ∈ R

given by

H(x, u, p) def= 〈p, f(x, u)〉 .

Then there exists a π such that

E1. [a∗, b∗] t → π(t) ∈ Rn is an absolutely continuous map;

E2. π(t) �= 0 for every t ∈ [a∗, b∗];E3. the “adjoint equation” holds, that is,

π(t) = −∂H

∂x(ξ∗(t), η∗(t), π(t))


is,

0 = H(ξ∗(t), η∗(t), π(t)) = max{H(ξ∗(t), u, π(t)) : u ∈ U

}for almost every t ∈ [a, b],

E5. the “transversality condition” holds, that is,

−π(b∗) ∈ C⊥ .

3. THE PROOF OF THE MAXIMUM PRINCIPLE

We now present a proof of Theorem 2.1.1. The basic strategy is to make“needle variations,” combine these variations into “packets of needle vari-ations,” “propagate” the effects of these variations to the terminal pointof the reference trajectory, and construct an “approximating cone” to thereachable set. Then a topological argument will be used to derive, from theseparation assumption for the reachable set and the set, the existence ofa hyperplane separating the corresponding tangent cones. By propagatingbackwards via the adjoint equation the linear functional that defines thishyperplane, we will get the desired momentum vector π.

In order to avoid technical complications, we will first do the proof underthe following additional simplifying assumption:

(SA) the reference control η∗ is continuous.


Assumption (SA) is not needed, but the proof without it is a little bit moretechnical, so we will first assume (SA), and then explain how to get rid ofthis condition.

So from now on we assume that the data n, m, Q, U , a, b, f , x, S,satisfy conditions C1-C7 of §2.1, TCP[a,b](Q, U, f) and R[a,b](Q, U, f ; x) aredefined by C8-C9 of §2.1, γ∗ = (ξ∗, η∗) satisfies C10 of §2.1, x is defined byC10 of §2.1, C satisfies C11 of §2.1, and (SA) holds.

3.1. The variational equation. Define

A(t) =∂f

∂x(ξ∗(t), η∗(t), t) .

Then, under our hypotheses—including the additional condition (SA)—themap [a, b] t → A(t) ∈ R

n×n is continuous.The linear time-varying ordinary differential equation

v(t) = A(t) · v(t)(3.1)

is the variational equation along the reference TCP (ξ∗, η∗).We use M(t, s) to denote the fundamental matrix solution of (3.1). That

is, for each s ∈ [a, b] the map

[a, b] t → M(t, s) ∈ Rn×n

is the solution of the matrix-valued initial value problem{X(t) = A(t) ·X(t) ,X(s) = 111n ,

(3.2)

where 111n is the identity matrix of Rn.

It then follows that the map

[a, b]× [a, b] (t, s) → M(t, s) ∈ Rn×n

is continuous and satisfies the integral equation

M(t, s) = 111n +∫ t

sA(r) · M(r, s) dr for t, s ∈ [a, b] .

3.2. Packets of needle variations. Fix

NV1. a positive integer kNV2. a k-tuple τ = (τ1, . . . , τk) (the “variation times”) such that

a ≤ τ1 < τ2 < · · · < τk < b ,

NV3. a k-tuple u = (u1, . . . , uk) (the “variation controls”) of members ofthe control set U .

Use MB([a, b], U) to denote the set of all controls defined on the interval[a, b]. (That is, the members of MB([a, b], U) are the bounded measurablemaps η : [a, b] → U .)

12 H. J. SUSSMANN

For a positive real number α, define

Rk+(α) def= {(x1, . . . , xk) ∈ R

k : 0 ≤ xj ≤ α for j = 1, . . . , k} .

Letε = min(τ2 − τ1, τ3 − τ2, . . . , τk − τk−1, b− τk) .

We then define the packet of needle variations of the reference control η∗associated to τ, u to be the map

Rk+(ε) ε = (ε1, . . . , εk) → η�τ,�u,�ε

∗ ∈ MB([a, b], U)

that associates to every ε = (ε1, . . . , εk) ∈ Rk+(ε) a control η�τ,�u,�ε

∗ defined asfollows:

η�τ,�u,�ε∗ (t) =

{η∗(t) if t ∈ [a, b]\

⋃kj=1[τj, τj + εj] ,

uj if t ∈ [τj, τj + εj ], j ∈ {1, . . . , k} .

3.3. The endpoint map. Given k, τ, u as above, we let ξ�τ ,�u,�ε∗ be, for

ε ∈ Rk+(ε), the unique solution t → ξ(t) of the equation

ξ(t) = f(ξ(t), η�τ,�u,�ε∗ (t), t)

such that ξ(a) = x.Uniqueness follows because the map

Q × [a, b] (x, t) → f(x, η�τ,�u,�ε∗ (t), t)

is Lipschitz with respect to x with a bounded Lipschitz constant as long as(x, t) belongs to a compact subset of Q × [a, b].

In principle, ξ�τ,�u,�ε∗ need not exist on the whole interval [a, b]. The end-

point mapε → E�τ,�u(ε)

associated to τ, u is the map that assigns to each ε ∈ Rk+(ε) the terminal

point

E�τ,�u(ε) def= ξ�τ,�u,�ε∗ (b)(3.3)

of the trajectory ξ�τ ,�u,�ε∗ . Precisely, the domain D�τ ,�u of E�τ,�u is the set of those

ε ∈ Rk+(ε) such that ξ

�τ ,�u,�ε∗ exists on the whole interval [a, b] and, for ε ∈ D�τ ,�u,

E�τ,�u(ε) is defined by (3.3).

3.4. The main technical lemma.

Lemma 3.4.1. Given k, τ , u as above, write

xj = ξ∗(τj) , u∗,j = η∗(τj) .

Then there exists an ε such that:

1. 0 < ε ≤ ε,2. R

k+(ε) ⊆ D�τ ,�u,

3. the endpoint map E�τ,�u is continuous on Rk+(ε),


4. E�τ,�u is differentiable at 0 ∈ Rk, and the differential DE�τ,�u(0) of E�τ ,�u

at 0 is the linear map

Rk α = (α1, . . . , αk) → Λ(α) ∈ R

n

given by

Λ(α) def=k∑

j=1

αj · M(b, τj) ·(f(xj, uj, τj) − f(xj, u∗,j, τj)

).

Conclusion 4 says that

E�τ ,�u(ε) = ξ∗(b) +k∑

j=1

εj · M(b, τj) ·(f(xj, uj, τj) − f(xj, u∗,j, τj)

)+ o(‖ε‖)

as ε → 0 via values in Rk+.

Proof. We fix a positive number δ such that the set

K = {(x, t) ∈ Rn × [a, b] : ‖x − ξ∗(t)‖ ≤ δ}

is entirely contained in Q × [a, b].We let U0 be the union of the sets {η∗(t) : a ≤ t ≤ b} and {u1, . . . , uk}.

Then U0 is compact. We define K = {(x, u, t) : (x, t) ∈ K ∧ u ∈ U0}, so Kis a compact subset of Q × U × [a, b].

Since f and ∂f∂x are continuous on Q×U × [a, b], we can pick a constant

κ such that

‖f(x, u, t)‖+∥∥∥∂f

∂x(x, u, t)

∥∥∥ ≤ κ whenever (x, u, t) ∈ Q × U × [a, b] .

For ε ∈ Rk+(ε), let β(ε) be the maximum of all real numbers β such that

(i) a ≤ β ≤ b, (ii) ξ�τ,�u,�ε∗ exists on [a, β], (iii) (ξ�τ,�u,�ε

∗ (t), t) ∈ K whenevert ∈ [a, β]. (It is easy to prove that such a maximum exists.)

Fix ε = (ε1, . . . , εk) ∈ Rk+(ε). If t ∈ [a, β(ε)], then

ξ�τ,�u,�ε∗ (t)− ξ∗(t) =

∫ t

a

(f(ξ�τ,�u,�ε

∗ (s), η�τ,�u,�ε∗ (s), s)− f(ξ∗(s), η∗(s), s)

)ds

=∫ t

a


∗ (s), η∗(s), s)− f(ξ∗(s), η∗(s), s))

ds

+∫

[a,t]∩E(�ε)


∗ (s), η�τ,�u,�ε∗ (s), s)− f(ξ�τ,�u,�ε

∗ (s), η∗(s), s))

ds ,

where

E(ε) =k⋃

j=1

[τj, τj + εj ] .

14 H. J. SUSSMANN

Then

‖ξ�τ,�u,�ε∗ (t) − ξ∗(t)‖ ≤

∫ t

a

∥∥∥f(ξ�τ,�u,�ε∗ (s), η∗(s), s)− f(ξ∗(s), η∗(s), s)

∥∥∥ds

+∫

[a,t]∩E(�ε)

∥∥∥f(ξ�τ ,�u,�ε∗ (s), η�τ,�u,�ε

∗ (s), s)− f(ξ�τ ,�u,�ε∗ (s), η∗(s), s)

∥∥∥ds

≤ κ

∫ t

a

∥∥∥ξ�τ ,�u,�ε∗ (s) − ξ∗(s)

∥∥∥ ds + 2κ‖ε‖ ,

where‖ε‖ = ε1 + . . . + εk .

Gronwall’s inequality then implies

‖ξ�τ,�u,�ε∗ (t) − ξ∗(t)‖ ≤ 2κ‖ε‖eκ(b−a) .(3.4)

Choose ε such that

0 < ε ≤ ε(3.5)

and4kκεeκ(b−a) ≤ δ .

Then

ε ∈ Rk+(ε) =⇒

(‖ξ�τ,�u,�ε

∗ (t) − ξ∗(t)‖ ≤ δ

2whenever t ∈ [a, β(ε)]

).

It then follows thatε ∈ R

k+(ε) =⇒ β(ε) = b ,

so that

Rk+(ε) ⊆ D�τ ,�u(3.6)

and(ε ∈ R

k+(ε) ∧ t ∈ [a, b]

)=⇒

(‖ξ�τ,�u,�ε

∗ (t) − ξ∗(t)‖ ≤ 2κ‖ε‖eκ(b−a) ≤ δ

2

).

A similar Gronwall inequality argument establishes that(ε ∈ R

k+(ε) ∧ ε′ ∈ R

k+(ε) ∧ t ∈ [a, b]

)=⇒

(‖ξ�τ,�u,�ε

∗ (t) − ξ�τ ,�u,�ε′∗ (t)‖ ≤ 2κ

( k∑j=1

|εj − ε′j|)eκ(b−a) ≤ δ

2

).

It follows, in particular, that

(ε ∈ R

k+(ε)∧ε′∈ R

k+(ε)

)⇒ ‖E�τ ,�u(ε)−E�τ,�u(ε′)‖ ≤ 2κ

( k∑j=1

|εj − ε′j |)eκ(b−a) ,

so the endpoint map E�τ ,�u is continuous on Rk+(ε).


Finally, we have to prove that E�τ ,�u is differentiable at 0 and that thedifferential is given by the formula of our statement. For this purpose, wego back to the formula

ξ�τ ,�u,�ε∗ (t) − ξ∗(t) =

∫ t

a

(f(ξ�τ ,�u,�ε

∗ (s), η∗(s), s)− f(ξ∗(s), η∗(s), s))

ds

+∫

[a,t]∩E(�ε)


∗ (s), η�τ,�u,�ε∗ (s), s)− f(ξ�τ,�u,�ε

∗ (s), η∗(s), s))

ds ,

that we rewrite as

ξ�τ ,�u,�ε∗ (t) − ξ∗(t) =

∫ t

aΔ1(s) ds +

∫[a,t]∩E(�ε)

Δ2(s) ds .(3.7)

We observe that

1. the difference

Δ1(s)def= f(ξ�τ,�u,�ε

∗ (s), η∗(s), s)− f(ξ∗(s), η∗(s), s)

is “approximately equal” to ∂f∂x(ξ∗(s), η∗(s), s) · (ξ�τ,�u,�ε

∗ (s)− ξ∗(s)), i.e.,to A(s) · (ξ�τ,�u,�ε

∗ (s) − ξ∗(s)), whenever t ∈ [a, b].2. the difference

Δ2(s)def= f(ξ�τ,�u,�ε

∗ (s), η�τ,�u,�ε∗ (s), s)− f(ξ�τ,�u,�ε

∗ (s), η∗(s), s)

is “approximately equal” to

wjdef= f(xj, uj, τj) − f(xj, u∗,j, τj)

whenever s ∈ [τj, τj + εj],

To make this precise, we let θ : [a, b] → Rn be the function such that

θ(t) ={

0 if t /∈ E(ε) ,f(xj, uj, τj) − f(xj, u∗,j, τj) if t ∈ [τj, τj + εj] ,

and we let μ : [a, b] → Rn be the solution of

μ(t) = A(t) · μ(t) + θ(t) , μ(a) = 0 .

Then

μ(t) =∫ t

a(A(s) · μ(s) + θ(s)) ds .(3.8)

If we subtract (3.8) from (3.7), we get

ξ�τ ,�u,�ε∗ (t) − ξ∗(t) − μ(t)

=∫ t

a

(Δ1(s)− A(s) · μ(s)

)ds +

∫[a,t]∩E(�ε)

(Δ2(s) − θ(s)

)ds

=∫ t

a

(A(s) · (ξ�τ,�u,�ε

∗ (s) − ξ∗(s) − μ(s))

ds + R(t) ,

16 H. J. SUSSMANN

where the remainder R(t) is given by

R(t) def=∫ t

a

(Δ1(s)−A(s)·(ξ�τ,�u,�ε

∗ (s)−ξ∗(s)))

ds+∫

[a,t]∩E(�ε)

(Δ2(s)−θ(s)

)ds .

Then Gronwall’s inequality yields

‖ξ�τ,�u,�ε∗ (t) − ξ∗(t) − μ(t)‖ ≤ eκ(b−a) sup{‖R(s)‖ : a ≤ s ≤ b} .(3.9)

The functions θ and μ depend on ε. The variations of constants formulayields

μ�ε(b) =∫ b

a

M(b, s) · θ�ε(s) ds ,

where we have now made the ε-dependence explicit. So

μ�ε(b) =k∑

j=1

∫ τj+εj

τj

M(b, s) · wj ds(3.10)

=k∑

j=1

εj · M(b, τj) · wj

+k∑

j=1

∫ τj+εj

τj

(M(b, s)− M(b, τj)

)· wj ds

=k∑

j=1

εj · M(b, τj) · wj + o(‖ε‖) .

If we apply (3.9) with t = b, and use (3.10), we get

‖E�τ ,�u(ε) − ξ∗(b)−k∑

j=1

εj · M(b, τj) · wj‖ ≤ eκ(b−a)ω(ε) + o(‖ε‖) ,(3.11)

whereω(ε) def= sup{‖R�ε(s)‖ : a ≤ s ≤ b} .

So our desired conclusion will follow if we prove that

ω(ε) = o(‖ε‖) as ε → 0 .

The proof of this is a fairly routine and somewhat tedious calculation, thatwill be done later.

TRANSVERSALITY,

SET SEPARATION,

AND THE PROOF OF

THE MAXIMUM PRINCIPLE,

PART II

Hector J. Sussmann — January 10, 2001

This is the continuationd of the notes of January 3. In particular, the pageand section numbering continue those of the previous set of notes.

CONTENTS

4. END OF THE PROOF OF THE MAIN TECHNICAL LEMMA 18

5. TRANSVERSALITY OF CONES AND SET SEPARATION 215.1 Transversality and strong transversality of cones 235.2 Transversality vs. strong transversality 245.3 The main topological lemma 255.4 The transversal intersection theorem 25

6. COMPLETION OF THE PROOF 306.1 Application of the transversal intersection theorem 306.2 The compactness argument 316.3 The momentum 326.4 Elimination of the continuity assumption on the reference control 33

18 H. J. SUSSMANN

4. END OF THE PROOF OF THE MAIN TECHNICALLEMMA

We have to show that

ω(�ε) = o(‖�ε‖) as �ε → 0 ,(4.1)

whereω(�ε) def= sup{‖R�ε(s)‖ : a ≤ s ≤ b} ,

R�ε(t) def=∫ t

a

(Δ�ε

1(s)− A(s) · (ξ�τ ,�u,�ε∗ (s) − ξ∗(s))

)ds

+∫

[a,t]∩E(�ε)

(Δ�ε

2(s)− θ�ε(s))

ds ,

Δ�ε1(t)

def= f(ξ�τ,�u,�ε∗ (t), η∗(t), t)− f(ξ∗(t), η∗(t), t) ,

Δ�ε2(t)

def= f(ξ�τ,�u,�ε∗ (t), η�τ,�u,�ε

∗ (t), t)− f(ξ�τ,�u,�ε∗ (t), η∗(t), t) ,

A(t) =∂f

∂x(ξ∗(t), η∗(t), t) ,

and

θ�ε(t) ={

0 if t /∈ E(�ε) ,f(xj, uj, τj) − f(xj, u∗,j, τj) if t ∈ [τj, τj + εj] .

First, we have

Δ�ε1(t) =

∫ 1

0

∂f

∂x

(ξ∗(t) + ν(ξ�τ ,�u,�ε

∗ (t) − ξ∗(t)), η∗(t), t)·(ξ�τ,�u,�ε∗ (t)− ξ∗(t)

)dν ,

so that

Δ�ε1(t) − A(t) ·

(ξ�τ,�u,�ε∗ (t)− ξ∗(t)

)=

∫ 1

0B�ε(ν, t) ·

(ξ�τ,�u,�ε∗ (t) − ξ∗(t)

)dν ,

where

B�ε(ν, t) =∂f

∂x

(ξ∗(t) + ν(ξ�τ,�u,�ε

∗ (t) − ξ∗(t)), η∗(t), t)− ∂f

∂x

(ξ∗(t), η∗(t), t

).

Since the function

Q × U × [a, b] � (x, u, t) �→ ∂f

∂x(x, u, t)

is continuous, the quantity

λ(r) def=

sup{∥∥∥∂f

∂x(x + v, u, t)− ∂f

∂x(x, u, t)

∥∥∥ : (x, u, t)∈K, (x+ v, u, t)∈K, ‖v‖≤r}

satisfieslimr↓0

λ(r) = 0 .

Using the bound

‖ξ�τ,�u,�ε∗ (t) − ξ∗(t)‖ ≤ 2κ‖�ε‖eκ(b−a)(4.2)


(cf. equation (7.4)) we get

‖B�ε(ν, t)‖ ≤ λ(2κ‖�ε‖eκ(b−a)

)whenever t ∈ [a, b], 0 ≤ ν ≤ 1 .

Then

‖Δ�ε1(t) − A(t) ·

(ξ�τ,�u,�ε∗ (t) − ξ∗(t)

)‖ ≤ λ

(2κ‖�ε‖eκ(b−a)

)· 2κ‖�ε‖eκ(b−a)

whenever t ∈ [a, b]. Therefore∥∥∥ ∫ t

a

(Δ�ε

1(s)− A(s) · (ξ�τ,�u,�ε∗ (s) − ξ∗(s))

)ds

∥∥∥(4.3)

≤ λ(2κ‖�ε‖eκ(b−a)

)· 2(b− a)κ‖�ε‖eκ(b−a)

whenever t ∈ [a, b]. This provides the desired estimate for the first of thetwo terms in the definition of R�ε(t).

To estimate the second term, we fix j, and then t ∈ [τj, τj + εj ], andwrite

Δ�ε2(t) = f(ξ�τ,�u,�ε

∗ (t), uj, t)− f(ξ�τ,�u,�ε∗ (t), η∗(t), t) .

ThenΔ�ε

2(t) − θ�ε(t) = σ�ε1(t) − σ�ε

2(t) ,

where

σ�ε1(t)

def= f(ξ�τ,�u,�ε∗ (t), uj, t)− f(ξ∗(τj), uj, τj) ,

σ�ε2(t)

def= f(ξ�τ,�u,�ε∗ (t), η∗(t), t)− f(ξ∗(τj), η∗(τj), τj) .

Let

λ(r) def= sup{‖F (s1) − F (s2)‖ : s1 ∈ [a, b], s2 ∈ [a, b], |s1 − s2| ≤ r} ,

whereF (s) def= f(ξ∗(s), η∗(s), s) .

Then

limr↓0

λ(r) = 0 ,(4.4)

because F is continuous, since f , ξ∗ and η∗ are continuous.Then

‖f(ξ�τ,�u,�ε∗ (t), η∗(t), t)− f(ξ∗(t), η∗(t), t)‖ ≤ κ‖ξ�τ ,�u,�ε

∗ (t) − ξ∗(t)‖(4.5)

≤ 2κ2‖�ε‖eκ(b−a) ,

and

‖f(ξ∗(t), η∗(t), t)− f(ξ∗(τj), η∗(τj), τj)‖ ≤ λ(‖�ε‖) .(4.6)

Thereforeσ�ε

2(t) ≤ 2κ2‖�ε‖eκ(b−a) + λ(‖�ε‖) .

20 H. J. SUSSMANN

To estimate σ�ε1(t), we define

λ(r) def=sup{‖Fj(s1)− Fj(s2)‖ : s1 ∈ [a, b], s2 ∈ [a, b], |s1 − s2| ≤ r, j = 1, . . . , k} ,

where

Fj(s)def= f(ξ∗(s), uj, s) .

Then

limr↓0

λ(r) = 0 ,

because the Fj are continuous, since f and ξ∗ are continuous.It follows that

‖f(ξ�τ,�u,�ε∗ (t), uj, t)− f(ξ∗(t), uj, t)‖ ≤ κ‖ξ�τ,�u,�ε

∗ (t) − ξ∗(t)‖≤ 2κ2‖�ε‖eκ(b−a) ,

and

‖f(ξ∗(t), uj, t)− f(ξ∗(τj), uj, τj)‖ ≤ λ(‖�ε‖) .

Therefore

σ�ε1(t) ≤ 2κ2‖�ε‖eκ(b−a) + λ(‖�ε‖) .

Hence

‖Δ�ε2(t) − θ�ε(t)‖ ≤ 4κ2‖�ε‖eκ(b−a) + λ(‖�ε‖) + λ(‖�ε‖) .

Then

‖∫

[a,t]∩E(�ε)

(Δ�ε

2(s)− θ�ε(s))

ds‖(4.7)

≤(4κ2‖�ε‖eκ(b−a) + λ(‖�ε‖) + λ(‖�ε‖)

)· ‖�ε‖ .

If we combine (4.3) and (4.7), we get the bound

‖R�ε(t)‖ ≤ μ(�ε) · ‖�ε‖ ,

where

μ(�ε) def=

λ(2κ‖�ε‖eκ(b−a)

)· 2(b− a)κeκ(b−a) + 4κ2‖�ε‖eκ(b−a) + λ(‖�ε‖) + λ(‖�ε‖) .

It follows that

‖ω(�ε)‖ ≤ μ(�ε) · ‖�ε‖ .

Since μ(�ε) → 0 as �ε → 0, (4.1) is proved. ♦


5. TRANSVERSALITY OF CONES AND SET SEPARATION

This section presents in detail the “topological argument” that plays a cru-cial role in the proof of the maximum principle. The purpose of the argumentis to extract information from the facts that

1. the reachable set R[a,b](Q, U, f ; x) and the set S are locally separatedat x,

2. S has a Boltyanskii tangent cone C,3. R[a,b](Q, U, f ; x) contains the image of a neighborhood of 0 relative to

the positive orthant Rk+ of R

k under the endpoint map E�τ ,�u,4. the map E�τ,�u is continuous near 0 and differentiable at 0.

Facts 2, 3 and 4 tell us that we have

a. two subsets S1 and S2 (given by S1 = R[a,b](Q, U, f ; x) and S2 = S)of a finite-dimensional real linear space Y = R

n,b. two convex cones C1 and C2 (given by C1 = R

k+ and C2 = C), in

linear spaces X1, X2 (where X1 = Rk and X2 = R

n),c. neighborhoods U1, U2 of the origin in X1, X2, respectively,d. continuous maps F1 : U1 ∩ C1 �→ S1 and F2 : U2 ∩ C2 �→ S2 (given by

F1 = E�τ ,�u and F2 = ϕ, where ϕ is the approximating map that occursin the definition of a Boltyanskii cone),

e. linear maps L1 : X1 �→ Y , L2 : X2 �→ Y such that F1(x) = L1x +o(‖x‖) as x → 0, x ∈ C1, and F2(x) = L2x+o(‖x‖) as x → 0, x ∈ C2.

Our goal is then to conclude that “the separation of the sets S1 and S2

implies that the linear approximations L1C1, L2C2 to these sets are sepa-rated in the linear sense.” Here “linear separation” of two cones K1, K2 inY means that “there exists a nontrivial linear functional λ : Y �→ R suchthat λ(v) ≥ 0 for v ∈ K1 and λ(v) ≤ 0 for v ∈ K2.”

Unfortunately, separation of two sets does not imply linear separation oftheir approximating cones. This can be see most easily by considering thefollowing trivial example. Let Y = R

2, and take S1, S2 to be the x axis andthe y axis, respectively. Then S1 and S2 are separated at the origin. On theother hand, it is clear that S1 and S2 are their own linear approximationsat 0. Yet S1 and S2 are not linearly separated.

The true corespondence between separation of two sets and linear sepa-ration of their approximating cones is given by a property which is slightlyweaker than linear separation of the approximating cones. Precisely:

i. linear separation of two cones is equivalent to the property that thecones are not “transversal,”

ii. there is another property, called “strong transversality,” which isslightly stronger than transversality,

22 H. J. SUSSMANN

iii. therefore the property of not being strongly transversal is slightlyweaker than non-transversality, i.e., slightly weaker than linear sepa-ration,

iv. strong transversality of the approximating cones implies that the setsare not separated, that is separation of the sets implies that the ap-proximating cones are not strongly transversal.

The notion of “transversality” of cones is a natural extension of the wellknown notion of transversality of linear subspaces. One says that two linearsubspaces A1, A2 of a finite-dimensional real linear space Y are transversal ifthe sum A1+A2 (that is, the set of all sums a1 +a2, a1 ∈ A1, a2 ∈ A2) is thewhole space Y . Naturally, when the Ai are subspaces we could equally wellhave used the difference A1 − A2 (that is, the set of all differences a1 − a2,a1 ∈ A1, a2 ∈ A2). It turns out that, once we use set difference rather thanset sum, the resulting notion of “transversality” is the one that works forcones as well.

The general philosophy of transversality theory is that, if two objects B1

and B2 have linear approximations A1, A2 at a point x, then B1 ∩B2 looks,near x, like A1 ∩ A2, if A1 and A2 are transversal. For example, supposeB1, B2 are smooth submanifolds of R

n of dimensions n1, n2, x ∈ B1 ∩ B2,and A1, A2 are the tangent spaces to B1, B2 at x. Then, if A1 and A2

are transversal, it follows that n1 + n2 ≥ n, and the intersection A1 ∩ A2 isa subspace of dimension ν = n1 + n2 − n. The implicit function theoremthen implies that, near x, B1 ∩ B2 is a ν-dimensional submanifold of R

n.(Transversality is essential here! For example, if we take n = 2, and let B1

be the x axis, and B2 be the parabola {(x, y) : y = x2}, we see that B1

and B2 intersect at the origin, and their tangent spaces A1, A2 at (0, 0) bothcoincide with x-axis. So A1 ∩A2 is one-dimensional, but B1 ∩B2 consists ofa single point, so B1 ∩B2 does not look at all like A1 ∩A2. This shows thatthe principle that “B1 ∩B2 looks near x like A1 ∩A2” can fail if A1 and A2

fail to be transversal.)In our case, the “general philosophy” suggest the following possibility: if

the approximating cones C1 and C2 are transversal, then S1∩S2 will containa nontrivial set if C1∩C2 does. Therefore, to guarantee that S1∩S2 containsa nontrivial set, we have to require

(a) that C1 and C2 be transversal,

and

(b) that C1 ∩ C2 be nontrivial, that is, that C1 ∩ C2 �= {0}.

The conjunction of these two conditions is precisely what we are going tocall “strong transversality.” It will then be a rigorous theorem that

(#a) strong tranversality of the approximating cones implies non-separationof the sets,


that is, that

(#′) separation of the sets implies that the approximating cones are notstrongly transversal.

Using (#′), we will conclude, in the proof of the maximum principle,that the cones DE�τ,�u(0)(Rk

+) and C are not strongly transversal. We willthen want to draw the stronger conclusion that the cones are not transversal.This will follow from the observation (proved in Lemma 5.2.1) that, as longas the cones under consideration are not both linear subspaces, transver-sality of two cones is in fact equivalent to strong transversality. That is,“transversality and strong transversality are essentially the same thing,”except only in the case when both cones are linear subspaces.

5.1. Transversality and strong transversality of cones. If S1, S2

are subsets of a real linear space X , and S3 ⊆ R, we write

S1 + S2def= {s1 + s2 : s1 ∈ S1, s2 ∈ S2} ,

S1 − S2def= {s1 − s2 : s1 ∈ S1, s2 ∈ S2} ,

S3 · S1def= {s3 · s1 : s1 ∈ S1, s3 ∈ S3} .

When one of the sets consists of a single point x, we will write “x” ratherthan “{x}” in the above formulae. Thus, for example, if S ⊆ X and x ∈ X ,then x + S is the translate of S by x, i.e., the set {x + s : s ∈ S}. Similarly,if S ⊆ X and r ∈ R, then r · S is the set {r · s : s ∈ S}. In particular, ifB is the closed unit ball of X centered at 0 (with respect to some norm onX) and x ∈ X , r ∈ R, r > 0, then x + r · B is the closed ball of radius rcentered at x.

Definition 5.1.1. Let C1, C2 be cones in a finite-dimensional real linearspace X . We say that C1 and C2 are transversal—and write C1∩|

−C2—if

C1 − C2 = X . ♦

Remark 5.1.2. If C1 and C2 are convex, then C1∩|−

C2 if and only if theredoes not exist a nonzero linear functional λ : X �→ R such that

λ(c) ≥ 0 whenever c ∈ C1 ,

λ(c) ≤ 0 whenever c ∈ C2 .

Equivalently,

C1∩|−

C2 ⇐⇒ (−C1)⊥ ∩ (C2)⊥ = {0} .(5.1)

To see that (5.1) holds, assume first that C1∩|−

C2. Let λ ∈ (−C1)⊥ ∩ (C2)⊥.Let x ∈ X . Write x = c1 − c2, c1 ∈ C1, c2 ∈ C2. Then λ(c1) ≥ 0,because λ ∈ (−C1)⊥, and then λ(c2) ≤ 0, because λ ∈ (C2)⊥. Thereforeλ(c1 − c2) ≥ 0. So λ(x) ≥ 0. Since this inequality is true for all x ∈ X , wecan take an x ∈ X and apply the inequality to −x, thereby concluding thatλ(−x) ≥ 0, i.e., that λ(x) ≤ 0. So λ(x) = 0 for all x ∈ X , i.e., λ = 0.

24 H. J. SUSSMANN

To prove the converse, we assume that (−C1)⊥∩ (C2)⊥ = {0} and try to

prove that C1∩|−

C2. Suppose it is not true that C1∩|−

C2. Let D = C1 − C2.Then D is a convex cone in X , and D �= X . So D is a proper convex conein X . Let E be the closure of D in X . Then E is a closed convex cone inX , and E �= X . (Here we are using the fact that X is finite-dimensional, toconclude that E �= X from the fact that D �= X . In infinite dimensions thiscan fail, because there are proper convex cones that are dense.) So by theHahn-Banach Theorem there exists a nonzero linear functional λ : X �→ R

such that λ(x) ≥ 0 for all x ∈ E. In particular, if c ∈ C1 then c ∈ E, soλ(c) ≥ 0. Therefore λ ∈ (−C1)⊥. Also, if c ∈ C2 then −c ∈ E, so λ(−c) ≥ 0,and then λ(c) ≤ 0. Therefore λ ∈ (C2)⊥. So λ ∈ (−C1)⊥ ∩ (C2)⊥ = {0}.Since λ �= 0, this contradicts the assumption that (−C1)⊥ ∩ (C2)⊥ = {0}.Therefore C1∩|

−C2. ♦

Definition 5.1.3. Let C1, C2 be cones in a finite-dimensional real lin-ear space X . We say that C1 and C2 are strongly transversal—and writeC1∩||

−C2—if C1∩|

−C2 and C1 ∩ C2 �= {0}. ♦

5.2. Transversality vs. strong transversality.

Lemma 5.2.1. Let C1, C2 be two convex cones in a finite-dimensionalreal linear space X . Then the following two conditions are equivalent:

1. C1∩|−

C2,2. either

a. C1∩||−

C2

orb. C1 and C2 are both linear subspaces and X = C1 ⊕ C2.

Proof. It is clear that 2 =⇒ 1. Let us show that 1 =⇒ 2. Assume thatC1∩|

−C2 but C1 is not strongly transversal to C2. We have to show that

Condition b holds. Clearly, our assumptions imply that C1 ∩ C2 = {0}.First, we show that C2 is a linear subspace. Let c ∈ C2. Since C1∩|

−C2,

we can write c = c1−c2, c1 ∈ C1, c2 ∈ C2. Then c+c2 = c1, so c1 ∈ C1∩C2.Therefore c1 = 0, so c + c2 = 0. It follows that −c = c2, so −c ∈ C2. Wehave thus shown that

(∀c)(c ∈ C2 =⇒ −c ∈ C2) .(5.2)

Since C2 is a convex cone, (5.2) implies that it is a linear subspace.Second, a similar argument shows that C1 is a linear subspace as well.

It then follows that C1 + C2 = X , because C1 − C2 = X , since C1∩|−

C2.Moreover, we know that C1 ∩ C2 = {0}. So the sum is direct, that is,X = C1 ⊕ C2. ♦


5.3. The main topological lemma.

Lemma 5.3.1. Let r ∈ R, r > 0, and let B be the ball of radius r in afinite-dimensional normed space Y . Let ρ be such that 0 < ρ < r, and letF : B → Y be a continuous map such that

‖F (y) − y‖ ≤ ρ whenever y ∈ B .

Then(r − ρ)B ⊆ F (B) .

Proof. Fix z ∈ (r − ρ)B. We want to find y ∈ B such that F (y) = z.Now, the equation F (y) = z is equivalent to y = y − F (y) + z. Let

G(y) = y − F (y) + z ,

for y ∈ B. If y ∈ B, then ‖y − F (y)‖ ≤ ρ. Since ‖z‖ ≤ r − ρ, we canconclude that ‖G(y)‖ ≤ r. So G is a continuous map from B to B. Bythe Brouwer fixed point theorem, G has a fixed point. That is, there existsy ∈ B such that G(y) = y. But then this y satisfies F (y) = z, and our proofis complete.

5.4. The transversal intersection theorem.

Theorem 5.4.1. Assume that the following conditions hold.

1. X1, X2, Y are finite-dimensional real linear spaces.2. C1, C2 are convex cones in X1, X2.3. U1, U2 are neighborhoods of 0 in X1, X2.4. F1 : U1 ∩ C1 �→ Y , F2 : U2 ∩ C2 �→ Y , are continuous maps such that

F1(0) = F2(0) = 0.5. F1 and F2 are differentiable at 0 along C1, C2 with differentials L1,

L2. (That is, for i = 1, 2, Li is a linear map from Xi to Y such thatFi(x) = Lix + o(‖x‖) as x → 0 via values in Ci.)

6. L1(C1)∩||−

L2(C2).

Then the sets F1(U1 ∩ C1) and F2(U2 ∩ C2) are not locally separated at 0.That is, F1(U1 ∩ C1) ∩ F2(U2 ∩ C2) ∩ V �= {0} for every neighborhood V of0 in Y .

Remark 5.4.2. The statement of the theorem involves (in Item 5) normson the spaces X1, X2, Y , but it is easy to see that the validity of the conditionon Item 5 does not depend on the choice of the norms. ♦

Proof. We fix once and for all norms on X1, X2, Y .Let (e1, . . . , en) be a basis of Y . Write e0 = −(e1 + . . .+en). Then every

y ∈ Y can be written uniquely as an affine combination of e0, e1, . . . , en.(Recall that an affine combination of vectors v1, . . . , vm is a linear combina-tion a1v1+. . .+amvm with scalar coefficients aj such that a1+. . .+am = 1. Ify ∈ Y , then we can write y = a1e1+. . .+anen and, using 0 = e0+e1+. . .+en,we can pick an arbitrary scalar r and write y = b0e0 +b1e1 + . . .+bnen, with

26 H. J. SUSSMANN

b0 = r and bj = aj +r for j = 1, . . . , n. Then b0+b1+ . . .+bn = a+(n+1)r,where a = a1 + . . .+am. By choosing r = 1−a

n+1 , we get b0 + b1 + . . .+ bn = 1.This yields an expression of the desired form for y. To prove uniqueness, as-sume y = b0e0+b1e1+. . .+bnen = c0e0+c1e1+. . .+cnen, with b0+b1+. . .+bn = c0+c1+. . .+cn = 1. Then (b0−c0)e0+(b1−c1)e1+. . .+(bn−cn)en = 0,so (b1 − c1 − (b0 − c0))e1 + . . . + (bn − cn − (b0 − c0))en = 0. Since the vec-tors e1, . . . , en are linearly independent, we find b1 − c1 − (b0 − c0) = . . . =bn − cn − (b0 − c0) = 0. Therefore b0 − c0 = b1 − c1 = . . . = bn − cn. But∑n

j=0 bj =∑n

j=0 cj = 1, and then∑n

j=0(bj − cj) = 0. So the n + 1 numbersbj−cj are equal and add up to zero. Therefore bj−cj = 0 for j = 0, 1, . . . , n.)Use aj(y) to denote the coefficients of this affine combination, so

y ∈ Y =⇒(

y =n∑

j=0

aj(y)ej andn∑

j=0

aj(y) = 1)

.

It is easy to see that the functions Y � y �→ aj(y) ∈ R are continuous. Since

0 = e0 + e1 + . . . + en =1

n + 1· e0 +

1n + 1

· e1 + . . . +1

n + 1· en ,

we haveaj(0) =

1n + 1

for j = 0, 1, . . . , n .

Since the functions aj are continuous, we can fix a positive number δ suchthat (

y ∈ Y and ‖y‖ ≤ δ)

=⇒(

aj(y) ≥ 0 for j = 0, 1, . . . , n)

.

Let B be the closed ball {y ∈ Y : ‖y‖ ≤ δ}. Then every member y of B is aconvex combination

∑nj=0 aj(y)ej of e0, e1, . . . , en.

The assumption that the cones L1(C1) and L2(C2) are transversal tellsus that we can find vectors e1,0, e1,1, . . . , e1,n in C1, e2,0, e2,1, . . . , e2,n in C2,such that

ej = L1e1,j − L2e2,j for j = 0, 1, . . . .n .

The assumption that L1(C1) and L2(C2) are strongly transversal impliesthat there exists a nonzero vector y ∈ Y such that y ∈ L1(C1) ∩ L2(C2).Write

y = L1v1 = L2v2 , v1 ∈ C1 , v2 ∈ C2 .

Then, if r ∈ R is arbitrary, we have

ej = L1(e1,j + rv1) − L2(e2,j + rv2) for j = 0, 1, . . . .n ,

because L1v1 − L2v2 = 0.We will choose r in a special way, and then define

e1,j = e1,j + rv1 , e2,j = e2,j + rv2 for j = 0, 1, . . . .n .

The choice of r is made by first fixing a linear functional μ : Y �→ R suchthat μ(y) = 1, and observing that the resulting vectors ei,j will then satisfy

μ(L1e1,j) = μ(L1e1,j) + r , μ(L2e2,j) = μ(L1e2,j) + r .


We choose r such that r > 0 and all the numbers μ(L1e1,j)+r, μ(L1e2,j)+rare ≥ 1.

With this choice of r, the vectors e1,j belong to C1, and the e2,j belongto C2. So we now have

ej = L1e1,j − L2e2,j ,e1,j ∈ C1 ,e2,j ∈ C2 ,

μ(L1e1,j) ≥ 1 ,and μ(L2e2,j) ≥ 1

⎫⎪⎪⎪⎪⎬⎪⎪⎪⎪⎭

for j = 0, 1, . . . .n .

We now define a positive number r and a map

]0, r]× B � (r, y) �→ H(r, y) ∈ Y

by letting

H(r, y) =1r

(F1

(rθ1(y)

)− F2

(rθ2(y)

))

whenever 0 < r ≤ r and y ∈ B. Here

θ1(y) def= a0(y)e1,0 + a1(y)e1,1 + . . . + an(y)e1,n ,

θ2(y) def= a0(y)e2,0 + a1(y)e2,1 + . . . + an(y)e2,n .

The number r is chosen so that

r(‖ei,0‖ + ‖ei,1‖ + . . . + ‖ei,n‖) ≤ δi for i = 1, 2 ,

where the numbers δi are such that

δi > 0 and (∀x)

((x ∈ Xi and ‖x‖ ≤ δi

)=⇒ x ∈ Ui

).

It follows from our choice of r that ‖rθi(y)‖ ≤ δi whenever i = 1, 2, y ∈ B,and 0 < r ≤ r. (Recall that the coefficients aj(y) satisfy 0 ≤ aj(y) ≤ 1whenever y ∈ B.) Therefore H is well defined on the set ]0, r] × B. It isthen clear that H is continuous.

Now use the assumption that

Fi(x) = Lix + o(‖x‖) as x → 0 , x ∈ Ci ,

to writeFi

(rθi(y)

)= Li

(rθi(y)

)+ o(r) as r ↓ 0 ,

since ‖θi(y)‖ is bounded by a fixed constant. Then

Fi

(rθi(y)

)= rLi

(θi(y)

)+ o(r) as r ↓ 0 ,

since Li is linear. So

H(r, y) = L1

(θ1(y)

)− L2

(θ2(y)

)+ o(1) as r ↓ 0 .

28 H. J. SUSSMANN

But

L1

(θ1(y)

)− L2

(θ2(y)

)= L1

( n∑j=0

aj(y)e1,j

)− L2

( n∑j=0

aj(y)e2,j

)

=n∑

j=0

aj(y)(L1e1,j − L2e2,j)

=n∑

j=0

aj(y)ej

= y .

Therefore

H(r, y) = y + o(1) as r ↓ 0 .

In other words, if we define Hr(y) = H(r, y), we find

limr↓0

Hr(y) = y , uniformly with respect to y ∈ B .

Pick any α such that 0 < α < δ. Then choose r such that 0 < r ≤ r havingthe property that ‖Hr(y)− y‖ ≤ α whenever 0 < r ≤ r. Then Lemma 5.3.1implies that

(δ − α)B ⊆ Hr(B) whenever 0 < r ≤ r .

In particular,

0 ∈ Hr(B) whenever 0 < r ≤ r .

This means that, for each r ∈]0, r], we can find a point yr ∈ B such that

Hr(yr) = 0 .

It then follows from the definition of Hr that

F1(rθ1(yr)) = F2(rθ2(yr)) .

So, if we let wrdef= F1(rθ1(yr)) = F2(rθ2(yr)), we have shown that

wr ∈ F1(U1 ∩ C1) ∩ F2(U2 ∩ C2) whenever 0 < r ≤ r .

Moreover, it is clear that

limr↓0

wr = 0 ,

since rθ1(yr) → 0 and F1 is continuous. It follows that, to conclude ourproof, it suffices to show that

wr �= 0 if r is small enough .


To see this, we estimate μ(wr). We have

wr = F1(rθ1(yr))= L1(rθ1(yr)) + o(r)

= L1

(r

n∑j=0

aj(yr)e1,j

)+ o(r)

=(r

n∑j=0

aj(yr)L1e1,j

)+ o(r) ,

and then

μ(wr) = μ(r

n∑j=0

aj(yr)L1e1,j

)+ o(r)

= rn∑

j=0

aj(yr)μ(L1e1,j) + o(r)

≥ rn∑

j=0

aj(yr) + o(r)

= r + o(r)> 0 if r is small enough ,

since μ(L1e1,j) ≥ 1. This concludes our proof.

Remark 5.4.3. The preceding proof is somewhat unusual, in that weactually end up proving more than is really needed. All we need to concludethat the sets Fi(Ui ∩ Ci) are not separated at 0 is to find a nonzero pointin F1(U1 ∩ C1) ∩ F2(U2 ∩ C2). To establish the stronger conclusion thatthe Fi(Ui ∩ Ci) are not locally separated at 0 it suffices to find nonzeropoints of F1(U1 ∩C1) ∩F2(U2 ∩ C2) arbitrarily close to zero. Yet, our proofactually yields a whole “continuous” one-parameter family {wr}r>0,r small ofsuch points! This is clearly an anomalous situation, suggesting that in factthe “true conclusion” of the theorem ought to be stronger, saying not onlythat these points exist, but that that there is a whole continuum of them,perhaps a continuous curve.

It turns out that the correct answer is not that the set F1(U1 ∩ C1) ∩F2(U2 ∩ C2) contains a whole continuous curve r �→ wr of nonzero points.Such a strong conclusion can fail to be true, but the weaker conclusion thatF1(U1 ∩ C1) ∩ F2(U2 ∩ C2) contains a nontrivial connected set containing 0is true, even though this set may fail to be path-connected. The proof of thisstronger conclusion is based on a theorem of Leray-Schauder on connectedsets of zeros of a homotopy. The issue is discussed in detail in the paper

30 H. J. SUSSMANN

Sussmann, H. J., “Transversality conditions and a strong maximumprinciple for systems of differential inclusions.” In Proceedings of the37th IEEE Conference on Decision and Control, Tampa, FL, Dec.1998. IEEE publications, New York, 1998, pp. 1-6.

A Postscript version of the paper (compressed with gzip) can be downloadedfrom the author’s Web page:

http://www.math.rutgers.edu/ sussmann ♦

6. COMPLETION OF THE PROOF

6.1. Application of the transversal intersection theorem. Fix k,�τ , �u as above.

Using the fact that C is a Boltyanskii tangent cone to S at x, we pick aneighborhood V of the origin in R

n and a continuous map ϕ : V ∩ C �→ Ssuch that ϕ(v) = x + v + o(‖v‖) as v → 0, v ∈ C.

Since the image of Rk+ under the endpoint map E�τ,�u is contained in the

reachable set R[a,b](Q, U, f ; x), the image of C under ϕ is contained inS,and the sets R[a,b](Q, U, f ; x) and S are locally separated at x, it followsthat the sets E�τ ,�u(Rk

+) and ϕ(C) are locally separated at x. The transversalintersection theorem then implies that the cones DE�τ ,�u(0)(Rk

+) and C arenot strongly transversal. Since C is not a linear subspace, we can concludefrom Lemma 5.2.1 that the cones DE�τ,�u(0)(Rk

+) and C are not transversal.Therefore there exists a nonzero π ∈ Rn such that

π · v ≤ 0 whenever v ∈ DE�τ,�u(0)(Rk+)

and

π · v ≥ 0 whenever v ∈ C .(6.1)

The formula for DE�τ,�u(0)(Rk+) given by the main technical lemma implies

that

M(b, τj)·(f(xj, uj, τj)−f(xj , u∗,j, τj)

)∈ DE�τ,�u(0)(Rk

+) for j = 1, . . . , k .

Therefore

π · M(b, τj) ·(f(xj, uj, τj) − f(xj, u∗,j, τj)

)≤ 0 for j = 1, . . . , k ,

and then

π ·M(b, τj) · f(xj, uj, τj) ≤ π · M(b, τj) · f(xj, u∗,j, τj)(6.2)for j = 1, . . . , k .

Clearly, we can also assume that

‖π‖ = 1 .(6.3)


6.2. The compactness argument. In the previous subsection, weproved that for every choice of k, �τ , �u, there exists a π for which (6.1), (6.2),(6.3) hold.

Given any subset W of [a, b]×U , let Π(W ) be the set of all π ∈ Rn thatsatisfy (6.1), (6.3), and

π · M(b, τ) · f(ξ∗(τ), u, τ) ≤ π ·M(b, τ) · f(ξ∗(τ), η∗(τ), τ)(6.4)for all (τ, u) ∈ W .

We have established that Π(W ) is nonempty whenever W is a finite set{(τ1, u1), . . . , (τk, uk)} such that a ≤ τ1 < τ2 < . . . < τk < b. Now supposethat W is any finite subset of [a, b[×U . Let

W = {(τ1, u1), . . . , (τk, uk)} , a ≤ τ1 ≤ τ2 ≤ . . . ≤ τk < b .

Let W� be the set

{(τ1, u1), (τ2 +1�, u2), (τ3 +

2�, u3), . . . , (τk +

k − 1�

, uk) .

Then Π(W�) �= ∅. Pick π� ∈ Π(W�). Then the π� belong to the unit sphere ofR

n, which is compact. So there is a subsequence {π�(ν)}ν∈N that convergesto a limit π. Then π ∈ Π(W ), so Π(W ) �= ∅.

A similar limiting argument then shows that Π(W ) �= ∅ if W is any finitesubset of [a, b]× U .

Now, it is clear that the set Π(W ) is compact for every W . Moreover,

Π(W1) ∩ . . . ∩ Π(Ws) = Π(W1 ∪ . . .∪ Ws) �= ∅

if W1, . . . , Ws are finite subsets of [a, b]× U .Let W be the set of all finite subsets of [a, b]× U . Then{

Π(W )}

W∈W

is a family of compact subsets of the unit sphere of Rn having the property

that every finite intersection of members of the family is nonempty. It followsthat ⋂{

Π(W ) : W ∈ W}�= ∅ .

Therefore

Π([a, b]× U) �= ∅ .

This means that there exists a π ∈ Rn that satisfies (6.1), (6.3), and is suchthat

π · M(b, τ) · f(ξ∗(τ), u, τ) ≤ π ·M(b, τ) · f(ξ∗(τ), η∗(τ), τ)(6.5)for all (τ, u) ∈ [a, b]× U .

32 H. J. SUSSMANN

6.3. The momentum. Let π be such that (6.1), (6.3), and (6.5) hold.Define

π(t) = π · M(b, t) .

Then π satisfies the adjoint equation

π(t) = −π(t) · A(t) .

(Proof. We know that

∂M

∂t(t, s) = A(t) · M(t, s) .

Moreover,M(t, s) · M(s, t) = 1n .

If we differentiate this with respect to s, we get

∂M

∂s(t, s) · M(s, t) + M(t, s) · A(s) ·M(s, t) = 0 .

Therefore∂M

∂s(t, s) + M(t, s) · A(s) = 0 .

So∂M

∂s(t, s) = −M(t, s) · A(s) .

Then

π(s) = π · ∂M

∂s(b, s) = −π ·M(b, s) · A(s) = −π(s) ·A(s) ,

as desired.)Condition (6.1) says that

−π(b) ∈ C⊥ .

Condition (6.3) implies that π is nonzero.Finally, Condition (6.5) says that

π(τ) · f(ξ∗(τ), u, τ) ≤ π(τ) · f(ξ∗(τ), η∗(τ), τ)(6.6)for all (τ, u) ∈ [a, b]× U .

Therefore

H(ξ∗(τ), u, π(τ), τ) ≤ H(ξ∗(τ), η∗(τ), π(τ), τ)for all (τ, u) ∈ [a, b]× U .

It then follows that

H(ξ∗(τ), η∗(τ), π(τ), τ) = max{H(ξ∗(τ), u, π(τ), τ) : u ∈ U}for all τ ∈ [a, b].

We have thus shown that π satisfies all the desired conditions. Our proofis therefore complete, under the extra assumption that η∗ is continuous.


6.4. Elimination of the continuity assumption on the referencecontrol. The assumption that η∗ is continuous was made to simplify somesteps of the proof, but is not really necessary. We now explain how to avoidthis hypothesis, and assume only that η∗ is a bounded, measurable U -valuedfunction on [a, b].

The continuity of η∗ was used in the proof exactly four times. We willnow show how, in each case, the hypothesis can be avoided.

1. In page 11, we defined the matrix-valued function [a, b] � t �→ A(t),and asserted that this function was continuous. If η∗ is only boundedand measurable, then the set {(ξ∗(t), η∗(t), t) : a ≤ t ≤ b} is containedin a compact subset of Q × U × [a, b], so A turns out to be boundedand measurable, though not necessarily continuous. This, however, isgood enough, and the properties of the fundamental solution M arethe same.

2. In page 13, we defined the set U0 and asserted that U0 is compact.If η∗ is not continuous, the conclusion that U0 is compact no longerfollows. On the other hand, we can define U0 in this case to be theclosure of the set of page 11, and this new set is compact. Using thenew U0 instead of the original one, all the arguments where U0 occursare still valid.

3. In page 19, we used the fact that the function F is continuous, whichdepended very strongly on the continuity of η∗. This problem is muchmore serious than the previous ones, and to solve it we need some basicfacts from the theory of functions of a real variable.Recall that, if [a, b] � t �→ ψ(t) ∈ R

n is an integrable function, aLebesgue point of ψ is a τ ∈]a, b[ such that

limh↓0

1h

∫ τ+h

τ−h‖ψ(t)− ψ(τ)‖ dt = 0

It is then a theorem that almost every point of [a, b] is a Lebesguepoint of ψ.To avoid invoking the fact that F is continuous, we make the furtherrestriction that the times τj where our needle variations are made areLebesgue points of the function F .The argument is then modified as follows. Estimate (4.5) is still validand useful, but (4.6)—which is still valid—is no longer useful, becausethere is no reason now for (4.4) to hold.On the other hand, if we let

ζ0(h) def=

1h

max{∫ τj+h

τj−h‖f(ξ∗(t), η∗(t), t)− f(ξ∗(τj), η∗(τj), τj)‖ dt : j = 1 . . . , k

},

ζ(h) def= sup{ζ0(h′) : 0 < h′ ≤ h}

34 H. J. SUSSMANN

then the condition that the τj are Lebesgue points says precisely that

limh↓0

ζ(h) = 0 .(6.7)

We then get the integral estimate∥∥∥∫[a,t]∩E(�ε)

(Δ�ε

2(s) − θ�ε(s))

ds∥∥∥ ≤

(4κ2‖�ε‖eκ(b−a) + kζ(‖�ε‖) + λ(‖�ε‖)

)· ‖�ε‖ ,

which replaces (4.7). The rest of proof of the main technical lemmaremains unchanged.

4. In the compactness argument in page 31, we approximated the τj ,that could in principle be equal, by points τj,� that are all different.In our new situation this argument has to be refined, because theapproximating τj,� have to be Lebesgue points of F , and the passageto the limit in (6.4) requires that

lim�→∞

f(ξ∗(τj,�), η∗(τj,�), τj,�) = f(ξ∗(τj), η∗(τj), τj) .

To take care of this we use Lusin’s theorem, which guarantees that forevery positive β there exists a compact subset Jβ of [a, b] such thatmeas([a, b]\Jβ) ≤ β having the property that the restriction of F toJβ is continuous.We pick sets Jβν as above, corresponding to a sequence {βν} thatconverges to zero and then, for each ν, we let Gν be the set of allpoints of Jβν that are also Lebesgue points of F , and let Gν be theset of all points of density of Gν . (Recall that a point of density ofa measurable set E is a point of E which is a Lebesgue point of theindicator function of E.) We let G =

⋃ν∈NGν. Then G is a subset of

full measure of [a, b], and it is easy to see that the limiting argumentworks when the τj belong to G.

The proof of the maximum principle, thus modified, works exactly as before,provided that the needle variations are made using points τj ∈ G. Theresult now is the same as before, except that the Hamiltonian maximizationcondition is only proved for t ∈ G. This explains why Condition E4 ofTheorem 2.1.1 says that the maximization holds “almost everywhere” ratherthan “everywhere.”

FINAL EXAM

Hector J. Sussmann—The Weizmann Institute—January 2001

This is a take-home exam, consisting of FOUR problems. You can send your solutionsto me by e-mail, to [email protected] . Send either a Plain TeX or LateX filewithout using any special Weizmann-Institute macros, or a Postscript file. Whenyou e-mail your file, do not use any “MIME-encoding” or Microsoft-dependentstuff.

PROBLEM 1: Do Problem 3 of the list in the first handout, dated November 1, 2000.(Recall: a norm on IRn is a function

IRn � x �→ ‖x‖ ∈ IR

such that (i) ‖x‖ > 0 for all x ∈ IRn such that x �= 0, (ii) ‖x + y‖ ≤ ‖x‖ + ‖y‖ for all x, y,(iii) ‖r · x‖ = |r| · ‖x‖ whenever r ∈ IR and x ∈ IRn.)

PROBLEM 2: Do Problem 11 of the list in the first handout, dated November 1, 2000.

PROBLEM 3: For Problem 16 of the list in the first handout, dated November 1, 2000,prove the following:

(a) For every optimal trajectory-control pair (ξ, η) with a continuous control, the controlη is is fact constant and equal to +1, −1 or 0; moreover, the zero control case can onlyoccur where ω = 0.

(b) For an optimal trajectory-control pair (ξ, η), it is not possible for the control η to havea jump discontinuity from 0 to 1 or −1 or from 1 or −1 to zero. (In other words, thereare no optimal “jump junctions” between bang and singular arcs.)

NOTE: Result (b) is fairly counterintuitive. One would imagine in view of (a) that, if youwant to control the vehicle from a certain initial position (x−, y−, θ−, ω−) to a terminalposition (x+, y+, θ+, ω+) which is very far away, then the right thing to do is to follow astraight line, if possible, and if this cannot be done then you follow a straight line most ofthe time, except that at the beginning and at the end you use some bang-bang manoeuversto get into the line. Now, motion along a straight line corresponds to θ = constant, soω = 0 and u = 0. Result (b) says that the “bang-bang manoeuvers” cannot be finiteconcatenations of +1 and −1 controls.

OPTIONAL PROBLEM: Give a complete description of the optimal trajectories for Prob-lem 3. NOTE: this is a hard open problem. If you solve it correctly then this will be amajor mathematical achievement that will make you famous.

PROBLEM 4: Fill in the details (for the fixed time interval case) for the result on “moregeneral endpoint conditions” outlined in the remark in Page 7 of the January 3 handout.In particular: (a) give a complete statement of the theorem, (b) give a complete proof thatthe statement follows from Theorem 1.3.1 of the handout.

18 examples of calculus of variations and optimal …kawski/classes/apm582/16... · 18 examples of...

Documents