costate estimation for optimal control problems...
Post on 02-Oct-2020
13 Views
Preview:
TRANSCRIPT
COSTATE ESTIMATION FOR OPTIMAL CONTROL PROBLEMS USINGORTHOGONAL COLLOCATION AT GAUSSIAN QUADRATURE POINTS
By
CAMILA CLEMENTE FRANCOLIN
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOLOF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
2013
c© 2013 Camila Clemente Francolin
2
To the ones who supported me: Mamae and Carol.
And to the ones who inspired me: Papai and Nicolas.
3
ACKNOWLEDGMENTS
Getting a PhD has been, hands down, the hardest thing I have accomplished thus
far in my life. It goes without saying I did none of it by myself, and I owe many thanks to
the people who, in many ways, helped me through this process. For obvious reasons,
none of this would have been possible without my faculty advisor, Dr. Anil Rao. I thank
you for always holding me to the highest standards. You were tough when you needed
to be, and encouraging the rest of the time. You helped me evolve and gain confidence
as a researcher, and I am a much better scientist for it. I would also like to thank my
committee members for helping me through the doctoral process: Dr. William Hager,
Dr. Richard Lind, and Dr. Warren Dixon. I am deeply grateful to Dr. William Hager for
patiently taking the time to meet and discuss my research, and kindly correcting me
when I was wrong.
I would also like to thank the Office of Naval Research, especially Dr. Maria
Medeiros and Dr. David Drumheller, for their financial support. The time I spent working
at the Naval Undersea Warfare Center was highly instructive. Thank you to Chris Duarte
and Gerry Martel for their mentorship during the time I spent there.
A PhD is not just about academic growth, but also about personal development. I
have a lot of people to thank for the latter part of my formation. I first thank those who,
through their love of science and discovery, inspired me to go down this path, as it is
not an easy one to pick. My first inspiration was my Dad, whom I watched go through
this process so many years ago. He was in a foreign country with two small children
and he still made it look easy. He taught me by example at a very young age to always
question things, and to never lose a sense of curiosity; it is still this sense curiosity that
propels me to keep learning. Nick, you were my second source of inspiration. I would
never have had the courage to take this path if you hadn’t been there, forging ahead with
no fear and showing me the way. You showed me this process was possible by taking
one step at a time, and I thank you for you inspiring me with your enthusiasm and love
4
of science. Next, I want to thank all those who encouraged me and helped me to keep
going when quitting would have been so much easier. Greg, you always believed in me,
even when I didn’t. Ed, thank you for keeping me motivated toward graduation by asking
me when I would be done each and every time you saw me. I’m so lucky to have you
in my life. Thomas, thank you for moving my futon all the eight times it took to get me
graduated. Thanks to all my family in Brazil: tia Terezinha, Thais, Luluca, Erika, Celma,
Joao Matheus, Maria Alice; every time I went to see you over the summers I came back
renewed. Maxie, you were always sitting at my feet through the ups and downs of the
research process. And always had healing licks when things didn’t go according to plan
(which, as it turned out, happened quite a lot).
I would like to thank all the members of VDOL. Especially Chris and Divya in the
beginning, and Begum, Matt, and Brian toward the end (yes Brian, you are an honorary
VDOL member). Matt, how could I ever thank you for all your support. Thank you for
patiently listening when I overindulged in telling you every gory detail of my research,
kindly telling me it would be okay when my results didn’t turn out as expected, and
sharing in my excitement when it finally did turn out as expected. Thank you for putting
a fence up in my backyard just so I could write this dissertation with no distractions from
my dog. You’re my lifeboat.
Finally, to my Mom and my Sister. Getting a PhD is just one of the things that I could
never have accomplished without you both by my side. You supported me emotionally,
financially, and any other way that is possible. Mom, thank you for each and every time
you helped me move, cleaned my house, or ran my errands just so I could have more
time to finish a piece of work. I hope to always make you proud. Carol, you showed
me that it was possible to succeed, and it was okay to fail, because the first follows the
latter. You are always there when I need someone to talk to (or when I have someone to
sue). Thank you.
5
TABLE OF CONTENTS
page
ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
CHAPTER
1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2 MATHEMATICAL BACKGROUND . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.1 Continuous-Time Bolza Optimal Control Problem . . . . . . . . . . . . . . 252.1.1 First-Order Optimality Conditions . . . . . . . . . . . . . . . . . . . 262.1.2 First-Order Optimality Conditions of Integral Formulation . . . . . . 272.1.3 Control Inequality Path Constraints . . . . . . . . . . . . . . . . . . 292.1.4 State Inequality Path Constraints . . . . . . . . . . . . . . . . . . . 31
2.2 Differential-Algebraic Equations . . . . . . . . . . . . . . . . . . . . . . . . 322.2.1 Index-Reduction for Differential-Algebraic Equations . . . . . . . . 342.2.2 Solutions of High-Index Differential-Algebraic Equations . . . . . . 37
2.3 State Inequality Path Constrained Optimal Control Problems . . . . . . . 402.3.1 Indirect Adjoining . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.3.2 Direct Adjoining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452.3.3 Indirect Adjoining With Continuous Multipliers . . . . . . . . . . . . 47
2.4 Numerical Properties of Orthogonal Collocation Methods . . . . . . . . . 492.4.1 Function Approximation and Interpolation . . . . . . . . . . . . . . 49
2.4.1.1 Family of Legendre-Gauss points . . . . . . . . . . . . . 512.4.2 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.4.2.1 Low-order integrators . . . . . . . . . . . . . . . . . . . . 542.4.2.2 Gaussian quadrature . . . . . . . . . . . . . . . . . . . . 57
2.5 Orthogonal Collocation for the Solution of Optimal Control Problems . . . 592.5.1 Global Collocation at LG Points . . . . . . . . . . . . . . . . . . . . 612.5.2 Global Collocation at LGR Points . . . . . . . . . . . . . . . . . . . 632.5.3 Global Collocation at Flipped LGR Points . . . . . . . . . . . . . . 652.5.4 Variable-Order Collocation at LG Points . . . . . . . . . . . . . . . 662.5.5 Variable-Order Collocation at LGR Points . . . . . . . . . . . . . . 682.5.6 Variable-Order Collocation at Flipped LGR Points . . . . . . . . . . 70
3 COSTATE ESTIMATION USING THE INTEGRAL FORMULATION . . . . . . . 72
3.1 Continuous-Time Bolza Optimal Control Problem . . . . . . . . . . . . . . 733.1.1 Differential and Integral Forms of Optimal Control Problem . . . . . 743.1.2 First-Order Optimality Conditions of Differential and Integral Forms 75
6
3.2 Costate Estimation Using Integral Legendre-Gauss Collocation . . . . . . 763.2.1 Differential Form of LG Collocation . . . . . . . . . . . . . . . . . . 763.2.2 KKT Conditions Using Differential LG Collocation . . . . . . . . . . 783.2.3 Integral Form of LG Collocation . . . . . . . . . . . . . . . . . . . . 803.2.4 KKT Conditions Using Integral LG Collocation . . . . . . . . . . . . 823.2.5 A Relationship Between Integral and Differential Costate Estimates 85
3.3 Costate Estimation Using Integral Legendre-Gauss-Radau Collocation . . 863.3.1 Differential Form of LGR Collocation . . . . . . . . . . . . . . . . . 873.3.2 KKT Conditions Using Differential LGR Collocation . . . . . . . . . 883.3.3 Integral Form of LGR Collocation . . . . . . . . . . . . . . . . . . . 913.3.4 KKT Conditions Using Integral LGR Collocation . . . . . . . . . . . 933.3.5 A Relationship Between Integral and Differential Costate Estimates 97
3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 973.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4 MOTIVATION FOR NEW COSTATE ESTIMATE . . . . . . . . . . . . . . . . . . 100
4.1 Continuous-Time Bolza Optimal Control Problem . . . . . . . . . . . . . . 1014.1.1 First-Order Optimality Conditions of Continuous Problem . . . . . . 102
4.2 Variable-Order Collocation at Legendre-Gauss Points . . . . . . . . . . . 1034.2.1 KKT Conditions of Variable-Order LG Collocation Method . . . . . 1044.2.2 Costate Estimate and Transformed Adjoint System . . . . . . . . . 105
4.3 Variable-Order Collocation at Flipped Legendre-Gauss-Radau Points . . . 1084.3.1 KKT Conditions of Variable-Order Flipped LGR Collocation Method 1094.3.2 Costate Estimate and Transformed Adjoint System . . . . . . . . . 110
4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5 COSTATE ESTIMATION FOR STATE CONSTRAINED PROBLEMS . . . . . . 116
5.1 Continuous-Time State-Constrained Optimal Control Problem . . . . . . . 1175.1.1 First-Order Optimality Conditions . . . . . . . . . . . . . . . . . . . 119
5.2 Costate Estimation Using Legendre-Gauss Collocation . . . . . . . . . . . 1205.2.1 Variable-Order Collocation at Flipped LG Points . . . . . . . . . . . 1205.2.2 Costate Estimate and Transformed Adjoint System . . . . . . . . . 122
5.3 Costate Estimation Using Flipped Legendre-Gauss-Radau Collocation . . 1275.3.1 Variable-Order Collocation at Flipped LGR Points . . . . . . . . . . 1275.3.2 Costate Estimate and Transformed Adjoint System . . . . . . . . . 129
5.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
6 EXAMPLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6.1 Example 1: Mayer Optimal Control Problem . . . . . . . . . . . . . . . . . 1406.1.1 Solution Using Collocation at LG Points . . . . . . . . . . . . . . . 1406.1.2 Solution Using Collocation at LGR Points . . . . . . . . . . . . . . 144
6.2 Example 2: Lagrange Optimal Control Problem . . . . . . . . . . . . . . . 1476.2.1 Solution Using Collocation at LG Points . . . . . . . . . . . . . . . 1486.2.2 Solution Using Collocation at LGR Points . . . . . . . . . . . . . . 151
7
6.3 Example 3: First-Order State Inequality Path Constraint Problem . . . . . 1546.3.1 Solution Using Collocation at LG Points . . . . . . . . . . . . . . . 154
6.3.1.1 Previously derived costate estimate . . . . . . . . . . . . 1576.3.1.2 Indirect adjoining with continuous multipliers . . . . . . . 160
6.3.2 Solution Using Collocation at Flipped LGR Points . . . . . . . . . . 1626.3.2.1 Previously derived costate estimate . . . . . . . . . . . . 1626.3.2.2 Indirect adjoining with continuous multipliers . . . . . . . 164
6.4 Example 4: Second-Order State Inequality Path Constraint Example . . . 1686.4.1 Solution Using Collocation at LG Points . . . . . . . . . . . . . . . 169
6.4.1.1 Previously derived costate estimate . . . . . . . . . . . . 1726.4.1.2 Indirect adjoining with continuous multipliers . . . . . . . 175
6.4.2 Solution Using Collocation at Flipped LGR Points . . . . . . . . . . 1786.4.2.1 Previously derived costate estimate . . . . . . . . . . . . 1816.4.2.2 Indirect adjoining with continuous multipliers . . . . . . . 184
7 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
BIOGRAPHICAL SKETCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
8
LIST OF TABLES
Table page
2-1 Absolute maximum error in v(t) and u(t) for problems A and B. . . . . . . . . 40
9
LIST OF FIGURES
Figure page
2-1 Error in the solution of DAE system . . . . . . . . . . . . . . . . . . . . . . . . . 39
2-2 Function approximation using uniformly spaced points . . . . . . . . . . . . . . 52
2-3 Distribution of Gaussian quadrature points . . . . . . . . . . . . . . . . . . . . . 53
2-4 Function approximation using LG points . . . . . . . . . . . . . . . . . . . . . . 55
2-5 Function approximation using LGR points . . . . . . . . . . . . . . . . . . . . . 56
2-6 Error associated with function approximation using uniform, LG, and LGR points 57
2-7 Approximation of integral using Trapezoid rule. . . . . . . . . . . . . . . . . . . 58
2-8 Error in approximation of integral using Trapezoid rule as a function of N . . . . 58
2-9 Error in approximation of integral using Gaussian quadrature as a function of N 60
2-10 Distribution of LG points for global collocation . . . . . . . . . . . . . . . . . . . 62
2-11 Distribution of LGR points for global collocation . . . . . . . . . . . . . . . . . . 64
2-12 Distribution of flipped LGR points for global collocation . . . . . . . . . . . . . . 65
2-13 Distribution of LG points for variable-order collocation . . . . . . . . . . . . . . 67
2-14 Distribution of LGR points for variable-order collocation . . . . . . . . . . . . . 69
2-15 Distribution of flipped LGR points for variable-order collocation . . . . . . . . . 71
4-1 Relationship Between the Direct and Indirect Methods . . . . . . . . . . . . . . 115
5-1 Equivalence of the Direct and Indirect Methods . . . . . . . . . . . . . . . . . . 138
6-1 Primal solution for Example 1 obtained using integral collocation at LG points. . 141
6-2 Costate solutions for Example 1 obtained using collocation at LG points. . . . . 142
6-3 Costate errors for Example 1 obtained using collocation at LG points. . . . . . 143
6-4 Primal solution for Example 1 obtained using integral collocation at LGR points. 145
6-5 Costate solutions for Example 1 obtained using collocation at LGR points. . . . 146
6-6 Costate errors for Example 1 obtained using collocation at LGR points. . . . . 146
6-7 State and control for Example 2 obtained using integral LG collocation. . . . . 148
6-8 Costate solutions for Example 2 obtained using collocation at LG points. . . . . 149
10
6-9 Costate errors for Example 2 obtained using collocation at LG points. . . . . . 150
6-10 State and control for Example 2 obtained using integral LGR. . . . . . . . . . . 152
6-11 Costate solutions for Example 2 obtained using collocation at LGR points. . . . 153
6-12 Costate errors for Example 2 obtained using collocation at LGR points. . . . . 153
6-13 Primal solution for Example 3 obtained using collocation at LG points. . . . . . 155
6-14 Errors in state and control for Example 3 obtained using LG collocation. . . . . 156
6-15 Costate estimate as derived by Ref. [1] for Example 3. . . . . . . . . . . . . . . 158
6-16 Costate errors for estimate derived in Ref.[1] for Example 3. . . . . . . . . . . . 159
6-17 Dual variables for Example 3 obtained using collocation at LG points. . . . . . 161
6-18 Costate errors for Example 3 obtained using collocation at LG points. . . . . . 161
6-19 Primal solution for Example 3 obtained using collocation at LGR points. . . . . 163
6-20 Errors for Example 3 obtained using collocation at LGR points. . . . . . . . . . 164
6-21 Costate Estimate as derived by Ref. [1] for Example 3. . . . . . . . . . . . . . . 165
6-22 Costate errors for estimate derived in Ref.[1] for Example 3. . . . . . . . . . . . 166
6-23 Costate estimate for Example 3 obtained using collocation at LGR points. . . . 167
6-24 Costate errors for Example 3 obtained using collocation at LGR points. . . . . 167
6-25 Primal solution for Example 4 obtained using LG collocation. . . . . . . . . . . 170
6-26 State and control errors for Example 4 using collocation at LG points. . . . . . 171
6-27 Costate Estimate as derived by Ref. [1] for Example 4. . . . . . . . . . . . . . . 173
6-28 Costate errors for estimate derived in Ref.[1] for Example 4. . . . . . . . . . . . 174
6-29 Costate Estimate for Example 4 obtained using collocation at LG points. . . . . 176
6-30 Costate errors for Example 4 obtained using collocation at LG points. . . . . . 177
6-31 Primal solution for Example 4 obtained using LGR collocation. . . . . . . . . . 179
6-32 State and control errors for Example 4 using collocation at LGR points. . . . . 180
6-33 Costate estimate as derived by Ref. [1] for Example 4. . . . . . . . . . . . . . . 182
6-34 Costate Errors using estimate derived in Ref.[1] for Example 4. . . . . . . . . . 183
6-35 Costate Estimate for Example 4 obtained using collocation at LGR points. . . . 185
11
6-36 Costate errors for Example 4 obtained using collocation at LGR points. . . . . 186
12
Abstract of Dissertation Presented to the Graduate Schoolof the University of Florida in Partial Fulfillment of theRequirements for the Degree of Doctor of Philosophy
COSTATE ESTIMATION FOR OPTIMAL CONTROL PROBLEMS USINGORTHOGONAL COLLOCATION AT GAUSSIAN QUADRATURE POINTS
By
Camila Clemente Francolin
August 2013
Chair: Anil V. RaoMajor: Aerospace Engineering
Computing the costate in an optimal control problem is important for verifying
the optimality of the solution and performing sensitivity analysis. This dissertation is
concerned with the problem of estimating the costate in an optimal control problem
using orthogonal collocation at Legendre-Gauss (LG) and Legendre-Gauss-Radau
(LGR) points. First, methods are presented for estimating the costate using orthogonal
collocation at the LG or LGR points when the dynamic constraints of the optimal control
problem are formulated in integral form. A new continuous-time dual variable called
the integral costate is introduced, where the integral costate is the Lagrange multiplier
of the integral dynamic constraint. The first-order optimality conditions of the integral
form of the optimal control problem are derived in terms of the integral costate. The
integral form of the optimal control problem is then discretized using the integral LG
and LGR collocation methods and relationship between the discrete form of the integral
costate and the costate of the original differential optimal control problem are developed.
It is shown that the LGR integration matrix that relates the differential costate to the
integral costate is singular while the corresponding LG integration matrix is full rank. The
approach developed in this research then provides a way to estimate the costate of the
original optimal control problem using the Lagrange multipliers of the integral form of the
LG and LGR collocation methods. Furthermore, the costate estimates presented in this
research result in a set of Karuhn-Kush-Tucker conditions of the nonlinear programming
13
problem which are a discrete approximation of the first-order optimality conditions of the
continuous-time optimal control problem both in differential and integral forms.
The second part of this research focuses on state inequality path constrained
optimal control problems. Problems with active state-inequality path constraints are
difficult to solve due to the high-index differential-algebraic equations (DAE) that result
from the constraint activity. This DAE index fluctuation in the solution domain results in
possible discontinuities in the dual variables which are hard to approximate numerically.
Due to these discontinuities, previous costate estimates for direct transcription methods
using collocation at LG or LGR points resulted in a transformed adjoint system
which was not a discrete approximation to the first-order optimality conditions in the
presence of state inequality path constraints. In this research a different set of costate
estimates are developed which result in a transformed adjoint system that is a discrete
approximation of the first-order optimality conditions of the continuous-time optimal
control problem. Specifically, a costate estimate using the method of indirect adjoining
with continuous multipliers is derived. The equivalence between the first-order optimality
conditions of the finite-dimensional nonlinear program and the first-order optimality
conditions of the continuous-time optimal control problem ensures convergence of the
discrete problem to a local minimum which satisfies the optimality conditions of the
original problem. This costate estimate can thus be used to verify the extremality of the
approximated solution.
14
CHAPTER 1INTRODUCTION
Many problems in engineering, economics, and biology can be modeled as
differential-algebraic systems. In addition, it is often desired to optimize the performance
of such systems. The goal of an optimal control problem is to determine the state and
control that optimize a given performance index subject to a set of differential-algebraic
constraints. In aerospace engineering, optimal control applications include trajectory
optimization, parameter estimation, and vehicle guidance. As alluded to earlier, the
constraints in an optimal control problem include differential equations that describe the
motion of the dynamical system, path constraints that define limits on the process, and
event constraints that define way points that must be met during the motion.
Optimal control problems that involve inequality path constraints are common in
aerospace engineering. Such constraints can be purely a function of the control (for
example, control limits such as maximum allowable thrust), purely a function of the
state (for example, no-fly zone constraints), or more generally a function of both the
control and the state (for example, maximum heating rate constraints). Quoting Ref. [2],
“Solving an optimal control or estimation problem is not easy”. Optimal control problems
with inequality path constraints are particularly challenging to solve because the optimal
trajectory may contain regions where the inequality constraint is active. Even more
challenging are problems with inequality path constraints that are purely a function of
the state, leading to high-index differential-algebraic equation (DAE) constraints [3–6].
Systems with state inequality path constraints of index one or less can generally be
solved numerically using numerical integrators. Systems with state inequality path
constraints of index greater than one, however, pose computational challenges for
numerical integration methods [3]. In the context of an optimal control problem, a state
inequality path constrained high-index differential-algebraic system have a non-smooth
state and possibly a discontinuous costate, while a control inequality constrained
15
problem can have a discontinuous optimal control [7, 8]. Such discontinuities can be
difficult to approximate accurately using numerical methods.
Methods for approximating solutions to optimal control problems fall into two broad
categories: indirect and direct methods. In an indirect method the first-order optimality
conditions are derived using the calculus of variations, resulting in a Hamiltonian
boundary-value problem (HBVP) [9]. In the case when the inequality path constraints
are inactive on the optimal solution, the HBVP is a two-point boundary value problem.
When the solution domain contains active/inactive switches in state inequality path
constraint activity, however, the HBVP will have interior-point constraints, resulting in a
multi-point boundary value problem [7].
A great deal of research has been done on solving optimal control problems with
state inequality path constraints using indirect methods [8, 10–13]. This research has
yielded a number of different ways to derive the necessary conditions for optimality,
each resulting in a different set of conditions. In the method of direct adjoining, the state
inequality path constraint is augmented to the Hamiltonian, and the first-order optimality
conditions are derived using the calculus of variations. This method results on a set
of “jump conditions” on the optimal costate which must be applied at the entrance and
exit of the constrained arc. In the aerospace engineering literature, state inequality
constraints have historically been handled through index-reduction of the high-index
differential-algebraic equation (DAE) system that results from the state constraint activity
[2]. The necessary conditions for optimality are derived from the calculus of variations
using an approach termed indirect adjoining in which the state inequality constraint is
differentiated before being adjoined to the Lagrangian [7]. Using this approach, Ref. [10]
develops a set of tangency conditions that are enforced at the entrance of a constrained
arc, often leading to discontinuities in the costate. The control along the constrained
arc is then defined by setting to zero the lowest derivative of the inequality constraint
that is an explicit function of the control variable. The costate discontinuities that arise
16
from the necessary conditions for optimality then become a function of the tangency
conditions. In Ref. [13] a modified problem is posed where the original path constraint
is augmented to the cost functional and the tangency conditions are applied at both
the entrance and exit of the constraint activity. The formulation of Ref. [13] leads to a
reduction in the dimension of the state space in the region of active constraint activity. In
[14] a numerical technique for dealing with these problems is developed using steepest
descent.
Another technique for solving state inequality path constrained optimal control
problems is the method of indirect adjoining approach with continuous multipliers
[15]. In this method, the discontinuity in the costate is “subtracted out,” leading to
a set of optimality conditions that yield a continuous costate even if if the solution
lies on a constrained arc [16–19]. Reference [15] summarizes the methods of direct
adjoining, indirect adjoining, and indirect adjoining with continuous multipliers used
in the derivation of the necessary conditions for optimality of a state inequality path
constrained optimal control problem.
Indirect methods are attractive because the solution of the HBVP is an extremal
and thus must satisfy the first-order optimality conditions from the calculus of variations.
Consequently, a solution obtained using an indirect method can be accurate. The
HBVP, however, generally does not have an analytic solution. Therefore, numerical
methods must be employed. Common numerical approaches for solving the HBVP are
shooting, multiple shooting, and collocation [20]. Numerical implementations of Indirect
methods pose a number of computational challenges. First, the first-order optimality
conditions are often difficult to derive. Second, the radius of convergence of the resulting
Hamiltonian boundary value problem can be notably small due to instabilities in the
Hamiltonian dynamics [21]. As a result, an indirect method often requires a good initial
guess for both the state and the costate [2, 7, 9]. However, providing an initial guess for
the costate is often difficult because the costate has no physical interpretation. Finally, in
17
the case when the optimal solution has constrained and unconstrained arcs, it becomes
necessary to estimate the constrained arc sequence [2]. Estimating switches in path
constraint activity is often difficult when no a-priori knowledge of the solution structure is
available.
The second class of numerical methods in optimal control are direct methods.
Different from indirect methods, direct methods parametrize the control and/or the state,
and the continuous-time problem is discretized and transcribed into a finite-dimensional
nonlinear programming problem (NLP). The resulting NLP can then be solved using
well developed optimization software [22–25]. Direct methods have gained a great deal
of popularity as they avoid a number of the pitfalls associated with indirect methods.
Specifically, because a direct method directly transcribes the optimal control problem
into a NLP, the lengthy derivations of the first-order optimality conditions are avoided.
Also, direct methods do not require an initial guess for the costate, and the problem
can be modified relatively easily without having to re-derive the optimality conditions
[2, 26, 27]. Many direct methods, however, are not as accurate as indirect methods and
they require further analysis to verify optimality once a solution is achieved.
Direct methods can employ either a sequential or a simultaneous optimization
approach. In a sequential approach the control is parametrized and the dynamics
are integrated over the trajectory domain. One example of a sequential optimization
method is the direct shooting method [28–30]. In a direct shooting method the
control is parametrized and the dynamics are integrated using numerical integration
methods. Direct shooting methods are useful when the control can be parametrized
using few parameters, keeping the problem size small. As the number of variables
needed to parametrize the control increases, however, convergence to a solution
using direct shooting methods becomes difficult. Direct multiple-shooting methods
improve convergence by subdividing the solution domain into multiple intervals [28]. The
shooting method is then applied in each interval, and continuity of the state is enforced
18
at the interval boundaries. Multiple-shooting methods have better convergence than
shooting methods because the integration of the state dynamics is done over shorter
intervals. Both direct shooting and direct multiple shooting methods, however, are not
computationally efficient due to the sequential numerical integration technique used to
integrate the dynamics. Furthermore, convergence still depends on a-priori knowledge
of the constrained and unconstrained arc sequence.
A particular direct methods known as a collocation method, employ a simultaneous
optimization approach [2, 30–36]. Collocation methods parametrize both the control and
the state, and the differential-algebraic equations are enforced at a set of discrete points
in the domain [2, 26, 37]. Direct collocation methods are attractive because they require
no a priori knowledge of the solution structure [38]. Furthermore, direct collocation
methods are less sensitive to the initial guess than the sequential approach of shooting
methods [2]. Well-known software implementations of direct collocation methods include
SOCS, DIDO, DIRCOL, and GPOPS [39–42].
Direct collocation methods can employ local h-method collocation, or global
p-method collocation. Often the class of Runge-Kutta methods is used to collocate
and integrate the system dynamics [2, 33, 43–45]. Runge-Kutta methods are usually
employed as h-methods in which the solution domain is subdivided into many intervals
and a fixed low-degree approximation is used in each interval. This type of scheme
is computationally efficient as it has a sparse structure that can be exploited by NLP
solvers [46]. Convergence of the numerical discretization using h-methods is then
achieved by increasing the number of intervals in the domain. Due to the polynomial
convergence rate of this kind of scheme, however, h-methods can lead to extremely
large NLP’s [45, 47, 48].
In contrast to local h-methods, global p-methods use a single polynomial to
approximate the state over the entire domain [26, 27, 49]. Convergence in a p-method
is then obtained by increasing the degree of the approximating polynomial. A p-method
19
has the advantage that it converges exponentially for problems for problems whose
solutions are smooth. In the case when the solution is not smooth (as often happens in
the presence of active inequality constraints) the convergence rate is significantly slower.
Furthermore, the NLP arising from a p-method is less sparse than the NLP arising from
an h-method.
This research will employ an hp-method using collocation at the LG and LGR points
[50, 51]. In an hp−method, or variable-order method, the solution domain is divided
into a mesh, and the degree of the approximating polynomial (that is, the number of LG
or LGR collocation points) in each interval is allowed to vary. Using an hp-method it is
possible to divide the problem into intervals such that the solution in each interval is
smooth. Thus convergence is achieved by increasing the degree of the approximating
polynomial in each interval. In this manner it is possible to achieve a high accuracy
solution solution while keeping the NLP smaller than what might be possible using an
h-method.
Over the last decade, one class of direct collocation methods which has risen
to prominence in the numerical solution of optimal control problems is the class of
orthogonal collocation methods [26, 27, 34–36, 42, 49, 52–65].Orthogonal collocation
methods parametrize the state using global polynomials and collocate the differential-algebraic
equations using nodes obtained from a Gaussian quadrature. The three most commonly
used sets of collocation points are Legendre-Gauss (LG), Legendre-Gauss-Radau
(LGR), and Legendre-Gauss-Lobatto (LGL) points. These three sets of points are
obtained from the roots of a Legendre polynomial and/or linear combinations of a
Legendre polynomial and its derivatives. All three sets of points are defined on the
domain [−1, 1], but differ significantly in that the LG points include neither of the
endpoints, the LGR points include one of the endpoints, and the LGL points include
both of the endpoints. In addition, the LGR points are asymmetric relative to the
origin and are not unique in that they can be defined using either the initial point or
20
the terminal point. Although collocation at the LGL points provides state and control
approximations at the endpoints, it was shown by Refs. [64, 65] that the control and
costate approximations using LGL points tends to be innacurate due to a rank-deficient
differentiation matrix. Furthermore Ref. [36] also shows that using collocation at the
LG and LGR points yields a highly accurate approximation to the optimal state, control,
and costate. Because collocation at the LG and LGR points provide similar accuracy
whereas collocation at LGL points can provide erroneous solutions, this research will
focus on using collocation at the LG and LGR points.
When approximating the solution to an optimal control problem using any numerical
method, it is important to analyze the solution in an attempt to verify the convergence of
the discrete problem to a local minima of the continuous-time problem [4, 5, 66, 67]. One
key advantage of an orthogonal collocation method is the elegant transformations of
the KKT conditions of the NLP to the first-order optimality conditions derived analytically
from the calculus of variations [36, 64, 65, 68]. Such transformations have previously
been derived for optimal control problems with no active state inequality path constraints
and when the dynamic constraints are formulated in their differential form. When
available, such transformations show that the first-order optimality conditions of the
discrete NLP are equivalent to the discrete form of the first-order optimality conditions
of the continuous-time optimal control problem derived from the calculus of variations.
Therefore, in this research a gap of costate estimation theory is closed using collocation
at LG and LGR points by deriving a mapping for the costate estimate for the case when
the dynamic constraints are expressed in integral form and in the presence of state
inequality path constraints.
While the LG and LGR methods are equivalent regardless of whether collocation
is performed in either differential or integral form, the differential form of either method
has been predominantly used. Recently, however, more practical work has been done
in implementing both the differential and integral forms of LG and LGR collocation using
21
so called variable-order methods where the time interval is partitioned into a mesh and
mesh refinement techniques are used to determine an appropriate mesh that meets a
specified solution accuracy tolerance [69, 70]. This research indicates strongly that their
may be computational advantages to using the integral form of LG and LGR collocation
over the differential form. In fact, the most current implementation of LGR collocation is
the MATLAB optimal control software GPOPS− II [69]. GPOPS− II uses the integral
form of LGR collocation as the default because it has been found through a variety
of examples that the integral form provides more consistent results. Moreover, the
use of the implicit integral form of LG and LGR collocation is most consistent with the
implementations used by many established optimal control software packages such as
SOCS [39], DIRCOL [41], OTIS [71], ICLOCS [72], and ACADO [73].
While the differential and integral forms of the LG and LGR methods are mathematically
equivalent with regard to the primal variables (that is, the state and control), the
two formulations produce completely different dual variables. In particular, the
relationship between the Lagrange multipliers of the collocation conditions of the
dynamic constraints and the costate of the optimal control problem has been well
documented [27, 36, 64, 65]. On the other hand, the corresponding relationship
between the Lagrange multipliers associated with the integral forms of LG and LGR
collocation and the costate of the optimal control problem has not been established.
When employing the integral forms of LG and LGR collocation, however, it may be of
interest to either verify optimality or perform sensitivity analysis in a manner consistent
with that which would be performed when using variational methods. In such cases it
is useful to obtain a costate estimate when using the integral forms of the LG and LGR
methods.
In this research a methods for estimating the optimal control costate using the
integral forms of LG and LGR collocation is developed. Specifically, transformations
are derived that relate the Lagrange multipliers of the integral forms of the LG and
22
LGR collocation methods to the costate of the original optimal control problem. These
transformations are derived by writing the original continuous-time optimal control
problem in integral form. A new continuous-time dual variable called the integral costate
is then introduced, where the integral costate is the Lagrange multiplier of the integral
dynamic constraint. The first-order optimality conditions of the integral form of the
optimal control problem are derived in terms of the integral costate. The integral form of
the optimal control problem is then discretized using the integral LG and LGR collocation
methods and the relationships between the discrete form of the integral costate and the
costate of the original differential optimal control problem are developed. It is shown
that the LGR integration matrix that relates the differential costate to the integral costate
is singular while the corresponding LG integration matrix is full rank. The approach
developed in this research then provides a way to estimate the costate of the original
optimal control problem using the Lagrange multipliers of the integral form of the LG and
LGR collocation methods.
Next, inequality path constrained optimal control problems are analyzed. Although
previous research has successfully derived a high-accuracy estimate of the costate from
the KKT multipliers of the NLP for the case of a problem with no active state inequality
path constraints, Ref. [1] subsequently showed that in the case when the costate is
discontinuous (as is the case in the presence of active state inequality path constraints),
this costate estimate leads to a set of first-order optimality conditions of the NLP that
are not equivalent to the discrete form of the variational optimality conditions. This lack
of equivalence leads to an inaccurate approximation of the costate. Therefore, in this
research this inaccuracy is rectified by developing a new approach for costate estimation
using the method of indirect adjoining with continuous multipliers [15, 19]. The costate
estimate derived in this research leads to a transformed adjoint system which is a
discrete approximation of the first-order optimality conditions of the continuous-time
problem.
23
The contributions of this research are as follows. First, costate estimates are
derived using collocation at Legendre-Gauss and Legendre-Gauss-Radau points for the
case when the dynamic constraints of the optimal control problem are formulated
in integral form. Second, it is demonstrated that the costate mapping derived for
collocation at the LG and LGR points leads to a set of transformed optimality conditions
of the NLP that are shown to be a discrete representation of the necessary conditions
for optimality of the continuous-time problem. Third, a relationship between the integral
and the differential forms of the costate estimate is given and it is shown that the
two sets of optimality conditions are equivalent. Fourth, a new costate estimate for
collocation at LG and LGR points is derived for problems with active state inequality
path constraints. This costate estimate is shown to lead to a transformed adjoint system
of the NLP which is a discrete approximation of the necessary conditions for optimality
of the continuous-time optimal control problem. Finally, examples are presented that
characterize the accuracy of the costate estimates presented in this research.
24
CHAPTER 2MATHEMATICAL BACKGROUND
In this chapter the mathematical background necessary to understand the scope of
the research is provided. First, a general continuous-time Bolza optimal control problem
is defined and the first-order optimality conditions of this continuous-time Bolza optimal
control problem arising from the calculus of variations are derived. Second, an overview
of methods for solving differential-algebraic equations (DAE) is presented. In particular,
it is shown that optimal control problems with active state inequality path constraints lead
to high-index DAEs which are inherently difficult to solve using numerical methods. A
method of index-reduction is then presented to overcome the numerical difficulties that
arise from high-index DAE systems. Third, various methods are presented to derive the
necessary conditions for optimality of state inequality path constrained continuous-time
optimal control problems. Fourth, methods for transcribing a general continuous-time
optimal control problem to a nonlinear program (NLP) using orthogonal collocation at
Legendre-Gauss and Legendre-Gauss-Radau points are described. Finally, in order
to explain and legitimize the use of orthogonal collocation methods to solve optimal
control problems, a brief background is provided in function interpolation and numerical
integration.
2.1 Continuous-Time Bolza Optimal Control Problem
Without loss of generality, consider the following optimal control problem in Bolza
form. Determine the state, y(t) ∈ Rn, and the control, u(t) ∈ R
m, that minimize the cost
functional
J = Φ(y(tf )) +
∫ tf
t0
g(y(t), u(t))dt (2–1)
subject to the dynamic constraint
dy
dt= y(t) = f(y(t), u(t)) ∈ R
n, (2–2)
25
the boundary condition
φ(y(t0)) = 0 ∈ Rq, (2–3)
and the state and control inequality path constraint
C(y(t), u(t)) ≤ 0 ∈ Rc . (2–4)
The cost functional of Eq. (2–1) consists of a Mayer cost, which is evaluated purely at
the end points of the domain, and a Lagrange, or integral, cost. The optimal control
problem of Eqs. (2–1)–(2–4) will be referred to as the continuous Bolza problem.
2.1.1 First-Order Optimality Conditions
The first-order necessary conditions for an extremal solution of the continuous
Bolza problem can be derived using the calculus of variations [9]. First, using Lagrange
multipliers, the constraints of the optimal control problem are augmented to the cost
functional to generate the augmented cost functional
Ja =Φ(y(tf ))−ψ⊤φ(y(t0)) (2–5)
+
∫ tf
t0
[
g(y(t), u(t))− λ⊤(t)(y(t)− f(y(t), u(t)))− µ⊤(t)C(y(t), u(t))]
dt,
where ψ, λ, and µ are the Lagrange multipliers associated, respectively, with the
boundary conditions of Eq. (2–3), the dynamic constraints of Eq. (2–2), and the
inequality path constraints of Eq. (2–4).
Next, taking the first variation of the augmented cost with respect to all free
variables (i.e., y(t),u(t), ψ,λ(t), and µ(t)), we obtain
δJa =∂Φ
∂y(tf )δyf − δψ⊤φ−ψ⊤
[
∂φ
∂y(t0)δy0
]
+
∫ tf
t0
[
∂g
∂yδy +
∂g
∂uδu
−δλ⊤(y− f) + λ⊤
(
δfδy
δy+δfδu
δu− δy
)
− δµ⊤C− µ⊤
(
δCδy
δy+δCδu
δu
)]
dt. (2–6)
26
The term λ⊤δy in Eq. (2–6) can be integrated by parts as follows
∫ tf
t0
λ⊤δydt = λ⊤(tf )δy(tf )− λ⊤(t0)δy(t0) +
∫ tf
t0
λ⊤δydt. (2–7)
Applying the relationship of Eq. (2–7) to Eq. (2–6), the first variation of the augmented
cost can then be rewritten as a function of the augmented Hamiltonian
H(y, u,λ,µ) = g + λ⊤f− µ⊤C (2–8)
as
δJa = δψ⊤φ+
(
−ψ⊤ ∂φ
∂y(t0)+ λ⊤(t0)
)
δy(t0) +
(
∂Φ
∂y(tf )− λ⊤(tf )
)
δy(tf )
+
∫ tf
t0
[(
∂H
∂y+ λ
)
δy+
(
∂H
∂u
)
δu− δλ⊤(y− f)− δµ⊤C]
dt.
An extremal solution will satisfy the condition δJa = 0. Because the variations of the
free variables are not zero, the only way to obtain an extremal solution is to satisfy the
following set of first-order optimality conditions:
y =f(y, u), 0 = φ(y(t0)), (2–9)
λ⊤(t0) =ψ⊤ ∂φ
∂y(t0), (2–10)
λ⊤(tf ) =∂Φ
∂y(tf ), (2–11)
∂H
∂u=∂g
∂u+ λ⊤ ∂f
∂u− µ⊤∂C
∂u= 0, (2–12)
∂H
∂y=∂g
∂y+ λ⊤ ∂f
∂y− µ⊤∂C
∂y= −λ⊤
, (2–13)
µ ≤0, µ⊤S(y) = 0. (2–14)
2.1.2 First-Order Optimality Conditions of Integral Formulation
The continuous Bolza problem given by Eqs. (2–1)–(2–4) can be reformulated such
that the dynamic constraint of Eq. (2–2) are written in integral form. Reformulating the
problem in this way will be of interest to this research so that a relationship between the
27
Lagrange multipliers of the integral form can be related to the Lagrange multipliers of the
original differential form. In integral form, the optimal control problem is to determine the
state, y(t) ∈ Rn, and the control, u(t) ∈ Rm, that minimize the cost functional
J = Φ(y(tf )) +
∫ tf
t0
g(y(t), u(t))dt (2–15)
subject to the integral constraint
y(t) = y(t0) +
∫ t
t0
f(y(t), u(t)) dt, (2–16)
and the boundary condition
φ(y(t0)) = 0 ∈ Rq. (2–17)
The optimal control problem given by Eq. (2–15), along with the constraints of
Eqs. (2–16) and (2–17), will be referred to as the integral Bolza Problem.
The first-order necessary conditions for an extremal solution of the integral Bolza
problem can again be derived through the calculus of variations. First, the constraints of
Eqs. (2–16) and (2–17) are augmented to the cost such that
Ja = Φ(tf ) +ψ⊤φt0 +
∫ tf
t0
(
g(y, u)− p⊤(
y− y(t0)−∫ τ
t0
f(y, u) dt
)
dτ , (2–18)
where p(t) and ψ are the Lagrange multipliers associated with the dynamic constraints
of Eq. (2–16) and the boundary conditions of Eq. (2–17), respectively. Next, the first
variation is taken with respect to all free variables (y, u, p, and ψ), such that
δJa =∂Φ
∂y(tf )δyf −ψ⊤
[
∂φ
∂y(t0)δy0
]
− δψ⊤φ+
∫ tf
t0
[
∂g(y, u)
∂yδy
+∂g(y, u)
∂uδu− δp⊤
(
y − y(t0)−∫ t
t0
f(y, u) dτ
)
− p⊤δy
+p⊤δy(t0) +
∫ tf
t
p⊤ dτ · ∂f(y, u)∂y
δy+
∫ tf
t
p⊤ dτ · ∂f(y, u)∂u
δu
]
dt.
(2–19)
28
It is noted that the following relationship was used in Eq. (2–19):
∫ tf
t0
[
q(t)
∫ t
t0
p(τ) dτ
]
dt =
∫ tf
t0
[
p(t)
∫ tf
t
q(τ) dτ
]
dt. (2–20)
Furthermore, note that the variation of the final state is not independent, but depends on
the variations of the initial states and the state and control at intermediate points. Thus,
the variation of the final state is given as
δyf = y0 +
∫ +1
−1
[
∂f
∂yδy +
∂f
∂uδu
]
dt. (2–21)
Eq. 2–21 can be substituted into Eq. (2–19) to obtain an expression for the first variation
of the cost with respect to the independent variables.
An extremal solution will satisfy the condition δJa = 0. Because the variations of the
free variables are not zero, the only way to obtain an extremal solution is by satisfying
the following set of first-order optimality conditions:
y = y(t0) +
∫ t
t0
f(y, u) dt, φ(y(t0)) = 0, (2–22)
0 =∂g
∂u+
(∫ tf
t
p⊤ dτ +∂Φ
y(tf )
)
· ∂f(y, u)∂u
, (2–23)
p⊤ =∂g
y+
(∫ tf
t
p⊤ dτ +∂Φ
∂y
)
· ∂f(y, u)∂y
, (2–24)
ψ⊤ ∂φ
y(t0)=
∫ tf
t0
p⊤ dt +∂Φ
∂y(tf ). (2–25)
2.1.3 Control Inequality Path Constraints
Now consider an optimal control problem with an active control inequality path
constraint. When a control constraint is inactive, the optimal control is determined using
the strong form of Pontryagin’s Minimum Principle, given by Eq. (2–12) [7, 9]. When the
inequality constraint is active, however, a subset of the optimal control is determined by
the relation
Ck(y(t), u(t)) = 0, t ∈ [t1, t2], (2–26)
29
where the subscript k denotes the subset of active constraints in the time interval
[t1, t2] ⊆ [t0, tf ].
A special case of an active control inequality constraint is one that results in a bang-
bang control. If the control appears linearly in the Hamiltonian defined by Eq. (2–8),
the strong form of Pontryagin’s Minimum Principle given by Eq. (2–12) provides no
information about the optimal control, and the weak form of Pontryagin’s Minimum
Principle must be used instead [9]. Denoting u∗ as the control that minimizes the cost
functional of Eq. (2–1), by definition the following must hold true:
J(u)− J(u∗) ≥ 0,
for all trajectories with the admissible control u sufficiently close to u∗. Furthermore, for
small variations around the optimal trajectory
J(u)− J(u∗) = δJ(u∗, δu) ≥ 0.
It is known that the variation in the cost is related to the Hamiltonian by
δJ(u∗, δu) =
∫ tf
t0
[
∂H
∂u
]
u∗δudt,
δJ(u∗, δu) =
∫ tf
t0
[H(y∗, u∗ + δu,λ∗)− H(y∗, u∗,λ∗)] dt.
Therefore,
H(y∗, u∗ + δu,λ∗)−H(y∗, u∗,λ∗) ≥ 0,
H(y∗, u∗ + δu,λ∗) ≥H(y∗, u∗,λ∗),
for all admissible variations in u. From this discussion it can be concluded that the
optimal control minimizes the Hamiltonian given in Eq. (2–8). This optimality condition is
called the weak form of Pontryagin’s Minimum Principle and is given as
u∗ = argminuH(y∗, u,λ∗). (2–27)
30
Active control inequality path constraints may cause finite discontinuities in the
control at the entering and exit corners of the constraint activity. Activity in these control
constraints, however, only produces discontinuities in the time derivative of the state and
costate; and in general the state and costate themselves are continuous across such
corners. Furthermore, the Hamiltonian will also be continuous in the presence of these
constraints.
2.1.4 State Inequality Path Constraints
Of primary interest in this research is the case when the inequality path constraint
of Eq. (2–4) is purely a function of the state and the independent variable (that is, the
inequality path constraint is not a function of the control) such as
S(y(t)) ≤ 0, t ∈ [t0, tf ]. (2–28)
For those time intervals where an extremal solution lies on the constraint boundary (and
contrary to the case of a control inequality path constraint), the optimal control cannot be
determined by the active constraint because it is not an explicit function of the control.
While on the constraint boundary, the DAEs describing the system will then have the
form
y(t) =f(y(t), u(t)), (2–29)
0 =Sk(y(t)), t ∈ [t1, t2], (2–30)
where k denotes the active constraint in the time interval [t1, t2] ⊆ [t0, tf ]. Equation (2–30)
is an algebraic constraint that, when satisfied, removes a degree of freedom from the
differential equations defined by Eq. (2–29). Removal of this degree of freedom results
in what the DAE literature refers to as a high-index differential-algebraic equation. In
general, high-index DAE are difficult to solve numerically. Methods for solving high-index
DAEs will be discussed in the following section.
31
Active state inequality path constraint will cause trouble not only when solving an
optimal control problem numerically, but also when solving an optimal control problem
analytically. The constraint activity introduces additional unknowns that cannot be
determined by applying the first-order optimality conditions of Eqs. (2–9)–(2–13).
Specifically, the additional unknowns include the times of the entrance and exit of the
constrained arc and the path constraint multiplier µ(t). Therefore, when solving these
problem analytically, the first-order optimality conditions derived in the previous section
must be modified to account for the constrained arcs. Active state inequality path
constraints will often produce discontinuities in the costate and the Hamiltonian at the
entrance and/or exit of the constraint activity. Furthermore, it is often the case that the
time derivative of the state will be discontinuous at the corners of a constrained arc. The
state and control, however, will generally not be discontinuous even in the presence of
active state inequality path constraints.
2.2 Differential-Algebraic Equations
The numerical solution of DAEs can be far more complicated than the numerical
solution of ordinary differential equations (ODEs). The accuracy of a numerical
method for the solution of a DAE depends upon on the DAE’s solvability, index, and
the consistency of initial conditions [3, 4]. The most general representation of a DAE is
given in the nonlinear implicit form
F(y(t), y(t), u(t)) = 0. (2–31)
The DAE is said to be solvable if a family of unique solutions exists locally. Furthermore,
the index of the DAE is defined as the minimum number of times that all or subsets of
the DAE given by Eq. (2–31) need to be differentiated with respect to time in order to
determine y(t) as a continuous function of the state and control. It is noted that the DAE
index can vary along a solution trajectory of a nonlinear DAE. Finally, the consistency
32
of initial conditions for a DAE system is defined by a set of initial conditions (y0, y0) that
satisfies the extended system (2–31) at time t0.
In order to better understand the difference between a DAE and an ODE, suppose
the DAE given by Eq. (2–31) has the form
F(y, y, u) = 0 = A(t)y+ B(t)y+D(t)u+ e(t). (2–32)
The Jacobian of Eq. (2–32) is defined as
∂F∂y= A(t). (2–33)
If the matrix A(t) is full rank, it is possible to solve Eq. (2–32) for the state as
[A(t)]−10 = [A(t)]−1 (A(t)y+ B(t)y+ D(t)u+ e(t)) ,
y = −[A(t)]−1B(t)y− [A(t)]−1D(t)u− [A(t)]−1e(t), (2–34)
Equation (2–34) is an ordinary differential equation. Therefore, in general, if the system
Jacobian given in Eq. (2–33) is invertible, Eq. (2–31) can be transformed into an ODE of
the form y = f(y, u). If, however, the Jacobian of Eq. (2–33) is singular, then Eq. (2–31)
is a differential-algebraic system which can be written in the semi-explicit form
y(t) =f(y(t), u(t)), (2–35)
0 =C(y(t), u(t)). (2–36)
Furthermore, when the DAE system is in the semi-explicit form of Eqs. (2–35)–(2–36)
and the matrix
G =∂C∂u
is singular, then the system is said to be a high-index DAE system.
One way to understand the solution of a DAE system, and why high-index systems
can be problematic, is by viewing the algebraic Eq. (2–36) as a way of “eliminating” the
control variable such that a standard ODE integration method can be used to obtain
33
a numerical solution. In that case one degree of freedom is removed from the ODEs
for each algebraic equation that is satisfied. For instance, using Newton’s method,
Eq. (2–36) can be solved iteratively for the control, u, as
uk = uk −G−1C, (2–37)
where uk is the new approximation at the k th iterative step. The resulting control could
then be substituted into the ODE of Eq. (2–35) such that
y(t) = f(y, u).
This last ODE could then be solved by employing available numerical ODE solvers.
Although this approach would be time consuming and is not a practical way of obtaining
a solution, it does illustrate the computational challenges associated with high-index
constraints. Namely, if C−1u is rank deficient then not only can the operation given by
Eq. (2–37) not be performed, but the algebraic constraints will also not uniquely specify
all of the degrees of freedom of the system.
2.2.1 Index-Reduction for Differential-Algebraic Equations
Differential Algebraic Equations of index at most one can generally be solved
numerically using methods developed for the solution of ODEs. For systems of index
higher than one, however, such methods may have poor convergence, may converge
to the wrong solution, or may not converge at all [3]. Two general approaches accepted
in the DAE literature exist for obtaining numerical solutions to these types of high-index
systems. The first is through the use of presently available numerical methods and
codes that are modified ODE solvers designed specifically for high-index DAE systems
such as backward-differentiation formulas (BDF). The second is through index reduction
of the system by symbolic manipulation of the DAE equations.
In the scope of this research, it is desired to not only find a numerical solution
to the DAE system but also perform the optimization described by Eqs. (2–1)–(2–4).
34
This optimization is done by transcribing the optimal control problem into a nonlinear
programming problem (NLP). Therefore, specialized codes for solving high-index
systems are not desirable as they would result in a computationally inefficient NLP
structure. Furthermore, the solution structure of the resulting optimal control problem
will often consist of both constrained and unconstrained arcs, making the DAE index
fluctuate throughout the solution. It is desirable, however, to have one DAE solver for
both constrained and unconstrained segments of the trajectory. For these reasons, the
method of index reduction might be preferred over specialized approaches for solving
the high-index DAE that result from state inequality constrained problems.
The method of index reduction is formulated as follows. Given the DAE
y(t) =f(y(t), u(t)), (2–38)
0 =S(y(t)), (2–39)
it is clear that the matrix
G =∂S∂u
is rank deficient. Therefore the system given by Eqs. (2–38)–(2–39) is a high-index DAE
system. Now if S = 0, then it must also be true that all its time derivatives are zero.
Therefore, Eq. (2–39) can be differentiated with respect to time such that
0 =∂S∂yy +
∂S∂uu,
0 =∂S∂yf(y, u) +
∂S∂uu.
(2–40)
This differentiation and back-substitution procedure can be repeated until the control
appears explicitly in the algebraic constraints. If r time derivatives are needed for the
derivative of the constraint with respect to the control
G(r) =∂rS∂ur
35
to be full rank, then the constraints are said to have index r. Once G(r) is full rank, the
DAE system of Eqs. (2–38)–(2–39) can be rewritten equivalently as
y(t) =f(y(t), u(t)),
u(t) =[G(r)]−1(
∂S∂yf(y, u)
)
.(2–41)
Equation (2–41) is an ODE system that can be solved using well-known algorithms.
Therefore, in general, the DAE index is the minimum number of times that the original
constraints must be differentiated with respect to time in order to obtain an ODE.
In the context of dynamic optimization, index reduction is used only to the point
where the optimal control can be explicitly defined by the active algebraic constraint.
Therefore, one less time derivative need be taken, such that the resulting DAE is defined
by
y(t) =f(y(t), u(t)),
0 =dq
dtqSk(y(t), u(t)),
(2–42)
where k denotes the active constraint. In the dynamic optimization literature, the
algebraic constraint given in Eq. (2–42) is termed a qth order constraint. Therefore, in
general, the order of a state inequality path constraint is the minimum number of times
that constraint must be differentiated with respect to time before the control appears
explicitly in the expression. Furthermore, the order of a constraint and the index of a
DAE are related by the expression q = r − 1. Finally, as the algebraic constraints are
differentiated, the intermediate q − 1 time derivatives are not discarded. Instead, they
are evaluated at the initial time t0 and used as a set of consistent initial conditions for the
new DAE given by Eq. (2–42):
S(y(t))
d
dtS(y(t))
...
d (q−1)
dt(q−1)S(y(t))
t=t0
= 0. (2–43)
36
2.2.2 Gaussian Quadrature Collocation Method for Solutions of High-Index DAEs
Suppose the state y from Eqs. (2–35)–(2–36) is approximated as
y(τ) ≈ Y(τ) =N+1∑
i=1
YiLi(τ), (2–44)
where τi , (i = 1, ... ,N + 1) are the discretization points in the domain τ ∈ [−1,+1],
Yi ∈ Rn is a row vector of the state approximated at τi , and the Lagrange polynomials
Li(τ) are defined as
Li(τ) =N+1∏
j=1
j 6=i
τ − τjτi − τj
. (2–45)
It is noted that the domain t ∈ [t0, tf ] can be transformed to τ ∈ [−1,+1] through the
affine transformation
t =tf − t02
τ +tf + t02. (2–46)
The time derivative of the state approximation given in Eq. (2–44) is then given as
y(τ) ≈ Y(τ) =N+1∑
i=1
Li(τ)Yi . (2–47)
While any set of points τi can be used as support points for the state approximation,
it has been shown that non-uniform discretization points obtained from the roots of
orthogonal polynomials such as Chebyshev or Legendre polynomials will minimize
the interpolation error associated with Runge phenomenon. In this research, the
Legendre-Gauss-Radau (LGR) points plus the initial point −1 are used as the support
points. Applying the time derivative of Eq. (2–47) at the N LGR points, (τ1, ... , τN), gives
y(τj) ≈ Y(τj) =N+1∑
i=1
Lj(τi)Yi =N+1∑
i=1
DjiYi , (j = 1, ... ,N), (2–48)
where Dji , (j = 1, ... ,N); (i = 1, ... ,N +1) is a (N)× (N + 1) matrix of known coefficients
known as the LGR differentiation matrix.
37
The DAE given by Eqs. (2–35)–(2–36) can then be approximated by the set of
algebraic equations
N+1∑
i=1
DjiYi −tf − t02f(Yj ,Uj) =0, (j = 1, ... ,N), (2–49)
C(Yj ,Uj) =0, (j = 1, ... ,N), (2–50)
where Uj ∈ Rm is a row vector of the control approximation at τj . Eqs. (2–49)–(2–50) are
a set of nonlinear equations that can be solved for the unknowns Yi , (i = 1, ... ,N + 1),
Uj , (j = 1, ... ,N) using known numerical methods.
Example: In order to illustrate the benefits of index reduction, consider the following
simple example where a double integrator has its state constrained:
ProblemA
x(t) =v(t),
v(t) =u(t),
x(t) =ℓ,
(2–51)
where (x , v) is the state, u is the control and t ∈ [−1,+1]. Because the algebraic
constraint must be differentiated twice with respect to time before the control appears
explicitly in the expression, Eq. (2–51) is an example of an index-3 DAE and a second-
order constraint. Index reduction results in the system
ProblemB
x(t) =v(t) , x(−1) =ℓ,
v(t) =u(t) , v(−1) =0,
u(t) =0,
(2–52)
which is equivalent to the original system given in Eq. (2–51). The analytic solution to
the DAE in Eq. (2–51) is given as
x(t) = ℓ, v(t) = 0,
u(t) =0.
(2–53)
38
Because the analytic solution to Eq. (2–51) is constant in the entire domain, it is
reasonable to expect small errors in the solution using low-degree polynomial approximations
(that is, a small number of collocation points).
ag
2 4 6 8 10 12 14 16 18
-2
-4
-6
-8
-10
-12
-14
-16
-18
Number of Collocation Points
log10
Abs
olut
eE
rror
inx(t)
Problem AProblem B
Figure 2-1. Base ten logarithm of the maximum absolute error in x(t) as a function ofthe number of LGR collocation points for problems A and B.
Both problems A and B, defined by Eqs. (2–51) and (2–52), respectively, were
solved using the method described by Eqs. (2–49)–(2–50). Figure 2-1 shows the
base ten logarithm of the maximum absolute error in the x(t) component of the state
as a function of the number of LGR collocation points. It can be seen that using the
formulation given in Eq. (2–52) results in much smaller errors as compared with
the formulation in Eq. (2–51). Furthermore, Eq. (2–51) requires a large number of
collocation points to achieve a reasonable accuracy tolerance of 10−8, whereas the
numerical solution of Eq. (2–52) has a much smaller error of approximately 10−16
regardless of the degree of the approximating polynomial. The errors found for the
second component of the state, v(t), and the control, u(t), are shown in Table 2-1. From
these results it can be seen that the disparity in the accuracy of the solution between
39
the formulations of Eqs. (2–51) and (2–52) is even larger for the variables that are
not explicitly defined by the algebraic constraint in the original problem formulation.
Furthermore, it can be seen that an accuracy of O(10−5) can still be attained without
performing index-reduction. Therefore it can be concluded that index reduction greatly
reduces the numerical error when solving high-index DAE systems, but when no
index-reduction is performed it is still possible to achieve a reasonable level of accuracy
when solving high-index DAE of order three.
Table 2-1. Absolute maximum error in v(t) and u(t) for problems A and B.
Number of LGR Pointsv(t) Max Absolute Error u(t) Max Absolute ErrorProblem A Problem B Problem A Problem B
3 3.38× 10−3 0 7.97× 10−3 06 1.29× 10−4 0 9.79× 10−4 09 1.80× 10−5 0 2.57× 10−4 0
12 5.49× 10−6 0 8.08× 10−5 015 2.57× 10−6 0 3.98× 10−5 018 1.13× 10−6 0 2.57× 10−5 0
2.3 State Inequality Path Constrained Optimal Control Problems
From the discussion of Section 2.2, it is clear that optimal control problems with
active state inequality constraints lead to high-index DAEs and are difficult to solve
numerically. Furthermore, it was shown that when solving these problems analytically
the first-order optimality conditions of Section 2.1 must be modified in order to account
for the extra unknowns introduced by the constraint activity. Many methods are available
in the literature for deriving the necessary conditions for optimality of a state-inequality
constrained problem. Of these methods, three of them will be discussed here [7, 8, 15].
The first method is called indirect adjoining [15]. In indirect adjoining the index-reduction
of the high-index system of DAEs is taken into account, as first derived by [7]. Indirect
adjoining results in a costate that has discontinuities at the entrance of the constraint
activity. The second method presented for solving state inequality path constrained
problems is called direct adjoining [15]. In direct adjoining, the state inequality path
constraint is directly augmented to the cost and the first-order optimality conditions are
40
derived. Using direct adjoining the costate may be discontinuous at the entrance or
exit of a constrained arc because of jumps in the state constraint multiplier. The third
method presented is called indirect adjoining with continuous multipliers [15]. In indirect
adjoining with continuous multiplers, the costate discontinuity is “subtracted out” from
the costate dynamics, yielding a costate that is continuous even in the presence of state
constraint activity.
Consider again the optimal control problem of Section 2.1, restated here such
that the inequality path constraint is a function of only the state. Determine the state,
y(t) ∈ Rn, and the control u(t) ∈ Rm, that minimize the cost functional
Φ(y(tf )) +
∫ tf
t0
g(y(t), u(t))dt (2–54)
subject to the dynamic constraint
y(t) = f(y(t), u(t)), (2–55)
the boundary condition
φ(y(t0)) = 0, (2–56)
and the state inequality path constraint
S(y(t)) ≤ 0. (2–57)
2.3.1 Indirect Adjoining
The first-order optimality conditions of the state inequality path constrained problem
of Eqs. (2–54)–(2–57) are now derived by applying the method of indirect adjoining.
It was previously shown in Section 2.2.1 that high-index DAE are difficult to solve
numerically. Thus, in the indirect adjoining method of deriving the first-order optimality
conditions, index reduction is performed on the high-index DAE system that results from
the state constraint activity, resulting in a modified optimal control problem formulation.
In order to simplify the analysis presented here, a scalar inequality path constraint is
41
considered. Generality is not lost, however, because index reduction can be applied
to a vector inequality path constraint by considering each component individually.
Furthermore, it is assumed the state constraint is active in an interval [t1, t2] such that
S(y(t)) = 0 ∈ [t1, t2] ⊆ [t0, tf ]. On the constrained arc, the state constraint must be
satisfied such that
S(y(t)) = 0, t ∈ [t1, t2]. (2–58)
Because Eq. (2–58) must be satisfied on the optimal solution, all derivatives of the
path constraint in [t1, t2] must also be zero. Performing index-reduction as described in
Section 2.2.1 , Eq. (2–58) is differentiated q times, where q is the lowest derivative of
S that is an explicit function of the control. The intermediate time derivatives are then
defined as
π(y(t)) ≡
S(y(t))
d
dtS(y(t))
...
d (q−1)
dt(q−1)S(y(t))
, t ∈ [t1, t2]. (2–59)
These intermediate time derivatives are evaluated at the entrance of the constrained
arc, t1, and act as consistent initial conditions for the modified DAE. Reference [7]
denotes these constraints as tangency conditions. Physically, the interpretation of these
conditions is that since the constraint function is controllable only by changing its qth
time derivative, there does not exist a finite control for which the system will remain on
the constraint boundary unless the tangency conditions are also satisfied [10].
Consequently, the state inequality path constraint given by Eq. (2–57) can be
replaced by the tangency conditions along with the following state and control inequality
path constraint:
d (q)
dt(q)S(y(t), u(t)) = C(y(t), u(t)) ≤ 0, t ∈ [t1, t2]. (2–60)
42
The optimal control problem of Eqs. (2–54)–(2–57) can then be modified as follows to
account for the DAE index reduction as follows. Minimize the cost functional
Φ(y(tf )) +
∫ tf
t0
g(y(t), u(t))dt (2–61)
subject to the dynamic constraint
y(t) = f(y(t), u(t)), (2–62)
the boundary condition
φ(y(t0)) = 0, (2–63)
the tangency conditions
π(y(t1)) = 0, (2–64)
and the state and control inequality path constraint
C(y(t), u(t)) ≤ 0, t ∈ [t1, t2]. (2–65)
The first-order optimality conditions of the modified problem of Eqs. (2–61)–(2–65) can
be obtained using the calculus of variations as previously described in Section 2.1. They
are given as the original constraints of Eqs. (2–62)–(2–65) along with the conditions
0 =∂H(y, u, λ, µ)
∂u, (2–66)
˙λ⊤ = −∂H(y, u, λ, µ)∂y
, (2–67)
λ⊤(t0) = ψ⊤ ∂φ
∂y(t0), (2–68)
λ⊤(t+1 ) = λ
⊤(t−1 ) + η
⊤(t1)∂π
∂y(t1), (2–69)
λ(t+2 ) = λ(t−2 ), (2–70)
λ⊤(tf ) =
∂Φ
∂y(tf ), (2–71)
S(y) ≤ 0, µ ≤ 0, µ⊤S(y) = 0. (2–72)
43
In the conditions of Eqs. (2–66)–(2–72), λ(t) ∈ Rn is the costate, µ(t) ∈ R is the
Lagrange multiplier associated with the path constraint of Eq. (2–65), ψ ∈ Rp is the
Lagrange multiplier associated with the boundary condition of Eq. (2–63), η ∈ Rq is
the Lagrange multiplier associated with the tangency conditions of Eq. (2–64), and the
augmented Hamiltonian is defined as
H = g(y, u) + λ⊤
f(y, u)− µ⊤C(y, u). (2–73)
The aforementioned approach is called indirect adjoining because the qth derivative of
the state constraint is adjoined to the Hamiltonian rather than the state constraint itself.
It can be seen that by applying these conditions, active state inequality path
constraints may lead to a discontinuous costate at the entrance, but not at the exit, of
the constrained arc, as can be seen by Eqs. (2–95) and (2–96). Furthermore, note that
the costate discontinuities at the entrance of the constrained arc are quantified by the
costate jump conditions of Eq. (2–95). Thus, Eq. (2–95) acts as terminal conditions for
the interval preceding the constrained arc, and the optimal control problem can be seen
as a three-point boundary value problem.
Although the method of indirect adjoining offers the obvious benefits of index-reduction,
as stated in Section 2.2.1, in practice this technique may be cumbersome to apply.
In particular, index-reduction requires a reformulation of the problem in which the
derivatives of the state inequality path constraint must be taken analytically. Furthermore,
the solution structure of the problem (that is, the sequence of constrained and
unconstrained arcs) must be known a priori. Because solutions to optimal control
problems are often obtained through an automated process (such as a mesh-refinement
technique), indirect adjoining is an impractical solution method.
It was seen in Section 2.2.1 that it is still possible to obtain an accuracy of O(10−5)
when estimating solutions to high-order DAE of up to order three. Because most
aerospace engineering applications generally do not formulate state inequality path
44
constraints of order higher than two, it is reasonable to explore alternate methods of
solving state inequality path constrained optimal control problems that do not involve
index-reduction. In Sections 2.3.2 and 2.3.3 two such methods will be discussed:
the method of direct adjoining and the method of indirect adjoining with continuous
multipliers.
2.3.2 Direct Adjoining
Using the method of direct adjoining, the first-order optimality conditions of the state
inequality path constrained problem of Eqs. (2–54)–(2–57) can be derived in the same
manner as was done in Section 2.1. These conditions are given as
y =f(y, u), 0 = φ(y(t0)), (2–74)
0 =∂H(y, u,λ,µ)
∂u, (2–75)
λ⊤=∂H(y, u,λ,µ)
∂y, (2–76)
λ⊤(t0) =ψ⊤ ∂φ
∂y(t0), (2–77)
λ⊤(tf ) =∂Φ
∂y(tf ), (2–78)
S(y) ≤0, µ ≤ 0, µ⊤S(y) = 0, (2–79)
where the augmented Hamiltonian is given as
H(y, u,λ,µ) = g(y, u) + λ⊤f(y, u)− µ⊤S(y). (2–80)
The first-order optimality conditions of Eqs. (2–74)–(2–79) are necessary conditions
for optimality of a pure state inequality path constrained problem. They are, however,
insufficient to fully determine an extremal solution. In particular, the state constraint
multipliers given by Eq. (2–79) may be discontinuous at the entrance and exit of
a constrained arc. Because the state constraint multipliers and the costate are
related through the costate dynamics of Eq. (2–76), the discontinuities in the state
constraint multiplier will in turn cause discontinuities in the costate. To quantify these
45
discontinuities, the costate dynamics of Eq. (2–76) can be integrated as
λ⊤(t+1 ) = λ⊤(t0)−
∫ t1
t0
[
∂g(y, u)
∂y+ λ⊤(t)
∂f(y, u)∂y
]
dt +
∫
(t0,t1]
µ⊤(t)∂S(y)∂ydt, (2–81)
where t0 ≤ t1 ≤ tf . Now, define the variable ν(t) = −µ(t). The integral of Eq. (2–81)
can then be re-written as
λ⊤(t+1 ) = λ⊤(t0)−
∫ t1
t0
[
∂g(y, u)
∂y+ λ⊤(t)
∂f(y, u)∂y
]
dt −∫
(t0,t1]
∂S⊤(y)
∂ydν(t). (2–82)
Because ν(t) is a function of a bounded variation, it can be decomposed uniquely as
ν(t) = ν1(t) + ν2(t), (2–83)
where ν1(t) is absolutely continuous with respect to t and ν2(t) is singular with
respect to t. Therefore, the second integral on the right-hand side of Eq. (2–82) can
be expressed as
∫
[t0,t1]
∂S⊤(y)
∂ydν(t) =
∫ t1
t0
∂S⊤(y)
∂yν1(t)dt +
∫
(t0,t1]
∂S⊤(y)
∂ydν2(t). (2–84)
Also, ν(t) is monotone so that it can have at most countably many jumps. Moreover,
it is reasonable to assume that ν(t) is sufficiently well behaved to have a piecewise
continuous derivative. In that case the second integral on the right-hand side of
Eq. (2–84) can be expressed as
∫
[t0,t1]
∂S⊤(y)
∂ydν2(t) =
∑
τi∈(t0,t1]
∂S⊤(y(τi))
∂y[ν(τ+1 )− ν(τ−i )], (2–85)
where τi are the points of discontinuity of ν(t). Next, defining the costate discontinuity
as
η(τ) = ν(τ−)− ν(τ+), (2–86)
the integral of Eq. (2–81) can be expressed as
λ⊤(t+1 ) = λ⊤(t0)−
∫ t1
t0
[
∂g(y, u)
∂y+ λ⊤(t)
∂f(y, u)∂y
− µ⊤(t)∂S(y)∂y
]
dt+∑
τi∈(t0,t1]
η(τ)⊤∂S(y(τi))
∂y.
46
Therefore, the costate dynamics can be expressed as
− λ⊤=
∂g
∂y+ λ⊤ ∂f
∂y− µ⊤∂S
∂y. (2–87)
Furthermore, let t = τ denote an entry time into a constrained arc, an exit time from
a constrained arc, or a single contact point in which S(y(τ)) = 0. Then the costate
trajectory may have a discontinuity given by the following jump conditions
λ⊤(τ+) = λ⊤(τ−) + η(τ)⊤∂S(y(τi))
∂y. (2–88)
Because ν(t) ≥ 0, the function ν(t) is non-decreasing in the solution domain,
and thus η(k) ≤ 0. Furthermore, the condition 〈η(k),S(Y(Tk))〉 = 0 must hold
true. Thus, using direct adjoining, the first-order optimality conditions for the state
inequality path constrained optimal control problem of Eqs. (2–54)–(2–57) are given by
Eqs. (2–74)–(2–76) along with the costate discontinuity conditions of Eq. (2–88).
The first-order optimality conditions derived here are very similar to the conditions
derived in Section 2.3.1. However, it is noted that in the conditions derived in Section
2.3.1 the qth derivative of the state inequality path constraint is enforced on the
constrained arc, whereas the original undifferentiated path constraint was enforced
in the problem formulation of this section. Moreover, the optimality conditions stated in
Section 2.3.1 normalize the jump conditions such that the costate is discontinuous only
at the entry time of the constrained arc, while the conditions stated in this section make
no such distinction.
2.3.3 Indirect Adjoining With Continuous Multipliers
The third method for deriving the necessary conditions for optimality of a state
inequality path constrained optimal control problem is the method of indirect adjoining
with continuous multipliers. Using the method of indirect adjoining with continuous
multipliers yields a costate that is continuous even in the presence of state inequality
path constraints. Because discontinuities are difficult to approximate numerically, the
47
method of indirect adjoining with continuous multipliers offers an advantage over the
methods of direct and indirect adjoining which approximate a discontinuous costate.
The first-order optimality conditions for the state inequality path constrained optimal
control problem given by Eqs. (2–54)–(2–57) are now derived by using the method of
indirect adjoining with continuous multipliers. Similar to the approach used in Section
2.3.2, the costate dynamics of Eq. (2–76) can be integrated to give
λ⊤(t+1 ) = λ⊤(t0)−
∫ t1
t0
[
∂g(y, u)
∂y+ λ⊤(t)
∂f(y, u)∂y
]
dt +
∫
(t0,t1]
µ⊤(t)∂S(y)∂ydt,
where t0 ≤ t1 ≤ tf . Now, let ν(t) = −µ(t). Furthermore, because µ(t) ≤ 0, ν(t) must
be a non-decreasing function of time. The costate discontinuity can now be “subtracted”
by defining a new costate p(t) such that
p⊤(t) = λ⊤(t) + ν⊤(t)∂S(y)∂y. (2–89)
The Hamiltonian can be defined in terms of the continuous costate p(t) such that
H(y, u, p,ν) = g(y, u) + p⊤f(y, u)− ν⊤S(y, u). (2–90)
It can be seen that this expression can be decomposed to the Hamiltonian by substituting
the relationship
S(y) ≡ ∂S(y)
∂y· y = ∂S(y)
∂y· f(y, u).
into Eq. (2–90) such that
H(y, u,λ) = g(y, u) + λ⊤f(y, u)←→ H(y, u, p,ν). (2–91)
48
The first-order optimality conditions are then given in terms of the continuous
costate as Eqs. (2–55)–(2–57) along with the conditions
0 =∂H(y, u, p,ν)
∂u, (2–92)
p⊤ = −∂H(y, u, p,ν)∂y
, (2–93)
p⊤(t0) = ψ⊤ ∂φ
∂y(t0)+ ν(t0)
⊤∂S(y(t0))∂y
, (2–94)
λ⊤(tf ) =∂Φ
∂y(tf )+ ν(tf )
⊤∂S(y(tf ))∂y
. (2–95)
ν(tf ) ≤ 0, ν ≥ 0, C(y) ∈ N (ν), (2–96)
Furthermore, let C(Rq) denote the space of continuous functions mapping [t0, tf ] to Rq.
Assuming ν is Lipschitz continuous and non-decreasing with ν(tf ) ≤ 0, the set-valued
map N (ν) is defined as
N (ν) = {z ∈ C(Rq) : z ≤ 0, ν⊤z = 0,ν⊤(tf )z(tf ) = 0}.
2.4 Numerical Properties of Orthogonal Collocation Methods
Numerically solving an optimal control problems require knowledge of a number of dif-
ferent concepts. In particular, two concepts are important in constructing a discretized finite-
dimensional optimization problem from a continuous-time optimal control problem: polynomial
approximation and numerical integration. Polynomial approximation is important because the
infinite-dimensional continuous functions (that is, the state components) of the optimal control
problem are approximated by a finite-dimensional Lagrange polynomial basis. Numerical integra-
tion methods are important because the dynamic constraints and the cost must be integrated as
part of the optimization. In this section, these important mathematical concepts that are used to
transcribe a continuous-time optimal control problem to a nonlinear programming problem (NLP)
using orthogonal collocation at Gaussian quadrature points are reviewed.
2.4.1 Function Approximation and Interpolation
Collocation methods for solving optimal control problems approximate the continuous
functions of time at a set of support points. In this research, Lagrange polynomials are used to
49
interpolate the state and the costate. Specifically, given a continuous function y(t), there exists
a unique polynomial Y (t) of degree N − 1 which uses N arbitrary support points (t1 ... , tN) ∈
[t0, tf ], such that
Y (ti) = y(ti), (i = 1, ... ,N). (2–97)
Furthermore, the unique polynomial can be described by Lagrange interpolation such that
Y (t) =
N∑
i=1
yiLi(τ), (2–98)
where yi = y(ti) and Li(t) are the Lagrange polynomials
Li(t) =
N∏
j=1
j 6=i
τ − τj
τi − τj. (2–99)
One important property of Lagrange interpolating polynomials is that they satisfy the isolation
property
Li(tj) =
1 for i = j ,
0 for i 6= j .(2–100)
Eq. (2–100) is important for this research because it leads to a sparse nonlinear program
transcription of the optimal control problem being solved. Thus the isolation property leads to a
transcription which can be efficiently solved by a nonlinear program solver. The error associated
with the Lagrange approximation of a Nth times differentiable function is given by
y(t)− Y (t) = (t − t1) ... (t − tN)N!
yN(ζ), (2–101)
where yN(ζ) is the Nth derivative of the function y(t) evaluated at a point ζ ∈ [t0, tf ]. It is
seen from this error formula that the error is exactly zero at any of the support points of the
interpolating polynomial. Furthermore, since the error is a direct function of the Nth derivative
of y(t), the Lagrange interpolation approximation using N support points will be exact for
polynomials of degree at most N − 1.
Although smooth functions can be accurately approximated as stated above, the behavior
of the interpolation error as N approaches infinity for non-smooth functions becomes erratic, a
behavior called Runge phenomenon. Runge phenomenon is characterized by large amplitude
50
oscillations in the interpolating polynomials near the domain boundaries when the support points
are uniformly distributed and N becomes large. In order to understand Runge phenomenon
better, consider the following function defined in the domain τ ∈ [−1,+1]
y(τ) =1
1 + 50τ2. (2–102)
The function in Eq. (2–102) was approximated using N uniformly distributed support points for
a basis of Lagrange interpolating polynomials and the result for N = 11 and N = 41 is shown
in Fig. (2-2). The approximations for N = 11 and N = 41 points correspond to polynomials
of degree 10 and 40. It can be seen that as the number of support points is increased, the
error in the interpolation becomes larger near the end points due to the large oscillations in
the polynomial approximations. Thus the interpolation does not converge to the function being
approximated as N is increased.
2.4.1.1 Family of Legendre-Gauss points
One way to rectify the effect of Runge phenomenon is to use a non-equally spaced set
support points. In this research the support points used are the points obtained from the roots of
a Legendre polynomial and/or linear combinations of a Legendre polynomial and its derivatives.
These points are known to have a distribution that minimizes Runge phenomenon. In particular,
two sets of points are of interest: the Ledendre-Gauss (LG) points, and the Legende-Gauss-
Radau (LGR) points. Both these sets of points are defined on the domain τ ∈ [−1,+1], but
differ significantly in that the LG points include neither of the endpoints whereas the LGR points
include one of the endpoints. In addition, the LGR points are asymmetric relative to the origin
and are not unique because they can be defined using either the initial or the terminal point. The
LGR points that include the terminal point are often called the flipped LGR points. The flipped
LGR points are a mirror image of the LGR points that include the initial point in the domain.
These points can be calculated as follows. Denoting the Nth degree Legendre polynomial by
PN(τ) =1
2NN!
dN
dτN
[
(τ2 − 1)N]
,
51
0
0 0.2 0.4
0.5
0.6 0.8
1
1-0.8 -0.6 -0.4 -0.2
-0.5
-1-1
2
3
2.5
3.5
1.5
τ
y(τ)
y(τ)
Y (τ)
(A) Approximation of the function given by Eq. (2–102) using 11uniformly spaced support points.
0
0 0.2 0.4
0.5
0.6 0.8 1-0.8 -0.6 -0.4 -0.2
-0.5
-1.5
-2
-1
-1-2.5
6x10
τ
y(τ)
y(τ)
Y (τ)
(B) Approximation of the function given by Eq. (2–102) using 41uniformly spaced support points.
Figure 2-2. Approximation of the function given by Eq. (2–102) using 11 and 41uniformly spaced support points.
52
0 0.2 0.4 0.6 0.8 1-0.8 -0.6 -0.4 -0.2-1τ
LGR
LGR-f
LG
Figure 2-3. Distribution of Legendre-Gauss, Legendre-Gauss-Radau, and FlippedLegendre-Gauss-Radau Points in the domain τ ∈ [−1,+1].
the LG points are defined as the roots of PN(τ) and the LGR points are defined as the roots
obtained from PN−1(τ) + PN(τ). Fig. (2-3) is a schematic representation of the LG, LGR, and
flipped LGR points in the domain [−1,+1] for N = 5.
The function given by Eq. (2–102) is now approximated using a basis of Lagrange interpo-
lating polynomial with LG and LGR support points. A function y(τ) can be approximated in the
domain [−1,+1] using a basis of Lagrange interpolating polynomials and the LG points as
Y (τ) =
N+1∑
i=0
y(τi)L)i(τ) (2–103)
where (τ1, ... , τN) are the N LG points, τ0 = −1, and τN+1 = +1. Similarly, the same function
y(τ) can be approximated in the domain [−1,+1] using a basis of Lagrange interpolating
polynomials and the LGR points as
Y (τ) =
N+1∑
i=1
y(τi)L)i(τ) (2–104)
where (τ1, ... , τN) are the N LG points, and τN+1 = +1. Fig. (2-4) shows the results of the
function approximation when N = 11 and N = 41 LG support points are used. Furthermore,
Fig. (2-5) shows the results of the function approximation when N = 11 and N = 41 LGR support
53
points are used. It can be seen that as the number of support points is increased the polynomial
approximation converges to the true function. In order to understand the behavior of the error,
Fig. (2-6) plots the base ten logarithm of the maximum absolute error defined as
Ey = log10 ||Y (τ)− y(τ)||∞
as a function of N for an approximation using Lagrange interpolating polynomials with uni-
formly spaced, LG, and LGR support points. It can be seen that as the number of support
points increases, the polynomial approximation which uses uniformly spaced support points
diverges from the function being approximated, whereas using LG and LGR support points the
approximation converges to the true function.
2.4.2 Numerical Integration
Numerical integration plays a key role when numerically approximating the solution to a
continuous-time optimal control problem. In particular, an optimal control problem requires that a
cost be minimized (or maximized), and this cost is integrated across the time domain of interest.
Furthermore, the dynamic constraints must be integrated. Therefore, an optimal control problem
optimizes and integrates simultaneously. A summary of numerical integration methods will now
be given in order to better understand the methods used in this research.
2.4.2.1 Low-order integrators
A common technique used to approximate the integral of a function is to use low-degree
polynomial approximations. The approximation of the integral is then found by summing the low-
order method integral approximations of each subinterval. One commonly used technique that
uses low-order methods is called the composite trapezoid rule [2]. The composite trapezoid rule
divides the domain of interest into many uniformly distributed sub-intervals and approximates
the function to be integrated with a straight line that passes through the function at the endpoints
of the subinterval. Therefore, for N approximating subintervals, the composite trapezoid rule is
given by
∫ tf
t0
f (t) dt ≈ tf − t02N
[f (t0) + 2f (t1) + 2f (t2) + ... + 2f (tN−1) + f (tN)] , (2–105)
54
1.2
0
0
0.2
0.2
0.4
0.4
0.6
0.6
0.8
0.8
1
1-0.8 -0.6 -0.4-0.2
-0.2-1τ
y(τ)
y(τ)
Y (τ)
(A) Approximation of the function given by Eq. (2–102) using 11Legendre-Gauss points.
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6
0.6
0.7
0.8
0.8
0.9
1
1-0.8 -0.6 -0.4 -0.2-1τ
y(τ)
y(τ)
Y (τ)
(B) Approximation of the function given by Eq. (2–102) using 41Legendre-Gauss points.
Figure 2-4. Approximation of the function given by Eq. (2–102) using 11 and 41Legendre-Gauss points.
55
1.2
0
0
0.2
0.2
0.4
0.4
0.6
0.6
0.8
0.8
1
1-0.8 -0.6 -0.4-0.2
-0.2-1τ
y(τ)
y(τ)
Y (τ)
(A) Approximation of the function given by Eq. (2–102) using 11Legendre-Gauss-Radau points.
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6
0.6
0.7
0.8
0.8
0.9
1
1-0.8 -0.6 -0.4 -0.2-1τ
y(τ)
y(τ)
Y (τ)
(B) Approximation of the function given by Eq. (2–102) using 41Legendre-Gauss-Radau points.
Figure 2-5. Approximation of the function given by Eq. (2–102) using 11 and 41Legendre-Gauss-Radau points.
56
0
0 30 40 50 60 70 80 90 100
-5
10
10-10
20
20
15
5
N
Ey
Uniform
LG
LGR
Figure 2-6. Base ten logarithm of infinity norm error as a function of number of supportpoints, N , for approximating the function given by Eq. (2–102).
where (t0, ... , tN) are the subinterval boundaries, or grid points, at which the function is being
evaluated. In order to demonstrate the use of the composite trapezoid rule, consider approximat-
ing the integral of the function f (τ) in the domain τ ∈ [−1,+1]:
f (τ) =
∫ +1
−1exp(τ) dτ . (2–106)
Figure (2-7) shows a graphical representation of the trapezoid rule approximation using three
intervals. Furthermore, Fig. (2-8) shows the base ten logarithm error of this integration as a
function of the base ten logarithm number of approximating intervals. It can be seen that the
convergence of this method is slow as 105 subintervals are necessary to reach an error of
O(10−10).
2.4.2.2 Gaussian quadrature
In contrast with low-order integrators such as the composite trapezoid rule, a Gaussian
quadrature is a high accuracy integrator which displays exponential convergence when approxi-
mating the integral of smooth functions. Gaussian quadrature rules approximate the integral of a
57
00 0.2 0.4
0.5
0.6 0.8
1
1-0.8 -0.6 -0.4 -0.2-1
2
3
2.5
1.5
f (τ)
f(τ)
Approx.
τ
Figure 2-7. Approximation of the integral of the function given by Eq. (2–106) using a 4interval Trapezoid rule.
−4
2 4 4.5 5
-5
32.5 3.5
-6
-7
-8
-9
-10
-11
log10
Err
or
log10 Number of Approximating Intervals
Figure 2-8. Base ten logarithm error of the integral of the function given by Eq. (2–106)as a function of the base ten logarithm number of approximating intervals.
58
function by evaluating the expression
∫ tf
t0
f (t) dt ≈N∑
i=1
wi f (τi), (2–107)
where wi are the quadrature weights associated with the set of points chosen to approximate the
integration. The three sets of points defined by Gaussian quadrature are the Legendre-Gauss-
Lobatto (LGL) points, the Legendre-Gauss-Radau (LGR) points, and the Legendre-Gauss (LG)
points. The LGL, LGR, and LG quadrature rules are exact for polynomials of degree at most
2N − 3, 2N − 2, and 2N − 1, respectively. In this research the LG and LGR points are used.
The N LG points are the roots of the Nth degree Legendre polynomial PN(τ), and the
corresponding LG quadrature weights are given as
wi =2
1− τ2i[PN(τi)]2
, (i = 1, ... ,N).
Similarly, the N LGR points are computed from the roots of PN(τ) + PN−1(τ), and the corre-
sponding LGR quadrature weights are given as
w1 =2
N2,
wi =1
(1− τi)[(PN−1(τi)]2, (i = 2, ... ,N).
Finally, the flipped LGR points and weights are simply the negative of the LGR points.
The accuracy of the LG and LGR quadrature methods can be seen from the function
f (τ) = exp(τ) in the domain τ ∈ [−1,+1]. Figure 2-9 shows the base ten log of the error
for the approximation of the integral in Eq. (2–106) as a function of N. It can be seen that the
convergence rate using Gaussian quadrature is exponential. Furthermore, it is seen that N = 6
LG or LGR points results in an error of less than O(10−10). For comparison, this same error
of O(10−10) required 105 subintervals using the composite Trapezoid rule. Thus, the benefits
of using Gaussian quadrature over low-order integrators when approximating the integral of a
smooth function is evident.
2.5 Orthogonal Collocation for the Solution of Optimal Control Problems
It is now possible to combine the concepts described in Section 2.4 in order to develop a
method to approximate the solution of the continuous-time optimal control problem of Section
59
6 7 8 9 10-16
-14
-4
-2
4 53
-6
-8
-10
-12
N
LG Points
LGR Points
log10
Err
or
Figure 2-9. Base ten logarithm of the error in the integration of the function given byEq. (2–106) as a function of number of LG and LGR Points Used.
2.1. In this section the methods of orthogonal collocation at both Legendre-Gauss (LG) and
Legendre-Gauss-Radau (LGR) points are described. Both these methods for approximating
solutions to optimal control problems can be used as either global collocation methods or
variable-order collocation methods. Global collocation methods use one single polynomial
approximation to collocate the differential-algebraic equations over the entire domain. Global
collocation at LG and LGR points is advantageous when solving problems whose solutions are
smooth, because the LG and LGR methods converge exponentially. When the solution is not
smooth, however, the convergence rate is significantly lower. In the case where the optimal
solution lies on a constrained arc for a subset of the solution domain, non-smooth features in the
solution state and/or control can occur. For such problems, it will thus be beneficial to employ
variable-order LG or LGR collocation. In a variable-order collocation scheme the solution domain
is divided into a mesh, and the degree of the approximating polynomial (that is, the number of
LG or LGR collocation points) in each mesh interval is allowed to vary. This method is useful
because it allows for capturing non-smoothness in the solution domain at interval boundaries.
Because both the LG and the LGR set of points are defined on the domain τ ∈ [−1,+1], the
following affine transformation will be used to map the time domain t ∈ [t0, tf ] to τ ∈ [−1,+1]
60
when using global collocation:
t =tf − t02
τ +tf + t02. (2–108)
Furthermore, it is noted that
d t
dτ=tf − t02
≡ h2, h = tf − t0.
When using variable-order collocation, the time domain t ∈ [t0, tf ] is divided into a mesh
consisting of K mesh intervals where the mesh points are t0 = T0 < T1 < · · · < TK−1 < TK =
tf , and the corresponding mesh intervals are [Tk−1,Tk ], (k = 1, ... ,K). Therefore each mesh
interval can be mapped to the domain τ ∈ [−1,+1] through the affine transformation
t =tk − tk−12
τ +tk + tk−12
, (k = 1, ... ,K).
It is also noted thatd t
dτ=tk − tk−12
≡ h(k)
2, h(k) = tk − tk−1.
The following notation and conventions will be used in the discussion that ensues. First, all
vector functions of time are denoted as row vectors, that is, if y(τ) ∈ Rn is a vector function of the
scalar variable τ , then y(τ) = [y1(τ), · · · , yn(τ)]. Next, any capital boldface character, Y, denotes
a matrix of size M × n, where each row of Yi corresponds to the evaluation of a function y(τ) at
a particular value τ = τi . Next, the notation Yi:j denotes rows i through j of the matrix Y, except
when referring to a differentiation matrix D, in which case Di refers to the i th column of D. Finally,
D⊤ denotes the transpose of matrix D, and D⊤i denotes the transpose of the i th column of D.
2.5.1 Global Collocation at Legendre-Gauss Points
The method for approximating solutions to optimal control problems using global orthogonal
collocation at Legendre-Gauss (LG) points is now described [27]. The LG points are defined
in the domain (−1,+1) which does not include either of the endpoints. However, the method
derived here for collocation at the LG points still approximates, but does not collocate, the state
at both endpoints τ0 = −1 and τN+1 = +1. Figure 2-10 shows the LG collocation points as well
as the endpoints at which the state is approximated but not collocated for various values of N.
61
0 0.2 0.4 0.6 0.8 1-0.8 -0.6 -0.4 -0.2-13
4
5
6
7
8
9
τ
Discretization Points
Collocation Points
Num
ber
ofLG
Poi
nts,N
Figure 2-10. Distribution of Legendre-Gauss discretization and collocation points in thedomain τ ∈ [−1,+1].
The state is then approximated using a Lagrange polynomial with support points at the N
LG points plus the noncollocated point τ0 = −1, such that
y(τ) ≈ Y(τ) =N∑
i=0
YiLi(τ), (2–109)
where the Lagrange polynomials Li(τ) are defined as
Li(τ) =
N∏
j=0
j 6=i
τ − τj
τi − τj; (i = 0, ... ,N). (2–110)
The state approximation is then differentiated at τ = τj , (j = 1, ... ,N) as
Y(τj) ≈N∑
i=0
Yi Li(τj) = [DY0:N ]j , (2–111)
where Dij = Li (τj) , (i = 1, ... ,N, j = 0, ... ,N) are the components of the N × (N + 1)
Legendre-Gauss (LG) differentiation matrix.
62
Next, the cost functional of Eq. (2–1) is approximated with a Gaussian quadrature. The
finite-dimensional transcription of the continuous-time optimal control problem of Eqs. (2–1)–
(2–4) then becomes to minimize the cost function
J = Φ(YN+1) +h
2
N∑
j=1
wjg(Yj ,Uj) (2–112)
subject to the algebraic constraints
DY0:N =h
2f(Y1:N,U1:N), (2–113)
YN+1 = Y0 +h
2
N∑
j=1
wj f(Yj ,Uj), (2–114)
φ(Y0) = 0, (2–115)
C(Y1:N ,U1:N) ≤ 0, (2–116)
where w = (w1, ... ,wN) are the LG quadrature weights. It is noted for LG collocation that
Eq. (3–24) provides an LG quadrature approximation of the state, YN+1, at the final noncollo-
cated point τN+1 = +1.
2.5.2 Global Collocation at Legendre-Gauss-Radau Points
The method for approximating solutions to optimal control problems using global orthogonal
collocation at Legendre-Gauss-Radau (LGR) points is now described [64]. The LGR points are
defined in the domain [−1,+1) such that τ1 = −1 is a LGR collocation point but τN+1 = +1 is
a noncollocated point. However, the method derived here for collocation at the LGR points still
approximates, but does not collocate, the state at the terminal point τN+1 = +1. Figure 2-11
shows the LGR collocation points as well as the noncollocated terminal point for various values
of N.
The state is then approximated using a Lagrange polynomial with support points at the N
LGR points plus the noncollocated point τN+1 = +1 such that
y(τ) ≈ Y(τ) =N+1∑
i=1
YiLi(τ), (2–117)
63
0 0.2 0.4 0.6 0.8 1-0.8 -0.6 -0.4 -0.2-13
4
5
6
7
8
9
τ
Discretization Points
Collocation Points
Num
ber
ofLG
RP
oint
s,N
Figure 2-11. Distribution of Legendre-Gauss discretization and collocation points in thedomain τ ∈ [−1,+1].
where the Lagrange polynomials Li(τ) are defined as
Li(τ) =
N+1∏
j=1
j 6=i
τ − τj
τi − τj; (i = 1, ... ,N + 1). (2–118)
The state approximation is then differentiated at τ = τj , (j = 1, ... ,N) as
Y(τj) ≈N+1∑
i=1
Yi Li(τj) = [DY1:N+1]j , (2–119)
where Dij = Li (τj) , (i = 1, ... ,N, j = 1, ... ,N + 1) are the components of the N × (N + 1)
Legendre-Gauss-Radau (LGR) differentiation matrix. Next, the cost functional of Eq. (2–1) is
approximated by an LGR quadrature. The finite-dimensional approximation of the continuous-
time optimal control problem of Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost
function
J = Φ(YN+1) +h
2
N∑
j=1
wjg(Yj ,Uj) (2–120)
64
0 0.2 0.4 0.6 0.8 1-0.8 -0.6 -0.4 -0.2-13
4
5
6
7
8
9
τ
Discretization Points
Collocation Points
Num
ber
ofF
lippe
dLG
RP
oint
s,N
Figure 2-12. Distribution of Legendre-Gauss discretization and collocation points in thedomain τ ∈ [−1,+1].
subject to the algebraic constraints
DY1:N+1 =h
2f(Y1:N ,U1:N), (2–121)
φ(Y1) = 0, (2–122)
C(Y1:N ,U1:N) ≤ 0, (2–123)
where w = (w1, ... ,wN) are the LGR quadrature weights.
2.5.3 Global Collocation at Flipped Legendre-Gauss-Radau Points
The method for approximating solutions to optimal control problems using global orthogonal
collocation at the flipped Legendre-Gauss-Radau (LGR) points is now described [64]. The LGR
points are defined in the domain (−1,+1] such that τN = +1 is a LGR collocation point but
τ0 = −1 is a noncollocated point. However, the method derived here for collocation at the LGR
points still approximates, but does not collocate, the state at the initial point τ0 = −1. Figure
(2-12) shows the flipped LGR collocation points as well as the initial point at which the state is
approximated but not collocated for various values of N.
65
The state is then approximated using a Lagrange polynomial with support points at the N
LGR points plus the noncollocated point τ0 = −1, such that
y(τ) ≈ Y(τ) =N∑
i=0
YiLi(τ), (2–124)
where the Lagrange polynomials Li(τ) are defined as
Li(τ) =
N∏
j=0
j 6=i
τ − τj
τi − τj; (i = 0, ... ,N). (2–125)
The state approximation is then differentiated at τ = τj , (j = 1, ... ,N) as
Y(τj) ≈N∑
i=0
Yi Li(τj) = [DY0:N ]j , (2–126)
where Dij = Li (τj) , (i = 1, ... ,N, j = 0, ... ,N) are the components of the N × (N + 1)
Legendre-Gauss-Radau (LGR) differentiation matrix.
Next, the cost functional of Eq. (2–1) is approximated by an LGR quadrature. The finite-
dimensional approximation of the continuous-time optimal control problem of Eqs. (2–1)–(2–4) is
then given as follows. Minimize the cost function
J = Φ(YN) +h
2
N∑
j=1
wjg(Yj ,Uj) (2–127)
subject to the algebraic constraints
DY0:N =h
2f(Y1:N ,U1:N), (2–128)
φ(Y0) = 0, (2–129)
C(Y1:N ,U1:N) ≤ 0, (2–130)
where w = (w1, ... ,wN) are the flipped LGR quadrature weights.
2.5.4 Variable-Order Collocation at Legendre-Gauss Points
The method for approximating solutions to optimal control problems using variable-order
collocation at the Legendre-Gauss (LG) points is now described. When implementing the
variable-order LG method, a single variable is used for the value of the state at the end of mesh
interval k and the start of mesh interval k + 1, that is, Y(k)Nk+1
≡ Y(k+1)0 , 1 ≤ k ≤ K − 1 such that
66
3
4
5
6
7
8
9
tT0 T1 T2 T3
Discretization Points
Collocation PointsNum
ber
ofLG
poin
tsP
erIn
terv
al,Nk
Figure 2-13. Distribution of multiple-interval Legendre-Gauss discretization andcollocation points for various values of Nk . The domain [t0, tf ] is split into K=3 intervalssuch that t0 = T0 and tf = T3.
continuity in the state is enforced. Hence, redundant variables defining the state at the interior
mesh points are eliminated. Figure 2-13 shows the LG collocation points as well as the mesh
points at which the state is approximated but not collocated for when K = 3 and for various
values of N.
In multiple-interval LG collocation, the state is approximated in each mesh interval k as
y(k)(τ) ≈ Y(k)(τ) =Nk∑
i=0
Y(k)i L
(k)i (τ), L
(k)i (τ) =
Nk∏
j=0
i 6=j
τ − τ(k)j
τi − τ(k)j
, (2–131)
Differentiating Y(k)(τ) in Eq. (2–131) with respect to τ , yields
Y(k)(τj) ≈Nk∑
i=0
Y(k)iL(k)i(τj) = [D
(k)Y(k)0:Nk]j , (2–132)
where D(k)ij= L
(k)i(τ)j , (i = 1, ... ,Nk , j = 0, ... ,Nk) are the components of the Nk × (Nk + 1)
Legendre-Gauss (LG) differentiation matrix in the kth mesh interval.
Next, the cost functional of Eq. (2–1) is approximated using a multiple-interval LG quadra-
ture. The finite-dimensional approximation of the continuous-time optimal control problem of
67
Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost function
J ≈ Φ(Y(K)NK+1) +K∑
k=1
Nk∑
j=1
h(k)
2w(k)j g(Y
(k)j ,U
(k)j ), (2–133)
subject to the algebraic constraints
D(k)Y(k)0:Nk
=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (2–134)
Y(k+1)0 = Y
(k)0 +
h(k)
2
Nk∑
j=1
w(k)j f(Y
(k)j ,U
(k)j ), (k = 1, ... ,K − 1), (2–135)
Y(K)N+1 = Y
(K)0 +
h(K)
2
Nk∑
j=1
w(K)j f(Y
(K)j ,U
(K)j ), (2–136)
φ(Y(1)0 ) = 0, (2–137)
C(Y(k)1:Nk,U(k)1:Nk ) ≤ 0, (k = 1, ... ,K), (2–138)
where w(k) = (w (k)1 , ... ,w(k)N k) are the LG quadrature weights in interval k . It is noted for LG
collocation that Eq. (2–135) provides an LG quadrature approximation, Y(k)0 , of the state at the
final noncollocated point τ (k)N+1 = +1 in interval (k = 1, ... ,K − 1), while Eq. (2–136) provides an
LG quadrature approximation, Y(K)N+1, of the state at the final noncollocated point of the domain,
tf = τ(k)N+1 = +1.
2.5.5 Variable-Order Collocation at Legendre-Gauss-Radau Points
The method for approximating solutions to optimal control problems using variable-order
collocation at the Legendre-Gauss-Radau (LGR) points is now described. When implementing
the variable-order LGR method, a single variable is used for the value of the state at the end of
mesh interval k and the start of mesh interval k + 1, that is, Y(k)Nk+1
≡ Y(k+1)1 , 1 ≤ k ≤ K − 1
such that continuity in the state is enforced. Hence, redundant variables defining the state at the
interior mesh points are eliminated. It is noted that the LGR points are particularly conducive
to this type of collocation; because only one of the domain endpoints is collocated, there is no
“double collocation” at the boundaries. Also, the only noncollocated point is the last point of the
final interval, tf = τ(K)NK+1
= +1. Figure (2-14) shows the LGR collocation points as well as the
terminal point at which the state is approximated but not collocated for when N = 3 and for
various values of K .
68
3
4
5
6
7
8
9
tT0 T1 T2 T3
Discretization Points
Collocation PointsNum
ber
ofLG
RP
oint
sP
erIn
terv
al,Nk
Figure 2-14. Distribution of multiple-interval Legendre-Gauss-Radau discretization andcollocation points for various values of Nk . The domain [t0, tf ] is split into K=3 intervalssuch that t0 = T0 and tf = T3.
In multiple-interval LGR collocation, the state is approximated in each mesh interval k as
y(k)(τ) ≈ Y(k)(τ) =Nk+1∑
i=1
Y(k)iL(k)i(τ), L
(k)i(τ) =
Nk+1∏
j=1
i 6=j
τ − τ(k)j
τi − τ(k)j
, (2–139)
Differentiating Y(k)(τ) in Eq. (2–139) with respect to τ , yields
Y(k)(τj) ≈Nk+1∑
i=1
Y(k)i L
(k)i (τj) = [D
(k)Y(k)1:Nk+1
]j , (2–140)
where D(k)ij = L(k)i (τ)j , (i = 1, ... ,Nk , j = 1, ... ,Nk + 1) are the components of the Nk × (Nk + 1)
Legendre-Gauss-Radau (LGR) differentiation matrix in the kth mesh interval.
Next, the cost functional of Eq. (2–1) is approximated using a multiple-interval LG quadra-
ture. The finite-dimensional approximation of the continuous-time optimal control problem of
Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost function
J ≈ Φ(Y(K)NK+1) +K∑
k=1
Nk∑
j=1
h(k)
2w(k)j g(Y
(k)j ,U
(k)j ), (2–141)
69
subject to the algebraic constraints
D(k)Y(k)1:Nk+1
=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (2–142)
φ(Y(1)1 ) = 0, (2–143)
C(Y(k)1:Nk,U(k)1:Nk ) ≤ 0, (k = 1, ... ,K), (2–144)
where w(k) = (w (k)1 , ... ,w(k)N k) are the LGR quadrature weights in interval k .
2.5.6 Variable-Order Collocation at Flipped Legendre-Gauss-Radau Points
The method for approximating solutions to optimal control problems using variable-order
collocation at the flipped Legendre-Gauss-Radau (LGR) points is now described. When
implementing the flipped variable-order LGR method, a single variable is used for the value
of the state at the end of mesh interval k and the start of mesh interval k + 1, that is, Y(k)Nk ≡
Y(k+1)0 , 1 ≤ k ≤ K − 1 such that continuity in the state is enforced. Hence, redundant
variables defining the state at the interior mesh points are eliminated. It is noted that the flipped
LGR points are particularly conducive to this type of collocation; since only one of the domain
endpoints are collocated, there is no “double collocation” at the boundaries. Also, the only
noncollocated point is the first point of the first interval, t0 = τ(1)0 = −1. Figure (2-15) shows the
flipped LGR collocation points as well as the initial point at which the state is approximated but
not collocated for when N = 3 and for various values of K .
In multiple-interval flipped LGR collocation, the state is approximated in each mesh interval
k as
y(k)(τ) ≈ Y(k)(τ) =Nk∑
i=0
Y(k)i L
(k)i (τ), L
(k)i (τ) =
Nk∏
j=0
i 6=j
τ − τ(k)j
τi − τ(k)j
, (2–145)
Differentiating Y(k)(τ) in Eq. (4–46) with respect to τ , yields
Y(k)(τj) ≈Nk∑
i=0
Y(k)i L
(k)i (τj) = [D
(k)Y(k)0:Nk]j , (2–146)
where D(k)ij = L(k)i (τ)j , (i = 1, ... ,Nk , j = 0, ... ,Nk) are the components of the Nk × (Nk + 1)
flipped Legendre-Gauss-Radau (LGR) differentiation matrix in the kth mesh interval.
70
3
4
5
6
7
8
9
t
T0 T1 T2 T3
Discretization Points
Collocation Points
Num
ber
ofF
lippe
dLG
RP
oint
sP
erIn
terv
al,Nk
Figure 2-15. Distribution of multiple-interval Flipped Legendre-Gauss-Radaudiscretization and collocation points for various values of Nk . The domain [t0, tf ] is splitinto K=3 intervals such that t0 = T0 and tf = T3.
Next, the cost functional of Eq. (2–1) is approximated using a multiple-interval LG quadra-
ture. The finite-dimensional approximation of the continuous-time optimal control problem of
Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost function
J ≈ Φ(Y(K)NK) +
K∑
k=1
Nk∑
j=1
h(k)
2w(k)jg(Y
(k)j,U(k)j), (2–147)
subject to the algebraic constraints
D(k)Y(k)0:Nk
=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (2–148)
φ(Y(1)0 ) = 0, (2–149)
C(Y(k)1:Nk,U(k)1:Nk ) ≤ 0, (k = 1, ... ,K), (2–150)
where w(k) = (w (k)1 , ... ,w(k)N k) are the flipped LGR quadrature weights in interval k .
71
CHAPTER 3COSTATE ESTIMATION USING THE INTEGRAL FORMULATION
As was previously discussed, the Legendre-Gauss (LG) and Legendre-Gauss-Radau
(LGR) methods for approximating solutions to optimal control problems are equivalent
regardless of whether the collocation is performed in differential or integral form.
Typically, however, the differential form of either method has been used. Therefore, the
relationship between the Lagrange multipliers of the differential form of the collocation
methods and the costate of the optimal control problem has been well documented.
On the other hand, the corresponding relationship between the Lagrange multipliers
associated with the integral forms of LG and LGR collocation and the costate of the
optimal control problem has not been established. In this chapter methods for estimating
the optimal control costate using the integral forms of LG and LGR collocation are
developed. Specifically, transformations are derived that relate the Lagrange multipliers
of the integral forms of the LG and LGR collocation methods to the costate of the
original optimal control problem. A new continuous-time dual variable called the inte-
gral costate is introduced, where the integral costate is the Lagrange multiplier of the
integral dynamic constraint. The first-order optimality conditions of the integral form of
the optimal control problem are derived in terms of the integral costate. The integral
form of the optimal control problem is then discretized using the integral LG and LGR
collocation methods and relationship between the discrete form of the integral costate
and the costate of the original differential optimal control problem are developed. The
approach developed in this research then provides a way to estimate the costate of the
original optimal control problem using the Lagrange multipliers of the integral form of the
LG and LGR collocation methods.
The following notation and conventions will be used throughout this chapter. First,
all vector functions of time are denoted as row vectors, that is, if y(τ) ∈ Rn is a vector
function of the scalar variable τ , then y(τ) = [y1(τ), · · · , yn(τ)]. Next, any capital
72
boldface character, Y, denotes a matrix of size M×n, where each row of Yi corresponds
to the evaluation of a function y(τ) at a particular value τ = τi . Next, the notation Yi:j
denotes rows i through j of the matrix Y, except when referring to a differentiation matrix
D or the integration matrix A, in which case Di and Ai refers to the i th column of D and
A. Finally, D⊤ denotes the transpose of matrix D, and D⊤i denotes the transpose of the
i th column of D. Given vectors x and y ∈ Rn, the notation 〈x, y〉 is used to denote the
standard inner product between x and y. Furthermore, if f : Rn −→ Rm, then ∇f is the
m by n Jacobian matrix whose i th row is ∇fi . In particular, the gradient of a scalar-valued
function is a row vector. If φ : Rm×n −→ R and Y is an m by n matrix, then ∇φ denotes
the m by n matrix whose (i , j) element is (∇φ(Y))ij = ∂φ(Y)/∂Yij .
This remainder of this chapter is organized as follows. In Section 3.1 the continuous-time
optimal control problem is presented with the dynamic constraints formulated in both
differential and integral form, and the first-order optimality conditions of each formulation
are given. In Sections 3.2 and 3.3, the Legendre-Gauss and Legendre-Gauss-Radau
collocation methods in both differential and integral forms are presented, and the
first-order optimality conditions of each form are derived. Furthermore, the transformed
adjoint system of the integral form is derived, and a costate estimate is presented in
terms of the Lagrange multipliers of the integral forms. Finally, Section 3.4 provides a
discussion of the differences between the collocation schemes at LG and LGR points.
3.1 Continuous-Time Bolza Optimal Control Problem
In order to make this exposition clearer, in this chapter the Bolza continuous-time
optimal control problem of Section 2.1 is formulated in the domain τ ∈ [−1,+1]. It is
noted that the time interval τ ∈ [−1,+1] can be transformed to the interval [t0, tf ] via the
affine transformation
t =tf − t02
τ +tf + t02. (3–1)
Furthermore, this problem formulation is stated with no inequality path constraints.
Inequality path constrained optimal control problems will be discussed in Chapter 5.
73
3.1.1 Differential and Integral Forms of Optimal Control Problem
Consider again the Bolza continuous-time optimal control problem from Section
2.1 defined on the interval τ ∈ [−1,+1]. Determine the state y(τ) ∈ Rn and the control
u(τ) ∈ Rm that minimize the cost functional
J = Φ(y(+1)) +
∫ +1
−1
g(y, u)dτ (3–2)
subject to the dynamic constraint
y(τ)− f(y, u) = 0. (3–3)
and the boundary condition
φ(y(−1)) = 0. (3–4)
Henceforth, Eqs. (3–2)–(3–4) will be referred to as the differential optimal control
problem.
The differential optimal control problem given in Eqs. (3–2)–(3–4) can be re-written
in the following integral form. In particular, integrating the dynamics given in Eq. (3–3),
yields
y(τ) = y(−1) +∫ τ
−1
f(y, u) dτ .
The optimal control problem in integral form is then stated as follows. Determine the
state, y(τ) ∈ Rn, and the control, u(τ) ∈ Rm, that minimize the cost functional
J = Φ(y(+1)) +
∫ +1
−1
g(y(τ), u(τ))dτ (3–5)
subject to the integral constraint
y(τ)− y(−1)−∫ τ
−1
f(y, u) dt = 0, (3–6)
and the boundary condition
φ(y(−1)) = 0. (3–7)
74
Henceforth, Eqs. (3–5)–(3–7) will be referred to as the integral optimal control problem.
3.1.2 First-Order Optimality Conditions of Differential and Integral Forms
The first-order optimality conditions of the differential optimal control problem
obtained from the calculus of variations were derived in Section 2.1.1 and are given as
y(τ) = f(y, u), φ(y(−1)) = 0, (3–8)
0 = ∇uH(y, u,λ), (3–9)
−λ = ∇yH(y, u,λ), (3–10)
λ(−1) = ∇y 〈ψ,φ(−1)〉, (3–11)
λ(+1) = ∇yΦ(+1), (3–12)
Next, the first-order optimality conditions of the integral optimal control problem
obtained from applying the calculus of variations were derived in Section 2.1.2 are given
as
y = y(−1) +∫ τ
−1
f(y, u) dτ , φ(y(−1)) = 0, (3–13)
0 = ∇ug + 〈(∫ +1
τ
p dt +∇yΦ(+1))
, ∇uf(y, u) 〉, (3–14)
p = ∇yg + 〈(∫ +1
τ
p dt +∇yΦ(+1))
, ∇yf(y, u)〉, (3–15)
∇y〈ψ,φ(−1)〉 =∫ +1
−1
p dτ +∇yΦ(+1), (3–16)
where
H(y, u,λ) = g(y, u) + 〈λ, f(y, u)〉, (3–17)
is the Hamiltonian, λ(τ) is the costate of the differential optimal control problem (and
will be referred to henceforth as the differential costate), p(τ) is the costate of integral
optimal problem (and will be referred to henceforth as the integral costate), and ψ is the
Lagrange multiplier associated with the boundary condition of Eq. (3–4). It is noted that
the differential costate and the integral costate are different in that λ(τ) is the Lagrange
75
multiplier associated with the differential equation constraint of Eq. (3–3) while p(τ) is
the Lagrange multiplier associated with the integral equation constraint of Eq. (3–6).
The differential and integral costate are related as
λ(τ) = ∇yΦ(+1) +∫ +1
τ
p dt. (3–18)
In particular, substituting Eq. (3–18) together with the relationship
λ(τ) = −p(τ) (3–19)
into the first-order optimality conditions of the integral optimal problem as given in
Eqs. (3–13)–(3–16) yields the the first-order optimality conditions of the differential
optimal control problem as given in Eqs. (3–8)–(3–12). The remainder of this chapter is
devoted to deriving two discrete approximations of the differential costate, λ(τ), using
discrete approximations of the integral costate, p(τ).
3.2 Costate Estimation Using Integral Legendre-Gauss Collocation
In this section a costate estimate for the differential optimal control problem is
developed via an estimate of the integral costate obtained using the integral form of
the Legendre-Gauss orthogonal collocation method. In Section 3.2.1 the differential
form of the Legendre-Gauss collocation method is described. In Section 3.2.2 the
first-order optimality conditions of the nonlinear programming problem described in
Section 3.2.1 are provided. In Section 3.2.3 the integral form of the Legendre-Gauss
collocation method is described. In Section 3.2.4 the first-order optimality conditions of
the nonlinear programming problem described in Section 3.2.1 are provided. Finally, in
Section 3.2.5 a differential costate estimate using the integral costate estimate derived in
Section 3.2.4 is developed.
3.2.1 Differential Form of Legendre-Gauss Collocation
The differential optimal control problem of Eqs. (3–2)–(3–4) can now be approximated
using collocation at Legendre-Gauss (LG) points as was described in Section 2.5.1. The
76
LG points are defined in the domain (−1,+1) which does not include either of the
endpoints. The state is approximated using a Lagrange polynomial with support points
at the N LG points plus the noncollocated point τ0 = −1, such that
y(τ) ≈ Y(τ) =N∑
i=0
YiLi(τ), Li(τ) =N∏
j=0
j 6=i
τ − τjτi − τj
; (i = 0, ... ,N). (3–20)
where Li(τ), (i = 0, ... ,N) is a basis of Lagrange polynomials of degree N with support
points (τ0, ... , τN). The time derivative of the state at τ = τj is then approximated as
Y(τj) ≈N∑
i=0
Yi Li(τj) = [DY0:N ]j , (3–21)
where D is the N × (N + 1) Legendre-Gauss differentiation matrix whose elements are
given as Dij = Li (τj) , (i = 1, ... ,N, j = 0, ... ,N). Furthermore, the cost functional of
Eq. (3–2) is approximated using a Legendre-Gauss quadrature as
J = Φ(YN+1) +
N∑
j=1
wjg(Yj ,Uj). (3–22)
The differential optimal control problem is then approximated via the following finite-dimensional
nonlinear programming problem. Minimize the cost function of Eq. (3–22) subject to the
algebraic constraints
DY0:N = f(Y1:N ,U1:N), (3–23)
YN+1 = Y0 +
N∑
j=1
wj f(Yj ,Uj), (3–24)
φ(Y0) = 0, (3–25)
where w = (w1, ... ,wN) is a vector of Legendre-Gauss quadrature weights. It is
noted for LG collocation that Eq. (3–24) provides an LG quadrature approximation,
YN+1, of the state at the final noncollocated point τN+1 = +1. The NLP described by
Eqs. (3–22)–(3–25) will be referred to as the differential Legendre-Gauss collocation
method.
77
3.2.2 KKT Conditions Using Differential Legendre-Gauss Collocation
The Karush-Kuhn-Tucker (KKT) first-order optimality conditions of the differential
Legendre-Gauss collocation method are now derived [27, 64]. First the Lagrangian is
formed such that
L = Φ(YN+1) + 〈w, g(Y1:N,U1:N)〉 − 〈ψ,φ(Y0)〉
− 〈Λ1:N ,DY0:N − f(Y1:N,U1:N)〉
− 〈ΛN+1,YN+1 −Y0 −w⊤f(Y1:N,U1:N)〉,
where (ψ, Λ1:N , ΛN+1) are the KKT multipliers associated with the constraints of
Eqs. (3–25), (3–23), and (3–24), respectively. Next, the KKT conditions are obtained by
differentiating the Lagrangian with respect to all variables in the NLP. They are given as
DY0:N = f(Y1:N,U1:N), φ(Y0) = 0, (3–26)
YN+1 = Y0 + w⊤f(Y1:N,U1:N), (3–27)
0 = ∇UHG(Y1:N ,U1:N,Λ1:N+1), (3–28)
W−1D⊤1:NΛ1:N = ∇YHG(Y1:N,U1:N ,Λ1:N+1) (3–29)
D⊤0 Λ1:N = ΛN+1 −∇Y〈ψ,φ(Y0)〉, (3–30)
ΛN+1 = ∇YΦ(YN+1). (3–31)
where
HG(Y1:N ,U1:N,Λ1:N+1) = 1⊤g(Y1:N,U1:N) + 〈W−1Λ1:N + 1ΛN+1, f(Y1:N,U1:N)〉 (3–32)
is the discrete Hamiltonian. Furthermore, 1 is an N × 1 column vector of all ones, andW
is a N × N diagonal matrix of LG weights. Suppose now that the following transformed
78
dual variables are introduced:
λ1:N =W−1Λ1:N + 1ΛN+1, (3–33)
λN+1 = ΛN+1, (3–34)
λ0 = ΛN+1 −D⊤0 Λ1:N . (3–35)
In addition, consider the N × (N + 1) matrix D†,
D†ij = −
wj
wiDji , and D
†i,N+1 =
N∑
j=1
D†ij (3–36)
for (i , j = 1, ... ,N). It was shown in Ref. [36] that D† is a differentiation matrix for the
space of polynomials of degree N . In other words, if b is a polynomial of degree at most
N and b ∈ RN+1 is the vector whose i th element is bi = b(τi), then
(D†b)i = b(τi). (3–37)
Using the transformations described in Eqs. (3–33)–(3–35) along with Eq. (3–36), the
KKT conditions of the differential Legendre-Gauss collocation method of Eqs. (3–26)–(3–31)
can be written as [64]
DY0:N = f(Y1:N,U1:N), φ(Y0) = 0, (3–38)
YN+1 = Y0 + wT f(Y1:N,U1:N), (3–39)
0 = ∇UH(Y1:N,U1:N ,λ1:N), (3–40)
D†λ1:N+1 = −∇YH(Y1:N ,U1:N,λ1:N), (3–41)
λ0 = ∇Y〈ψ,φ(Y0)〉, (3–42)
λN+1 = ∇YΦ(YN+1), (3–43)
where H is a discrete form of the Hamiltonian given by Eq. (3–17). It is seen by
examination that Eqs. (3–38)–(3–43) is a discrete form of Eqs. (3–8)–(3–12).
79
It is noted in this transcription that the state is being differentiated by a matrix D
[given by Eq. (3–21)] which is based on the derivatives of polynomials of degree N with
coefficients at the N LG points plus the initial noncollocated point τ0 = −1, whereas the
costate is being differentiated by a matrix D† [given by Eq. (3–36)] which is based on
the derivatives of polynomials of degree N with coefficients at the N LG points plus the
terminal noncollocated point τN+1 = +1.
3.2.3 Integral Form of Legendre-Gauss Collocation
The integral optimal control problem is now discretized using the integral form of
Legendre-Gauss collocation. It has been shown in Ref. [64] that the differential LG
transcription method given in Section 3.2.1 can equivalently be expressed as an implicit
integration scheme. In particular, let p be any polynomial of degree at most N . By the
construction of the N × (N + 1) matrix D, then Dp = p where
pi = p(τi), (i = 0, ... ,N),
pi = p(τi), (i = 0, ... ,N).(3–44)
Now, let 1 be a N × 1 column vector composed of ones. since D is a differentiation
matrix, it follows that the components of the vector D1 are the derivatives at the
collocation points of the constant polynomial p(τ) = 1. Therefore, D1 = 0, which
implies that D1 = D0 +D1:N1 = 0. Rearranging,
D0 = −D1:N1. (3–45)
It has been shown by Ref. [64] that the matrix D1:N is full rank. Therefore, multiplying by
D−11:N gives the relationship
−D−11:ND0 = 1. (3–46)
Furthermore, the expression p can equivalently be written as
p = Dp = D0p0 +D1:Np1:N. (3–47)
80
premultiplying by D−11:N and utilizing relationship given by Eq. (3–46), the following is
obtained
pi = p0 +(
D−11:N p
)
i, (i = 1, ... ,N). (3–48)
Now, to show that the N × N matrix D−11:N is an integration matrix, a different
expression for pi − p0 can be obtained based on the integration of the interpolant of the
derivative. Let L†j (τ) be the Lagrange polynomial basis given by
L†j =
N∏
i=1i 6=j
τ − τiτj − τi
, (j = 1, ... ,N). (3–49)
Notice that the Lagrange polynomials Li defined in the differential problem formulation,
given by Eq. (3–20) are degree N while the Lagrange polynomials L†j are degree N − 1.
Then because p is a polynomial of degree at most N − 1, it can be interpolated exactly
by the Lagrange polynomials L†j :
p =
N∑
j=1
pjL†j (τ) (3–50)
Integrating p from −1 to τi , the following relationship is obtained
p(τi) = p(−1) +N∑
j=1
pjAij ,
Aij =
∫ τi
−1
L†j (τ)dτ , (i = 1, ... ,N),
(3–51)
which can equivalently be written as
pi = p0 + (Ap)i , (i = 1, ... ,N). (3–52)
The relations (3–48) and (3–52) are satisfied for any polynomial of degree at most N . By
equating (3–48) and (3–52) the following becomes true
Ap = D−11:Np.
81
Thus, the dynamic constraints of Eq. (3–6) can be approximated as
Y1:N = 1Y0 + Af(Y1:N,U1:N), (3–53)
where 1 denotes a N × 1 column vector of the constant 1 for every component, and
A = D−11:N is the N × N Legendre-Gauss integration matrix defined by Eq. (3–51).
Thus the state at each Legendre-Gauss point is approximated via quadrature using the
Legendre-Gauss integration matrix. Next, the state at τ = +1 is approximated using a
Legendre-Gauss quadrature as
YN+1 = Y0 +
N∑
i=1
wi f(Yi ,Ui).
Using these properties of LG collocation, the integral optimal control problem of
Eqs. (3–5)–(3–7) can be approximated via the following finite-dimensional nonlinear
programming problem. Minimize the cost function of Eq. (3–22) subject to the algebraic
constraints
Y1:N = 1Y0 + Af(Y1:N,U1:N), (3–54)
YN+1 = Y0 +wT f(Y1:N,U1:N), (3–55)
φ(Y0) = 0. (3–56)
The NLP described by Eqs. (3–22) and (3–54)–(3–56) will be referred to as the integral
Legendre-Gauss collocation method.
3.2.4 KKT Conditions Using Integral Legendre-Gauss Collocation
Similarly as was done for the differential form of the differential LG transcription, the
Karush-Kuhn-Tucker (KKT) first-order optimality conditions of the integral Legendre-Gauss
82
collocation method are now derived. First, the Lagrangian is defined as
L = Φ(YN+1) + 〈w, g(Y1:N,U1:N)〉 − 〈ψ,φ(Y0)〉
− 〈P1:N ,Y1:N − 1Y0 − Af(Y1:N,U1:N)〉
− 〈ΛN+1,YN+1 −Y0 −w⊤f(Y1:N,U1:N)〉,
where (ψ, P1:N , ΛN+1) are the KKT multipliers associated with the constraints of
Eqs. (3–56), (3–54), and (3–55), respectively. Next, the KKT conditions are obtained by
differentiating the Lagrangian with respect to all free variables in the NLP. They are given
as
Y1:N = 1Y0 + Af(Y1:N,U1:N), φ(Y0) = 0, YN+1 = Y0 + w⊤f(Y1:N,U1:N), (3–57)
0 = ∇U(
w⊤g(Y1:N,U1:N) + 〈A⊤P, f(Y1:N,U1:N)〉+ 〈ΛN+1,w⊤f(Y1:N,U1:N)〉)
, (3–58)
P1:N = ∇Y(
w⊤g(Y1:N,U1:N) + 〈A⊤P, f(Y1:N,U1:N)〉+ 〈ΛN+1,w⊤f(Y1:N,U1:N)〉)
, (3–59)
ΛN+1 = ∇Y〈ψ,φ(Y0)〉 − 1⊤P1:N, (3–60)
ΛN+1 = ∇YΦ(YN+1), (3–61)
Suppose now that the following transformed dual variables are introduced:
p1:N =W−1P1:N, λN+1 = ΛN+1. (3–62)
In addition, consider the N × N matrix A†,
A†ij = −
wj
wiAji , (3–63)
for (i , j = 1, ... ,N). It will now be proven that A† is a backward integration matrix for the
space of polynomials of degree N − 1.
Theorem 1. The matrix A† described by Eq. (3–63) is a backwards integration matrix for
the space of polynomials of degree N−1. More specifically, if p is a polynomial of degree
83
at most N − 1 and p ∈ RN is the vector with i th component pi = p(τi), (i = 1, ... ,N), then
(A†p)i =
∫ +1
τi
p(t)dt (3–64)
Proof. Let p, q denote polynomials of degree at most N − 1 such that pj = p(τj) and
qj = q(τj), (j = 1, ... ,N). The order of integration in a double integral can be switched by
changing the limits of integration such that
∫ +1
−1
[
q(τ) ·∫ τ
−1
p(t) dt
]
dτ =
∫ +1
−1
[
p(τ) ·∫ +1
τ
q(t) dt
]
dτ . (3–65)
Now, since p and q are polynomials of degree at most N − 1, then p ·∫ +1
τq dt and
q ·∫ τ
−1p dt are polynomials of degree at most 2N − 1. Since Gauss quadrature is exact
for polynomials of degree 2N − 1, the integrals in Eq. (3–65) can be replaced by their
quadrature equivalents to obtain
N∑
j=1
wjqj ·∫ τ
−1
p(t) dt =
N∑
j=1
wjpj ·∫ +1
τ
q(t) dt. (3–66)
Substituting∫ τ
−1p(t)dt = Ap and
∫ +1
τq(t) dt = A†q the following expression is obtained
(Wq)⊤Ap = (A†q)⊤Wp,
q⊤(A⊤W −WA†)p = 0.
(3–67)
Since p and q are arbitrary vectors, it must be true that
A⊤W −WA† = 0, (3–68)
which implies that
A†ij =wj
wiAji , (i , j = 1, ... ,N).
Furthermore, a polynomial of degree N − 1 is uniquely defined by its value at N points,
and can thus be exactly interpolated by a Lagrange interpolating polynomial of degree
84
N − 1. Therefore:
∫ +1
τi
q(τ)dτ =N∑
j=1
A†ijqj , A
†ij =
∫ +1
τi
L†j (τ)dτ , (3–69)
where L† is the basis of interpolating polynomials of degree N − 1 defined in Eq. (3–51).
Using the transformations described in Eq. (3–62) along with the definition of A† in
Eq. (3–63), the KKT conditions of the integral form of the Legendre-Gauss collocation
method as shown in Eqs. (3–57)–(3–61) can be written as
Y1:N = 1Y0 +Af(Y1:N ,U1:N), φ(Y0) = 0, YN+1 = Y0 + w⊤f(Y1:N,U1:N), (3–70)
0 = ∇Ug(Y1:N ,U1:N) + 〈(
A†p1:N +∇YΦ(YN+1))
, ∇Uf(Y1:N ,U1:N)〉, (3–71)
p1:N = ∇Yg(Y1:N,U1:N) + 〈(
A†p1:N +∇YΦ(YN+1))
, ∇Yf(Y1:N,U1:N)〉, (3–72)
w⊤p1:N = ∇Y〈ψ,φ(Y0)〉 − ∇YΦ(YN+1), (3–73)
λN+1 = ∇YΦ(YN+1). (3–74)
It is seen by examination that Eqs. (3–70)–(3–74) are a discrete form of the necessary
conditions for optimality of the integral optimal control problem given in Eqs. (3–13)–(3–16).
It must be noted that in the integral Legendre-Gauss collocation method this
transcription that the state, y(τ), and the dual variable, p(τ), are approximated using
an integration matrix for the space of polynomials of degree N − 1, the difference being
that the state is approximated using a forward quadrature using the matrix A given in
Eq. (3–51) while the integral costate, p(τ), is approximated using a backward quadrature
using the matrix A† given in Eq. (3–69).
3.2.5 Differential Costate Estimate Using Integral Legendre-Gauss Collocation
The results of Sections 3.2.1–3.2.4 can now be used to define an estimate
for the differential costate using the estimate of the integral costate. In particular,
the transformed necessary conditions of Eq. (3–38)–(3–43) are equivalent to the
85
transformed necessary conditions of Eqs. (3–70)–(3–73) if the discrete approximations
of λ(τ) and p(τ) are related as
D†λ1:N+1 = −p1:N , (3–75)
where D† is as defined in Eq. (3–36).
It has been shown by Ref.[64] that the matrix D† has the following properties
that are similar to the properties the matrix D: (a) the square matrix D†1:N obtained by
removing the last column of D† is full-rank, and (b) −[D†1:N ]
−1D†N+1 = 1. Using these
properties, Eq. (3–75) can be rewritten as
λ1:N = 1λN+1 − [D†1:N ]
−1p1:N ,
λ1:N = 1λN+1 + A†p1:N,
(3–76)
where the matrix A† is the integration matrix defined by Eq. (3–69). From this relationship,
it can be seen that the differential and integral dual variable estimates are related
by [D†1:N ]
−1 = −A†. Furthermore, the costate at the initial noncollocated point τ0 is
approximated through a Gaussian quadrature such that
λ0 = λN+1 + w⊤p1:N. (3–77)
It can be seen that applying the transformations of Eqs. (3–75)–(3–77) to the transformed
first-order optimality conditions given in Eqs. (3–70)–(3–74) will result in the transformed
first-order optimality conditions given in Eqs. (3–38)–(3–43).
3.3 Costate Estimation Using Integral Legendre-Gauss-Radau Collocation
In this section a costate estimate for the differential optimal control problem is
developed via an estimate of the integral costate obtained using the integral form
of the Legendre-Gauss-Radau orthogonal collocation method. In Section 3.3.1 the
differential form of the Legendre-Gauss collocation method is revisited. In Section 3.3.2
the first-order optimality conditions of the nonlinear programming problem described
86
in Section 3.3.1 are derived. In Section 3.3.3 the integral form of the Legendre-Gauss
collocation method is described. In Section 3.3.4 the first-order optimality conditions of
the nonlinear programming problem described in Section 3.3.1 are derived. Finally, in
Section 3.3.5 a differential costate estimate using the integral costate estimate derived in
Section 3.3.4 is presented.
3.3.1 Differential Form of Legendre-Gauss-Radau Collocation
The differential optimal control problem of Eqs. (3–2)–(3–4) can now be approximated
using collocation at Legendre-Gauss-Radau (LGR) points as was described in Section
2.5.2. The LGR points are defined in the domain [−1,+1) such that τ1 = −1 is a LGR
collocation point but τN+1 = +1 is a noncollocated point. The state is approximated
using a Lagrange polynomial with support points at the N LGR points plus the
noncollocated point τN+1 = +1, such that
y(τ) ≈ Y(τ) =N+1∑
i=1
YiLi(τ), Li(τ) =
N+1∏
j=1
j 6=i
τ − τjτi − τj
; (i = 1, ... ,N + 1). (3–78)
where Li(τ), (i = 1, ... ,N + 1) is a basis of Lagrange polynomials of degree N
with support points (τ1, ... , τN+1). The time derivative of the state at τ = τj is then
approximated as
Y(τj) ≈N+1∑
i=1
Yi Li(τj) = [DY1:N+1]j , (3–79)
where D is the N × (N + 1) Legendre-Gauss-Radau differentiation matrix whose
elements are given as Dij = Li (τj) , (i = 1, ... ,N, j = 1, ... ,N + 1). Furthermore, the
cost functional of Eq. (3–2) is approximated using a Legendre-Gauss-Radau quadrature
as
J = Φ(YN+1) +
N∑
j=1
wjg(Yj ,Uj). (3–80)
The differential optimal control problem is then approximated via the following finite-dimensional
nonlinear programming problem. Minimize the cost function of Eq. (3–80) subject to the
87
algebraic constraints
DY1:N+1 = f(Y1:N,U1:N), (3–81)
φ(Y1) = 0, (3–82)
where w = (w1, ... ,wN) is a vector of Legendre-Gauss-Radau quadrature weights. The
NLP described by Eqs. (3–80)–(3–82) will be referred to as the differential Legendre-
Gauss-Radau method.
3.3.2 KKT Conditions Using Differential Legendre-Gauss-Radau Collocation
Similarly as was done for collocation at the LG points, the Karush-Kuhn-Tucker
(KKT) first-order optimality conditions of the differential Legendre-Gauss-Radau
collocation method are now derived [64]. First the Lagrangian is formed such that
L = Φ(YN+1) + 〈w, g(Y1:N,U1:N)〉 − 〈ψ,φ(Y1)〉
− 〈Λ1:N ,DY1:N+1 − f(Y1:N ,U1:N)〉,
where (ψ, Λ1:N) are the KKT multipliers associated with the constraints of Eqs. (3–82)
and (3–81), respectively. Next, the KKT conditions are obtained by differentiating the
Lagrangian with respect to all variables in the NLP. They are given as
DY1:N+1 = f(Y1:N,U1:N), φ(Y1) = 0, (3–83)
0 = ∇UHR(Y1:N,U1:N,Λ1:N), (3–84)
D⊤1:NΛ1:N = ∇YHR(Y1:N,U1:N,Λ1:N)− e1∇Y〈ψ,φ(Y1)〉, (3–85)
D⊤N+1Λ1:N = ∇YΦ(YN+1), (3–86)
where
HR(Y1:N ,U1:N,Λ1:N) = w⊤g(Y1:N ,U1:N) + 〈Λ1:N , f(Y1:N,U1:N)〉 (3–87)
88
is the discrete Hamiltonian, and e1 is the first column of the identity matrix. Suppose now
that the following transformed dual variables are introduced:
λ1:N =W−1Λ1:N, (3–88)
λN+1 = D⊤N+1Λ1:N. (3–89)
In addition, consider the N × N matrix D†,
D†11 = −D11 −
1
w1and D
†ij = −
wj
wiDji otherwise (3–90)
for (i , j = 1, ... ,N). It was shown in Ref. [36] that D† is a differentiation matrix for the
space of polynomials of degree N − 1. In other words, if b is a polynomial of degree at
most N − 1 and b ∈ RN is the vector i th element bi = b(τi), then
(D†b)i = b(τi). (3–91)
Using the transformations described in Eqs. (3–88)–(3–89) along with Eq. (3–90),
the KKT conditions of differential Legendre-Gauss-Radau collocation method of
Eqs. (3–83)–(3–86) can be written as[36]
DY1:N+1 = f(Y1:N,U1:N), φ(Y1) = 0, (3–92)
0 = ∇UH(Y1:N,U1:N ,λ1:N), (3–93)
D†λ1:N = −∇YH(Y1:N ,U1:N,λ1:N) +e1
w1(∇Y〈ψ,φ(Y1)〉 − λ1) , (3–94)
λN+1 = ∇YΦ(YN+1), (3–95)
These equations are incomplete because a new variable λN+1 was introduced without
adding a new equation. An equation for this new variable can be developed by
manipulating the matrix D. Because D is a differentiation matrix, it has the property
that D1 = 0, where 1 is a vector whose components are all constant and equal to 1. This
89
implies that
DN+1 = −N∑
j=1
D1:N, j ,
D⊤N+1Λ =
N∑
i=1
Di,N+1Λi = −N∑
i=1
N∑
j=1
Di, jΛi ,
λN+1 = λ1 +
N∑
i=1
N∑
j=1
wiλjD†i,j = λ1 +
N∑
j=1
wj(D†λ)j , (3–96)
where the relationships in (3–88) and (3–91) were used to obtain Eq. (4–70). It can be
seen that this relationship approximates the integral of the costate dynamics across the
domain via a Radau quadrature. Combining Eqs. (3–92)–(3–95) with Eq. (4–70), the
complete transformed adjoint system can then be written as
DY1:N+1 = f(Y1:N,U1:N), φ(Y1) = 0, (3–97)
0 = ∇UH(Y1:N,U1:N ,λ1:N), (3–98)
D†λ1:N = −∇YH(Y1:N ,U1:N,λ1:N) +e1
w1(∇Y〈ψ,φ(Y1)〉 − λ1) , (3–99)
λN+1 = ∇Y〈ψ,φ(Y0)〉 −N∑
i=1
wi∇YH(Yi ,Ui ,λi), (3–100)
λN+1 = ∇YΦ(YN+1), (3–101)
where H is a discrete form of the Hamiltonian given by Eq. (3–17). It is seen in
Eq. (3–100) that the costate at the noncollocated final point τ = +1 is approximated
via a Legendre-Gauss-Radau quadrature of the costate dynamics across the solution
domain. Consequently, Eq. (3–100) is a subtle way of enforcing the relationship λ1 =
∇Y〈ψ,φ(Y1)〉 and it is expected that the last term of Eq. (3–99) will be small while the
remaining terms in Eq. (3–99) are a collocation collocation scheme for the continuous
adjoint equation. The transformed optimality conditions of Eqs. (3–97)–(3–101) are,
thus, a discrete form of the necessary conditions for optimality of the differential optimal
control problem described by Eqs. (3–8)–(3–12). It is noted in these conditions that
90
the time derivative of the state is being approximated using the differentiation matrix D
for the space of polynomials of degree N [see Eq. (3–79)], while the costate is being
differentiated by a differentiation matrix D† for the space of polynomials of degree N − 1
[see Eq. (3–90)].
3.3.3 Integral Form of Legendre-Gauss-Radau Collocation
The integral optimal control problem is now discretized using the integral form of
Legendre-Gauss-Radau collocation. It has been shown in Ref. [64] that the differential
LG transcription method given in Section 3.2.1 can equivalently be expressed as an
implicit integration scheme. In particular, suppose that p is any polynomial of degree
at most N . Then, by the construction of the N × (N + 1) differentiation matrix D, then
Dp = p where
pi = p(τi), (i = 1, ... ,N + 1),
i = p(τi), (i = 1, ... ,N + 1).(3–102)Now, let 1 be a N × 1 column vector composed
of ones. since D is a differentiation matrix, it follows that the components of the vector
D1 are the derivatives at the collocation points of the constant polynomial p(τ) = 1.
Therefore, D1 = 0, which implies that D1 = D1:N1+DN+1 = 0. Rearranging,
DN+1 = −D1:N1. (3–103)
It has been shown by Ref. [64] that the matrix D1:N is full rank. Therefore, pre-multiplying
by D−11:N gives the relationship
−D−11:NDN+1 = 1. (3–104)
Furthermore, the expression p can equivalently be written as
p = Dp = D1:Np1:N +DN+1pN+1. (3–105)
premultiplying by D−11:N and utilizing the relationship given by Eq. (3–104), the following is
obtained
pi = pN+1 +(
D−11:Np
)
i, (i = 1, ... ,N). (3–106)
91
Next, let Lj(τ) be Lagrange polynomial basis given by
Lj(τ) =N∏
i=1i 6=j
τ − τiτj − τi
, (j = 1, ... ,N).
Then p can be interpolated exactly as
p(τ) =
N∑
j=1
pj Lj(τ).
Integrating from +1 to τi ,
pi = pN+1 + (Ap)i , (i = 1, ... ,N), (3–107)
Aij =
∫ τi
+1
Lj(τ) dτ , (i , j = 1, ... ,N). (3–108)
By (3–106) and (3–107), it can be seen that A = D−11:N . Thus, the dynamic constraints of
Eq. (3–6) can be approximated as
Y1:N = 1YN+1 + Af(Y1:N,U1:N), (3–109)
where 1 denotes a N × 1 column vector of the constant 1 for every component, and
A = D−11:N is the N × N Legendre-Gauss-Radau integration matrix given by Eq. (3–108).
The integral optimal control problem can then be approximated via the following
finite-dimensional nonlinear programming problem. Minimize the cost function of
Eq. (3–80) subject to the algebraic constraints
Y1:N = 1YN+1 +Af(Y1:N ,U1:N), (3–110)
φ(Y1) = 0, (3–111)
The NLP described by Eqs. (3–80), (3–110), and (3–111) will be referred to as the
integral Legendre-Gauss-Radau collocation method.
92
3.3.4 KKT Conditions Using Integral Legendre-Gauss-Radau Collocation
The Karush-Kuhn-Tucker (KKT) first-order optimality conditions of the integral
Legendre-Gauss-Radau collocation method are given as
Y1:N = 1YN+1 + Af(Y1:N,U1:N), φ(Y1) = 0, (3–112)
0 = ∇U(
w⊤g(Y1:N ,U1:N) + 〈A⊤P1:N , f(Y1:N ,U1:N)〉)
(3–113)
P1:N = ∇Y(
w⊤g(Y1:N,U1:N) + 〈A⊤P1:N , f(Y1:N,U1:N)〉)
− e1∇Y〈ψ , φ(Y1)〉 (3–114)
N∑
i=1
Pi = −∇YΦ(YN+1). (3–115)
where (P1:N, ψ) are the KKT multipliers associated with the constraints of Eqs. (3–110)
and (3–111), respectively. Suppose now that we introduce the following transformed
dual variables:
p2:N =W−12:N,2:NP2:N, λN+1 = −
N∑
i=1
Pi . (3–116)
In addition, suppose we introduce the N × N matrix A†,
A†ij =wj
wiAji + wj , (3–117)
for (i , j = 1, ... ,N). We will now prove that A† is a backward integration matrix for the
space of polynomials of degree N − 2.
Theorem 2. The matrix A† described by Eq. (3–117) is a backwards integration matrix
for the space of polynomials of degree N − 2. More specifically, let p be a polynomial of
degree at most N − 2 and p ∈ RN−1 be the vector with i th component pi = p(τi), (i =
2, ... ,N). Since a polynomial of degree N − 2 is uniquely defined by its values at N − 1
points, p can be expressed exactly as
p(τ) =
N∑
j=2
pi · L†j (τ), L†j (τ) =N∏
i=2j 6=i
τ − τiτj − τi
. (3–118)
93
Furthermore, let p1 = p(τ1) be the extrapolated value of p(τ) evaluated at τ1 such that
p(τ1) ≈ p1 =N∑
j=2
pjL†j (τ1). (3–119)
Then the matrix A† approximates the integral
(A†p)i =
∫ +1
τi
p(t) dt. (3–120)
Proof. Let p denote the polynomials of degree at most N − 2 defined in the statement
of the theorem. Furthermore, let q denote a polynomial of degree N − 1 which satisfies
qj = q(τj), (j = 1, ... ,N). If p and q are smooth, real-valued functions then integration by
parts gives
∫ +1
−1
[
p(τ) ·∫ τ
+1
q(t) dt
]
dτ = −∫ −1
+1
p(τ) dτ ·∫ −1
+1
q(τ) dτ +
∫ +1
−1
[
q(τ) ·∫ +1
τ
p(t) dt
]
dτ .
(3–121)
Now, since p is a polynomial of degree at most N − 2 and q is a polynomial of degree at
most N − 1, then q ·∫ +1
τp dt and p ·
∫ τ
+1q dt are polynomials of degree at most 2N − 2.
Since Radau quadrature is exact for polynomials of degree 2N − 2, the integrals in
Eq. (3–121) can be replaced by their quadrature equivalents to obtain
N∑
j=1
wjpj ·∫ τ
+1
q(t) dt = −N∑
i=1
wipi ·N∑
j=1
wjpj +
N∑
j=1
wjqj ·∫ +1
τ
p(t) dt. (3–122)
Substituting∫ τ
+1q(t)dt = Aq and
∫ +1
τp(t) dt = A†p with the first column of A† consisting
of a N × 1 column vector of zeros, the following expression is obtained
(Aq)⊤Wp = −(Wq)⊤1N×NWp+ (Wq)⊤A†p,
q⊤(A⊤W +W1N×NW −WA†)p = 0.
(3–123)
Since p and q are arbitrary vectors, then
A⊤W +W1N×NW −WA† = 0, (3–124)
94
which implies that
A†ij =wj
wiAji + wj , (i , j = 1, ... ,N).
It has previously been shown that the matrix A defined as
Aij =wj
wiAji , (i , j = 1, ... ,N) (3–125)
has the form
Aij = −∫ τi
−1
L†j (τ)dτ + w1L
†j (τ1), (i , j = 2, ... ,N),
Ai1 = −w1, (i = 1, ... ,N),
A1j = w1L†j (τ1), (j = 2, ... ,N).
(3–126)
Since A†ij = Aij + wj , for (i , j = 1, ... ,N), then A† has the form
A†ij = −
∫ τi
−1
L†j (τ)dτ + w1L
†j (τ1) + wj , (i , j = 2, ... ,N),
A†i1 = 0, (i = 1, ... ,N),
A†1j = w1L
†j (τ1) + wj , (j = 2, ... ,N).
(3–127)
To better understand the structure of the matrix A†, suppose the N × 1 vector p is
multiplied by A† where pi = p(τi) and p(τ) is the polynomial of degree N − 2 defined in
Eq. (3–118). The resulting operation yields
(A†p)i =
N∑
j=2
w1pjL†j (τ1) +
N∑
j=2
wjpj −N∑
j=2
pj
∫ τi
−1
L†j (τ). (3–128)
The summation∑N
j=2 pjL†j (τ1) extrapolates the (N − 2)th-degree polynomial p(τ) to
the initial point τ1. Therefore the sum of the first and second terms in Eq. (3–128) is
a Legendre-Gauss-Radau quadrature approximation of∫ +1
−1p(τ)dτ . Furthermore,
because p(τ) is a polynomial of degree N − 2 it can be interpolated exactly as defined
in Eq. (3–118). Therefore the final term of Eq. (3–128) approximates∫ τ
−1p(τ)dτ . This
shows that A† is thus an integration matrix for the space of polynomials of degree N − 2
95
which approximates
(A†p)i =
∫ +1
τi
p(τ)d(τ).
Recall that the first column of A† is all zeros (that is, A†1 = 0), and that p is being
approximated by a polynomial of degree N − 2 which is uniquely defined by its values at
N − 1 points. Using the transformations described in Eq. (3–116) along with Eq. (3–117),
the KKT conditions of the integral Legendre-Gauss-Radau collocation method as shown
in Eqs. (3–112)–(3–115) can be written as
Y1:N = 1Y1 + Af(Y1:N,U1:N), φ(Y1) = 0, (3–129)
0 = ∇Ug(Y1:N,U1:N) + 〈(
A†2:Np2:N +∇YΦ(YN+1)
)
, ∇Uf(Y1:N ,U1:N)〉, (3–130)
p2:N = ∇Yg(Y2:N ,U2:N) + 〈(
A†2:N,2:Np2:N +∇YΦ(YN+1)
)
, ∇Yf(Y2:N,U2:N)〉, (3–131)
1
w1(P1 +∇Y〈ψ , φ(Y1)〉)
= ∇Yg(Y1,U1) + 〈(
A†1,2:Np2:N +∇YΦ(YN+1)
)
, ∇Yf(Y1,U1)〉, (3–132)
P1 = −N∑
i=2
wipi −∇YΦ(YN+1). (3–133)
Substituting the values of pi , (i = 2, ... ,N), and P1 obtained from Eqs. (3–131) and
(3–132), respectively, into Eq. (3–133) yields
N∑
i=1
wi
(
∇Yg(Yi ,Ui) + 〈(
A†i,2:Np2:N+∇YΦ(YN+1) , ∇Yf(Yi ,Ui)〉
=∇Y〈ψ , φ(Y1)〉 − ∇YΦ(YN+1),(3–134)
Equation (3–134) is a Legendre-Gauss-Radau quadrature approximation to the
continuous-time condition of Eq. (3–16). It can be seen that Eqs. (3–130) and (3–131)
are approximations to the continuous-time optimality conditions of Eqs. (3–14) and
(3–15), respectively. Therefore, it has been shown that Eqs. (3–129)–(3–131) along
with Eq. (3–134) form a set of necessary conditions for optimality and are a discrete
96
approximation to the necessary conditions for optimality of the integral form of the
optimal control problem as defined in Eqs. (3–5)–(3–7).
3.3.5 Differential Costate Estimate Using Integral Legendre-Gauss-RadauCollocation
The results of Sections 3.3.1–3.3.4 can now be used to define an estimate for the
differential costate using the integral costate estimate from the Legendre-Gauss-Radau
collocation method. In particular, the transformed necessary conditions of Eq. (3–97)–(3–101)
are equivalent to the transformed necessary conditions of Eq. (3–129)–(3–131) along
with Eq. (3–134) if the discrete approximations of λ(τ) and p(τ) are related as
λ1:N = λN+1 + A†p2:N , (3–135)
where the matrix A† is the integration matrix for the space of polynomials of degree N−2
defined in Eq. (3–117). Equation (3–135) can equivalently be written in differential form
such that
D†λ1:N = −p1:N (3–136)
where the matrix D† is a differentiation matrix for the space of polynomials of degree N −
1 defined by Eq. (3–90). It can be seen that applying the transformation of Eq. (3–135)
to the optimality conditions of the integral Legendre-Gauss-Radau collocation method
as given in Eqs. (3–129)–(3–131) along with Eq. (3–134) will result in the first-order
optimality conditions of problem the differential Legendre-Gauss-Radau collocation
method as given in Eqs. (3–97)–(3–101).
3.4 Discussion
While it may appear at first glance as if the costate estimate using either the integral
Legendre-Gauss or integral Legendre-Gauss-Radau method is the same, the estimate
obtained using either of the methods has nuances that distinguish it from the estimate
obtained using the other method. First, the N ×N Legendre-Gauss-Radau differentiation
matrix of Eq. (3–90) is singular for the space of polynomials of degree N − 1, while the
97
N × (N + 1) Legendre-Gauss differentiation matrix of Eq. (3–36) is full rank for the space
of polynomials of degree N . Second, the N × N matrix A† in Eq. (3–117) associated with
the integral Legendre-Gauss-Radau method is singular and simultaneously integrates
and interpolates a polynomial of degree N − 2, while the matrix N × N matrix A† in
Eq. (3–69) is an integration matrix for the space of polynomials of degree N − 1. Finally,
although both integral collocation methods provide estimates of the differential costate,
λ(τ), at all discretization points in the domain (including both endpoints), the integral
Legendre-Gauss collocation method produces an approximation of integral costate,
p(τ), at only the N Legendre-Gauss points while the integral Legendre-Gauss-Radau
collocation method provides estimates of p(τ) at N − 1 interior Legendre-Gauss-Radau
points and extrapolates the value of p(τ) to the initial (τ = −1) Legendre-Gauss-Radau
point.
3.5 Concluding Remarks
A method was presented for costate estimation of an optimal control problem
using orthogonal collocation at Legendre-Gauss and Legendre-Gauss-Radau points
when the dynamic constraints are presented in integral form. A key feature of these
collocation schemes is that the inverse of the matrix associated with an implicit LG (or
LGR) integration scheme is the LG (or LGR) differentiation matrix. Hence, the methods
presented in this chapter can be thought of as either an implicit integration method or
a differential method. It was shown that the KKT multipliers stemming from the implicit
integration transcription can be related to the costate of the differential form of the
problem via an integration matrix. The LG collocation scheme yielded a costate estimate
which was approximated by a polynomial of degree N , whereas the costate estimate for
the LGR collocation scheme was approximated by a polynomial of degree N − 1. The
relationship between the costate of the differential formulation and the dual multipliers
of the implicit integral formulation provided an equivalence between the first-order
98
optimality conditions of the optimal control problem when posed with the constraints in
either form.
99
CHAPTER 4MOTIVATION FOR NEW COSTATE ESTIMATE
In this chapter a motivation is given for developing new costate estimates for
variable-order collocation at the Legendre-Gauss (LG) and flipped Legendre-Gauss-Radau
(LGR) points when solving problems with active state inequality path constraints. In
particular, a previously derived costate estimate for variable-order collocation at the LG
and flipped LGR points will be presented [1]. It will be shown that in the case when the
costate is discontinuous (as is the case in the presence of active state inequality path
constraints), this costate estimate leads to a set of first-order optimality conditions of the
NLP that are not equivalent to the discrete form of the variational optimality conditions.
This lack of equivalence leads to an inaccurate approximation of the costate.
The following notation and conventions will be used throughout this chapter in
order to make the exposition more clear. First, all vector functions of time are denoted
as row vectors, that is, if y(τ) ∈ Rn is a vector function of the scalar variable τ , then
y(τ) = [y1(τ), · · · , yn(τ)]. Next, any capital boldface character, Y, denotes a matrix of
size M × n, where each row of Yi corresponds to the evaluation of a function y(τ) at a
particular value τ = τi . Next, the notation Yi:j denotes rows i through j of the matrix Y,
except when referring to a differentiation matrix D or the integration matrix A, in which
case Di and Ai refers to the i th column of D and A. Finally, D⊤ denotes the transpose
of matrix D, and D⊤i denotes the transpose of the i th column of D. Given vectors x and
y ∈ Rn, the notation 〈x, y〉 is used to denote the standard inner product between x
and y. Furthermore, if f : Rn −→ Rm, then ∇f is the m by n Jacobian matrix whose
i th row is ∇fi . In particular, the gradient of a scalar-valued function is a row vector. If
φ : Rm×n −→ R and Y is an m by n matrix, then ∇φ denotes the m by n matrix whose
(i , j) element is (∇φ(Y))ij = ∂φ(Y)/∂Yij .
The remainder of this chapter is organized as follows. Section 4.1 reformulates the
continuous-time Bolza optimal control problem of Section 2.1 such that the domain is
100
divided into a mesh and each mesh interval is defined in the domain τ ∈ [−1,+1]. The
transformation of variables is done to facilitate comparison of the first-order optimality
conditions of the continuous-time problem with the modified optimality conditions
of the variable-order collocation methods. Next, Section 4.2 presents a previously
derived costate estimate for variable-order collocation at the LG points, develops the
transformed adjoint system, and shows that the costate estimate is only valid for the
case when the costate is continuous across interval boundaries in the domain. Similarly,
in Section 4.3 a previously derived costate estimate for variable-order collocation at
the flipped LGR points is presented, the transformed adjoint system is developed, and
it is shown that the costate estimate does not lead to an accurate approximation of
the continuous-time optimal costate in the case when the costate is discontinuous.
Finally, Section 4.4 discusses the implications of the inaccuracy in the costate estimates
presented.
4.1 Continuous-Time Bolza Optimal Control Problem
Consider again the continuous Bolza problem that was presented in Section 2.1. To
simplify comparisons with the transformed adjoint system, the domain t ∈ [t0, tf ] = I
is now divided into K sub-intervals Sk = [Tk−1,Tk ] ⊆ [t0, tf ], (k = 1, ... ,K), where
T0 = t0, TK = tf , Tk−1 < Tk , (k = 1, ... ,K), and⋃K
k=1 Sk = I. Furthermore,
without loss of generality the optimal control problem can be scaled by transforming the
independent variable in each sub-interval from t ∈ [Tk−1,Tk ] to τ (k) ∈ [−1,+1] via the
affine transformation
t =Tk −Tk−12
τ (k) +Tk + Tk−12
(4–1)
such that
dt =Tk − Tk−12
dτ (k) ≡ h(k)
2, where h(k) ≡ Tk − Tk−1. (4–2)
The optimal control problem problem then becomes to determine the state y(k)(τ) ∈ Rn,
and the control u(k)(τ) ∈ Rm in each sub-interval (k = 1, ... ,K), to minimize the cost
101
functional
J = Φ(y(K)(+1)) +
K∑
k=1
h(k)
2
∫ +1
−1
g(y(k)(τ), u(k)(τ))dτ (4–3)
subject to the dynamic constraints
y(k)(τ) =h(k)
2f(y(k)(τ), u(k)(τ)), (k = 1, ... ,K), (4–4)
the boundary conditions
φ(y(1)(−1)) = 0, (4–5)
and the state and control inequality path constraint
C(y(k)(τ), u(k)(τ)) ≤ 0, (k = 1, ... ,K). (4–6)
4.1.1 First-Order Optimality Conditions of Continuous Problem
The first-order optimality conditions of the optimal control problem given by
Eqs. (4–3)–(4–6) can be derived from the calculus of variations in the manner described
by Section 2.1. They are given as
y(k) = f(y(k), u(k)), φ(y(1)(−1)) = 0, (k = 1, ... ,K), (4–7)
0 = ∇uH(y(k), u(k),λ(k),µ(k)), (k = 1, ... ,K), (4–8)
−λ(k) = h(k)
2∇yH(y(k), u(k),λ(k),µ(k)), (k = 1, ... ,K), (4–9)
λ(1)(−1) = ∇y 〈ψ,φ(y(1))〉|τ=−1, (4–10)
λ(K)(+1) = ∇yΦ(y(K))|τ=+1, (4–11)
C(y(k), u(k)) ≤ 0, µ(k)(τ) ≤ 0, 〈µ(k),C(y(k), u(k))〉 = 0, (k = 1, ... ,K), (4–12)
where (λ(k),µ(k)) are the Lagrange multipliers associated with the constraints of
Eqs. (4–4) and (4–6) in interval k , and ψ are the Lagrange multipliers associated with
the boundary conditions of Eq. (4–5). Furthermore, the Hamiltonian in interval k is given
as
H(y(k), u(k),λ(k),µ(k)) = g(k) + 〈λ(k), f(k)〉 − 〈µ(k),C(k)〉. (4–13)
102
4.2 Variable-Order Collocation at Legendre-Gauss Points
The optimal control problem of Eqs. (4–3)–(4–6) is now discretized using variable-order
collocation at the Legendre-Gauss (LG) points as described in Section 2.5.4. First, recall
that the LG points (τ0, ... , τN+1) are defined in the domain τ ∈ (−1,+1) such that
τ0 = −1 and τN+1 = +1 are noncollocated points. The state is then approximated in
each mesh interval k as
y(k)(τ) ≈ Y(k)(τ) =Nk∑
i=0
Y(k)i L
(k)i (τ), L
(k)i (τ) =
Nk∏
j=0
i 6=j
τ − τ (k)j
τi − τ (k)j, (4–14)
Differentiating Y(k)(τ) in Eq. (4–14) with respect to τ , yields
Y(k)(τj) ≈Nk∑
i=0
Y(k)i L
(k)i (τj) = [D
(k)Y(k)0:Nk]j , (4–15)
where D(k)ij = L(k)i (τ)j , (i = 1, ... ,Nk, j = 0, ... ,Nk) are the components of the
Nk × (Nk + 1) Legendre-Gauss (LG) differentiation matrix in the k th mesh interval. It is
noted that when implementing the variable-order LG method, a single variable is used
for the value of the state at the end of mesh interval k and the start of mesh interval
k + 1, that is, Y(k)Nk+1 ≡ Y(k+1)0 , 1 ≤ k ≤ K − 1 such that continuity in the state is enforced.
Next, the cost functional of Eq. (2–1) is approximated using a multiple-interval LG
quadrature. The finite-dimensional approximation of the continuous-time optimal control
problem of Eqs. (2–1)–(2–4) is then given as follows. Minimize the cost function
J ≈ Φ(Y(K)NK+1) +K∑
k=1
Nk∑
j=1
h(k)
2w(k)j g(Y
(k)j ,U
(k)j ), (4–16)
103
subject to the algebraic constraints
D(k)Y(k)0:Nk
=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K) (4–17)
Y(k+1)0 = Y
(k)0 +
h(k)
2
Nk∑
j=1
w(k)j f(Y
(k)j ,U
(k)j ), (k = 1, ... ,K − 1) (4–18)
Y(K)N+1 = Y
(K)0 +
h(K)
2
Nk∑
j=1
w(K)j f(Y
(K)j ,U
(K)j ), (4–19)
φ(Y(1)0 ) = 0, (4–20)
C(Y(k)1:Nk,U(k)1:Nk ) ≤ 0, (k = 1, ... ,K), (4–21)
where w(k) = (w (k)1 , ... ,w(k)N k) are the LG quadrature weights in interval k . It is noted
for LG collocation that Eq. (4–18) provides an LG quadrature approximation, Y(k)0 , of
the state at the final noncollocated point τ (k)N+1 = +1 in interval (k = 1, ... ,K − 1), while
Eq. (4–19) provides an LG quadrature approximation, Y(K)N+1, of the state at the final
noncollocated point of the domain, tf = τ (k)N+1 = +1.
4.2.1 KKT Conditions of Variable-Order Legendre-Gauss Collocation Method
The first-order optimality conditions of the discrete problem given by Eqs. (4–16)–(4–21),
also called the KKT conditions of the NLP, are now derived. First, the Lagrangian is
defined as
L = Φ(Y(K)NK+1)− 〈ψ,φ(Y(1)0 )〉+
K∑
k=1
Nk∑
i=1
(
h(k)
2w(k)i g
(k)i − 〈Γ(k)i ,C(k)i 〉
)
−K∑
k=1
Nk∑
i=1
(
〈Λ(k)i ,D(k)i,0:NkY(k)0:Nk− h
(k)
2f(k)i 〉
)
−K−1∑
k=1
Nk∑
i=1
(
〈Λ(k)Nk+1,Yk+10 − Y(k)0 −
h(k)
2w(k)i f(k)i 〉
)
−NK∑
i=1
(
〈Λ(K)NK+1,Y(K)NK+1
− Y(K)0 − h(K)
2w(K)i f(K)i 〉
)
(4–22)
where (Λ(k)i ,Λ(k)Nk+1,Γ(k)i ) are the Lagrange multipliers associated with the dynamic
constraints of Eq. (4–17), the quadrature constraints of Eqs. (4–18)–(4–19), and the
104
inequality path constraint of Eq. (4–21), respectively, in interval k at the LGR point
τi . Furthermore ψ denotes the Lagrange multipliers associated with the boundary
conditions of Eq. (4–20). Note that function dependencies have been omitted for clarity,
such that g(k)i ≡ g(Y(k)i ,U(k)i ), and similarly f(k)i ≡ f(Y(k)i ,U(k)i ) and C(k)i ≡ C(Y(k)i ,U(k)i ).
The KKT conditions of the NLP are then given as
D(k)0:NkY(k)0:Nk=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk ), φ(Y
(1)0 ) = 0, (k = 1, ... ,K), (4–23)
Y(k+1)0 = Y
(k)0 +
Nk∑
i=1
w(k)i f(Y
(k)i ,U
(k)i ), (k = 1, ... ,K − 1), (4–24)
Y(k)NK+1
= Y(K)0 +
NK∑
i=1
w(K)i f(Y
(K)i ,U
(K)i ), (4–25)
0 = ∇UH(Y(k)1:Nk,U(k)1:Nk ,Λ
(k)1:Nk+1
,Γ(k)1:Nk), (k = 1, ... ,K), (4–26)
D(k)⊤1:NkΛ(k)1:Nk=h(k)
2∇YH(Y(k)1:Nk ,U
(k)1:Nk,Λ(k)1:Nk+1
,Γ(k)1:Nk), (k = 1, ... ,K), (4–27)
D(1)⊤0 Λ(1)1:N1= Λ
(1)N1+1
−∇Y〈ψ,φ(Y(1)0 )〉, (4–28)
D(k)⊤0 Λ(k)1:N1= Λ
(k)Nk+1
− Λ(k−1)Nk+1, (k = 2, ... ,K), (4–29)
Λ(K)NK+1
= ∇YΦ(Y(K)NK+1), (4–30)
C(Y(k)1:Nk ,U(k)1:Nk) ≤ 0, Γ(k)1:Nk ≤ 0, 〈Γ
(k)1:Nk,C(Y(k)1:Nk ,U
(k)1:Nk)〉 = 0, (k = 1, ... ,K). (4–31)
The discrete Hamiltonian in interval k is given as
H(Y(k)1:Nk,U1:Nk ,Λ
(k)1:Nk+1
,Γ(k)1:Nk) = w (k)⊤g
(k)1:Nk+ 〈Λ(k)1:Nk +W
(k)1Λ(k)Nk+1, f(k)1:Nk〉−〈 2h(k)Γ(k)1:Nk,C(k)1:Nk〉,
whereW(k) is a N × N diagonal matrix of LG quadrature weights in interval k , and 1 is a
N × 1 column vector of ones.
4.2.2 Costate Estimate and Transformed Adjoint System
Consider the following costate estimate, first derived by Ref. [1], that relates the
KKT conditions multipliers of Eqs. (4–23)–(4–31) to the dual variables of the first-order
105
optimality [given by Eqs. (4–7)–(4–12)] of the continuous-time optimal control problem:
µ(k)i =
2
h(k)Γ(k)i
w(k)i
, (i = 1, ... ,Nk), (k = 1, ... ,K), (4–32)
λ(k)i = Λ
(k)Nk+1
+Λ(k)i
w(k)i
, (i = 1, ... ,Nk), (k = 1, ... ,K), (4–33)
λ(k)0 = Λ
(k)Nk+1
− D(k)⊤
0 Λ(k)1:Nk, (k = 1, ... ,K), (4–34)
λ(k)Nk+1
= Λ(k)Nk+1, (k = 1, ... ,K). (4–35)
Next, let D† be an Nk × (Nk + 1) matrix defined as follows:
D†ij = −
wj
wiDji , and D
†i,N+1 = −
N∑
j=1
D†ij (4–36)
for i = 1, ... ,N . Based on the theory developed in [36], D† is a differentiation matrix for
the space of polynomials of degree N . That is, if b is a polynomial of degree at most N
and b ∈ RN is the vector with i-th element bi = b(τi), then
(D†b)i = b(τi).
106
Using the adjoint differentiation matrix defined in Eq. (4–36) along with the transformations
given by Eqs. (4–32)–(4–35), the KKT system of Eqs. (4–23)–(4–31) can be rewritten as
D(k)0:NkY(k)0:Nk=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk ), φ(Y
(1)0 ) = 0, (k = 1, ... ,K), (4–37)
Y(k+1)0 = Y
(k)0 +
Nk∑
i=1
w(k)i f(Y
(k)i ,U
(k)i ), (k = 1, ... ,K − 1), (4–38)
Y(K)NK+1
= Y(K)0 +
NK∑
i=1
w(K)i f(Y
(K)i ,U
(K)i ), (4–39)
0 = ∇UH(Y(k)1:Nk ,U(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk), (k = 1, ... ,K), (4–40)
D†(k)1:Nkλ(k)1:Nk= −h
(k)
2∇YH(Y(k)1:Nk ,U
(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk), (k = 1, ... ,K), (4–41)
λ(1)0 = ∇Y〈ψ,φ(Y(1)0 )〉, (4–42)
λ(k)0 = λ
(k−1)Nk+1, (k = 2, ... ,K), (4–43)
λ(K)NK+1
= ∇YΦ(Y(K)NK+1), (4–44)
C(Y(k)1:Nk ,U(k)1:Nk) ≤ 0, µ
(k)1:Nk≤ 0, 〈µ(k)1:Nk ,C(Y
(k)1:Nk,U(k)1:Nk )〉 = 0, (k = 1, ... ,K). (4–45)
where H(Y(k)1:Nk ,U(k)1:Nk,λ(k)1:Nk+1
,µ(k)1:Nk) is a discrete form of the Hamiltonian given by
Eq. (4–13). It can be seen that the transformed optimality conditions of Eqs. (4–37)–(4–42)
are a discrete form of the first-order optimality conditions of the continuous-time optimal
control problem given by Eqs. (4–7)–(4–10). Furthermore, the costate terminal condition
given by Eq. (4–11) is satisfied in the discrete transformed adjoint system by Eq. (4–44),
and the complementary slackness conditions of Eq. (4–12) are satisfied by the discrete
condition of Eq. (4–45). From the relationship given in the transformed adjoint system
by Eq. (4–43), however, it is seen that in the discrete problem the costate across
interval boundaries must be continuous. It was previously shown in Section 2.3
that discontinuities in the costate stem from inequality path constraint activity in the
solution domain. Therefore in the presence of state inequality constraints, the costate
becomes discontinuous, and the transformed adjoint system of the NLP is not a discrete
approximation of the first-order optimality conditions of the continuous-time problem.
107
4.3 Variable-Order Collocation at Flipped Legendre-Gauss-Radau Points
The optimal control problem of Eqs. (4–3)–(4–6) is now discretized using variable-order
collocation at the flipped Legendre-Gauss-Radau points as described in Section
2.5.6. First, recall that the flipped LGR points (τ0, ... , τN) are defined in the domain
τ ∈ (−1,+1] such that τ0 = −1 is a noncollocated point. The state in each mesh interval
k is approximated as
y(k)(τ) ≈ Y(k)(τ) =Nk∑
i=0
Y(k)i L
(k)i (τ), L
(k)i (τ) =
Nk∏
j=0
i 6=j
τ − τ(k)j
τ (k)i − τ (k)j, (4–46)
Differentiating Y(k)(τ) in Eq. (4–46) with respect to τ , yields
Y(k)(τj) ≈Nk∑
i=0
Y(k)i L
(k)i (τj) = [D
(k)Y(k)0:Nk]j , (4–47)
where D(k)ij = L(k)i (τ)j , (i = 1, ... ,Nk, j = 0, ... ,Nk) are the components of the
Nk×(Nk+1) flipped Legendre-Gauss-Radau (LGR) differentiation matrix in the k th mesh
interval.
The optimal control problem then becomes to minimize the cost
J ≈ Φ(Y(K)NK ) +K∑
k=1
Nk∑
j=1
h(k)
2w(k)j g(Y
(k)j ,U
(k)j ), (4–48)
subject to the algebraic constraints
D(k)Y(k)0:Nk
=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (4–49)
φ(Y(1)0 ) = 0, (4–50)
C(Y(k)1:Nk,U(k)1:Nk ) ≤ 0, (k = 1, ... ,K), (4–51)
where w(k) = (w (k)1 , ... ,w(k)N k) are the flipped LGR quadrature weights in interval k .
When implementing the flipped variable-order LGR method, a single variable is used for
the value of the state at the end of mesh interval k and the start of mesh interval k + 1,
that is, Y(k−1)Nk≡ Y(k)0 , 2 ≤ k ≤ K such that continuity in the state is enforced.
108
4.3.1 KKT Conditions of Variable-Order Flipped Legendre-Gauss-Radau Colloca-tion Method
The first-order optimality conditions of the discrete problem given by Eqs. (4–48)–(4–51),
also called the KKT conditions of the NLP, are now derived. First, the Lagrangian is
defined as
L = Φ(Y(K)NK )− 〈ψ,φ(Y(1)0 )〉+
K∑
k=1
Nk∑
i=1
(
h(k)
2w(k)i g
(k)i − 〈Γ(k)i ,C(k)i 〉
)
−N1∑
i=1
(
〈Λ(1)i ,D(1)i,0 Y(1)0 + D(1)i,1:N1Y(1)1:N1− h
(k)
2f(1)i 〉
)
−K−1∑
k=1
Nk∑
i=1
(
〈Λ(k)i ,D(k)i,0 Y(k−1)N + D(k)i,1:NkY(k)1:Nk− h
(k)
2f(k)i 〉
)
(4–52)
where (Λ(k)i ,Γ(k)i ) are the Lagrange multipliers associated with the dynamic constraints
of Eq. (4–49) and the inequality path constraint of Eq. (4–51) in interval k at the LGR
point τi . Furthermore ψ denotes the Lagrange multipliers associated with the boundary
conditions of Eq. (4–50). Note that function dependencies have been omitted for clarity,
such that g(k)i ≡ g(Y(k)i ,U(k)i ), and similarly f(k)i ≡ f(Y(k)i ,U(k)i ) and C(k)i ≡ C(Y(k)i ,U(k)i ).
The KKT conditions of the NLP are then given as
D(k)0:NkY(k)0:Nk=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk ), φ(Y
(1)0 ) = 0, (k = 1, ... ,K), (4–53)
0 = ∇UH(Y(k)1:Nk,U(k)1:Nk ,Λ
(k)1:Nk,Γ(k)1:Nk), (k = 1, ... ,K), (4–54)
D(k)⊤1:NkΛ(k)1:Nk=h(k)
2∇YH(Y(k)1:Nk ,U
(k)1:Nk,Λ(k)1:Nk,Γ(k)1:Nk)
− eNkD(k+1)0 Λ
(k+1)1:Nk, (k = 1, ... ,K − 1),
(4–55)
D(K)⊤1:NKΛ(K)1:NK=h(k)
2∇YH(Y(K)1:NK ,U
(K)1:NK,Λ(K)1:NK,Γ(K)1:NK) + eNK∇YΦ(Y(K)NK ), (4–56)
D(1)⊤0 Λ(1)1:N1= −∇Y〈ψ,φ(Y(1)0 )〉, (4–57)
C(Y(k)1:Nk ,U(k)1:Nk) ≤ 0, Γ(k)1:Nk ≤ 0, 〈Γ
(k)1:Nk,C(Y(k)1:Nk ,U
(k)1:Nk)〉 = 0, (k = 1, ... ,K), (4–58)
109
where eN denotes the N th column of the identity matrix and the discrete Hamiltonian in
interval k is
H(Y(k)1:Nk,U1:Nk ,Λ
(k)1:Nk) = w (k)⊤g
(k)1:Nk+ 〈Λ(k)1:Nk , f
(k)1:Nk〉 − 〈 2
h(k)Γ(k)1:Nk,C(k)1:Nk〉. (4–59)
4.3.2 Costate Estimate and Transformed Adjoint System
Consider the following costate estimate, first derived by Ref. [1], that relates the
KKT conditions multipliers of Eqs. (4–53)–(4–58) to the dual variables of the first-order
optimality [given by Eqs. (4–7)–(4–12)] of the continuous-time optimal control problem:
µ(k)i =
2
h(k)Γ(k)i
w(k)i
, (i = 1, ... ,Nk), (k = 1, ... ,K), (4–60)
λ(k)i =
Λ(k)i
w(k)i
, (i = 1, ... ,Nk), (k = 1, ... ,K), (4–61)
λ(k)0 = −D(k)
⊤
0 Λ(k)1:Nk, (k = 1, ... ,K). (4–62)
Next, let D† be an Nk × Nk matrix defined as follows:
D†ij =
−DNN + 1wN, i = j = N
−wjwiDji , i , j = 2, ... ,N.
(4–63)
Based on the theory developed in [36], D† is a differentiation matrix for the space of
polynomials of degree N − 1. That is, if b is a polynomial of degree at most N − 1 and
b ∈ RN is the vector with i-th element bi = b(τi), then
(D†b)i = b(τi).
110
Using the adjoint differentiation matrix defined in Eq. (4–63) along with the transformations
given by Eqs. (4–60)–(4–62), the KKT system of Eqs. (4–53)–(4–58) can be rewritten as
D(k)0:NkY(k)0:Nk=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk ), φ(Y
(1)0 ) = 0, (k = 1, ... ,K), (4–64)
0 = ∇UH(Y(k)1:Nk ,U(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk), (k = 1, ... ,K), (4–65)
D† (k)1:Nkλ(k)1:Nk= −h
(k)
2∇YH(Y(k)1:Nk ,U
(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk)
− eNkwNk
(
λ(k+1)0 − λ(k)N
)
, (k = 1, ... ,K − 1), (4–66)
D† (K)1:NK
λ(K)1:NK= −h
(k)
2∇YH(Y(K)1:NK ,U
(K)1:NK,λ(K)1:NK,µ(K)1:NK)
− eNKwNK
(
∇YΦ(Y(K)N )− λ(K)N)
, (4–67)
λ(1)0 = ∇Y〈ψ,φ(Y(1)0 )〉, (4–68)
C(Y(k)1:Nk ,U(k)1:Nk) ≤ 0, µ
(k)1:Nk≤ 0, 〈µ(k)1:Nk ,C(Y
(k)1:Nk,U(k)1:Nk )〉 = 0, (k = 1, ... ,K), (4–69)
where H(Y(k)1:Nk ,U(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk) is a discrete form of the Hamiltonian given by
Eq. (4–13). These equations are incomplete because a new variable λ(k)0 was
introduced without adding a new equation. An equation for this new variable can be
developed by manipulating the matrix D(k). Consider now a (N + 1) × 1 column vector
composed of ones. The components of the vector D(k)1 are the derivatives at the
collocation points of the polynomial whose value is 1 at τi , (i = 1, ... ,N + 1) in interval
k . The derivative of the constant polynomial is zero everywhere. Thus, D(k)1 = 0, which
implies that
D(k)0 = −Nk∑
j=1
D(k)1:Nk , j ,
D(k)⊤0 Λ(k) =
Nk∑
i=1
D(k)i, 0Λ(k)i = −
Nk∑
i=1
Nk∑
j=1
D(k)i, j Λ(k)i ,
−λ(k)0 = −λ(k)Nk +Nk∑
i=1
Nk∑
j=1
w(k)i λ
(k)j D†(k)
i,j = −λ(k)Nk +Nk∑
j=1
w(k)j [D
†(k)λ(k)]j ,
(4–70)
111
where the relationships in (4–61)–(4–62) and (4–63) were used to obtain Eq. (4–70).
It can be seen that this relationship approximates the integral of the costate dynamics
across the interval k via a Radau quadrature. That is, it approximates the relationship
−λ(−1) = −λ(+1) +∫ +1
−1
λ(τ)dτ .
Combining Eqs. (4–66)–(4–67) with Eq. (4–70), the complete transformed adjoint
system can then be written as
D(k)0:NkY(k)0:Nk=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk ), φ(Y
(1)0 ) = 0, (k = 1, ... ,K), (4–71)
0 = ∇UH(Y(k)1:Nk ,U(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk), (k = 1, ... ,K), (4–72)
D† (k)1:Nkλ(k)1:Nk= −h
(k)
2∇YH(Y(k)1:Nk ,U
(k)1:Nk,λ(k)1:Nk,µ(k)1:Nk)
− eNkwNk
(
λ(k+1)0 − λ(k)N
)
, (k = 1, ... ,K − 1), (4–73)
λ(k+1)0 = λ
(k)0 −
h(k)
2
Nk∑
j=1
w(k)j ∇YH(Y(k)j ,U(k)j ,λ(k)j ,µ(k)j ), (k = 1, ... ,K − 1), (4–74)
D† (K)1:NK
λ(K)1:NK= −h
(k)
2∇YH(Y(K)1:NK ,U
(K)1:NK,λ(K)1:NK,µ(K)1:NK)
− eNKwNK
(
∇YΦ(Y(K)N )− λ(K)N)
, (4–75)
∇YΦ(Y(K)N ) = λ(K)0 −h(K)
2
NK∑
j=1
w(K)j ∇YH(Y(K)j ,U(K)j ,λ(K)j ,µ(K)j ), (4–76)
λ(1)0 = ∇Y〈ψ,φ(Y(1)0 )〉, (4–77)
C(Y(k)1:Nk ,U(k)1:Nk) ≤ 0, µ(k)1:Nk ≤ 0, 〈µ
(k)1:Nk,C(Y(k)1:Nk ,U
(k)1:Nk)〉 = 0, (k = 1, ... ,K), (4–78)
It is seen that Eq. (4–76) is a Legendre-Gauss-Radau quadrature of the costate
dynamics across interval K . Consequently, the right-hand side of Eq. (4–76) approximates
the costate at the final point in the domain. Eq. (4–76) is thus a subtle way of enforcing
the relationship λ(K)N = ∇YΦ(YNk) and it is expected that the last term of Eq. (4–75)
will be small while the remaining terms in Eq. (4–75) are a collocation collocation
scheme for the continuous adjoint equation in the final interval K . Similarly, the
112
right-hand side of Eq. (4–74) approximates the costate, λ(k)N , at the terminal point
in interval k via a Legendre-Gauss-Radau quadrature of the costate dynamics for
(k = 1, ... ,K − 1). Equation (4–74) is therefore a subtle way of enforcing the relationship
λ(k)N = λ
(k+1)0 , (k = 1, ... ,K − 1). If the costate is continuous across an interval
boundary, the relationship λ(k)N = λ(k+1)0 holds true and the last term of Eq. (4–73)
will be small while the remaining terms in Eq. (4–73) are a collocation scheme for the
continuous adjoint equation. Therefore in cases when the costate is continuous across
an interval boundary, the transformed optimality conditions of Eqs. (4–71)–(4–78) are
a discrete form of the first-order optimality conditions [given by Eqs. (4–7)–(4–12)] of
the continuous-time optimal control problem. However, it has previously been shown in
Section 2.3 that the presence of active state inequality path constraints in the solution
domain may cause discontinuities in the costate. Therefore, in the presence of active
state inequality path constraints in the solution domain, λ(k)N 6= λ(k+1)0 at the entrance
or exit of a constrained arc, and the last term of Eq. (4–73) will not be small. Therefore
Eq. (4–73) will not be a collocation scheme for the continuous adjoint equation using this
costate estimate.
4.4 Discussion
In this chapter a method first derived by Ref. [1] for obtaining costate estimates
from the KKT multipliers of the NLP was presented. This derivation showed that if
the costate is continuous, variable-order collocation at the LG and LGR points yields
a set of transformed optimality conditions of the KKT system which are a discrete
representation of the continuous-time first-order necessary conditions of the optimal
control problem, as can be seen in Fig. (4-1). If the costate is discontinuous, however,
variable-order collocation at the LG and LGR points yields a set of transformed
optimality conditions of the KKT system which are an inexact discrete representation
of the continuous-time first-order necessary conditions. This result was first shown by
113
Ref. [74], who suggested that the costate estimate must be modified in order to account
for the costate discontinuities:
“. . . high-accuracy approximations are achieved by the proposed
hp-method if the costate is continuous. If mesh points are at the location
of discontinuity in the costate, the transformed adjoint system is an inexact
discrete representation of the continuous-time first-order necessary
conditions. For a dynamic refinement algorithm that will exactly locate
the switch in activity of inequality path constraints, it is likely that the costate
may be discontinuous at mesh points. It is necessary to determine how
to make the transformed adjoint system a discrete representation of the
continuous-time first-order necessary conditions for a discontinuous costate
solution if mesh points are at the location of discontinuity in the costate”.
It was previously shown in Section 2.3 that discontinuities in the costate stem from
inequality path constraint activity in the solution domain. Therefore in the presence
of state inequality constraints, the costate becomes discontinuous, and the first-order
optimality conditions of the NLP are not a discrete approximation of the first-order
optimality conditions of the continuous-time problem. Therefore, in this research a new
method of costate estimation for variable-order collocation at LG and LGR points will be
derived using three different methods. Specifically, a costate estimate using the method
indirect adjoining with continuous multipliers will be presented. It will be shown that
this method for costate estimation using variable-order collocation at the LG and LGR
points leads to a transformed adjoint system which is a discrete representation of the
continuous-time first-order necessary conditions even in the presence of state inequality
path constraints.
114
Figure 4-1. Relationship between the direct and indirect methods for solving an optimalcontrol problem. In the indirect method, the problem is first optimized through thecalculus of variations, leading to a set of conditions which can then be discretized andsolved. In the direct method, the problem is first discretized and transcribed to an NLP,then it is optimized by solving the KKT system. The two systems are equivalent onlywhen the costate is continuous in the solution domain.
115
CHAPTER 5COSTATE ESTIMATION FOR STATE CONSTRAINED PROBLEMS
As was shown in Chapter 4, previous research has successfully derived a
high-accuracy estimate of the costate using collocation at the Legendre-Gauss and
Legendre-Gauss-Radau points for the case of a problem with no active state inequality
path constraints. However, Ref. [1] showed that in the case when the costate is
discontinuous (as is the case in the presence of active state inequality path constraints),
this previously derived costate estimate leads to a set of first-order optimality conditions
of the NLP that are not equivalent to the discrete form of the variational optimality
conditions. This non-equivalence leads to an inaccurate approximation of the costate. In
order to rectify this inacuracy, in this chapter a method for estimating the costate of state
inequality path constrained optimal control problems using collocation at LG and LGR
points is developed using the method of indirect adjoining with continuous multipliers.
The method of indirect adjoining with continuous multipliers was chosen to
develop a costate estimate over the direct and indirect adjoining methods for two
reasons. First, the method of indirect adjoining requires a modification of the original
problem formulation through index-reduction of the differential-algebraic equations. The
reformulation of the problem must be done analytically, and requires prior knowledge
of the solution structure. Thus, when using an automated solution process (such as
a mesh refinement technique), this procedure might be cumbersome to implement.
Second, both the methods of indirect and direct adjoining result in a discontinuous
costate. Because discontinuities are difficult to approximate numerically, both these
methods may yield large errors in the costate estimate if the location of the discontinuity
is not exact. Thus, because the method of indirect adjoining with continuous multipliers
yields a continuous costate, it offers an advantage over the methods of direct and
indirect adjoining which approximate a discontinuous costate.
116
Similar to the approach of Chapter 4, the following notation and conventions will
be used throughout this chapter to make the exposition more clear. First, all vector
functions of time are denoted as row vectors, that is, if y(τ) ∈ Rn is a vector function of
the scalar variable τ , then y(τ) = [y1(τ), · · · , yn(τ)]. Next, any capital boldface character,
Y, denotes a matrix of size M × n, where each row of Yi corresponds to the evaluation
of a function y(τ) at a particular value τ = τi . Next, the notation Yi:j denotes rows
i through j of the matrix Y, except when referring to a differentiation matrix D or the
integration matrix A, in which case Di and Ai refers to the i th column of D and A. Finally,
D⊤ denotes the transpose of matrix D, and D⊤i denotes the transpose of the i th column
of D. Given vectors x and y ∈ Rn, the notation 〈x, y〉 is used to denote the standard inner
product between x and y. Furthermore, if f : Rn −→ Rm, then ∇f is the m by n Jacobian
matrix whose i th row is ∇fi . In particular, the gradient of a scalar-valued function is a row
vector. If φ : Rm×n −→ R and Y is an m by n matrix, then ∇φ denotes the m by n matrix
whose (i , j) element is (∇φ(Y))ij = ∂φ(Y)/∂Yij .
The remainder of this chapter is organized as follows. First, Section 5.1 formulates
the continuous-time state inequality path constrained optimal control problem and
states the first order optimality conditions of the continuous problem. Next, in Sections
5.2 and 5.3 a new costate estimate is derived using variable-order collocation at the
Legendre-Gauss and flipped Legendre-Gauss-Radau points, respectively, through the
method of indirect adjoining with continuous multipliers. It is shown for each of these
derived costate estimates that the transformed first-order optimality conditions of the
NLP are a discrete form of the first-order optimality conditions of the continuous-time
optimal control problem. Finally, in Section 5.4 the derived costate estimates are
discussed, and conclusions are given.
5.1 Continuous-Time State Inequality Path Constrained Optimal Control Problem
The state inequality path constrained optimal control problem to be studied in
the remainder of this chapter is now presented. To simplify comparisons with the
117
transformed adjoint system, the domain t ∈ [t0, tf ] = I is divided into K intervals
Sk = [Tk−1,Tk ] ⊆ [t0, tf ], (k = 1, ... ,K), where T0 = t0, TK = tf , Tk−1 < Tk , (k =
1, ... ,K), and⋃K
k=1 Sk = I. Furthermore, without loss of generality the optimal control
problem can be scaled by transforming the independent variable in each interval from
t ∈ [Tk−1,Tk ] to τ (k) ∈ [−1,+1] via the affine transformation
t =Tk −Tk−12
τ (k) +Tk + Tk−12
(5–1)
such that
dt =Tk − Tk−12
dτ (k) ≡ h(k)
2, where h(k) ≡ Tk − Tk−1. (5–2)
The state inequality constrained optimal control problem problem is stated as
follows. Determine the state y(k)(τ) ∈ Rn, and the control u(k)(τ) ∈ Rm in each interval
(k = 1, ... ,K), to minimize the cost functional
J = Φ(y(K)(+1)) +
K∑
k=1
h(k)
2
∫ +1
−1
g(y(k)(τ), u(k)(τ))dτ (5–3)
subject to the dynamic constraints
y(k)(τ) =h(k)
2f(y(k)(τ), u(k)(τ)), (k = 1, ... ,K), (5–4)
the boundary conditions
φ(y(1)(−1)) = 0, (5–5)
and the state inequality path constraint
S(y(k)(τ)) ≤ 0, (k = 1, ... ,K). (5–6)
The continous-time optimal control problem of Eqs. (5–3)–(5–6) will be the topic of the
remainder of this chapter.
118
5.1.1 First-Order Optimality Conditions Using Method of Indirect Adjoining withContinuous Multipliers
The first-order optimality conditions of the state inequality path constrained optimal
problem given by Eqs. (5–3)–(5–6) were derived in Section 2.3.3 using the method of
indirect adjoining with continuous multipliers. These conditions are repeated here as
y(k) = f(y(k), u(k)), φ(y(1)(−1)) = 0, (k = 1, ... ,K), (5–7)
0 = ∇uH(y(k), u(k), p(k),ν(k)), (k = 1, ... ,K), (5–8)
−p(k) = h(k)
2∇yH(y(k), u(k), p(k),ν(k)), (k = 1, ... ,K), (5–9)
p(1)(−1) = ∇y(〈ψ,φ(y(1))〉+ 〈ν(k),S(y(k))〉)∣
∣
τ=−1(5–10)
p(K)(+1) = ∇y(Φ(y(K)) + 〈ν(K),S(y(K))〉)∣
∣
τ=+1(5–11)
ν(K)(+1) ≤ 0, ν(k) ≥ 0, S(y(k)) ∈ N (ν(k)), (k = 1, ... ,K), (5–12)
where p(k)(τ) and ν(k)(τ) are the Lagrange multipliers associated with the dynamic
constraints of Eq. (5–4) and the state inequality path constraints of Eq. (5–6), respectively,
in interval k . Furthermore, ψ is the Lagrange multiplier associated with the boundary
conditions of Eq. (5–5). The Hamiltonian in interval k , H(y(k), u(k), p(k),ν(k)), is defined
as
H(y(k), u(k), p(k),ν(k)) = g(y(k), u(k)) + 〈p(k), f(y(k), u(k))〉 − 〈ν(k), S(y(k))〉, (5–13)
where S(k) ≡ ∇S(y(k))f(y(k), u(k)). Let S(Rq) denote the space of continuous functions
mapping [t0, tf ] to Rq. Assuming ν(k) is Lipschitz continuous and nondecreasing with
ν(K)(+1) ≤ 0, the set-valued map N (ν(k)) is defined as
N (ν(k)) = {z(k) ∈ S(Rq) : z(k) ≤ 0, 〈ν(k), z(k)〉 = 0, 〈ν(K)(+1), z(K)(+1)〉 = 0},
for (k = 1, ... ,K).
119
5.2 Costate Estimation Using Legendre-Gauss Collocation
The optimal control problem of Eqs. (5–3)–(5–6) is now discretized using variable-order
collocation at the Legendre-Gauss points as described in Section 2.5.4. Unlike previous
implementations of the LG collocation method, the state inequality path constraint is
enforced at all LG points and all interior mesh points (T1, ... ,TK−1) but is not enforced
at the endpoints T0 = −1 and TK = +1. LG collocation provides an approximation to
the state, but not the control, at the mesh points. Therefore it is impossible to enforce
inequality path constraints that are a function of the control at any of the mesh points.
However, because this research is concerned with inequality path constraints that are
a function of purely the state, it is possible to enforce the inequality path constraint at
all interior mesh points. Indeed, it will be seen that an accurate approximation of the
costate using variable-order collocation at the LG points can only be achieved when the
state inequality path constraint is enforced at all the interior mesh points.
5.2.1 Variable-Order Collocation at Legendre-Gauss Points
Recall that the LG points (τ1, ... , τN) are defined in the domain τ ∈ (−1,+1)
such that τ0 = −1 and τN+1 = +1 are noncollocated points. When implementing the
variable-order LG method, a single variable is used for the value of the state at the end
of mesh interval k and the start of mesh interval k + 1, that is, Y(k)Nk+1 ≡ Y(k+1)0 , (k =
1, ... ,K − 1) such that continuity in the state is enforced.
The NLP is then given as follows. Minimize the cost function
J ≈ Φ(Y(K)NK+1) +K∑
k=1
Nk∑
j=1
h(k)
2w(k)j g(Y
(k)j ,U
(k)j ), (5–14)
120
subject to the algebraic constraints
D(k)Y(k)0:Nk=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (5–15)
Y(k+1)0 = Y
(k)0 +
h(k)
2
Nk∑
j=1
w(k)j f(Y
(k)j ,U
(k)j ), (k = 1, ... ,K − 1), (5–16)
Y(K)N+1 = Y
(K)0 +
h(K)
2
Nk∑
j=1
w(K)j f(Y
(K)j ,U
(K)j ), (5–17)
φ(Y(1)0 ) = 0, (5–18)
S(Y(k)0:Nk) ≤ 0, (k = 2, ... ,K), (5–19)
S(Y(1)1:N1) ≤ 0. (5–20)
It is seen that the quadrature constraints of Eqs. (5–16) and (5–17) provide an
approximation to the final state in intervals (k = 1, ... ,K − 1) and K , respectively,
through a Gaussian quadrature approximation to the integral of the state dynamics
across that interval.
The first-order optimality (KKT) conditions of the discrete problem given by
Eqs. (5–14)–(5–19) are derived in the same manner of Section 4.2. First, the Lagrangian
is defined as
L = Φ(Y(K)NK+1)− 〈ψ,φ(Y(1)0 )〉+
K∑
k=1
Nk∑
i=1
(
h(k)
2w(k)i g
(k)i − 〈Γ(k)i ,S(k)i 〉
)
−K∑
k=1
Nk∑
i=1
(
〈Λ(k)i ,D(k)i,0:NkY(k)0:Nk− h
(k)
2f(k)i 〉
)
−K−1∑
k=1
(
〈Λ(k)Nk+1,Yk+10 − Y(k)0 −
h(k)
2w(k)i f(k)i 〉 − 〈Γ(k)Nk+1,S
(k+1)0 〉
)
−NK∑
i=1
(
〈Λ(K)NK+1,YKNK+1
− Y(K)0 − h(K)
2w(K)i f(K)i 〉
)
(5–21)
where Λ(k)1:Nk+1 and Γ(k)1:Nk+1 are the Lagrange multipliers associated with the dynamic
constraints of Eq. (5–15), the quadrature constraints of Eqs. (5–16)–(5–17), and the
inequality path constraint of Eq. (5–19), respectively, in interval k . Furthermore ψ
121
denotes the Lagrange multipliers associated with the boundary conditions of Eq. (5–18).
The KKT conditions of the NLP are then given as
D(k)0:NkY(k)0:Nk=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk ), φ(Y
(1)0 ) = 0, (k = 1, ... ,K), (5–22)
Y(k+1)0 = Y
(k)0 +
Nk∑
i=1
w(k)i f(Y
(k)i ,U
(k)i ), (k = 1, ... ,K − 1), (5–23)
Y(k)NK+1
= Y(K)0 +
NK∑
i=1
w(K)i f(Y
(K)i ,U
(K)i ), (5–24)
0 = ∇UH(Y(k)1:Nk,U(k)1:Nk ,Λ
(k)1:Nk+1
,Γ(k)1:Nk), (k = 1, ... ,K), (5–25)
D(k)⊤1:NkΛ(k)1:Nk=h(k)
2∇YH(Y(k)1:Nk ,U
(k)1:Nk,Λ(k)1:Nk+1
,Γ(k)1:Nk), (k = 1, ... ,K), (5–26)
D(1)⊤0 Λ(1)1:N1= Λ
(1)N1+1
−∇Y〈ψ,φ(Y(1)0 )〉, (5–27)
D(k)⊤0 Λ(k)1:N1= Λ
(k)Nk+1
− Λ(k−1)Nk+1−∇Y〈Γ(k−1)N+1 ,S(Y
(k)0 )〉, (k = 2, ... ,K), (5–28)
Λ(K)NK+1
= ∇YΦ(Y(K)NK+1), (5–29)
S(Y(k)1:Nk ) ≤ 0, Γ(k)1:Nk≤ 0, 〈Γ(k)1:Nk ,S(Y
(k)1:Nk)〉 = 0, (k = 1, ... ,K), (5–30)
S(Y(k)0 ) ≤ 0, Γ(k−1)N+1 ≤ 0, 〈Γ(k−1)N+1 ,S(Y(k)0 )〉 = 0, (k = 2, ... ,K). (5–31)
The discrete Hamiltonian in interval k is given as
H(Y(k)1:Nk,U1:Nk ,Λ
(k)1:Nk+1
) = w (k)⊤g(k)1:Nk+ 〈Λ(k)1:Nk +W
(k)1Λ(k)Nk+1, f(k)1:Nk〉 − 〈 2
h(k)Γ(k)1:Nk,S(k)1:Nk〉,
whereW(k) is a N × N diagonal matrix of LG quadrature weights in interval k , and 1 is a
N × 1 column vector of ones.
5.2.2 Costate Estimate and Transformed Adjoint System
In order to relate the necessary conditions for optimality of the continuous problem
[given by Eqs. (5–7)–(5–12)] to the discrete KKT conditions of the NLP [given by
122
Eqs. (5–22)–(5–31)], consider the following transformed dual variables
p(k)1:Nk= [W(k)]−1Λ
(k)1:Nk+ 1Λ
(k)N+1 +∇Y〈ν(k)1:Nk ,S(Y
(k)1:Nk)〉, (k = 1, ... ,K), (5–32)
p(k)Nk+1
= Λ(k)Nk+1
+∇Y 〈ν(k)Nk+1,S(Y(k)Nk+1)〉, (k = 1, ... ,K), (5–33)
Γ(k)1:N = −W(k)D†(k)ν1:N+1, (k = 1, ... ,K), (5–34)
ν(K)NK+1
= Γ(K)NK+1
= 0, (5–35)
ν(k)Nk+1
= Γ(k)Nk+1
+ ν(k+1)0 , (k = 1, ... ,K), (5–36)
p(k)0 = p
(k)Nk+1
−D(k)⊤0 W(k)[p(k)1:Nk− 1 · p(k)Nk+1], (k = 1, ... ,K), (5–37)
ν(k)0 = ν
(k)Nk+1
+
Nk∑
i=1
Γ(k)i , (k = 1, ... ,K). (5–38)
where D† is a N × (N + 1) matrix defined by
D†ij = −
wj
wiDji , and D
†i,N+1 =
N∑
j=1
Dij (5–39)
for i = 1, ... ,N . Based on the theory developed in [36], D† is a differentiation matrix for
the space of polynomials of degree N . That is, if b is a polynomial of degree at most N
and b ∈ RN+1 is the vector with i-th element bi = b(τi), then
(D†b)i = b(τi). (5–40)
Furthermore, it can be shown that D† has similar properties to the Gauss differentiation
matrix D. Specifically, as was seen in Chapter 3, the following hold true: (a) the
square matrix D†1:N obtained by removing the last column of D† is full-rank, and (b)
−(D†1:N)
−1D†N+1 = 1. Using these properties, Eq. (5–34) can be rewritten as
ν(k)1:Nk= ν
(k)N+1 − [W(k)D†(k)
1:N ]−1Γ
(k)1:Nk
= ν(k)Nk+1
− A†(k)[W(k)]−1Γ(k)1:Nk,
(5–41)
where the matrix A† is a backward integration matrix for the space of polynomials of
degree N − 1. Specifically, let L†i (τ) be a basis of Lagrange interpolating polynomials of
123
degree N − 1,
L†i (τ) =
N∏
j=1
j 6=i
τ − τjτi − τj
. (5–42)
Then if q is a polynomial of degree at most N − 1 with q(τi) = qi , it can be interpolated
exactly by the Lagrange polynomials L†i such that
q(τ) =N∑
i=1
qiL†i (τ). (5–43)
Integrating this expression backwards yields
q(τj) = q(+1) +N∑
i=1
A†ji qi , A
†ji =
∫ τj
+1
L†i (τ)dτ . (5–44)
Furthermore, a constant of integration (in this case a terminal condition) is needed for
the integration of the KKT multipliers Γ. Since the constraint S(Y(k)1:N) ≤ 0 is not enforced
at the final point τ (K)NK+1 = +1 in the final interval K , by definition its associated multiplier
is zero. Thus, ν(K)NK+1 = Γ(K)NK+1
= 0. The terminal condition for the remainder of the
intervals, (k = 1, ... ,K − 1), is defined by Eq. (5–36).
Expressions are now developed that justify Eqs. (5–37) and (5–38) as approximations
of the state constraint multiplier and the costate, respectively, at τ (k) = −1, (k =
1, ... ,K). First, let ν(k)(τ) be the polynomial of degree N that satisfies ν(k)(τi) = ν(k)i for
(i = 1, ... ,N + 1) in interval k , (k = 1, ... ,K). Since the Legendre-Gauss quadrature is
exact for a polynomial of degree N − 1, then
ν(k)0 = ν
(k)N+1 −
∫ +1
−1
ν(k)(τ)dτ = ν(k)N+1 −N∑
i=1
w(k)j ν(k)(τi). (5–45)
Furthermore, D† is a differentiation matrix for the space of polynomials of degree N , as
seen by Eq. (5–40). Therefore, Eq. (5–45) becomes
ν(k)0 = ν
(k)N+1 −
Nk∑
i=1
w(k)j D
†(k)i,1:N+1ν
(k)1:N+1 = ν
(k)N+1 +
Nk∑
i=1
Γi(k),
where Eq. (5–34) was used in the last substitution.
124
Next, it is shown that the costate approximation given by Eq. (5–37) is equivalent to
applying the Fundamental Theorem of Calculus. Let 1 ∈ RN+1 denote the vector whose
elements are all unity. Because the components of the vector D(k)1 are the derivatives
of the constant polynomial q(τ) = 1 at the LG points, D(k)1 = 0, which implies that
D(k)0 = −
Nk∑
j=1
D(k)j . (5–46)
Taking the transpose of Eq. (5–46) and substituting the rows of D†(k) for the rows of D(k),
the following expression is obtained:
D(k)0 =
N∑
i=1
N∑
j=1
wi
wjD
†(k)ij (5–47)
Finally, post-multiplying the result byW(k)[p(k)1:N−1 ·p(k)N+1], and subtracting p(k)N+1 from both
sides yields
−p(k)N+1 +D(k)⊤0 W(k)[p(k)1:N − 1 · p(k)N+1]
= −p(k)N+1 +Nk∑
j=1
w(k)j D
†(k)j,1:N+1p
(k)1:N+1,
(5–48)
for each interval (k = 1, ... ,K). Now let p(k)(τ) be the polynomial of degree N that
satisfies p(k)(τi) = p(k)i for (i = 1, ... ,N + 1), (k = 1, ... ,K), then using the same logic as
was done for Eq. (5–45)
−p(k)0 = −p(k)N+1 +∫ +1
−1
p(k)(τ)dτ = −p(k)N+1 +Nk∑
i=1
w(k)i p
(k)(τi).
Comparing this expression with Eq. (5–48), it is seen that p(k)0 , (k = 1, ... ,K) given by
Eq. (5–37) is consistent with the Fundamental Theorem of Calculus.
125
Using the transformations described in Eqs. (5–32)–(5–38) along with Eq. (5–39),
the KKT conditions of the NLP given by Eqs. (5–22)–(5–31) can be written as
D(k)Y(k)0:N = f(Y
(k)1:N,U
(k)1:N), φ(Y
(1)0 ) = 0, (k = 1, ... ,K), (5–49)
Y(k+1)0 = Y
(k)0 + w
(k)⊤f(Y(k)1:N ,U
(k)1:N), (k = 1, ... ,K − 1), (5–50)
Y(K)N+1 = Y
(K)0 +w(k)⊤f(Y
(k)1:N,U
(k)1:N), (5–51)
0 = ∇UH(Y(k)1:N,U(k)1:N, p(k)1:N,ν(k)1:N), (k = 1, ... ,K), (5–52)
D†(k)p(k)1:N+1 = −∇YH(Y(k)1:N,U(k)1:N, p(k)1:N ,ν(k)1:N), (k = 1, ... ,K), (5–53)
p(k)0 = ∇Y
(
〈ψ,φ(Y(1)0 )〉+ 〈ν(1)0 ,S(Y(1)0 )〉)
, (5–54)
p(k+1)0 = p
(k)N+1, (k = 1, ... ,K − 1), (5–55)
p(K)N+1 = ∇Y
(
Φ(Y(K)N+1) + 〈ν(K)N+1,S(YN+1)〉
)
, (5–56)
0 = 〈D†(k)ν(k)1:N+1,S(Y
(k)1:N)〉, (k = 1, ... ,K), (5–57)
S(Y(k)1:N) ≤ 0, D†(k)ν
(k)1:N+1 ≥ 0, ν
(K)N+1 = 0, (k = 1, ... ,K), (5–58)
where H(k) is a discrete form of the Hamiltonian in interval k given by Eq. (5–13).Furthermore,
the partial differentials of the Hamiltonian in interval k given in Eqs. (5–52) and (5–53)
are given as
∇UH(k) =∇Ug(Y(k)i ,U(k)i ) + 〈p(k)i ,∇Uf⊤(Y(k)i ,U(k)i )〉 − 〈ν(k)i ,∇YS⊤(Y(k)i )∇Uf⊤(Y(k)i ,U(k)i )〉,
and
∇YH(k) =∇Yg(Y(k)i ,U(k)i ) + 〈p(k)i ,∇Yf⊤(Y(k)i ,U(k)i )〉 − 〈ν(k)i ,∇YS⊤(Y(k)i )∇Yf⊤(Y(k)i ,U(k)i )〉
− 2
h(k)D
†(k)i,1:N+1 · ∇Y〈ν(k)1:N+1,S(Y(k)1:N+1)〉+ 〈
2
h(k)D
†(k)i,1:N+1ν
(k)1:N+1,∇YS(Y(k)i )〉,
for i = 1, ... ,N . Note that the product rule was used to differentiate the state inequality
constraint, that is, the following identity was used:
〈ν, ddt∇yS⊤(y)〉 =
d
dt〈ν,∇yS⊤(y)〉 − 〈
d
dtν,∇yS⊤(y)〉.
126
Careful comparison of the necessary conditions for optimality of the discrete and
continuous problems [given by Eqs. (5–49)–(5–58) and Eqs. (5–7)–(5–12), respectively]
reveals their equivalence. Note the costate is continuous across interval boundaries, as
given by Eq. (5–55). Furthermore, it is reinforced that the state is being differentiated by
a matrix D which is based on the derivatives of polynomials of degree N with coefficients
at the N LG points plus the initial uncollocated point τ0 = −1, whereas the costate and
the state constraint multipliers are being differentiated by a matrix D† which is based
on the derivatives of polynomials of degree N with coefficients at the N LG points plus
the terminal uncollocated point τN+1 = +1. Finally, note that the integration matrix
A associated with the state dynamics integrates the state forward in the domain, and
requires an initial condition, whereas the integration matrix A† associated with the
costate and the state constraint multipliers is a backward integration matrix which
requires a terminal condition.
5.3 Costate Estimation Using Flipped Legendre-Gauss-Radau Collocation
The optimal control problem of Eqs. (5–3)–(5–6) is now discretized using variable-order
collocation at the flipped Legendre-Gauss-Radau points as described in Section 2.5.6.
It is noted that the flipped LGR points are particularly conducive to variable-order
collocation; since only one of the domain endpoints are collocated, there is no “double
collocation” at the boundaries. Also, the only noncollocated point is the first point of the
first interval, t0 = τ (1)0 = −1. Thus, the flipped LGR points are an improvement over
the LG points, which provided no information on the optimal control at any of the mesh
points.
5.3.1 Variable-Order Collocation at Flipped Legendre-Gauss-Radau Points
Recall that the flipped LGR points are defined on the domain (−1,+1] such that
τN = +1 is a LGR collocation point but τ0 = −1 is a noncollocated point. When
implementing the flipped variable-order LGR method, a single variable is used for the
value of the state at the end of mesh interval k and the start of mesh interval k + 1, that
127
is, Y(k)Nk ≡ Y(k+1)0 , 1 ≤ k ≤ K − 1 such that continuity in the state is enforced. Hence,
redundant variables defining the state at the interior mesh points are eliminated.
The optimal control problem of Eqs. (5–3)–(5–6) is now discretized using variable-order
collocation at the flipped Legendre-Gauss-Radau points as described in Section 2.5.6.
The NLP is then given as follows. Minimize the cost function
J ≈ Φ(Y(K)NK ) +K∑
k=1
Nk∑
j=1
h(k)
2w(k)j g(Y
(k)j ,U
(k)j ), (5–59)
subject to the algebraic constraints
D(k)Y(k)0:Nk=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk), (k = 1, ... ,K), (5–60)
φ(Y(1)0 ) = 0, (5–61)
S(Y(k)1:Nk) ≤ 0, (k = 1, ... ,K). (5–62)
The first-order optimality (KKT) conditions of the discrete problem given by
Eqs. (5–59)–(5–62) are derived in the same manner of Chapter 4. First, the Lagrangian
is defined as
L = Φ(Y(K)NK )− 〈ψ,φ(Y(1)0 )〉+
K∑
k=1
Nk∑
i=1
(
h(k)
2w(k)i g
(k)i − 〈Γ(k)i ,S(k)i 〉
)
−N1∑
i=1
(
〈Λ(1)i ,D(1)i,0 Y(1)0 + D(1)i,1:N1Y(1)1:N1− h
(k)
2f(1)i 〉
)
−K−1∑
k=1
Nk∑
i=1
(
〈Λ(k)i ,D(k)i,0 Y(k−1)N +D(k)i,1:NkY(k)1:Nk− h
(k)
2f(k)i 〉
)
,
(5–63)
where Λ(k)i and Γ(k)i are the Lagrange multipliers associated with the dynamic constraints
of Eq. (5–60) and the inequality path constraint of Eq. (5–62),respectively, in interval
k at the LGR point τi . Furthermore ψ is the Lagrange multipliers associated with the
boundary conditions of Eq. (5–61).
128
The KKT conditions of the NLP are then given as
D(k)0:NkY(k)0:Nk=h(k)
2f(Y
(k)1:Nk,U(k)1:Nk ), φ(Y
(1)0 ) = 0, (k = 1, ... ,K), (5–64)
0 = ∇UH(Y(k)1:Nk,U(k)1:Nk ,Λ
(k)1:Nk,Γ(k)1:Nk), (k = 1, ... ,K), (5–65)
D(k)⊤1:NkΛ(k)1:Nk=h(k)
2∇YH(Y(k)1:Nk ,U
(k)1:Nk,Λ(k)1:Nk,Γ(k)1:Nk)
− eNkD(k+1)0 Λ
(k+1)1:Nk, (k = 1, ... ,K − 1), (5–66)
D(K)⊤1:NKΛ(K)1:NK=h(k)
2∇YH(Y(K)1:NK ,U
(K)1:NK,Λ(K)1:NK,Γ(K)1:NK) + eNK∇YΦ(Y(K)NK ), (5–67)
D(1)⊤0 Λ(1)1:N1= −∇Y〈ψ,φ(Y(1)0 )〉, (5–68)
S(Y(k)1:Nk ) ≤ 0, Γ(k)1:Nk≤ 0, 〈Γ(k)1:Nk ,S(Y
(k)1:Nk)〉 = 0, (k = 1, ... ,K), (5–69)
where eN denotes the N-th column of the identity matrix and the discrete Hamiltonian in
interval k is
H(Y(k)1:Nk,U1:Nk ,Λ
(k)1:Nk) = w (k)⊤g
(k)1:Nk+ 〈Λ(k)1:Nk , f
(k)1:Nk〉 − 〈h
(k)
2Γ(k)1:Nk,S(k)1:Nk〉. (5–70)
5.3.2 Costate Estimate and Transformed Adjoint System
In order to relate the necessary conditions for optimality of the continuous problem
[given by Eqs. (5–7)–(5–12)] to the discrete KKT conditions of the NLP [given by
Eqs. (5–64)–(5–69)], consider the following transformed dual variables
p(k)1:N = [W
(k)]−1Λ(k) +∇Y〈ν(k)1:N,S(Y(k)1:N)〉, (k = 1, ... ,K), (5–71)
Γ(K)1:N = −W(K)D(K)ν(K)1:N , (5–72)
p(k)0 = −D(k)⊤0 W(k)p
(k)1:N, (k = 1, ... ,K), (5–73)
ν(k)0 = −D(k)⊤0 W(k)ν
(k)1:N , (k = 1, ... ,K), (5–74)
whereW(k) is the diagonal matrix with the quadrature weights w on the diagonal in
interval k and D is defined by
D = −W−1DT1:NW. (5–75)
129
Next, let D† be an Nk × Nk matrix defined as follows:
D†ij =
−DNN + 1wN, i = j = N
−wjwiDji , i , j = 2, ... ,N.
(5–76)
Based on the theory developed in [36], D† is a differentiation matrix for the space of
polynomials of degree N − 1. That is, if b is a polynomial of degree at most N − 1 and
b ∈ RN is the vector with i-th element bi = b(τi), then
(D†b)i = b(τi).
The matrix D is identical to the differentiation matrix D† introduced in Eq. (5–76) except
for the (N,N) element:
DNN = −DNN = D†NN −
1
wN. (5–77)
Because D equals D† except for the single element in row N given in Eq. (5–77), it
follows that
(Db)i =
b(τi), 1 ≤ i ≤ N − 1,
b(τN)− b(τN)/wN, i = N.(5–78)
Hence, D behaves like a differentiation matrix except for the last row that both
differentiates and evaluates. Note that D† is singular while D is invertible since D1:N
is invertible. In particular, by Eq. (5–75), D can be inverted as
D−1 =W−1D−T1:NW.
Now let A denote D−1, by Eq. (5–72), the following representation for ν(K) in terms of
µ(K) in the last mesh interval K is obtained:
ν(K)1:N = −[W(K)D(K)]−1Γ(K)1:N = −A(K)[W(K)]−1Γ(K)1:N . (5–79)
130
It can be shown that A is an integration matrix which also extrapolates the final value of
the vector it operates on. Specifically, the elements of A are given as
Aij =
∫ τi
+1
Lj(τ)dτ + wN Lj(τN), (i , j = 1, ... ,N − 1),
AiN = −wN , (i = 1, ... ,N),
ANj = wN Lj(τN), (j = 1, ... ,N − 1),
(5–80)
where the N − 1 Lagrange interpolating polynomials Lj(τ) are defined as
Lj (τ) =
N−1∏
i=1j 6=i
τ − τiτj − τi
, j = 1, ... ,N − 1. (5–81)
It is known that the KKT multiplier Γ is related to the dual variables of the continuous-time
problem as follows
Γ(k)i =
µ(k)i
w(k)i
, (i = 1, ... ,N − 1),
Γ(k)N =
µ(k)N + η
(k)
w(k)N
,
(5–82)
where it is recalled from Chapter 2 that µ(k) = ν(k), and η is the multiplier associated
with the “jump”, or discontinuity of the state constraint multiplier. Next, let the continuous-time
state constraint multiplier µ be a polynomial of degree at most N−2 such that µi = µ(τi)
for (i = 1, ... ,N − 1). This polynomial can be described exactly using the Lagrange
interpolating bases of Eq. (5–81) such that
µ(τ) =
N−1∑
j=1
µj Lj . (5–83)
Substituting the expressions from Eq. (5–82) into Eq. (5–80) it is seen that
A(k)i Γ
(k)1:N =
∫ τi
+1
µ(k) dτ +N−1∑
j=1
µj L(k)j (τN)− w (k)N Γ(k)N , (i = 1, ... ,N − 1),
N−1∑
j=1
µj L(k)j (τN)− w (k)N Γ(k)N , (i = N).
(5–84)
131
Thus, it is seen that the matrix A integrates and extrapolates the multiplier µ for (i =
1, ... ,N − 1). Furthermore, the expression for (i = N) amounts to an approximation of
the jump multiplier η(k). Consequently, the right hand side of Eq. (5–79) is equivalent to
integrating the state inequality constraint multipliers backward from τ (K) = +1 in the last
mesh interval K . Furthermore, because Γ(k) is being integrated backward across each
mesh interval, the value of the integration in interval k + 1 must be added as an initial
condition of the integration in interval k , yielding:
ν(k)1:N = 1ν(k+1)0 − A(k)[W(k)]−1Γ(k)1:N, (k = 1, ... ,K − 1), (5–85)
where 1 is a N × 1 column vector composed of ones.
To justify that Eq. (5–74) is an approximation of the state constraint multiplier
at τ (k) = −1 for (k = 1, ... ,K), it is now shown that this definition is equivalent to
applying the Fundamental Theorem of Calculus. Let 1 ∈ RN+1 denote the vector whose
elements are all unity. Because the components of the vector D1 are the derivatives of
the constant polynomial q(τ) = 1 at the LGR points, D1 = 0, which implies that
D0 = −N∑
j=1
Dj . (5–86)
Taking the transpose of Eq. (5–86), substituting the rows of D for the rows of DT , and
multiplying the result on the right byW(K)ν(K)1:N yields
D(K)0
⊤W(K)ν(K)1:N = −ν(K)0 =
N∑
j=1
w(K)j D
(K)j,1:Nν
(K)1:N . (5–87)
Next, let ν(τ) be the polynomial of degree N − 1 that satisfies ν(τi) = νi for 1 ≤ i ≤ N .
Using Eq. (5–78) together with the fact that the Legendre-Gauss-Radau quadrature is
exact for a polynomial of degree N − 2, then
−ν0 = −νN +N∑
i=1
wj ν(τi) = −νN +∫ +1
−1
ν(τ)dτ .
132
Because ν0 is the approximation to the state constraint multiplier at τ = −1, while νN
is the approximation at τ = +1, it is seen that ν0 in Eq. (5–74) is consistent with the
Fundamental Theorem of Calculus. Using the same logic as above, let 1 ∈ RN denote
the vector whose elements are all unity. Because the components of the vector D†1 are
the derivatives of the constant polynomial q(τ) = 1 at the LGR points, D†1 = 0, which
implies:N∑
j=1
D†j = 0. (5–88)
Because D is identical to the differentiation matrix D† introduced in Eq. (5–76) except for
the (N,N) element, substituting the values of D into Eq. (5–88) results in
[D1]i =
0 for (i = 1, ... ,N − 1),
− 1wN
for (i = N).(5–89)
Thus, substituting the values of ν(k)1:N obtained from Eq. (5–85) into the right-hand side of
Eq. (5–87) results in the relationship
D(k)0
⊤W(k)ν(k)1:N = −ν(k)0 = −ν(k+1)0 +
N∑
j=1
w(k)j [W
−1Γ(k)1:N]j , (k = 1, ... ,K − 1). (5–90)
Furthermore, substituting the expressions given by Eq. (5–82) into Eq. (5–90) results in
the expression
− ν(k)0 = −ν(k+1)0 − η(k) +N∑
j=1
w(k)j µj , (k = 1, ... ,K − 1). (5–91)
which says that the value of the state constraint mutliplier at the first point of an interval
is given by a Radau quadrature which approximates the backward integral of the state
constraint dynamics across that interval summed with a terminal condition given by
ν(k)N = ν
(k+1)0 + η(k). Finally, in a similar fashion, it can be shown that Eq. (5–73) implies
133
the following relation:
D0Wp1:N = −p0 = −pN +N∑
j=1
wjD†j,1:Np1:N. (5–92)
Because D† is a differentiation matrix, it is seen that Eq. (5–73) is also consistent with
the Fundamental Theorem of Calculus.
Now, using the transformations described in Eqs. (5–71)–(5–74) along with
Eqs. (5–75) and (5–77), the KKT conditions of the NLP given by Eqs. (5–64)–(5–69)
can be written as
D(k)Y(k)0:Nk = f(Y(k)1:Nk,U(k)1:Nk ), φ(Y
(1)0 ) = 0, (k = 1, ... ,K), (5–93)
0 = ∇UH(Y(k)1:N,U(k)1:N, p(k)1:N,ν(k)1:N), (k = 1, ... ,K), (5–94)
D†(k)1:Nkp(k)1:Nk= −∇YH(Y(k)1:N,U(k)1:N , p(k)1:N,ν(k)1:N) (5–95)
+eNkwNk
[
p(k)Nk− p(k+1)0
]
, (k = 1, ... ,K − 1), (5–96)
D†(K)1:NKp(K)1:NK= −∇YH(Y(K)1:N ,U(K)1:N , p(K)1:N ,ν(K)1:N)
+eNKwNK
[
p(K)N −∇Y
(
Φ(Y(K)N )− 〈ν(K)N ,S(Y(K)N )〉
)]
(5–97)
p(1)0 = ∇Y
(
〈ψ,φ(Y(1)0 )〉+ 〈ν0,S(Y0)〉)
(5–98)
S(Y(k)1:N) ≤ 0, D(k)ν(k)1:N ≥ 0, 〈D(k)ν(k)1:N,S(Y(k)1:N)〉 = 0, (k = 1, ... ,K), (5–99)
where H is the continuous Hamiltonian defined in Eq. (5–13). Next, substituting
Eq. (5–97) into Eq. (5–92) yields
−p(K)0 = −∇Y(
Φ(Y(K)N )− 〈ν(K)N ,S(Y(K)N )〉
)
−NK∑
i=1
w(K)i ∇YH(Y(K)i ,U(K)i , p(K)i ,ν(K)i ). (5–100)
If p(K)i were the continuous costate evaluated at τi in interval K , then by the continuous
adjoint equation, the sum in Eq. (5–100) approximates the integral of p(K) between −1
134
and +1. Eq. (5–100) amounts to an approximation to the relation
p(K)N = ∇Y
(
Φ(Y(K)N ) + 〈ν(K)N ,S(Y(K)N )〉
)
, (5–101)
where Eq. (5–101) is a discrete form of the continuous optimality condition given in
Eq. (5–11). Consequently, it is expected that the eN term in Eq. (5–97) should be
small, while the remaining terms in Eq. (5–97) amount to a collocation scheme for the
continuous adjoint equation. Next, it is shown that the last term in the brackets on the
right-hand side of Eq. (5–96) will be small. Substituting Eq. (5–96) into Eq. (5–92) yields
p(k+1)0 = p
(k)0 −
Nk∑
i=1
w(k)i ∇YH(Y(k)i ,U(k)i , p(k)i ,ν(k)i ), (k = 1, ... ,K − 1). (5–102)
If p(k)i were the continuous costate evaluated at τi in interval k , then by the continuous
adjoint equation, the sum in Eq. (5–102) approximates the integral of p(K) between −1
and +1. Eq. (5–102) amounts to an approximation to the relation
p(k)Nk= p
(k+1)0 . (5–103)
This condition shows that the costate will be continuous across mesh interval boundaries.
Consequently, it is expected that the eN term in Eq. (5–96) should be small, while the
remaining terms in Eq. (5–96) amount to a collocation scheme for the continuous adjoint
equation.
The connection between the transformed optimality conditions and the original
continuous optimality conditions is quite subtle. For example, the nonnegativity
conditions for the derivative of the state multiplier ν(k) and the complementary slackness
conditions in Eq. (5–12) are embedded in a very unusual way in the discrete optimality
conditions. As pointed out in Eq. (5–78), if the discrete multiplier νk)1:N associated with
the state constraint is interpolated by a polynomial ν(τ) of degree N − 1, then the
nonnegativity conditions in Eq. (5–98) only ensure nonnegativity of the polynomial
135
derivative at τ1 through τN−1. At τN , the discrete positivity condition amounts to
ν(K)(τN)−ν(K)(τN)
w(K)N
≥ 0, (5–104)
ν(K)(τN) +−ν(k)(τN) + ν(k+1)(τ0)
w(k)N
≥ 0, (k = 1, ... ,K). (5–105)
To illustrate how these conditions work, suppose that the state constraint is inactive over
the entire final interval [−1,+1] for the discrete problem. That is, S(Y(K)i ) < 0 for all i . In
this case, complementary slackness implies that
ν(K)(τi) = 0 for 1 ≤ i ≤ N − 1. (5–106)
Because the derivative of a polynomial of degree N − 1 is N − 2, the N − 1 conditions
Eq. (5–106) imply that the derivative is identically zero. Hence, ν(K)(τN) = 0 in
Eq. (5–104), and it is concluded that ν(K)(τN) = ν(+1) ≤ 0. Finally, from the
complementary slackness condition,
0 = S(Y(K)N )
T(ν(K)(τN)− ν(K)(τN)/wN)
= −S(Y(K)N )Tν(K)(τN)/wN ,
which implies that ν(K)(τN) = ν(K)N = 0 when S(Y(K)N ) < 0. Hence, the continuous
optimality conditions ν(K)(+1) ≤ 0 and 〈ν(K)(+1),S(y(K)(+1))〉 = 0 are satisfied
in the discrete problem. Furthermore, if complementary slackness holds in interval
K , then ν(K) is a non-decreasing function in interval K , and ν(K)0 ≥ ν(K−1)N , thus the
second term in Eq. (5–105) will be greater than zero. A similar argument can be made
over each interval, such that the condition ν(k) ≥ 0 is satisfied for (k = 1, ... ,K).
Thus, it has been shown that the conditions of the transformed adjoint system given
by Eqs. (5–93)–(5–99) are a discrete form of the first-order optimality conditions of the
continuous-time optimal control problem given by Eqs. (5–7)–(5–12).
The transformed adjoint system for variable-order collocation at the LGR points is
complex. For instance, the differentiation matrix associated with the state dynamics, D,
136
is a N × (N + 1) full-rank differentiation matrix associated with the space of polynomials
of degree at most N . Conversely, the differentiation matrix associated with the costate
dynamics, D† is a rank-defficient N × N differentiation matrix associated with the space
of polynomials of degree at most N − 1. Finally, the matrix associated with the state
constraint multiplier, D, is a full-rank N × N matrix that differentiates and evaluates the
terminal condition simultaneously.
5.4 Discussion
In this chapter costate estimates were derived for estimating the costate of state
inequality path constrained optimal control problems using orthogonal collocation
at the Legendre-Gauss and the flipped Legengre-Gauss-Radau Points. These
conditions result in a continuous approximation to the costate even in the presence
of state inequality path constraints. Furthermore, the costate estimate derived here
reduces to the costate estimate given by Ref. [1], presented in Chapter 4, when no
state inequality path constraints are present in the optimal control problem. Finally,
It was shown that the costate estimate using the method of indirect adjoining with
continuous multipliers resulted in a transformed adjoint system that is a discrete form of
the first-order optimality conditions of the continuous-time problem. Fig. (5-1) illustrates
the equivalence between the transformed adjoint system derived from the NLP and the
first-order optimality conditions of the continuous-time problem derived from the calculus
of variations.
137
Figure 5-1. Relationship between the direct and indirect methods for solving an optimalcontrol problem. In the indirect method, the problem is first optimized through thecalculus of variations, leading to a set of conditions which can then be discretizedand solved. In the direct method, the problem is first discretized and transcribed toan NLP, then it is optimized by solving the KKT system. The two systems are shown tobe equivalent even in the presence of a discontinuous costate.
138
CHAPTER 6EXAMPLES
In this chapter, four examples are studied using the methods developed in Chapters
3 and 5. The first two examples demonstrate the effectiveness of the costate estimation
methods derived in Chapter 3 using the integral form of LG and LGR collocation.
The first example is a single state nonlinear Mayer optimal control problem while
the second example is a single state nonlinear Lagrange optimal control problem.
Next, two state inequality path constrained optimal control problems are solved using
variable-order LG and LGR collocation as described by Chapter 5. The first state
inequality constrained example contains a first-order state inequality path constraint,
while the second state inequality constrained example contains a second-order state
inequality path constraint. The LG and LGR costate estimates derived in Ref. [1]
are shown to produce inaccurate estimates of the dual variables for both examples,
while the LG and LGR costate estimates using the method of indirect adjoining with
continuous multipliers (as described in Chapter 5) are shown to produce accurate
approximations of the dual variables.
Three main observations are made from the examples solved in this chapter. First,
it is shown that variable-order collocation at the LG and LGR points produces accurate
approximations to state inequality path constrained optimal control problems. Because
collocation at the LG points does not provide an approximation to the optimal control at
any of the mesh points whereas collocation at the LGR points provides an approximation
of the optimal control at all interior mesh points, collocation at the LGR method is found
to be the preferred method of solution. Second, it is shown that for state inequality path
constraints of at most order two, it is not necessary to reformulate the optimal control
problem by reducing the index of the DAE in order to obtain an accurate approximation.
Index-reduction requires an analytic reformulation of the optimal control problem which
may be cumbersome, if not impossible, to implement when using mesh refinement.
139
Third, because the method of indirect adjoining with continuous multipliers produces
a costate estimate that is continuous even in the presence of state inequality path
constraint, highly accurate estimates of the costate can be obtained even when using
low-order polynomial approximations in the state (that is, a small number of collocation
points).
6.1 Example 1: Mayer Optimal Control Problem
The first example considered is a nonlinear one-dimensional Mayer optimal control
problem [64]. It is stated as follows:
Minimize J = −y(2) subject to
y = 52(−y + yu − u2),
y(0) = 1.
The optimal solution is given as
y ∗(t) =4
a(t), u∗(t) =
y ∗(t)
2,
p∗y(t) = −(15 exp(5t/2)− 1)(−3 exp(5t/2)− 1)(2b exp(5t/2))
, λ∗y(t) = −exp(2 ln(a(t))− 5t/2)
b,
where a(t) = 1 + 3 exp(5t/2), and b = exp(−5) + 6 + 9 exp(5). This example was solved
using the integral LG and LGR collocation methods using the NLP solver SNOPT [22],
where SNOPT was implemented using optimality and feasibility tolerances of 1 × 10−8
and 2 × 10−8, respectively. The initial guess used for the state and control was a linear
interpolation from the initial state value to zero. For collocation at either set of points,
the integral costate, py(τ), was estimated from the KKT multipliers of the NLP, and
the differential costate, λy(τ), was subsequently computed from the integral costate
approximation.
6.1.1 Solution Using Collocation at Legendre-Gauss Points
Example 1 was solved using integral collocation at LG points as described in
Chapter 3. Figure 6-1 shows the state and control approximation obtained using N = 20
140
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6
0.6
0.7
0.8
0.8
0.9
1.2 1.4 1.6 1.8
1
1 2t
Sta
te
y∗(t)
y(t)
(A) State.
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6 0.8 1.2 1.4 1.6 1.8
0.05
0.15
0.25
0.35
0.45
1 2t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-1. Primal solution for Example 1 obtained using integral collocation at LGpoints.
LG collocation points. It is seen that integral collocation at the LG points provides a
highly accurate approximation to the optimal solution.
Next, the integral costate, py (τ), was computed at the LG points using Eq. (3–62),
and the differential costate, λy(τ), was estimated at the LG points plus the noncollocated
endpoints τ0 and τN+1 using the results of Section 3.2.5. Figure 6-2 shows both the
141
Cos
tate
-0.5
0
0
0.5
0.5
1.5
1.5
2.5
-1
1
1
2
2t
p∗y
λ∗
y
py
λy
Figure 6-2. Integral and differential costate solutions for Example 1 obtained using LGcollocation.
integral and the differential costates obtained using N = 20 LG collocation points.
It is seen that the costate estimate is indistinguishable from the optimal costate.
Furthermore, Fig. 6-3 shows the base ten logarithm of the L∞-norm error for the
integral and differential costates when approximated using (N = 2, 4, 6, ... , 20) LG
collocation points. It is interesting to note that the differential costate estimate converges
exponentially as a function of N and reaches an accuracy of O(10−12) for N = 20,
whereas the integral costate estimate achieves an accuracy of approximately O(10−11)
for N = 20.
142
0
0
-8
-6
-4
-2
4 8 12-12 16 20
-10
N
log10
Infin
ity-N
orm
Err
or
∥
∥λy − λ∗y
∥
∥
∞
∥
∥py − p∗y∥
∥
∞
Figure 6-3. Integral and differential costate errors for Example 1 obtained using LGcollocation.
143
6.1.2 Solution Using Collocation at Legendre-Gauss-Radau Points
Next, Example 1 was solved using integral LGR collocation as described in Chapter
3. Figure 6-4 shows the state and control approximation obtained using N = 20
LGR collocation points. It is seen that, similar to collocation at the LG points, integral
collocation at the LGR points provides highly accurate approximation to the optimal
solution. However, unlike collocation at the LG points, collocation at the LGR points
provides an approximation of the control at the terminal boundary point, making
collocation at the LGR points more desirable than collocation at the LG points.
Next, the integral costate, py (τ), was computed at the LGR points using Eq. (3–116),
and the differential costate, λy(τ), was estimated at the LGR points plus the noncollocated
endpoint τN+1 using the results of Section 3.3.5. Note that the value py(τ1) (where
τ1 = −1 for the integral LGR collocation method) was found by extrapolating the
Lagrange interpolating polynomial as described by Eqs. (3–118) and (3–119). Figure
6-5 shows both the integral and the differential costates obtained using N = 20
LGR collocation points. It is seen that the costate estimate is indistinguishable from
the optimal solution. Furthermore, Fig. 6-6 shows the base ten logarithm of the
L∞-norm error for the integral and differential costates when approximated using
(N = 2, 4, 6, ... , 20) LGR collocation points. It is seen that the differential and integral
costate estimates converges exponentially as a function of N until the error reaches
approximately O(10−10) for N = 20.
144
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6
0.6
0.7
0.8
0.8
0.9
1.2 1.4 1.6 1.8
1
1 2t
Sta
te
y∗(t)
y(t)
(A) State.
00
0.1
0.2
0.2
0.3
0.4
0.4
0.5
0.6 0.8 1.2 1.4 1.6 1.8
0.05
0.15
0.25
0.35
0.45
1 2t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-4. Primal solution for Example 1 obtained using integral collocation at LGRpoints.
145
Cos
tate
-0.5
0
0
0.5
0.5
1.5
1.5
2.5
-1
-1.5
1
1
2
2t
p∗y
λ∗
y
py
λy
Figure 6-5. Integral and differential costate solutions for Example 1 obtained using LGRcollocation.
0
0
-8
-6
-4
-2
4 8 12 16 20
-10
-12
N
log10
Infin
ity-N
orm
Err
or
∥
∥λy − λ∗y
∥
∥
∞
∥
∥py − p∗y∥
∥
∞
Figure 6-6. Integral and differential costate errors for Example 1 obtained using LGRcollocation.
146
6.2 Example 2: Lagrange Optimal Control Problem
This second example considered is a nonlinear one-dimensional Lagrange optimal
control problem given as follows.
Minimize J = 12
∫ tf
0
(log2 y + u2)dt subject to
y = y log y + yu,
y(0) = 5,
y(tf ) = 3.
The optimal solution to this example is given as
y ∗(t) = exp(x∗(t)),
λ∗y(t) = λ∗
x(t)/y∗(t),
p∗y (t) = − λ∗x(t)y
∗(t)− λ∗x(t)y
∗(t)
(y ∗(t))2,
(6–1)
where
x∗(t) = c1 exp(−t√2) + c2 exp(t
√2),
λ∗x(t) = c1(1 +
√2) exp(−t
√2) + c2(1−
√2) exp(t
√2),
(6–2)
and
c1
c2
=
1 1
exp(−tf√2) exp(tf
√2)
log y0
log yf
. (6–3)
The example was solved using the integral LG and LGR collocation methods using the
NLP solver SNOPT [22], where SNOPT was implemented using optimality and feasibility
tolerances of 1 × 10−8 and 2 × 10−8, respectively, with the exact state and control
evaluated at the discretization points as the initial guess. For collocation at either set of
points, the integral costate, py (t), was estimated from the KKT multipliers of the NLP,
and the differential costate, λy(t), was subsequently computed from the integral costate
approximation.
147
3.5
3.5
4.5
4.5
5.5
0 0.5
1.5
1.5
2.5
2.51 1
2
2
3
3
4
4
5
5t
Sta
te
y∗(t)
y(t)
(A) State.
-3.5
-2.5
-1.5
3.5 4.5
-0.5
0
0
0.5
0.5 1.5 2.5-4
-3
-2
-1
1 2 3 4 5t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-7. State and control for Example 2 obtained using integral LG collocation.
6.2.1 Solution Using Collocation at Legendre-Gauss Points
Example 2 was solved using integral collocation at LG points, as described in
Chapter 3. Figure 6-7 shows the primal solution (that is, the state and control) obtained
using N = 32 LG collocation points. It is seen that integral collocation at the LG points
provides highly accurate approximation to the optimal solution.
148
-0.8
-0.6
-0.4
-0.2
0
0
0.2
0.4
0.6
0.8
1
1 2 3 4 5t
Cos
tate
p∗y
λ∗
y
py
λy
Figure 6-8. Integral and differential costate for Example 2 obtained using LG collocation.
Next, the integral costate, py (τ), was estimated at the LG points using Eq. (3–62),
and the differential costate, λy(τ), was estimated at the LG points plus the noncollocated
endpoints τ0 and τN+1 using the results of Section 3.2.5. Figure 6-8 shows both the
integral and the differential costates obtained using N = 32 LG collocation points. It is
seen that both the differential and integral costate estimates are indistinguishable from
the optimal costates. Figure 6-9 shows the base ten logarithm of the L∞-norm error for
the integral and differential costates for (N = 4, 8, 12, ... , 32) LG collocation points. Both
the differential and integral costate estimates converges exponentially as a function of N
until the error reaches approximately O(10−12) for N = 32.
149
322824-12
0
0
-8
-6
-4
-2
4 8 12 16 20
-10
N
log10
Infin
ity-N
orm
Err
or
∥
∥λy − λ∗y
∥
∥
∞
∥
∥py − p∗y∥
∥
∞
Figure 6-9. Integral and differential costate errors for Example 2 obtained using integralLG collocation.
150
6.2.2 Solution Using Collocation at Legendre-Gauss-Radau Points
Next, Example 2 was solved using integral collocation at LGR points, as described
in Chapter 3. Figure 6-10 shows the state and control approximations obtained using
N = 32 LGR collocation points. It is seen that, similar to collocation at the LG points,
integral collocation at the LGR points provides highly accurate approximation to the
optimal solution. However, unlike collocation at the LG points, collocation at the LGR
points provides an approximation of the control at the terminal boundary point, making
collocation at the LGR points more desirable than collocation at the LG points.
The integral costate, py (τ), was computed at the LGR points using Eq. (3–116), and
the differential costate, λy(τ), was estimated at the LGR points plus the noncollocated
endpoint τN+1 using the results of Section 3.3.5. Note that the value py(τ1) (where
τ1 = −1 for the integral LGR collocation method) was found by extrapolating the
Lagrange interpolating polynomial as described by Eqs. (3–118) and (3–119). Figure
6-11 shows both the integral and the differential costates obtained using N = 32
LGR collocation points. It is seen that the costate estimate is indistinguishable from
the optimal solution. Furthermore, Fig. 6-12 shows the base ten logarithm of the
L∞-norm error for the integral and differential costates when approximated using
(N = 4, 8, 12, ... , 32) LGR collocation points. It is seen that the differential and integral
costate estimates converges exponentially as a function of N until the error reaches
approximately O(10−12) for N = 32.
151
3.5
3.5
4.5
4.5
5.5
0 0.5
1.5
1.5
2.5
2.51 1
2
2
3
3
4
4
5
5t
Sta
te
y∗(t)
y(t)
(A) State.
-3.5
-2.5
-1.5
3.5 4.5
-0.5
0
0
0.5
0.5 1.5 2.5-4
-3
-2
-1
1 2 3 4 5t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-10. State and control for Example 2 obtained using integral LGR.
152
-0.8
-0.6
-0.4
-0.2
0
0
0.2
0.4
0.6
0.8
1
1 2 3 4 5t
Cos
tate
p∗y
λ∗
y
py
λy
Figure 6-11. Integral and differential costate for Example 2 obtained using integral LGRcollocation with N = 32.
322824-12
0
0
-8
-6
-4
-2
4 8 12 16 20
-10
N
log10
Infin
ity-N
orm
Err
or
∥
∥λy − λ∗y
∥
∥
∞
∥
∥py − p∗y∥
∥
∞
Figure 6-12. Integral and differential costate errors for Example 2 obtained using integralLGR collocation.
153
6.3 Example 3: First-Order State Inequality Path Constraint Problem
Consider the following state inequality path constrained optimal control problem:
Minimize
∫ 3
0
e−tu dt subject to
y = u, y(0) = 0,
y − 1 + (t − 2)2 ≥ 0,
0 ≤ u ≤ 3.
(6–4)
The optimal state and control for this example are given as
y ∗ =
0 , t ∈ [0, 1),
1− (t − 2)2 , t ∈ [1, 2],
1 , t ∈ (2, 3],
u∗ =
0 , t ∈ [0, 1),
2(2− t) , t ∈ [1, 2],
0 , t ∈ (2, 3],
(6–5)
This example was solved using LG and flipped LGR collocation with the NLP solver
SNOPT, where SNOPT was implemented using default settings. The solution domain
was divided into three intervals with N collocation points in each interval. The boundaries
between the intervals were chosen to be the time instants where the state constraint
changes between active and inactive, namely, the interval boundaries were at t = 1 and
t = 2. Furthermore, a straight line initial guess between the initial state and unity was
used for the state and control.
6.3.1 Solution Using Collocation at Legendre-Gauss Points
Example 3 was solved using variable-order collocation at the Legendre-Gauss
points, as described in Chapter 2. Figure 6-13 shows the state and control approximations
obtained using N = 10 collocation points per interval. It is seen that variable-order LG
collocation provides highly accurate approximations to the optimal solution even though
index-reduction of the state inequality path constraint was not performed. Figure 6-14
154
0
0
0.2
0.4
0.5
0.6
0.8
1.2
1.5 2.5
1
1 2 3-0.2
t
Sta
te
y∗(t)
y(t)
(A) State.
00
0.5
0.5
1.5
1.5
2.5
2.5
1
1
2
2 3t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-13. Primal solution for Example 3 obtained using collocation at LG points.
shows the base ten logarithm of the L∞-norm error for the state and the control for
(N = 2, 4, 6, ... , 20) collocation points per interval. It is interesting to see that the LG
state and control approximations are highly accurate even using low-degree state
approximations (that is, using a small number of collocation points).
155
-8.5
-9.5
-10.5
-11
-11.5
12
-9
-8
2 4 6 8 10 12 14 16 18 20
-10
N
log10
Infin
ity-N
orm
Err
or
‖y − y∗‖∞
‖u − u∗‖∞
Figure 6-14. State and control errors for Example 3 obtained using LG collocation.
156
6.3.1.1 Previously derived costate estimate
The accuracy of the costate estimate of Ref. [1] is now compared to the costate
estimate derived in Chapter 5. The analytic optimal costate for Example 3 using
the method of direct adjoining can be found by applying the first-order optimality
conditions derived in Section 2.3.2. The costate and state constraint multipliers are
given, respectively, as
λ∗ =
−e−1 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3],
µ∗ =
0 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3].
(6–6)
Figure 6-15 shows the result of the costate approximation for N = 10 collocation
points per interval. It can be seen that the costate, λ(t), is not approximated correctly at
the interval boundaries where the discontinuities occur. Furthermore, the approximation
of the state constraint multiplier, µ, is quite poor. Figure 6-16 shows the base ten
logarithm of the L∞-norm error for the costate and the state constraint mutlipliers for
(N = 2, 4, 6, ... , 20) collocation points per interval. It is seen that the costate has large
errors near the known discontinuities in the optimal costate. Furthermore, the state
constraint multiplier estimate diverges.
157
0
0 0.5 1.5 2.5
0.05
1 2 3-0.4
-0.35
-0.3
-0.25
-0.15
-0.1
-0.05
-0.2
t
Cos
tate
λ∗(t)
λ(t)
(A) Costate.
-0.5
0
0 0.5 1.5 2.5-4
-3
-2
-1
1 2 3
-3.5
-2.5
-1.5
t
Sta
teC
onst
rain
tMul
tiplie
r
µ∗(t)
µ(t)
(B) State Constraint Multiplier.
Figure 6-15. Costate Estimate as derived by Ref. [1] for Example 3 obtained usingcollocation at LG points.
158
0
1.2
1
2 4 6 8 10 12 14 16 18 20
-0.2
-0.4
-0.6
0.2
0.4
0.6
0.8
N
log10
Infin
ity-N
orm
Err
or
‖λ− λ∗‖∞
‖µ− µ∗‖∞
Figure 6-16. Errors in costate estimate derived by Ref. [1] for Example 3 obtained usingLG collocation.
159
6.3.1.2 Costate estimate using method of indirect adjoining with continuousmultipliers
The costate estimate derived using the method of indirect adjoining with continuous
multipliers for collocation at the LG points, described in Chapter 5, is now analyzed. The
optimal costate for Example 3 using the method of indirect adjoining with continuous
multipliers can be found by applying the first-order optimality conditions derived in
Section 2.3.3. The costate and the state constraint multipliers are given, respectively, as
p∗ =
0 , t ∈ [0, 1),
0 , t ∈ [1, 2],
0 , t ∈ (2, 3],
ν∗ =
−e−1 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3].
(6–7)
Figure 6-17 shows the costate approximation for N = 10. It can be seen that the
estimates presented in Chapter 5 provide an accurate costate approximation of the
continuous optimal control problem. Figure 6-18 shows the base ten logarithm of the
L∞-norm error for the costate and state constraint multiplier. It can be seen that the
error on the state inequality constraint multiplier decreases as the number of collocation
points is increased. Furthermore, the error on the costate approximation remains
approximately zero. Therefore, the costate estimate produces an accuracy of O(10−12)
even when inacuracies in the state constraint multiplier are present due to the use of
low-order polynomial approximations in the state.
160
0.5 1 1.5 2 2.5 3-0.4
-0.35
-0.3
-0.25
-0.2
-0.15
-0.1
-0.05
0
0
0.05
t
Dua
lVar
iabl
es
p∗(t)
ν∗(t)
p(t)
ν(t)
Figure 6-17. Dual variables for Example 3 obtained using collocation at LG points.
0
-2
-4
-6
-8
-10
-12
-142 4 6 8 10 12 14 16 18 20
N
log10
Abs
olut
eE
rror
ν(t)
p(t)
Figure 6-18. Costate errors for Example 3 obtained using collocation at LG points.
161
6.3.2 Solution Using Collocation at Flipped Legendre-Gauss-Radau Points
Example 3 is now solved using variable-order flipped LGR collocation as described
in Chapter 2. Figure 6-19 shows the state and control approximations obtained using
N = 10 collocation points per interval. It is seen that variable-order collocation at the
LG points provides highly accurate approximations to the optimal solution even though
index-reduction of the state inequality path constraint was not performed. Figure 6-20
shows the base ten logarithm of the L∞-norm error for the state and the control using
(N = 2, 4, 6, ... , 20) collocation points per interval. It is seen that the solution using LGR
collocation is less accurate than the solution obtained using collocation at the LG points.
This difference in accuracy is expected, as the LG points are known to have a higher
accuracy quadrature than the LGR points. Furthermore, it is seen that the state and
control reach accuracies of O(10−5) and O(10−8), respectively, for N = 20.
6.3.2.1 Previously derived costate estimate
The accuracy of the costate estimate of Ref. [1] using flipped LGR collocation is
now compared against the accuracy of the costate estimate derived in this research.
The optimal costate for Example 3 using the method of direct adjoining can be found by
applying the first-order optimality conditions derived in Section 2.3.2. The costate and
state constraint multipliers are given, respectively, as
λ∗ =
−e−1 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3],
µ∗ =
0 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3].
(6–8)
Figure 6-21 shows the result of the costate approximation when N = 10 collocation
points per interval were used. It can be seen that although the costate is approximated
162
0
0
0.2
0.4
0.5
0.6
0.8
1.2
1.5 2.5
1
1 2 3-0.2
t
Sta
te
y∗(t)
y(t)
(A) State.
00
0.2
0.4
0.5
0.6
0.8
1.2
1.4
1.5
1.6
1.8
2.5
1
1
2
2 3t
Con
trol
u∗(t)
u(t)
(B) Control.
Figure 6-19. Primal solution for Example 3 obtained using variable-order collocation atLGR points.
163
−16
−14
12
-8
-6
-4
-2
2 4 6 8 10 12 14 16 18 20
-10
N
log10
Infin
ity-N
orm
Err
or
‖y − y∗‖∞
‖u − u∗‖∞
Figure 6-20. State and control errors for Example 3 obtained using variable-ordercollocation at LGR points.
accurately, the state constraint multiplier, µ, is approximated very poorly where the
costate is discontinuous. Figure 6-22 shows the base ten logarithm of the L∞-norm
error for the costate and the state constraint mutliplier when approximated using
(N = 2, 4, 6, ... , 20) collocation points per interval. It is seen that the costate estimate
has large errors near the known discontinuities in the optimal costate. Furthermore, it is
seen that the state constraint multiplier estimate diverges.
6.3.2.2 Costate estimation using method of indirect adjoining with continuousmultipliers
The costate estimate derived using the method of indirect adjoining with continuous
multipliers for collocation at the flipped LGR points,described in Chapter 5, is now
analyzed. The analytic optimal costate for Example 3 using the method of indirect
adjoining with continuous multipliers can be found by applying the first-order optimality
conditions derived in Section 2.3.3. The costate and the state constraint multipliers are
164
Dua
l Sol
utio
n
0
0 0.5 1.5 2.51 2 3-0.4
-0.35
-0.3
-0.25
-0.15
-0.1
-0.05
-0.2
t
λ∗(t)
λ(t)
(A) Costate.
−14
−12
−10
−8
−6
Dua
l Sol
utio
n
0
0 0.5 1.5 2.5
-4
-2
1 2 3t
µ∗(t)
µ(t)
(B) State Constraint Multiplier.
Figure 6-21. Costate Estimate as derived by Ref. [1] for Example 3 obtained using LGRcollocation.
165
−0.5
0.5
0
1.5
1
2
2 4 6 8 10 12 14 16 18 20N
log10
Infin
ity-N
orm
Err
or‖λ− λ∗‖
∞
‖µ− µ∗‖∞
Figure 6-22. Errors in costate estimate derived by Ref. [1] for Example 3 obtained usingLGR collocation.
given, respectively, as
p∗ =
0 , t ∈ [0, 1),
0 , t ∈ [1, 2],
0 , t ∈ (2, 3],
ν∗ =
−e−1 , t ∈ [0, 1),
−e−t , t ∈ [1, 2],
0 , t ∈ (2, 3].
(6–9)
Figure 6-23 shows the costate approximation for N = 10 obtained by using the
method described in Chapter 5 for collocation at LGR points. It can be seen that the
estimate presented in this research provides an accurate approximation of the costate.
Figure 6-24 shows the base ten logarithm of the L∞-norm error for the costate and state
constraint multiplier approximations. It can be seen that the error on the state inequality
constraint multiplier decreases exponentially as the number of collocation points is
increased. Furthermore, the error on the costate approximation remains approximately
zero. Therefore, the costate estimate produces an accuracy of O(10−12) even when
166
0.5 1 1.5 2 2.5 3-0.4
-0.35
-0.3
-0.25
-0.2
-0.15
-0.1
-0.05
0
0
0.05
t
Dua
lVar
iabl
es
p∗(t)
ν∗(t)
p(t)
ν(t)
Figure 6-23. Costate estimate for Example 3 obtained using collocation at LGR points.
0
-2
-4
-6
-8
-10
-12
-142 4 6 8 10 12 14 16 18 20
N
log10
Abs
olut
eE
rror
ν(t)
p(t)
Figure 6-24. Costate errors for Example 3 obtained using collocation at LGR points.
inacuracies in the state constraint multiplier are present due to the use of low-order
polynomial approximations in the state.
167
6.4 Example 4: Second-Order State Inequality Path Constraint Example
Consider the following second-order state inequality constrained optimal control
problem from Ref. [10]:
minimize 12
∫ 1
0
u2dt subject to
x = v ,
v = u,
x(0) = 0,
x(1) = 0,
v(0) = 1
v(1) = −1,
x(t) ≤ ℓ.
It is known for this example that the inequality path constraint is inactive for ℓ > 1/4,
is active at only a single point for 1/6 < ℓ ≤ 1/4, and is active along a nonzero duration
arc for 0 < ℓ ≤ 1/6. In the case where 0 < ℓ ≤ 1/6, the optimal state and control are
given as
x∗(t) =
ℓ[
1−(
1− t3ℓ
)3]
,
ℓ,
ℓ[
1−(
1− 1−t3ℓ)3]
,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1],
v∗(t) =
(
1− t3ℓ
)2,
0,
−(
1− 1−t3ℓ)2,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1],
u∗(t) =
− 23ℓ(
1− t3ℓ
)
,
0,
− 23ℓ(
1− 1−t3ℓ)
,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1],
A value of ℓ = 1/10 was used in the analysis of this example. The solution domain was
divided into three intervals with N collocation points in each interval. The boundaries
between the intervals were chosen to be the time instants where the state constraint
changes between active and inactive, namely, t = 3/10 and t = 7/10. The solution was
168
approximated using N = 5 collocation points. All problems were solved using the NLP
solver SNOPT with default optimality and feasibility tolerances. [22]. The initial guess
used was the exact solution.
6.4.1 Solution Using Collocation at Legendre-Gauss Points
Example 4 was solved using variable-order collocation at the Legendre-Gauss
points, as described in Chapter 2. Figure 6-25 shows the state and control approximations
obtained using N = 5 collocation points per interval. It is seen that variable-order
collocation at the LG points provides highly accurate approximations to the optimal
solution even though index-reduction of the state inequality path constraint was not
performed. Figure 6-26 shows the base 10 logarithm of the L∞-norm error for the state
and the control when approximated using (N = 2, 3, ... , 10) collocation points per
interval. It is interesting to see that the errors in the primal solution for this example are
larger than the errors observed for the primal solution of Example 3. This difference
in accuracy can be attributted to the increase in the order of the state inequality path
constraint. Although the errors in this example are larger than for Example 3, it can be
seen that an accuracy of O(10−6) and O(10−5) for the state and control, respectively,
can be obtained using N = 3 collocation points per interval.
169
00
0.1
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.02
0.04
0.06
0.08
0.12
t
Sta
teC
ompo
nent
x∗(t)
x(t)
(A) x(t).
0
0 0.1
0.2
0.2 0.3
0.4
0.4 0.5
0.6
0.6 0.7
0.8
0.8 0.9
1
1
-0.2
-0.4
-0.6
-0.8
-1t
Sta
teC
ompo
nent
v∗(t)
v(t)
(B) v(t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
-4
-3
-2
1
1
-5
-6
-7
-1
t
Con
trol
u∗(t)
u(t)
(C) u(t).
Figure 6-25. Primal solution for Example 4 obtained using LG collocation.
170
N
0
-8
-7
-6
-5
-4
-3
-2
-1
2 3 4 5 6 7 8 9 10
log10
Infin
ity-N
orm
Err
or
x(t)
v(t)
u(t)
Figure 6-26. State and control errors for Example 4 using collocation at LG points.
171
6.4.1.1 Previously derived costate estimate
The accuracy of the costate estimate using LG collocation derived in Ref. [1] is
now compared against the accuracy of the costate estimate derived in this research.
The optimal costate for Example 4 using the method of direct adjoining can be found by
applying the first-order optimality conditions derived in Section 2.3.2. The costate and
state constraint multiplier are given, respectively, as
λ∗x =
29ℓ2
, t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],
− 29ℓ2
, t ∈ [1− 3ℓ, 1],
λ∗v =
23ℓ
(
1− t3ℓ
)
, t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],23ℓ
(
1− 1−t3ℓ
)
, t ∈ [1− 3ℓ, 1],
µ∗ =
0 , t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],
0 , t ∈ [1− 3ℓ, 1].
(6–10)
Figure 6-27 shows the costate approximation for N = 5 collocation points per
interval. It can be seen that the costate, λ(t), is not approximated correctly at the
interval boundaries where the discontinuities occur. Furthermore, the state constraint
multiplier, µ, is approximated very poorly. Figure 6-28 shows the base ten logarithm of
the L∞-norm error for the costate and the state constraint mutliplier when approximated
using (N = 2, 3, ... , 10) collocation points per interval. It can be seen that large errors
around the costate discontinuities prevent the costate estimate from converging to its
optimal solution.
172
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
25
20
15
10
5
-5
-10
-15
-20
-25t
Cos
tate
Com
pone
nt
λ∗
x (t)
λx (t)
(A) Costate.
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
6
7
-1
1
1
2
3
4
5
t
Cos
tate
Com
pone
nt
λ∗
v (t)
λv(t)
(B) Costate.
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-50
-100
-150
-200
-250
-300
-350
-400t
µ∗(t)
µ(t)
Sta
teC
onst
rain
tMul
tiplie
r
(C) State Constraint Multiplier.
Figure 6-27. Costate Estimate as derived by Ref. [1] for Example 4 obtained usingcollocation at LG points.
173
1.5
2.5
1
2
2
3
3 4 5 6 7 8 9 10
3.5
0.5
N
log10
Infin
ity-N
orm
Err
or
‖λx − λ∗x ‖∞
‖λv − λ∗v ‖∞
‖µ− µ∗‖∞
Figure 6-28. Errors in costate estimate derived by Ref. [1] for Example 4 obtained usingLG collocation.
174
6.4.1.2 Costate Estimation using method of indirect adjoining with continuousmultipliers
The costate estimate derived using the method of indirect adjoining with continuous
multipliers for collocation at the LG points, described in Chapter 5, is now analyzed. The
optimal costate for Example 4 using the method of indirect adjoining with continuous
multipliers can be found by applying the first-order optimality conditions derived in
Section 2.3.3. The costate and the state constraint multiplier are given, respectively, as
p∗x(t) =
{
− 29ℓ2, t ∈ [0, 1],
p∗v (t) =
23ℓ
(
1− t3ℓ
)
,
0,
23ℓ
(
1− 1−t3ℓ
)
,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1].
ν∗(t) =
− 49ℓ2,
− 29ℓ2,
0,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1].
Figure 6-29 shows the dual variable approximations for collocation at the LG points.
It can be seen that the mapping presented in Chapter 5 provides an accurate estimate
for the dual variables of the continuous optimal control problem. Figure 6-30 shows the
base ten logarithm of the L∞-norm error for the costate and state constraint multiplier
approximations obtained using Eqs. (5–32)–(5–38) and Eq. (5–79) for N collocation
points in each of the three mesh intervals. It can be seen that the error on the dual
variables reach an accuracy of O(10−5) for low-order polynomial approximations in the
state (that is, a small number of collocation numbers).
175
-22.5
-22.4
-22.3
-22.2
-22.1
-22
-21.9
-21.8
-21.7
-21.6
-21.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1t
p∗x (t)
px (t)
Cos
tate
Com
pone
nt
(A) px(t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
6
7
-1
1
1
2
3
4
5
t
p∗v (t)
pv (t)
Cos
tate
Com
pone
nt
(B) pv (t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-5
-10
-15
-20
-25
-30
-35
-40
-45t
ν∗(t)ν(t)
Sta
teC
onst
rain
tMul
tiplie
r
(C) ν(t).
Figure 6-29. Costate estimate for Example 4 obtained using collocation at LG points.
176
0
-6
-5
-4
-3
-2
-1
1
2
2 3 4 5 6 7 8 9 10
N
log10
Abs
olut
eE
rror
ν(t)
px(t)
pv (t)
Figure 6-30. Costate errors for Example 4 obtained using collocation at LG points.
177
6.4.2 Solution Using Collocation at Flipped Legendre-Gauss-Radau Points
Next, Example 4 was solved using variable-order collocation at the flipped
Legendre-Gauss-Radau points, as described in Chapter 2. Figure 6-31 shows the
state and control approximations obtained using N = 5 collocation points per interval. It
is seen that variable-order collocation at the LG points provides accurate approximations
to the optimal solution even though index-reduction of the state inequality path constraint
was not performed. Figure 6-32 shows the base 10 logarithm of the L∞-norm error for
the state and the control when approximated using (N = 2, 3, ... , 10) collocation points
per interval. It is seen that the state reaches an accuracy of O(10−5) for N = 3, whereas
the control only reaches an accuracy of O(10−2) for N = 10.
178
0.01
0.03
0.07
0.09
00
0.1
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.05
1
0.02
0.04
0.06
0.08
t
Sta
teC
ompo
nent
x∗(t)
x(t)
(A) x(t).
0
0 0.1
0.2
0.2 0.3
0.4
0.4 0.5
0.6
0.6 0.7
0.8
0.8 0.9
1
1
-0.2
-0.4
-0.6
-0.8
-1t
Sta
teC
ompo
nent
v∗(t)
v(t)
(B) v(t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
-4
-3
-2
1
-5
-6
-7
-1
t
Con
trol
u∗(t)
u(t)
(C) u(t).
Figure 6-31. Primal solution for Example 4 obtained using LGR collocation.
179
N
0
-5
2 3 4
5
5 6 7 8 9 10
-10
-20
-25
-30
-35
-15
log10
Infin
ity-N
orm
Err
or
x(t)
v(t)
u(t)
Figure 6-32. State and control errors for Example 4 using collocation at LGR points.
180
6.4.2.1 Previously derived costate estimate
he accuracy of the costate estimate using LGR collocation derived in Ref.[1] is now
compared against the accuracy of the costate estimate derived in this research. The
optimal costate for Example 4 using the method of direct adjoining can be found by
applying the first-order optimality conditions derived in Section 2.3.2. The costate and
state constraint multipliers are given, respectively, as
λ∗x =
29ℓ2
, t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],
− 29ℓ2
, t ∈ [1− 3ℓ, 1],
λ∗v =
23ℓ
(
1− t3ℓ
)
, t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],23ℓ
(
1− 1−t3ℓ
)
, t ∈ [1− 3ℓ, 1],
µ∗ =
0 , t ∈ [0, 3ℓ],
0 , t ∈ [3ℓ, 1− 3ℓ],
0 , t ∈ [1− 3ℓ, 1].
(6–11)
Figure 6-33 shows the costate approximation for N = 5 collocation points per
interval.. It can be seen that although the costate is approximated accurately, the state
constraint multiplier, µ, is approximated very poorly. Figure 6-34 shows the base ten
logarithm of the L∞-norm error for the costate and the state constraint mutliplier when
approximated using (N = 2, 3, ... , 10) collocation points per interval. It can be seen
that large errors around the costate discontinuities prevent the costate estimate from
converging to its optimal solution.
181
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
25
20
15
10
5
-5
-10
-15
-20
-25t
Cos
tate
Com
pone
nt
λ∗
x (t)
λx (t)
(A) Costate.
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
6
7
-1
1
1
2
3
4
5
t
Cos
tate
Com
pone
nt
λ∗
v (t)
λv(t)
(B) Costate.
−2000
−1500
−1000
−500
500
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1t
µ∗(t)
µ(t)
Sta
teC
onst
rain
tMul
tiplie
r
(C) State Constraint Multiplier.
Figure 6-33. Costate Estimate as derived by Ref. [1] for Example 4 obtained usingcollocation at LGR points.
182
0
-10
12
-8
-6
-4
-2
2
2 3 4 5 6 7 8 9 10N
log10
Infin
ity-N
orm
Err
or
‖λx − λ∗x ‖∞
‖λv − λ∗v ‖∞
‖µ− µ∗‖∞
Figure 6-34. Errors in costate estimate derived by Ref. [1] for Example 4 obtained usingLGR collocation.
183
6.4.2.2 Costate Estimation using method of indirect adjoining with continuousmultipliers
The costate estimate derived using the method of indirect adjoining with continuous
multipliers for collocation at the flipped LGR points, described in Chapter 5, is now
analyzed. The analytic optimal costate for Example 4 using the method of indirect
adjoining with continuous multipliers can be found by applying the first-order optimality
conditions derived in Section 2.3.3. The costate and the state constraint multiplier are
given, respectively, as
p∗x(t) =
{
− 29ℓ2, t ∈ [0, 1],
p∗v (t) =
23ℓ
(
1− t3ℓ
)
,
0,
23ℓ
(
1− 1−t3ℓ
)
,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1].
ν∗(t) =
− 49ℓ2,
− 29ℓ2,
0,
t ∈ [0, 3ℓ],
t ∈ [3ℓ, 1− 3ℓ],
t ∈ [1− 3ℓ, 1].
Figure 6-35 shows the dual variable approximations obtained by using the method
described in this paper for collocation at LGR points. It can be seen that the mapping
presented in Chapter 5 provides an accurate estimate for the dual variables of the
continuous optimal control problem. Figure 6-36 shows the base 10 logarithm of the
L∞-norm error for the costate and state constraint multiplier approximations for N
collocation points in each of the three mesh intervals. It can be seen that the error
on the estimate of the state inequality constraint multiplier and the costate reach an
accuracy of O(10−5) for N larger than two.
184
-22.5
-22.4
-22.3
-22.2
-22.1
-22
-21.9
-21.8
-21.7
-21.6
-21.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1t
p∗x (t)
px (t)
Cos
tate
Com
pone
nt
(A) px(t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
6
7
-1
1
1
2
3
4
5
t
p∗v (t)
pv (t)
Cos
tate
Com
pone
nt
(B) pv (t).
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-5
-10
-15
-20
-25
-30
-35
-40
-45t
ν∗(t)ν(t)
Sta
teC
onst
rain
tMul
tiplie
r
(C) ν(t).
Figure 6-35. Costate estimate for Example 4 obtained using collocation at LGR points.
185
0
-6
-5
-4
-3
-2
-1
1
2
2 3 4 5 6 7 8 9 10
N
log10
Abs
olut
eE
rror
ν(t)
px(t)
pv (t)
Figure 6-36. Costate errors for Example 4 obtained using collocation at LGR points.
186
CHAPTER 7CONCLUSIONS
Solving an optimal control problem is not easy. For most engineering applications,
it is impossible to derive an analytic solution to an optimal control problem using the
first-order optimality conditions derived from the calculus of variations. Thus, numerical
methods must be used to approximate the solution to the continuous-time problem.
Applying the first-order optimality conditions of the continuous-time optimal control
problem result in a Hamiltonian boundary-value problem which must be solved.
Numerical methods that aproximate a solution to the Hamiltonian boundary-value
problem stemming from the first-order optimality conditions of the optimal control
problem are called indirect methods. Numerical methods that employ the indirect
method tend to result in a highly accurate solution because the first-order optimality
conditions of the optimal control problem are satisfied. Convergence using indirect
methods, however, can be very hard to achieve due to the unstable nature of the
Hamiltonian boundary-value problem. Thus, intuitive initial guesses are required to
achieve convergence using indirect methods.
Numerical methods for solving optimal control problems that do not formulate the
first-order optimality conditions of the continous-time problem are called direct methods.
Direct methods convert the infinite-dimensional continuous control problem into a
finite-dimensional discrete nonlinear programming problem (NLP). The resulting NLP
can then be solved by well-developed NLP algorithms. Direct methods are attractive
because the first-order optimality conditions need not be derived. Furthermore, because
the Hamiltonian boundary-value problem is not formulated, convergence using direct
methods is usually easier to obtain. In this research a direct orthogonal collocation
method using collocation at the Legendre-Gauss and Legendre-Gauss-Radau
points was analyzed. In particular, an estimate of the continuous-time costate of the
187
continuous-time optimal control problem was derived from the KKT multipliers of the
nonlinear programming problem of the discrete problem.
Costate estimation is an important step in the numerical solution of optimal control
problems. Mapping the dual variables of the numerical solution to the costate of the
continuous-time problem not only allows for a verification of the dual solution, but
also allows the first-order optimality conditions of the nonlinear programming problem
(NLP) to take a form that is equivalent to the first-order optimality conditions of the
continuous-time problem. Thus, having an accurate costate estimate shows that the
KKT conditions satisfied by the NLP are a discrete form of the first-order optimality
conditions of the continuous problem given by the calculus of variations, and will
converge to an optimal solution of the continuous-time problem if discretized correctly.
Costate estimates for direct methods using orthogonal collocation at the Legendre-Gauss
and Legendre-Gauss-Radau points have previously been derived for optimal control
problems with no active state inequality path constraints and when the dynamic
constraints are formulated in their differential form. In this research a gap of costate
estimation theory was closed by deriving a mapping for the costate estimate for the case
when the dynamic constraints are expressed in integral form and in the presence of
state inequality path constraints.
In the first part of this research a costate estimate was developed for problem stated
with integral constraints. While the differential and integral forms of the LG and LGR
methods are mathematically equivalent with regard to the primal variables (that is, the
state and control), the two formulations produce completely different dual variables. In
particular, the relationship between the Lagrange multipliers of the collocation conditions
of the dynamic constraints and the costate of the optimal control problem has been
well documented. On the other hand, the corresponding relationship between the
Lagrange multipliers associated with the integral forms of LG and LGR collocation and
the costate of the optimal control problem has not been established. When employing
188
the integral forms of LG and LGR collocation, however, it may be of interest to either
verify optimality or perform sensitivity analysis in a manner consistent with that which
would be performed when using variational methods. In such cases it is useful to
obtain a costate estimate when using the integral forms of the LG and LGR methods.
Thus, in this research, a costate estimate for collocation at Legendre-Gauss and
Legendre-Gauss-Radau points was derived for the case when the dynamic constraints
of the optimal control problem are formulated in integral form. It was demonstrated
that the costate mapping derived for collocation at the LG and LGR points leads to a
set of transformed optimality conditions of the NLP which were shown to be a discrete
representation of the necessary conditions for optimality of the continuous-time problem.
Finally, a relationship between the integral and the differential forms of the costate
estimate was given and it was shown that the two sets of optimality conditions are
equivalent.
The second part of this research focused on problems with active state inequality
path constraints. Although previous research has successfully derived a high-accuracy
estimate of the costate from the KKT multipliers of the NLP for the case of a problem
with no active state inequality path constraints, Ref. [1] subsequently showed that in the
case when the costate is discontinuous (as is the case in the presence of active state
inequality path constraints), this costate estimate leads to a set of first-order optimality
conditions of the NLP that are not equivalent to the discrete form of the variational
optimality conditions. This lack of equivalence leads to an inaccurate approximation
of the costate. Therefore, in this research costate estimates for collocation at LG and
LGR points were derived for problems with active state inequality path constraints.
The derived costate estimate was shown to lead to a transformed adjoint system of
the NLP which is a discrete approximation of the necessary conditions for optimality
of the continuous-time optimal control problem. This equivalence was not existent with
prior costate estimates using LG and LGR collocation. The costate estimates derived
189
in this dissertation were implemented in four problems to assess their accuracy. It was
shown that each discrete costate estimate led to an accurate approximation of the
continuous-time costate.
190
REFERENCES
[1] Darby, C. L., Garg, D., and Rao, A. V., “Costate Estimation Using Multiple-IntervalPseudospectral Methods,” Journal of Spacecraft and Rockets, Vol. 48, No. 5,September–October 2011, pp. 856–866.
[2] Betts, J., Practical Methods for Optimal Control and Estimation Using NonlinearProgramming, SIAM Press, Philadelphia, 2nd ed., 2009.
[3] Unger, J., Kroner, A., and Marquardt, W., “Structural analysis ofdifferential-algebraic equation systems - theory and applications,” Computersand Chemical Engineering, Vol. 19, No. 8, 1995, pp. 867 – 882.
[4] Feehery, W. F. and Barton, P. I., “Dynamic optimization with state variable pathconstraints,” Computers and Chemical Engineering, Vol. 22, No. 9, 1998, pp. 1241– 1256.
[5] Maurer, H. and Pesch, H. J., “Direct optimization methods for solving a complexstate-constrained optimal control problem in microeconomics,” Applied Mathematicsand Computation, Vol. 204, No. 2, 2008, pp. 568 – 579.
[6] Boskens, Christof, M. H., “SQP-methods for solving optimal control problems withcontrol and state constraints: adjoint variables, sensitivity analysis and real-timecontrol,” J. Comput. Appl. Math., Vol. 120, No. 1-2, Aug. 2000, pp. 85–108.
[7] Bryson, A. E. and Ho, Y.-C., Applied Optimal Control , Hemisphere Publishing, NewYork, 1975.
[8] Speyer, J. L., “Necessary Conditions for Optimality For Paths Lying on a Corner,”Management Science, Vol. 19, No. 11, July 1973.
[9] Kirk, D. E., Optimal Control Theory: An Introduction, Dover Publications, Mineola,New York, 2004.
[10] Bryson, A. E., Denham, W. F., and Dreyfus, S. E., “Optimal Programming Problemswith Inequality Constraints I: Necessary Conditions for Extremal Solutions,” AIAAJournal , Vol. 1, No. 11, 1962, pp. 2544–2550.
[11] Dreyfus, S. E., Variational Problems with State Variable Inequality Constraints,Ph.D. thesis, Harvard University, Cambridge, Massachussets, 1962.
[12] Jacobson, D. H., Lele, M. M., and Speyer, J. L., “New Necessary Conditions ofOptimality for Control Problems with State-Variable Inequality Constraints,” Tech.rep., Division of Engineering and Applied Physics - Harvard University, Cambridge,Massachussets, 1969.
[13] Speyer, J. L. and Bryson, A. E., “Optimal Programming Problems with a BoundedState Space,” AIAA Journal , Vol. 6, No. 8, August 1968, pp. 1488–1491.
191
[14] Denham, W. F. and Bryson, A. E., “Optimal Programming Problems with InequalityConstraints II: Solution by Steepest-Ascent,” AIAA Journal , Vol. 2, No. 1, January1964, pp. 25–34.
[15] Hartl, R. F., Sethi, S. P., and Vickson, R. G., “A Survey of the Maximum Principlesfor Optimal Control Problems with State Constraints,” SIAM Review , Vol. 37, No. 2,June 1995, pp. 181–218.
[16] Gollan, B., “On Optimal Control Problems with State Constraints,” Journal ofOptimization Theory and Applications, Vol. 32, 1980, pp. 75–80.
[17] Russak, I. B., “On Problems with Bounded State Variable,” Journal of OptimizationTheory and Applications, Vol. 5, 1970, pp. 114–157.
[18] Russak, I. B., “On General Problems with Bounded State Variable,” Journal ofOptimization Theory and Applications, Vol. 6, 1970, pp. 424–452.
[19] Vinter, R., Optimal Control (Systems & Control: Foundations and Applications),Birkhauser, Boston, 2000.
[20] Russel, R. and Shampine, L., “A Collocation Method for Boundary Value Problems,”Numerical Mathematics, Vol. 19, 1972, pp. 1–28.
[21] Rao, A. V. and Mease, K. D., “Dichotomic Basis Approach to solvingHyper-Sensitive Optimal Control Problems,” Automatica, Vol. 35, No. 4, April1999, pp. 633–642.
[22] Gill, P. E., Murray, W., and Saunders, M. A., “SNOPT: An SQP Algorithm forLarge-Scale Constrained Optimization,” SIAM Review , Vol. 47, No. 1, January2005, pp. 99–131.
[23] Biegler, L. T. and Zavala, V. M., “Large-Scale Nonlinear Programming Using IPOPT:An Integrating Framework for Enterprise-Wide Optimization,” Computers andChemical Engineering, Vol. 33, No. 3, March 2008, pp. 575–582.
[24] Wachter, A., An Interior Point Algorithm for Large-Scale Nonlinear Optimizationwith Applications in Process Engineering, Ph.D. thesis, Department of ChemicalEngineering, Carnegie-Mellon University, Pittsburgh, PA, 2002.
[25] Wachter, A. and Biegler, L. T., “On the Implementation of an Interior-Point FilterLine-Search Algorithm for Large-Scale Nonlinear Programming,” MathematicalProgramming, Vol. 106, No. 1, 2006, pp. 25–57.
[26] Fahroo, F. and Ross, I. M., “Costate Estimation by a Legendre PseudospectralMethod,” Journal of Guidance, Control, and Dynamics, Vol. 24, No. 2, March–April2001, pp. 270–277.
[27] Benson, D. A., Huntington, G. T., Thorvaldsen, T. P., and Rao, A. V., “DirectTrajectory Optimization and Costate Estimation via an Orthogonal Collocation
192
Method,” Journal of Guidance, Control, and Dynamics, Vol. 29, No. 6,November-December 2006, pp. 1435–1440.
[28] Bock, H. G. and Plitt, K. J., “A Multiple Shooting Algorithm for Direct Solution ofOptimal Control Problems,” IFAC 9th World Congress, Budapest, Hungary, 1984.
[29] Williamson, W. E., “Use of Polynomial Approximations to Calculate SuboptimalControls,” AIAA Journal , Vol. 9, No. 11, 1971, pp. 2271–2273.
[30] Kraft, D., On Converting Optimal Control Problems into Nonlinear ProgrammingCodes, Vol. F15 of NATO ASI Series, in Computational Mathematical Program-ming, Springer-Verlag, Berlin, 1985, pp. 261–280.
[31] Von-Stryk, O. and Bulirsch, R., “Direct and Indirect Methods for TrajectoryOptimization,” Annals of Operations Research, Vol. 37, 1992, pp. 357–373.
[32] Hargraves, C. R. and Paris, S. W., “Direct Trajectory Optimization Using NonlinearProgramming Techniques,” Journal of Guidance, Control, and Dynamics, Vol. 10,No. 4, July–August 1987, pp. 338–342.
[33] Enright, P. J. and Conway, B. A., “Discrete Approximations to Optimal TrajectoriesUsing Direct Transcription and Nonlinear Programming,” Journal of Guidance,Control, and Dynamics, Vol. 19, No. 4, July–August 1996, pp. 994–1002.
[34] Benson, D. A., A Gauss Pseudospectral Transcription for Optimal Control , Ph.D.thesis, Department of Aeronautics and Astronautics, Massachusetts Institute ofTechnology, Cambridge, Massachusetts, 2004.
[35] Elnagar, G., Kazemi, M., and Razzaghi, M., “The Pseudospectral Legendre Methodfor Discretizing Optimal Control Problems,” IEEE Transactions on AutomaticControl , Vol. 40, No. 10, 1995, pp. 1793–1796.
[36] Garg, D., Patterson, M. A., Darby, C. L., Francolin, C., Huntington, G. T., Hager,W. W., and Rao, A. V., “Direct Trajectory Optimization and Costate Estimationof Finite-Horizon and Infinite-Horizon Optimal Control Problems using a RadauPseudospectral Method,” Computational Optimization and Applications, Vol. 49,No. 2, June 2011, pp. 335–358.
[37] Biegler, L. T., Ghattas, O., Heinkenschloss, M., and van Bloemen Waanders,B., editors, Large-Scale PDE Constrained Optimization, Lecture Notes inComputational Science and Engineering, Vol. 30, Springer-Verlag, Berlin, 2003.
[38] Betts, J. T., Campbell, S. L., and Engelsone, A., “Direct Transcription Solution ofOptimal Control Problems with Higher Order State Constraints: Theory vs Practice,”Optimization and Engineering, Vol. 8, No. 1, 2007, pp. 1–19.
[39] Betts, J. T. and Huffman, W. P., “Sparse Optimal Control Software – SOCS,” Tech.Rep. MEA-LR-085, Boeing Information and Support Services, Seattle, Washington,July 1997.
193
[40] Ross, I. M. and Fahroo, F., User’s Manual for DIDO 2001 α: A MATLAB Applicationfor Solving Optimal Control Problems, Department of Aeronautics and Astronautics,Naval Postgraduate School, Technical Report AAS-01-03, Monterey, California,2001.
[41] Von-Stryk, O., User’s Guide for DIRCOL Version 2.1: A Direct Collocation Methodfor the Numerical Solution of Optimal Control Problems, Technische UniversitatDarmstadt, Darmstadt, Germany, 1999.
[42] Rao, A. V., Benson, D. A., Darby, C. L., Francolin, C., Patterson, M. A., Sanders,I., and Huntington, G. T., “Algorithm 902: GPOPS, A Matlab Software for SolvingMultiple-Phase Optimal Control Problems Using the Gauss PseudospectralMethod,” ACM Transactions on Mathematical Software, Vol. 37, No. 2, April–June2010, pp. 22:1–22:39.
[43] J.T.Betts, “Survey of Numerical Methods for Trajectory Optimization,” Journal ofGuidance, Control, and Dynamics, Vol. 21, No. 2, March–April 1998, pp. 193–207.
[44] Hager, W. W., “Runge-Kutta Methods in Optimal Control and the TransformedAdjoint System,” Numerische Mathematik , Vol. 87, 2000, pp. 247–282.
[45] Dontchev, A. L., Hager, W. W., and Veliov, V. M., “Second-Order Runge-KuttaApproximations In Constrained Optimal Control,” SIAM Journal on NumericalAnalysis, Vol. 38, 2000, pp. 202–226.
[46] Betts, J. T., “Sparse Jacobian Updates in the Collocation Method for OptimalControl Problems,” Journal of Guidance, Control, and Dynamics, Vol. 13, No. 3,May–June 1990, pp. 409–415.
[47] Dontchev, A. L., Hager, W. W., and Malanowski, K., “Error Bounds for the EulerApproximation and Control Constrained Optimal Control Problem,” NumericalFunctional Analysis and Applications, Vol. 21, 2000, pp. 653–682.
[48] Dontchev, A., Hager, W., and Veliov, V., “Uniform Convergence and Meshindependence of Newton’s method for Discretized Variational Problems,” SIAMJournal on Control and Optimization, Vol. 39, 2000, pp. 961–980.
[49] Kameswaran, S. and Biegler, L. T., “Convergence Rates for Direct Transcriptionof Optimal Control Problems Using Collocation at Radau Points,” ComputationalOptimization and Applications, Vol. 41, No. 1, 2008, pp. 81–126.
[50] Darby, C. L., Hager, W. W., and Rao, A. V., “An hp-Adaptive Pseudospectral Methodfor Solving Optimal Control Problems,” Optimal Control Applications and Methods,Vol. 32, No. 4, July–August 2011, pp. 476–502.
[51] Darby, C. L., Hager, W. W., and Rao, A. V., “Direct Trajectory Optimization Using aVariable Low-Order Adaptive Pseudospectral Method,” Journal of Spacecraft andRockets, Vol. 48, No. 3, May–June 2011, pp. 433–445.
194
[52] Vlassenbroeck, J. and Dooren, R. V., “A Chebyshev Technique for SolvingNonlinear Optimal Control Problems,” IEEE Transactions on Automatic Control ,Vol. 33, No. 4, 1988, pp. 333–340.
[53] Vlassenbroeck, J., “A Chebyshev Polynomial Method for Optimal Control with StateConstraints,” Automatica, Vol. 24, No. 4, 1988, pp. 499–506.
[54] Elnagar, G. and Razzaghi, M., “A Collocation-Type Method for Linear QuadraticOptimal Control Problems,” Optimal Control Applications and Methods, Vol. 18,No. 3, 1998, pp. 227–235.
[55] Elnagar, G. and Kazemi, M., “Pseudospectral Chebyshev Optimal Control ofConstrained Nonlinear Dynamical Systems,” Computational Optimization andApplications, Vol. 11, No. 2, 1998, pp. 195–217.
[56] Elnagar, G. N. and Razzaghi, M. A., “Pseudospectral Legendre-Based OptimalComputation of Nonlinear Constrained Variational Problems,” Journal of Computa-tional and Applied Mathematics, Vol. 88, No. 2, March 1998, pp. 363–375.
[57] Williams, P., “Jacobi Pseudospectral Method for Solving Optimal Control Problems,”Journal of Guidance, Control, and Dynamics, Vol. 27, No. 2, March–April 2004,pp. 293–297.
[58] Williams, P., “Application of Pseudospectral methods for Receding Horizon Control,”Journal of Guidance, Control, and Dynamics, Vol. 27, No. 2, 2004, pp. 310–314.
[59] Williams, P., “Hermite-Legendre-Gauss-Lobatto Direct Transcription Methods inTrajectory Optimization,” Americal Astronautical Society, Spaceflight MechanicsMeeting, August 2005.
[60] Huntington, G. T., Advancement and Analysis of a Gauss Pseudospectral Tran-scription for Optimal Control , Ph.D. thesis, Massachusetts Institute of Technology,Cambridge, Massachusetts, 2007.
[61] Huntington, G. T., Benson, D. A., and Rao, A. V., “Optimal Configuration ofTetrahedral Spacecraft Formations,” The Journal of the Astronautical Sciences,Vol. 55, No. 2, April-June 2007, pp. 141–169.
[62] Huntington, G. T. and Rao, A. V., “Optimal Reconfiguration of SpacecraftFormations Using the Gauss Pseudospectral Method,” Journal of Guidance,Control, and Dynamics, Vol. 31, No. 3, May-June 2008, pp. 689–698.
[63] Fahroo, F. and Ross, I. M., “Direct Trajectory Optimization by a ChebyshevPseudospectral Method,” Journal of Guidance, Control, and Dynamics, Vol. 25,No. 1, 2002, pp. 160–166.
[64] Garg, D., Patterson, M. A., Hager, W. W., Rao, A. V., Benson, D. A., and Huntington,G. T., “A Unified Framework for the Numerical Solution of Optimal Control Problems
195
Using Pseudospectral Methods,” Automatica, Vol. 46, No. 11, December 2010,pp. 1843–1851.
[65] Garg, D., Hager, W. W., and Rao, A. V., “Pseudospectral Methods for SolvingInfinite-Horizon Optimal Control Problems,” Automatica, Vol. 47, No. 4, April 2011,pp. 829–837.
[66] Dontchev, A. L. and Hager, W. W., “The Euler Approximation in State ConstrainedOptimal Control,” Mathematics of Computation, Vol. 70, 2001, pp. 173–203.
[67] Maurer, H. and Gillessen, W., “Application of multiple shooting to the numericalsolution of optimal control problems with bounded state variables,” Computing,Vol. 15, 1975, pp. 105–126.
[68] Garg, D., Advances In Global Pseudospectral Methods For Optimal Control , Ph.D.thesis, University of Florida, Gainesville, Florida, 2011.
[69] Patterson, M. A. and Rao, A. V., “GPOPS-II: A MATLAB Software for SolvingMultiple-Phase Optimal Control Problems Using hp-Adaptive Gaussian QuadratureCollocation Methods and Sparse Nonlinear Programming,” ACM Transactions onMathematical Software, February 2013, Submitted.
[70] Patterson, M. A., Hager, W. W., and Rao, A. V., “A ph-Collocation Scheme forOptimal Control,” Automatica, January 2013, Submitted.
[71] Vlases, W. G., Paris, S. W., Lajoie, R. M., Martens, M. J., and Hargraves, C. R.,“Optimal Trajectories by Implicit Simulation,” Tech. Rep. WRDC-TR-90-3056,Boeing Aerospace and Electronics, Wright-Patterson Air Force Base, Ohio, 1990.
[72] Falugi, P., Kerrigan, E., and van Wyk, E., Imperial College London Optimal ControlSoftware User Guide (ICLOCS), Department of Electrical Engineering, ImperialCollege London, London, UK, May 2010.
[73] Houska, B., Ferreau, H. J., and Diehl, M., “ACADO Toolkit–An Open-SourceFramework for Automatic Control and Dynamic Optimization,” Optimal ControlApplications and Methods, Vol. 32, No. 3, 2011, pp. 298–312.
[74] Darby, C. L., hp-Pseudospectral Method For Solving Continuous-Time NonlinearOptimal Control Problems, Ph.D. thesis, University of Florida, Gainesville, Florida,2011.
196
BIOGRAPHICAL SKETCH
Camila Francolin was born in Rio de Janeiro, Brazil. She received her dual Bachelor
of Science degrees in aerospace and mechanical engineering in December 2007
from the University of Florida. She then received her Master of Science degree in
aerospace engineering in May 2010, and her Doctor of Philosophy in aerospace
engineering in August 2013 from the University of Florida. Her research interests include
numerical approximations to differential equations, optimal control theory, and numerical
approximations to the solution of optimal control problems.
197
top related