theory, computation, and design 2nd editionjbraw/mpc/mpc-book-2... · 2020. 10. 9. · no part of...
TRANSCRIPT
-
Model Predictive Control:
Theory, Computation, and Design
2nd Edition
-
Model Predictive Control:
Theory, Computation, and Design
2nd Edition
James B. Rawlings
Department of Chemical Engineering
University of California
Santa Barbara, California, USA
David Q. Mayne
Department of Electrical and Electronic Engineering
Imperial College London
London, England
Moritz M. Diehl
Department of Microsystems Engineering and
Department of Mathematics
University of Freiburg
Freiburg, Germany
DNob Hill Publishing
Santa Barbara, California
-
This book was set in Lucida using LATEX, and printed and bound by
Worzalla.
Cover design by Cheryl M. and James B. Rawlings.
Copyright © 2020 by Nob Hill Publishing, LLC
All rights reserved.
Nob Hill Publishing, LLC
Cheryl M. Rawlings, publisher
Santa Barbara, CA 93101
http://www.nobhillpublishing.com
No part of this book may be reproduced, in any form or by any means,
without permission in writing from the publisher.
Library of Congress Control Number: 2020942771
Printed in the United States of America.
First Edition
First Printing August 2009
Electronic Download (1st) November 2013
Electronic Download (2nd) April 2014
Electronic Download (3rd) July 2014
Electronic Download (4th) October 2014
Electronic Download (5th) February 2015
Second Edition
First Printing October 2017
Electronic Download (1st) October 2018
Electronic Download (2nd) February 2019
Paperback Edition
Third Printing October 2020
Electronic Download (3rd) October 2020
-
To Cheryl, Josephine, and Stephanie,
for their love, encouragement, and patience.
-
Preface to the Second Edition
In the eight years since the publication of the ®rst edition, the ®eld
of model predictive control (MPC) has seen tremendous progress. First
and foremost, the algorithms and high-level software available for solv-
ing challenging nonlinear optimal control problems have advanced sig-
ni®cantly. For this reason, we have added a new chapter, Chapter 8,
ªNumerical Optimal Control,º and coauthor, Professor Moritz M. Diehl.
This chapter gives an introduction into methods for the numerical so-
lution of the MPC optimization problem. Numerical optimal control
builds on two ®elds: simulation of differential equations, and numeri-
cal optimization. Simulation is often covered in undergraduate courses
and is therefore only brie¯y reviewed. Optimization is treated in much
more detail, covering topics such as derivative computations, Hessian
approximations, and handling inequalities. Most importantly, the chap-
ter presents some of the many ways that the speci®c structure of opti-
mal control problems arising in MPC can be exploited algorithmically.
We have also added a software release with the second edition of
the text. The software enables the solution of all of the examples and
exercises in the text requiring numerical calculation. The software is
based on the freely available CasADi language, and a high-level set of
Octave/MATLAB functions, MPCTools, to serve as an interface to CasADi.
These tools have been tested in several MPC short courses to audiences
composed of researchers and practitioners. The software can be down-
loaded from www.chemengr.ucsb.edu/~jbraw/mpc.
In Chapter 2, we have added sections covering the following topics:
• economic MPC
• MPC with discrete actuators
We also present a more recent form of suboptimal MPC that is prov-
ably robust as well as computationally tractable for online solution of
nonconvex MPC problems.
In Chapter 3, we have added a discussion of stochastic MPC, which
has received considerable recent research attention.
In Chapter 4, we have added a new treatment of state estimation
with persistent, bounded process and measurement disturbances. We
have also removed the discussion of particle ®ltering. There are two
vi
-
vii
reasons for this removal; ®rst, we wanted to maintain a manageable
total length of the text; second, all of the available sampling strate-
gies in particle ®ltering come up against the ªcurse of dimensionality,º
which renders the state estimates inaccurate for dimension higher than
about ®ve. The material on particle ®ltering remains available on the
text website.
In Chapter 6, we have added a new section for distributed MPC of
nonlinear systems.
In Chapter 7, we have added the software to compute the critical
regions in explicit MPC.
Throughout the text, we support the stronger KL-de®nition of asymp-
totic stability, in place of the classical de®nition used in the ®rst edition.
The most signi®cant notational change is to denote a sequence with
a; b; c; : : : instead of with fa;b; c; : : :g as in the ®rst edition.
JBR
Madison, Wis., USA
DQM
London, England
MMD
Freiburg, Germany
Added for the second edition, third printing
The second edition, ®rst printing was made available electronically in
October 2018. The February 2019 second (electronic only) printing
mainly corrected typographical errors. This third printing was printed
as a paperback and made available electronically in October 2020.
In this third printing, besides removing typographical and other er-
rors, Chapter 4 was revised signi®cantly. The analysis of Moving Hori-
zon Estimation and Full Information Estimation with bounded distur-
bances has improved signi®cantly in the last several years due to the
research efforts of several groups. We have attempted to bring the
material in Chapter 4 up to date with this current literature.
Moreover, the section in Chapter 3 on Stochastic MPC was updated,
and a new section on discrete actuators was added to Chapter 8.
JBR
Santa Barbara, CA, USA
DQM
London, England
MMD
Freiburg, Germany
-
Preface
Our goal in this text is to provide a comprehensive and foundational
treatment of the theory and design of model predictive control (MPC).
By now several excellent monographs emphasizing various aspects of
MPC have appeared (a list appears at the beginning of Chapter 1, and
the reader may naturally wonder what is offered here that is new and
different. By providing a comprehensive treatment of the MPC foun-
dation, we hope that this text enables researchers to learn and teach
the fundamentals of MPC without continuously searching the diverse
control research literature for omitted arguments and requisite back-
ground material. When teaching the subject, it is essential to have a
collection of exercises that enables the students to assess their level of
comprehension andmastery of the topics. To support the teaching and
learning of MPC, we have included more than 200 end-of-chapter exer-
cises. A complete solution manual (more than 300 pages) is available
for course instructors.
Chapter 1 is introductory. It is intended for graduate students in en-
gineering who have not yet had a systems course. But it serves a second
purpose for those who have already taken the ®rst graduate systems
course. It derives all the results of the linear quadratic regulator and
optimal Kalman ®lter using only those arguments that extend to the
nonlinear and constrained cases to be covered in the later chapters.
Instructors may ®nd that this tailored treatment of the introductory
systems material serves both as a review and a preview of arguments
to come in the later chapters.
Chapters 2±4 are foundational and should probably be covered in
any graduate level MPC course. Chapter 2 covers regulation to the ori-
gin for nonlinear and constrained systems. This material presents in a
uni®ed fashion many of the major research advances in MPC that took
place during the last 20 years. It also includes more recent topics such
as regulation to an unreachable setpoint that are only now appearing in
the research literature. Chapter 3 addressesMPCdesign for robustness,
with a focus on MPC using tubes or bundles of trajectories in place of
the single nominal trajectory. This chapter again uni®es a large body of
research literature concerned with robust MPC. Chapter 4 covers state
estimation with an emphasis on moving horizon estimation, but also
viii
-
ix
covers extended and unscented Kalman ®ltering, and particle ®ltering.
Chapters 5±7 present more specialized topics. Chapter 5 addresses
the special requirements of MPC based on output measurement instead
of state measurement. Chapter 6 discusses how to design distributed
MPC controllers for large-scale systems that are decomposed intomany
smaller, interacting subsystems. Chapter 7 covers the explicit optimal
control of constrained linear systems. The choice of coverage of these
three chapters may vary depending on the instructor's or student's own
research interests.
Three appendices are included, again, so that the reader is not sent
off to search a large research literature for the fundamental arguments
used in the text. Appendix A covers the required mathematical back-
ground. Appendix B summarizes the results used for stability analysis
including the various types of stability and Lyapunov function theory.
Since MPC is an optimization-based controller, Appendix C covers the
relevant results from optimization theory. In order to reduce the size
and expense of the text, the three appendices are available on the web:
www.chemengr.ucsb.edu/~jbraw/mpc. Note, however, that all mate-
rial in the appendices is included in the book's printed table of contents,
and subject and author indices. The website also includes sample ex-
ams, and homework assignments for a one-semester graduate course
in MPC. All of the examples and exercises in the text were solved with
Octave. Octave is freely available from www.octave.org.
JBR DQM
Madison, Wisconsin, USA London, England
-
Acknowledgments
Both authors would like to thank the Department of Chemical and Bio-
logical Engineering of the University of Wisconsin for hosting DQM's
visits to Madison during the preparation of this monograph. Funding
from the Paul A. Elfers Professorship provided generous ®nancial sup-
port.
JBR would like to acknowledge the graduate students with whom
he has had the privilege to work on model predictive control topics:
Rishi Amrit, Dennis BonnÂe, John Campbell, John Eaton, Peter Findeisen,
Rolf Findeisen, Eric Haseltine, John Jùrgensen, Nabil Laachi, Scott Mead-
ows, Scott Middlebrooks, Steve Miller, Ken Muske, Brian Odelson, Mu-
rali Rajamani, Chris Rao, Brett Stewart, Kaushik Subramanian, Aswin
Venkat, and Jenny Wang. He would also like to thank many colleagues
with whom he has collaborated on this subject: Frank AllgÈower, Tom
Badgwell, Bhavik Bakshi, Don Bartusiak, Larry Biegler, Moritz Diehl,
Jim Downs, Tom Edgar, Brian Froisy, Ravi Gudi, Sten Bay Jùrgensen,
Jay Lee, Fernando Lima, Wolfgang Marquardt, Gabriele Pannocchia, Joe
Qin, Harmon Ray, Pierre Scokaert, Sigurd Skogestad, Tyler Soderstrom,
Steve Wright, and Robert Young.
DQM would like to thank his colleagues at Imperial College, espe-
cially Richard Vinter and Martin Clark, for providing a stimulating and
congenial research environment. He is very grateful to Lucien Polak
and Graham Goodwin with whom he has collaborated extensively and
fruitfully over many years; he would also like to thank many other col-
leagues, especially Karl ÊAstrÈom, Roger Brockett, Larry Ho, Petar Koko-
tovic, and Art Krener, from whom he has learned much. He is grateful
to past students who have worked with him on model predictive con-
trol: Ioannis Chrysochoos, Wilbur Langson, Hannah Michalska, Sasa
RakoviÂc, and Warren Schroeder; Hannah Michalska and Sasa RakoviÂc,
in particular, contributed very substantially. He owes much to these
past students, now colleagues, as well as to Frank AllgÈower, Rolf Find-
eisen Eric Kerrigan, Konstantinos Kouramus, Chris Rao, Pierre Scokaert,
and Maria Seron for their collaborative research in MPC.
Both authors would especially like to thank Tom Badgwell, Bob Bird,
Eric Kerrigan, Ken Muske, Gabriele Pannocchia, and Maria Seron for
their careful and helpful reading of parts of themanuscript. John Eaton
x
-
xi
again deserves specialmention for his invaluable technical support dur-
ing the entire preparation of the manuscript.
Added for the second edition. JBR would like to acknowledge the
most recent generation of graduate students with whom he has had the
privilege to work on model predictive control research topics: Doug Al-
lan, Travis Arnold, Cuyler Bates, Luo Ji, Nishith Patel, Michael Risbeck,
and Megan Zagrobelny.
In preparing the second edition, and, in particular, the software re-
lease, the current group of graduate students far exceeded expectations
to help ®nish the project. Quite simply, the project could not have been
completed in a timely fashion without their generosity, enthusiasm,
professionalism, and sel¯ess contribution. Michael Risbeck deserves
special mention for creating the MPCTools interface to CasADi, and
updating and revising the tools used to create the website to distribute
the text- and software-supporting materials. He also wrote code to cal-
culate explicit MPC control laws in Chapter 7. Nishith Patel made a
major contribution to the subject index, and Doug Allan contributed
generously to the presentation of moving horizon estimation in Chap-
ter 4.
A research leave for JBR in Fall 2016, again funded by the Paul A.
Elfers Professorship, was instrumental in freeing up time to complete
the revision of the text and further develop computational exercises.
MMD wants to especially thank Jesus Lago Garcia, Jochem De Schut-
ter, Andrea Zanelli, Dimitris Kouzoupis, Joris Gillis, Joel Andersson,
and Robin Verschueren for help with the preparation of exercises and
examples in Chapter 8; and also wants to acknowledge the following
current and former team members that contributed to research and
teaching on optimal and model predictive control at the Universities of
Leuven and Freiburg: Adrian BÈurger, Hans Joachim Ferreau, JÈorg Fis-
cher, Janick Frasch, Gianluca Frison, Niels Haverbeke, Greg Horn, Boris
Houska, Jonas Koenemann, Attila Kozma, Vyacheslav Kungurtsev, Gio-
vanni Licitra, Rien Quirynen, Carlo Savorgnan, Quoc Tran-Dinh, Milan
Vukov, and Mario Zanon. MMD also wants to thank Frank AllgÈower, Al-
berto Bemporad, Rolf Findeisen, Larry Biegler, Hans Georg Bock, Stephen
Boyd, SÂebastien Gros, Lars GrÈune, Colin Jones, John Bagterp Jùrgensen,
Christian Kirches, Daniel Leineweber, Katja Mombaur, Yurii Nesterov,
Toshiyuki Ohtsuka, Goele Pipeleers, Andreas Potschka, Sebastian Sager,
Johannes P. SchlÈoder, Volker Schulz, Marc Steinbach, Jan Swevers, Phil-
ippe Toint, Andrea Walther, Stephen Wright, Joos Vandewalle, and Ste-
fan Vandewalle for inspiring discussions on numerical optimal control
-
xii
methods and their presentation during the last 20 years.
All three authors would especially like to thank Joel Andersson and
Joris Gillis for having developed CasADi and for continuing its support,
and for having helped to improve some of the exercises in the text.
Added for the second edition, third printing. The authors would
like to acknowledge and thank Doug Allan again for his suggestions
and help with the revision of Chapter 4. Much of the new material
on Full Information Estimation and Moving Horizon Estimation is a di-
rect result of Doug's research papers and 2020 PhD thesis on state
estimation. Koty McAllister provided expert assistance in the update
of stochastic MPC in Chapter 3. Finally, Adrian Buerger and Pratyush
Kumar provided valuable assistance on the addition of the discrete ac-
tuator numerics to Chapter 8.
-
Contents
1 Getting Started with Model Predictive Control 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Models and Modeling . . . . . . . . . . . . . . . . . . . . . . 1
1.2.1 Linear Dynamic Models . . . . . . . . . . . . . . . . . 2
1.2.2 Input-Output Models . . . . . . . . . . . . . . . . . . 3
1.2.3 Distributed Models . . . . . . . . . . . . . . . . . . . 4
1.2.4 Discrete Time Models . . . . . . . . . . . . . . . . . . 5
1.2.5 Constraints . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.6 Deterministic and Stochastic . . . . . . . . . . . . . 9
1.3 Introductory MPC Regulator . . . . . . . . . . . . . . . . . . 11
1.3.1 Linear Quadratic Problem . . . . . . . . . . . . . . . 11
1.3.2 Optimizing Multistage Functions . . . . . . . . . . . 12
1.3.3 Dynamic Programming Solution . . . . . . . . . . . 18
1.3.4 The In®nite Horizon LQ Problem . . . . . . . . . . . 21
1.3.5 Controllability . . . . . . . . . . . . . . . . . . . . . . 23
1.3.6 Convergence of the Linear Quadratic Regulator . . 24
1.4 Introductory State Estimation . . . . . . . . . . . . . . . . . 26
1.4.1 Linear Systems and Normal Distributions . . . . . 27
1.4.2 Linear Optimal State Estimation . . . . . . . . . . . 29
1.4.3 Least Squares Estimation . . . . . . . . . . . . . . . 33
1.4.4 Moving Horizon Estimation . . . . . . . . . . . . . . 39
1.4.5 Observability . . . . . . . . . . . . . . . . . . . . . . . 41
1.4.6 Convergence of the State Estimator . . . . . . . . . 43
1.5 Tracking, Disturbances, and Zero Offset . . . . . . . . . . 46
1.5.1 Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . 46
1.5.2 Disturbances and Zero Offset . . . . . . . . . . . . . 49
1.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2 Model Predictive ControlÐRegulation 89
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
2.2 Model Predictive Control . . . . . . . . . . . . . . . . . . . . 91
2.3 Dynamic Programming Solution . . . . . . . . . . . . . . . 107
2.4 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
2.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 112
xiii
-
xiv Contents
2.4.2 Stabilizing Conditions . . . . . . . . . . . . . . . . . 114
2.4.3 Exponential Stability . . . . . . . . . . . . . . . . . . 120
2.4.4 Controllability and Observability . . . . . . . . . . . 120
2.4.5 Time-Varying Systems . . . . . . . . . . . . . . . . . 123
2.5 Examples of MPC . . . . . . . . . . . . . . . . . . . . . . . . . 131
2.5.1 The Unconstrained Linear Quadratic Regulator . . 132
2.5.2 Unconstrained Linear Periodic Systems . . . . . . . 133
2.5.3 Stable Linear Systems with Control Constraints . 135
2.5.4 Linear Systems with Control and State Constraints 136
2.5.5 Constrained Nonlinear Systems . . . . . . . . . . . 139
2.5.6 Constrained Nonlinear Time-Varying Systems . . . 141
2.6 Is a Terminal Constraint Set Xf Necessary? . . . . . . . . 144
2.7 Suboptimal MPC . . . . . . . . . . . . . . . . . . . . . . . . . 147
2.7.1 Extended State . . . . . . . . . . . . . . . . . . . . . . 150
2.7.2 Asymptotic Stability of Difference Inclusions . . . 150
2.8 Economic Model Predictive Control . . . . . . . . . . . . . 153
2.8.1 Asymptotic Average Performance . . . . . . . . . . 155
2.8.2 Dissipativity and Asymptotic Stability . . . . . . . 156
2.9 Discrete Actuators . . . . . . . . . . . . . . . . . . . . . . . . 160
2.10 Concluding Comments . . . . . . . . . . . . . . . . . . . . . 163
2.11 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
2.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
3 Robust and Stochastic Model Predictive Control 193
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
3.1.1 Types of Uncertainty . . . . . . . . . . . . . . . . . . 193
3.1.2 Feedback Versus Open-Loop Control . . . . . . . . 195
3.1.3 Robust and Stochastic MPC . . . . . . . . . . . . . . 200
3.1.4 Tubes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
3.1.5 Difference InclusionDescription of Uncertain Sys-
tems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
3.2 Nominal (Inherent ) Robustness . . . . . . . . . . . . . . . . 204
3.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 204
3.2.2 Difference Inclusion Description of Discontinu-
ous Systems . . . . . . . . . . . . . . . . . . . . . . . 206
3.2.3 When Is Nominal MPC Robust? . . . . . . . . . . . . 207
3.2.4 Robustness of Nominal MPC . . . . . . . . . . . . . 209
3.3 Min-Max Optimal Control: Dynamic Programming Solution 214
3.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 214
3.3.2 Properties of the Dynamic Programming Solution 216
-
Contents xv
3.4 Robust Min-Max MPC . . . . . . . . . . . . . . . . . . . . . . 220
3.5 Tube-Based Robust MPC . . . . . . . . . . . . . . . . . . . . 223
3.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 223
3.5.2 Outer-Bounding Tube for a Linear System with
Additive Disturbance . . . . . . . . . . . . . . . . . . 224
3.5.3 Tube-BasedMPC of Linear Systemswith Additive
Disturbances . . . . . . . . . . . . . . . . . . . . . . . 228
3.5.4 Improved Tube-BasedMPC of Linear Systemswith
Additive Disturbances . . . . . . . . . . . . . . . . . 234
3.6 Tube-Based MPC of Nonlinear Systems . . . . . . . . . . . 236
3.6.1 The Nominal Trajectory . . . . . . . . . . . . . . . . 238
3.6.2 Model Predictive Controller . . . . . . . . . . . . . . 238
3.6.3 Choosing the Nominal Constraint Sets ÅU and ÅX . . 242
3.7 Stochastic MPC . . . . . . . . . . . . . . . . . . . . . . . . . . 246
3.7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 246
3.7.2 Stability of Stochastic MPC . . . . . . . . . . . . . . 248
3.7.3 Tube-based stochastic MPC . . . . . . . . . . . . . . 250
3.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
3.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
4 State Estimation 269
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
4.2 Full Information Estimation . . . . . . . . . . . . . . . . . . 269
4.2.1 Nominal Estimator Stability . . . . . . . . . . . . . . 279
4.2.2 Robust Estimator Stability . . . . . . . . . . . . . . . 284
4.2.3 InterludeÐLinear System Review . . . . . . . . . . . 287
4.3 Moving Horizon Estimation . . . . . . . . . . . . . . . . . . 292
4.3.1 Zero Prior Weighting . . . . . . . . . . . . . . . . . . 293
4.3.2 Nonzero Prior Weighting . . . . . . . . . . . . . . . . 296
4.3.3 RGES of MHE under exponential assumptions . . . 297
4.4 Other Nonlinear State Estimators . . . . . . . . . . . . . . . 302
4.4.1 Particle Filtering . . . . . . . . . . . . . . . . . . . . . 302
4.4.2 Extended Kalman Filtering . . . . . . . . . . . . . . . 302
4.4.3 Unscented Kalman Filtering . . . . . . . . . . . . . . 304
4.4.4 EKF, UKF, and MHE Comparison . . . . . . . . . . . 306
4.5 On combining MHE and MPC . . . . . . . . . . . . . . . . . 312
4.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
4.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
-
xvi Contents
5 Output Model Predictive Control 333
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
5.2 A Method for Output MPC . . . . . . . . . . . . . . . . . . . 335
5.3 Linear Constrained Systems: Time-Invariant Case . . . . 338
5.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 338
5.3.2 State Estimator . . . . . . . . . . . . . . . . . . . . . . 338
5.3.3 Controlling Ãx . . . . . . . . . . . . . . . . . . . . . . . 340
5.3.4 Output MPC . . . . . . . . . . . . . . . . . . . . . . . . 342
5.3.5 Computing the Tightened Constraints . . . . . . . 346
5.4 Linear Constrained Systems: Time-Varying Case . . . . . 347
5.5 Offset-Free MPC . . . . . . . . . . . . . . . . . . . . . . . . . 347
5.5.1 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 349
5.5.2 Control . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
5.5.3 Convergence Analysis . . . . . . . . . . . . . . . . . 354
5.6 Nonlinear Constrained Systems . . . . . . . . . . . . . . . . 357
5.7 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
5.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
6 Distributed Model Predictive Control 363
6.1 Introduction and Preliminary Results . . . . . . . . . . . . 363
6.1.1 Least Squares Solution . . . . . . . . . . . . . . . . . 364
6.1.2 Stability of Suboptimal MPC . . . . . . . . . . . . . . 369
6.2 Unconstrained Two-Player Game . . . . . . . . . . . . . . . 374
6.2.1 Centralized Control . . . . . . . . . . . . . . . . . . . 376
6.2.2 Decentralized Control . . . . . . . . . . . . . . . . . 377
6.2.3 Noncooperative Game . . . . . . . . . . . . . . . . . 378
6.2.4 Cooperative Game . . . . . . . . . . . . . . . . . . . . 386
6.2.5 Tracking Nonzero Setpoints . . . . . . . . . . . . . . 392
6.2.6 State Estimation . . . . . . . . . . . . . . . . . . . . . 399
6.3 Constrained Two-Player Game . . . . . . . . . . . . . . . . 400
6.3.1 Uncoupled Input Constraints . . . . . . . . . . . . . 402
6.3.2 Coupled Input Constraints . . . . . . . . . . . . . . 405
6.3.3 Exponential Convergence with Estimate Error . . . 407
6.3.4 Disturbance Models and Zero Offset . . . . . . . . 409
6.4 Constrained M-Player Game . . . . . . . . . . . . . . . . . . 413
6.5 Nonlinear Distributed MPC . . . . . . . . . . . . . . . . . . . 415
6.5.1 Nonconvexity . . . . . . . . . . . . . . . . . . . . . . . 415
6.5.2 Distributed Algorithm for Nonconvex Functions . 417
6.5.3 Distributed Nonlinear Cooperative Control . . . . 419
6.5.4 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . 422
-
Contents xvii
6.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
6.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
7 Explicit Control Laws for Constrained Linear Systems 445
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
7.2 Parametric Programming . . . . . . . . . . . . . . . . . . . . 446
7.3 Parametric Quadratic Programming . . . . . . . . . . . . . 451
7.3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . 451
7.3.2 Preview . . . . . . . . . . . . . . . . . . . . . . . . . . 452
7.3.3 Optimality Condition for a Convex Program . . . . 453
7.3.4 Solution of the Parametric Quadratic Program . . 456
7.3.5 Continuity of V0� and u0� . . . . . . . . . . . . 4607.4 Constrained Linear Quadratic Control . . . . . . . . . . . . 461
7.5 Parametric Piecewise Quadratic Programming . . . . . . . 463
7.6 DP Solution of the Constrained LQ Control Problem . . . 469
7.7 Parametric Linear Programming . . . . . . . . . . . . . . . 470
7.7.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . 470
7.7.2 Minimizer u0x is Unique for all x 2 X . . . . . . 4727.8 Constrained Linear Control . . . . . . . . . . . . . . . . . . 475
7.9 Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . 476
7.10 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477
7.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478
8 Numerical Optimal Control 485
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
8.1.1 Discrete Time Optimal Control Problem . . . . . . 486
8.1.2 Convex Versus Nonconvex Optimization . . . . . . 487
8.1.3 Simultaneous Versus Sequential Optimal Control 490
8.1.4 Continuous Time Optimal Control Problem . . . . 492
8.2 Numerical Simulation . . . . . . . . . . . . . . . . . . . . . . 495
8.2.1 Explicit Runge-Kutta Methods . . . . . . . . . . . . . 496
8.2.2 Stiff Equations and Implicit Integrators . . . . . . . 500
8.2.3 Implicit Runge-Kutta and Collocation Methods . . 501
8.2.4 Differential Algebraic Equations . . . . . . . . . . . 505
8.2.5 Integrator Adaptivity . . . . . . . . . . . . . . . . . . 507
8.3 Solving Nonlinear Equation Systems . . . . . . . . . . . . . 507
8.3.1 Linear Systems . . . . . . . . . . . . . . . . . . . . . . 507
8.3.2 Nonlinear Root-Finding Problems . . . . . . . . . . 508
8.3.3 Local Convergence of Newton-Type Methods . . . 511
8.3.4 Af®ne Invariance . . . . . . . . . . . . . . . . . . . . . 513
8.3.5 Globalization for Newton-Type Methods . . . . . . 513
-
xviii Contents
8.4 Computing Derivatives . . . . . . . . . . . . . . . . . . . . . 514
8.4.1 Numerical Differentiation . . . . . . . . . . . . . . . 515
8.4.2 Algorithmic Differentiation . . . . . . . . . . . . . . 516
8.4.3 Implicit Function Interpretation . . . . . . . . . . . 517
8.4.4 Algorithmic Differentiation in Forward Mode . . . 520
8.4.5 Algorithmic Differentiation in Reverse Mode . . . 522
8.4.6 Differentiation of Simulation Routines . . . . . . . 525
8.4.7 Algorithmic and Symbolic Differentiation Software 527
8.4.8 CasADi for Optimization . . . . . . . . . . . . . . . . 527
8.5 Direct Optimal Control Parameterizations . . . . . . . . . 530
8.5.1 Direct Single Shooting . . . . . . . . . . . . . . . . . 532
8.5.2 Direct Multiple Shooting . . . . . . . . . . . . . . . . 534
8.5.3 Direct Transcription and Collocation Methods . . 538
8.6 Nonlinear Optimization . . . . . . . . . . . . . . . . . . . . . 542
8.6.1 Optimality Conditions and Perturbation Analysis 543
8.6.2 Nonlinear Optimization with Equalities . . . . . . . 546
8.6.3 Hessian Approximations . . . . . . . . . . . . . . . . 547
8.7 Newton-Type Optimization with Inequalities . . . . . . . 550
8.7.1 Sequential Quadratic Programming . . . . . . . . . 551
8.7.2 Nonlinear Interior Point Methods . . . . . . . . . . 552
8.7.3 Comparison of SQP and Nonlinear IP Methods . . 554
8.8 Structure in Discrete Time Optimal Control . . . . . . . . 555
8.8.1 Simultaneous Approach . . . . . . . . . . . . . . . . 556
8.8.2 Linear Quadratic Problems (LQP) . . . . . . . . . . . 558
8.8.3 LQP Solution by Riccati Recursion . . . . . . . . . . 558
8.8.4 LQP Solution by Condensing . . . . . . . . . . . . . 560
8.8.5 Sequential Approaches and Sparsity Exploitation 562
8.8.6 Differential Dynamic Programming . . . . . . . . . 564
8.8.7 Additional Constraints in Optimal Control . . . . . 566
8.9 Online Optimization Algorithms . . . . . . . . . . . . . . . 567
8.9.1 General Algorithmic Considerations . . . . . . . . . 568
8.9.2 Continuation Methods and Real-Time Iterations . 571
8.10 Discrete Actuators . . . . . . . . . . . . . . . . . . . . . . . . 574
8.11 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579
8.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581
Author Index 600
Citation Index 608
Subject Index 614
-
Contents xix
A Mathematical Background 624
A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624
A.2 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 624
A.3 Range and Nullspace of Matrices . . . . . . . . . . . . . . . 624
A.4 Linear Equations Ð Existence and Uniqueness . . . . . . 625
A.5 Pseudo-Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . 625
A.6 Partitioned Matrix Inversion Theorem . . . . . . . . . . . . 628
A.7 Quadratic Forms . . . . . . . . . . . . . . . . . . . . . . . . . 629
A.8 Norms in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
A.9 Sets in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
A.10Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 632
A.11Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 633
A.12Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636
A.13Convex Sets and Functions . . . . . . . . . . . . . . . . . . . 641
A.13.1Convex Sets . . . . . . . . . . . . . . . . . . . . . . . . 641
A.13.2Convex Functions . . . . . . . . . . . . . . . . . . . . 646
A.14Differential Equations . . . . . . . . . . . . . . . . . . . . . . 648
A.15Random Variables and the Probability Density . . . . . . 654
A.16Multivariate Density Functions . . . . . . . . . . . . . . . . 659
A.16.1Statistical Independence and Correlation . . . . . . 668
A.17Conditional Probability and Bayes's Theorem . . . . . . . 672
A.18Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 678
B Stability Theory 693
B.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693
B.2 Stability and Asymptotic Stability . . . . . . . . . . . . . . 696
B.3 Lyapunov Stability Theory . . . . . . . . . . . . . . . . . . . 700
B.3.1 Time-Invariant Systems . . . . . . . . . . . . . . . . 701
B.3.2 Time-Varying, Constrained Systems . . . . . . . . . 707
B.3.3 Upper boundingK functions . . . . . . . . . . . . . 709B.4 Robust Stability . . . . . . . . . . . . . . . . . . . . . . . . . 709
B.4.1 Nominal Robustness . . . . . . . . . . . . . . . . . . 709
B.4.2 Robustness . . . . . . . . . . . . . . . . . . . . . . . . 711
B.5 Control Lyapunov Functions . . . . . . . . . . . . . . . . . . 713
B.6 Input-to-State Stability . . . . . . . . . . . . . . . . . . . . . 717
B.7 Output-to-State Stability and Detectability . . . . . . . . . 719
B.8 Input/Output-to-State Stability . . . . . . . . . . . . . . . . 720
B.9 Incremental-Input/Output-to-State Stability . . . . . . . . 722
B.10 Observability . . . . . . . . . . . . . . . . . . . . . . . . . . . 722
B.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 724
-
xx Contents
C Optimization 729
C.1 Dynamic Programming . . . . . . . . . . . . . . . . . . . . . 729
C.1.1 Optimal Control Problem . . . . . . . . . . . . . . . 731
C.1.2 Dynamic Programming . . . . . . . . . . . . . . . . . 733
C.2 Optimality Conditions . . . . . . . . . . . . . . . . . . . . . 737
C.2.1 Tangent and Normal Cones . . . . . . . . . . . . . . 737
C.2.2 Convex Optimization Problems . . . . . . . . . . . . 741
C.2.3 Convex Problems: Polyhedral Constraint Set . . . 743
C.2.4 Nonconvex Problems . . . . . . . . . . . . . . . . . . 745
C.2.5 Tangent and Normal Cones . . . . . . . . . . . . . . 746
C.2.6 Constraint Set De®ned by Inequalities . . . . . . . 750
C.2.7 Constraint Set; Equalities and Inequalities . . . . . 753
C.3 Set-Valued Functions and Continuity of Value Function . 755
C.3.1 Outer and Inner Semicontinuity . . . . . . . . . . . 757
C.3.2 Continuity of the Value Function . . . . . . . . . . . 759
C.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767
-
List of Figures
1.1 System with input u, output y , and transfer function ma-
trix G connecting them; the model is y Gu. . . . . . . . . 31.2 Typical input constraint sets U for (a) continuous actua-
tors and (b) mixed continuous/discrete actuators. . . . . . 9
1.3 Output of a stochastic system versus time. . . . . . . . . . . 10
1.4 Two quadratic functions and their sum. . . . . . . . . . . . . 15
1.5 Schematic of the moving horizon estimation problem. . . . 39
1.6 MPC controller consisting of: receding horizon regulator,
state estimator, and target selector. . . . . . . . . . . . . . . 52
1.7 Schematic of the well-stirred reactor. . . . . . . . . . . . . . . 54
1.8 Three measured outputs versus time after a step change
in inlet ¯owrate at 10 minutes; nd 2. . . . . . . . . . . . . 571.9 Two manipulated inputs versus time after a step change
in inlet ¯owrate at 10 minutes; nd 2. . . . . . . . . . . . . 571.10 Three measured outputs versus time after a step change
in inlet ¯owrate at 10 minutes; nd 3. . . . . . . . . . . . . 581.11 Two manipulated inputs versus time after a step change
in inlet ¯owrate at 10 minutes; nd 3. . . . . . . . . . . . . 591.12 Plug-¯ow reactor. . . . . . . . . . . . . . . . . . . . . . . . . . . 60
1.13 Pendulum with applied torque. . . . . . . . . . . . . . . . . . 62
1.14 Feedback control system with output disturbance d, and
setpoint ysp. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
2.1 Example of MPC. . . . . . . . . . . . . . . . . . . . . . . . . . . 101
2.2 Feasible region U2, elliptical cost contours and ellipsecenter ax, and constrained minimizers for different
values of x. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
2.3 First element of control constraint set U3x (shaded)and control law �3x (line) versus x cos�; sin�,� 2 ��;� on the unit circle for a nonlinear systemwithterminal constraint. . . . . . . . . . . . . . . . . . . . . . . . . 106
2.4 Optimal cost V03 x versus x on the unit circle. . . . . . . . 107
xxi
-
xxii List of Figures
2.5 Closed-loop economic MPC versus tracking MPC starting
at x �8;8 with optimal steady state 8;4. Both con-trollers asymptotically stabilize the steady state. Dashed
contours show cost functions for each controller. . . . . . . 159
2.6 Closed-loop evolution under economic MPC. The rotated
cost function Ve 0 is a Lyapunov function for the system. . . 1602.7 Diagram of tank/cooler system. Each cooling unit can be
either on or off, and if on, it must be between its (possibly
nonzero) minimum and maximum capacities. . . . . . . . . 163
2.8 Feasible sets XN for two values of ÇQmin. Note that forÇQmin 9 (right-hand side), XN for N � 4 are discon-nected sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
2.9 Phase portrait for closed-loop evolution of cooler system
with ÇQmin 9. Line colors show value of discrete actuatoru2. 1652.10 Region of attraction (shaded region) for constrained MPC
controller of Exercise 2.6. . . . . . . . . . . . . . . . . . . . . . 174
2.11 The region Xf , in which the unconstrained LQR control
law is feasible for Exercise 2.7. . . . . . . . . . . . . . . . . . . 175
2.12 The region of attraction for terminal constraint xN 2Xf and terminal penalty Vf x 1=2x0�x and the es-timate of ÅXN for Exercise 2.8. . . . . . . . . . . . . . . . . . . 177
2.13 Inconsistent setpoint xsp; usp, unreachable stage cost
`x;u, and optimal steady states xs ; us, and stage costs
`sx;u for constrained and unconstrained systems. . . . 181
2.14 Stage cost versus time for the case of unreachable setpoint. 182
2.15 Rotated cost-function contour èx;u 0 (circles) for� 0;�8;�12. Shaded region shows feasible regionwhereèx;u < 0. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
3.1 Open-loop and feedback trajectories. . . . . . . . . . . . . . . 198
3.2 The sets XN , Rb, and Rc . . . . . . . . . . . . . . . . . . . . . . 2143.3 Outer-bounding tube Xz; Åu. . . . . . . . . . . . . . . . . . . . 228
3.4 Minimum feasible � for varying N. Note that we require
� 2 0;1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2323.5 Bounds on tightened constraint set ÅZ for varyingN. Bounds
are jx1j � �1, jx2j � �2, and juj � �. . . . . . . . . . . . . . . 2333.6 Comparison of 100 realizations of standard and tube-
based MPC for the chemical reactor example. . . . . . . . . 244
3.7 Comparison of standard and tube-based MPC with an ag-
gressive model predictive controller. . . . . . . . . . . . . . . 245
-
List of Figures xxiii
3.8 Concentration versus time for the ancillary model predic-
tive controller with sample time � 12 (left) and � 8(right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
3.9 Observed probability "test of constraint violation for i 10. Distribution is based on 500 trials for each value of
". Dashed line shows the outcome predicted by formula
(3.25), i.e., "test ". . . . . . . . . . . . . . . . . . . . . . . . . . 2553.10 Closed-loop robust MPC state evolution with uniformly
distributed jwj � 0:1 from four different x0. . . . . . . . . . 263
4.1 Smoothing update. . . . . . . . . . . . . . . . . . . . . . . . . . 299
4.2 Comparison of ®ltering and smoothing updates for the
batch reactor system. Second column shows absolute es-
timate error. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
4.3 Evolution of the state (solid line) and EKF state estimate
(dashed line). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
4.4 Evolution of the state (solid line) and UKF state estimate
(dashed line). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
4.5 Evolution of the state (solid line) and MHE state estimate
(dashed line). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
4.6 Perturbed trajectories terminating in Xf . . . . . . . . . . . . 315
4.7 Closed-loop performance of combined nonlinearMHE/MPC
with no disturbances. First column shows system states,
and second column shows estimation error. Dashed line
shows concentration setpoint. Vertical lines indicate times
of setpoint changes. . . . . . . . . . . . . . . . . . . . . . . . . 317
4.8 Closed-loop performance of combined nonlinearMHE/MPC
for varying disturbance size. The system is controlled be-
tween two steady states. . . . . . . . . . . . . . . . . . . . . . . 318
5.1 State estimator tube. The solid line Ãxt is the center of
the tube, and the dashed line is a sample trajectory of xt. 336
5.2 The system with disturbance. The state estimate lies in
the inner tube, and the state lies in the outer tube. . . . . . 337
6.1 Convex step from up1 ; u
p2 to u
p11 ; u
p12 . . . . . . . . . . 380
6.2 Ten iterations of noncooperative steady-state calculation. . 397
6.3 Ten iterations of cooperative steady-state calculation. . . . 397
6.4 Ten iterations of noncooperative steady-state calculation;
reversed pairing. . . . . . . . . . . . . . . . . . . . . . . . . . . 398
-
xxiv List of Figures
6.5 Ten iterations of cooperative steady-state calculation; re-
versed pairing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
6.6 Cooperative control stuck on the boundary of U under
coupled constraints . . . . . . . . . . . . . . . . . . . . . . . . 406
6.7 Cost contours for a two-player, nonconvex game. . . . . . . 416
6.8 Nonconvex function optimized with the distributed gra-
dient algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
6.9 Closed-loop state and control evolutionwith x10; x20 3;�3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
6.10 Contours of Vx0;u1;u2 for N 1. . . . . . . . . . . . . . 4266.11 Optimizing a quadratic function in one set of variables at
a time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
6.12 Constrained optimality conditions and the normal cone. . 439
7.1 The sets Z, X, and Ux. . . . . . . . . . . . . . . . . . . . . . 4487.2 Parametric linear program. . . . . . . . . . . . . . . . . . . . . 448
7.3 Unconstrained parametric quadratic program. . . . . . . . . 449
7.4 Parametric quadratic program. . . . . . . . . . . . . . . . . . . 449
7.5 Polar cone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
7.6 Regions Rx; x 2 X for a second-order example. . . . . . . . 4627.7 Solution to a parametric LP. . . . . . . . . . . . . . . . . . . . 473
7.8 Solution times for explicit and implicit MPC for N 20. . . 480
8.1 Feasible set and reduced objective u0 of the non-
linear MPC Example 8.1. . . . . . . . . . . . . . . . . . . . . . . 490
8.2 Performance of different integration methods. . . . . . . . . 499
8.3 Polynomial approximationxe1t and true trajectoryx1tof the ®rst state and its derivative. . . . . . . . . . . . . . . . 504
8.4 Performance of implicit integration methods on a stiff ODE. 506
8.5 Newton-type iterations for solution of Rz 0 from Ex-ample 8.5. Left: exact Newton method. Right: constant
Jacobian approximation. . . . . . . . . . . . . . . . . . . . . . 510
8.6 Convergence of different sequences as a function of k. . . 512
8.7 Relaxed and binary feasible solution for Example 8.17. . . 578
8.8 A hanging chain at rest. See Exercise 8.6(b). . . . . . . . . . 585
8.9 Direct single shooting solution for (8.65)without path con-
straints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 587
8.10 Open-loop simulation for (8.65) using collocation. . . . . . . 590
8.11 Gauss-Newton iterations for the direct multiple-shooting
method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 592
-
List of Figures xxv
A.1 The four fundamental subspaces of matrix A . . . . . . . . 626
A.2 Matrix A maps into RA. . . . . . . . . . . . . . . . . . . . . . 627A.3 Pseudo-inverse of A maps into RA0. . . . . . . . . . . . . . 627A.4 Subgradient. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640
A.5 Separating hyperplane. . . . . . . . . . . . . . . . . . . . . . . 642
A.6 Polar cone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645
A.7 A convex function. . . . . . . . . . . . . . . . . . . . . . . . . . 646
A.8 Normal distribution. . . . . . . . . . . . . . . . . . . . . . . . . 658
A.9 Multivariate normal in two dimensions. . . . . . . . . . . . . 660
A.10 The geometry of quadratic form x0Ax b. . . . . . . . . . . 661A.11 A nearly singular normal density in two dimensions. . . . . 665
A.12 The region Xc for y maxx1; x2 � c. . . . . . . . . . . . 667A.13 A joint density function for the two uncorrelated random
variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 670
A.14 The probability distribution and inverse distribution for
random variable �. . . . . . . . . . . . . . . . . . . . . . . . . . 687
B.1 Stability of the origin. . . . . . . . . . . . . . . . . . . . . . . . 697
B.2 An attractive but unstable origin. . . . . . . . . . . . . . . . . 698
C.1 Routing problem. . . . . . . . . . . . . . . . . . . . . . . . . . . 730
C.2 Approximation of the set U . . . . . . . . . . . . . . . . . . . . 738
C.3 Tangent cones. . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
C.4 Normal at u. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
C.5 Condition of optimality. . . . . . . . . . . . . . . . . . . . . . . 745
C.6 Tangent and normal cones. . . . . . . . . . . . . . . . . . . . . 747
C.7 Condition of optimality. . . . . . . . . . . . . . . . . . . . . . . 749
C.8 FUu 6 TUu. . . . . . . . . . . . . . . . . . . . . . . . . . . . 751C.9 Graph of set-valued function U�. . . . . . . . . . . . . . . . 756C.10 Graphs of discontinuous set-valued functions. . . . . . . . . 757
C.11 Outer and inner semicontinuity of U�. . . . . . . . . . . . . 758C.12 Subgradient of f�. . . . . . . . . . . . . . . . . . . . . . . . . 762
-
List of Examples and Statements
1.1 Example: Sum of quadratic functions . . . . . . . . . . . . . 15
1.2 Lemma: Hautus lemma for controllability . . . . . . . . . . . 24
1.3 Lemma: LQR convergence . . . . . . . . . . . . . . . . . . . . . 24
1.4 Lemma: Hautus lemma for observability . . . . . . . . . . . 42
1.5 Lemma: Convergence of estimator cost . . . . . . . . . . . . 43
1.6 Lemma: Estimator convergence . . . . . . . . . . . . . . . . . 44
1.7 Assumption: Target feasibility and uniqueness . . . . . . . 48
1.8 Lemma: Detectability of the augmented system . . . . . . . 50
1.9 Corollary: Dimension of the disturbance . . . . . . . . . . . 50
1.10 Lemma: Offset-free control . . . . . . . . . . . . . . . . . . . . 52
1.11 Example: More measured outputs than inputs and zero
offset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
1.12 Lemma: Hautus lemma for stabilizability . . . . . . . . . . . 68
1.13 Lemma: Hautus lemma for detectability . . . . . . . . . . . . 72
1.14 Lemma: Stabilizable systems and feasible targets . . . . . . 83
2.1 Proposition: Continuity of system solution . . . . . . . . . . 94
2.2 Assumption: Continuity of system and cost . . . . . . . . . 97
2.3 Assumption: Properties of constraint sets . . . . . . . . . . 98
2.4 Proposition: Existence of solution to optimal control problem 98
2.5 Example: Linear quadratic MPC . . . . . . . . . . . . . . . . . 99
2.6 Example: Closer inspection of linear quadratic MPC . . . . 101
2.7 Theorem: Continuity of value function and control law . . 104
2.8 Example: Discontinuous MPC control law . . . . . . . . . . . 105
2.9 De®nition: Positive and control invariant sets . . . . . . . . 109
2.10 Proposition: Existence of solutions to DP recursion . . . . . 110
2.11 De®nition: Asymptotically stable and GAS . . . . . . . . . . 112
2.12 De®nition: Lyapunov function . . . . . . . . . . . . . . . . . . 113
2.13 Theorem: Lyapunov stability theorem . . . . . . . . . . . . . 113
2.14 Assumption: Basic stability assumption . . . . . . . . . . . . 114
2.15 Proposition: The value function V0N� is locally bounded . 1152.16 Proposition: Extension of upper bound to XN . . . . . . . . 1152.17 Assumption: Weak controllability . . . . . . . . . . . . . . . . 116
2.18 Proposition: Monotonicity of the value function . . . . . . . 118
2.19 Theorem: Asymptotic stability of the origin . . . . . . . . . 119
xxvii
-
xxviii List of Examples and Statements
2.20 De®nition: Exponential stability . . . . . . . . . . . . . . . . . 120
2.21 Theorem: Lyapunov function and exponential stability . . 120
2.22 De®nition: Input/output-to-state stable (IOSS) . . . . . . . . 121
2.23 Assumption: Modi®ed basic stability assumption . . . . . . 121
2.24 Theorem: Asymptotic stability with stage cost `y;u . . . 122
2.25 Assumption: Continuity of system and cost; time-varying
case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
2.26 Assumption: Properties of constraint sets; time-varying case 124
2.27 De®nition: Sequential positive invariance and sequential
control invariance . . . . . . . . . . . . . . . . . . . . . . . . . . 125
2.28 Proposition: Continuous system solution; time-varying case 125
2.29 Proposition: Existence of solution to optimal control prob-
lem; time-varying case . . . . . . . . . . . . . . . . . . . . . . . 125
2.30 De®nition: Asymptotically stable andGAS for time-varying
systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
2.31 De®nition: Lyapunov function: time-varying, constrained
case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
2.32 Theorem: Lyapunov theorem for asymptotic stability (time-
varying, constrained) . . . . . . . . . . . . . . . . . . . . . . . . 126
2.33 Assumption: Basic stability assumption; time-varying case 127
2.34 Proposition: Optimal cost decrease; time-varying case . . . 127
2.35 Proposition: MPC cost is less than terminal cost . . . . . . . 127
2.36 Proposition: Optimal value function properties; time-varying
case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
2.37 Assumption: Uniform weak controllability . . . . . . . . . . 128
2.38 Proposition: Conditions for uniform weak controllability . 128
2.39 Theorem: Asymptotic stability of the origin: time-varying
MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
2.40 Lemma: Entering the terminal region . . . . . . . . . . . . . . 146
2.41 Theorem: MPC stability; no terminal constraint . . . . . . . 146
2.42 Proposition: Admissible warm start in Xf . . . . . . . . . . . 149
2.43 Algorithm: Suboptimal MPC . . . . . . . . . . . . . . . . . . . 149
2.44 Proposition: Linking warm start and state . . . . . . . . . . . 150
2.45 De®nition: Asymptotic stability (difference inclusion) . . . 150
2.46 De®nition: Lyapunov function (difference inclusion) . . . . 151
2.47 Proposition: Asymptotic stability (difference inclusion) . . 151
2.48 Theorem: Asymptotic stability of suboptimal MPC . . . . . 151
2.49 Assumption: Continuity of system and cost . . . . . . . . . 154
2.50 Assumption: Properties of constraint sets . . . . . . . . . . 154
2.51 Assumption: Cost lower bound . . . . . . . . . . . . . . . . . 154
-
List of Examples and Statements xxix
2.52 Proposition: Asymptotic average performance . . . . . . . . 155
2.53 De®nition: Dissipativity . . . . . . . . . . . . . . . . . . . . . . 156
2.54 Assumption: Continuity at the steady state . . . . . . . . . . 157
2.55 Assumption: Strict dissipativity . . . . . . . . . . . . . . . . . 157
2.56 Theorem: Asymptotic stability of economic MPC . . . . . . 157
2.57 Example: Economic MPC versus tracking MPC . . . . . . . . 158
2.58 Example: MPC with mixed continuous/discrete actuators . 162
2.59 Theorem: Lyapunov theorem for asymptotic stability . . . 177
2.60 Proposition: Convergence of state under IOSS . . . . . . . . 178
2.61 Lemma: An equality for quadratic functions . . . . . . . . . 178
2.62 Lemma: Evolution in a compact set . . . . . . . . . . . . . . . 179
3.1 De®nition: Robust global asymptotic stability . . . . . . . . 207
3.2 Theorem: Lyapunov function and RGAS . . . . . . . . . . . . 208
3.3 Theorem: Robust global asymptotic stability and regular-
ization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
3.4 Proposition: Bound for continuous functions . . . . . . . . . 211
3.5 Proposition: Robustness of nominal MPC . . . . . . . . . . . 214
3.6 De®nition: Robust control invariance . . . . . . . . . . . . . 217
3.7 De®nition: Robust positive invariance . . . . . . . . . . . . . 217
3.8 Assumption: Basic stability assumption; robust case . . . . 218
3.9 Theorem: Recursive feasibility of control policies . . . . . . 218
3.10 De®nition: Set algebra and Hausdorff distance . . . . . . . . 224
3.11 De®nition: Robust asymptotic stability of a set . . . . . . . 230
3.12 Proposition: Robust asymptotic stability of tube-based
MPC for linear systems . . . . . . . . . . . . . . . . . . . . . . 230
3.13 Example: Calculation of tightened constraints . . . . . . . . 231
3.14 Proposition: Recursive feasibility of tube-based MPC . . . . 235
3.15 Proposition: Robust exponential stability of improved tube-
based MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
3.16 Proposition: Implicit satisfaction of terminal constraint . . 239
3.17 Proposition: Properties of the value function . . . . . . . . . 240
3.18 Proposition: Neighborhoods of the uncertain system . . . . 241
3.19 Proposition: Robust positive invariance of tube-basedMPC
for nonlinear systems . . . . . . . . . . . . . . . . . . . . . . . 241
3.20 Example: Robust control of an exothermic reaction . . . . . 243
3.21 Assumption: Stabilizing conditions, stochastic MPC: Ver-
sion 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
3.22 Assumption: Stabilizing conditions, stochastic MPC: Ver-
sion 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
-
xxx List of Examples and Statements
3.23 Proposition: Expected cost bound . . . . . . . . . . . . . . . . 250
3.24 Assumption: Robust terminal set condition . . . . . . . . . 253
3.25 Example: Constraint tightening via sampling . . . . . . . . . 254
4.1 De®nition: State Estimator . . . . . . . . . . . . . . . . . . . . 271
4.2 De®nition: Robustly globally asymptotically stable esti-
mation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
4.3 Proposition: RGAS plus convergent disturbances imply
convergent estimates . . . . . . . . . . . . . . . . . . . . . . . . 273
4.4 Example: The Kalman ®lter of a linear system is RGAS . . . 273
4.5 De®nition: i-IOSS . . . . . . . . . . . . . . . . . . . . . . . . . . 275
4.6 Proposition: RGAS estimator implies i-IOSS . . . . . . . . . . 276
4.7 De®nition: i-IOSS Lyapunov function . . . . . . . . . . . . . . 277
4.8 Theorem: i-IOSS and Lyapunov function equivalence . . . . 277
4.9 De®nition: Incremental Stabilizability with respect to stage
cost L� . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2774.10 Assumption: Continuity . . . . . . . . . . . . . . . . . . . . . . 277
4.11 Assumption: Positive-de®nite stage cost . . . . . . . . . . . . 278
4.12 Assumption: Stabilizability . . . . . . . . . . . . . . . . . . . . 278
4.13 Assumption: Detectability . . . . . . . . . . . . . . . . . . . . 278
4.14 De®nition: Q-function for estimation . . . . . . . . . . . . . 283
4.15 Theorem: Q-function theorem for global asymptotic stability 283
4.16 Theorem: Stability of full information estimation . . . . . . 283
4.17 Assumption: Stage cost under disturbances . . . . . . . . . 284
4.18 Assumption: Stabilizability under disturbances . . . . . . . 284
4.19 De®nition: Exponentially i-IOSS . . . . . . . . . . . . . . . . . 285
4.20 De®nition: Robustly globally exponentially stable estimation 285
4.21 Proposition: Equivalent de®nition of RGES . . . . . . . . . . 286
4.22 Assumption: Power-law bounds for stage costs . . . . . . . 286
4.23 Assumption: Exponential stabilizability . . . . . . . . . . . . 286
4.24 Assumption: Exponential detectability . . . . . . . . . . . . . 286
4.25 Theorem: Robust stability of full information estimation . 287
4.26 Lemma: Duality of controllability and observability . . . . . 291
4.27 Theorem: Riccati iteration and regulator stability . . . . . . 291
4.28 De®nition: Observability . . . . . . . . . . . . . . . . . . . . . 293
4.29 De®nition: Final-state observability . . . . . . . . . . . . . . . 294
4.30 De®nition: GloballyK-continuous . . . . . . . . . . . . . . . 2944.31 Proposition: Observable and globalK-continuous imply FSO2944.32 De®nition: RGAS estimation (observable case) . . . . . . . . 295
4.33 Theorem: MHE is RGAS (observable case) . . . . . . . . . . . 295
-
List of Examples and Statements xxxi
4.34 De®nition: Full information arrival cost . . . . . . . . . . . . 297
4.35 Lemma: MHE and FIE equivalence . . . . . . . . . . . . . . . . 297
4.36 Assumption: MHE prior weighting bounds . . . . . . . . . . 297
4.37 Theorem: MHE is RGES . . . . . . . . . . . . . . . . . . . . . . 298
4.38 Example: Filtering and smoothing updates . . . . . . . . . . 300
4.39 Example: EKF, UKF, and MHE performance comparison . . 306
4.40 De®nition: i-UIOSS . . . . . . . . . . . . . . . . . . . . . . . . . 312
4.41 Assumption: Bounded estimate error . . . . . . . . . . . . . 313
4.42 De®nition: Robust positive invariance . . . . . . . . . . . . . 313
4.43 De®nition: Robust asymptotic stability . . . . . . . . . . . . 314
4.44 De®nition: ISS Lyapunov function . . . . . . . . . . . . . . . . 314
4.45 Proposition: ISS Lyapunov stability theorem . . . . . . . . . 314
4.46 Theorem: Combined MHE/MPC is RAS . . . . . . . . . . . . . 316
4.47 Example: Combined MHE/MPC . . . . . . . . . . . . . . . . . . 317
5.1 De®nition: Positive invariance; robust positive invariance . 339
5.2 Proposition: Proximity of state and state estimate . . . . . 339
5.3 Proposition: Proximity of state estimate and nominal state 341
5.4 Assumption: Constraint bounds . . . . . . . . . . . . . . . . . 342
5.5 Algorithm: Robust control algorithm (linear constrained
systems) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
5.6 Proposition: Exponential stability of output MPC . . . . . . 345
5.7 Algorithm: Robust control algorithm (offset-free MPC) . . . 353
6.1 Algorithm: Suboptimal MPC (simpli®ed) . . . . . . . . . . . . 369
6.2 De®nition: Lyapunov stability . . . . . . . . . . . . . . . . . . 370
6.3 De®nition: Uniform Lyapunov stability . . . . . . . . . . . . 371
6.4 De®nition: Exponential stability . . . . . . . . . . . . . . . . . 371
6.5 Lemma: Exponential stability of suboptimal MPC . . . . . . 372
6.6 Lemma: Global asymptotic stability and exponential con-
vergence with mixed powers of norm . . . . . . . . . . . . . 373
6.7 Lemma: Converse theorem for exponential stability . . . . 374
6.8 Assumption: Unconstrained two-player game . . . . . . . . 380
6.9 Example: Nash equilibrium is unstable . . . . . . . . . . . . . 383
6.10 Example: Nash equilibrium is stable but closed loop is
unstable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
6.11 Example: Nash equilibrium is stable and the closed loop
is stable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
6.12 Example: Stability and offset in the distributed target cal-
culation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
-
xxxii List of Examples and Statements
6.13 Assumption: Constrained two-player game . . . . . . . . . . 401
6.14 Lemma: Global asymptotic stability and exponential con-
vergence of perturbed system . . . . . . . . . . . . . . . . . . 408
6.15 Assumption: Disturbance models . . . . . . . . . . . . . . . . 409
6.16 Lemma: Detectability of distributed disturbance model . . 409
6.17 Assumption: Constrained M-player game . . . . . . . . . . . 414
6.18 Lemma: Distributed gradient algorithm properties . . . . . 418
6.19 Assumption: Basic stability assumption (distributed) . . . . 420
6.20 Proposition: Terminal constraint satisfaction . . . . . . . . 421
6.21 Theorem: Asymptotic stability . . . . . . . . . . . . . . . . . . 423
6.22 Example: Nonlinear distributed control . . . . . . . . . . . . 423
6.23 Lemma: Local detectability . . . . . . . . . . . . . . . . . . . . 437
7.1 De®nition: Polytopic (polyhedral) partition . . . . . . . . . . 450
7.2 De®nition: Piecewise af®ne function . . . . . . . . . . . . . . 450
7.3 Assumption: Strict convexity . . . . . . . . . . . . . . . . . . . 451
7.4 De®nition: Polar cone . . . . . . . . . . . . . . . . . . . . . . . 453
7.5 Proposition: Farkas's lemma . . . . . . . . . . . . . . . . . . . 453
7.6 Proposition: Optimality conditions for convex set . . . . . . 453
7.7 Proposition: Optimality conditions in terms of polar cone 455
7.8 Proposition: Optimality conditions for linear inequalities . 455
7.9 Proposition: Solution of Pw, w 2 R0x . . . . . . . . . . . . 4577.10 Proposition: Piecewise quadratic (af®ne) cost (solution) . . 458
7.11 Example: Parametric QP . . . . . . . . . . . . . . . . . . . . . . 458
7.12 Example: Explicit optimal control . . . . . . . . . . . . . . . . 459
7.13 Proposition: Continuity of cost and solution . . . . . . . . . 461
7.14 Assumption: Continuous, piecewise quadratic function . . 464
7.15 De®nition: Active polytope (polyhedron) . . . . . . . . . . . 465
7.16 Proposition: Solving P using Pi . . . . . . . . . . . . . . . . . 465
7.17 Proposition: Optimality of u0xw in Rx . . . . . . . . . . . . 468
7.18 Proposition: Piecewise quadratic (af®ne) solution . . . . . . 468
7.19 Proposition: Optimality conditions for parametric LP . . . 472
7.20 Proposition: Solution of P . . . . . . . . . . . . . . . . . . . . . 475
7.21 Proposition: Piecewise af®ne cost and solution . . . . . . . 475
8.1 Example: Nonlinear MPC . . . . . . . . . . . . . . . . . . . . . 489
8.2 Example: Sequential approach . . . . . . . . . . . . . . . . . . 492
8.3 Example: Integration methods of different order . . . . . . 498
8.4 Example: Implicit integrators for a stiff ODE system . . . . 505
8.5 Example: Finding a ®fth root with Newton-type iterations . 510
-
List of Examples and Statements xxxiii
8.6 Example: Convergence rates . . . . . . . . . . . . . . . . . . . 511
8.7 Theorem: Local contraction for Newton-type methods . . . 512
8.8 Corollary: Convergence of exact Newton's method . . . . . 513
8.9 Example: Function evaluation via elementary operations . 516
8.10 Example: Implicit function representation . . . . . . . . . . 518
8.11 Example: Forward algorithmic differentiation . . . . . . . . 520
8.12 Example: Algorithmic differentiation in reverse mode . . . 522
8.13 Example: Sequential optimal control using CasADi from
Octave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
8.14 Theorem: KKT conditions . . . . . . . . . . . . . . . . . . . . . 543
8.15 Theorem: Strong second-order suf®cient conditions for
optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545
8.16 Theorem: Tangential predictor by quadratic program . . . 545
8.17 Example: MPC with discrete actuator . . . . . . . . . . . . . . 577
A.1 Theorem: Schur decomposition . . . . . . . . . . . . . . . . . 629
A.2 Theorem: Real Schur decomposition . . . . . . . . . . . . . . 630
A.3 Theorem: Bolzano-Weierstrass . . . . . . . . . . . . . . . . . . 632
A.4 Proposition: Convergence of monotone sequences . . . . . 633
A.5 Proposition: Uniform continuity . . . . . . . . . . . . . . . . . 634
A.6 Proposition: Compactness of continuous functions of com-
pact sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635
A.7 Proposition: Weierstrass . . . . . . . . . . . . . . . . . . . . . 636
A.8 Proposition: Derivative and partial derivative . . . . . . . . 637
A.9 Proposition: Continuous partial derivatives . . . . . . . . . . 638
A.10 Proposition: Chain rule . . . . . . . . . . . . . . . . . . . . . . 638
A.11 Proposition: Mean value theorem for vector functions . . . 638
A.12 De®nition: Convex set . . . . . . . . . . . . . . . . . . . . . . . 641
A.13 Theorem: Caratheodory . . . . . . . . . . . . . . . . . . . . . . 641
A.14 Theorem: Separation of convex sets . . . . . . . . . . . . . . 642
A.15 Theorem: Separation of convex set from zero . . . . . . . . 643
A.16 Corollary: Existence of separating hyperplane . . . . . . . . 643
A.17 De®nition: Support hyperplane . . . . . . . . . . . . . . . . . 644
A.18 Theorem: Convex set and halfspaces . . . . . . . . . . . . . . 644
A.19 De®nition: Convex cone . . . . . . . . . . . . . . . . . . . . . . 644
A.20 De®nition: Polar cone . . . . . . . . . . . . . . . . . . . . . . . 644
A.21 De®nition: Cone generator . . . . . . . . . . . . . . . . . . . . 645
A.22 Proposition: Cone and polar cone generator . . . . . . . . . 645
A.23 Theorem: Convexity implies continuity . . . . . . . . . . . . 647
A.24 Theorem: Differentiability and convexity . . . . . . . . . . . 647
-
xxxiv List of Examples and Statements
A.25 Theorem: Second derivative and convexity . . . . . . . . . . 647
A.26 De®nition: Level set . . . . . . . . . . . . . . . . . . . . . . . . 648
A.27 De®nition: Sublevel set . . . . . . . . . . . . . . . . . . . . . . 648
A.28 De®nition: Support function . . . . . . . . . . . . . . . . . . . 648
A.29 Proposition: Set membership and support function . . . . . 648
A.30 Proposition: Lipschitz continuity of support function . . . 648
A.31 Theorem: Existence of solution to differential equations . 651
A.32 Theorem: Maximal interval of existence . . . . . . . . . . . . 651
A.33 Theorem: Continuity of solution to differential equation . 651
A.34 Theorem: Bellman-Gronwall . . . . . . . . . . . . . . . . . . . 651
A.35 Theorem: Existence of solutions to forced systems . . . . . 653
A.36 Example: Fourier transform of the normal density. . . . . . 659
A.37 De®nition: Density of a singular normal . . . . . . . . . . . . 662
A.38 Example: Marginal normal density . . . . . . . . . . . . . . . 663
A.39 Example: Nonlinear transformation . . . . . . . . . . . . . . . 666
A.40 Example: Maximum of two random variables . . . . . . . . . 667
A.41 Example: Independent implies uncorrelated . . . . . . . . . 668
A.42 Example: Does uncorrelated imply independent? . . . . . . 669
A.43 Example: Independent and uncorrelated are equivalent
for normals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 671
A.44 Example: Conditional normal density . . . . . . . . . . . . . 674
A.45 Example: More normal conditional densities . . . . . . . . . 675
B.1 De®nition: Equilibrium point . . . . . . . . . . . . . . . . . . . 694
B.2 De®nition: Positive invariant set . . . . . . . . . . . . . . . . . 694
B.3 De®nition: K,K1,KL, and PD functions . . . . . . . . . . 695B.4 De®nition: Local stability . . . . . . . . . . . . . . . . . . . . . 696
B.5 De®nition: Global attraction . . . . . . . . . . . . . . . . . . . 697
B.6 De®nition: Global asymptotic stability . . . . . . . . . . . . . 697
B.7 De®nition: Various forms of stability . . . . . . . . . . . . . . 698
B.8 De®nition: Global asymptotic stability (KL version) . . . . . 699
B.9 Proposition: Connection of classical andKL global asymp-
totic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 699
B.10 De®nition: Various forms of stability (constrained) . . . . . 699
B.11 De®nition: Asymptotic stability (constrained, KL version) . 700
B.12 De®nition: Lyapunov function (unconstrained and con-
strained) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 701
B.13 Theorem: Lyapunov function and GAS (classical de®nition) 702
B.14 Lemma: From PD toK1 function (Jiang and Wang (2002)) 703
-
List of Examples and Statements xxxv
B.15 Theorem: Lyapunov function and global asymptotic sta-
bility (KL de®nition) . . . . . . . . . . . . . . . . . . . . . . . . 703
B.16 Proposition: Improving convergence (Sontag (1998b)) . . . 705
B.17 Theorem: Converse theorem for global asymptotic stability 705
B.18 Theorem: Lyapunov function for asymptotic stability (con-
strained) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706
B.19 Theorem: Lyapunov function for exponential stability . . . 706
B.20 Lemma: Lyapunov function for linear systems . . . . . . . . 707
B.21 De®nition: Sequential positive invariance . . . . . . . . . . . 707
B.22 De®nition: Asymptotic stability (time-varying, constrained) 707
B.23 De®nition: Lyapunov function: time-varying, constrained
case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 708
B.24 Theorem: Lyapunov theorem for asymptotic stability (time-
varying, constrained) . . . . . . . . . . . . . . . . . . . . . . . . 708
B.25 Proposition: Global K function overbound . . . . . . . . . . 709
B.26 De®nition: Nominal robust global asymptotic stability . . . 710
B.27 Theorem: Nominal robust global asymptotic stability and
Lyapunov function . . . . . . . . . . . . . . . . . . . . . . . . . 710
B.28 De®nition: Positive invariance with disturbances . . . . . . 711
B.29 De®nition: Local stability (disturbances) . . . . . . . . . . . . 712
B.30 De®nition: Global attraction (disturbances) . . . . . . . . . . 712
B.31 De®nition: GAS (disturbances) . . . . . . . . . . . . . . . . . . 712
B.32 De®nition: Lyapunov function (disturbances) . . . . . . . . . 712
B.33 Theorem: Lyapunov function for global asymptotic sta-
bility (disturbances) . . . . . . . . . . . . . . . . . . . . . . . . 713
B.34 De®nition: Global control Lyapunov function (CLF) . . . . . 714
B.35 De®nition: Global stabilizability . . . . . . . . . . . . . . . . . 714
B.36 De®nition: Positive invariance (disturbance and control) . . 715
B.37 De®nition: CLF (disturbance and control) . . . . . . . . . . . 715
B.38 De®nition: Control invariance (constrained) . . . . . . . . . 715
B.39 De®nition: CLF (constrained) . . . . . . . . . . . . . . . . . . . 716
B.40 De®nition: Control invariance (disturbances, constrained) 716
B.41 De®nition: CLF (disturbances, constrained) . . . . . . . . . . 716
B.42 De®nition: Input-to-state stable (ISS) . . . . . . . . . . . . . . 717
B.43 De®nition: ISS-Lyapunov function . . . . . . . . . . . . . . . . 718
B.44 Lemma: ISS-Lyapunov function implies ISS . . . . . . . . . . 718
B.45 De®nition: ISS (constrained) . . . . . . . . . . . . . . . . . . . 718
B.46 De®nition: ISS-Lyapunov function (constrained) . . . . . . . 718
B.47 Lemma: ISS-Lyapunov function implies ISS (constrained) . 719
B.48 De®nition: Output-to-state stable (OSS) . . . . . . . . . . . . 720
-
xxxvi List of Examples and Statements
B.49 De®nition: OSS-Lyapunov function . . . . . . . . . . . . . . . 720
B.50 Theorem: OSS and OSS-Lyapunov function . . . . . . . . . . 720
B.51 De®nition: Input/output-to-state stable (IOSS) . . . . . . . . 721
B.52 De®nition: IOSS-Lyapunov function . . . . . . . . . . . . . . . 721
B.53 Theorem: Modi®ed IOSS-Lyapunov function . . . . . . . . . 721
B.54 Conjecture: IOSS and IOSS-Lyapunov function . . . . . . . . 722
B.55 De®nition: Incremental input/output-to-state stable . . . . 722
B.56 De®nition: Observability . . . . . . . . . . . . . . . . . . . . . 722
B.57 Assumption: Lipschitz continuity of model . . . . . . . . . . 723
B.58 Lemma: Lipschitz continuity and state difference bound . 723
B.59 Theorem: Observability and convergence of state . . . . . . 723
C.1 Lemma: Principle of optimality . . . . . . . . . . . . . . . . . 734
C.2 Theorem: Optimal value function and control law from DP 734
C.3 Example: DP applied to linear quadratic regulator . . . . . . 736
C.4 De®nition: Tangent vector . . . . . . . . . . . . . . . . . . . . 739
C.5 Proposition: Tangent vectors are closed cone . . . . . . . . 739
C.6 De®nition: Regular normal . . . . . . . . . . . . . . . . . . . . 739
C.7 Proposition: Relation of normal and tangent cones . . . . . 740
C.8 Proposition: Global optimality for convex problems . . . . 741
C.9 Proposition: Optimality conditionsÐnormal cone . . . . . . 742
C.10 Proposition: Optimality conditionsÐtangent cone . . . . . 743
C.11 Proposition: Representation of tangent and normal cones . 743
C.12 Proposition: Optimality conditionsÐlinear inequalities . . 744
C.13 Corollary: Optimality conditionsÐlinear inequalities . . . . 744
C.14 Proposition: Necessary condition for nonconvex problem . 746
C.15 De®nition: General normal . . . . . . . . . . . . . . . . . . . . 748
C.16 De®nition: General tangent . . . . . . . . . . . . . . . . . . . . 748
C.17 Proposition: Set of regular tangents is closed convex cone 748
C.18 De®nition: Regular set . . . . . . . . . . . . . . . . . . . . . . . 749
C.19 Proposition: Conditions for regular set . . . . . . . . . . . . 749
C.20 Proposition: Quasiregular set . . . . . . . . . . . . . . . . . . 751
C.21 Proposition: Optimality conditions nonconvex problem . . 752
C.22 Proposition: Fritz-John necessary conditions . . . . . . . . . 753
C.23 De®nition: Outer semicontinuous function . . . . . . . . . . 757
C.24 De®nition: Inner semicontinuous function . . . . . . . . . . 758
C.25 De®nition: Continuous function . . . . . . . . . . . . . . . . . 758
C.26 Theorem: Equivalent conditions for outer and inner semi-
continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 759
C.27 Proposition: Outer semicontinuity and closed graph . . . . 759
-
List of Examples and Statements xxxvii
C.28 Theorem: Minimum theorem . . . . . . . . . . . . . . . . . . . 760
C.29 Theorem: Lipschitz continuity of the value function, con-
stant U . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 761
C.30 De®nition: Subgradient of convex function . . . . . . . . . . 762
C.31 Theorem: Clarke et al. (1998) . . . . . . . . . . . . . . . . . . 762
C.32 Corollary: A bound on du;Ux0 for u 2 Ux . . . . . . 763C.33 Theorem: Continuity of U� . . . . . . . . . . . . . . . . . . . 765C.34 Theorem: Continuity of the value function . . . . . . . . . . 765
C.35 Theorem: Lipschitz continuity of the value functionÐUx 766
-
Notation
Mathematical notation
9 there exists2 is an element of8 for all=) (= implies; is implied by6=) 6(= does not imply; is not implied bya : b a is de®ned to be equal to b.a : b b is de®ned to be equal to a.� approximately equalV� function VV : A! B V is a function mapping set A into set Bx , Vx function V maps variable x to value Vx
x value of x at next sample time (discrete time system)
Çx time derivative of x (continuous time system)
fx partial derivative of fx with respect to x
r nabla or del operator� unit impulse or delta function
jxj absolute value of scalar; norm of vector (two-norm unlessstated otherwise); induced norm of matrix
x sequence of vector-valued variable x, x0; x1; : : :
kxk sup norm over a sequence, supi�0 jxijkxka:b maxa�i�b jxijtrA trace of matrix A
detA determinant of matrix A
eigA set of eigenvalues of matrix A
�A spectral radius of matrix A, maxi j�ij for �i 2 eigAA�1 inverse of matrix A
Ay pseudo-inverse of matrix A
A0 transpose of matrix A
inf in®mum or greatest lower bound
min minimum
sup supremum or least upper bound
max maximum
xxxix
-
xl Notation
arg argument or solution of an optimization
s.t. subject to
I integers
I�0 nonnegative integers
In:m integers in the interval n;m
R real numbers
R�0 nonnegative real numbers
Rn real-valued n-vectors
Rm�n real-valuedm�n matricesC complex numbers
B ball in Rn of unit radiusx � px random variable x has probability density pxEx expectation of random variable xvarx variance of random variable x
covx;y covariance of random variables x and y
Nm;P normal distribution (meanm, covariance P ), x � Nm;Pnx;m;P normal probability density, pxx nx;m;P; the empty setaffA af®ne hull of set A
intA interior of set A
coA convex hull of the set A
A closure of set A
leva V sublevel set of function V , fx j Vx � agf � g composition of functions f and g, f � g s : fgsa� b maximum of scalars a and b, Chapter 4Ln
i1 ai a1 � a2 � � � � � an, Chapter 4A� B set addition of sets A and B, Chapters 3 and 5A B set subtraction of set B from set AA n B elements of set A not in set BA[ B union of sets A and BA\ B intersection of sets A and BA � B set A is a subset of set BA � B set A is a superset of set BA � B set A is a proper (or strict) subset of set BA � B set A is a proper (or strict) superset of set Bda;B Distance between element a and set B
dHA;B Hausdorff distance between sets A and B
x & y x % y x converges to y from above (below)satx saturation, satx x if jxj � 1;�1 if x < �1;1 if x > 1
-
Notation xli
Symbols
A;B;C system matrices, discrete time, x Ax Bu, y CxAc ; Bc system matrices, continuous time, Çx Acx BcuAij state transition matrix for player i to player j
Ai state transition matrix for player i
ALi estimate error transition matrix Ai � LiCiBd input disturbance matrix
Bij input matrix of player i for player j's inputs
Bi input matrix of player i
Cij output matrix of player i for player j's interaction states
Ci output matrix of player i
Cd output disturbance matrix
C controllability matrixC� polar cone of cone C
d integrating disturbance
E; F constraint matrices, Fx Eu � ef ;h system functions, discrete time, x fx;u, y hxfcx;u system function, continuous time, Çx fcx;uFx;u difference inclusion, x 2 Fx;u, F is set valuedG input noise-shaping matrix
Gij steady-state gain of player i to player j
H controlled variable matrix
Ix;u index set of constraints active at x;u
I0x index set of constraints active at x;u0x
k sample time
K optimal controller gain
`x;u stage cost
`Nx;u ®nal stage cost
L optimal estimator gain
m input dimension
M cross-term penalty matrix x0Mu
M number of players, Chapter 6
M class of admissible input policies, � 2Mn state dimension
N horizon length
O observability matrix, Chapters 1 and 4O compact robust control invariant set containing the origin,
Chapter 3
p output dimension
-
xlii Notation
p optimization iterate, Chapter 6
p� probability density of random variable �
psx sampled probability density, psx P
iwi�x � xiP covariance matrix in the estimator
Pf terminal penalty matrix
P polytopic partition, Chapter 3P polytopic partition, Chapter 7PNx MPC optimization problem; horizon N and initial state x
q importance function in importance sampling
Q state penalty matrix
r controlled variable, r HyR input penalty matrix
s number of samples in a sampled probability density
S input rate of change penalty matrix
Sx;u index set of active polytopes at x;u
S0x index set of active polytopes at x;u0x
t time
T current time in estimation problem
u input (manipulated variable) vector
ue warm start for input sequenceu improved input sequence
UNx control constraint setU input constraint set
v output disturbance, Chapters 1 and 4
v nominal control input, Chapters 3 and 5
VNx;u MPC objective function
V 0Nx MPC optimal value function
VT �;! Full information state estimation objective function at time T
with initial state � and disturbance sequence !
ÃVT �;! MHE objective function at time T with initial state � and distur-
bance sequence !
Vf x terminal penalty
VNz nominal control input constraint setV output disturbance constraint set
w disturbance to the state evolution
wi weights in a sampled probability density, Chapter 4
wi convex weight for player i, Chapter 6
wi normalized weights in a sampled probability density
W class of admissible disturbance sequences, w 2W
-
Notation xliii
W state disturbance constraint set
x state vector
xi sample values in a sampled probability density
xij state interaction vector from player i to player j
x0 mean of initial state density
Xk;x;� state tube at time k with initial state x and control policy �
Xj set of feasible states for optimal control problem at stage jX state constraint set
Xf terminal region
y output (measurement) vector
Y output constraint set
z nominal state, Chapters 3 and 5
ZT x full information arrival cost
ÃZT x MHE arrival cost
ZeT x MHE smoothing arrival costZ system constraint region, x;u 2 ZZf terminal constraint region, x;u 2 ZfZNx;u constraint set for state and input sequence
Greek letters
�T � MHE prior weighting on state at time T
� sample time
� control law
�j control law at stage j
�f control law applied in terminal region Xf
�ix control law at stage i
�x control policy or sequence of control laws
� output disturbance decision variable in estimation problem
� cost-to-go matrix in regulator, Chapter 1
� covariance matrix in the estimator, Chapter 5
�i objective function weight for player i
�i Solution to Lyapunov equation for player i
�k;x;u state at time k given initial state x and input sequence u
�k;x; i;u state at time k given state at time i is x and input sequence u
�k;x;u;w state at time k given initial state is x, input sequence is u, and
disturbance sequence is w
� state decision variable in estimation problem
! state disturbance decision variable in estimation problem
-
xliv Notation
Subscripts, superscripts, and accents
Ãx estimate
Ãx� estimate before measurement
xe estimate errorxs steady state
xi subsystem i in a decomposed large-scale system
xsp setpoint
V 0 optimal
Vuc unconstrained
V sp unreachable setpoint
-
Acronyms
AD algorithmic (or automatic) differentiation
AS asymptotically stable
BFGS Broyden-Fletcher-Goldfarb-Shanno
CIA combinatorial integral approximation
CLF control Lyapunov function
DAE differential algebraic equation
DARE discrete algebraic Riccati equation
DDP differential dynamic programming
DP dynamic programming
END external numerical differentiation
FIE full information estimation
FLOP ¯oating point operation
FSO ®nal-state observable
GAS globally asymptotically stable
GES globally exponentially stable
GL Gauss-Legendre
GPC generalized predict