-
LQ Dynamic Optimization and
Differential Games
Jacob Engwerda
Tilburg University, The Netherlands
JOHN WILEY & SONS, LTD
Innodata0470015519.jpg
-
LQ Dynamic Optimization and
Differential Games
-
LQ Dynamic Optimization and
Differential Games
Jacob Engwerda
Tilburg University, The Netherlands
JOHN WILEY & SONS, LTD
-
Copyright # 2005 John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester,West Sussex PO19 8SQ, England
Telephone (+44) 1243 779777
Email (for orders and customer service enquiries): [email protected] our Home Page on www.wiley.com
All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted inany form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, exceptunder the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issuedby the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without thepermission in writing of the Publisher. Requests to the Publisher should be addressed to the PermissionsDepartment, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ,England, or emailed to [email protected], or faxed to (+44) 1243 770620.
Designations used by companies to distinguish their products are often claimed as trademarks. All brand namesand product names used in this book are trade names, service marks, trademarks or registered trademarks of theirrespective owners. The Publisher is not associated with any product or vendor mentioned in this book.
This publication is designed to provide accurate and authoritative information in regard to thesubject matter covered. It is sold on the understanding that the Publisher is not engaged in rendering professionalservices. If professional advice or other expert assistance is required, the servicesof a competent professional should be sought.
Other Wiley Editorial Offices
John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA
Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA
WileyVCH Verlag GmbH, Boschstr. 12, D-69469 Weinheim, Germany
John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia
John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop # 02-01, Jin Xing Distripark, Singapore 129809
John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not beavailable in electronic books.
Library of Congress Cataloging-in-Publication Data
p. cm.ISBN 0-470-85756-01. CancerRisk factorsMathematical models. I. Edler, Lutz, 1945- II. Kitsos, Christos Par., 1951-RC268.48.Q34 2004616.99040710 015118dc22 2004028290
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
ISBN-13 978-0-470-01524-7 (HB)ISBN-10 0-470-01524-1 (HB)
Typeset in 10/12pt Times by Thomson Press (India) Limited, New DelhiPrinted and bound in Great Britain by Antony Rowe Ltd, Chippenham, WiltshireThis book is printed on acid-free paper responsibly manufactured from sustainable forestryin which at least two trees are planted for each one used for paper production.
http://www.wiley.com
-
Contents
Preface ix
Notation and symbols xi
1 Introduction 11.1 Historical perspective 1
1.2 How to use this book 10
1.3 Outline of this book 10
1.4 Notes and references 14
2 Linear algebra 152.1 Basic concepts in linear algebra 15
2.2 Eigenvalues and eigenvectors 21
2.3 Complex eigenvalues 23
2.4 CayleyHamilton theorem 31
2.5 Invariant subspaces and Jordan canonical form 34
2.6 Semi-definite matrices 42
2.7 Algebraic Riccati equations 43
2.8 Notes and references 54
2.9 Exercises 55
2.10 Appendix 59
3 Dynamical systems 633.1 Description of linear dynamical systems 64
3.2 Existenceuniqueness results for differential equations 70
3.2.1 General case 70
3.2.2 Control theoretic extensions 74
3.3 Stability theory: general case 78
3.4 Stability theory of planar systems 83
3.5 Geometric concepts 91
3.6 Performance specifications 96
3.7 Examples of differential games 105
3.8 Information, commitment and strategies 114
3.9 Notes and references 114
3.10 Exercises 115
3.11 Appendix 118
-
4 Optimization techniques 1214.1 Optimization of functions 121
4.2 The EulerLagrange equation 125
4.3 Pontryagins maximum principle 133
4.4 Dynamic programming principle 154
4.5 Solving optimal control problems 162
4.6 Notes and references 162
4.7 Exercises 163
4.8 Appendix 170
5 Regular linear quadratic optimal control 1755.1 Problem statement 175
5.2 Finite-planning horizon 177
5.3 Riccati differential equations 192
5.4 Infinite-planning horizon 196
5.5 Convergence results 209
5.6 Notes and references 218
5.7 Exercises 219
5.8 Appendix 224
6 Cooperative games 2296.1 Pareto solutions 230
6.2 Bargaining concepts 240
6.3 Nash bargaining solution 246
6.4 Numerical solution 251
6.5 Notes and references 253
6.6 Exercises 254
6.7 Appendix 259
7 Non-cooperative open-loop information games 2617.1 Introduction 264
7.2 Finite-planning horizon 265
7.3 Open-loop Nash algebraic Riccati equations 278
7.4 Infinite-planning horizon 283
7.5 Computational aspects and illustrative examples 299
7.6 Convergence results 305
7.7 Scalar case 312
7.8 Economics examples 319
7.8.1 A simple government debt stabilization game 320
7.8.2 A game on dynamic duopolistic competition 322
7.9 Notes and references 326
7.10 Exercises 327
7.11 Appendix 340
8 Non-cooperative feedback information games 3598.1 Introduction 359
8.2 Finite-planning horizon 362
vi CONTENTS
-
8.3 Infinite-planning horizon 371
8.4 Two-player scalar case 383
8.5 Computational aspects 389
8.5.1 Preliminaries 390
8.5.2 A scalar numerical algorithm: the two-player case 393
8.5.3 The N-player scalar case 399
8.6 Convergence results for the two-player scalar case 403
8.7 Notes and references 412
8.8 Exercises 413
8.9 Appendix 421
9 Uncertain non-cooperative feedback information games 4279.1 Stochastic approach 428
9.2 Deterministic approach: introduction 433
9.3 The one-player case 435
9.4 The one-player scalar case 444
9.5 The two-player case 450
9.6 A fishery management game 455
9.7 A scalar numerical algorithm 461
9.8 Stochastic interpretation 472
9.9 Notes and references 474
9.10 Exercises 475
9.11 Appendix 481
References 485
Index 495
CONTENTS vii
-
Preface
Interaction between processes at various levels takes place everywhere around us. Either
in industry, economics, ecology or on a social level, in many places processes influence
each other. Particularly in those cases where people can affect the outcome of a process,
the question arises how do people come to take a particular action.
To obtain a clearer view of this question within mathematics the paradigms of optimal
control theory and game theory evolved, and dynamic game theory resulted as a merging
of both these topics. This theory brings together the issues of optimizing behavior, the
presence of multiple agents, enduring consequences of decisions and robustness with
respect to variability in the environment. Within this field, linear quadratic differential
games developed and these play an important role for three reasons: first, many
applications of differential game theory fall into this category, secondly, there are
many analytical results available and, thirdly, efficient numerical solution techniques
can be used to solve these games.
Going through the literature, one can find many instances of results dealing with linear
quadratic differential games. Unfortunately, there is no textbook focusing on this subject
and giving a rigorous self-contained treatment of this theory. This textbook intends on the
one hand to fill this gap and on the other hand to show the relevance of this theory by
illustrating the theoretical concepts and results by means of simple economics examples.
Given this background, the organization of the book was chosen as follows. The last
four chapters of the book deal with differential games. Since the theoretical development
of these chapters sets out some knowledge on the one-player case and dynamic
optimization techniques, these chapters are preceded by a rigorous treatment of this
one-player case, the so-called regular linear quadratic control problem, and a chapter on
dynamic optimization theory. To tackle these issues, however, one needs to be familiar
with some preliminary work on linear algebra and dynamical systems. For that reason
those subjects are dealt with first. The first chapter gives some historical developments
and an outline of the book.
Having worked for more than two years on this book I ask myself whether it was worth
all the trouble. Was it worth spending so much spare time and research time to produce
something from which all one can hope is that it will be digestible and does not contain
too many flaws and, on the other hand, does not contain a number of closely related
interesting issues leaving the reader with information without the theory to back it up?
The main motivation for writing this book was in the hope that it somehow might
contribute a small amount to peoples understanding and that some people will appreciate
this book and use it for this purpose. . .
-
Last, but not least, I would like to thank the people who have contributed, most of them
unknowingly, to this book. First of all my wife, Carine, and children, Elsemiek, Ton and
Heiko, for offering me the chance to use my spare time for doing this. Next, I am
indebted to Hans Schumacher for many discussions over the past decade concerning
various issues on dynamic games, and to both him and Malo Hautus for the use of their
excellent course notes on dynamic optimization. Furthermore Arie Weeren, Rudy Douven
and Bram van den Broek contributed indirectly to the sections on differential games
through their Ph.D. theses. Finally, Hendri Adriaens is acknowledged for his technical
support on LaTEX and Tomasz Michalak for his proof-reading of an early version of this
book.
Answers to the exercises included in this book can be found at http://www.wiley.
com/go/engwerda
x PREFACE
-
Notation and symbols
Matrix (Operations)
xT transpose of the vector x
k x k2 length xTx of vector x0 zero; zero vector; zero matrix
I identity matrix
A1 inverse of matrix AAT transpose of matrix A
adj(A) adjoint of matrix A
det(A) determinant of matrix A
rank(A) rank of matrix A
trace(A) trace of matrix A
A spectrum of matrix AAjS spectrum of matrix A restricted to the invariant subspace SAib matrix A with ith column replaced by bAji matrix A with row j and column i deleted
Ker A null space of A
NA null space of AIm A column space of A
RA column space of AdiagfA,Bg diagonal matrix with diagonal entries A and Bdim(S) dimension of subspace S
E eigenspace corresponding with eigenvalue Eg generalized eigenspace corresponding with eigenvalue
Ec center subspace
Es stable subspace
Eu unstable subspace
vec(A) vector of stacked columns of matrix A
A B Kronecker product of matrices A and BS V direct sum subspaces S and VS? orthogonal complement of subspace SC0 set of complex numbers with non-positive real partC0 set of complex numbers with non-negative real partCn set of complex vectors with n entries
Cnm set of n m matrices with complex entries
-
C1 set of continuous differentiable functions
CD set of continuous functions satisfying some differentiability properties
(definition 3.4)
i set of control functions of player iaff set of affine functions of the state variableF set of all stabilizing feedback matricesL2 set of all measurable Lebesgue square integrable functions on 0;1L2;loc set of all measurable functions that are square integrable over all
finite intervals 0; T Minv set of M-invariant subspacesN set of natural numbers
Ppos set of n-dimensional M-invariant graph subspacesQ set of rational numbers
R set of real numbers
Rn set of vectors with n real entries
Rnm set of n m matrices with real entriesNd set of bargaining problems (see Chapter 6)U set of control functions for which the differential equation has a solution
in the extended sense
Ure fu 2 UjJu exists as a finite number and limt!1 xt exists gUs set of locally square integrable control functions yielding a stable
closed-loop system
Miscellaneous
ARE algebraic Riccati equation
ij Kronecker , ij 1 if i j, ij 0 otherwisef x0; h Frechet/Gateaux differential of f at x0 with increment h@f Frechet derivative of f@if ith Frechet partial derivative of fES; d egalitarian bargaining solutioninf infimum
IS; d ideal pointKS; d KalaiSmorodinsky bargaining solutionlim limit
NS; d Nash bargaining solution0h higher-order terms in hni1pi product of the n pi variablesRDE Riccati differential equation
sup supremum
jzj modulus,ffiffiffiffiffiffiffiffiffiffiffiffiffiffix2 y2
p, of complex number z x iy
Re(z) real part, x, of complex number z x iyx y vector inequality xi > yi; i 1; . . . ;N
xii NOTATION AND SYMBOLS
-
1
Introduction
1.1 Historical perspective
Dynamic game theory brings together four features that are key to many situations in
economics, ecology and elsewhere: optimizing behavior, the presence of multiple agents/
players, enduring consequences of decisions and robustness with respect to variability in
the environment.
To deal with problems which have these four features the dynamic game theory
methodology splits the modeling of the problem into three parts. One part is the modeling
of the environment in which the agents act. To obtain a mathematical model of the agents
environment a set of differential or difference equations is usually specified. These
equations are assumed to capture the main dynamical features of the environment. A
characteristic property of this specification is that these dynamic equations mostly contain
a set of so-called input functions. These input functions model the effect of the actions
taken by the agents on the environment during the course of the game. In particular, by
viewing nature as a separate player in the game who can choose an input function that
works against the other player(s), one can model worst-case scenarios and, consequently,
analyze the robustness of the undisturbed game solution.
A second part is the modeling of the agents objectives. Usually the agents objectives
are formalized as cost/utility functions which have to be minimized. Since this
minimization has to be performed subject to the specified dynamic model of the
environment, techniques developed in optimal control theory play an important role in
solving dynamic games. In fact, from a historical perspective, the theory of dynamic
games arose from the merging of static game theory and optimal control theory. However,
this merging cannot be done without further reflection. This is exactly what the third
modeling part is about. To understand this point it is good to summarize the rudiments of
static games.
Most research in the field of static game theory has been and is being concentrated
on the normal form of a game. In this form all possible sequences of decisions of each
player are set out against each other. So, for example, for a two-player game this results in
a matrix structure. Characteristic for such a game is that it takes place in one moment of
LQ Dynamic Optimization and Differential Games J. Engwerda# 2005 John Wiley & Sons, Ltd