# linear programming: foundations and extensions cdn. programming... linear programming: foundations...

Post on 25-Mar-2020

5 views

Embed Size (px)

TRANSCRIPT

Linear Programming: Foundations and Extensions

Robert J. Vanderbei

DEPARTMENT OF OPERATIONS RESEARCH AND FINANCIAL ENGINEERING, PRINCETON UNIVERSITY, PRINCETON, NJ 08544

E-mail address: rvdb@princeton.edu

LINEAR PROGRAMMING: FOUNDATIONS AND EXTENSIONS Second Edition

Copyright c©2001 by Robert J. Vanderbei. All rights reserved. Printed in the United States of America. Except as permitted under the United States Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a data base or retrieval system, without the prior written permission of the publisher.

ISBN 0-0000-0000-0

The text for this book was formated in Times-Roman and the mathematics was formated in Michael Spivak’s Mathtimes

usingAMS-LATEX(which is a macro package for Leslie Lamport’s LATEX, which itself is a macro package for Donald

Knuth’s TEXtext formatting system) and converted from device-independent to postscript format usingDVIPS. The fig-

ures were produced usingSHOWCASE on a Silicon Graphics, Inc. workstation and were incorporated into the text as

encapsulated postscript files with the macro package calledPSFIG.TEX.

To my parents, Howard and Marilyn,

my dear wife, Krisadee,

and the babes, Marisa and Diana

Contents

Preface xiii

Preface to 2nd Edition xvii

Part 1. Basic Theory—The Simplex Method and Duality 1

Chapter 1. Introduction 3 1. Managing a Production Facility 3 2. The Linear Programming Problem 6 Exercises 8 Notes 10

Chapter 2. The Simplex Method 13 1. An Example 13 2. The Simplex Method 16 3. Initialization 19 4. Unboundedness 22 5. Geometry 22 Exercises 24 Notes 27

Chapter 3. Degeneracy 29 1. Definition of Degeneracy 29 2. Two Examples of Degenerate Problems 29 3. The Perturbation/Lexicographic Method 32 4. Bland’s Rule 36 5. Fundamental Theorem of Linear Programming 38 6. Geometry 39 Exercises 42 Notes 43

Chapter 4. Efficiency of the Simplex Method 45 1. Performance Measures 45 2. Measuring the Size of a Problem 45

vii

viii CONTENTS

3. Measuring the Effort to Solve a Problem 46 4. Worst-Case Analysis of the Simplex Method 47 Exercises 52 Notes 53

Chapter 5. Duality Theory 55 1. Motivation—Finding Upper Bounds 55 2. The Dual Problem 57 3. The Weak Duality Theorem 58 4. The Strong Duality Theorem 60 5. Complementary Slackness 66 6. The Dual Simplex Method 68 7. A Dual-Based Phase I Algorithm 71 8. The Dual of a Problem in General Form 73 9. Resource Allocation Problems 74 10. Lagrangian Duality 78 Exercises 79 Notes 87

Chapter 6. The Simplex Method in Matrix Notation 89 1. Matrix Notation 89 2. The Primal Simplex Method 91 3. An Example 96 4. The Dual Simplex Method 101 5. Two-Phase Methods 104 6. Negative Transpose Property 105 Exercises 108 Notes 109

Chapter 7. Sensitivity and Parametric Analyses 111 1. Sensitivity Analysis 111 2. Parametric Analysis and the Homotopy Method 115 3. The Parametric Self-Dual Simplex Method 119 Exercises 120 Notes 124

Chapter 8. Implementation Issues 125 1. Solving Systems of Equations:LU -Factorization 126 2. Exploiting Sparsity 130 3. Reusing a Factorization 136 4. Performance Tradeoffs 140 5. Updating a Factorization 141 6. Shrinking the Bump 145

CONTENTS ix

7. Partial Pricing 146 8. Steepest Edge 147 Exercises 149 Notes 150

Chapter 9. Problems in General Form 151 1. The Primal Simplex Method 151 2. The Dual Simplex Method 153 Exercises 159 Notes 160

Chapter 10. Convex Analysis 161 1. Convex Sets 161 2. Carath́eodory’s Theorem 163 3. The Separation Theorem 165 4. Farkas’ Lemma 167 5. Strict Complementarity 168 Exercises 170 Notes 171

Chapter 11. Game Theory 173 1. Matrix Games 173 2. Optimal Strategies 175 3. The Minimax Theorem 177 4. Poker 181 Exercises 184 Notes 187

Chapter 12. Regression 189 1. Measures of Mediocrity 189 2. Multidimensional Measures: Regression Analysis 191 3. L2-Regression 193 4. L1-Regression 195 5. Iteratively Reweighted Least Squares 196 6. An Example: How Fast is the Simplex Method? 198 7. Which Variant of the Simplex Method is Best? 202 Exercises 203 Notes 208

Part 2. Network-Type Problems 211

Chapter 13. Network Flow Problems 213 1. Networks 213

x CONTENTS

2. Spanning Trees and Bases 216 3. The Primal Network Simplex Method 221 4. The Dual Network Simplex Method 225 5. Putting It All Together 228 6. The Integrality Theorem 231 Exercises 232 Notes 240

Chapter 14. Applications 241 1. The Transportation Problem 241 2. The Assignment Problem 243 3. The Shortest-Path Problem 244 4. Upper-Bounded Network Flow Problems 247 5. The Maximum-Flow Problem 250 Exercises 252 Notes 257

Chapter 15. Structural Optimization 259 1. An Example 259 2. Incidence Matrices 261 3. Stability 262 4. Conservation Laws 264 5. Minimum-Weight Structural Design 267 6. Anchors Away 269 Exercises 272 Notes 272

Part 3. Interior-Point Methods 275

Chapter 16. The Central Path 277 Warning: Nonstandard Notation Ahead 277 1. The Barrier Problem 277 2. Lagrange Multipliers 280 3. Lagrange Multipliers Applied to the Barrier Problem 283 4. Second-Order Information 285 5. Existence 285 Exercises 287 Notes 289

Chapter 17. A Path-Following Method 291 1. Computing Step Directions 291 2. Newton’s Method 293 3. Estimating an Appropriate Value for the Barrier Parameter 294

CONTENTS xi

4. Choosing the Step Length Parameter 295 5. Convergence Analysis 296 Exercises 302 Notes 306

Chapter 18. The KKT System 307 1. The Reduced KKT System 307 2. The Normal Equations 308 3. Step Direction Decomposition 310 Exercises 313 Notes 313

Chapter 19. Implementation Issues 315 1. Factoring Positive Definite Matrices 315 2. Quasidefinite Matrices 319 3. Problems in General Form 325 Exercises 331 Notes 331

Chapter 20. The Affine-Scaling Method 333 1. The Steepest Ascent Direction 333 2. The Projected Gradient Direction 335 3. The Projected Gradient Direction with Scaling 337 4. Convergence 341 5. Feasibility Direction 343 6. Problems in Standard Form 344 Exercises 345 Notes 346

Chapter 21. The Homogeneous Self-Dual Method 349 1. From Standard Form to Self-Dual Form 349 2. Homogeneous Self-Dual Problems 350 3. Back to Standard Form 360 4. Simplex Method vs Interior-Point Methods 363 Exercises 367 Notes 368

Part 4. Extensions 371

Chapter 22. Integer Programming 373 1. Scheduling Problems 373 2. The Traveling Salesman Problem 375 3. Fixed Costs 378

xii CONTENTS

4. Nonlinear Objective Functions 378 5. Branch-and-Bound 380 Exercises 392 Notes 393

Chapter 23. Quadratic Programming 395 1. The Markowitz Model 395 2. The Dual 399 3. Convexity and Complexity 402 4. Solution Via Interior-Point Methods 404 5. Practical Considerations 406 Exercises 409 Notes 411

Chapter 24. Convex Programming 413 1. Differentiable Functions and Taylor Approximations 413 2. Convex and Concave Functions 414 3. Problem Formulation 414 4. Solution Via Interior-Point Methods 415 5. Successive Quadratic Approximations 417 6. Merit Functions 417 7. Parting Words 421 Exercises 421 Notes 423

Appendix A. Source Listings 425 1. The Self-Dual Simplex Method 426 2. The Homogeneous Self-Dual Method 429

Answers to Selected Exercises 433

Bibliography 435

Index 443

Preface

This book is about constrained optimization. It begins with a thorough treatment of linear programming and proceeds to convex analysis, network flows, integer pro- gramming, quadratic programming, and convex optimization. Along the way, dynamic programming and the linear complementarity problem are touched on as well.

The book aims to be a first introduction to the subject. Specific examples and concrete algorithms precede more abstract topics. Nevertheless, topics covered are developed in some depth, a large number of numerical examples are worked out in detail, and many recent topics are included, most notably interior-point methods. The exercises at the end of each chapter both illustrate the theory and, in some cases, extend it.

Prerequisites.The book is divided into four parts. The first two parts assume a background only in linear algebra. For the last two parts, some knowledge of multi- variate calculus is necessary. In particular, the student should know how to use La- grange multipliers to solve simple calculus problems in 2 and 3 dimensions.

Associated software.It is good to be able to solve small problems by hand, but the problems one encounters in practice are large, requiring a computer for their solution. Therefore, to fully appreciate the subject, one needs to solve large (practical) prob- lems on a computer. An important feature of this book is that it comes with software implementing the major algorithms described herein. At the time of writing, software for the following five algorithms is available:

• The two-phase simplex method as shown in Figure 6.1. • The self-dual simplex method as shown in Figure 7.1. • The path-following method as shown in Figure 17.1. • The homogeneous self-dual method as shown in Figure 21.1. • The long-step homogeneous self-dual method as described in Exercise 21.4.

The programs that implement these algorithms are written in C and can be easily compiled on most hardware platforms. Students/instructors are encouraged to install and compile these programs on their local hardware. Great pains have been taken to make the source code for these programs readable (see Appendix A). In pa

Recommended