linear programming: formulations, geometry and simplex method yi zhang january 21 th, 2010
DESCRIPTION
Inequality form of LPs An LP in inequality form (x in R n ) Matrix notationTRANSCRIPT
![Page 1: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/1.jpg)
Linear Programming: Formulations, Geometry
and Simplex MethodYi Zhang
January 21th, 2010
![Page 2: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/2.jpg)
Outline Different forms of LPs Geometry of LPs Solving an LP: Simplex Method Summary
![Page 3: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/3.jpg)
Inequality form of LPs An LP in inequality form (x in Rn)
Matrix notation
![Page 4: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/4.jpg)
Why is inequality form useful? Intuitive: sketching an LP Understand the geometry of LPs
![Page 5: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/5.jpg)
Standard form of LPs An LP in standard form
Matrix notation
![Page 6: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/6.jpg)
Why is standard form useful? Easy for computers to operate
Search “corners” of the feasible region Transform of constraints E.g., simplex method works in standard form
![Page 7: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/7.jpg)
Inequality form standard form Add slack variables
![Page 8: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/8.jpg)
Stanford form inequality form Make and drop slack variables
![Page 9: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/9.jpg)
General form of LPs An LP in general form
Transform to Inequality form: sketching, geometry Standard form: simplex method
![Page 10: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/10.jpg)
Outline Different forms of LPs Geometry of LPs
Half space and polyhedron Extreme points, vertices and basic feasible solution Optimality of LPs at extreme points
Solving an LP: Simplex Method Summary
![Page 11: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/11.jpg)
Half space and polyhedron An inequality constraint a half space A set of inequality constraints a polyhedron
[Boyd & Vandenberghe]
![Page 12: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/12.jpg)
Geometry of LPs An LP in inequality form (x in Rn)
[Boyd & Vandenberghe]
![Page 13: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/13.jpg)
Geometry of LPs An LP in inequality form (x in Rn)
Also, an LP can be Infeasible Unbounded
![Page 14: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/14.jpg)
Geometry of LPs Three important concepts of an LP
Extreme points Vertices Basic feasible solutions
[Boyd & Vandenberghe]
![Page 15: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/15.jpg)
Concept 1: extreme points A point x in P is an extreme point:
It can not be represented as , Not in the middle of any other two points in P
[Boyd & Vandenberghe]
![Page 16: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/16.jpg)
Concept 2: vertices A point x in P is a vertex:
It is uniquely optimal for some objective function
[Boyd & Vandenberghe]
![Page 17: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/17.jpg)
Concept 3:Basic feasible solutions An inequality constraint is active at x:
The constraint holds with equality at x
[Boyd & Vandenberghe]
![Page 18: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/18.jpg)
Concept 3:Basic feasible solutions A point x is a basic solution:
There exist n linearly independent active constraints at x
[Boyd & Vandenberghe]
![Page 19: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/19.jpg)
Concept 3:Basic feasible solutions A point x is a basic feasible solution:
A basic solution that satisfies all constraints (i.e., stay in P)
[Boyd & Vandenberghe]
![Page 20: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/20.jpg)
Equivalence of three definitions Extreme points, vertices and basic feasible
solution are equivalent Extreme points: not in the middle of any two Vertices: uniquely optimal for some objective Basic feasible solutions: n indep. active constraints
Intuition of proofs Vertex extreme point Extreme point basic feasible solution Basic feasible solution vertex
![Page 21: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/21.jpg)
Why are these definitions useful? Equivalent ways to define “corners”
Extreme points Vertices Basic feasible solutions
Optimality of LPs at “corners”
![Page 22: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/22.jpg)
Optimality of extreme points Given an LP
If The polyhedron P has at least one extreme point Optimal solutions exist (not unbounded or
infeasible) Then
At least one optimal solution is an extreme point
![Page 23: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/23.jpg)
Search basic feasible solutions! Solve an LP: search over extreme points Extreme points basic feasible solutions
Search over basic feasible solutions! Basic idea of simplex method
![Page 24: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/24.jpg)
Outline Different forms of LPs Geometry of LPs Solving an LP: Simplex Method Summary
![Page 25: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/25.jpg)
Search basic feasible solutions Optimality of extreme points Extreme points basic feasible solutions Solve LP: search over basic feasible solutions!
![Page 26: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/26.jpg)
Search basic solutions in standard form Simplex method operates in standard form
Understand the geometry in inequality form Search basic solutions in standard form ?
![Page 27: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/27.jpg)
Inequality form vs. standard form
![Page 28: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/28.jpg)
Search basic solutions in standard form
How to get a basic solution in standard form? Pick a basis (m independent columns) Fix the rest (n-m) non-basic vars to 0 Solve for m basic vars
![Page 29: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/29.jpg)
Search basic solutions in standard form
![Page 30: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/30.jpg)
Trick: monitor the objective function during the search
![Page 31: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/31.jpg)
Simplex method Simplex method
Search over basic feasible solutions Repeatedly move to a neighbor bfs to improve
objective Stop at “local” optimum
![Page 32: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/32.jpg)
Simplex method: an example Maximize Z = 5x1 + 2x2 + x3
x1 + 3x2 - x3 ≤ 6,
x2 + x3 ≤ 4,
3x1 + x2 ≤ 7,
x1, x2, x3 ≥ 0.
![Page 33: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/33.jpg)
Simplex method: an example Maximize Z = 5x1 + 2x2 + x3
x1 + 3x2 - x3 + x4 = 6,
x2 + x3 + x5 = 4,
3x1 + x2 + x6 = 7,
x1, x2, x3, x4, x5, x6 ≥ 0.
Go through the example …
![Page 34: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/34.jpg)
Outline Different forms of LPs Geometry of LPs Solving an LP: Simplex Method Summary
![Page 35: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/35.jpg)
Summary Different forms of LPs
Inequality, standard, general .. Geometry of LPs
Focus on Inequality form LPs Half space and polyhedron Extreme points, vertices and basic feasible
solutions – three definitions of “corners” Optimality at “corners”
![Page 36: Linear Programming: Formulations, Geometry and Simplex Method Yi Zhang January 21 th, 2010](https://reader036.vdocuments.site/reader036/viewer/2022062413/5a4d1b737f8b9ab0599b620e/html5/thumbnails/36.jpg)
Summary Simplex method
Operate in standard form Search over “corners” Start from a basic feasible solution (i.e., a basis) Search over neighboring basis
Improve the objective Keep feasibility
Stop at local(?) optimum