connecting calculus to linear programming · linear programming simplex method applications...
TRANSCRIPT
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Connecting Calculus to Linear Programming
Marcel Y. Blais, Ph.D.Worcester Polytechnic Institute
Dept. of Mathematical Sciences
July 2017
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Motivation
Goal: To help students make connections between high schoolmath and real world applications of mathematics.
Linear programming is based on a simple idea fromcalculus.
We can connect calculus to real-world industrial problems.
We’ll explore the some basic principles of linearprogramming and some modern-day applications.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Motivation
Goal: To help students make connections between high schoolmath and real world applications of mathematics.
Linear programming is based on a simple idea fromcalculus.
We can connect calculus to real-world industrial problems.
We’ll explore the some basic principles of linearprogramming and some modern-day applications.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Motivation
Goal: To help students make connections between high schoolmath and real world applications of mathematics.
Linear programming is based on a simple idea fromcalculus.
We can connect calculus to real-world industrial problems.
We’ll explore the some basic principles of linearprogramming and some modern-day applications.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Continuous Functions on Closed Intervals
x-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
y
-2
-1
0
1
2
3
4
5
a b
What can we say about a continuous function on a closedinterval?
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Continuous Functions on Closed Intervals
x-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
y
-2
-1
0
1
2
3
4
5
a b
Theorem
If f is continuous on [a, b] then f attains both an absoluteminimum value and an absolute maximum value on [a, b].
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Continuous Functions on Closed Intervals
x-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
y
-2
-1
0
1
2
3
4
5
a b
Theorem
If f is continuous on [a, b] then f attains both an absoluteminimum value and an absolute maximum value on [a, b].Further these occur at critical points of f or endpoints of [a, b].
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Linear Case
Theorem
If f is continuous linear on [a, b] then f attains both anabsolute minimum value and an absolute maximum value on[a, b], and these occur at endpoints of [a, b].
x0 1 2 3 4 5 6 7 8 9 10
y
0
5
10
15
20
25
30
35
a b
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Higher-Dimensional Optimization
1
0.8
0.6
0.4
x
0.2
000.10.2
y
0.30.40.50.60.70.80.91
-0.2
-0.5
-0.4
-0.3
0.2
-0.1
0
0.1
z
What does this mean for a continuous function f (x , y) on aclosed and bounded plane region R?
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Higher-Dimensional Optimization
1
0.8
0.6
0.4
x
0.2
000.10.2
y
0.30.40.50.60.70.80.91
-0.2
-0.5
-0.4
-0.3
0.2
-0.1
0
0.1
z
Theorem
If f (x , y) is continuous on closed and bounded plane region Rthen f attains both an absolute minimum value and an absolutemaximum value on R. Further, these values occur either atcritical points of f in the interior of R or on the boundary of R.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Higher-Dimensional Optimization
1
0.8
0.6
0.4
x
0.2
000.10.2
y
0.30.40.50.60.70.80.91
-0.2
-0.5
-0.4
-0.3
0.2
-0.1
0
0.1
z
Theorem
If f (x1, x2, . . . , xn) is continuous on closed and bounded regionR ⊂ Rn then f attains both an absolute minimum value and anabsolute maximum value on R. Further, these values occureither at critical points of f in the interior of R or on theboundary of R.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Higher-Dimensional Linearity
10.90.80.70.60.50.40.3
x
0.20.100
0.2
y
0.4
0.6
0.8
4.5
0
0.5
1
5
1.5
2
2.5
3
3.5
4
1
z
Theorem
If f (x1, x2, . . . , xn) is linear on closed and bounded regionR ⊂ Rn defined by a system of linear constraints then f attainsboth an absolute minimum value and an absolute maximumvalue on R. Further, these values occur on the boundary of R.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Higher-Dimensional Linearity
10.90.80.70.60.50.40.3
x
0.20.100
0.2
y
0.4
0.6
0.8
4.5
0
0.5
1
1.5
2
2.5
3
3.5
4
5
1
z
Theorem
If f (x1, x2, . . . , xn) is linear on closed and bounded regionR ⊂ Rn defined by a system of linear constraints then f attainsboth an absolute minimum value and an absolute maximumvalue on R. Further, these values occur on corner points of theboundary of R.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Higher-Dimensional Linearity
1
0.8
0.6
0.4
x0.2
00
0.1
0.2
0.3
y
0.4
0.5
0.6
0.7
0.8
0.9
0.1
0.6
0
1
0.9
0.8
0.7
0.2
0.5
0.4
0.3
1
z
Example: A possible feasible region f (x , y , z) = ax + by + czsubjuect to 5 linear constraints.
A solution to minimize f over this region will occur at a corner.Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Linear Programming
The Simplex Method was developed by George Dantzig in1947.
“programming” synonymous with “optimization”.
Algorithm to traverse the corner points of the feasiblepolyhedron for a linear programming problem to find anoptimal feasible solution.
Standard form for a linear programming problem:
min xTc such that Ax ≤ b and x ≥ 0. (1)
x, c ∈ Rn,b ∈ Rm,A ∈ Rm×n - n variables and m constraints.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Simplex Method
min xTc such that Ax ≤ b and x ≥ 0.
For each constraint (row i of A), add a new slackvariable yi .
New system: min xTc such that [A + I ]
[xy
]= b,
x ≥ 0, y ≥ 0.
Underdetermined (more variables than equations). A + I
The slack now in the system is the key.
Each slack variable is associated with a constraintboundary
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Simplex Method
Simplex algorithm:
Split entries of
[xy
]into 2 subsets:
Basic variables: xB ∈ Rm (non-zero valued)Non-basic variables: xN ∈ Rn (zero valued)Represents a corner point of the feasible region.
If not optimal, move to an adjacent corner point byswapping one entry in xB with one entry in xN andre-solving the system of equations.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Simplex Geometry
1
0.8
0.6
0.4
x0.2
00
0.1
0.2
0.3
y
0.4
0.5
0.6
0.7
0.8
0.9
0.1
0.6
0
1
0.9
0.8
0.7
0.2
0.5
0.4
0.3
1
z
Figure: Feasible Region
Example: min f (x1, x2, x3) = ax1 + bx2 + cx3 subject to 5 linearconstraints.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Simplex Geometry
1
0.8
0.6
0.4
x0.2
00
0.1
0.2
0.3
y
0.4
0.5
0.6
0.7
0.8
0.9
0.1
0.6
0
1
0.9
0.8
0.7
0.2
0.5
0.4
0.3
1
z
Example: min f (x1, x2, x3) = ax1 + bx2 + cx3 subject to 5 linearconstraints.
Choose initial basis to correspond to the origin.Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Simplex Geometry
1
0.8
0.6
0.4
x0.2
00
0.1
0.2
0.3
y
0.4
0.5
0.6
0.7
0.8
0.9
0.1
0.6
0
1
0.9
0.8
0.7
0.2
0.5
0.4
0.3
1
z
Example: min f (x1, x2, x3) = ax1 + bx2 + cx3 subject to 5 linearconstraints.
If the objective function is not optimal, move to a betteradjacent corner.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Simplex Geometry
1
0.8
0.6
0.4
x0.2
00
0.1
0.2
0.3
y
0.4
0.5
0.6
0.7
0.8
0.9
0.1
0.6
0
1
0.9
0.8
0.7
0.2
0.5
0.4
0.3
1
z
Example: min f (x1, x2, x3) = ax1 + bx2 + cx3 subject to 5 linearconstraints.
Repeat until the objective function is minimized (no remainingadjacent corners will reduce it).
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Application: Shipping Network
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem
K warehouses: S1,S2, . . . ,SK
L retail locations: D1,D2, . . . ,DL
Cost to ship x amount of product from Si to Dj is cijx
Warehouse Si can supply si amount of product
Retail location Dj demands dj amount of product
Problem: Design a shipping schedule that satisfies demandat all retail locations at a minimal cost.
Example: As of July 2017, Target Corp. has 37distribution centers and 1, 802 stores.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem
K warehouses: S1,S2, . . . ,SK
L retail locations: D1,D2, . . . ,DL
Cost to ship x amount of product from Si to Dj is cijx
Warehouse Si can supply si amount of product
Retail location Dj demands dj amount of product
Problem: Design a shipping schedule that satisfies demandat all retail locations at a minimal cost.
Example: As of July 2017, Target Corp. has 37distribution centers and 1, 802 stores.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem
K warehouses: S1,S2, . . . ,SK
L retail locations: D1,D2, . . . ,DL
Cost to ship x amount of product from Si to Dj is cijx
Warehouse Si can supply si amount of product
Retail location Dj demands dj amount of product
Problem: Design a shipping schedule that satisfies demandat all retail locations at a minimal cost.
Example: As of July 2017, Target Corp. has 37distribution centers and 1, 802 stores.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem
K warehouses: S1,S2, . . . ,SK
L retail locations: D1,D2, . . . ,DL
Cost to ship x amount of product from Si to Dj is cijx
Warehouse Si can supply si amount of product
Retail location Dj demands dj amount of product
Problem: Design a shipping schedule that satisfies demandat all retail locations at a minimal cost.
Example: As of July 2017, Target Corp. has 37distribution centers and 1, 802 stores.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem
K warehouses: S1,S2, . . . ,SK
L retail locations: D1,D2, . . . ,DL
Cost to ship x amount of product from Si to Dj is cijx
Warehouse Si can supply si amount of product
Retail location Dj demands dj amount of product
Problem: Design a shipping schedule that satisfies demandat all retail locations at a minimal cost.
Example: As of July 2017, Target Corp. has 37distribution centers and 1, 802 stores.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem
K warehouses: S1,S2, . . . ,SK
L retail locations: D1,D2, . . . ,DL
Cost to ship x amount of product from Si to Dj is cijx
Warehouse Si can supply si amount of product
Retail location Dj demands dj amount of product
Problem: Design a shipping schedule that satisfies demandat all retail locations at a minimal cost.
Example: As of July 2017, Target Corp. has 37distribution centers and 1, 802 stores.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem
K warehouses: S1,S2, . . . ,SK
L retail locations: D1,D2, . . . ,DL
Cost to ship x amount of product from Si to Dj is cijx
Warehouse Si can supply si amount of product
Retail location Dj demands dj amount of product
Problem: Design a shipping schedule that satisfies demandat all retail locations at a minimal cost.
Example: As of July 2017, Target Corp. has 37distribution centers and 1, 802 stores.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Network
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Network
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem Mathematical Model
Constraints
xij ≥ 0 - amount of product shipped from Si to Dj
Amount leaving Si : xi1 + xi2 + · · ·+ xiL ≤ si
Amount entering Dj : x1j + x2j + · · ·+ xKj = dj
Objective
Total cost of shipping schedule:K∑i=1
L∑j=1
cijxij
Minimize cost of shipping such that demand is satisfied
Target example: 37× 1, 802 = 66, 674 variables
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem Mathematical Model
Constraints
xij ≥ 0 - amount of product shipped from Si to Dj
Amount leaving Si : xi1 + xi2 + · · ·+ xiL ≤ si
Amount entering Dj : x1j + x2j + · · ·+ xKj = dj
Objective
Total cost of shipping schedule:K∑i=1
L∑j=1
cijxij
Minimize cost of shipping such that demand is satisfied
Target example: 37× 1, 802 = 66, 674 variables
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem Mathematical Model
Constraints
xij ≥ 0 - amount of product shipped from Si to Dj
Amount leaving Si : xi1 + xi2 + · · ·+ xiL ≤ si
Amount entering Dj : x1j + x2j + · · ·+ xKj = dj
Objective
Total cost of shipping schedule:K∑i=1
L∑j=1
cijxij
Minimize cost of shipping such that demand is satisfied
Target example: 37× 1, 802 = 66, 674 variables
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem Mathematical Model
Constraints
xij ≥ 0 - amount of product shipped from Si to Dj
Amount leaving Si : xi1 + xi2 + · · ·+ xiL ≤ si
Amount entering Dj : x1j + x2j + · · ·+ xKj = dj
Objective
Total cost of shipping schedule:K∑i=1
L∑j=1
cijxij
Minimize cost of shipping such that demand is satisfied
Target example: 37× 1, 802 = 66, 674 variables
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem Mathematical Model
Constraints
xij ≥ 0 - amount of product shipped from Si to Dj
Amount leaving Si : xi1 + xi2 + · · ·+ xiL ≤ si
Amount entering Dj : x1j + x2j + · · ·+ xKj = dj
Objective
Total cost of shipping schedule:K∑i=1
L∑j=1
cijxij
Minimize cost of shipping such that demand is satisfied
Target example: 37× 1, 802 = 66, 674 variables
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem Mathematical Model
Constraints
xij ≥ 0 - amount of product shipped from Si to Dj
Amount leaving Si : xi1 + xi2 + · · ·+ xiL ≤ si
Amount entering Dj : x1j + x2j + · · ·+ xKj = dj
Objective
Total cost of shipping schedule:K∑i=1
L∑j=1
cijxij
Minimize cost of shipping such that demand is satisfied
Target example: 37× 1, 802 = 66, 674 variables
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem Mathematical Model
Amount leaving Si : xi1 + xi2 + · · ·+ xiL ≤ si
Amount entering Dj : x1j + x2j + · · ·+ xKj = dj
A =
1 . . . 1
1 . . . 1. . .
1 . . . 1
I I I
minK∑i=1
L∑j=1
cijxij such that Ax≤=
[sd
], x ≥ 0.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
Shipping Problem Mathematical Model
Amount leaving Si : xi1 + xi2 + · · ·+ xiL ≤ si
Amount entering Dj : x1j + x2j + · · ·+ xKj = dj
A =
1 . . . 1
1 . . . 1. . .
1 . . . 1
I I I
minK∑i=1
L∑j=1
cijxij such that Ax≤=
[sd
], x ≥ 0.
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
TV Project - MA 3231
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
TV Project - MA 3231
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
TV Project - MA 3231
Blais MIST 2017
MIST 2017
Blais
Optimization
Linearity
Higher-DimensionalOptimization
Higher-DimensionalLinearity
LinearProgramming
Simplex Method
Applications
References
1 Blais, Marcel., MA 3231 Linear Programming LectureNotes, WPI Department of Mathematical Sciecnes, 2016.
2 Griva, Igor, Stephen G. Nash, and Ariela Sofer., Linear andNonlinear Optimization, Second Edition. SIAM, 2009.
3 Hillier, Frederick and Gerald Lieberman., Introduction toOperations Research, Tenth Edition. McGraw HillEducation, 2015.
Blais MIST 2017