mathematical optimization: introductionmathematical modelling model: a structure used to represent...
TRANSCRIPT
Mathematical Optimization: introduction
Carlo Mannino
University of Oslo, INF-MAT5360 - Autumn 2011 (Mathematical optimization)
Practical Information
Lecturer: Carlo Mannino, [email protected]
2 to 4 obligatory projects (assignments)
Written exams (theoretical arguments from the lectures, exercises (including, possibly, short proofs of properties).
Structure of the lesson
14:15 – 15:00 main topic
15:15 – 16 main topic
16:15 – 17 Exercises and questions (unless more time is needed)
Normally the lecture notes will be posted on the web-page the day before the lecture.
Reference Texts
Course Reference Texts:
Alexander Schrijver, A course in combinatorial optimization, http://homepages.cwi.nl/~lex/files/dict.pdf (Chap. 1)
Geir Dahl, Carlo Mannino, Notes on combinatorial optimization http://heim.ifi.uio.no/~geird/comb_notes.pdf
Geir Dahl, An introduction to convexity http://heim.ifi.uio.no/~geird/conv.pdf
Geir Dahl, Network flows and combinatorial matrix theory http://heim.ifi.uio.no/~geird/CombMatrix.pdf
Bradley, Hax, Magnanti, Applied Mathematical Programming http://web.mit.edu/15.053/www/AMP-Chapter-01.pdf
ILOG OPL Development Studio 5.5
Mathematical Models and Optimization
Mathematical Modelling
Model: a structure used to represent and analyze specific features or characteristics of concrete objects or systems
Mathematical Model: makes use of the algebraic formalism, i.e. variables, equations, inequalities, to represent concrete relations.
Good motives to build mathematical models:
- Mathematical analysis may reveal non-apparent properties
- Find quantitative solutions which may be used in practical decisions
- Allow for quick simulations (what-if scenarios)
Mathematical Optimization
Optimization Model: you want to find the best possible out of a set of feasible alternatives, according to some evaluation criterion
Mathematical Optimization Model: min {𝑐 𝑥 : 𝑥 ∈ 𝑆}
𝑐: 𝑆 → ℝ objective function, 𝑆 feasible solutions set
Typically S•⊆ ℝn is expressed by means of constraints, i.e.
𝑆 = 𝑥 ∈ ℝn: 𝑔𝑖 𝑥 ≤ 𝑏𝑖 , 𝑖 = 1, … , 𝑚 with 𝑔𝑖 : ℝ
n → ℝ
𝑔𝑖 , 𝑐 linear functions → linear programming (VS non-linear progr.)
and S ⊆ ℤ𝑛→ integer linear programming
|S| finite → combinatorial optimization
Convex programming (Linear programming, Second order cone
programming, Semidefinite programming, Conic programming,
Geometric programming), Integer Programming, Quadratic
programming , Fractional programming, Nonlinear programming,
Stochastic programming, Robust programming, Combinatorial
optimization, Infinite-dimensional optimization, Metaheuristics,
Constraint satisfaction, Constraint programming, Disjunctive
programming, Dynamic programming, Multi-objective optimization, …
Major Subfields (Wikipedia)
Mathematical Optimization (or Mathematical Programming): a list of major subfields (according to Wikipedia)
The course will focus on Combinatorial Optimization and its links to linear and integer programming.
Mathematical Optimization provides the theoretical and computational background for Operations Research (OR) (= Management Science or Decision Science)
Operations Research
OR (Wikipedia): an interdisciplinary mathematical science that focuses on the effective use of technology by organizations; it arrives at optimal or near-optimal solutions to complex decision-making problems.
… And what field is all of “management, scientific and technical”?
Operations Research, of course!
Michael Trick, Dean of Tepper School of Business, www.mat.tepper.cmu.edu/blog/
Management, Scientific, and Technical Consulting Services will grow
by 84% from 2008 to 2018
According to the US Bureau of Labour Statistics www.bls.gov:
(www.npr.org/templates/story/story.php?storyId=121875404)
Input problem
Design solution algorithm
Analyze solutions
Revise
algorithm
Revise model
Analyze model
Build model
Optimization Problem
)(min xfSx
Modelling approach
Bad quality solutions
Unusable solutions
Example: Product Mix
A factory produces 5 products: (PR1, …, PR5)
PR1 PR2 PR3 PR4 PR5
550 600 350 400 200
Unit profit (euro)
Each unit of product must be processed by two machines
Unit time (hours)
Process PR1 PR2 PR3 PR4 PR5
Grinding 12 20 0 25 15
Drilling 10 8 16 0 0
Weekly available hours: grinding 288, drilling 192.
Want: find the blend of products which maximized weekly profit.
Product Mix: a linear program
Decision variables xi : quantity of PRi, i = 1, ….,5
Profit: 550 x1 + 600 x2 + 350 x3 + 400 x4 + 200 x5
Constraints: we cannot exceed grinding and drilling available hours
Grinding: 12 x1 + 20 x2 + 25 x4 + 15 x5 ≤ 288
Drilling: 10 x1 + 8 x2 + 16 x3 ≤ 192
𝑚𝑎𝑥 +550𝑥1 +600𝑥2 +350𝑥3 +400𝑥4 +200𝑥5
+12𝑥1 +20𝑥2 +25𝑥4 +15𝑥1 ≤ 288
+10𝑥1 +8𝑥2 +16𝑥3 ≤ 192
𝑥𝑖 ≥ 0 𝑖 = 1, … , 5
Canonical lp form: 𝑚𝑎𝑥 𝑐𝑇𝑥
𝐴𝑥 ≤ 𝑏𝑥 ≥ 𝑂
The complete LP-program:
Linear Programming: applications
There is a very large number of real-life problems that can be modeled by LP. Here is some examples
Petroleum Industry (refinery optimization problem)
Manufacturing Industry (lot sizing, stock control)
Transport, distribution, transhipment
Finance (portfolio optimization)
Energy
Defence
….
Example: project selection
• Projects A e B
• Budget constraint < D =10
• Costs cA =5, cB =8
Project selection problem:
Find the fraction of each project to
buy so as to satisfy the budget
constraint and maximize profit.
max 10 xA + 20 xB
5 xA + 8 xB < 10
0 < xA < 1
0 < xB < 1
• Profits wA = 10 e wB = 20
Suppose first we can activate fractions of projects (e.g. shares)
Decision variables xA , xB : project fraction
xA = 1
xB = 1
1
1
2
Feasible set 0.4
! If there is an optimal solution then there is one in a vertex
Example 0,1 LP: project selection
• Projects A e B
• Budget constraint < D =10
• Costs cA =5, cB =8
B
AS
1
0 ,
0
1 ,
0
0
Project selection problem:
Find a selection of projects
satisfying the budget constraint
and maximizing profit.
c({}) = 0, c({A}) = 5, c({B}) = 8,
c({A, B}) = 13 > D {A, B} not feasible
Feasible Solutions {{}, {A}, {B}}
xS
max 10 xA + 20 xB
S incident vectors of
the feasible solutions
• Profits wA = 10 e wB = 20
The problem can be written as a 0,1 linear program.
Solving 0,1 linear program is “difficult” (in general).
0,1 LP: formulations
S = {(0,0)T, (0,1)T, (1,0)T}
𝑚𝑎𝑥 +10𝑥𝐴 +20𝑥𝐵
+5𝑥𝐴 +8𝑥𝐵 ≤ 10
𝑥𝐴, 𝑥𝐵 ∈ {0,1}
We will show that 0,1 LP’s and Combinatorial Optimization problems (with linear objective function) coincide.
A very large number of real-life problems can be modeled as combinatorial optimization problems.
0
1
0
0
1
0
1
1
2
0.4
S is the set of 0,1 points contained in the region identified by the linear constraints and the box constraints (0 ≤ x ≤ 1)
LP-relaxation: remove integrality.
0,1 linear program
Linear objective functions
Linear constraints
0,1 variables
Combinatorial optimization: applications
17
Network design
Example: connectivity
You need to connect a source s (of
water, packets, …) to a number of
locations (farms, computers).
Each connection (pipe, fiber) has a cost
WANT: find a minimum cost network
connecting all locations to the source
s
2
3 7
5
2 1
3 s 11
20
1
7
1
4
s
18
Example: flow
RM MI FR ZU
RM - 30 - 80
MI - - 40 14
FR - 14 - 14
ZU - - 30 -
Roma
Milano
Zurich
Frankfurt
Expected demand:
from
to
Roma-Milano: 120 sits
Milano-Zurich: 150 sits
Frankfurt -Milano: 30 sits
Zurich- Frankfurt : 80 sits
•Flights (available sits):
30+80<120 •80+40+
•14+14 <150
•14+14<30
•40+30<80
Passengers should
reach their destinations
be assigned to departing flights
…. Satisfying capacity constraints
•WAITING LIST
Example: Project Scheduling
1
4
2
3
6
8
5
1
0
7
9 SSmin(2)
FFmax(3)
FFmax(4)
SSmin(0)
SSmin(2)
SSmin(1)
SSmin(1)
SSmin(2)
SFmin(10) SFmin(6)
SFmax(4)
FSmin(4)
FFmin(2)
FSmin(0)
FFmin(2)
FSmin(2)
0
7
2
3 4
4
6
4
0
5
Projects decompose into activities
Precedence Relations exist
between activities.
WANT: find a schedule of the activities satisfying all precedence
constraints and minimizing the project completion time
Activities may require resources,
which in turn may be limited
5 7 SFmin(6)
= “7 must Start at least 6 hours after 5 is Finished”
Simple version equivalent to the shortest path problem (LP).
Example: Job-Shop Scheduling
• A product (job) must be processed on
different machines in a production line
• Processing a job on a machine is
called operation
• Each machine can process at most k
jobs at a time.
• WANT: find a schedule of
the operations satisfying
machine capacities and
additional precedence
constraints.
Example: vehicle routing
Transfer goods from origins to destinations
• Minimizing transportation costs
• Satisfying:
• - constraints on vehicle capacities
• - connectivity constraints
• …
Several parameters are involved
1. Origin and destination position
2. Demand level
3. Fleet size
…
A
C B
• Every vehicle visits a subset of clients and return to depot
• A Solution is a set of (Hamilton) tours
Vehicle routing: solutions
A famous instance
Standard test instance G-n262-k25 (Gillett & Johnson 1976)
"The world record" for G-n262-k25: 5685 vs. 6119 (SINTEF 2003)
Inventory routing problem (IRP):
establish when and how much
produce and stock
Inventory level,
production
•Time •LB
•UB
Mixed problem: plan production and
transport of goods so as production is
never halted and warehouses are
never empty.
Lot sizing and rounting
•Production port
•Consumption port
•Product 1
•Product 2
•Kjøpsvik
•Brevik
•Alta
•Mo i
Rana
•Trondheim
•Karmøy
•Årdal
A Norwegian case (SINTEF)
Mathematical Optimization Course
Basic combinatorial optimization problems: shortest path, minimum spanning tree, maximum flow, minimum cut.
Connections with linear programming and polyhedral theory: convexity, representation of polyhedra, formulations
Integer Polyhedra
Methods: heuristic algorithms
Methods: exact approaches
Outline of the course
LP-duality, basic relations
Dual Pair
Given a linear program (primal) (P)
min cTx
Ax b
x 0
(P)
A Rm×n, c Rn , b Rm x Rn
We associate a dual problem (D)
OBS a dual variable for each primal constraint, a dual constraint for each primal variable
y Rm
max bTy
ATy c
y 0
(D)
(P) can be infeasible, or unbounded, or the minimum is finite.
Example
y R3
0
1
4
1
4
1
A
1
1c
1
8
5
b
014
141TA
min x1 + x2
x1 + 4x2 5
4x1 + x2 8
x1 1
x1 , x2 0, x R2
max 5y1 + 8y2 + y3
y1 + 4y2 + y3 1
4y1 + y2 1
y1 , y2 , y3 0, y R3
General Form
PRIMAL DUAL
constraints
min max
bi
bi
= bi
0
constraints variables
variables 0
free
0
0
free
cj
cj
= cj
Dual variables correspond to primal constraints
Dual constraints correspond to primal variables
Exercise: Derive some of these rules from the canonical forms
Strong duality
(i) if (P) is unbounded then (D) is infeasible.
(ii) if (P) is infeasible then (D) is either unbounded or infeasible
(iii) if (P) has an optimal solution x* then (D) has an optimal
solution y* and
cTx* = bTy*
Proposition (strong duality)
(i) Let x’ be primal feasible and y’ be dual feasible. If
cTx’ = bTy’
Then x’ is optimal for the primal and y’ is optimal for the dual.
Proposition
Slacks
Denote by aj the j-th column of A (as row
vector) and by ai the i-th row
min cTx
Ax b
x 0
max bTy
ATy c
y 0
Then the i-th constraint of (P) is ai x ≥ bi i = 1, …,m
Let 𝑥 be a primal feasible solution, the quantity (ai 𝑥 - bi) is the
slack of the i-th constraint.
The j-th constraint of (D) is aj y ≤ cj i = 1, …,m
Let 𝑦 be a feasible dual solution, the quantity (cj - aj 𝑦)
is the slack of the j-th dual constraint.
(D)
(P)
Complementary slackness
(i) Let x* be primal optimal, let y* be dual optimal. Then
(cj – aj y
*) xj* = 0 j
(ai x* - bi) yi
*= 0 i
Proposition (complementary slackness conditions)
aj is the j-column (as row vector) of A and ai is the i-th row of A.
(i) Let x’ be primal feasible, let y’ be dual feasible and suppose
the complementary slackness conditions are satisfied. Then x’
is optimal for the primal and y’ is optimal for the dual.
Proposition (complementary slackness)
Exercise
Model this problem as a linear program (example taken from
Bradley, Hax, Magnanti, Applied Mathematical Programming)