column generation: theory, algorithms and applications dr natashia boland department of mathematics...
TRANSCRIPT
![Page 1: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/1.jpg)
Column Generation:Theory, Algorithms and Applications
Dr Natashia Boland
Department of Mathematics and StatisticsThe University of Melbourne
![Page 2: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/2.jpg)
Outline
Linear and integer programs
A shipping problem: column generation formulation
Column generation and why it works
Successful applications
Difficulties
Towards a cure: duality
A general framework for stabilisation
Progress in column generation and the future
![Page 3: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/3.jpg)
Linear Programs
xi = ith decision variable, i = 1,…,n
min c1x1 + c2x2 +…+ cnxn
s.t. a11x1 + a12x2 +…+ a1nxn b1
a21x1 + a22x2 +…+ a2nxn b2
am1x1 + am2x2 +…+ amnxn bm
linear objective function
m linearconstraints
xi integer, i = 1,…,n
Integer
![Page 4: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/4.jpg)
A Shipping Problem*
Delivering product to customers around the world
Shipping contracts entered into for next 12 months
Contract specifies ship s can be used at most once in any ms consecutive months – the spread
Contract specifies min and max limits on number of times ship used over year
Operations Research Group Pty Ltd J. Horton, T. Surendonk, E. Jago
* See Boland and Surendonk, 2001
![Page 5: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/5.jpg)
Exercising a Shipping Contract
Jan Feb Mar Apr May Jun Jul Aug Sep
Spread = 3 months
Spread = 4 months
CUSTOMER DEMAND
![Page 6: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/6.jpg)
Exercising a Shipping Contract
Jan Feb Mar Apr May Jun Jul Aug Sep
Spread = 3 months
Spread = 4 months
CUSTOMER DEMAND
1
2 X
![Page 7: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/7.jpg)
An Integer Programming Model
xist = 1, if ship s delivers to customer i in period t
0, otherwise
min costisxisti,s,t
capacitysxist demand of customer i in period ts
s.t.
![Page 8: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/8.jpg)
Contract Constraints
i, tmins xist maxs
(xist + xis(t+1) + … + xis(t+ms-1) ) 1i
For each ship s
for all t=1,…
spread of ms periods
![Page 9: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/9.jpg)
Outcomes
Implemented using top-grade commercial integer programming solver (ILOG Cplex)
16 customers, about 30 ships
Ran for many hours – out of memory
Best solution found might be as bad as 16% from optimal
WHAT’S GOING WRONG?
![Page 10: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/10.jpg)
Integrality Gap
Integer program
min cx
s.t. Ax b
x {0,1}n
Value of integer program >> Value of relaxation
Linear programming
relaxation
min cx
s.t. Ax b
0 x 1
![Page 11: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/11.jpg)
Re-formulation: Ship Combinations
Jan Feb Mar Apr May Jun Jul Aug Sep
capacitysxist demand of customer i in period ts
Which combination of ships should be used to meet customer i’s demand inperiod t?
![Page 12: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/12.jpg)
A Column Generation Model
yikt = 1, if combination k used for
customer i in period t
0, otherwise
For customer i, period tKit = set of all combinations of ships that can
meet customer’s demand in that period
For k Kit
![Page 13: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/13.jpg)
The Model
min ( costis)yikti,t,kKit
yikt = 1k Kit
s.t.
sk
customer i demand met in period t
mins yikt maxsi,t,k: s k
(yikt + yik(t+1) + … + yik(t+ms-1) ) 1i,k: s k
contract constraintsfor ship s
for all t=1,…
![Page 14: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/14.jpg)
Issues
There are a very large number of variables - how can we solve even the linear programming relaxation?
Why will this reformulation help?
ANSWERS:
Column generation
The linear program better approximates the integer program - WHY?
![Page 15: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/15.jpg)
Column Generation
1. Select a small number of variables and solve the linear program using only these.
2. Find an unused variable which, if included, would most improve the objective value, or determine that there is none - the linear program has been solved: stop.
3. Include the variable in the linear program, re-
solve it, and go to step 2.
Model this as an optimization problemSolve the column generation subproblem
![Page 16: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/16.jpg)
The Shipping Subproblem
For each customer i and period t, find a combination of ships that will most improve the objective:
capacitysvs demand of customer i in period ts
vs = 1, if ship s used in combination
0, otherwise
min f(costis,s)vs
s.t.
dual variables from linear program
![Page 17: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/17.jpg)
Shipping Column Generation
1. Select a small number of combinations for each customer and period. Solve the linear program using only these.
2. For each customer and period, solve the shipping subproblem: if its value < cut-off include the optimal combination in the linear program.
3. If no new combinations included in step 2, the linear program has been solved: stop.
4. Re-solve the linear program and go to step 2.
![Page 18: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/18.jpg)
Outcomes
Implemented using ILOG Cplex (and AMPL)
Solved in less than 2 minutes
Step 2 performed 34 times
1700 combinations generated
Solution to linear program was integer optimal
WHAT’S GOING RIGHT?!
![Page 19: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/19.jpg)
Dantzig-Wolfe Decomposition
Integer program
min cx
s.t. Ax b
Dx g
x {0,1}n
Value integer program >> Value linear relaxation
0 x 1
Linear relaxation D-W Decomposition
min cx
s.t. Ax b
x conv Dy g
y {0,1}n
Value integer program Value D-W decomposition
contract
demand
![Page 20: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/20.jpg)
Geometry
demand
linearrelaxation
D-W relax.
contract
![Page 21: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/21.jpg)
Successful Applications: A Brief History
Cutting stock problems
Air crew scheduling
Aircraft fleeting and routing
Crew rostering
Vehicle routing
Global shipping
Multi-item lot-sizing
Optical telecommunications network design
Cancer radiation treatment using IMRT
1960
1980
1990
2000
![Page 22: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/22.jpg)
Difficulties with Column Generation
Tailing off – slow convergence
Getting integrality – column generation within branch-and-bound and in combination with cutting planes
Columns with set-up costs
![Page 23: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/23.jpg)
Difficulties with Column Generation
Tailing off – slow convergence
Getting integrality – column generation within branch-and-bound and in combination with cutting planes
Columns with set-up costs
![Page 24: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/24.jpg)
A Dual Approach
Integer program
min cx
s.t. Ax = b
Dx g
x {0,1}n
Lagrangian Relaxation
min cx
s.t. x Y
Value integer program Value Lagrangian relaxation
Y
+ (b – Ax)
For all :
= z()
![Page 25: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/25.jpg)
Properties of the Lagrangian Relaxation
z() is a piecewise affine concave function
To find the best lower bound, solve
Lagrangian Dual
max z()
s.t. m
To find a subgradient of z(), solve the Lagrangian relaxation
![Page 26: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/26.jpg)
Nonsmooth Methods
Kelley’s cutting plane
Subgradient optimisation (Held & Karp)
Projection methods (Conn & Cornuejols)
Method of analytic centres (Goffin et al.)
Box-step method (Marsten)
Bundle methods (Lemarechal, Kiwiel)
Linear-norm bundle methods (Kim et al.)
Penalty-step method (du Merle, Desrosiers et al.)
Partial-step method (Neame)
= column generation!
![Page 27: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/27.jpg)
A Common Framework
Iteratively solve
max ( min { cx + (b – Ax) : x Yi} )
s.t. m
+ si()
where Yi Y
Joint work with Danny Ralph & Phil Neame
stabilisation function
![Page 28: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/28.jpg)
Stabilisation Functions
Box-step
Bundle
Bundle – 1-norm
du Merle et al.
![Page 29: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/29.jpg)
Better Theory and Algorithms
An algorithm for general stabilisation functions
General conditions on stabilisation functions under which the algorithm will
- Converge- Terminate finitely
Termination of earlier methods: special cases These imply conditions on the parameters
defining the stabilisation functions New (hybrid) stabilisation functions
![Page 30: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/30.jpg)
Numerical Improvements
Binary cutting stock problem
Average results over 20 problems drawn at random from 3 problem classes
CPU Time (s)
Standard
Col. Gen.
Box-step
Variant
Bundle
1-norm
Hybrid
16 15 18 20
319 117 121 113
216 123 111 113
![Page 31: Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne](https://reader035.vdocuments.site/reader035/viewer/2022062417/551af48a55034606048b6257/html5/thumbnails/31.jpg)
Progress in Column Generation
Tailing off – slow convergence
Getting integrality – column generation within branch-and-bound and in combination with cutting planes*
Columns with set-up costs**
* See Barnhart, Boland et al. 1998 & Boland et al. 2000
** See Vanderbeck 2000