is703: decision support and optimization - mysmu.edu is703: decision support and optimization ... 2...
TRANSCRIPT
![Page 1: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/1.jpg)
1
IS703:Decision Support and Optimization
Week 5: Mathematical Programming
Lau Hoong ChuinSchool of Information Systems
![Page 2: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/2.jpg)
2
Mathematical Programming - Scope
• Linear Programming
• Integer Programming
• Network flows (an introduction in previous class)
• Not Covered:– Convex Programming– Non-Linear Programming– etc
![Page 3: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/3.jpg)
3
Linear Programming(Adapted from http://www.cs.princeton.edu/~wayne/cs423/lectures/lp.ppt)
Powerful model generalizes many classic problems:– shortest path, max flow, multicommodity flow, MST, matching, 2-
person zero sum gamesEssential tool for optimal allocation of scarce resources, among a number of competing activities.Ranked among most important scientific advances of 20th century.
– accounts for a major proportion of all scientific computationHelps find "good" solutions to NP-hard optimization problems.
– optimal solutions (branch-and-cut)– provably good solutions (randomized rounding)
![Page 4: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/4.jpg)
4
Linear ProgrammingLP "standard" form.
Input data: rational numbers cj, bi, aij .Maximize linear objective function.Subject to linear inequalities.
njx
mibxa
xc
j
in
jjij
n
jjj
≤≤≥
≤≤≤∑
∑
=
=
10
1t. s.
max(P)
1
1
0t. s.
max(P)
≥≤
•
xbAx
xc
![Page 5: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/5.jpg)
5
LP: GeometryGeometry.
Forms an n-dimensionalpolyhedron.
Convex: if y and z are feasible solutions, then so is αy + (1-α)z, for all 0<α<1.
njx
mibxa
xc
j
in
jjij
n
jjj
≤≤≥
≤≤≤∑
∑
=
=
10
1t. s.
max(P)
1
1
z
y
zy
Convex Not convex
Extreme points
![Page 6: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/6.jpg)
6
LP: GeometryExtreme Point Theorem. If there exists an optimal solution to standard form LP (P), then there exists one that is an extreme point.
Only need to consider finitely many possible solutions.
Greed. Local optima areglobal optima.
![Page 7: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/7.jpg)
7
LP: AlgorithmsSimplex. (Dantzig 1947)
Developed shortly after WWII in response to logistical problems:used for 1948 Berlin airlift.Practical solution method that moves from one extreme point to a neighboring extreme point.Finite (exponential) complexity, but no polynomial implementation known.
![Page 8: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/8.jpg)
8
LP: Polynomial AlgorithmsEllipsoid. (Khachian 1979, 1980)
Solvable in polynomial time: O(n4 L) bit operations.– n = # variables – L = # bits in input
Theoretical tour de force.Not remotely practical.
Karmarkar's algorithm. (Karmarkar 1984)O(n3.5 L).Polynomial and reasonably efficientimplementations possible.
Interior point algorithms. O(n3 L).Competitive with simplex!
– will likely dominate on large problems soonExtends to even more general problems.
![Page 9: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/9.jpg)
9
LP Application: Weighted Bipartite MatchingAssignment problem. Given a complete bipartite network Kn,n and edge weights cij, find a perfect matching of minimum weight.
Birkhoff-von Neumann theorem (1946, 1953). All extreme points of the above polyhedron are {0-1}-valued.
Corollary. Can solve assignment problem using LP techniques since LP algorithms return optimal solution that is an extreme point.
Remarks. (1) The above theorem is directly proven by applying the concept of totally unimodular matrix.
(2) Polynomial combinatorial algorithms also exist.
njix
njx
nix
xc
ij
niij
njij
njijij
ni
≤≤≥
≤≤=∑
≤≤=∑
∑∑
≤≤
≤≤
≤≤≤≤
,10
11
11t. s.
min
1
1
11
![Page 10: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/10.jpg)
10
LP Application: Multicommodity FlowMulticommodity flow. Given a network G = (V, E) with edge capacities u(e) ≥ 0, edge costs c(e) ≥ 0, set of commodities K, and supply / demand dk(v) for commodity k at node v, find a minimum cost flow that satisfies all of the demand.
Applications.Transportation networks.Communication networks.VLSI design.
KkEeex
Eeeuex
KkVvvdexex
exec
kKk
k
k
ve
k
ve
kEe
kk
Kk
∈∈≥
∈≤
∈∈=−
∑
∑∑
∑∑
∈
∈∈
,0)(
)()(
,)()()( t.s.
)()(min
ofout in to
Remark. The problem is NP-hard if flows are integral!
![Page 11: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/11.jpg)
Integer ProgrammingAn Integer Programming model is a linear programming problem where some or all of the variables are required to be non-negative integers.
These models are in general substantially harder than solving linear programming models.
Network models are special cases of integer programming models and are very efficiently solvable.
Several applications of integer programming models.
Branch and bound technique, one of the most popular algorithm to solve integer programming models.
![Page 12: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/12.jpg)
12
Integer ProgrammingProblem: Given rational numbers aij, bi, cj, find integers xj that solve:
Claim. INTEGER-PROGRAMMING is NP-hard.Proof. VERTEX-COVER ≤ P INTEGER-PROGRAMMING.
njxnjx
mibxa
xc
j
j
in
jjij
n
jjj
≤≤≤≤≥
≤≤≥∑
∑
=
=
1integral10
1t. s.
min
1
1
![Page 13: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/13.jpg)
13
Weighted Vertex CoverWeighted vertex cover. Given an undirected graph G = (V, E) with vertex weights wv ≥ 0, find a minimum weight subset of nodes S such that every edge is incident to at least one vertex in S.
NP-hard even if all weights are 1.
Integer programming formulation.
If x* is optimal solution to (ILP),then S = {v ∈ V : x*v = 1} is a minweight vertex cover.
3
6
10
7
A
E
H
B
D I
C
F
J
G
6
16
10
7
23
9
10
9
33
Total weight = 55.
32
VvxEwvxx
xwILP
v
wv
Vvvv
∈∈∈≥+
∑∈
}1,0{),(1t. s.
min)(
![Page 14: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/14.jpg)
Classifications of IP ModelsPure IP Model: Where all variables must take integer values.
Maximize z = 3x1 + 2x2subject to x1 + x2 ≤ 6
x1, x2 ≥ 0, x1 and x2 integer
Mixed IP Model: Where some variables must be integer while others can take real values.
Maximize z = 3x1 + 2x2subject to x1 + x2 ≤ 6
x1, x2 ≥ 0, x1 integer
0-1 IP Model: Where all variables must take values 0 or 1.
Maximize z = x1 - x2subject to x1 + 2x2 ≤ 2
2x1 - x2 ≤ 1, x1, x2 = 0 or 1
![Page 15: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/15.jpg)
Classifications of IP Models (contd.)LP Relaxation: The LP obtained by omitting all integer or 0-1 constraints on variables is called the LP relaxation of IP.
IP:Maximize z = 21x1 + 11x2subject to
7x1 + 4x2 ≤ 13x1, x2 ≥ 0, x1 and x2 integer
LP Relaxation:
Maximize z = 21x1 + 11x2subject to
7x1 + 4x2 ≤ 13x1, x2 ≥ 0
Fact:Optimal objective value of IP ≤
Optimal objective value of LP relaxation
![Page 16: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/16.jpg)
Solving IP via LP relaxation
Solve the LP relaxationRound-off the solution to the nearest feasible integer solution
Any potential difficulty with this approach?
x x xx
xx
xx
x1
x2
1 2 3
1
3
2
7x1 + 4x2= 13
![Page 17: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/17.jpg)
Capital Budgeting ProblemStockco Co. is considering four investments
It has $14,000 available for investment
Formulate an IP model to maximize the NPV obtained from the investments
IP:
Maximize z = 16x1 + 22x2 + 12x3 + 8x4subject to
5x1 + 7x2 + 4x3 + 3x4 ≤ 14x1, x2,,x3, x4 = 0, 1
Investmentchoice
1 2 3 4
Cashoutflow
$5000 $7000 $4000 $3000
NPV $16000 $22000 $12000 $8000
![Page 18: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/18.jpg)
Fixed Charge ProblemGandhi cloth company manufactures three types of clothing: shirts, shorts, and pants
Machinery must be rented on a weekly basis to make each type of clothing. Rental Cost:
$200 per week for shirt machinery$150 per week for shorts machinery$100 per week for pants machinery
There are 150 hours of labor available per week and 160 square yards of cloth
Find a solution to maximize the weekly profitLabor hr Cloth yd Var. cost Price
Shirts 3 4 $12 $6Shorts 2 3 $8 $4Pants 6 4 $15 $8
![Page 19: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/19.jpg)
Fixed Charge Problem (contd.)Decision Variables:
x1 = number of shirts produced each weekx2 = number of shorts produced each week x3 = number of pants produced each week
y1 = 1 if shirts are produced and 0 otherwisey2 = 1 if shorts are produced and 0 otherwisey3 = 1 if pants are produced and 0 otherwise
Formulation:
Max. z = 6x1 + 4x2 + 7x3 - 200y1 - 150 y2 - 100y3subject to
3x1 + 2x2 + 6x3 ≤ 1504x1 + 3x2 + 4x3 ≤ 160x1 ≤ M y1, x2 ≤ M y2, x3 ≤ M y3
x1, x2,,x3 ≥ 0, and integer; y1, y2,,y3 ≥ 0, and integer
![Page 20: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/20.jpg)
Either-Or ConstraintsDorian Auto is considering manufacturing three types of auto: compact, midsize, large.
Resources required and profits obtained from these cars are given below.
We have 6,000 tons of steel and 60,000 hours of labor available.
If any car is produced, we must produce at least 1,000 units of that car.
Find a production plan to maximize the profit.Compact Midsize Large
Steel Req. 1.5 tons 3 tons 5 tonsLabor Req. 30 hours 25 hours 40 hoursProfit $2000 $3000 $4000
![Page 21: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/21.jpg)
Either-Or Constraints (contd.)Decision Variables:
x1, x2, x3 = number of compact, midsize and large cars producedy1, y2, y3 = 1 if compact , midsize and large cars are produced or
not
Formulation:
Maximize z = 2x1 + 3x2 + 4x3subject tox1 ≤ My1; x2 ≤ My2; x3 ≤ My31000 - x1 ≤ M(1-y1)
1000 - x2 ≤ M(1-y2)1000 - x3 ≤ M(1-y3)
1.5 x1 + 3x2 + 5x3 ≤ 600030 x1 + 25x2 + 40 x3 ≤ 60000
x1, x2, x3 ≥ 0 and integer; y1, y2, y3 = 0 or 1
![Page 22: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/22.jpg)
Set Covering ProblemsWestern Airlines has decided to have hubs in USA.
Western runs flights between the following cities: Atlanta, Boston, Chicago, Denver, Houston, Los Angeles, New Orleans, New York, Pittsburgh, Salt Lake City, San Francisco, and Seattle.
Western needs to have a hub within 1000 miles of each of these cities.Determine the minimum number of hubs
Cities within 1000 milesAtlanta (AT) AT, CH, HO, NO, NY, PIBoston (BO) BO, NY, PIChicago (CH) AT, CH, NY, NO, PIDenver (DE) DE, SLHouston (HO) AT, HO, NOLos Angeles (LA) LA, SL, SFNew Orleans (NO) AT, CH, HO, NONew York (NY) AT, BO, CH, NY, PIPittsburgh (PI) AT, BO, CH, NY, PISalt Lake City (SL) DE, LA, SL, SF, SESan Francisco (SF) LA, SL, SF, SESeattle (SE) SL, SF, SE
![Page 23: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/23.jpg)
Formulation of Set Covering ProblemsDecision Variables:
xi = 1 if a hub is located in city ixi = 0 if a hub is not located in city i
Minimize xAT + xBO + xCH + xDE + xHO + xLA + xNO + xNY + xPI + xSL+ xSF + xSE
subject toAT BO CH DE HO LA NO NY PI SL SF SE Required
AT 1 0 1 0 1 0 1 1 1 0 0 0 xAT >= 1BO 0 1 0 0 0 0 0 1 1 0 0 0 xBO >= 1CH 1 0 1 0 0 0 1 1 1 0 0 0 xCH >= 1DE 0 0 0 1 0 0 0 0 0 1 0 0 xDE >= 1HO 1 0 0 0 1 0 1 0 0 0 0 0 xHO >= 1LA 0 0 0 0 0 1 0 0 0 1 1 0 xLA >= 1NO 1 0 1 0 1 0 1 0 0 0 0 0 xNO >= 1NY 1 1 1 0 0 0 0 1 1 0 0 0 xNY >= 1PI 1 1 1 0 0 0 0 1 1 0 0 0 xPI >= 1SL 0 0 0 1 0 1 0 0 0 1 1 1 xSL >= 1SF 0 0 0 0 0 1 0 0 0 1 1 1 xSF >= 1SE 0 0 0 0 0 0 0 0 0 1 1 1 xSE >= 1
![Page 24: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/24.jpg)
IP: Logistics/Supply Chain Applications
Set covering and partitioningScheduling airline crewsManpower rostering
Facility locationLocating warehouses or franchises
Multicommodity distributionDistributing auto parts
Traveling salesman / Vehicle routingRouting pickups and deliveries
Exercise: Model TSP as an IP
![Page 25: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/25.jpg)
Branch and BoundThe most popular approach for solving integer programming problems
Enumerate the entire solution space but only implicity; hence they are called implicit enumeration algorithms.
A general-purpose solution technique which must be specialized for individual IP's.
Running time grows exponentially with the problem size, but small to moderate size problems can be solved in reasonable time.
A divide and conquer algorithm, where a problem is divided into smaller and smaller subproblems. Each subproblem is solved separately, and the best solution is taken.
![Page 26: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/26.jpg)
26
Branch-and-Bound Search
• Represent search space as trees• Search tree
– Each leaf node represents a feasible solution– Non-leaf node represents a subset of feasible solutions – Parent-child relationship represents partition of search
space into sub-spaces• Idea: if we can drastically reduce solution space
from one level to next, total number of nodes explored will be reduced
![Page 27: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/27.jpg)
27
Branch-and-Bound Search
S
S1
S2S3
S4
S1 S2 S3 S4
S24
(a)
(b)
(c)
S1
S4
S21
S22
S31
S32
*
*
* = does not contain optimal solution
S1 S2 S3 S4
S21 S22 S23
Figure 1: Illustration of the search space of B&B.
![Page 28: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/28.jpg)
28
Elements of Branch-and-Bound1. Branching rule
• Rule for dividing the solution space into sub-spaces (sub-problem)2. Node-selection (search/exploration) strategy
• Select next node to be explored next• DFS, BFS, Best-First Search
3. Bounding function • Function that computes the lower bound for the best solution
obtainable in that search space4. Upper bounding
• Process of obtaining an upper bound for each subproblem5. Pruning strategy
• If for a subproblem, UB ≤ LB, then the subproblem need not be explored further
![Page 29: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/29.jpg)
29
Bounding Function
g
S
P
f
Figure 2: The relation between the bounding function g and the objective func-tion f on the sets S and P of feasible and potential solutions of a problem.
![Page 30: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/30.jpg)
30
Branch-and-Bound Algorithm (sketch)• Start with complete feasible solution space and a naïve
initial solution• upper bound = objective value of best solution found so far• Among unexplored nodes do
– Select a non-leaf node based on selection strategy– Partition the feasible space into sub-spaces based on
branching rule– For each sub-space do
• If leaf node reached and beats best solution, update best solution, else discard and exit
• Compute lower-bound• If lower bound >= upper bound, prune the sub-space • Else create a sub-space node and add to list of
unexplored nodes
![Page 31: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/31.jpg)
31
Example: TSP• Find optimal tour by systematically considering all
tours• Strategy 1 (exhaustive search):
– Consider all permutations of the cities (vertices)– Evaluate each tour (given by particular vertex order)– Keep track of shortest tour– (n-1)! permutations, each takes O(n) time to evaluate
• Don’t look at all n permutations, since we don’t care about starting point of tour: A,B,C,(A) is same tour as C,A,B,(C)
– Unacceptable for large n• Strategy 2 (dynamic programming):
– O(n22n)
![Page 32: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/32.jpg)
32
Example: TSP• Strategy 3: Branch-and-Bound• Construct a tree with a root that represents all tours
(tours are feasible solutions)• Each node has two children
– One child represents tours with a particular edge– Other child represents tours without that edge– Choose edges in lexicographic order, i.e. ab, ac, ad, etc
![Page 33: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/33.jpg)
33
TSP Search Tree
all tours
tours with(a,b)
tours without(a,b)
tours with(a,b)
and (a,c)
tours with(a,b)
not (a,c)
tours with(a,c)
not (a,b)
tours without(a,b)
or (a,c)
tours with(a,b)
and (a,c)not (a,d)
tours with(a,b) and(a,d) not
(a,c)
tours with(a,b) not(a,c) or(a,d)
tours with(a,d) not(a,b) or
(a,c)
tours without(a,d),
(a,b) or(a,c)
not shown:pruned
(infeasible)
![Page 34: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/34.jpg)
34
Lower bound• Compute a lower bound on the cost of feasible
solutions “below” node n– If current solution is better (lower) than lower bound for
node n, need not explore any nodes below n• For TSP, lower bound on cost of tour
– What’s an obvious loose lower bound?– Tighter bound: half the sum over all nodes n of the 2
least cost edges incident upon n– More precisely, let e1(n) and e2(n) be the two least cost
edges incident to n, then LB = (∑n∈V e1(n)+e2(n)) / 2– Proof. Left as an exercise. (Hint: consider situation at
each node, and sum up over all nodes.)
![Page 35: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/35.jpg)
35
Examplea
c
b
e
d
33
44
8
7
2
6
6
5
2 least cost edges adjacent to a are: (a,d) and (a,b) for total of 2+3=52 least cost edges adjacent to b are: (a,b) and (b,e) for total of 3+3=6c, d, e: total of 8, 7, 9 respectively⇒ lower bound on cost of tour is (5+6+8+7+9)/2 = 17.5
![Page 36: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/36.jpg)
36
Lower Bound for Subset of Tours• Lower bound on the cost of node in search tree
– Each node represents tours defined by a set of edges that must be in tour and set of edges that may not be included
• Assume we must include (a,e) and exclude (b,c)a: (a,d) and (a,e), total 9 // must include (a,e) instead of (a,b)b: (a,b) and (b,e), total 6c: (a,c) and (c,d), total 9 // cannot use (b,c)d: (a,d) and (c,d), total 7e: (a,e) and (b,e), total 10 // must include (a,e) instead of (e,d)
⇒ lower bound = (9+6+9+7+10)/2 = 20.5
![Page 37: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/37.jpg)
37
Reconstruct Search Tree• Each time we branch, decide on which edge to
include/exclude based on:– Include (x,y) if it becomes impossible for x or y to have two adjacent
edges in tour otherwise– Exclude (x,y) if it raises degree of x or y above 2– Exclude (x,y) if it would complete a non-tour cycle with selected
edges
• After branch, compute lower bounds for both children
• Prune child whose lower bound is as high or higher than best tour found so far
• If neither child can be pruned, explore child with smaller lower bound first
![Page 38: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/38.jpg)
38
New Search Tree
B 17.5ab
C 18.5ab
D 20.5ac ad ae
E 18ac
G 23ad ae
F 18ad ae
H
bc bd bece de cd
tour=abcedacost=23
I
bc cd cebd de be
tour=abecdacost=21
pruned after discovery of I
pruned after discovery
of I
J 18.5ac
K 21ac ad ae
pruned
M 23.5ad ae
L 18.6ad ae
pruned
N
bc cd cebd de be
tour=acbedacost=19
P
bc bd bece de cd
tour=acebdacost=23
A 17.5no constraints
LB
![Page 39: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/39.jpg)
39
Analysis
• Search space: still (n-1)! in worst-case• But much better on average
– Depends on the 3 elements of B&B
![Page 40: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/40.jpg)
40
A simple way to get the initial solution:
Set xi = 1 for i = 1,2,… until the constraint is violated.
e.g. x1=1, x2=1, x3=0, x4=0.
max z = 16x1+22x2+12x3+8x4
subject to the following constraints:
5x1+7x2+4x3+3x4 <= 14
xj= 0 or 1, for all j
Lower bound = 16x1 + 22x2 = 38
Any sub-space that has an upper bound <=38 can be pruned. What is a good upper bound?
0-1 Knapsack Problem
Integer programming (IP)formulation
![Page 41: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/41.jpg)
41
Upper Bounding Function:Fractional Knapsack Problem
0,5.0
,1
4
3
21
==
==
xx
xx :solution LPj p j w j ratio ranking
1 16 5 3.2 12 22 7 3.1 23 12 4 3.0 34 8 3 2.7 4
value ingnonincreas to according items order :it solve Toj
j
wp
Greedy! i.e. possible, as large as order this in s' the make and jx
Linear programming(LP) Relaxation
max z = 16x1+22x2+12x3+8x4
subject to the following constraints:
5x1+7x2+4x3+3x4 <= 14
0<= xj<=1, for all j
![Page 42: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/42.jpg)
42
Branch and Bound TreeSubproblem 1z = 44, LB=38
x1 = x2 = 1x3 =.5
1
x3 = 1
2
Subproblem 3z = 43.4, LB=38
x1 =x3= 1, x2 = .7, x4=0
Subproblem 6z = 42, LB=38x1 =0, x2=x3=1
x4 = 15
Subproblem 7LB = 42
Infeasible6
x1 = 0 x1 = 1
x3 = 0
7
Subproblem 2z = 43.3, LB=38
x1 = x2=1x3 = 0, x4 =.67
x4 = 1
8 9
Subproblem 8z = 38, LB=42
x1 = x2=1x3 = x4 = 0
Subproblem 9z= 42.85, LB=42
x1 = x4 =1x3 = 0, x2 = .85
x4 = 0
pruned
x2 = 0
Subproblem 5z = 43.6, LB=38x1 =.6, x2=x3=1
x4 = 0
Subproblem 4z = 36, LB=38
x1 = x3=1x2 = 0, x4 =1
x2 = 1
3 4
pruned
LB=42
pruned
![Page 43: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/43.jpg)
43
Practical Developments• Good formulations, heuristics and theory
Goal: to get LP solution as close as possible to IP solutionDisaggregation, adding constraints (cuts)
• PreprocessingAutomatic methods for reformulationSome interesting graph theory is involved
• Cut generation (branch-and-cut)Add cuts during the branch-and-bound
• Column generationImprove formulation by introducing an exponential number of variables.
![Page 44: IS703: Decision Support and Optimization - mysmu.edu IS703: Decision Support and Optimization ... 2 integer Mixed IP Model: Where some variables must be integer while others can take](https://reader031.vdocuments.site/reader031/viewer/2022030415/5aa1549a7f8b9a84398b820c/html5/thumbnails/44.jpg)
44
Exercise: Generalized Assignment Problem
In this problem, we assign m jobs to n machines– Machine: The total available time of machine j is Tj
– Jobs: If job i is performed on machine j, it costs aij, and requires tij time units. A job must be assigned to one machine in its entirety
The problem is to find a minimum cost assignment of all jobs.
Design a B&B approach to find the optimal solution.