unit.9 integer programming - xiaoxi li's homepage · pdf filesome innovative uses of...

58
Unit.9 Integer Programming Xiaoxi Li EMS & IAS, Wuhan University Dec. 22 - 29, 2016 (revised) Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 1 / 58

Upload: vuonghanh

Post on 13-Mar-2018

245 views

Category:

Documents


6 download

TRANSCRIPT

Page 1: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Unit.9 Integer Programming

Xiaoxi Li

EMS & IAS, Wuhan University

Dec. 22 - 29, 2016 (revised)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 1 / 58

Page 2: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Organization of this lecture

Contents:

Integer Programming: Modeling Approach

Basic Models and Classifications: IP, MIP, BIPA Prototype Example and Some BIP ApplicationsSome Innovative Uses of Binary Variables in Model FormulationSome Formulation Examples

Solving an IP (1): the Branch-and-Bound Algorithm

The Branch-and-Bound Algorithm for BIPThe Branch-and-Bound Algorithm for MIP

Solving an IP (2): the Cutting Plane Approach

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 2 / 58

Page 3: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Integer Programming: Modeling Approach

The notion of Integer Programming (IP) has the following two senses.

The broad sense. An IP is a Mathematical Programming (LP orNon LP) with the restrictions that some decision variables takeinteger values.The narrow sense: An IP is a LP with the restrictions that somedecision variables take integer values.

In this course, we restrict ourselves to the narrow sense. Thus a LPcan be regarded as a relaxation of some IP.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 3 / 58

Page 4: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Integer Programming: Modeling Approach (cont.)

Classifications of IP:

MIP: some variables are asked to take integer valuesPure IP: all variables are asked to take integer valuesBIP: all variables are asked to take 0-1 values

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 4 / 58

Page 5: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

A Prototype Example with BIP Modeling: TheCalifornia Manufacturing Company

The California Manufacturing Company is a company withfactories and warehouses throughout California.It is currently considering whether to build a new factory in LosAngeles and/or San Francisco.Management is also considering building one new warehousewhere a new factory has been recently built.Should the CMC build factories and/or warehouses in Los Angelesand/or San Francisco?

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 5 / 58

Page 6: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

A Prototype Example with BIP Modeling: TheCalifornia Manufacturing Company (cont.)

There are (at most) four possible decisions to make:

To build a factory in LA? (x1)To build a factory in SF? (x2)To build a warehouse in LA (if a factory has been built there)? (x3)To build a warehouse in SF (if a factory has been built there)? (x4)

So the decision variables take the binary form, to be Yes or No:

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 6 / 58

Page 7: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

A Prototype Example with BIP Modeling: TheCalifornia Manufacturing Company

The following restrictions exist amongst xj:

x3 and x4 are mutually exclusive alternatives: x3 + x4 ≤ 1;(since at most ONE warehouse is to be constructed)x3 is a contingent variable on x1: x3 ≤ x1

(same for (resp. x4 ≤ x2)

).

(a warehouse is constructed only where a factory is constructed)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 7 / 58

Page 8: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

A Prototype Example with BIP Modeling: TheCalifornia Manufacturing Company (cont.)

The BIP for the problem is as follows:

Max Z = 9x1 +5x2 +6x3 +4x4

s.t.

6x1 +3x2 +5x3 +2x4 ≤ 10x3 + x4 ≤ 1x3 ≤ x1

x4 ≤ x2

xj ∈ {0,1}, j = 1,2,3,4.Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 8 / 58

Page 9: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Some Other Applications

Investment AnalysisSite SelectionDesigning a Production and Distribution NetworkDispatching ShipmentsScheduling Interrelated ActivitiesAirline Application...

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 9 / 58

Page 10: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Some Innovative Uses of Binary Variables in ModelFormulation

To name a few:Either-Or ConstraintK out of N Constraints Must HoldFunctions with N Possible ValuesThe Fixed-Charge Problem

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 10 / 58

Page 11: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Either-Or Constraint

Suppose you have a situation where you have two potentialconstraints, but only one of them can hold. For example:

either "5x1 +9x2 ≤ 24" (Material 1 be used for production) – 1

or "8x1 +6x2 ≤ 35" (Material 2 be used for production) – 2

To handle this, we introduce a binary variable y1, a big M > 0, andadd to one constraint the term My1 to the RHS;add to the other constraint the term M(1− y1) to the RHS.

To obtain: "5x1 +9x2 ≤ 24+My1 and 8x1 +6x2 ≤ 35+M(1− y1)".

Why they are equivalent?

When y1 = 0, Constraint 1 is active and Constraint 2 isautomatically satisfied for any feasible solution (always bounded);

When y1 = 1, Constraint 2 is active and Constraint 1 isautomatically satisfied for any feasible solution (always bounded).

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 11 / 58

Page 12: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

K out of N Constraints Must Hold

There are N constraints among which only K (must) hold (K < N).Introduce binary variables yi, i = 1, ...,N; add to Constraint i theterm Myi, and another constraint ∑

Ni=1 yi = N−K.

For an example (M = 3 and K = 1):

5x1 +12x2 ≤ 34+My16x1 +11x2 ≤ 43+My27x1 +10x2 ≤ 57+My3y1 + y2 + y3 = 1

Why they are equivalent?

The same reasoning as the Either-Or Constraint (M = 2, K = 1);"yi = 0" corresponds to the case that "Constraint i " is active, so∑

Ni=1 yi = N−K is the number of constraints that do not hold.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 12 / 58

Page 13: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Functions with N Possible Values

An example: the constraint is "5x1 +31x2 = 25 or 50 or 75".We introduce binary variables y1,y2 and y3 to obtain the equivalent:

5x1 +31x2 = 25y1 +50y2 +75y3 and y1 + y2 + y3 = 1.

Indeed, "the RHS taking the i-th value" corresponds to the case

”yi = 1 and yj = 0,∀j 6= i.”

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 13 / 58

Page 14: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

The Fixed-Charge Problem

The cost function for producing activity j is

fj(xj) =

{kj + cjxj if xj > 0

0 if xj = 0

where xj is the level of activity j, kj is the fixed (setup) cost and cj isthe marginal cost.The objective function is not linear (mainly at the point 0).To handle the issue, we introduce for each j

the binary variable yj for the decision "if activity j is produced";the continuous variable xj for the level of activity j.Thus yj is seen as a contingent variable of xj, i.e. "yj = 1 iff. xj > 0".

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 14 / 58

Page 15: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

The Fixed-Charge Problem (cont.)

Here, it is somehow more complicated (than the previous case)because xj can take very large value (a priori no given bounded).

Let M > 0 be sufficiently large that exceeds the maximum feasiblevalue of any xj, j = 1, ...,n.Introduce the constraints "xj ≤Myj, j = 1, ...,n".The objective function is Z = ∑

nj=1(cjxj + kjyj).

Why they are equivalent to the constraints "yj = 1 iff. xj > 0, j = 1, ...,n"?

First of all, "xj ≤Myj" isIt is straightforward that "xj ≤Myj" implies "yj = 1 if xj > 0";On the other hand, if xj = 0, yj could be 0 or 1 as a feasiblesolution. However, by the objective function, at optimum, weshould have yj = 0 whenever xj = 0.

This leads us to a MIP.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 15 / 58

Page 16: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Example 1: Making Choices When the DecisionVariables Are Continuous

Objective: Maximize ProfitsDecision Variables: Product 1, Product 2, and Product 3Constraints:

Production time available for Plants 1 and 2At most two out of the three products can be producedOnly one of the two plants can produce the new products

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 16 / 58

Page 17: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Example 1: Making Choices When the DecisionVariables Are Continuous (cont.)

The data for the problem:

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 17 / 58

Page 18: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Example 1: Making Choices When the DecisionVariables Are Continuous (cont.)

We construct the following MIP for the problem:

Max Z = 5x1 +7x2 +3x3

s.t.

x1−My1 ≤ 0, x2−My2 ≤ 0, x3−My3 ≤ 0y1 + y2 + y3 ≤ 2x1 ≤ 7,x2 ≤ 5,x3 ≤ 93x1 +4x2 +2x3 ≤ 30+Mz1

4x1 +6x2 +2x3 ≤ 30+M(1− z1)

x1,x2,x3 ≥ 0, y1,y2,y3,z1 ∈ {0,1}.

Interpretations. In class.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 18 / 58

Page 19: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Example 2: Proportionality Assumption Violated

Objective: Maximize ProfitsDecision Variables: Number of TV spots for Product 1, Product 2,and Product 3Constraints: Number of TV spots allocated to the three productscannot be more than fiveMajor Issue: Proportionality Assumption is violated

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 19 / 58

Page 20: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Example 2: Proportionality Assumption Violated(cont.)

The data for the problem:

Note: Proportionality Assumption Violated!

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 20 / 58

Page 21: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Example 2: Proportionality Assumption Violated(cont.)

We construct the following MIP for the problem (solution 1):

Max Z = y11 +3y12 +3y13 +2y22 +3y23− y31 +2y32 +4y33

s.t.

y11 + y12 + y13 ≤ 1y21 + y22 + y23 ≤ 1y31 + y32 + y33 ≤ 1y11 +2y12 +3y13 + y21 +2y22 +3y23 + y31 +2y32 +3y33 = 5yij ∈ {0,1}, i, j = 1,2,3.

For each i, the auxiliary binary variable yij is introduced for eachpossible value "j" of the variable xi i.e. yij = 1 if xi = j and yij = 0 if xi 6= j.

Interpretations. In class.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 21 / 58

Page 22: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Example 2: Proportionality Assumption Violated(cont.)

We construct the following MIP for the problem (solution 2):

Max Z = y11 +2y12 +2y22 + y23− y31 +3y32 +2y33

s.t.

y12 ≤ y11, y13 ≤ y12

y22 ≤ y21, y23 ≤ y22

y32 ≤ y31, y33 ≤ y32

y11 + y12 + y13 + y21 + y21 + y22 + y23 + y31 + y32 + y33 = 5yij ∈ {0,1}, i, j = 1,2,3.

For each i, the auxiliary binary variable yij is introduced for possiblevalue no smaller than "j" of the variable xi, i.e. yij = 1 if xi ≥ j and yij = 0if xi < j. =⇒ yij are the increments and xi = yi1 + yi2 + yi3.

Interpretations. In class.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 22 / 58

Page 23: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Example 3: Covering All Characteristics

Exercise.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 23 / 58

Page 24: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: Some General Comments

Some general comments on solving IP.

Solving an IP is NOT easier than solving a LP as was thought.Having a finite number of feasible solutions ensures that it is readilysolvable. Wrong!Removing some feasible solutions (the non-integer ones) from a LPmakes an IP easier to solve. Wrong!

The fact that an IP is in general more difficult to solve than a LP(relaxation) can be understood as follows.

The simplex method (solving LP) visits only the corner-pointfeasible solutions, which generally have a much smaller numberthan the integer feasible solutions.Most successful IP algorithms incorporate a LP algorithm byrelating an IP to its LP relaxation.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 24 / 58

Page 25: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: Some General Comments (cont.)

A LP relaxation of an IP is to keep all the original program except forremoving the integer-value restrictions on variables.

Consider a maximization problem. The value of a LP relaxation isalways NO SMALLER than that of an IP. Thus solving acorresponding LP relaxation gives a upper bound for an IP.It is fortuitous if a LP relaxation’s optimal solution happens to beinteger values, in which case the solution is also optimal to the IP.However, this is in general not guaranteed.Nevertheless, in some specific applications, it is fortuitous to havethis solution structure: minimum cost flow problem and its specialcases (transportation problem, assignment problem, shortest-pathproblem, maximum flow problem).

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 25 / 58

Page 26: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: Some General Comments (cont.)

One temptation for solving an IP is to first solve its LP relaxation, andthen round the optimal solution to integer values.

This is an approximation method which may not always lead to anoptimal solution. The approximation might be good if the variables’values are quite large.But in general, it is not accepted due to the following two pitfalls:

The rounded "optimal" solution may not be feasible.Even if feasibility is not a problem, the rounded "optimal" solutionmay be far from optimal.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 26 / 58

Page 27: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: Some General Comments (cont.)

Pitfall 1. The rounded "optimal" solution may not be feasible.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 27 / 58

Page 28: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: Some General Comments (cont.)

Pitfall 2. The rounded "optimal" solution may be far from optimal.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 28 / 58

Page 29: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm

The Branch-and-Bound Algorithm: a motivating example of Poker.Objective. Within a random bundle of Poker, find out one card withthe highest value and has a red-heart format.Tool. Among any bundle of cards, a machine that is able to pickout the card(*) with the highest value (no ability to read theformat).An algorithm.

Put the whole bundle into the machine and obtain the card(*) withthe highest value. STOP if the card’s format is red-heart (the wholebundle is discarded); otherwise branch the bundle into two.In a general step, for a newly branched bundle, detect it by themachine for a highest-valued card(*). Several possibilities:

Discard this bundle if this card(*) is red-heart, and call it incumbent ifits value is higher than the current incumbent one;Discard this bundle if this card(*)’s value is lower than the currentincumbent one;Otherwise, brunch the bundle for further detection by machine.

STOP when all cards are discarded. Incumbent = optimal value.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 29 / 58

Page 30: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Objective. For a LP problem, find out one feasible solution with thehighest value and take binary values.Tool. The simplex method to solve for a LP (relaxation).An algorithm.

Apply the simple method to the LP relaxation to solve for an optimalsolution. STOP if the optimal solution take binary values; otherwise,brunch the problem into two subproblems by letting x1 = 0 or x1 = 1.In a general step, for a newly branched subproblem, solve its LPrelaxation by the simplex method. Several possibilities:

Discard (fathom) this subproblem if this optimal solution take integervalues, and call this solution incumbent if its value is higher than thecurrent incumbent one;Discard this subproblem if this optimal value is no larger than thecurrent incumbent one or if it is infeasible;Otherwise, brunch the subproblem by xj = 0 or xj = 1 (j in order).

STOP when all subproblems are either branched or fathomed.Incumbent = optimal solution.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 30 / 58

Page 31: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

The Branch-and-Bound Algorithm to solving an BIP: an example.

The (whole) problem is

Max Z = 9x1 +5x2 +6x3 +4x4

s.t.

6x1 +3x2 +5x3 +2x4 ≤ 10x3 + x4 ≤ 1−x1 + x3 ≤ 0−x2 + x4 ≤ 0xj ∈ {0,1}, j = 1,2,3,4.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 31 / 58

Page 32: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Branching. Branch the BIP into two by letting x1 = 0 or x1 = 0.

Subproblem 1 (x1 = 0).

Max Z = 5x2 +6x3 +4x4

s.t.

3x2 +5x3 +2x4 ≤ 10x3 + x4 ≤ 1x3 ≤ 0−x2 + x4 ≤ 0xj ∈ {0,1}, j = 2,3,4.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 32 / 58

Page 33: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Branching. Branch the BIP into two by letting x1 = 0 or x1 = 0.

Subproblem 2 (x1 = 1).

Max Z = 9+5x2 +6x3 +4x4

s.t.

3x2 +5x3 +2x4 ≤ 4x3 + x4 ≤ 1x3 ≤ 1−x2 + x4 ≤ 0xj ∈ {0,1}, j = 2,3,4.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 33 / 58

Page 34: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Branching. Branch the BIP into two by letting x1 = 0 or x1 = 0. Thebranching tree created by the branching for the first iteration.

x1 is called the branching variable.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 34 / 58

Page 35: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Bounding. This procedure involves solving the LP relaxation of asubproblem so as to obtain an upper bound on the BIP’s optimal value.Consider for example the whole problem. Its LP relaxation is

Max Z = 9x1 +5x2 +6x3 +4x4

s.t.

6x1 +3x2 +5x3 +2x4 ≤ 10x3 + x4 ≤ 1−x1 + x3 ≤ 0−x2 + x4 ≤ 00≤ xj ≤ 1, j = 1,2,3,4.

Solving it by simplex method: (x1,x2,x3,x4) = (56 ,1,0,1) with Z = 16 1

2 .Therefore any feasible solution for BIP has a value ≤ 16 1

2 .Coefficients in the objective function are integers, so 16 1

2 can berounded to 16.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 35 / 58

Page 36: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Bounding. Now we use the similar approach to bound theSubproblem 1 (x1 = 0) and the Subproblem 2 (x1 = 1).

Subproblem 1 relaxed (x1 = 0).

Max Z = 5x2 +6x3 +4x4

s.t.

3x2 +5x3 +2x4 ≤ 10x3 + x4 ≤ 1x3 ≤ 0−x2 + x4 ≤ 00≤ xj ≤ 1, j = 2,3,4.

Solving it by simplex method: (x1,x2,x3,x4) = (0,1,0,1) with Z = 9.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 36 / 58

Page 37: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Bounding.

Subproblem 1 relaxed (x1 = 1).

Max Z = 9+5x2 +6x3 +4x4

s.t.

3x2 +5x3 +2x4 ≤ 4x3 + x4 ≤ 1x3 ≤ 1−x2 + x4 ≤ 00≤ xj ≤ 1, j = 2,3,4.

Solving it by simplex method: (x1,x2,x3,x4) = (1, 45 ,1,

45) with Z = 16 1

5 .

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 37 / 58

Page 38: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Fathoming.

The Whole Problem: not fathomed because its LP relaxation hasan optimal (feasible) solution which is not binary valued, and thereis no previous incumbent value (think to be Z∗−∞).The Subproblem 1 (x1 = 0): fathomed because its LP relaxationhas an optimal solution that is binary valued. Moreover, thesolution (0,1,0,1) and its value Z∗ = 9 are stored as the incumbentsince there is no previous ones.The Subproblem 2 (x1 = 1): not fathomed because its LPrelaxation has an optimal (feasible) solution which is not binaryvalued, and its value Z = 16 is higher than the incumbent valueZ∗ = 9.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 38 / 58

Page 39: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Fathoming.

General rules for fathoming test. A (sub)problem is fathomed if itsatisfies one of the following three.

1 Its bound ≤ Z∗;2 Its LP relaxation has no feasible solution.3 Its LP relaxation has integer (binary) solution (which is stored as

the incumbent if its value is higher than the current incumbentone).

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 39 / 58

Page 40: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 40 / 58

Page 41: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example (after Iteration 1)

Iteration 2. Subproblem 3 (x1 = 1,x2 = 0).

Max Z = 9+6x3 +4x4

s.t.

5x3 +2x4 ≤ 10x3 + x4 ≤ 1x3 ≤ 1

x4 ≤ 0xj ∈ {0,1}, j = 3,4.

Its LP relaxation is to replace xj ∈ {0,1} by 0≤ xj ≤ 1 for j = 3,4.Solving it by simplex method: (x1,x2,x3,x4) = (1,0, 4

5 ,0), which isnot integer valued, and with Z = 13 4

5 .Bound for Subproblem 3: Z ≤ 13, larger than the incumbent Z∗ = 9Thus Subproblem 3 is not fathomed.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 41 / 58

Page 42: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example (after Iteration 1)

Iteration 2. Subproblem 4 (x1 = 1,x2 = 1).

Max Z = 14+6x3 +4x4

s.t.

5x3 +2x4 ≤ 1x3 + x4 ≤ 1x3 ≤ 1

x4 ≤ 1xj ∈ {0,1}, j = 3,4.

Its LP relaxation is to replace xj ∈ {0,1} by 0≤ xj ≤ 1 for j = 3,4.Solving it by simplex method: (x1,x2,x3,x4) = (1,1,0, 1

2 , which is notinteger valued, and with Z = 16.Bound for Subproblem 4: Z ≤ 16, larger than the incumbent Z∗ = 9Thus Subproblem 4 is not fathomed.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 42 / 58

Page 43: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example (after Iteration 2)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 43 / 58

Page 44: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example (...after Iteration 3)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 44 / 58

Page 45: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example (...after Iteration 4)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 45 / 58

Page 46: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example: the Iteration Procedure

Operation 1. Bounding Problem [all], to obtain the solutionXall = (5

6 ,1,0,1) and the bound Zall = 16.Operation 2. Branching Problem [all] since X0 is notinteger-valued, into Subproblem [0] (x0 = 0) and [1] (x1 = 1).

Operation 2.1. Bounding Subproblem [0], to obtain the solutionX0 = (0,1,0,1) and the bound Z0 = 9. [0] fathomed and set theincumbent Z∗ = Z0 = 9 since X0 is the first integer-valued solution sofar.Operation 2.2. Bounding Subproblem [1], to obtain the solutionX1 = (1, 4

5 ,0,45 ) and the bound Z1 = 16. [1] not fathomed because X1

is not integer-valued .

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 46 / 58

Page 47: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example: the Iteration Procedure (cont.)

Operation 3. Branching Problem [1] since Z1 = 16 > 9 = Z∗, intoSubproblem [10] (x2 = 0) and [11] (x2 = 1).

Operation 3.1. Bounding Subproblem [10], to obtain the solutionX10 = (1,0, 4

5 ,0) and the bound Z10 = 13 > 9 = Z∗. [10] not fathomed.Operation 3.2. Bounding Subproblem [11], to obtain the solutionX11 = (1,1,0, 1

2 ) and the bound Z11 = 16 > 9 = Z∗. [11] not fathomed.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 47 / 58

Page 48: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Operation 4. Branching Subproblem [11] before Subproblem [10]because Z11 = 16 > 13 = Z10, into Subproblem [110] (x3 = 0) and[111] (x3 = 1).

Operation 4.1. Bounding Subproblem [110], to obtain the solutionX110 = (1,1,0, 1

2 ) and the bound Z110 = 16 > 9 = Z∗.[110] not fathomed.Operation 4.2. Bounding Subproblem [111], to obtain no feasiblesolution so [111] fathomed.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 48 / 58

Page 49: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Branch-and-Bound Algorithm (cont.)

Completing the example: the Iteration Procedure (cont.)

Operation 5. Branching Subproblem [110] rather than Subproblem[10] because [110] is newly created, into Subproblem [1100](x4 = 0) and Subproblem [1101] (x4 = 1).

Operation 5.1. Bounding Subproblem [1100], to obtain the solutionX1100 = (1,1,0,0) and the bound Z1100 = 14.Subproblem [1100] fathomed and revise the incumbentZ∗ = Z1100 = 14 as X1100 is integer-valued and 14 > 9.Operation 5.2. Bounding Subproblem [1101] to obtain no feasiblesolution thus Subproblem [1101] fathomed.

Operation 6. Fathoming Subproblem [10] due to the fact that therevised incumbent value Z∗ = 14 now is larger than Z10 = 13.STOP because all subproblems are either fathomed orbranched. The optimal solution is X] = X1100 = (1,1,0,0) and theoptimal value is Z] = Z∗ = 14.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 49 / 58

Page 50: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving a MIP: the Branch-and-Bound Algorithm

A mixed integer programming problem (MIP) in general form:

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 50 / 58

Page 51: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving a MIP: the Branch-and-Bound Algorithm(cont.)

A variation of the Branch-and Bound algorithm solves MIP. The onlydifferences are:

1 Selecting the branching variable. In the optimal solution for the LPrelaxation, consider the integer-restricted variables havingnon-integer values, and pick the first one in natural ordering;

2 Branching into two subproblems. Let x∗j be the optimal(non-integer) value of the branching variable in the optimalsolution of the LP relaxation, then the two subproblemscorrespond to "xj ≤ [x∗j ]" and "xj ≥ [x∗j ]+1".

3 Bounding without rounding. This is due to the fact that somevariables are not integer-restricted, so the optimal value is ingeneral not required to be integers.

4 Fathoming conditions. The subproblem is fathomed if the optimalsolution for its LP relaxation have integer values for thoseinteger-restricted variables.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 51 / 58

Page 52: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving a MIP: the Branch-and-Bound Algorithm(cont.)

Solving a MIP with the Branch-and-Bound Algorithm: an example.

See textbook Hillier and Lieberman (10e) Section 12.7.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 52 / 58

Page 53: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Cutting Plane Approach

An example of adding "cuts" to obtain an optimal solution with integervalues for an IP.

Below we follow here Operations Research: Applications andAlgorithms by W.L. Winston, Section 9.8, to present the Cutting PlaneApproach for IP.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 53 / 58

Page 54: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Cutting Plane Approach (cont.)

Optimality test. At optimum, x1 = 3.75 and x2 = 2.25, both arenon-integer values. We pick an constraint with the RHSnon-integer. For example,

x1−1.25s1 +0.24s2 = 3.75 −−−− (∗)

Looking for a cut.Now we write each variable’s coefficient and the constraint’s RHSof Equation (**) in the form x = [x]+ f with 0≤ f < 1, to obtain:

x1−2s1 +0.5s1 +0s2 +0.25s2 = 3+0.75.

Group integer parts into the LHS and non-integer parts the RHS:

x1−2s1−3 = 0.75−0.5s1−0.25s2 −−−− (∗∗)

The "cut" of this iteration is obtained as "RHS of (**) ≤ 0 ", i.e.

0.75−0.5s1−0.25s2 ≤ 0 −−−− (∗∗∗)

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 54 / 58

Page 55: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Cutting Plane Approach (cont.)

The above-obtained cut (constraint) has the following two properties:

1 Any feasible point for the IP will satisfy the cut.2 Current optimal solution to LP relaxation doesn’t satisfy the cut.

Why?

Remark.

A cut "cut off" the current optimal solution to the LP relaxation, butnot any feasible solution to the IP.When the cut to the LP relaxation is added, it is hoped that theoptimal solution to the LP relaxation with the cut constraint addedwill be integer-valued. If it is the case, an optimal solution to the IPis obtained; if not, we look for a new cut.Gomory (1958) found that this process will yield an optimalsolution to an IP after a finite number of iterations (cuts).

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 55 / 58

Page 56: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Cutting Plane Approach (cont.)

Substituting "s1 = 6− x1− x2" and "s2 = 45−9x1−5x2" into the cut"0.75−0.5s1−0.25s2 ≤ 0" to obtain the cut constraint ""3x1 +2x2 ≤ 15"thus the cutting plane "3x1 +2x2 = 15".

Illustration of the cut to the LP relaxation.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 56 / 58

Page 57: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

Solving an IP: the Cutting Plane Approach (cont.)

We put the obtained cut constraint "3x1 +2x2 ≤ 15" into the LPrelaxation. Introducing the slack variable s3 into this constraint, weobtain the following tableau:

Next we use the dual simplex algorithm to solve the above LP(optimal but not feasible): select s3 as the leaving variable since−0.75 < 0 and then select s1 as the entering variable since −0.75 < 0and |1.25÷ (−0.75)|< |0.75÷ (−0.25)|.

Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 57 / 58

Page 58: Unit.9 Integer Programming - Xiaoxi Li's homepage · PDF fileSome Innovative Uses of Binary Variables in Model Formulation ... decision variables take integer values. ... j =0, y j

The dual simplex algorithm, applied to LP satisfying the optimality(positive coefficient in Eq.(0)

)but not the feasibility condition (positive

RHS), is essentially a set of operations on constraints so as reachingfeasibility condition while keeping optimality condition (see Section 8.1in Hillier and Lieberman (10e) for more details).

Using "−0.75" (row-s3, column-s1) as the pivot number, we perform theGaussian elimination operations on rows to obtain

(the same as the

simplex method, to have the column-s1 finally read as (0,0,0,1)T):

At optimum, all variables are integers so it is feasible thus optimal to IP.Operations Research (Li, X.) Unit.9 Integer Programming Dec. 22 - 29, 2016 (revised) 58 / 58