Lecture 6: Cutting Plane Methodand Strong Valid Inequalities

(3 units)

Outline

I Valid inequality

I Gomory’s cutting plane method

I Polyhedra, faces and facets

I Cover inequalities for 0-1 knapsack set

I Lifting and separation of cover inequalities

I Branch-and-cut algorithm

1 / 35

Valid Inequality

I Let X = {x | Ax ≤ b, x ∈ Zn+}. Consider the following linear

program:

(CIP) min cT x

s.t. x ∈ conv(X ).

Then (IP) is equivalent to (CIP).

I Question: How to describe or approximateconv(X ) = {x | Ax ≤ b, x ≥ 0}?

I An inequality πT x ≤ π0 is said to be a valid inequality for Xif πT x ≤ π0 for all x ∈ X .

2 / 35

I Question:(1) How to find good or useful valid inequality?(2) How to use valid inequality to solve an integer program?

I Chvatal-Gomory procedure for VI. X = {x ∈ Zn+ | Ax ≤ b},

A = (a1, . . . , an).(i) (surrogate)

∑nj=1 µTajxj ≤ µTb, µ ≥ 0.

(ii) (round off)∑n

j=1bµTajcxj ≤ µTb.

(iii)∑n

j=1bµTajcxj ≤ bµTbc.I Theorem: every valid inequality of X can be obtained by

applying Chvatal-Gomory procedure for a finite number oftimes.

3 / 35

Gomory’s Cutting Plane Procedure for IP

I Integer programming:

min{cT x | x ∈ Zn+,Ax = b}.

I Create the valid inequalities (cutting planes) directly from thesimplex tableau

I Given an (optimal) LP basis B, write the (pure) IP as

min cTB B−1b +

∑

j∈NB

cjxj

s.t. (xB)i +∑

j∈NB

aijxj = bi , i = 1, 2, ...m,

xj ∈ Z1+, j = 1, 2, ...n,

NB is the set of nonbasic variables.

I cj ≥ 0, j ∈ NB, bi ≥ 0, i = 1, . . . ,m.

4 / 35

I If the LP solution is not integral, then there exists some row iwith bi 6∈ Z.

I The C-G cut for row i is

(xB)i +∑

j∈NB

baijcxj ≤ bbic.

I Substitute for (xB)i to get

∑

j∈NB

(aij − baijc)xj ≥ bi − bbic.

I Let fij = aij − baijc, fj = bi − bbic.∑

j∈NB

fijxj ≥ fi .

I Since the LP optimal solution x∗j = 0 for j ∈ NB, and0 ≤ fij < 1, 0 < fi < 1, this inequality cut off x∗!

5 / 35

I Example 1

min−5x1 − 8x2

s.t. x1 + x2 ≤ 6

5x1 + 9x2 ≤ 45

x1, x2 ∈ Z+

I The feasible region is

6 / 35

I Optimal Simplex Tableau:

x1 x2 x3 x4 b

1 0 2.25 −0.25 2.250 1 −1.25 0.25 3.750 0 1.25 0.75 41.25

I The Gomory cut from the second row of the tableau is:

0.75x3 + 0.25x4 ≥ 0.75

7 / 35

I The modified feasible set is

I The fractional optimal solution x = (2.35, 3.75) violates thecutting plane and is removed from the new feasible set.

8 / 35

I Example 2

min−4x1 + x2

s.t. 7x1 − 2x2 ≤ 14

x2 ≤ 3

2x1 − 2x2 ≤ 3

x1, x2 ∈ Z+

9 / 35

I Optimal Simplex Tableau:

x1 x2 x3 x4 x5 b

1 0 17

27 0 20

70 1 0 1 0 30 0 −2

7107 1 23

7

0 0 47

17 0 −59

7

Cut from the first row of the tableau is:

1

7x3 +

2

7x4 ≥ 6

7.

10 / 35

I Reoptimization:

x1 x2 x3 x4 x5 x6 b

1 0 0 0 0 1 20 1 0 0 −1

2 1 12

0 0 1 0 −1 −5 10 0 0 1 1

2 6 52

0 0 0 0 12 3 −15

2

Cut from the second row of the tableau is:

1

2x5 ≥ 1

2.

11 / 35

I Reoptimization:

x1 x2 x3 x4 x5 x6 x7 b

1 0 0 0 0 1 0 20 1 0 0 0 1 −1 10 0 1 0 0 −5 −2 20 0 0 1 0 6 1 20 0 0 0 1 0 −1 1

0 0 0 0 0 3 1 −7

Done! Optimal solution x∗ = (2, 1)T .

12 / 35

Linear algebra and convex analysis review

I Affine independence: A finite collection of vectors x1, . . . ,xk ∈ Rn is affinely independent if the unique solution to

k∑

i=1

αixi = 0,

k∑

i=1

αi = 0,

is αi = 0, i = 1, 2, ..., k.

I Linear independence implies affine independence, but not viceversa.

I The following statements are equivalent:

1. x1, . . . , xk ∈ Rn are affinely independent.2. x2 − x1, . . . , xk − x1 are linearly independent.3. (x1, 1), . . . , (xk , 1) ∈ Rn+1 are linearly independent.

13 / 35

I A polyhedron is a set of the form{x ∈ Rn | Ax ≤ b} = {x ∈ Rn | aix ≤ bi , i ∈ M}, whereA ∈ Rm@n and b ∈ Rm.

I Dimension of Polyhedra: A polyhedron P is of dimension k,denoted dim(P) = k, if the maximum number of affinelyindependent points in P is k + 1. A polyhedron P ∈ Rn isfull-dimensional if dim(P) = n. LetM = {1, ...,m},M= = {i ∈ M | aix = bi ,∀x ∈ P}, (the equality set),M≤ = M \M= (the inequality set).Let (A=, b=), (A≤, b≤) be the corresponding rows of (A, b).If P ∈ Rn, then dim(P) + rank(A=, b=) = n.

14 / 35

Valid Inequalities and Faces

I The inequality denoted by (π, π0) is called a valid inequalityfor P if πx ≤ π0,∀x ∈ P.

I Note that (π, π0) is a valid inequality if and only if P lies inthe half-space {x ∈ Rn | πx ≤ π0}.

I If (π, π0) is a valid inequality for P andF = {x ∈ P | πx = π0}, F is called a face of P and we saythat (π, π0) represents or defines F .

I A face is said to be proper if F 6= ∅ and F 6= P. Note that aface has multiple representations.

I The face represented by (π, π0) is nonempty if and only ifmax{πx | x ∈ P} = π0.

I If the face F is nonempty, we say it supports P. Note that theset of optimal solutions to an LP is always a face of thefeasible region.

15 / 35

Facets

I Let P be a polyhedron with equality set M=. IfF = {x ∈ P | πx = π0} is nonempty, then F is a polyhedron.We can get the polyhedron F by taking some of theinequalities of P and making them equalities.

I The number of distinct faces of P is finite.

I A face F is said to be a facet of P if dim(F ) = dim(P)− 1.The inequality corresponding to a facet is called a strong validinequality.

I If F is a facet of P, then in any description of P, there existssome inequality representing F . (By setting the inequality toequality, we get F ).

I Every inequality that represents a face that is not a facet isunnecessary in the description of P.

16 / 35

An example

17 / 35

18 / 35

19 / 35

20 / 35

0-1 Knapsack Inequalities

I Valid Inequalities for the Knapsack Problem. We areinterested in valid inequalities for the knapsack set

S = {x ∈ {0, 1}n |N∑

j=1

ajxj ≤ b}.

N = {1, 2, . . . , .n}. Assume that aj > 0, j ∈ N, aj < b, j ∈ N.We are interested in finding facets of conv(S).

I Simple facets. What is dim(conv(S))? 0, ej , j ∈ N are n + 1affinely independent points in conv(S), so dim(conv(S)) = n.

I xk ≥ 0 is a facet of conv(S).

I Proof. 0, ej , j ∈ N \ {k} are n affinely independent pointsthat satisfy xk = 0.

21 / 35

I xk ≤ 1 is a facet of conv(S) if aj + ak ≤ b, ∀j ∈ N \ {k}.I Proof. ek , ej + ek , j ∈ N \ {k} are n affinely independent

points that satisfy xk = 1.

I A set C ⊆ N is a cover if∑

j∈C aj > b. A cover C is aminimal cover if C \ {j} is not a cover ∀j ∈ C .

I If C ⊆ N is a cover, then the cover inequality

∑

j∈C

xj ≤ |C | − 1

is a valid inequality for S .

22 / 35

I Example:

S = {x ∈ B7 | 11x1+6x2+6x3+5x4+5x5+4x6+x7 ≤ 19}.I Minimal covers:

C = {1, 2, 3},C = {1, 2, 6},C = {1, 5, 6},C = {3, 4, 5, 6}.

23 / 35

Can we do better?

I Are these inequalities the strongest ones we can come upwith? What does strongest mean? We all know that facetsare the “strongest”, but can we say anything else?

I If πx ≤ π0 and µx ≤ µ0 are two valid inequalities for P ∈ Rn+,

we say that πx ≤ π0 dominates πx ≤ µ0 if ∃u ≥ 0 such thatπ ≥ uµ, π0 ≤ uµ0 and (π, π0) 6= u(µ, µ0).

I If πx ≤ π0 dominates πx ≤ µ0, then

{x ∈ Rn+ | πx ≤ π0} ⊆ {x ∈ Rn

+ | µx ≤ µ0}.

24 / 35

I Strengthening cover inequalities. If C ⊆ N is a minimal cover,the extended cover E (C ) is defined asE (C ) = C ∪ {j ∈ N | aj ≥ ai ,∀i ∈ C}. If E (C ) is an extendedcover for S , then the extended cover inequality

∑

j∈E(C)

xj ≤ |C | − 1

is a valid inequality for S .

I The cover inequality x3 + x4 + x5 + x6 ≤ 3 is dominated bythe extended cover inequality x1 + x2 + x3 + x4 + x5 + x6 ≤ 3.

25 / 35

I Let C be a minimal cover. If C = N, then

∑

j∈C

xj ≤ |C | − 1

is a facet of conv(S).

I Proof Rk = C \ {k}, ∀k ∈ C . xRk satisfies

∑

j∈C

xRkj = |C | − 1.

Also, xR1 , . . . , xR|C | are affinely independent. Since C = N,there are n affinely independent vectors satisfy∑

j∈C xRkj = |C | − 1 at equality.

26 / 35

I Let C = {j1, . . . , jr} be a minimal cover. Letp = min{j | j ∈ N \ E (C )}. If C = E (C ), and∑

j∈C\{j1} aj + ap ≤ b, then∑

j∈C xj ≤ |C | − 1 is a facet ofconv(S).

I Proof. Tk = C \ {j1} ∪ {k}, ∀k ∈ N \ E (C ).|Tk ∩ E (C )| = |C | − 1 and

∑

j∈Tk∩E(C)

xTkj = |C | − 1.

|Rk |+ |Tk | = N. xRk , k ∈ C , xTk , k ∈ N \ C , are n affinelyindependent vectors.

27 / 35

I Example:

S =∑

{x ∈ {0, 1}5 | 79x1+53x2+53x3+45x4+45x5 ≤ 178}.

I Consider minimal cover C = {1, 2, 3}. The valid inequality is:

x1 + x2 + x3 ≤ 2.

C = E (C ). p = 4, C \ {1} ∪ {4} = {2, 3, 4}.53 + 53 + 45 = 151 < 178. So x1 + x2 + x3 = 2 gives a facetof conv(S).

28 / 35

Lifting Cover Inequalities

I Question: Can we find the the a valid inequality as strong aspossible?

I Example: C = {3, 4, 5, 6}, the valid inequality for C is:

x3 + x4 + x5 + x6 ≤ 3.

I Setting x1 = x2 = x7 = 0, the cover inequalitiesx3 + x4 + x5 + x6 ≤ 3 is valid for{x ∈ {0, 1}4 | 6x3 + 5x4 + 5x5 + 4x6 ≤ 19}.

I If x1 is not fixed at 0, can we strengthen the inequality? Forwhat values of α1 is the inequality

α1x1 + x3 + x4 + x5 + x6 ≤ 3

valid forP2,7 = {x ∈ {0, 1}5 | 11x1 + 6x3 + 5x4 + 5x5 + 4x6 ≤ 19}.

29 / 35

I ⇔ α1 + x3 + x4 + x5 + x6 ≤ 3 is valid for all x ∈ {0, 1}4

satisfying 6x3 + 5x4 + 5x5 + 4x6 ≤ 19− 11;⇔ α1 + ζ ≤ 3, where

ζ = max{x3 + x4 + x5 + x6 | 6x3 + 5x4 + 5x5 + 4x6 ≤ 8}.

I ζ = 1 ⇒ α1 ≤ 2. Thus α1 = 2 gives the strongest inequality.

I How to find the best value αj , j ∈ N \ C such that

∑

j∈C

xj +∑

j∈N\Cαjxj ≤ |C | − 1

is valid for S?

30 / 35

I Lifting ProcedureI Let j1, . . . , jr be an ordering of N \ C . Set t = 1.I The valid inequality

t−1∑

i=1

αji xji +∑

j∈C

xj ≤ |C | − 1

is given. Solve the following knapsack problem:

ζt = maxt−1∑

i=1

αji xji +∑

j∈C

xj

s.t.t−1∑

i=1

aji xji +∑

j∈C

ajxj ≤ b − ajt

x ∈ {0, 1}|C |+t−1.

I Set αjt = |C | − 1− ζt . Stop if t = r .

31 / 35

I Example: C = {3, 4, 5, 6}, j1 = 1, j2 = 2, j3 = 7. α1 = 2.Consider x2, we have

ζ2 = max 2x1 + x3 + x4 + x5 + x6

s.t. 11x1 + 6x3 + 5x4 + 5x5 + 4x6 ≤ 19− 6 = 13,

x ∈ {0, 1}5.

So ζ2 = 2 and αj2 = α2 = 3− 2 = 1.Consider x7 now, we have

ζ3 = max 2x1 + x2 + x3 + x4 + x5 + x6

s.t. 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 ≤ 19− 1 = 18,

x ∈ {0, 1}6.

So ζ3 = 3 and αj3 = α7 = 3− 3 = 0. We obtain a validinequality:

2x1 + x2 + x3 + x4 + x5 + x6 + x7 ≤ 3.

32 / 35

Separation of Cover Inequalities

I We often try to solve problems that have knapsack rows withlots more variables than that... Obviously I do not want toadd all of those facets at once. What to do?

I Given some P, find an inequality of the form∑j∈C xj ≤ |C | − 1 such that

∑j∈C x∗j > |C | − 1. This is

called a separation problem.

I Note that∑

j∈C xj ≤ |C | − 1 can be rewritten as

∑

j∈C

(1− xj) ≥ 1.

33 / 35

I Separation Problem: Given a fractional LP solution x∗, does ∃cover C ⊆ N such that

∑j∈C (1− x∗j ) < 1? or is

γ = minC⊆N

{∑

j∈C

(1− xj) |∑

j∈C

aj > b} < 1?

I If γ ≥ 1, then x∗ satisfies all the cover inequalities.

I If γ < 1 with optimal solution zR , then∑

j∈R xj ≤ |R| − 1 is aviolated cover inequality.

34 / 35

Branch-and-Cut Method

I Branch and cut is an LP-based branch and bound scheme inwhich the linear programming relaxations combined with bycutting plane method.

I The valid inequalities are generated dynamically usingseparation procedures.

I At each node of the search tree, cuts are generated and usedto improve the LP relaxation.

I Branch-and-cut method is very efficient for some hard integerprogramming problems.

35 / 35