ie 5531: engineering optimization i - university of … · ie 5531: engineering optimization i ......

IE 5531: Engineering Optimization I

Lecture 7: Duality and applications

Prof. John Gunnar Carlsson

September 29, 2010

Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 29, 2010 1 / 30

Administrivia

PS 2 posted this evening


Recap

Any linear program

minimize cTx s.t.

Ax = b

x ≥ 0

has a dual of the form

maximize bTy s.t.

ATy ≤ c

If both problems are feasible, we have cTx ≥ bTy for all feasible x and

y

At optimality, we have cTx∗ = bTy∗

�The dual of the dual is the primal�


Duality relationships

Primal minimize cTx maximize bTy Dual

A AT

≥ bi ≥ 0

constraints ≤ bi ≤ 0 variables

= bi free

≥ 0 ≤ cjvariables ≤ 0 ≥ cj constraints

free = cj


Today

Weak & strong duality

Duality examples

Complementary slackness


Weak duality

The weak duality theorem says that if x is a feasible point for the

primal (minimization) problem and y is a feasible point for the dual

(maximization) problem, then we have

bTy ≤ cTx

If optimal solutions exist for both problems, the strong duality theorem

says that

bTy∗ = cTx∗


Proof of weak duality

Let x and y be feasible primal-dual pairs for a standard-form LP, so

that

Ax = b; x ≥ 0; ATy ≤ c

We �nd that

bTy = xTATy ≤ cTx

since x ≥ 0


Proof of strong duality

Suppose that x∗ is an optimal solution to a standard-form LP that we

found via the simplex method, let B denote the basic indices at x∗,and let N denote the non-basic indices

Let x∗B = A−1B b and x∗N = 0

Recall the de�nition of the reduced cost vector r in the simplex

method (the top row):

r = c− AT(A−1B

)TcB

When the simplex method terminates, the reduced cost vector must

be nonnegative, thus

c− AT(A−1B

)TcB︸︷︷︸

y

≥ 0


Proof of strong duality

De�ne y by setting y =(A−1B

)TcB as in the preceding; clearly we have

c ≥ ATy

and thus y is dual feasible

Finally, note that

bTy = bT(A−1B

)TcB =

(A−1B b

)TcB = cTB x

∗B = cTx∗

which completes the proof

y must be optimal for the dual problem, by weak duality


Unboundedness

Note that if one of the problems is unbounded, then the other must be

infeasible

Suppose that cTx is unbounded below in a standard-form LP; weak

duality says that

bTy ≤ cTx

for all x and y

Suppose that y is feasible in the dual; we can choose x to make cTx

arbitrarily small and thus bTy ≤ −∞, a contradiction


Infeasibility

On the other hand, it is not true that if one problem is infeasible, the

other must be unbounded

The following primal-dual pairs are both infeasible:

minimize x1 + 2x2 s.t.

x1 + x2 = 1

2x1 + 2x2 = 3

with dual

maximize y1 + 3y2 s.t.

y1 + 2y2 = 1

y1 + 2y2 = 2


Feasibility and boundedness

An obvious corollary: if the primal problem is infeasible and the dual

problem has a feasible point, then it must be unbounded

A less obvious corollary: unless both problems are infeasible, at least

one of them must have an unbounded feasible set

The following relationships between primal and dual LPs are possible:

Primal

Finite optimum Unbounded Infeasible

Finite optimum Possible X X

Dual Unbounded X X Possible

Infeasible X Possible Possible


Duality example

Consider a transportation problem:

We have m warehouses, each containing an amount ai of a product

We have n stores that need to receive an amount bj of that product

It costs cij to transport one unit of the product from warehouse i to

store j

How can we minimize our costs while satisfying all stores' demands?

Assume that∑m

i=1ai =

∑nj=1

bj (total supply equals total demand)


Duality example

The LP formulation is

minimize

m∑i=1

n∑j=1

cijxij s.t.

m∑j=1

xij = ai ∀i

n∑i=1

xij = bj ∀j

xij ≥ 0 ∀i , j


Duality example

The dual of the transportation problem is

maximize

m∑i=1

aiui +n∑

j=1

bjvj s.t.

ui + vj ≤ cij ∀i , j

Remember:

ai is the amount of the product available at warehouse i

bj is the amount of the product required at store j

cij is the transportation cost from i to j


Duality example

Since there is no sign constraint, we can set ui 7→ −ui (forinterpretation purposes):

maximize

n∑j=1

bjvj −m∑i=1

aiui s.t.

vj − ui ≤ cij ∀i , j

Interpretation: think of the ui 's and vj 's as prices

Speci�cally, think of a new logistics company that wants to ship these

products

The company will buy the goods at each warehouse at a price ui and

sell them back to the warehouses at a price vjIn order to be competitive with the original company's transportation

modes, the new company must have vj − ui ≤ cij ∀i , j (otherwise the

original transportation company won't use them)


Maximum �ow

Recall from earlier the maximum �ow problem:

Given a directed, weighted graph G = (V ,E ,W ) and a pair of nodes

s and t

Think of the edge weights wij as the �capacity� of that edge

What's the largest amount of ��ow� we can send from s to t, subject

to the capacity constraints?


Maximum �ow

In lecture 1 we derived an LP formulation: let xij denote the amount of

�ow across edge (i , j); the problem is

maximize

∑(s,i)∈E

xsi s.t.

xij ≤ wij ∀ (i , j) ∈ E∑(i ,j)∈E

xij =∑

(j ,i)∈E

xji ∀i 6= s, t

xij ≥ 0 ∀ (i , j) ∈ E


Maximum �ow

The dual of the maximum �ow problem is

minimize

∑(i ,j)∈E

wijzij s.t.

zij = yi − yj ∀ (i , j) ∈ E

ys − yt = 1

zij ≥ 0 ∀ (i , j) ∈ E

(see B&T 7.5 for derivation)

We're assigning values yi to the nodes, and incurring costs yi − yj for

all (i , j) ∈ E

We can assume WLOG that ys = 1 and yt = 0

It is not hard to show that, in fact, yi ∈ {0, 1} for all i


Min cut

The preceding problem is equivalent to �nding a subset S of vertices,

such that s ∈ S , that minimizes the cut length of S , i.e.∑i∈S :j /∈S

wij

Thus, strong duality says that the maximum �ow we can push through

the network is equal to the smallest cut size of any subset S of vertices

Corollary: if the value of a �ow is equal to the value of some cut, then

both are optimal


Max-�ow min-cut theorem

Input graph



Maximum �ow



Minimum cut



Consider a standard-form linear program and let Ai denote column i of

A

We know that any constraint in the primal corresponds to a variable in

the dual, and vice versa

An important relationship between optimal primal and dual solutions is

known as complementary slackness, or complementarity:



Theorem

Let x and y be feasible solutions to the primal and dual problem,

respectively. The vectors x and y are optimal solutions for the two

respective problems if and only if

xi > 0 =⇒ ATi y = ci

ATi y < ci =⇒ xi = 0

in other words,

xi

(ATi y − ci

)= 0 ∀i


Proof of complementary slackness

Suppose that

xi > 0 =⇒ ATi y = ci

ATi y < ci =⇒ xi = 0

as in the theorem; then certainly(ATy − c

)Tx =

(yTA− cT

)x

=n∑

i=1

(yTAi − ci

)xi

= 0



Since(yTA− cT

)x = 0, we know that

0 =(yTA− cT

)x

= yTAx− cTx

= yTb− cTx

cTx = bTy

which completes one direction of the proof by weak duality.



To prove the other direction, suppose that x and y are optimal, so

that cTx = bTy (strong duality)

We can write

cTx− bTy = 0

cTx− (Ax)T y = 0

cTx− yTAx = 0(cT − yTA

)x = 0



We can rewrite the equation(cT − yTA

)x = 0

as a summation:n∑

i=1

(ci − yTAi

)xi = 0

Since x and y are feasible, we have

ci − yTAi ≥ 0 ∀ixi ≥ 0 ∀i

so(ci − yTAi

)xi for all i


Non-standard form LPs

Complementarity holds for LPs that are not in standard form as well

Let aTj denote the jth row of A and let Ai denote the ith column of A

A primal-dual pair x, y is optimal if and only if

yj

(aTj x− bj

)= 0 ∀j ∈ {1, . . . ,m}(

ci − yTAi

)xi = 0 ∀i ∈ {1, . . . , n}

where A ∈ Rm×n

In other words: take each variable, multiply it by its associated

constraint; the product must be 0


ie 5531: engineering optimization i - university of … · ie 5531: engineering optimization i ......

Documents