1

Linear Programming:Duality theory

Duality Theory The theory of duality is a very

elegant and important concept within the field of operations research. This theory was first developed in relation to linear programming, but it has many applications, and perhaps even a more natural and intuitive interpretation, in several related areas such as nonlinear programming, networks and game theory.

The notion of duality within linear programming asserts that every linear program has associated with it a related linear program called its dual. The original problem in relation to its dual is termed the primal.

it is the relationship between the primal and its dual, both on a mathematical and economic level, that is truly the essence of duality theory.

Examples

There is a small company which has recently become engaged in the production of office furniture. The company manufactures tables, desks and chairs. The production of a table requires 8 kgs of wood and 5 kgs of metal and is sold for $80; a desk uses 6 kgs of wood and 4 kgs of metal and is sold for $60; and a chair requires 4 kgs of both metal and wood and is sold for $50. We would like to determine the revenue maximizing strategy for this company, given that their resources are limited to 100 kgs of wood and 60 kgs of metal.

Problem P1

maxxZ x x x 80 60 501 2 3

8 6 4 100

5 4 4 60

0

1 2 3

1 2 3

1 2 3

x x x

x x x

x x x

, ,

Problem1 Now consider that there is a much

bigger company which has been the lone producer of this type of furniture for many years. They don't appreciate the competition from this new company; so they have decided to tender an offer to buy all of their competitor's resources and therefore put them out of business.

Problem D1

8 5 80

6 4 60

4 4 50

0

1 2

1 2

1 2

1 2

y y

y y

y y

y y

,

minyw y y 100 601 2

A Diet Problem An individual has a choice of two types of food

to eat, meat and potatoes, each offering varying degrees of nutritional benefit. He has been warned by his doctor that he must receive at least 400 units of protein, 200 units of carbohydrates and 100 units of fat from his daily diet. Given that a kg of steak costs $10 and provides 80 units of protein, 20 units of carbohydrates and 30 units of fat, and that a kg of potatoes costs $2 and provides 40 units of protein, 50 units of carbohydrates and 20 units of fat, he would like to find the minimum cost diet which satisfies his nutritional requirements

Problem P2

80 40 400

20 50 200

30 20 100

0

1 2

1 2

1 2

1 2

x x

x x

x x

x x

,

minxZ x x 10 21 2

Examples

Now consider a chemical company which hopes to attract this individual away from his present diet by offering him synthetic nutrients in the form of pills. This company would like determine prices per unit for their synthetic nutrients which will bring them the highest possible revenue while still providing an acceptable dietary alternative to the individual.

Problem D2

maxyw y y y 400 200 1001 2 3

80 20 30 101 2 3y y y

40 50 20 21 2 3y y y

y y y1 2 3 0, ,

FINDING THE DUAL OF A STANDARD LINEAR PROGRAM

In this section we formalize the intuitive feelings we have with regard to the relationship between the primal and dual versions of the two illustrative examples

The important thing to observe is that the relationship - for the standard form - is given as a definition.

Standard form of the Primal Problem

a x a x a x b

a x a x a x b

a x a x a x b

x x x

n n

n n

m m mn n m

n

11 1 12 2 1 1

21 1 22 2 2 2

1 1 2 2

1 2 0

. . .

. ..

. .. .. . . . . .. .

. .. .. . . . . .. .

. ..

, , .. .,

maxx

j jj

nZ c x

1

Standard form of the Dual Problem

a y a y a y c

a y a y a y c

a y a y a y c

y y y

m m

m m

n n mn m n

m

11 1 21 2 1 1

12 1 22 2 2 2

1 1 2 2

1 2 0

.. .

. . .

. .. .. . . . . .. .

. .. .. . . . . .. .

. ..

, , .. . ,

miny

ii

m

iw b y 1

L.P. 15

Linear Programming Duality

max becomes min

RHS coefficients swap places with objective function coefficients

sense changes

x variables go away

y variables appear

L.P. 16

Duality Example

FINDING THE DUAL OF NONSTANDARD LINEAR PROGRAMS

The approach here is similar when we dealt with non-standard formulations in the context of the simplex method.

There is one exception: we do not add artificial variables. We handle “=“ constraints by writing them as “<=“ constraints.

Possibility This is possible here because we do

not require here that the RHS is non-negative.

d x eii

ki

1

d x e

d x e

ii

k

i

ii

k

i

1

1

d x e

d x e

ii

k

i

ii

k

i

1

1

Standard form!

Example

2 4

3 4 5

2 3

0

2 3

1 2 3

1 2

1 2 3

x x

x x x

x x

x x x

, ; urs:= unrestricted sign

maxxZ x x x 1 2 3

Conversion Multiply through the greater-than-or-

equal-to inequality constraint by -1 Use the approach described above

to convert the equality constraint to a pair of inequality constraints.

Replace the variable unrestricted in sign, , by the difference of two nonnegative variables.

2 4

3 4 4 5

3 4 4 5

2 3

0

2 3 3

1 2 3 3

1 2 3 3

1 2

1 2 3 3

x x x

x x x x

x x x x

x x

x x x x

, ,,

, ,,

, ,,

, ,,, , ,

max , ,,

xZ x x x x 1 2 3 3

Dual

y2 y3 y4 1

2y1 3y2 3y3 2y4 1

y1 4y2 4y3 1

y1 4y2 4y3 1

y1, y2, y3, y4 0

minyw y y y y 4 5 5 31 2 3 4

Streamlining the conversion ...

An equality constraint in the primal generates a dual variable that is unrestricted in sign.

An unrestricted in sign variable in the primal generates an equality constraint in the dual.

Example min

yw y y y y 4 5 5 31 2 3 4

y2 y3 y4 1

2y1 3y2 3y3 2y4 1

y1 4y2 4y3 1

y1 4y2 4y3 1

y1, y2, y3, y4 0

y y

y y y

y y

y y y

2 3

1 2 3

1 2

1 3 2

1

2 3 2 1

4 1

0

, ,

, , ,

, ,

, , ,, ;

urs

min,

, , ,

yw y y y 4 5 31 2 3 +

+correction

Primal-Dual relationship

Primal Problem

opt=max

Constraint i : <= form = form

Variable j: xj >= 0 xj urs

opt=min

Dual Problem

Variable i : yi >= 0 yi urs

Constraint j: >= form = form

Example

3 8 6

6 5

8 10

0

1 2

1 2

1

2 1

x x

x x

x

x x

; urs

maxxZ x x 5 41 2

equivalent non-standard form

3 8 6

6 5

8 10

0

1 2

1 2

1

2 1

x x

x x

x

x x; urs

maxxZ x x 5 41 2

Dual from the recipe

3 8 5

8 6 4

0

1 2 3

1 2

1 2 3

y y y

y y

y y y; , urs

minyw y y y 6 5 101 2 3

L.P. Jose Rolim 31

Weak Linear Programming Duality

Any feasible solution to primal LP has value no greater Any feasible solution to primal LP has value no greater than that of any feasible solution to the dual LP.than that of any feasible solution to the dual LP.

L.P. 32

Weak Linear Programming Duality (continued)

L.P. 33

Finding a Dual Solution

Finding a dual solution whose value is equal to that of Finding a dual solution whose value is equal to that of an optimal primal solution…an optimal primal solution…

L.P. 34

Optimality

..

As you remember Given a directed graph ,

two vertices and a capacity for each edge

We want to find a flow functionso that the flow value is maximal

Maximum Flow

:c E N

( , )G V E,s t V

( , )

v V s

f s v

But, we are subject to some rules: What goes in must come out:

Capacity restrictions:

Maximum Flow

( , ) , ( , ) ( , )u v E f u v f v u

, , ( , ) ( , )u V u V

v V s t f u v f v u

( , ) , ( , ) ( , )u v E f u v c u v

We’ll show by a simple example:

Max Flow with Linear Programming

s t

a

b32

13

1

Formally:

Max Flow with Linear Programming

saf

s t

a

b 32

13

1

sbf

abf

atf

btf

atfabfsaf

abfsbf

3211300

But we don’tpermit

equalities

saf sbfmax

btf

So we’ll add another edge, and change the problem’s representation a little.

Max Flow with Linear Programming

s t

a

b 32

13

1

∞

saf

sbf

abf

atf

btf

atfabfsaf

abfsbf

3211300

saftsf 0

0sbf

tsfatf btf

btf

tsfmax

max

max 0 0 0 0 0 1

ts

sa

sb

ab

at

bt

ts

f

ff

fc f

f

f

f

1 0 0 0 0 0 3

0 1 0 0 0 0 2

0 0 1 0 0 0 1

0 0 0 1 0 0 1

0 0 0 0 1 0 3

1 0 1 1 0 0 0

0 1 1 0 1 0 0

1 1 0 0 0 1 0

0 0 0 1 1 1 0

sa

sb

ab

at

bt

ts

A b

ff

f

f

f

f

f

As you remember from Algorithms I Given a directed graph ,

two vertices and a weight for each edge

We want to find a minimal-weight subset of edges such that if we’ll remove them, we won’t be able to travel from to .

Minimum Cut

:c E N

( , )G V E,s t V

s t

In other words: We’ll choose ,

where: , , such that the cut value, is minimal.

Minimum Cut

,s tV V Vss V tt V s tV V V

,

( , )s

t

u Vv V

c u v

Going back to the example:

Min Cut with Linear Programming

s t

a

b32

13

1

Min Cut with Linear Programming

s t

a

b 32

13

1 What about:

Is that enough? We haven’t ensured paths from to

are cut.

( , )

min ( , )

. .

( , ) , 0,1

uvu v E

uv

c u v y

s t

u v E y

This look like

And we know thatIP

IP NPhard

s t

Min Cut with Linear Programming

s t

a

b 32

13

1 Beside a variable for

every edge , we’ll want a variable for every vertex , such that:

uvy

( , )u vuz u V

1

0u s

u t

z u V

z u V

Min Cut with Linear Programming

s t

a

b 32

13

1 Let’s take for instance:

If is in the cut that means that ,

If is not in the cut , that means that either or or

So it is the same to constraint:

( , )a b( , )a b ( 1)aby

,s ta V b V ( 1, 0)a bz z

( , )a b ( 0)aby 1, 1a bz z

0ab a by z z

0, 0a bz z 0, 1a bz z

Formally:

Max Flow with Linear Programming

0sa s ay z z

s t

a

b 32

13

1min 3 2 3sa sb ab at bty y y y y

0sb s by z z

0ab a by z z

0at a ty z z

0bt b ty z z

1s tz z In order to ensure

ss V tt V

min 3 2 3

min 3 2 1 1 3 0 0 0 0

sa sb ab at bt

sa

sb

ab

at

bt

a

b

s

t

y y y y y

y

y

y

yby

z

z

z

z

min 3 2 3

min 3 2 1 1 3 0 0 0 0

sa sb ab at bt

sa

sb

ab

at

bt

a

b

s

t

y y y y y

y

y

y

yby

z

z

z

z

1 0 0 0 0 1 0 1 0 0

0 1 0 0 0 0 1 1 0 0

0 0 1 0 0 1 1 0 0 0

0 0 0 1 0 1 0 0 1 0

0 0 0 0 1 0 1 0 1 0

0 0 0 0 0 0 0 1 1 1

saT

sb

ab

at

bt

a

b

s

t

yA cy

y

y

y

z

z

z

z

We now see that the problems are dual

So also according to the Strong Duality Theorem, Max Flow = Min Cut

Max Flow – Min Cut Theorem