primal dan dualitas

Duality R. Van Slyke p. 1 November 22, 2004

DUALITY

Dual Problems:

It is convenient to start our discussion of duality using the symmetric form of linear programs inmatrix form, which we shall call the primal problem for reasons that will become clear shortly:

(P) minimize z = cxsubject to Ax $ band x $ 0

Associated with (P) is its dual, (D), given by:

(D) maximize w = ybsubject to yA # cand y $ 0.

Note that x is a column vector with n components, y is a row vector with m components, b is acolumn vector with m components, c is a row vector with n components, and A has m rows and ncolumns. We will say that a vector x is feasible for (P) if it satisfies the constraints; it may or maynot minimize the objective. Similarly, y is feasible in (D) if it satisfies the constraints of (D). Wewill often use x and y to denote optimal solutions to (P) and (D), respectively. The two problems* *

(P) and (D) are intimately related. They are called duals of one another. We will prove a theoremcalled the duality theorem that pins this down shortly based on the results of our discussion of therevised simplex method. For now we are content with the:

Weak Duality:

Theorem (Weak Duality): For any x feasible in (P) and y feasible in (D) we have cx $ yb.

Proof: Consider yAx. We have yAx $ yb from the constraint of (P), and we have yAx # cx from theconstraint of (D). Thus cx $ yAx $ yb.

This is a very simple case of the Weak Saddle Point Theorem from “Notes.”

This has immediate, important consequences. Let x denote an optimal solution to (P) and let y* *

denote an optimal solution to (D) we have:

Corollary 1: cx $ yb for any feasible y; that is, yb, for any feasible y is a lower bound on the*

optimal solution of (P).

Similarly.

Corollary 2: y b # cx for any feasible x.*

Finally.

Corollary 3: y b # cx .* *

Of course we are implicitly assuming that optimal solutions to (P) and/or (D) exist when we write x*


and/or y , respectively.*

Note, that if yb = cx for any feasible x and y, then both must be optimal for their respectiveproblems. For example, by the theorem y'b # cx, for any y'. That is, cx is an upper bound for (D). But yb equals the upper bound so that no y' can do better; therefore, y is optimal. A similarargument holds for x. The strong duality theorem will establish that equality holds if both problemsare feasible.

Corollary 4: If cx = yb for any x and y, feasible for (P) and (D), respectively, then x and y are bothoptimal.

Complementary Slackness:

In order to gain a little facility with matrix and vector notation, let us assume, for the time being,that equality is possible. What are conditions for this to be so. From the theorem we see that wemust have cx* = y*Ax* = by* for optimal solutions x* and y*. From the first equality we have that(c-y*A)x* = 0. To explore further the implications of this, let us expand the vector/matrixrepresentation using summations:

Note that the product in each term of the sum is the product of two non-negative values, so thateach term is non-negative, so that each term separately has to be zero. In order for this to happen,one or both of the factors must be zero. Similarly, the implication of yAx = by is that y(Ax-b) = 0,and:

We can summarize all this by:

Theorem (Complementary Slackness): Let x and y be feasible for (P) and (D) respectively, thencx = yb if and only if:

i i iFor each i, i= 1,...,m we have y = 0 or A x = b ,

and

j jFor each j, j = 1,...,n we have x = 0 or yA = c .j

Duality Calculus:

The first thing we observe (which helps motivate the term "duality") is that the dual of the dual isthe primal.


To see this we convert (D) into the form of (P). That is, we make (D) into a minimization problemwith the inequalities running in the other direction. (D) 6 (D') where (D') is given by (thesuperscript T, indicates the array is transposed):

(D') z' = - minimum -b yT T

subject to -A y $ -cT T T

y $ 0.T

We then take the dual of (D') according to the model above, obtaining:

w' = - maximum -u cT T

subject to -u A # -bT T T

u $ 0,

which can be rewritten:

w' = minimize cu subject to Au $ b u $ 0,

which is (P) back again.

As our next exercise with the transformations, we show that duality for linear programs insymmetric form is equivalent to duality for linear programs in the standard form.

The linear program in standard form is:

(SP) minimize cxsubject to Ax = b

x $ 0

We now use our transformations to convert this to the symmetric form, because that is the form weknow the dual of.

(SP') minimize cxsubject to Ax $ b

-Ax $ -b x $ 0

So now we need twice as many dual variables. We call the dual variables associated with the firstset of inequalities y and the set associated with the second set y . We then take the dual of (P')+ -

according to the rules for duals of symmetric programs obtaining:

(SD') maximize y b - y b+ -

subject to y A -y A # c+ -

y $ 0, y $ 0+ -

Now we just note that we have the consequence of one of the transformations we studied. We justset y = y - y . Then y becomes an unrestricted variable; that is, it is not required to be non-+ -

negative. We then reach the final form of the dual:


(SD) maximize ybsubject to yA # c.

Be sure to note that y is unrestricted in sign. Now making use of our discussion of the FundamentalTheorem of Linear Programming, we know that if a linear program in standard form (SP) has anoptimal solution then there is a vector y solving (SD). Thus we have:

Theorem (Strong Duality): If both the primal and dual problems are feasible, they both haveoptima, and they are equal.

Similar manipulations with the transformations yields the following table which allows one tocalculate the dual of any linear program without converting it first into the symmetric or standard forms.

DUALITY CALCULUS

Minimize cx Maximize yb

i i iA x = b y unrestricted

i i iA x $ b y $ 0

j jx $ 0 yA # cj

j jx unrestricted yA = cj

Example 1: What is the dual of:

1 2 3 4Minimize z = 3x + 2x - 5x + x1 2 3 4subject to x + x + x + x = 11 2 3 4 -x + x - 3x - x # 22 x $ 0

we get as its dual:

1 2w = maximize y - 2y1 2subject to y + y = 31 2 y - y # 21 2 y + 3y = -51 2 y + y = 12 y $ 0.

Example 2: What is the dual for:

maximize cx + fysubject to Ax # b

By = dx $ 0

Since the dual of the dual is the primal we can take the given system to be a “dual” and usetransforms from the second column to the first in our table.


This yields:

minimize ub + vds.t. uA $ c

vB = fu $ 0.

The way to check the answer is to see if they two systems lead to weak duality relations. That is,can we prove that cx + fy # ub + vd for feasible (x,y) and (u,v)?

cx + fy # (uA)x + (vB)y = u(Ax) + v(By) # ub + vd using the second set of equations for the firstinequality and the original set of equations for the second inequality. If both solutions (x,y) and(u,v) are optimal, then the above relation holds with equality and we get the complementaryslackness conditions:

i i i i iA x < b => u = 0 u > 0 => A x = 0

j j j juA > c => x = 0 x > 0 => uA = cj j

Theorems of Alternatives:

A useful and fun application of duality is to derive theorems of alternatives. Theorems ofalternatives take the form of two systems of linear relations, in which one or the other is feasiblebut not both. The first systematic treatment of such theorems that I am aware of is in Chapter 2 ofOlvi Mangasarian’s book: Nonlinear Programming, McGraw-Hill, 1963 (recently republished bythe SIAM). He discusses almost twenty such theorems. We will use one, for example, to prove theKarush-Kuhn-Tucker Conditions. However, Mangasarian does not use duality theory to prove themas we do here. The most famous of the theorems of alternatives is Farka’s Lemma:

Farka’s Lemma (1902): Either

Ax = b, x $ 0 has a solution

or

uA # 0, ub > 0 has a solution

but not both.

Remark: Farka’s Lemma has a nice geometrical interpretation. Consider the columns of A asvectors in E . Then the set {Ax| x $ 0} is the convex cone starting at the origin generated by them

columns. The first relations assert that the column b, is in that cone.

The second set of relations can be interpreted as there being a hyperplane {x|ux = 0}, with the coneon one side and b on the other.


Geometric View of Farka’s Lemma

Proof: We assume without loss of generality that b $ 0 since we can multiply any equation of Ax =b by -1. Consider the following “Phase I” problem:

minimize w = eysubject to Ax + Iy = b, x $ 0 y $ 0

Where e is a m dimensional row vector of all ones, and A is mxn. Observe that the supplementalproblem is feasible; e.g., y = b, x = 0. And w is bounded below by 0. So our problem must have anoptimal solution. Either the solution is 0 or it is greater than 0. If it is zero we can take out all theauxilliary variables y, and we have a solution to the first system. If the minimum is greater than 0,we apply the duality theorem to the first system. This tells us that there exits an m-component rowvector u, satisfying uA # 0, uI # e and ub = min w >0. This implies the second system.

Both systems can’t be true. Suppose they were, then multiply the first on the left by u. This givesus: uAx = ub. The left is # 0, because uA # 0 and x $ 0. The right is positive, which yields acontradiction.

Other theorems of alternatives: Gordan’s Theorem (1873): Either Ax > 0 has a solution, x, or yA = 0; y $ 0, y � 0 has a solution,but not both.

Proof: First we note that Ax > 0 has a solution if and only if Ax $ e has a solution, where e is an m-component column vector of all ones. Clearly a solution of the second satisfies the first. Now takeany solution of the first and multiply each component of the solution by 1/d where d is the smallest


component of Ax. This new vector x’ solves the second system.

As before we construct a “Phase I” problem:

Minimize euSubject to Ax + Iu = e

u$0

Our auxiliary, Phase I, problem has the feasible solution x = 0, u = e. The objective is boundedbelow by 0. Therefore the auxiliary problem has an optimal solution. If the value of the optimalobjective is 0 we can take out the artificial variables because they must all be zero, and the resultingx values provide a solution for the first system in the theorem statement. If the optimum value ofthe auxiliary problem is positive, we take the dual. It is:

Maximize yeSubject to yA = 0

y $ 0y # e

As in Farka’s Lemma, both primal and dual have optimal solutions and the values of these solutionsare equal. Therefore ye > 0, which implies that y cannot be 0. This shows that y solves the secondsystem in the theorem statement. To show that both systems can’t be feasible consider yAx. Fromthe second system (yA)x =0. From the first Ax is a positive vector, and y is not zero, so y(Ax) > 0.Contradiction.

Stiemke’s Theorem (1915):Either Ax $ 0 has a solution, x, with Ax � 0, oryA = 0, y > 0 has a solution,but not both.

Proof: We turn the crank again. Consider the auxilliary problem:

Minimize uSubject to Ax - Is + eu = 0

es + u $ 1 u $ 0

We call the variables s, slack variables. They do not change the problem. They simply represent thedifference between the left and right sides of Ax $ 0. The variable u, however, is called an artificialvariable because it does change things and we eventually want it to be zero so we can get rid of it.That is why we minimize u (notice that u is a scalar and e is a column vector of all ones).

The inequality es + u = 1 guarantees that Ax � 0. Again there is a feasible solution to theauxiliary problem and its objective is bounded below so that both the auxiliary problem and its dualhave optimal solutions. If the primal optimal value is 0, the first set of relations in the theoremstatement is feasible. If not, we turn to the dual, which is:

Maximize wSubject to yA # 0

-yI + we # 0


ye + w # 1

For the optimal dual solution, w > 0. This and the second inequality of the dual implies y > 0.Again we can derive a contradiction if we assume both systems in the theorem statement havesolutions by examining yAx.

Gale’s Theorem (1960):Either Ax # c has a solution, x, oryA = 0, yc = -1, y $ 0 has a solution y,but not both.

Note: c is not generally non-negative otherwise the first system would be trivial.

Proof: Consider the auxiliary system:

Minimize euSubject to -Ax + ue $-cIt has a feasible solution for x = 0, and u equal to or larger than the absolute value of the mostnegative component of c. There is an obvious initial feasible solution and the objective is boundedbelow, so both the primal and dual have optimal solutions. If the value of the optimal solution tothe auxiliary problem is 0, we have a feasible solution to the first of Gale’s systems. If not we lookat the dual:

Maximize -ycSubject to -yA = 0

yI # e y $ 0

For the optimal solution of the dual, we have -yc > 0. This implies there is a y so that yA = 0, yc <0, y $ 0. Similar to the logic we used in the proof of Gordon’s Theorem, y can be scaled so that yc= -1. Finally, considering yAx leads to a proof that both systems can’t be feasible.

primal dan dualitas

Documents