johan h ogdahl, chapter 9: linear solution methods...

DiscreteOptimization

Johan Hogdahl,Victoria Svedberg

Chapter 9: LinearProgramming:Solution methods

9.7 The Dual SimplexMethod

9.8 The RevisedSimplex Method

Chapter 10:NetworkOptimizationProblems andSolutions

10.1 Networkfundamentals

10.2 A Class of EasyNetwork Problems

10.3 TotallyUnimodular Matrices

10.4 The NetworkSimplex Method

Discrete OptimizationSeminar 8: Chapter 9 and 10 in Chen et al.

Johan Hogdahl Victoria Svedberg

April 28, 2015











Outline

Chapter 9: Linear Programming: Solution methods9.7 The Dual Simplex Method9.8 The Revised Simplex Method

Chapter 10: Network Optimization Problems and Solutions10.1 Network fundamentals10.2 A Class of Easy Network Problems10.3 Totally Unimodular Matrices10.4 The Network Simplex Method











9.7 The Dual Simplex Method

The dual simplex method is usefull in the following cases:

(1) Reoptimizing a LP after one or more constraints havebeen added.

(2) Reoptimizing a LP after changing the right-hand side ofthe constraints.

(3) Solve LP problems.

In case (1) and (2) it is possible that negative values on theRHS is introduced, which makes the current basic solutioninfeasible. But since the values in the objective row is allnonnegative, the dual problem is still feasible. Since it is anefficient way of reoptimizing LP problems when addingconstraints the dual simplex method is important in IPsolvers as branch-and-cut for example.












Example

Introducing infeasibility when adding a constraint.

Basic Variable z x1 x2 s1 s2 s3 RHS

z 1 0 0 2 0 0 20

x2 0 0 1 2 -1 0 4x1 0 1 0 -1 1 0 2s3 0 0 0 -1 -1 1 -2












The Dual Simplex Method

I Step 0 (Initialize): Obtain a starting dual feasiblesolution (*).

I Step 1 (Optimality test): Check if b ≥ 0, then stop.

I Step 2 (Leaving variable): Determine the leavingvariable, xi , by selecting a pivot row r with the mostnegative valued RHS.)

I Step 3 (Infeasibility test): If arj ≥ 0 for all j , then theproblem has no feasible solution, then stop.

I Step 4 (Entering variable): Determine the enteringvariable, xk , by selecting the pivot column k based onthe minimum ratio test (**).

I Step 5 (Pivoting): Update the current simplex tableauby pivoting. Return to step 1.












Remarks

(*) The dual feasible solution is obtained by introducingnew constraints using the slackvariable as a basic variable.If the tableau is not in canonical form, then make neccecaryrow manipulations to obtain a simplex tableau. The dualsolution is now feasible if all components in the objectiverow is nonnegative. If the dual solution is infeasible one canintroduce the following artificial constraint∑

j∈Nxj ≤ M

and perform neccecary operations.

(**) The minimum ratio test in the dual simplex context is

ck

−ark= min

j

{ck

−ark; arj < 0

}












Example

Consider a LP where that is solved to optimality using thesimplex method. Suppose we want to add the constraint3x1 + 2x2 ≤ 12. Reoptimize using the dual simplex method.The simplex tableau from the initial LP is:

Basic Variable z x1 x2 s1 s2 RHS

z 1 0 0 2 1 20

x2 0 0 1 2 -1 4x1 0 1 0 -1 1 2












Example

Adding the slackvariable s3 to the new constraint andinserting it into the simplex tableau using s3 as a basicvariable gives us


z 1 0 0 2 0 0 20

x2 0 0 1 2 -1 0 4x1 0 1 0 -1 1 0 2s3 0 3 2 0 0 1 12

But this is not in canonical form.












Example

Some simple row manipulations gives us the followingfeasible solution.


z 1 0 0 2 0 0 20

x2 0 0 1 2 -1 0 4x1 0 1 0 -1 1 0 2s3 0 0 0 -1 -1 1 -2

We determine s3 as the leaving variable, since its RHS ismost negative. By computing min{2/1, 0/1} we determines2 as the entering variable.












Example

Updating the simplex tableau by pivoting now gives us


z 1 0 0 2 0 0 20

x2 0 0 1 3 0 -1 6x1 0 1 0 -2 0 1 0s2 0 0 0 1 1 1 2

Since each component in the RHS is nonnegative we havefound the optimal solution.(The tableau in the book contains typos)











9.8 The revised simplex method

I For the standard simplex method much computationtime is needed just for updating the columns.

I Since not all the information in the standard simplex isneeded computations can be done more efficient.

I The revised simplex method is an common softwareimplementation of the simplex method.

I Recall the LP problem in partitioned form:

maximize z = cTB xB + cTNxN

subject to BxB + NxN = b

xB , xN ≥ 0












By multiplying the equality constraint with B−1, we obtain

xB + B−1NxN = B−1b (1)

The objective can then be expressed by the nonbasicvariables as

maximize z = cTBB−1b + (cTN − cTBB

−1N)xN (2)

By transferring the variable term in (2) to the left hand sidewe obtain the ordinary simplex tableau

z xB xN RHS

1 0 cTBB−1N− cTN cTBB

−1b

0 I B−1N B−1b












Observations:

z xB xN RHS


−1b

0 I B−1N B−1b

I Once the set of basic variables xB is specified, thecorresponding simplex tableau can be calculated directlyfrom the original data.

I There is no need to update and store the coefficientmatrix associated with the basic variables because thatmatrix is always identity I, and the reduced costassociated with each basic variables is always zero.












Observations:

z xB xN RHS


−1b

0 I B−1N B−1b

I All other entries of the ordinary simplex tableau arecharacterized by pre-multiplying the original data byB−1 . Therefore, only B−1 is required to be updated.

I The reduced cost vector cTN associated with thenonbasic variables is calculated by cTN = cTBB

−1N− cTN ,whose j ’th entry c j = cTBB

−1aj − cj , where columnaj ∈ N.












Observations:

z xB xN RHS


−1b

0 I B−1N B−1b

I The dual solution uT is updated by uT = cTBB−1 ,

which requires to be computed only once for eachsimplex iteration.

I The updated coefficient columns associated withnonbasic variables are B−1N, whose j ’th column isaj = B−1aj .

I The primal solution is computed by xB = B−1b = b,xN = 0, and z = cTBB

−1b = cTBb.












Based on the observations we may now reduce the simplextableau from

z xB xN RHS


−1b

0 I B−1N B−1b

into the revised simplex tableau

cTBB−1 = uT cTBB

−1b = z

B−1 B−1b = b











9.8 The revised simplex methodThe revised simplex method may then be formulated as

I Step 0 (Initialization): Find an initial revised simplextableau. In presence of artificial variables the two phasemethod is applied as usual to find a basic feasiblesolution. Initially B = B−1 = I, uT = cTBB

−1 = cTB ,B−1b = b and z = cTBB

−1b = cTBb.

I Step 1 (Pivot Column): For all nonbasic j andaj ∈ N = A \ B, compute

aj = B−1aj

c j = uTaj − cj

determine the entering variable xk by selecting the mostnegative ck and the pivot column(

ckak

)












The revised simplex method may then be formulated as

I Step 2 (Optimality test):A finite maximum solution hasbeen found if ck ≥ 0 and at least one entry of ak ispositive. An unbounded maximum exists if ck ≥ 0 andak ≤ 0. Otherwise, go to step 3.

I Step 3 (Pivot Row): Append pivot column k anddetermine the leaving variable xB , or pivot row r by

r =

{i : min

i

(bi

aik, aik > 0

)}

I Step 4 (Pivoting): Update the revised simplex tableauby pivoting at ark . Go to step 1.











Network Fundamentals

Definition (10.1)

A network (or graph) G is a collection of nodes (or vertices)and a collection of arcs (or edges) joining pairs of nodes,denoted G = (V ,E ), V = {1, 2, . . . ,m}, andE = {(i , j) : i , j ∈ V }

Definition (10.2)

A path (from node i0 to ip) is a sequence of arcs{(i0, i1), (i1, i2), . . . , (ip−1, ip)} in which the initial node ofeach arc is the same as the terminal node of the precedingnode in the sequence and all nodes i0, i1, . . . , ip are distinct.A chain is a sequence of arcs similar to a path, except thatnot all arcs are necessarily directed toward node ip. Thus,every path is a chain but a chain may not be a path.












Definition (10.3)

Given a network G (V ,E ) and a distance (cost) cijassociated with each directed arc (i , j), the problem todetermine a path from a specified node to another specifiednode with a minimal total distance is called a shortest path(route) problem.

Definition (10.4)

A circuit is a path from some node i0 to ip plus the returnarc (ip, i0). Thus, a circuit is a closed path. Similarly, a cycleis a closed chain and every circuit is a cycle but a cycle maynot be a circuit.












Definition (10.5)

Given a network with m nodes, a tree with k (2 ≤ k ≤ m)nodes is a subnetwork that connects all k nodes with nocycles. A spanning tree is a tree that connects all m nodesin the given network with no cycles.

Properties of a tree

I Every tree (including spanning tree) with k nodes hasexactly (k − 1) arcs.

I Adding any new arc from the original network to aspanning tree results in a unique cycle.

I Every pair of nodes in a tree can be connected by aunique chain.











A Class of Easy Network Problems

The Minimum Cost Network Flow Problem (MCNF)

I Given a directed network G=(V,E) consisting of mnodes and n arcs.

I Each node is either a supply node, a demand node or atransshipment node.

I Each arc is associated with capacity constraints (lowerand upper)) and unit shipping costs for shippingthrough the arc.

I The problem is to determine the amount of flow througheach arc so that the total shipping cost is minimum.

I (We assume that the total supply equals the totaldemand. If that is not the case so called dummy nodescan be added.)












The Minimum Cost Network Flow Problem (MCNF)

The MCNF can be expressed as

minimize z =∑i

∑j

cijxij

subject tom∑j=1

xij −m∑

k=1

xki = bi for each i (3)

xij ≤ Uij for each arc (i , j) (4)

xij ≥ Lij for each arc (i , j) (5)












Formulating the Transportation-Assignment Problem asan MCNF Problem

I Given m1 source nodes, each with si units of supply, andm2 destination nodes, each with dj units of demand.

I Let cij be the unit flow cost from node i to node j andxij be the amount of flow.

I The problems is then to determine xij from each sourceto each destination so that the transportation cost isminimized.












Formulating the Transportation-Assignment Problem asan MCNF ProblemMathematically that can be formulated as

minimize z =∑i

∑j

cijxij

subject to

m2∑j=1

xij = si i = 1, 2, . . . ,m1 (6)

m1∑i=1

xij = di j = 1, 2, . . . ,m2) (7)

xij ≥ 0 for all (i , j) (8)












Formulating the Transportation-Assignment Problem asan MCNF ProblemUsing the following procedure we can reformulate thetransportation-assignment problem as an MCNF problem.

1. Renumber m1 source and m2 destination nodes using acommon index i == 1, 2, . . . ,m1,m1 + 1,m1 + 2, . . . ,m,where m = m1 + m2. The unit transportation costs cijare also renumbered accordingly.

2. Set bi = si for i=1, 2, . . . ,m1 and set bm1+j = −dj forj = 1, 2, . . . ,m2. Note that

∑i si =

∑j dj , implies∑m1

i=1 bi +∑m2

i=1 bm1+i =∑m

i=1 bi = 0.

3. Create a dummy source node (say node 0) withb0 =

∑m1i=1 bi and connect arcs (0, i) for

i = 1, 2, . . . ,m1, with unit cost c0i = 0.












Formulating the Transportation-Assignment Problem asan MCNF ProblemUsing the following procedure we can reformulate thetransportation-assignment problem as an MCNF problem.

5. Create a dummy sink node (say node m + 1) withbm+1 =

∑mi=m1+1 bi and connect arcs (m1 + 1,m + 1)

for i = 1, 2, . . . ,m2 with unit cost ci ,m+1 = 0.

6. Add a return arc (m + 1, 0) with cm+1,0 = 0.

7. Ignore the upper bound constraints (5).












In a similiar way it is also possible to formulate the followingproblems as MCNF problems:

I The assignment problem

I Transshipment Problem

I Maximum Flow Problem

I Shortest Path Problem











Totally unimodular matrices

An important property of MCNF problems is that every basicfeasible solution is naturally integer provided that all Ui andbi is integer. Total unimodularity is a sufficient conditionsthat ensures this happens.

Definition (10.6)

A square matrix whose determinant is 0, 1, or -1 is calledunimodular. A matrix M is totally unimodular (TU) if thedeterminant of every square submatrix of M has value 0, 1,or -1.












Theorem (10.1)

A m × n matrix M is totally unimodular if the followingconditions hold:

1. Every element of M is 0, 1 or -1.

2. Each column of M contains at most two nonzeroelements.

3. The m rows of M can be partitioned into two mutuallyexclusive subsets M1 and M2 such that

(a) If any column contains two nonzero elements of thesame sign, one element can be placed in M1 and theother in M”2.

(b) If any column contains two nonzero elements of theopposite signs, both elements can be placed in the samesubsets.












Example

Check if the following matrix satisfies the conditions oftheorem 10.1.

M =

−1 0 1 0 0 00 0 −1 −1 0 00 1 0 −1 1 01 −1 0 0 0 10 0 0 0 1 −1

Clearly M satisfies condition 1 and 2 in theorem 10.1. If Msatisfies condition 3 in theorem 10.1, then M is TU.












M =

−1 0 1 0 0 00 0 −1 −1 0 00 1 0 −1 1 01 −1 0 0 0 10 0 0 0 1 −1

I Since the nonzero elements of column 1 is of opposite

sign we place row 1 and 4 in the same subset.M1 = {R1,R4}.

I Since row 1 and 4 belongs to M1 and since the nonzeroelements of column 2, 3 and 6 are of opposite signs wemust also place row 2, 3 and 5 in the same subset as 1and 4. M1 = {R1,R2,R3,R4,R6}

I The nonzero elements of column 4 are of the same signand if condition 3 shall be satisfied row 2 and 3 must beplaced in different subsets, which is not possible sincethey already belongs to M1.












Observations

I Every matrix that satisfies condition 1 and 2 in theorem10.1 and whose nonzero elements is of the opposite signin each column is TU.

I Incidence matrices of bipartite networks is TU.

I The incidence matrix of the transportation and theassignment problems are examples off such matrices.

Definition (10.7)

A network G(V,E) is called bipartite if there exists twosubsets of nodes (vertices), V1 and V2, such thatV1 ∪ V2 = V and V1 ∩ V2 = ∅, and every arc (edge) of G isincident to exactly one node of V1 and one node of V2.












Properties of totally unimodular matrices

1. The matrix obtained by adding (deleting) an identitymatrix to (from) a TU matrix is also TU.

2. The transpose of a TU matrix is also TU.

3. The matrix obtained by pivoting on a TU matrix is alsoTU.

4. The matrix obtained by multiplying any row (column)of a TU matrix by -1 is also TU.

5. The matrix obtained by interchanging any two rows(columns) of a TU matrix is also TU.

6. The matrix obtained by deleting (adding) a unit row(column) of a TU matrix is also TU.












Definition (10.8)

A 0-1 matrix is called an interval matrix if in each column(or row) the 1’s appear consecutively (allowing wrappingaround).

It can be shown that an interval matrix is TU.

Example

The A-matrix in the workforce scheduling problem is aninterval matrix.

A =

1 0 0 1 11 1 0 0 11 1 1 0 00 1 1 1 00 0 1 1 1












Matrix structure of the MCNF problem

Consider a traffic problem formulated as a MCNF problem.The flow conservation equations is in matrix form Ax = b,where A may be for instance

A =

1 1 0 0 0 0 0 0 −1−1 0 1 1 0 0 0 0 00 −1 0 0 1 1 0 0 00 0 −1 0 −1 0 1 0 00 0 0 −1 0 −1 0 1 00 0 0 0 0 0 −1 −1 1

The upper boundary constraints is in matrix form Ix ≤ U.Since A is TU and since we know that when appending anidentity matrix to a TU matrix we obtain another TUmatrix. Therefore, we have that the constraints of the trafficproblem is TU.












Naturally Integer Solution for the MCNF problem

The following theorem is the main result of this chapter.

Theorem (10.2)

Every basic feasible (including basic optimal) solution to anLP problem P = {maxcT x : Ax ≤ b, x ≥ 0} is always aninteger solution if A is TU, b is integer, and P has a finiteoptimal solution. In addition, the statement is also true ifIx ≤ u is adjoined to the constraint set of P, provided u isinteger.

From theorem 10.2 it immediately follows that:

Corollary (10.1)

The MCNF problem has integer basic feasible solution andoptimal solution if b and u are integer-valued vectors.












Naturally Integer Solution for the MCNF problem

Proof of theorem 10.2.Rewrite the inequality constraints into equalities. Partitionthe the set of variables into basic and nonbasic variables.That is, x = (xB , xN)T and A = (B,N). ThenAx = b ⇐⇒ BxB + NxN = b. Let xN = 0 and solveBxB = b. Assuming that B is nonsingular, Cramers rule gives

xj =detBj

detB

Where Bj is obtained from replacing the j ’th column of Bwith b. Since b is a vector of integers and every element ofB is integer ⇒ detBj is integer. Since A is TU and Bnonsingular ⇒ detB = ±1. Therefore; xj is integer.











The Network Simplex Method

I A smiplex method that takes advantage of the specialnetwork structure of the MCNF problem.

I Instead of simplex tableau’s it iterates on the networkitself and the only required operations is additions andsubtractions.












Feasible spanning trees

I In the ordinary simplex method we have basic andnonbasic variables, and the basic solution has the samedimension as the rank of A.

I In the network simplex method we instead divide thenetwork in basic and nonbasic arcs. And the basicsolution is a spanning tree.

I Recall that a spanning tree of a m-node networkconsists of m − 1 arcs and that the simplex methodalways starts with a full-rank (m) constraint matrix.

I Therefore an artificial variable is required. That isequivalent of creating an extra arc from some node.This particular arc is called a root arc and theassociated node a root node. The feasible spanning treewith a root node is called a rooted feasible tree.












I Step 0 (Initialization): Find a rooted feasible spanningtree comprising m-1 basic arcs:

(a) Check if∑

bi = 0 If not, add a dummy node and itsassociated dummy arcs.

(b) Designate a demand node (bi < 0) as a root node andcreate a rooted arc incident to the root node.

(c) Create a feasible spanning tree in the following process.Begin with the end nodes and proceed toward therooted node (say m + 1). Assign flows to arcs so that ateach node the net flow (=outflows-inflows) equals bi .

I Step 1 (Node Potential): Compute the node potentialsui , (also called dual variables or simplex multipliers) forall nodes as follows. Begin with the rooted node andset um+1 = 0, and proceed toward the end nodes.Determine ui , iteratively by ui − uj = cij .












I Step 2 (Entering Arc): Compute the reduced costs c ijfor all nonbasic arcs by c ij = ui − uj − cij and thencheck the optimality (minimization) conditions for bothtypes of nonbasic variables. If xij = 0, then the optimalcondition is c ij ≤ 0. If xij = Uij , then the optimalcondition is c ij ≥ 0. If the current solution is notoptimal, then choose an entering arc that most violateseither optimal condition.

I Step 3 (Leaving arc): Form a unique cycle by adding theentering arc to the current spanning tree. Determinethe amount of arc flow δ that can increase withoutexceeding any arc capacity Uij in the cycle and that candecrease without violating the lower bound xij . Theleaving arc will be the one that first hits either 0 or Uij .












I Step 4 (Updating): Find the new feasible spanning tree(feasible solution) by adjusting the arc flows in the cycleso that the flow conservation at each node ismaintained. Go to step 1.

johan h ogdahl, chapter 9: linear solution methods...

Documents