linear programming - university of albertaaksikas/6opt_lp_simplex.pdf · linear programming our...

© Fraser Forbes, 1997 LP Simplex 1

Linear Programming

Our market gardener example had the form:

We need a more systematic approach to solving theseproblems, particularly when there are many variables andconstraints.

� SIMPLEX method (Dantzig).� always move to a vertex which improves the value of

the objective function.

[ ]

[ ]

min - 900 1500

subject to:

1.5 220 60

3

60

where: acres cabbages acres tomatoes T

x x

x

x

Ñ

ÓÒ

á

ãâ �

Ñ

ÓÒá

ãâ

¢ .


Linear Programming

SIMPLEX Algorithm

1) Convert problem to standard form with:• positive right-hand sides,• lower bounds of zero.

2) Introduce slack / surplus variables:• change all inequality constraints to equality

constraints.

3) Define an initial feasible basis:• choose a starting set of variable values which

satisfy all of the constraints.

4) Determine a new basis which improves objective function:• select a new vertex with a better value of P(x).

5) Transform the equations:• perform row reduction on the equation set.

6) Repeat Steps 4 & 5 until no more improvement in the objective function is possible.


Linear Programming

Market Gardener Revisited

The problem was written:

1) in standard form, the problem is:

[ ]

[ ]

min - 900 1500

subject to:

1.5 220 60

3

60

where c t T

x x

x

x 0

x

Ñ

ÓÒ

á

ãâ �

Ñ

ÓÒá

ãâ

�

¢ .

maxc,t 900 c + 1500 t

subject to: 1.5 c + 2 t 3 20 c + 60 t 60

�

�

trucking

fertilizer


Linear Programming


2) Introduce slack variables ( or convert all inequality constraints to equality constraints):

or in matrix form:

minc,t - 900 c - 1500 t

subject to: 1.5 c + 2 t + s = 3 20 c + 60 t + s = 60 c, t, s , s 0

1

2

1 2 �

[ ]

[ ]

min - 900 1500 0 0

subject to:

1.5 2 1 0

3

60

where c t s s1 2T

x x

x

x 0

x

20 60 0 1Ñ

ÓÒ

á

ãâ =

Ñ

ÓÒá

ãâ

�

¢ .

trucking

fertilizer


Linear Programming


3) Define an initial basis and set up tableau:

• choose the slacks as the initial basic variables,• this is equivalent to starting at the origin.

the initial tableau is:

objective function value

c t s1 s2 b

s1s2

1.520.0

-900

2.060.0

-1500

1.00.0

0.0

0.01.0

0.0

3.060.0

0.0

basisvariables

objective function coefficients


Linear Programming


4) Determine the new basis:• examine the objective function coefficients and

choose a variable with a negative weight. (youwant to decrease the objective function becauseyou are minimizing. Usually we will choose themost negative weight, but this is not necessary).

• this variable will be brought into the basis.• divide each element of b by the corresponding

constraint coefficient of the new basic variable.• the variable which will be removed from the basis

is in the pivot row (given by the smallest positiveratio of bi/aij ).

c t s1 s2 b

s1s2

1.520.0

-900

2.060.0

-1500

1.00.0

0.0

0.01.0

0.0

3.060.0

0.0

most negative coefficient,bring "t" into the basis

b1/a12 = 3/2b2/a22 = 1

pivot

new basicvariable

new non-basicvariable


Linear Programming


5) Transform the constraint equations:• perform row reduction on the constraint equations

to make the pivot element 1 and all other elementsof the pivot column 0.

c t s1 s2 b

s1t

5/61/3

-400

0.01.0

0.0

1.00.0

0.0

-1/301/60

25.0

1.01.0

1500

i) new row #2 = row #2 / 60ii) new row #1 = row #1 - 2*new row #2iii) new row #3 = row #3 + 1500* new row #2


Linear Programming


6) Repeat Steps 4 & 5 until no more improvement is possible:

c t s1 s2 b

s1t

5/61/3

-400

0.01.0

0.0

1.00.0

0.0

-1/301/60

25.0

1.01.0

1500

c t s1 s2 b

ct

1.00.0

0.0

0.01.0

0.0

6/5-1/3

480

-1/253/100

9.0

6/53/5

1980

pivot

the new tableau is:

no further improvements are possible, since there are nomore negative coefficients in the bottom row of the tableau.


Linear Programming


The final tableau is:

optimal acres ofcabbages & tomatoes

c t s1 s2 b

ct

1.00.0

0.0

0.01.0

0.0

6/5-1/3

480

-1/253/100

9.0

6/53/5

1980

optimal basis contains bothcabbages and tomatoes. maximum

profit

shadow prices


Linear Programming


Recall that we could graph the market garden problem:

We can track how the SIMPLEX algorithm moved through thefeasible region. The SIMPLEX algorithm started at the origin.Then moved along the tomato axis (this was equivalent tointroducing tomatoes into the basis) to the fertilizer constraint.The algorithm then moved up the fertilizer constraint until itfound the intersection with the trucking constraint, which isthe optimal solution.

c (a

cres

)

t (acres)

fertilizer

trucking

increasingprofit

feasibleregion

1 2

1

2

3

profit contoursx*

start


Linear Programming

Another SIMPLEX Example:

We have been commissioned by one of Ontario’s most famouswineries to schedule their production. The winery makes threetypes of wine: sweet, regular and dry. The profit per litre,required raw materials and labour for each of these types ofwine is:

The winery has available 80 kilograms of sugar, 225 hours oflabour, 150 bushels each of grade A and grade B grapes perday. What product mix of wines will maximize the companiesdaily profit.

1) What is the physical interpretation of each slack variable?2) What can we do to increase daily profit for the winery?3) What will it cost us to deviate from the optimum mix?

Grade Agrapes (bu)

0.100.20

0

Grade Bgrapes (bu)

0.100

0.20

sugar(kg)

0.20.10

labour(hr)

0.40.60.2

profit

$1.40$1.00$1.60

winetype

sweetregular

dry


Linear Programming

Some Other Considerations

1) Negative lower bounds on variables:• SIMPLEX method requires all variables to have

non-negative lower bounds (i.e. x � 0).• problems can have variables for which negative

values are physically meaningful (e.g. inter-mediate cross-flows between parallel processunits).

• introduce artificial variables.

If we have bounds on the variable x1 such that:-a � x1 � b

Then we introduce two new artificial variables (e.g. x5, x6)where:

x1 = x5 - x6

and we can require:x5 � 0 , x6 � 0

while satisfying the bounds on x1.


Linear Programming


2) Mixed constraints:• until this point we have always been able to

formulate our constraints sets using only �constraints (with positive right-hand sides).

• how can we handle constraint sets that alsoinclude � constraints (with positive right-handsides).

• introduce surplus variables.

Suppose we have the constraint: [ai1 ... ain] x � bi

The we can introduce a surplus variable si to form an equalityconstraint by setting:

ai1 x1 +... + ain xn - si = bi

We have a problem if we start with an initial feasible basissuch that x = 0 (i.e. starting at the origin). Then:

si = -bi

which violates the non-negativity constraint on all variables(including slack / surplus variables).

We need a way of finding a feasible starting point for thesesituations.



3) Origin not in the feasible region:

The SIMPLEX algorithm requires a feasible starting point. Wehave been starting at the origin, but In this case the origin isnot in the feasible region. There are several differentapproaches:

� “Big M” method (Murty, 1983).

Linear Programming

increasingprofit

feasibleregion

profit contours

x1

x2


Linear Programming

“Big M” Method

Consider the minimization problem with mixed constraints:

1) As well as the usual slack / surplus variables, introduce a separate artificial variable “ai” into each equality and “�” constraint:

2) Augment the objective function by penalizing non-zero values of the artificial variables:

minx x + 2x

subject to: 3x + 4x 5 x + 2x = 3 x + x 4

1 2

1 2

1 3

1 2

�

�

3x + 4x - s + a = 5 x + 2x + a = 3 x + x + s 4

1 2 1 1

1 3 2

1 2 2 =

P( ) = + MaTi

i= 1

m

x c x Í


Linear Programming

“Big M” Method

Choose a large value for “M” ( say 10 times the magnitude of the largest coefficient in the objective function). For the example minimization problem:

3) Set up the tableau as usual. The initial basis will include all of the artificial variables and the required slack variables:

P( ) = x + 2x + 20a + 20 a1 2 1 2x

x1 x2 x3 s1 s2 a1 a2 b

a1a2s2

3.01.01.0

1.0

4.00.01.0

2.0

0.02.00.0

0.0

-1.0 0.0 0.0

0.0

5.03.04.0

0

0.0 0.0 1.0

0.0

1.0 0.0 0.0

20.0

0.0 1.0 0.0

20.0


Linear Programming

“Big M” Method

Phase 1 modify the objective row in the tableau toreduce the coefficients of the artificial variables to zero:

x1 x2 x3 s1 s2 a1 a2 b

a1a2s2

3.01.01.0

-59

4.00.01.0

-78

0.02.00.0

0.0

-1.0 0.0 0.0

20.0

5.03.04.0

-100

0.0 0.0 1.0

0.0

1.0 0.0 0.0

0.0

0.0 1.0 0.0

20.0

x1 x2 x3 s1 s2 a1 a2 b

a1a2s2

3.01.01.0

-79

4.00.01.0

-78

0.02.00.0

-40

-1.0 0.0 0.0

20

5.03.04.0

-160

0.0 0.0 1.0

0.0

1.0 0.0 0.0

0.0

0.0 1.0 0.0

0.0


Linear Programming

“Big M” Method

4) Phase 2, Solve the problem as before starting from the final tableau from Phase 1.

x1 x2 x3 s1 s2 a1 a2 b

a1a2s2

3.01.01.0

-79

4.00.01.0

-78

0.02.00.0

-40

-1.0 0.0 0.0

20

5.03.04.0

-160

0.0 0.0 1.0

0.0

1.0 0.0 0.0

0.0

0.0 1.0 0.0

0.0

x1 x2 x3 s1 s2 a1 a2 b

x1a2s2

1.00.00.0

0.0

4/3-4/3-1/3

82/3

0.02.00.0

-40

-1/3 1/3 1/3

-19/3

5/34/37/4

-85/3

0.0 0.0 1.0

0.0

1/3 -1/3 -1/3

79/3

0.0 1.0 0.0

0.0


Linear Programming

the final tableau is:

The optimal solution is:x1 = 5/3, x2 = 0, x3 = 2/3, P(x) = 5/3

What would be the effect of:1) introducing x2 into the basis?2) introducing s1 into the basis?3) introducing a1 or a2 into the basis?

x1 x2 x3 s1 s2 a1 a2 b

x1x3s2

1.00.00.0

0.0

4/3-2/3-1/3

2/3

0.01.00.0

0.0

-1/3 1/6 1/3

1/3

5/32/37/3

-5/3

0.0 0.0 1.0

0.0

1/3 -1/6 -1/3

59/3

0.0 1/2 0.0

20


Linear Programming

SIMPLEX Method Summary

1) Write the problem in standard form,

2) Introduce slack, surplus and artificial variables asrequired,

3) Choose a starting point (usually x = 0),

4) Determine a new basis which improves the value ofthe objective function,

5) Transform the equation set,

6) Repeat steps 4 & 5 until no further improvement ispossible.


SIMPLEX Algorithm (Matrix Form)

To develop the matrix form for the Linear Programmingproblem:

we introduce the slack variables and partition x, A and c asfollows:

Then, the Linear Programming problem becomes:

min

:x

c x

Ax bx 0

T

subject to≤≥

[ ]

xx

x

A B N

cc

c

=

=

=

B

N

B

N

− − −

− − −

M

basic

non-basic

min

:x

c x c x

Bx Nx bx x 0

+ ,

T TB B N N

B N

B N

subject to

+

=≥



Feasible values of the basic variables (xB) can be defined interms of the values for non-basic variables (xN):

The value of the objective function is given by:

or:

Then, the tableau we used to solve these problems can berepresented as:

[ ]x B b NxB N = --1

[ ]P B N N N( )x c B b Nx c x = -T -1 T+

[ ]P B N B N( )x c B b c c B N x = -T -1 T T -1+

I B-1 NxB

xBT xN

T

0

b

B-1b

[ ]- -T T -1c c B NN B c B bBT -1



The Simplex Algorithm is:

1) form the B and N matrices. Calculate B-1.

2) calculate the shadow prices (reduced costs) of the non- basic variables (xN):

3) calculate B-1 N and B-1 b.

4) find the pivot element by performing the ratio test using the column corresponding to the most negative shadow price.

5) the pivot column corresponds to the new basic variable and the pivot row corresponds to the new non-basic variable. Modify the B and N matrices accordingly. Calculate B-1.

6) repeat steps 2 through 5 until there are no negative shadow prices remaining.

[ ]- -T T -1c c B NN B



This method is computationally inefficient because you mustcalculate a complete inverse of the matrix B at each iterationof the SIMPLEX algorithm. There are several variations onthe Revised SIMPLEX algorithm which attempt to minimizethe computations and memory requirements of the method.Examples of these can be found in:

ChvatalEdgar & Himmelblau (references in §7.7)FletcherGill, Murray and Wright

linear programming - university of albertaaksikas/6opt_lp_simplex.pdf · linear programming our...

Documents