Dual simplex method

Submitted by:Pooja jain

M.com second semesterGroup A roll no. 37

Linear ProgrammingDuring the Second World War, the soviet army was probably more efficient thanks to Leonid Kantorovich. In 1939, this mathematician and economist came up with a new mathematical technic to solve linear programs, hence improving plans of production at lower costs. In 1975, he became the only Nobel Prize winner in Economy from the USSR. After the war, methods of linear programming kept secret got published, including the simplex method by the American mathematician George Dantzig and the duality theory by the Hungarian-American mathematician John von Neumann.Applications in company daily planning rose, increasing efficiency thanks to linear programs. Applications also include transport, supply chains, shift scheduling, telecommunications, yield management, assignment, set partitioning, set covering, advertisement. I’ve also heard of DNA analysis, and I personally use it for cake-cutting.Let’s be smart robbersSuppose we have just attacked a village and we want to steal as much as we can. There’s gold and dollar bills. But we only have one knapsack with a limited volume and we know we can’t take it all. So we’ll have to choose the volumes of each thing to steal. Also assume that we are not strong enough to carry too much weight (I do play sports but I don’t go to the gym, so I have a weak upper body…).You face what is called a mathematical program or an optimization problem. You need to maximize the value of your robbery by choosing volumes of gold and 1 dollar bills with constraints on total volume and total weight. One volume of gold is worth more than one volume of dollar bills but it’s also heavier.The value of the robbery is called the objective function. It’s what we want to maximize or minimize. Volumes of gold and bills are called the variables. Mathematically, we usually write the mathematical program as follows.

Of course, we usually have complicated notations instead of words… But we’ll get to that!So far, what we have here is a mathematical program. There are plenty of methods to solve a mathematical program but none of them can guarantee for sure that we find the solution in a reasonable time, especially if the number of variable is higher than 10. But in our case, we can write our mathematical program as a linear program.A linear program is a mathematical program with a linear objective function and linear constraints.In our case, the objective function is linear since, whenever you add a volume of any product, the total value increases by the value of that volume of that product. That means that we can write the objective function in the following way:

A linear constraint is a constraint that says that a linear function of variables must be higher, lesser or equal to a constant. Now, our first constraint says that the sum of all volumes must be less than some constant limit. This is obviously a linear constraint. Similarly, because the weight

of volumes is the sum of weights of each volume, the total weight is also a linear function. Hence, the second constraint is also linear: our program is linear.Cool! Are we done writing our linear program?Almost. We forgot two constraints. So far volumes are assumed to be real numbers, but they actually need to be non-negatives. I mean stealing -1 litre of gold… that has no meaning. So we just need to add constraints that say that volumes are positive. Yet, these are linear constraint too! So our program is totally linear:

Our problem has two variables. Now any two variables will not necessarily satisfy all the constraints, in which case we will say that the solution is infeasible. For instance, if the volume of gold is -1 and the volume of bills is 0, then the solution is infeasible because the constraint   is not satisfied. If our program had no constraints, any solution would be feasible, but constraints modify the feasible set.What’s interesting is how each constraint modify the feasible set. As a matter of fact, each inequality constraint divides the space into two areas, one for solutions satisfying the constraint, the other for solutions that don’t satisfy the constraint.

The feasible set is the white area. We can see that they we have a finite number of extreme points and that we have the other points inside are points in-between the extreme points. A theorem says that there always exists an extreme point that is one of the points that maximize the objective function.It’s possible that other points also maximize the objective function. Still, it means that if we know all the extreme points of our feasible set, we can easily solve our problem by testing them all! And that’s not all! Any mathematical program can be associated with a dual

program, which is another mathematical program that has interesting properties related to the initial mathematical program. In the case of linear programming, the dual program has even more interesting properties and many reasoning in this dual program enable major improvements.

Duality of LP problemsEach LP problem (called as Primal in this context) is associated with its counterpart known as Dual LP problem. Instead of primal, solving the dual LP problem is sometimes easier when a) the dual has fewer constraints than primal (time required for solving LP problems is directly affected by the number of constraints,) and b) the dual involves maximization of an objective function (it may be possible to avoid artificial variables that otherwise would be used in a primal minimization problem).Duality in Linear ProgrammingIn order to explain duality to, continuing with the example of the smart robber. Basically, the smart robber wants to steal as much gold and dollar bills as he can. He is limited by the volume of his backpack and the maximal weight he can carry. Now, let’s notice that we can write the problem as follows.

The problem we have written here is what we call the primal linear program. The dual program will totally change our understanding of the problem, and that’s why it’s so cool. In the primal program, constraints had constant numbers on their right. These constant numbers

are our resources. They say what we are able to do concerning each constraint. The dual problem consists in evaluating how much our combined resources are worth. If the offer meets the

demand, our resources will be just as much as their potentials, which is the worth of the robbery.

Let’s go more into details. In the dual problem we will attribute values to the resources (as in “how much they’re worth”). These values are called the dual variables. In our case, we have two constraints, so we will have 2 dual variables.

The first dual variable, let’s call it ValueVolume refers to the value of one unit of volume. The second dual variable refers to the value of one unit of weight. Value Weight seems like a right name for it. Now, we can write the value of the robbery with these two new variables… Let’s see if we get the same result.

But how are the values per volume and per weight determined?If I wanted to sell my resources, potential buyers are going to minimize the value of my resources. So their valuations are the minimum of the total value. But as a seller, I will argue that each of my resource is worth a lot, because it enables the robbery of more gold and more bills.Obviously, the values of resources depend on the actual values of gold and bills per volume. Let’s have a thought about the value of gold (and then we’ll be able to apply the same reasoning to bills). If the constraints enabled us to steal one more volume of gold, then incremental value of the robbery would be at least the value of this one volume of gold, right? It could be more, if we use the new constraints to steal something else than gold that’s worth more. What I’m saying is that, if the total volume enabled us to steal one more unit of volume of gold, and if we could carry one more unit of weight of one volume of gold, then the value of this incremental steal would be at least the value of one more volume of gold. Let’s write it.

From this we deduce a constraint on dual variables. As we can see, any variable in the primal problem is associated with a constraint in the dual problem and vice-versa.We are almost done. Let’s notice the fact that if we increase the total volume, then we have more possibilities for the primal variables, which are the volumes of stolen gold and bills. Therefore, the value of a unit of volume cannot be negative. This adds two more constraints on the sign of dual variables. Now, we’re done and we can write the dual problem.

The dual program of a linear program is a linear program! Let’s have a look at the feasible set. It’s important to notice that the information of the primal objective functions appear in the dual feasible set. The most important result is the strong duality property: optimal values of the primal and dual problems are the same. We can solve the primal problem simply by solving the dual problem! And sometimes, the dual problem can be much more simple than the primal one.

Primal-Dual relationshipsFollowing points are important to be noted regarding primal-dual relationship:

Primal DualMaximization MinimizationMinimization Maximization

Ith variable ith constraintjth constraint jth variable

xi >=0 Inequality sign of Ith constraint:<= if dual is maximization>= if dual is minimization

Ith variable unrestricted Ith constraint with = signjth constraint with = sign jth variable unrestricted

RHS of jth constraint Cost coefficient associated with jth

variable in the objective functionCost coefficient associated with ith

variable in the objective functionRHS of ith constraint

Primal and Dual Simplex MethodsWe’ll consider the smart robber problem, that is we can steal bills or gold, but the total stolen volume is limited, as well as the total weight stolen. The simplex methodRecall that, in a linear program, there is necessarily an extreme point that is the optimum.So we could just list all the extreme points and choose the best one…There are two problems with listing all the extreme points. First, there can be a lot of them, especially when considering problems with millions of variables and hundreds of thousands constraints. Second, finding an extreme point can be quite difficult as it involves solving a system with all the constraints that may not lead to a feasible solution. Shortly said, listing all the extreme points is a bad idea.What the simplex method does is moving from extreme points to strictly better extreme points until finding an optimal extreme points. This simple idea is considered as one of the

main breakthroughs of the 20th century in computing! At any extreme point, the simplex method seeks for a next feasible constraint base to visit, which is strictly better than the current constraint base. In order to do that, it looks at the possible direction where it can go to reach a next extreme point. The figure shows what the simplex method would do in our case:The simplex method has become famous and has been used a lot as it enabled the resolution of problems with millions of variables and hundreds

of thousands of constraints in reasonable time. However, it faces problems in cases of degeneracy.

Solving an example with simplex method:Pancakes3 cups Bisquick1 cup Milk2 EggsServes: 6

Waffles2 cups Bisquick2 cup Milk2 EggsServes: 5

You have 24 cups of Bisquick, 18 cups of milk, and 20 eggs. If you want to feed as many people as possible, how many batches of each should you make?

Step 1 All information about example

Resource Pancakes (p) Waffles (w) Constraints

Bisquick (cups) 3 2 24

Milk (cups) 1 2 18

Eggs 2 2 20

Serves 6 5

Objective Function Z= 6p+5wBisquick Constraint 3p+2w ≤ 24

Milk Constraint p+2w ≤ 18Egg Constraint 2p+2w ≤ 20

Non-negativity conditions p,w ≤ 0

The first step of the simplex method requires that each inequality be converted into an equation. ”less than or equal to” inequalities are converted to equations by including slack variables.Suppose S1 bisquick cups, S2 milk cups and S3 eggs remain unused in a week. The constraints become;3p+2w+S1 = 24p+2w+S2 = 182p+2w+S3 = 20As unused hours result in no profit, the slack variables can be included in the objective function with zero coefficients:Z= 6p+5w+0S1+0S2+0S3

The problem can now be considered as solving a system of 3 linear equations involving the 5 variables in such a way that Z has the maximum value;

Now, the system of linear equations can be written in matrix form. The initial tableau is;Step 2

Table 1

Cj6 5 0 0 0

Basic variable

Solution P w S1 S2 S3

0 S1 24 3 2 1 0 00 S2 18 1 2 0 1 00 S3 20 2 2 0 0 1

Zj 0 0 0 0 0 0Cj-Zj - 6 5 0 0 0

Step 3 Select the pivot column (determine which variable to enter into the solution mix). Choose the column with the “most positive” element in the objective function row.

Table 2

Cj6 5 0 0 0

Basic variable

Solution P w S1 S2 S3

0 S1 24 3 2 1 0 00 S2 18 1 2 0 1 00 S3 20 2 2 0 0 1

Zj 0 0 0 0 0 0Cj-Zj - 6 5 0 0 0

p should enter into the solution mix because each batch of p (pancake) serves 6 compared with only 5 for each batch of w (waffle).Step 4 Select the pivot row (determine which variable to replace in the solution mix). Divide the solution element in each row by the corresponding element in the pivot column. The pivot row is the row with the smallest non-negative result.

Table 3

Cj6 5 0 0 0

Basic variable

Solution

P w S1 S2 S3 Ratio

0 S1 24 3 2 1 0 0 8

0 S2 18 1 2 0 1 0 180 S3 20 2 2 0 0 1 10

Zj 0 0 0 0 0 0Cj-Zj - 6 5 0 0 0

Pivot column

Pivot row and 3 is pivot number

S1 Should be replaced by p in the solution mix. 10 pancakes can be made with 20 unused eggs and 18 pancakes can be made with 18 unused milk cups but only 8 pancakes can be made with 24 bisquick cups. Therefore we decide to make 8 tables.Now calculate new values for the pivot row. Divide every number in the row by the pivot number.

Table 4

Cj6 5 0 0 0

Basic variable

Solution P w S1 S2 S3

6 p 8 1 2/3 1/3 0 00 S2 18 1 2 0 1 00 S3 20 2 2 0 0 1

Zj 0 0 0 0 0 0Cj-Zj - 6 5 0 0 0

Use row operations to make all numbers in the pivot column equal to 0 except for the pivot number which remains as 1.

Table 5

Cj6 5 0 0 0

Basic variable

Solution P w S1 S2 S3

6 P 8 1 2/3 1/3 0 00 S2 10 0 4/3 -1/3 1 00 S3 4 0 2/3 -2/3 0 1

Zj 48 6 4 2 0 0Cj-Zj - 0 1 -2 0 0

If 8 pancakes are made, then the unused milk cups are reduced by 8 cups (1 cup/pancake multiplied by 8 pancakes); the value changes from 18 cups to 10 cups. similarly, unused eggs are reduced by 16 (2 egg/pancake multiplied by 8 pancakes); the value changes from 20 eggs to 4 eggs . Making 8 pancakes results in the service being increased by 48; the value changes from 0 to 48.Now repeat the steps until there are no positive numbers in the last row. Select the new pivot column. w should enter into the solution mix.Select the new pivot row. S3 should be replaced by w in the solution mix.

Table 6

Cj6 5 0 0 0

Basic variable

Solution P w S1 S2 S3 Ratio

6 P 8 1 2/3 1/3 0 0 120 S2 10 0 4/3 -1/3 1 0 15/20 S3 4 0 2/3 -2/3 0 1 6

Zj 48 6 4 2 0 0Cj-Zj - 0 1 -2 0 0

Calculate new values for the pivot row. Divide every number in the row by the pivot number.

R1/3

R2 - R1

R3 -2 R1

Use row operations to make all numbers in the pivot column equal to 0 except for the pivot number.

Table 7

Cj6 5 0 0 0

Basic variable

Solution P w S1 S2 S3

6 P 4 1 0 1 0 -10 S2 2 0 0 1 1 -25 W 6 0 1 -1 0 3/2

Zj 54 6 5 1 0 3/2Cj-Zj - 0 0 -1 0 -3/2

If 6 batches of waffles are made, then the number of pancakes is reduced by 4 of batches pancakes (2/3 pancake/waffle multiplied by 6 batches of waffles); the value changes from 8 batches of pancakes to 4 batches of pancakes. The replacement of 4 batches of pancakes by 6 of batches waffles results in the serving being increased by 8; the value changes from 48 to 54.As the last row contains no positive numbers, this solution gives the maximum value of Z.

The dual simplexI’m now going to explain what’s happening in the dual as we apply the simplex method to the primal. This will help us define another way of solving linear programs, known as the dual simplex. In order to have interesting things to talk about, we’ll assume that the optimal solution is to steal bills only. This gives us the following primal-dual spaces.

We must choose the worst better dual value because we need to keep the dual reduced costs positive (if the dual program is a minimization program), so that when we reach a feasible dual base, we’ll be at the optimum for sure.Each pivot in the dual space should be done as the one we just did. From the dual orange dot, only the dual yellow constraint isn’t valid, so we have to get it into the dual constraint base, which leads us to choose the worst dual value between the yellow and pink dots. The pink dot has a worse dual value, so we pivot towards it.Then, from the dual pink dot, only the blue constraint isn’t satisfied, so we’ll get it into the dual constraint base. We then have to choose between the cyan and the white dots, but, as the white dot has a worse dual value, we cannot go there. So we’ll go to the cyan dot. The cyan dot is feasible, thus it is the dual optimum.Dual Simplex MethodComputationally, dual simplex method is same as simplex method. However, their approaches are different from each other. Simplex method starts with a non-optimal but feasible solution whereas dual simplex method starts with an optimal but infeasible solution.Simplex method maintains the feasibility during successive iterations whereas dual simplex method maintains the optimality. Steps involved in the dual simplex method are:The procedure for forming the dual problem is summarized below:Formation of the dual problemGiven the maximization problem with the problem constraints,Step1. Use the coefficients and constants in the problem constraints and the objective function to form a matrix A with the coefficients of the objective function in the last row.Step2. Interchange the rows and columns of matrix AT, the transpose of A.Step3. Use the rows of AT to form a minimization problem with ≥ problem constraints.

Forming the dual problem

Primal problem in algebraic formMaximise Z= 6p+5wSubject to3p+2w ≤ 24p+2w ≤ 182p+2w ≤ 20And p,w ≥ 0

Primal problem

2 2 241 2 182 2 206 5 1

Dual problem

2 1 2 62 2 2 524 18 20 1

Dual problem in algebraic formMinimiseZ= 24x+18y+20zSubject to3x+y+2z ≥ 62x+2y+2z ≥ 5And x,y,z ≥ 0

Step4. Modified problem, as in step 3, is expressed in the form of a simplex tableau.Table 8

Cj24 18 20 0 0 M M

Basic variable

Solution

X Y Z S1 S2 A1 A2 Ratio

M A1 6 3 1 2 -1 0 1 0 2M A2 5 2 2 2 0 -1 0 1 5/2

Zj 11M 5M 3M 4M -M -M M MCj-Zj - 24-5M 18-3M 20-4M M M 0 0

Step5. Selection of exiting variable: The basic variable with the highest negative value is the exiting variable. If there are two candidates for exiting variable, any one is selected. The row of the selected exiting variable is marked as pivotal row.Step6. Selection of entering variable: The column corresponding to minimum ratio is identified as the pivotal column and associated decision variable is the entering variable.Step7. Pivotal operation: Pivotal operation is exactly same as in the case of simplex method, considering the pivotal element as the element at the intersection of pivotal row and pivotal column.Step8. Check for optimality: If all the basic variables have nonnegative values then the optimum solution is reached. Otherwise, Steps 3 to 5 are repeated until the optimum is reached.

Table 9

Cj24 18 20 0 0 M

Basic variable

Solution X Y Z S1 S2 A2 Ratio

24 X 2 1 1/2 2/3 -1/3 0 0 4M A2 1 0 1 2/3 2/3 -1 1 5/2

Zj 48+M 24 12+M16+2/3

M-8+2/3M -M M

Cj-Zj - 0 6-M 4-2/3M 8+2/3 M M 0

Table 10

Cj24 18 20 0 0

Basic variable

Solution X Y Z S1 S2

24 X 3/2 1 0 1/3 -2/3 1/218 Y 1 0 1 2/3 2/3 -1

Zj 54 24 18 20 -4 -6Cj-Zj - 0 0 0 4 6

Economic interpretation of the dualThe optimal solution to primal problem has already been found and presented in table 7. If we compare table 7 and 10, we observe that objective functions of the two tables assume identical values, i.e., 54. We also note that values in the opportunity cost row under column S1 and S2 of the optimal table of the dual are the same as values under the “solution” column in the optimal table of the primal problem. We further observe that magnitude of variables x and y are exactly the same as the entries (with sign changed) in the opportunity cost row under columns S1 and S3 of the optimal table of the primal problem.AdvantagesKnowledge of dual is useful because solution of a linear programming problem may be easier to obtain through the dual than through the primal problem. For example, consider a primal problem involving three products all of which have to be processed on eight machines. The initial simplex table for this problem will have eight rows. Where the same problem can be solved via its dual, for which the initial table will have only three rows.There’s something else quite interesting about duality: It gives directly a sensitivity analysis. Consider our primal problems. It’d be interesting to know how much more I could get, had my knapsack been larger or my body stronger. This looks like a difficult question in the primal program, but it’s very obvious in the dual program. If I can have one unit of volume more in the knapsack, then the incremental worth of the robbery will be Value Volume.Modern advances in linear programming theory establish the use of the dual simplex algorithm as a powerful optimization tool. While the performance of the dual simplex was originally considered to lag that of its most popular primal variant, the dual simplex is widely used today in practice. One of the more popular applications of the algorithm includes large-scale mixed integer programming, where row additions break primal feasibility, but typically, dual feasibility remains intact.All these interesting properties of the method, coupled with the fact that linear optimization is a fundamental topic in operations research, ensure that advances in this area will continue to be positioned at the forefront of the discipline.