Linear Programming

What’s Linear Programming?

Often your try is to maximize or minimize an objective within given constraints

A linear programming problem can be expressed as a linear function of certain variables and the constraints can be expressed as equalities or inequalities on those variables

Linear programming could be a minimization linear program or a maximization linear program

2

Election Spending Problem

You’re running for office

100K urban voters, 200K suburban, and 50K rural voters are your target

You want to win over at least 50% of voters in each area

There’s a catch. These voters may be persuaded differently by advertisements on different issues

3

Election Spending ProblemThe effect of policies (and hence advertising on those policies) on thousands of voters who can be won over (or lost) by spending $1,000 on particular issue.

4

Policy urban suburban rural

building roads -2 5 3

gun control 8 2 -5

farm subsidies 0 0 10

gasoline tax 10 0 -2

Evolving the Linear ProgramRather than trial and error, you can formulate the problem

Let x1 be the number of thousands of dollars spent on advertising on building roads, x2 ... on gun control, x3 ... on farm subsidies, and x4 ... on gasoline tax (for public transit improvements)

You can formulate the requirement to win half urban voters (50k out of 100k voters) as

-2x1 + 8x2 + 0x3 + 10x4 >= 50

...(similarly for other areas)

You have to ensure xi >= 0 (as spending can’t be negative) and minimize x1 + x2 + x3 + x4 5

Linear Program for Election $Minimize x1 + x2 + x3 + x4

subject to

-2x1 + 8x2 + 0x3 + 10x4 >= 50

5x1 + 2x2 + 0x3 + 0x4 >= 100

3x1 - 5x2 + 10x3 - 2x4 >= 25

x1, x2, x3, x4 >= 0

Solution to the linear program yields the optimal strategy.

6

General Linear ProgramsOptimize a linear function subject to a set of linear inequalities

Given a set of real numbers a1, a2, ..., an and a set of variables x1, x2, ..., xn, we define a linear function f on those variables:

f(x1, x2, ..., xn) = a1x1 + a2x2 + ... + anxn = !1<=i<=n aixi

plus linear constraints: either a linear equality f(x1, x2, ..., xn) = b or linear inequalities f(x1, x2, ..., xn) <= b or f(x1, x2, ..., xn) >= b.

7

Canonical Forms of LP

Standard

Maximization of a linear function subject to linear inequalities

Slack

Maximization of a linear function subject to linear equalities

8

Feasible SolutionA linear program has n variables subject to a set of m linear inequalities

A feasible solution is any values for variables that satisfy all the constraints

Feasible region is set of all feasible solutions

The maximization function is called the objective function and the value of this function at particular point in the feasible region an objective value

There may be infinite number of points in a feasible region, so we need a way to practically find the optimal solution for the objective value

9

Solving Two VariablesYou can plot the constraints on a cartesian coordinate and find an optimal solution for the objective value

10

f1f2

f3

Objective function

OptimalSolution

The optimal solution is the vertex of the feasible region

If it is a line, you can use end points of the line segment

Solving Multiple VariablesEach constrain forms a half-space in n-th dimensional space

Feasible region formed by the intersection of these half-spaces is called simplex

The objective function is a hyperplane and the optimal solution occurs at the vertex of the simplex

The simplex algorithm moves through the exterior of the simplex, determining the vertex with maximum value that satisfy the objective function

11

The Simplex AlgorithmIterative algorithm

Set each nonbasic variable to 0 and compute the values of the basic variables from the equality constraints

In each iteration you convert from one slack form to another slack form

You improve the objective value in each iteration

We increase values (subject to constraints) of nonbasic variables from 0 in order to maximize the objective value

You convert the equations into equivalent forms until the objective function has all variables with negative coefficient

12

Example of Simplex AlgGiven the linear program in standard form

maximize 3x1 + x2 + 2x3

subject to

x1 + x2 + 3x3 <= 30

2x1 + 2x2 + 5x3 <= 24

4x1 + x2 + 2x3 <= 36

x1, x2, x3 >= 013

Convert to Slack Form

z = 3x1 + x2 + 2x3

x4 = 30 - x1 - x2 - 3x3

x5 = 24 - 2x1 - 2x2 - 5x3

x6 = 36 - 4x1 - x2 - 2x3

Nonbasic variables on the right, basic variables on the left

14

Giving Valuesz = 3x1 + x2 + 2x3

x4 = 30 - x1 - x2 - 3x3

x5 = 24 - 2x1 - 2x2 - 5x3

x6 = 36 - 4x1 - x2 - 2x3

Giving values of 0 to nonbasic variables, we have (0, 0, 0, 30, 24, 36) and z gets a value of 0.

Now we want to reformulate so the basic solution has a greater objective value.

15

Reformulatingz = 3x1 + x2 + 2x3

x4 = 30 - x1 - x2 - 3x3

x5 = 24 - 2x1 - 2x2 - 5x3

x6 = 36 - 4x1 - x2 - 2x3

Take a nonbasic variable whose coefficient in the objective function is positive and increase the value as much as possible without violating constraints. This variable becomes basic and some other variable becomes nonbasic.

16

Reformulatingz = 3x1 + x2 + 2x3

x4 = 30 - x1 - x2 - 3x3

x5 = 24 - 2x1 - 2x2 - 5x3

x6 = 36 - 4x1 - x2 - 2x3

Increasing x1 will result in decreasing x4, x5, and x6.

However, non of them can become negative due to constraints. x4, x5, and x6, restrict x1 to 30, 12, and 9, respectively.

The third constraint is the most limiting. We switch roles of x1 and x6. 17

Pivoting

z = 3x1 + x2 + 2x3

x4 = 30 - x1 - x2 - 3x3

x5 = 24 - 2x1 - 2x2 - 5x3

x1 = 9 - x2/4 - x3/2 - x6/4

You pivot—that is you chose an entering variable (x1) and a leaving variable (x6) and exchange their roles.

You now replace all x1

18

Pivotingz = 27 + x2/4 + x3/2 - 3x6/4

x1 = 9 - x2/4 - x3/2 - x6/4

x4 = 21 - 3x2/4 - 5x3/2 + x6/4

x5 = 6 - 3x2/2 - 4x3 + x6/2

You’re ready to pick the next pivot, an entering variable with positive coefficient in the objective function and a corresponding leaving variable.

Let’s chose x3 and that will lead to the tightest function and hence the leaving variable x5.

19

Pivoting

z = 111/4 + x2/16 - x5/8 - 11x6/16

x1 = 33/4 - x2/16 + x5/8 - 5x6/16

x3 = 3/2 - 3x2/8 - x5/4 + x6/8

x4 = 69/4 + 3x2/16 + 5x5/8 - x6/16

The lone standing positive coefficient nonbasic variable in objective function is x2

Using that as a pivot,...

20

Pivoting

z = 28 - x3/6 - x5/6 - 2x6/3

x1 = 8 + x3/6 + x5/6 - x6/3

x2 = 4 - 8x3/3 - 2x5/3 + x6/3

x4 = 18 - x3/2 + x5/2

Giving zero values to the nonbasic variables, we get x1 = 8, x2 = 4, x3 = 0, and the optimal solution z = 28.

21

Slack Variablesmaximize 3x1 + x2 + 2x3

x1 + x2 + 3x3 <= 30 12 <= 30, slack is 18 which is x4

2x1 + 2x2 + 5x3 <= 24 24 <= 24, slack is 0 which is x5

4x1 + x2 + 2x3 <= 36 36 <= 36, slack is 0 which is x6

x1, x2, x3 >= 0

Solution x1 = 8, x2 = 4, x3 = 0, and the optimal solution z = 28.

x4, x5, and x6 are the slack variable. The slacks are 18, 0, and 0. If you plug in values for x1, x2, and x3 into original inequalities, you can see the slack.

22

Pivot

23

Simplex Algorigthm

24

Issues to considerIt is possible that the feasible region is unbounded

It is possible that the iteration leaves the objective value unchanged—known as degeneracy

You need to look for cycles where the slack forms simply cycle through transformations

Since Simplex algorithm is deterministic, it will cycle over the same sequence

You can determine it cycles if it does not terminate in at most ( ) iterations

25

n+mm