CSE 101, Winter 2018

Design and Analysis of Algorithms

Lecture 13: Linear Programming

Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/

Worked Example: Final Gf

S B

D

T1

11

S

D

E

T

1

1

1

C1

S B

D

T1

11

S

A

B

D

E

T1

8

2

5

1

1

1

2

2

C3

4

1

Final Gf

E

2

2

1

1

#2

#3

#4

S

A D

T2

22

#1

S

A

B

D

E

T3

10

2

6

1

5

1

2

2

C4

Initial Gf

Lecture 12

Ford-Fulkerson Algorithm – Summary• Initialize f = 0• REPEAT:

– Construct the residual graph Gf

– Find a path P from S to T in Gf

– If there is no such path, HALT– cp = minimum cf-capacity edge on path P– Increase f by cp units along path P

• Flow always increases the algorithm terminates• But, if capacities are B-bit integers, can take up to

|E| 2B iterations in worst case

S

A

B

T106

1

106

106

106

The Max-Flow Min-Cut Theorem• Ford-Fulkerson constructively proves that the maximum

flow equals the minimum cut• Define an (L,R) cut in the original network G as follows:

– L = nodes reachable from S in final residual graph Gf

– R = rest of nodes = V – L Note that T cannot be in L T must be in R

• Consider edges between L and R in original network G– Edges e going from L R : must have flow = capacity (by def. of L, R)– Edges e’ going from R L : must have flow = 0 (by def. of L, R)

The flow across this cut = LR edges e c(e) = the capacity of the cut

Since any flow is ≤ any cut, we have max flow = min cut

RL S T

e

e’

Three DP Examples

(Lecture 11 Tablet, Slides 44-46)

Longest Palindromic Subsequence (DPV 6.7)

• Subproblem definition

– Goal

• Recurrence

https://www.geeksforgeeks.org/print-longest-palindromic-subsequence/

Min-Edit “Palindromization”• Given a string, find an efficient algorithm that

computes the minimum number of letters that must be inserted to make the string into a palindrome.

loyal loayaol (2 insertions)aaaabbb bbbaaaabbb (3 insertions)

• Subproblem

– Goal

• Recurrence

Longest Common Substring (DPV 6.8)

• Subproblem definition

– Goal

• Recurrence

Linear Programming (LP)

• Tool for optimal allocation of scarce resources– Optimizations subject to “compatibility constraints”

• Powerful and general problem-solving method– Shortest paths, maximum flows, min-cost flows, MST, matching, 2-

person games, …

• Significance and Practice– Among most important scientific advances of 20th century– Dominates industrial practice

• Delta Airlines: $100M/year benefit from use of LP

– Commercial solvers (CPLEX, COIN, OSL), modeling languages (AMPL)– General tool for attacking intractable (NP-hard) optimization problems

LP Example: Production of Bowls vs. Mugs

Labor Clay RevenuePRODUCT (hr/unit) (lb/unit) ($/unit)Bowl 1 4 40Mug 2 3 50

Constraints: There are 40 hours of labor and 120 pounds of clay available each day

Decision variables: x1 = number of bowls to producex2 = number of mugs to produce

RESOURCE COSTS REVENUE

• Note 1: x1, x2 must be non-negative• Note 2: x1, x2 can be fractional (non-integer)

LP Example: Production of Bowls vs. Mugs

Labor Clay RevenuePRODUCT (hr/unit) (lb/unit) ($/unit)Bowl 1 4 40Mug 2 3 50

Constraints: There are 40 hours of labor and 120 pounds of clay available each day

Decision variables: x1 = number of bowls to producex2 = number of mugs to produce

RESOURCE COSTS REVENUE

Maximize Z = $40 x1 + 50 x2

Subject to x1 + 2x2 40 hr (labor constraint)4x1 + 3x2 120 lb(clay constraint)

x1 , x2 0

Example solution: x1 = 24 bowls x2 = 8 mugsRevenue = $1,360

Geometric Interpretation

4 x1 + 3 x2 120 lb

x1 + 2 x2 40 hr

Feasible Region= Area common toboth constraints

50 –

40 –

30 –

20 –

10 –

0 – |10

|60

|50

|20

|30

|40 x1

x2

x1 = #bowlsx2 = #mugs

Feasible Region Has Extreme Points

x1 = 24 bowlsx2 =8 mugsZ = $1,360 x1 = 30 bowls

x2 =0 mugsZ = $1,200

x1 = 0 bowlsx2 =20 mugsZ = $1,000

A

BC|

20|

30|

40|

10 x1

x2

40 –

30 –

20 –

10 –

0 –

• The feasible region is an intersection of half-spacesthat arise from the constraints

– Vertices of the feasible region = where 2 constraints are tight– Vertices are like “corners”

Extreme Point = Solution of Simultaneous Equations

x1 + 2x2 = 404x1 + 3x2 = 1204x1 + 8x2 = 160

-4x1 - 3x2 = -1205x2 = 40x2 = 8

x1 + 2(8) = 40x1 = 24

4 x1 + 3 x2 120 lb

x1 + 2 x2 40 hr

40 –

30 –

20 –

10 –

0 – |10

|20

|30

|40

x1

x2

Z = $50(24) + $50(8) = $1,360

248

Geometry: Convex, Concave Annotation added (see also Slide 18)

The closed line segment with endpoints a and b consists of all points a + (1 - ) b, 0 1. These points are convex combinations of a and b.

Geometry: Convex, Concave, Extreme• Inequalities induce halfspaces with respect to hyperplanes• Bounded feasible region: convex polygon or polytope• Feasible region = convex set: If a, b feasible, so is (a+b)/2• Extreme point: Feasible solution x that cannot be written as

(a+b)/2 for two distinct feasible solutions a and b

Functions: Convex, Concave, and Linear

• Function f is convex if f[x1 + (1-)x2] ≤ f(x1) + (1-)f(x2)– f defined over a convex domain– If f is convex, then a local minimum is a global minimum

• Function f is concave if –f is convex– If f is concave, then a local minimum can occur only at an extreme

point of the domain of f

• KEY OBSERVATION: A linear function is both convex and concave (!!!)– Local minima occur only at extreme points (from concavity)– Any local minimum is a global minimum (from convexity)– To find a global minimum, only need to look at extreme points

0 1

Geometry: Convex Sets, Convex Functions• Inequalities induce halfspaces with respect to hyperplanes• Bounded feasible region: convex polygon or polytope• Feasible region = convex set: If a, b feasible, so is (a+b)/2• Extreme point: Feasible solution x that cannot be written as

(a+b)/2 for two distinct feasible solutions a and b

• Function f is convex if f[x1 + (1-)x2] ≤ f(x1) + (1-)f(x2)– f defined over a convex domain– If f is convex, then a local minimum is a global minimum

0 1

4x1 + 3x2 120 lb

x1 + 2x2 40 hr

40 –

30 –

20 –

10 –

0 –

B

|10

|20

|30

|40 x1

x2

C

A

New Z = 70x1 + 20x2New Optimum:x1 = 30 bowlsx2 =0 mugsZ = $2,100

Geometric Interpretation of Objective Function

• A global optimum has maximum projection on (or, maximum inner product with) the objective function

“Level set” of objective function value

Minimization Formulation

CHEMICAL CONTRIBUTIONBrand Nitrogen (lb/bag) Phosphate (lb/bag)

Gro-plus 2 4Crop-fast 4 3

Minimize Z = $6x1 + $3x2

subject to2x1 + 4x2 16 lb of nitrogen4x1 + 3x2 24 lb of phosphate

x1, x2 0

14 –

12 –

10 –

8 –

6 –

4 –

2 –

0 – |2

|4

|6

|8

|10

|12

|14 x1

x2

C

Geometric Interpretation

x1 = 0 bags of Gro-plusx2 = 8 bags of Crop-fastZ = $24

Z = 6x1 + 3x2

• Possible scenarios: (1) Optimal solution can be a vertex of the feasible region, or (2) optimal solution can be unbounded, or (3) there is no feasible region at all

A

B

Simplex Method• Invented by George Dantzig in 1947

– Developed shortly after WWII in response to logistics challenges– Used for 1948 Berlin airlift

• Objective function is linear, and feasible region is convex can use a greedy strategy (called “hill-climbing” in textbook)

– Start at an extreme point– “Pivot” from one extreme point to a neighboring one

• NEVER WORSEN THE OBJECTIVE FUNCTION IN DOING SO !!!

– Stop when the current extreme point is a local optimum

• Which (improving) neighbor?• How to avoid degeneracy / stalling (= new set of

“tight constraints” but same extreme point)• Cycling • Known implementations can take exponential in

worst-case, but in practice see linear #pivots

Network Flow Can Be Solved as a LP

• Optimization variables: fSA, fBA, etc.• Capacity constraints: 0 ≤ fSA ≤ 3, 0 ≤ fET ≤ 2, etc.• Conservation constraints: fSA + fBA = fAD, etc.• Non-negative flow constraints: fIJ 0 for all I,J

• LP: maximize F = fSA + fSB subject to constraints• Observation: “Max Flow reduces to LP”• Observation: “Matching reduces to Max Flow” “Matching reduces to LP”

S

A

B

D

ET

3

1

1

1

1

1

2

2

5

What is the maximum flow?

Matching Reduces to Max FlowWhat is the size of the maximum matching between Workers and Jobs?W1

W2

W4

W3

J1

J2

J4

J3

“boils down to”,“can be solved using”Slide 3,

Lec 12 Tablet

Matching Reduces to Max Flow

S T

What is the size of the maximum flow from S to T? (All edge capacities = 1; red-colored edges used by max flow.)

W1

W2

W4

W3

J1

J2

J4

J3

What is the size of the maximum matching between Workers and Jobs?

System of Distinct Representatives

LP Example: Fitting a Line

• General form of a line: ax + by = c• Find a, b, c such that maxi |axi + byi – c| is minimized

How do we write this down as an LP?

Problem: Given points (x1,y1), …, (xm,ym), find the best-fit line.

“Linear Regression Modeling”

LP Example: Fitting a Line

• General form of a line: ax + by = c• Find a, b, c such that maxi |axi + byi – c| is minimized• Trick: Introduce a new variable e = max deviation from line• LP: minimize e

s.t. axi + byi – c ≤ e for all i = 1, …, maxi + byi – c -e for all i = 1, …, m

• E.g., with m = 7 points, we get an LP with 14 constraints

Problem: Given points (x1,y1), …, (xm,ym), find the best-fit line.

“Linear Regression Modeling”

LP: “Dummy Fill” in Chip Design• Manufacturing of metal wires in a chip requires planarization of each layer

before the next layer is fabricated• Varying pattern density of wires causes unevenness in planarization result• Chip layouts have “density uniformity rules”, e.g., “In any 200um x 200um

window of the layout of each wiring layer, the metal density must be between 40% and 60%.”

• Designers satisfy these rules by adding “dummy fill” into the chip layout

0 Amount of Dummy Fill “Slack”

= unused slack= dummy fill= orig layout

= fill “slack”

= fill “slack”

LP: “Dummy Fill” in Chip Design• Manufacturing of metal wires in a chip requires planarization of each layer

before the next layer is fabricated• Varying pattern density of wires causes unevenness in planarization result• Chip layouts have “density uniformity rules”, e.g., “In any 200um x 200um

window of the layout of each wiring layer, the metal density must be between 40% and 60%.”

• Designers satisfy these rules by adding “dummy fill” into the chip layout

• Problem: Given a layout of wires and a set of “windows” to verify, insert the dummy fill to minimize the maximum difference of window densities. w/r

Overlapping windows

w

n

tile

+ = fill slack= fill= orig layout

LP for Dummy Fill

• Same trick of introducing a new variable, M• M is lower-bounded by the difference of densities between any

two windows pushing down M means more uniform density• Minimize M s.t.

M |dens(Wi) – dens(Wj)| for all windows i,j// force minimum variation of post-fill window densities

dens(W) = density(orig layout + fill) in all tiles of W// density of a window = original + added fill

0 ≤ fillij ≤ slackij for all tiles Tij

// cannot put more fill than will fit (i.e., more than “slack”)// or negative amount of fill into any window

w/r

Overlapping windows

w

n

tile