COSC 6114

Prof. Andy Mirzaian

General Overview Introduction Fundamentals Duality Major Algorithms Open Problems

2D Linear Programming O(n log n) time by computation of feasible region O(n) time Randomized Incremental Method O(n) time Deterministic Prune-&-Search Method Smallest Enclosing Disk

TOPICS

References:• [M. de Berge et al] chapter 4• [CLRS] chapter 29• [Edelsbrunner ‘87] chapter 10• Lecture Notes 4, 7-13• Reference books on “Optimization” listed on the course web-site

General Overview

The LP Problem

subject to:

dd xcxcxc 2211

ndndnn

dd

dd

bxaxaxa

bxaxaxa

bxaxaxa

2211

22222121

11212111

maximize

Applications

• The most widely used Mathematical Optimization Model.

• Management science (Operations Research).

• Engineering, technology, industry, commerce, economics.

• Efficient resource allocation:

– Airline transportation,

– Communication network – opt. transmission routing,

– Factory inventory/production control,

– Fund management, stock portfolio optimization.

• Approximation of hard optimization problems.

• . . .

FeasibleRegion

Example in 2D

optimum basic

constraints

max x1 + 8x2

subject to:

(1) x1 3 (2) x2 2(3) –3x1 + 4x2 14(4) 4x1 – 3x2 25(5) x1 + x2 15

x2

x1

(1)

(2)

(4)

(5)(3)

(5)(3)

optimum x1 + 8x

2

x1 = 46/7x2 = 59/7

= 74

Example in 3D

subject to: zmaximize zOptimum

(x,y,z)=(0,0,3)

x

y

z

0

0

0

2

3

z

y

x

y

zyx

History of LP 3000-200 BC: Egypt, Babylon, India, China, Greece: [geometry & algebra]

Egypt: polyhedra & pyramids. India: Sulabha suutrah (Easy Solution Procedures) [2 equations, 2 unknowns] China: Jiuzhang suanshu (9 Chapters on the Mathematical Art) [Precursor of Gauss-Jordan elimination method on linear equations] Greece: Pythagoras, Euclid, Archimedes, …

825 AD: Persia: Muhammad ibn-Musa Alkhawrazmi (author of 2 influential books):“Al-Maqhaleh fi Hisab al-jabr w’almoqhabeleh” (An essay on Algebra and equations) “Kitab al-Jam’a wal-Tafreeq bil Hisab al-Hindi” (Book on Hindu Arithmetic).

originated the words algebra & algorithm for solution procedures of algebraic systems.

Fourier [1826], Motzkin [1933] [Fourier-Motzkin elimination method on linear inequalities]

Minkowski [1896], Farkas [1902], De la Vallée Poussin [1910], von Neumann [1930’s], Kantorovich [1939], Gale [1960] [LP duality theory & precursor of Simplex]

George Dantzig [1947]: Simplex algorithm. Exponential time in the worst case, but effective in practice.

Leonid Khachiyan [1979]: Ellipsoid algorithm.

The first weakly polynomial-time LP algorithm: poly(n,d,L).

Narendra Karmarkar [1984]: Interior Point Method. Also weakly polynomial-time. IPM variations are very well studied.

Megiddo-Dyer [1984]: Prune-&-Search method. O(n) time if the dimension is a fixed constant. Super-exponential on dimension.

LP in matrix form:

subject to: Ax b

maximize cT x ),,,( 21 dT cccc

nidjijaA ..1

..1)(

),,,( 21 nT bbbb

Canonical Form: ),,,( 21 dT xxxx

We assume all vectors are column vectors. Transpose (T) if you need a row vector.

LP feasible regiond

idid22i11i in plane-hyper bxaxaxa

F = { xd | Ax b }

didid22i11i in space-half bxaxaxa

Primal Dual LPs:Primal:maximize cT xsubject to: Ax b

(canonical form)

Dual: minimize yT bsubject to: yT A = cT

y 0 (standard form)

A

xT

y b

cT

n

d

Primal-Dual LP Correspondence

PRIMAL DUAL

max cTx min yTb

Constraint: aiT x bi

ajT x bj

akT x = bk

(aiT = i-th row of constraint matrix A)

Variable: yi 0 yj 0 yk unrestricted

Variable: xi 0 xj 0 xk unrestricted

Constraint: yTAi ci yTAj cj yTAk = ck

(Ai = i-th column of constraint matrix A)

FACT: Dual of the Dual is the Primal.

Example 1: Primal-Dual

PRIMAL:max 16 x1 - 23 x2 + 43 x3 + 82 x4

subject to: 3 x1 + 6 x2 - 9 x3 + 4 x4 239 -9 x1 + 8 x2 + 17 x3 - 14 x4 = 582 5 x1 + 12 x2 + 21 x3 + 26 x4 -364

x10, x20, x40

DUAL:min 239 y1 + 582 y2 - 364 y3

subject to: 3 y1 - 9 y2 + 5 y3 16 6 y1 + 8 y2 + 12 y3 -23 -9 y1 + 17 y2 + 21 y3 = 43 4 y1 - 14 y2 + 26 y3 82

y10, y30

Example 2: The Diet Problem

This is one of the earliest Primal-Dual LP formulations. Dantzig attributes it to:[G. J. Stigler, “The cost of subsistence,” J. Farm Econ. 27, no. 2, May 1945, 303-314].

Primal: A homemaker has n choices of food to buy, and each food has some of each of m nutrients:

Given: aij = amount of the ith nutrient in a unit of the jth food, i=1..m, j=1..n,

ri = yearly minimum requirement of the ith nutrient, i=1..m,

cj = cost per unit of the jth food, j=1..n.

Determine: xj = yearly consumption of the jth food, j=1..n.

Objective: Minimize total food purchasing cost cTx for the homemaker.

Constraints: Meet yearly nutritional requirement (Ax r) with nonnegative amount of each food (x 0).

Primal: min cTx subject to: Ax r , x 0.

Example 2: The Diet ProblemPrimal: min cTx subject to: Ax r , x 0.

Dual: max rTy subject to: ATy c , y 0.

Dual interpretation:

A pill-maker wishes to market pills containing each of the m nutrients, yi = the price per unit of nutrient i, i=1..m.He wishes to be competitive with the price of real food, andmaximize his profit from the sale of pills for an adequate diet.

The dual constraints

m

1ijiij 1..njcya

express the fact that the cost in pill form of all the nutrients in the j th food

is no greater than the cost of the jth food itself.

The objective function rTy is simply the pill maker’s profit for an adequate diet.

Example 3: Network Max Flow

Flow Network: N=(V,E,s,t,c) V = vertex setE = directed edge sets = source vertext = terminal vertexc: E + edge capacities

find max flow f: E + satisfying flow conservation & capacity constraints

see alsoAAW

maximize v Maximize Net flow out of sourcesubject to:

v for x=s

Sy fxy - Sy fyx = -v for x=t Flow Conservation Law

0 for xV - {s,t}

fe ce for e E Flow Capacity Constraintsfe 0 for e E Flow Non-negativity Constraints

Assume flow is 0 on non-edges.

Example 3: Network Max Flow

e

i

j

i

j

e

+1

-1

0

0

0

fe

ce

A =

A: vertex-edge incidence matrix Rows indexed by vertices V, columns by edges E. For edge e=(i,j)E : Aie = +1, Aje = -1, Ake = 0 kV-{i,j}.

Example 3: Network Max Flow

e

i

j

fe

ce

s)constraint(capacity

n)oconcervati (flow

0

0

:s.t.

maximize

:Flow)t -s(Max :Primal

cf

other rows

row t v

row s v

Af

v

e

i

j

e

ce0σ

1ππ

0σππ :s.t.

σc minimize

:Cut)t -s(Min : Dual

ts

ijji

Eeee

E)j,i(

LP: Fundamental Facts

LP optimal solution set Feasible region F = { xd | Ax b } is the intersection of n half-spaces. F is a convex polyhedron (possibly empty or unbounded). Convexity: local optimality global optimality.

A supporting hyper-plane is one that does not intersect the interior of F. A face of F is the intersection of F with some supporting hyper-plane.

We consider F itself as a (full dimensional) face of F. is also a face of F. Vertex 0 dimensional face. Edge (bounded) 1 dimensional face.

optimum objective value z* = max { cTx : xF} (may not exist or be unbounded)

supporting hyper-plane cTx = z* optimum face F* = { x : cTx = z* , xF} F* interior(F) =

F is pointed if it doesn’t contain any line. Hence, F= or has a vertex (i.e., rank(A)=d).

F is pointed every face of F is pointed. F* & pointed F* contains a vertex of F (an optimal vertex).

[In that case, we can focus on searching for an optimal vertex.]

For further details see the upcoming slides.

Separation LemmaThe Hahn-Banach Separation Theorem:

Consider any non-empty closed convex set P and a point b in n .bP there is a hyper-plane that separates b from P, i.e.,

hyper-plane yTx = for some yn and , such that yTb < and yTz zP.

yT x =

P

b

Proof: Consider the closest point pP to point b. Take the hyper-plane through p and orthogonal to segment bp.

p

Extreme PointsFACT: A non-empty closed convex set P has an extreme point

if and only if P does not contain any line.

Proof (): Suppose P contains a line L.We must prove that no point xP is an extreme point of P.

Let L’ be the line parallel to L that passes through x.

CLAIM: P contains L’. This claim implies x is not an extreme point of P. (Why?)

Refutation of the Claim leads to a contradiction as follows:Suppose there is a point b L’ – P. By the Separation Lemma, there is a hyperplane H that separates b from P.H intersects L’ (at some point between x and b).H must also intersect line L (which is parallel to L’).That would contradict the fact that H separates b from P L.This contradiction proves the Claim.

L

x L’b

H

Extreme PointsFACT: A non-empty closed convex set P has an extreme point

if and only if P does not contain any line.

Proof (): Suppose P does not contain any line.We apply induction on the dimension.If P is a single point, then it is an extreme point.

Now suppose dim(P) > 0.Since P does not contain any line,

P is a proper subset of aff(P) (affine hull of P). So, there is a point b aff(P) – P. By the Separation Lemma,

there is a hyperplane H separating b from P in aff(P).

Move H parallelwise toward P until H is tangent to P.Let f = HP. dim(f) dim(H) < dim(P).

f is a non-empty closed convex set that contains no line.So, by the induction hypothesis, f must have an extreme point x.x is an extreme point of P also. (Why?)

H

P

f

Extreme PointsThe Krein-Milman Theorem:A closed bounded convex set is equal to the convex hull of its extreme points.

Proof: By convexity & closedness, the given set P contains the convex hull of its extreme points.

The converse, i.e., every point xP can be represented as a convex combination of extreme points of P, is proved by induction on the dimension.

If x is an extreme point of P, we are done. Otherwise, draw a line L through x.Let yz be the maximal segment of L in P.x is a convex combination of the boundary points y and z (as shown).By induction, y & z can be represented as convex combinations of extreme points of the lower dimensional closed bounded convex sets PH1 and PH2, respectively.Hence, x is a convex combination of extreme points of P (details for exercise).

H1

H2

yx z

P

L

Farkas LemmaFarkas Lemma:Let And, bn. Then exactly one of the following must hold:

(1) xd : Ax b.(2) yn : yTA = 0, yTb < 0, y0.

Proof: [(1) & (2) cannot both hold] Else: 0 = 0x = (yTA)x = yT(Ax) yTb < 0.

[ (1) (2)]: P = { zn | z=Ax+s, xd , sn , s0 } is convex & non-empty(0P).

(1) Þ bP y,: yTb < yTz zP by Separation LemmaÞ y,: yTb < yT(Ax+s) s 0,xÞ y,: yTb < (yTA)x+ yTs s 0,xÞ y,: yTb < yTs s 0, & yTA = 0 if yTA0, take x=(-1- yTs) ATy ||yTA||-2

Þ y,: yTb < yTs s 0, & yTA = 0, & y 0 if yi <0, take si= - (1+||)/yi & sj=0 for jiÞ y,: yTb < 0 yTs s 0, & yTA = 0, & y 0 if >0, take s = (/2)y ||y||-2

Þ y: yTb < 0, yTA = 0, y 0Þ (2)

Farkas LemmaFarkas Lemma (alternative versions): Let And, bn.

1) xd : Ax b yn : yTA = 0, yTb < 0, y0.

2) xd : Ax b, x 0 yn : yTA 0, yTb < 0, y0.

3) xd : Ax = b yn : yTA = 0, yTb < 0.

4) xd : Ax = b, x 0 yn : yTA 0, yTb < 0.

Convex Cones, Sets & Polyhedra

cone(R) = {m1r1+m2r2+ … +mkrk | mi0 } cone(R)={0} if R=. Extreme rays of the cone are shown bold.

0

Minkowski sum (of vector sets): P+Q = {x+y | xP, yQ}

CH(V) = {1v1+2v2+ … +mvm | Si i=1,i0 }

Recession Coneof the polyhedron

Convex Cone: C ( d), 0 C, C + C C, and 0: C C.

Finitely Generated Convex Cones and Convex Sets: Let V = {v1, … ,vm} and R = {r1, … ,rk}

+ =

CH(V) + cone(R):

polyhedron

Caratheodory’s Theorem

Let X be a non-empty set in d.

(a) xcone(X) & x0 if and only if x= m1x1+m2x2+ … +mkxk for some m1, … ,mk > 0

and for some linearly independent x1, … ,xk X.(Hence, k d.)

(b) xCH(X) if and only if x= m1x1+m2x2+ … +mkxk for some m1, … ,mk > 0, Si mi

=1. and for some affinely independent x1, … ,xk X.(I.e., x2-x1, … ,xk-x1 are linearly independent. Hence, k d+1.)

0

x

cone(X)

(a)

CH(X)x

(b)

X

x1

x2

x1

x2

x3

Caratheodory’s Theorem - Proof

(a) Consider a minimal k such thatx = m1x1+m2x2+ … +mkxk , mi >0, xiX, i=1..k.

(xi) are linearly dependent r1, … ,rk : at least one ri >0 & 0 = r1x1+r2x2+ … +rkxk.

So, x = (m1 – a r1) x1+ (m2 – a r2) x2+ … + (mk – a rk) xk .

Take a = min { mi / ri | ri >0}.

All coefficients remain non-negative and at least one becomes 0.

This contradicts minimality of k.

CH(X)

cone(Y)

Y

x

(b) Apply proof of (a) in d+1 to cone(Y) where Y = CH(X){1} = { (x,1) d+1 | xCH(X) }.

Polyhedra - Theorem

Fundamental Theorem of Polyhedra [Minkowski-Weyl]:

A subset P d is the intersection of a finite set of half-spaces:

P = { xd | Ax b } for some A nd, b n

if and only if

P is the sum of convex hull of a finite set of points plus conical combination of a finite set of vectors:

P = CH(V) + cone(R) for some V dm , R dm’ = { x d | x = Vl + Rm, m0, l0, Si li = 1 }.

P is a polytope P is bounded R=, V = vertex set of P.

Corollary: A polytope can be viewed in 2 equivalent ways: 1) as intersection of half spaces (defined by its facets), 2) as convex hull of its vertices (cf. Krein-Milman Theorem).

If P is pointed, columns of V are vertices of P, columns of R are extreme rays of P.

Polyhedra – Example

2x3x,1x,0xx|xP 211212

V = (v1 , v2)

R = ( r1 , r2)

1λ,0λ,0μ,

μμ

1103

λλ

1111

xx

xP iii2

1

2

1

2

12

v2

v1

r2

r1

P

x1

x2

RV

Polyhedra – Proof

Proof of (): [Assume polyhedron P is pointed.] Let Q = { x d | x = Vl + Rm, m0, l0, Si li = 1 } where columns vi of V are

extreme points of P, and columns ri of R are extreme rays of P. We claim Q = P:Q P: xQ implies x =y + z for some y=V , l l0, Si li = 1, and z= Rm, m0.

So yP, since y is a convex combination of extreme points of P. Also, z is a conic combination of extreme rays of P, so y+z P. So xP.

Q P: WLOG assume P. If xQ, then ( ,l m) mm’ such that v1 l1 + v2 l2 + …+ vm lm + r1 m1 + r2 m2 +… + rm’ mm’ = x l1 + l2 + …+ lm = 1 l 0, m 0.

So, by Farkas Lemma (p,po)d

such that i: pTvi + po 0, pTri 0, pTx + po > 0.Now consider the LP max {pTy | yP}.For all rays ri of P, pTri 0. Hence, the LP has a bounded optimum.So, the optimum must be achieved at an extreme point of P.But for every extreme point vi of P we have pTx > -po pTvi. So, pTx > -po max {pTy | yP}. (The hyperplane pTy = -po separates x from P.)Therefore, xP. [What happens if P is not pointed? See exercise 7.]

Polyhedra – Proof

Proof of ():

Q is the projection onto x of the polyhedron

{ (x, ,l m) dmm’ | x - Vl – R = 0m , m0, l0, Si li = 1 }.

Hence, Q is also a polyhedron (i.e, intersection of a finite number of half-spaces).

Bases & Basic Solutions

Bases & Basic Solutions:

Basis B: Any set of d linearly independent rows of A.

B dd and det(B) 0. N = remaining non-basic rows of A.

Basic Solution: BxB = bB xB = B-1 bB .

Basic Feasible Solution (BFS): If xB is feasible, i.e., NxB bN .

Each BFS corresponds to a vertex of the feasible polyhedron.

xTB

bB

bN

B

N

A b

Fundamental Theorem of LPFor any instance of LP exactly one of the following three possibilities holds:

(a) Infeasible.(b) Feasible but no bounded optimum.(c) Bounded optimum.

[Note: Feasible polyhedron could be unbounded even if optimum is bounded. It depends on the direction of the objective vector.]

Moreover, if A has full rank (i.e., basis), then every nonempty face of the feasible polyhedron contains a BFS, and this implies:

(1) feasible solution BFS.(2) optimum solution optimum solution that is a BFS.

c

Proof: If there is a basis, the basic cone contains the feasible region but does not contain any line. So the feasible region doesnot contain any line, hence it is pointed. So everynon-empty face of it (including the optimal face, if non-empty) is pointed, and thus contains a vertex.(For details see exercise 4.)

LP Duality TheoremPrimal: max { cTx | Ax b }Dual: min { yTb | yTA = cT, y 0 }

Weak Duality: x Primal feasible & y Dual feasible cTx yTb.

Proof: cTx = (yTA)x = yT(Ax) yTb.

Corollary:(1) Primal unbounded optimum Dual infeasible(2) Dual unbounded optimum Primal infeasible(3) Primal & Dual both feasible both have bounded optima.

[Note: Primal & Dual may both be infeasible. See Exercise 5.]

Strong Duality: If x & y are Primal/Dual feasible solutions, then

x & y are Primal/Dual optima cTx = yTb.

Proof: By Farkas Lemma. See Exercise 8.

Example 1

P: max -18 x1 + 27 x2 subject to: -4 x1 + 5 x2 8 x1 + x2 7

x10, x2 0

D: min 8 y1 + 7 y2

subject to: -4 y1 + y2 -18 5 y1 + y2 27

y10, y2 0

Optimum Solution for P: x1 = 3, x2 = 4

opt objective value =

-18*3 + 27*4 = 54

Optimum Solution for D: y1 = 5, y2 = 2

opt objective value =

8*5 + 7*2 = 54

Example 2: The Diet Problem

Primal: min cTx subject to: Ax r , x 0. [Homemaker]

Dual: max rTy subject to: ATy c , y 0. [Pill Maker]

Week Duality:

For every primal feasible x and dual feasible y: cTx rTy.

Interpretation: the pill maker cannot beat the real food marketfor an adequate diet.

Strong Duality:

min {cTx | Ax r , x 0} = max { rTy | ATy c , y 0}.

Interpretation: homemaker’s minimum cost food budget is equal to pill maker’s maximum profit for an adequate diet.

LP Optimization Feasibility

Primal:maximize cT xsubject to: Ax b

Dual: minimize yT bsubject to: yT A = cT

y 0

Is there (x,y) that satisfies:

bT y - cT x 0

Ax bAT y = c y 0

Complementary Slackness

Primal:maximize cT xsubject to: aT

i x bi , i=1..n

xj 0 , j=1..d

Dual: minimize yT b

subject to: ATj y cj , j=1..d

yi 0 , i=1..n

Complementary Slackness Condition (CSC): yi (bi - aT

i x ) = 0 , i=1..n

xj (ATj y - cj ) = 0 , j=1..d

FACT: Suppose x & y are Primal-Dual feasible. Then the following statements are equivalent:

(a) x and y are Primal-Dual optimal,(b) cTx = yTb,(c) x & y satisfy the CSC.

Example 3: Max-Flow Min-Cut

See AAW for animation:

cf

v

v

Af

v

0

rowsother 0

t row

s row

:s.t.

maximize

:Flow)t -s(Max :Primal

0σ

1ππ

0σππ :s.t.

σc minimize

:Cut)t -s(Min : Dual

ts

ijji

Eeee

E)j,i(

S

(v net out-flow)

t

f = c , =1

f = 0 , = 0

= 0

= 0 = 0CC

= 1

Min-Max Saddle-Point Property

FACT: For all functions f : XY :

max min f (x,y) min max f (x,y) . xX yY yY xX

Proof:

Suppose LHS = f(x1,y1) and RHS = f(x2,y2).

Then, f(x1,y1) ≤ f(x1,y2) ≤ f(x2,y2) .

x1

x2

y1 y2

Min-Max Saddle-Point Property

FACT: f has the SPP if and only if

max min f (x,y) = min max f (x,y) . xX yY yY xX

Function f has the Saddle-Point Property (SPP) if it has a saddle-point, i.e., a point (x0, y0) XY such that

xX f (x , y0) ≤ f (x0 , y0) ≤ f(x0 , y) yY .

Duality via LagrangianP* = max {cT x | Ax b } x

D* = min { yT b | ATy = c } y0

Lagrangian: L(x,y) = cT x + yT(b – Ax) = yT b + xT(c – ATy)

Fact: P* = max min L(x,y) , D* = min max L(x,y) . x y0 y0 x

Proof: See Exercise 9.

Weak Duality : max min L(x,y) min max L(x,y) x y0 y0 x

P* D* .

Strong Duality : P or D feasible Lagrangian has SPP.

max min L(x,y) = min max L(x,y) x y0 y0 x

P* = D* .

George Dantzig [1947]: Simplex algorithm. Exponential time in the worst case,

but effective in practice.

Leonid Khachiyan [1979]: Ellipsoid algorithm.

The first weakly polynomial-time LP algorithm: poly(n,d,L).

Narendra Karmarkar [1984]: Interior Point Method. Also weakly polynomial-time.

IPM variations are very well studied.

Dantzig – Khachiyan – Karmarkar

Dantzig: Simplex Algorithm

Pivots along the edges of the feasible polyhedron,monotonically improving the objective value.

c

Pivot Rule: Decides which row leaves & which row enters the basis (i.e., which edge of thefeasible polyhedron to follow next).

Each BFS corresponds to a vertex of the feasible polyhedron.

Simplex Pivot:A basic row leaves the basis anda suitable row enters the basis. A non-degenerate pivot moves the BFS along an edge of the feasible polyhedronfrom one vertex to an adjacent one.

Center of GravityA set of points K d is called a convex body

if it is a full dimensional, closed, bounded, convex set.

FACT: [Grünbaum 1960]Let point c be the center of gravity of a convex body K and H be an arbitrary hyperplane passing through point c. Suppose H divides K into two bodies K+ and K– . Then the volumes vol(K+) and vol(K–) satisfy:

.)(vol11})(vol,)(volmax{ 11 KKK

d

d

cK+

K– H

d

d 1111

e

d

dd

1111lim 11

Iterative Central Bisection

FACT: Suppose the feasible set is a convex body F which is contained in a ball C0 of radius R and contains a ball C1 of radius r.Suppose we start with (the center of) the larger ball and iteratively perform central bisection until center of gravity of the remaining convex body falls within F.The number of such iterations is O(d log R/r).

Proof: vol(C0) = vdRd vol(C1) = vdrd

vd = volume of the unit ball in d .

With each iteration, the volume drops by a constant factor.

# iterations = O( log vol(C0)/vol(C1) ).

r

R

F

C0

C1

Iterative Central Bisection

The # central bisection iterations O(d log R/r)is polynomial in the bit-size of the LP instance.

However, each iteration involves computing the volume or center of gravity of the resulting convex body.

This task seems to be hard even for LP instances. We don’t know how to compute them in polynomial time.

So, we resort to approximation1) approximate the center of gravity2) approximate the convex body

What simple approximating “shape” to use? Bounding Box, Ball, ellipsoid, …?

Ellipsoids

1

1

1

1

2

1

2

oTT

od

od

odd

zxAAzx|x

zxA|x

y,Ayzx:y|xE

zo = center of ellipsoid E. A = a dd non-singular matrix.

An ellipsoid is an affine transform of the Euclidean unit ball.

zo

E

Ayzxy o

xy

0

Ellipsoids

Example:

1

00

00

22

22

21

212

2

1

2

1

a

x

a

xxx

E

z,a

aA o

1a

2a

1x

2x

E

Ellipsoids

)Adet(v)E( dvol

( vd = volume of the unit ball in d .)

zo

E

cxopt

}|{maxarg Exxcx Txopt

cAcAAz

T

T

o

12 y,Ayzx:y|xE o

dd

Löwner-John Ellipsoid

FACT 1: [K. Löwner 1934, Fritz John, 1948]Let K be a convex body in d. Then1. There is a unique maximum volume ellipsoid Ein(K) contained in

K.2. K is contained in the ellipsoid that results from Ein(K) by

centrally expanding Ein(K) by a factor of d (independent of K).

3. If K is centrally symmetric, then the factor d in part (2) can be improved to d .

Löwner-John Ellipsoid

FACT 2: [K. Löwner 1934, Fritz John, 1948]Let K be a convex body in d. Then1. There is a unique minimum volume ellipsoid Eout(K) that

contains K.2. K contains the ellipsoid that results from Eout(K) by centrally

shrinking Eout(K) by a factor of d (independent of K).

3. If K is centrally symmetric, then the factor d in part (2) can be improved to d .

Löwner-John metric approximation

Corollary: Any metric in d can be approximated by a Löwner-Johnmetric with distortion factor at most d .

Proof: Let || . ||B denote an arbitrary metric norm with (centrally symmetric) unit ball B.

|| x ||B = min { l | x lB } for any x d .

Let ELJ = { xd | || A–1 x ||2 1 } be the min volume Löwner-John ellipsoidthat contains B.

The metric norm with ELJ as the unit ball is || x ||LJ = || A–1 x ||2 for any x d .

.1 have We LJLJ Ed

BE

.|||||||||||| Therefore, LJBLJ xdxx

ELJ

B

LJEd

1

c

xk-1

xopt

zo

Iterative Löwner-John Ellipsoidal Cut

)()( ooutT

ooptT zxczxc

)()1()( 01 zxcxxc opt

Tdkopt

T

dedxxc

xxc

koptT

koptT

/111)(

)(

1

)()1( 11

koptT

d xxc

)( okT zxcd )()( kopt

Toopt

T xxcdzxcd

xk

xout

 

dkexxc

xxc

optT

koptT

/

)(

)(

0

Iterative Löwner-John Ellipsoidal Cut

iterations

)(log

after

)(

0

xxc d Ok

xxc

optT

koptT

dkexxc

xxc

optT

koptT

/

)(

)(

0

# iterations is polynomial in the bit-size of the LP instance.

However, each iteration is expensive and involves computing

Löwner-John Ellipsoid of a convex polytope.

What next? Approximate the convex body by an ellipsoid.

Khachiyan: Ellipsoid MethodCircumscribed ellipsoid volume reduction by cuttings.

Ek

)1(21

)(vol

)(vol 1

de

k

k

E

E

zk

Ek+1

zk+1

# iterations = O(d2 log R/r)

Time per iteration = O(d n)

The New York Times, November 7, 1979

Continued on next slide.

Karmarkar: Interior Point Method

maximize cTx + i log si

subject to Ax+s=b, s > 0

= =1

=0.01

=

=1=0.01

Central path

Logarithmic Barrier and Central Path

IPM: Primal-Dual Central Path(P) max cTx (D) min bTy

s.t. Ax+s=b, s 0 s.t. ATy=c, y 0

Optimality Conditions on (x,s,y):

Ax+s=b, s 0 (Primal feasibility)

ATy=c, y 0 (Dual feasibility)

si yi = 0 i (Complementary Slackness)

Primal-Dual Central Path parameterized by > 0:

Ax+s=b, s > 0 (Primal strict feasibility)

ATy=c, y > 0 (Dual strict feasibility)

si yi = i (Centrality)

IPM: Primal-Dual Central Path

Algorithmic Framework:

Assume both (P) and (D) have strictly feasible solutions. Then, for any > 0, the

above non-linear system of equations has a unique solution (x(), s(), y()).These points form the Primal-Dual Central Path { (x(), s(), y()) | (0, ) }.

lim 0 (x(), s(), y()) = (x*, s*, y*) is optimal solution of the Primal-Dual LP.

Update approximate solution to the system while iteratively reducing towards 0.

Central Path:Ax+s=b, s > 0 ATy=c, y > 0 si yi = i

Open Problem

Can LP be solved in strongly polynomial time?

Strongly Polynomial:(i) Time = poly(n,d) [i.e., independent of input coefficient sizes]

(ii) Bit-length of each program variable = poly(n,d,L).

Weakly Polynomial:(i) Time = poly(n,d,L), [L = bit-length of input coefficients]

(ii) Bit-length of each program variable = poly(n,d,L).

[ Steve Smale, “Mathematical Problems for the Next Century’’, Mathematical Intelligencer, 20:7–15, 1998] lists 18 famous open problems and invites the scientific community to solve them in the 21st century. Problem 9 on that list is the above open problem on LP.

Best known upper-bound [Kalai, Kleitman 1992]:

D(d,n) nlog d + 2

For a recent survey on this topic click here.

Hirsch’s Conjecture [1957]Hirsch’s Conjecture:

Diameter of any d-polyhedron with n facets satisfiesD(d,n) n – d.

That is, any pair of its vertices are connected by a path of at most n d edges.

This conjecture was disproved by Francisco Santos in June 2010.Click here to see Santos’s paper.

LP in dimension 2

2D Linear Programming

Fy

x

Objective function = c1 x + c2 y, cT =(c1 , c2)Feasible region F

c

optimummax = c 1 x

+ c 2 y

2D Linear Programming

Feasible region F is the intersection of n half-planes.

F is (empty, bounded or unbounded) convex polygon with n vertices.

F can be computed in O(n log n) time by divide-&-conquer

(See Lecture-Slide 3).

If F is empty, then LP is infeasible. Otherwise, we can check its vertices, and its possibly up to

2 unbounded edges, to determine the optimum. The latter step can be done by binary search in O(log n) time. If objective changes but constraints do not, we can update the optimum

in only O(log n) time. (We don’t need to start from scratch).

Improvement Next:

Feasible region need not be computed to find the optimum vertex.

Optimum can be found in O(n) time both randomized & deterministic.

2D LP Example: Manufacturing with Molds

2D LP Example: Manufacturing with Molds

The Geometry of Casting: Is there a mold for an n-faceted 3D polytope P such that P can be removed from the mold by translation?

Lemma: P can be removed from its mold with a single translation in direction d

d makes an angle 90 with the outward normal of all non-top facets of P.

mold

P

f

f ’

(f ‘) = - (f)

d

Corollary: Many small translations possible Single translation possible.

2D LP Example: Manufacturing with Molds

The Geometry of Casting: Is there a mold for an n-faceted 3D polytope P such that P can be removed from the mold by translation?

mold

P

f

f ’

(f ‘) = - (f)

d

(f) = ( x(f) , y(f) , z(f) ) outward normal to facet f of P.

dT.(f) ≤ 0 non-top facet f of P

x(f) . x + y(f) . y + z(f) ≤ 0 f n-1 constraints

x

y

z

z=1d=(x,y,1)

THEOREM: The mold casting problem can be solved in O(n log n) time. (This will be improved to O(n) time on the next slides.)

Randomized IncrementalAlgorithm

Randomization

Random(k): Returns an integer i 1..k, each with equal probability 1/k. [Use a random number generator.]

Algorithm RandomPermute (A) O(n) time

Input: Array A[1..n]

Output: A random permutation of A[1..n] with each of n! possible permutations equally likely.

for k n downto 2 do Swap A[k] with A[Random(k)]

end.

This is a basic “initial” part of many randomized incremental algorithms.

2D LP: Incremental Algorithm

Method: Add constraints one-by-one, while maintaining the current optimum vertex.

Input: (H, c), H = { H(1), H(2), … , H(n)} n half-planes, c = objective vectorOutput: Infeasible: (i,j,k), or

Unbounded: , or Optimum: v = argmaxx { cTx | x H(1)H(2)…H(n) }.

Define: C(i) = H(1)H(2)…H(i) , for i = 1..n v(i) = optimum vertex of C(i) , for i=2..n. cTv(i) = max { cTx | x C(i) }

Note: C(1) C(2) … C(n).

e

v

Infeasible Unbounded Non-unique optimum Unique optimum

cPossible Outcomes:

H(i) H(j)

H(k)

2D LP: Incremental Algorithm

LEMMA: (1) v(i-1) H(i) v(i-1) C(i) v(i) v(i-1). (2) v(i-1) H(i) (2a) C(i) = , or (2b) v(i) L(i) C(i-1), L(i) =bounding-line of H(i).

H(i) v(i)=v(i-1)

(1)

C(i-1)

H(i)

(2a)

C(i-1)

v(i-1)

H(j)

H(k)

H(i)

(2b)

C(i-1)

v(i-1)

v(i)

L(i)L(i)

c

2D LP: Incremental Algorithm

Algorithm PreProcess (H,c) O(n) time 1. min { angle between c and outward normal of H(i) | i=1..n }

H(i) = most restrictive constraint with angle Swap H(i) with H(1) (L(1) is bounding-line of H(1)).

2. If (parallel) H(j) with angle and H(1)H(j)= then return “infeasible”.3. If L(1) H(j) is unbounded for all H(j) H, then

most restrictive L(1) H(j) over all H(j) H

return (“unbounded”, )4. If L(1) H(j) is bounded for some H(j)H, then

Swap H(j) with H(2) and return “bounded”.

c L(1)

H(1)

L(1)H(j)

H(2)H(1)

(1) (3) (4)

L(1)

H(1)

(2)

H(j)

Disjoint p

arallel.

v(2)

Input: (H, c), H = { H(1), H(2), … , H(n)} n half-planes, c = objective vector

Output: Solution to max { cTx | x H(1)H(2)…H(n) }

1. if PreProcess(H,c) returns (“unbounded”, ) or “infeasible”

then return the same answer

(* else bounded or infeasible *)

2. v(2) vertex of H(1) H(2)

3. RandomPermute (H[3..n])

4. for i 3..n do

5. if v(i-1) H(i) then v(i) v(i-1)

6. else v(i) optimum vertex p of L(i)(H(1)…H(i-1)) (* 1D LP *)

7. if p does not exist then return “infeasible”

8. end-for

9. return (“optimum”, v(n))

end.

Randomized Incremental 2D LP Algorithm

THEOREM: 2D LP Randomized Incremental algorithm has the following complexity: Space complexity = O(n) Time Complexity: (a) O(n2) worst-case

(b) O(n) expected-case.

Randomized Incremental 2D LP Algorithm

.)()(3

2

n

i

nOiO

Proof of (a): Line 6 is a 1D LP with i-1constraints and takes O(i) time.

Total time over for-loop of lines 4-8:

Proof of (b): Define 0/1 random variables

Lines 5-7 take O( i*X(i) +1) time. Total time is

Randomized Incremental 2D LP Algorithm

.n..3ifor otherwise0

H(i)1)v(iif1)i(X

n

3i

)i(X*)i(O)n(OT

])i(X*)i(O[E)n(O]T[En

3i

v(i) is defined by 2 H(j)’s. The probability that one of them is H(i) is 2/(i-2).This does not depend on C(i). Hence, remove the “Fix” assumption.

n

i

nOi

iOnOTE3

).(2

2*)()(][ :Therefore

Expected time:

)2/(i-2 ]H(i))1v(i-[Pr ]X(i)[E

])i(X[E*)i(O)n(On

3i

BackwardsAnalysis

linearity ofexpectation

“Fix” C(i) = { H(1), H(2)} {H(3), …, H(i)}

Random : C(i-1) = C(i) – {H(i)} random

Randomized Incremental LP in d dimension

Randomized Incremental algorithm can be used (with minor modifications)in d dimensions, recursively.

For example, for 3D LP line 6 becomes a 2D LP with i-1 constraints.In general, it will be a (d-1) dimensional LP.

Define: T(n,d) = running time of algorithm for n constraints & d variables.

i

]di)1d,1i(T[)dn(O)d,n(T

This would be exponential in both n and d.

otherwise0

H(i)1)v(iif1)i(X

Expected time (by backwards analysis):

slide]next on [proof . dn for ))(!()],([

*)]1,1([)()],([

.]1[Pr][

1

dndOdnTE

di

ddiTEdnOdnTE

d/(i-d) H(i))v(i- X(i)En

di

Randomized Incremental LP in d dimension

(1)i!

1F whered))(nO(d!dFd!3d)c(nd)T(n,

1d

0idd:Solution

.dFd!3d)c(n

1)d(ncddFd!3d)c(n

d)1)(ncd(ddd)c(n3Fd!d)c(n3cdn

d)1)(ncd(d1)!(d

1Fd!d)c(n3cdn

d)1)(ncd(dd)(nFcd!3cdn

1)(dF1)!3(dd)c(idi

dcdnd)T(n,

:1)d(n

d

2d

d

d

1d

n

1di1d

StepInduction

.cdFcd!3i!

2icd! 1)d-T(d,d 1)cd(d d)1,T(d

:1)d(n

:non induction By

d

1d

0iexpansioniterativeby

)1d(G)d(G

Basis:Proof

[For nd, T(n,d) = O(dn2), by Gaussian elimination.]

n

1di

dfor n )1,d1T(icdnT(n,d) did

Randomized Incremental Algorithm for

Smallest Enclosing Disk

Smallest Enclosing Disk

pi+1

Lemma: (1) pi D(i-1) D(i) = D(i-1) (2) pi D(i-1) pi lies on the boundary of D(i).

pi

D(i+1)

D(i)=D(i-1)

Input: A set P={p1, p2, … , pn } of n points in the plane.

Output: Smallest enclosing disk D of P.

Lemma: Output is unique.

Incremental Construction:P[1..i] = {p1, p2, … , pi }

D(i)= smallest enclosing disk of P[1..i] .

Smallest Enclosing Disk

LEMMA: Let P and R be disjoint point sets in the plane. pP, R possibly empty.Define MD(P, R) = minimum disk D such that P D & R D ( D = boundary of D). (1) If MD(P, R) exists, then it’s unique,(2) p MD(P-{p}, R) MD(P,R) = MD(P-{p}, R),(3) p MD(P-{p}, R) MD(P, R) = MD(P-{p}, R {p}).

D(0)

D(1)R

p

D()

(3) D(0) MD(P-{p}, R) D(1) MD(P, R) D() (1-) D(0) + D(1) 0 1

As goes from 0 to 1, D() continuously deforms from D(0) to D(1) s.t.

D(0) D(1) D().

p D(1) – D(0) by continuity smallest *, 0 < * 1 s.t.

p D(*) p D(*).

P D(*) & R D(*) *=1 by uniqueness.

Therefore, p is on the boundary of D(1).

Proof: (1) If non-unique smaller such disk:

(2) is obvious.

Smallest Enclosing Disk

Algorithm MinDisk (P[1..n])1. RandomPermute(P[1..n])2. D(2) smallest enclosing disk of P[1..2]3. for i 3..n do4. if pi D(i-1) then D(i) D(i-1)5. else D(i) MinDiskWithPoint (P[1..i-1] , pi) 6. return D(n)

Procedure MinDiskWithPoint (P[1..j],q)1. RandomPermute(P[1..j])

2. D(1) smallest enclosing disk of p1 and q3. for i 2..j do

4. if pi D(i-1) then D(i) D(i-1)

5. else D(i) MinDiskWith2Points (P[1..i-1] , q, pi) 6. return D(j)

Procedure MinDiskWith2Points (P[1..j],q1,q2)

1. D(0) smallest enclosing disk of q1 and q2

2. for i 1..j do

3. if pi D(i-1) then D(i) D(i-1)

4. else D(i) Disk (q1, q2, pi) 5. return D(j)

Sho

w r

ando

m p

erm

utat

ion

only

onc

e is

eno

ugh

Smallest Enclosing Disk

Proof: Space O(n) is obvious.MinDiskWith2Points (P,q1,q2) takes O(n) time.MinDiskWithPoint (P,q) takes time:

n

2i

)i(X*)i(O)n(OT where otherwise0

)1i(D(i)Dpif1)i(X i

n

2i

).n(Oi

2*)i(O)n(O]T[E

analysis) backwards(by i/2)]i(X[E

Apply this idea once more: expected running time of MinDisk is also O(n).

P[1..i]

q

“Fix” P[1..i] = {p1, … , pi} backwards P[1..i-1] = {p1, … , pi} - {pi}

one of these is pi with prob. 2/i

THEOREM: The smallest enclosing disk of n points in the plane can be computed in randomized O(n) expected time and O(n) space.

2D LPPrune-&-Search

2D LP: Megiddo(83), Dyer(84): Prune-&-Search

STEP 1: Transform so that the objective hyper-plane becomes horizontal. (a,b) (0,0), so WLOG b 0. Coordinate transform:

Y = ax + by i = ai – (a/b) bi

X = x i = bi /b

min { Y | i X + i Y + ci 0 , i=1..n }

X

Y

Ffeasibleregion

optimum

Original LP: min { ax + by | ai x + bi y + ci 0 , i=1..n }

2D LP: Prune-&-Search

STEP 2: Partition the constraints (the index set [1..n])I0 = { i | i = 0 } vertical line constraints determine feasible x-interval

I- = { i | i < 0 } I+ = { i | i > 0 }

STEP 3:

0,Ii|c

minU

0,Ii|c

maxU

UXU

i0i

i2

i0i

i1

21

X

Y

F

U1 U2

STEP 4:

IiXY : become Iin sconstraint

IiXY : become Iin sconstraint So

c, Define

ii

ii

i

ii

i

ii

min { Y | i X + i Y + ci 0 , i=1..n }

2D LP: Prune-&-Search

STEP 5: )X(max)X(F,)X(min)X(F Define iiIi

iiIi

F+

F-

So, the transformed constraints are: F-(X) Y F+ (X).Since we want to minimize Y, our objective function is F-(X).

STEP 6: Our new (transformed) problem is:

X

Y

F

U1U2

21 UXU

)X(F)X(F s.t.

)X(F minimize

optimum

F+

F-

2D LP: Prune-&-Search

7(a): Given X [U1 , U2 ] , we can determine in O(n) time:

F-(X) = max { i X + i | i I- }

F+(X) = min { i X + i | i I+ }

f- (L)

(X) = left slope of F-(X) at X

f- (R)

(X) = right slope of F-(X) at X

f+ (L)

(X) = left slope of F+(X) at X

f+ (R)

(X) = right slope of F+(X) at X

Note: if only one i I- achieves the “max” in F-(X), then f-

(L) (X) = f-

(R) (X) = i . Similarly with F+(X).

X’ X

F-

STEP 7: An Evaluation Stage:

2D LP: Prune-&-Search

7(b): In O(n) time we can determine for X [U1 , U2 ], whether: X achieves the minimum of F-(X) and is feasible (hence optimum). X is infeasible & there is no solution to the LP X is infeasible and we know which side (left/right) of X the feasible region F lies. X is feasible and we know which side of X the optimum solution lies.

In the latter two cases, shrink the interval [U1 , U2 ] accordingly.

F-(X)

F+(X)

X X

XX

f- (L)

(X) > f+ (L)

(X)

infeasibleF to

the left

f- (R)

(X) < f+ (R)

(X)

F to the right

f- (L)

(X) f+ (L)

(X) & f- (R)

(X) f+ (R)

(X) F = ; LP is infeasible

OR

STEP 7: An Evaluation Stage:

2D LP: Prune-&-Search

METHOD:

A series of evaluation stages . Prune at least a fixed fraction (1-) of the constraints in each stage.

These constraints won’t play a role in determining the optimum vertex. After the k-th stage, # constraints remaining k n. At most log n stages. Total time:

).n(OnOnαO α11

nlog

0k

kα

2D LP: Prune-&-Search

METHOD: (continued)

= 3/4 is achievable. Which X to select to evaluate then prune? Pair up constraints in I+ (do the same for I- ).

Suppose i,j I+ are paired up.

para

llel

iiXY

jjXY eliminate

U1 U2

eliminateXij

U1 U2

eliminate Xij

U1 U2

Xij eliminateneither one

METHOD: (continued)

Next evaluation point:

X* = median of the Xij’s of the non-eliminated pairs. O(n) time.

Half of the Xij’s will be on the wrong side of X*.

We will eliminate one line from at least half of the pairs. So, at least ¼ of the lines will be eliminated in this stage. At most = ¾ of the lines remain.

THEOREM: A 2-variable LP with n constraints can be solved in O(n) timein the worst-case by Megiddo-Dyer’s Prune-&-Search method.

GENERALIZATION: Megiddo[84] showed d dim. LP can also be solved by this technique in O(n) time assuming d is a fixed constant.[However, the hidden constant in O(n) is super-exponential in d.]

2D LP: Prune-&-Search

Extensions & Applications

The Smallest Enclosing Disk:2

i2

i

ni1y,x

)yy()xx(maxmin

THEOREM [Megiddo]: The smallest enclosing disk can be solved in O(n) timein the worst-case by the Prune-&-Search method.

See Lecture Notes 7-13 for more on this topic.

n..1ifor )yy()xx(z s.t.

z minimize

2i

2i A Quadratic

Program.See Exercise 16.

Exercises

1. A Non-Linear Primal Dual Example: [Rochat, Vecten, Fauguier, Pilatte, 1811] We are given 3 points A,B,C in general position in the plane.

Primal: Consider any equilateral triangle PQR whose 3 sides pass through A,B,C, respectively. (See Figure below.) Let H denote the altitude of the equilateral triangle PQR. Find PQR to maximize H.

Dual: Consider any point S, with L as the sum of the lengths of the 3 segments L = SA + SB + SC. Find S to minimize L. [The optimum S is called the Steiner point of ABC.]

Prove the following claims:a) For any primal-dual feasible solutions H L.b) At optimality max H = min L.c) At optimality SA QR, SB RP, SC PQ,

where means perpendicular.

A

B

C P

Q

R

S

2. LP Primal Dual Example: Give the dual of the following LP instance:

max 5 x1 + 12 x2 - 6 x3 + 9 x4 + 4 x5 subject to: -2 x1 + 5 x2 + 13 x3 + 6 x4 + 7 x5 = 53 11 x1 - 4 x2 + 5 x3 + 8 x4 - 2 x5 31 -4 x1 + 6 x2 + 14 x3 + 2 x4 + 7 x5 43 8 x1 - 3 x2 - 7 x3 + 9 x4 - 14 x5 61 x10, x3 0, x40, x50

3. Fractional Knapsack Problem (FKP):

FKP is the LP max { vTx | wTx W, 0xi1, i=1..n }, where vT=(v1 ,…, vn ) and wT =(w1 ,…, wn ) are vectors of item values and weights,and W is the knapsack capacity. Assume these are all positive reals.The real variable xi , 0xi1, is the selected fraction of the i-th item, for i=1..n.

a) Give the dual of FKP.

b) Use the Duality Theorem to show that a simple greedy algorithm obtains optimal primal-dual solutions. [Hint: sort vi / wi , for i=1..n.]

4. Polyhedral Vertices versus Basic Solutions: Consider the (possibly empty) polyhedron F = { xd | Ax b } for some And, bn, d n. Assume rank(A)=d (i.e., A contains d linearly independent rows). Show the following facts:

a) F has a basic solution (not necessarily feasible).

b) The cone defined by the constraints corresponding to a basis is pointed (i.e., does not contain any line).

c) F is pointed.

d) Every non-empty face of F has a vertex.

e) Every vertex of F corresponds to at least one basic feasible solution.

f) A vertex could correspond to more than one basic feasible solution.

g) Every vertex of F is the unique optimum LP solution for somelinear objective vector and with F as the feasible region.

5. Give LP Primal-Dual examples where:

a) Both the Primal and the Dual are infeasible.

b) The Primal is unbounded and the Dual is infeasible.

c) Both the Primal and the Dual have bounded optima.

6. Farkas Lemma (Alternative versions):Prove the remaining 3 versions of Farkas Lemma.

7. Fundamental Theorem of Polyhedra [Minkowski-Weyl] when P is not pointed:a) Example 1: Consider the non-pointed polyhedron P={(x1, x2)T2 | x11, -x11}. Show that P={ x 2 | x = Vl + Rm, m0, l0, Si li = 1 }, where V consists of the 2 columns v1=(1,0)T, v2=(-1,0)T, and R consists of the 2 columns r1=(0,1)T, r2=(0,-1)T. [Note that v1 and v2 are not extreme points of P.]b) Example 2: Show the alternative representation (according to the Theorem) for the non-pointed polyhedron P = {(x1, x2, x3)T3 | x1+x2 -x31, -x1+x2 -x31}. c) Prove the () part of the Theorem when P is not pointed. [Our referenced book [Ziegler, “Lectures on Polytopes”] contains a proof using the Fourier-Motzkin elimination method.]

8. Strong Duality Theorem: Prove the Strong Duality Theorem using Farkas Lemma. [Hint: a = min{yTb : yTA=c, y0} implies the system {yTb – a < 0, yTA - c = 0, y0} has no solution in y. Reformulate this as F.L.(2) using an extra variable y0. What is F.L.(1)?]

9. LP Duality via Lagrangian: Consider the primal-dual pair of linear programs (Primal) P* = max { cTx | Ax < b } (Dual) D* = min { bTy | ATy = c and y > 0 }

and their Lagrangian L(x,y) = cTx + yT (b – Ax) = bTy + xT (c – ATy) .

Define the functions p(x) = miny>0 L(x,y) andd(y) = maxx L(x,y).

That is, for any x, p(x) is obtained by a y > 0 that minimizes the Lagrangian (for the given x). Similarly, for any y > 0, d(y) is obtained by an x that maximizes the Lagrangian (for the given y).

a) Show that

b) Show that P* = maxx p(x) = maxx miny>0 L(x,y) and D* = miny>0 d(y) = miny>0 maxx L(x,y).

c) Show that if at least one of the Primal or the Dual LP has a feasible point, thenthe Lagrangian has the Saddle Point Property, i.e., maxx miny>0 L(x,y) = miny>0 maxx L(x,y).

[Can you prove this without assuming the LP Strong Duality Theorem?]

. otherwise

if

, otherwise

if

cyAybyd

bAxxcxp

TT

T

)(

)(

10. In the restricted version of the mold casting problem we insist that the object is removed from its mold using a vertical translation (perpendicular to the top facet).(a) Prove that in this case only a constant number of top facets are possible.(b) Give a linear time algorithm that determines all such possible top facets.

11. Let H be a set of at least 3 half-planes. A half-plane hH is called redundant if itsremoval does not change H (the intersection of half-planes in H). Prove that h is redundant if and only if h’h’’ h, for some h’,h’’H – {h}.

12. Prove that RandomPermute(A[1..n]) is correct, i.e., each of the n! permutationsof A can be the output with equal probability 1/n!.

13. Show how to implement MinDisk using a single routine MinDiskWithPoints(P,R)that computes MD(P,R) as defined before. Your algorithm should compute only asingle random permutation during the entire computation.

14. Show the “bridge finding” problem in Kirkpatrick-Seidel’s O(n log h) time 2D CHalgorithm can be solved in O(n) time by Megiddo-Dyer’s technique.

15. Minimum Area Annulus: An annulus is the portion of the plane contained between two concentric circles. Let a set P of n points in the plane be given. Give an efficient algorithm to find the minimum area annulus that contains P.[Hint: This can be solved by an LP. We want to find a center (x,y) and two radii R and r to minimize R2 – r2 subject to the 2n constraints r2 (x-xi)2 + (y – yi)2 R2 . Now instead of the variables x,y,r,R, use the variables x, y, w x2 +y2 - R2 and z x2 +y2 - r2. Also ensure that the LP solution satisfies x2 +y2 = w + R2 = z + r2 max{w,z}. How?]

16. Minimum Enclosing Disk in 2D: Show this problem can be solved in O(n) time by prune-&-search.[Hint: First show the restricted problem where the disk center lies on a given line can be solvedin O(n) time. Then show it can be decided in O(n) time on which side of this line the center of the (unrestricted) minimum enclosing disk lies. Then apply prune-&-search to this scheme.]

17. Maximum Inscribed Sphere: Let P d be a convex polytope, given as intersection of n half-spaces. Give a formulation of the problem to determine the largest sphere inscribed in P.You don’t need to solve the problem; just give a good formulation of it.[The center of this sphere is called the Chebyshev center of the polytope.]

END