# mathematical and algorithmic foundations linear ... linear programming introduction 1 linear...

Post on 05-Jul-2020

1 views

Embed Size (px)

TRANSCRIPT

Adavnced Algorithms Lectures

Mathematical and Algorithmic Foundations Linear Programming and Matchings

Paul G. Spirakis

Department of Computer Science University of Patras and Liverpool

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 1 / 91

Outline

1 Linear programming

2 Bipartite matching

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 2 / 91

Linear programming Introduction

1 Linear programming Introduction The geometry of LP Feasibility of LP Duality Solving a linear program

2 Bipartite matching

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 3 / 91

Linear programming Introduction

Linear programs

A linear program (LP) is the problem of optimizing (i.e., minimizing or maximizing) a linear objective function subject to linear equality or inequality constraints.

In standard form, it is expressed as:

minimize ∑n

j=1 cjxj (objective function)

subject to: ∑n

j=1 aijxj = bi , i = 1, . . . ,m (constraints)

xj ≥ 0 , j = 1, . . . , n (non-negativity constraints)

where aij , bi , cj are given for all i , j .

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 4 / 91

Linear programming Introduction

Expressing LP using matrices

A linear program is expressed more conveniently using matrices:

min cT x subject to

{ Ax = b x ≥ 0

where x = (xj)j∈{1,...,n} ∈ Rn×1 b = (bi )i∈{1,...,m} ∈ Rm×1 c = (cj)j∈{1,...,n} ∈ Rn×1 A = (aij)i∈{1,...,m},j∈{1,...,n} ∈ Rm×n

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 5 / 91

Linear programming Introduction

Basic terminology

If x satisfies Ax = b, x ≥ 0, then x is feasible. An LP is feasible if there exists a feasible solution, otherwise it is said to be infeasible.

An optimal solution x∗ is a feasible solution such that

cT x∗ = min{cT x : Ax = b, x ≥ 0} .

An LP is unbounded (from below) if for all λ ∈ R, there exists a feasible x∗ such that

cT x∗ ≤ λ .

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 6 / 91

Linear programming Introduction

Equivalent forms

An LP can take several forms:

We might be maximizing instead of minimizing;

We might have a combination of equality and inequality constraints;

Some variables may be restricted to be non-positive instead of non-negative, or be unrestricted in sign.

Two forms are said to be equivalent if they have the same set of optimal solutions or are both infeasible or both unbounded.

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 7 / 91

Linear programming Introduction

Equivalent forms

A maximization problem can be expressed as a minimization problem:

max cT x ⇔ min −cT x

An equality can be represented as a pair of inequalities:

aTi = bi ⇔ {

aTi x ≤ bi aTi x ≥ bi

⇔ {

aTi x ≤ bi −aTi x ≤ −bi

By adding a slack variable, an inequality can be represented as a combination of equality and non-negativity constraints:

aTi x ≤ bi ⇔ aTi x + si = bi , si ≥ 0

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 8 / 91

Linear programming Introduction

Equivalent forms

Non-positivity constraints can be expressed as non-negativity constraints: To express xj ≤ 0, replace xj everywhere with −yj and impose the condition yj ≥ 0. If xj is unrestricted in sign, i.e., non-positive or non-negative, replace everywhere xj by x

+ j − x

− j , adding the constraints x

+ j , x

− j ≥ 0.

In general, an inequality can be represented using a combination of equality and non-negativity constraints, and vice versa.

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 9 / 91

Linear programming Introduction

Canonical and standard forms

1 LP in canonical form:

min{cT x : Ax ≥ b}

2 LP in standard form:

min{cT x : Ax = b, x ≥ 0}

Using the rules described before, an LP in standard form may be written in canonical form, and vice versa.

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 10 / 91

Linear programming Introduction

Example

The optimal solution of the following LP is (4, 2) of cost 2:

min x2 s.t. x1 ≥ 2

3x1 − x2 ≥ 0 x1 + x2 ≥ 6

−x1 + 2x2 ≥ 0

x 2

x 1

-x 1 +2x

2 ≥ 0

x 1 ≥ 2

(4,2) x 1 +x

2 ≥ 6

3x 1 -x 2 ≥ 0

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 11 / 91

Linear programming Introduction

Example

x 2

x 1

-x 1 +2x

2 ≥ 0

x 1 ≥ 2

(4,2) x 1 +x

2 ≥ 6

3x 1 -x 2 ≥ 0

If we were maximizing x2 instead of minimizing under some feasible region, the resulting LP would be unbounded.

For any objective function for which the LP is bounded, there seems to exist an optimal solution which is a “corner” of the feasible section.

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 12 / 91

Linear programming The geometry of LP

Vertices

Consider an LP in standard form:

min{cT x : Ax = b, x ≥ 0}

Let P = {x : Ax = b, x ≥ 0} ⊆ Rn.

Definition

x is a vertex of P if @y 6= 0 such that x + y ∈ P and x − y ∈ P.

Theorem

Assume min{cT x : x ∈ P} is finite. Then, for all x ∈ P, there exists a vertex x ′ ∈ P such that

cT x ′ ≤ cT x .

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 13 / 91

Linear programming The geometry of LP

Vertices

Proof.

Recall that P = {x : Ax = b, x ≥ 0} ⊆ Rn. If x is a vertex then take x ′ = x .

If x is not a vertex then, by definition, there exists y 6= 0 such that x + y ∈ P and x − y ∈ P. Then A(x + y) = b and A(x − y) = b, implying that Ay = 0. W.l.o.g. assume cT y ≤ 0 (take either y or −y). If cT y = 0 choose y such that ∃j : yj < 0. Since y 6= 0 and cT y = cT (−y) = 0, this must be true for either y or −y .

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 14 / 91

Linear programming The geometry of LP

Vertices

Proof (continued). Consider x + λy , λ > 0. Since cT y ≤ 0,

cT (x + λy) = cT x + λcT y ≤ cT x .

Case 1: ∃j : yj < 0. As λ increases, component j decreases until x + λy is no longer feasible.

Choose the largest λ such that x + λy ≥ 0, i.e.,

λ = min j :yj

Linear programming The geometry of LP

Vertices

Proof (continued). Case 2: yj ≥ 0 for all j .

By assumption, cT y < 0.

Moreover,

A(x + λy) = Ax + λAy = Ax = b and

x + λy ≥ x ≥ 0 .

But cT (x + λy) = cT x + λcT y → −∞ as λ→∞, implying LP is unbounded, a contradiction.

Case 1 can happen at most n times (the number of components of x). By induction on the number of nonzero components of x , we obtain a vertex x ′. �

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 16 / 91

Linear programming The geometry of LP

Vertices and optimal solutions

Corollary

If min{cT x : Ax = b, x ≥ 0} is finite, there exists an optimal solution x∗ which is a vertex.

Proof. Suppose not. Take an optimal solution. By the previous theorem, there exists a vertex costing no more than the optimal. This vertex must be optimal as well! �

Corollary

If P = {x : Ax = b, x ≥ 0} 6= ∅ (i.e., if the LP is feasible), then P has a vertex.

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 17 / 91

Linear programming The geometry of LP

Vertices and optimal solutions

Corollary

If min{cT x : Ax = b, x ≥ 0} is finite, there exists an optimal solution x∗ which is a vertex.

Proof. Suppose not. Take an optimal solution. By the previous theorem, there exists a vertex costing no more than the optimal. This vertex must be optimal as well! �

Corollary

If P = {x : Ax = b, x ≥ 0} 6= ∅ (i.e., if the LP is feasible), then P has a vertex.

Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 17 / 91

Linear programming The geometry of LP

A characterization of vertices

Theorem

Let P = {x : Ax = b, x ≥ 0}. For x ∈ P, let Ax be a submatrix of A corresponding to j such that xj > 0. Then x is a vertex iff Ax has linearly independent columns (i.e., Ax has full column rank).

Example: Let

A =

2 1 3 07 3 2 1 0 2 0 5

, x =

2 0 1 0

. Then Ax corresponds to the positive components of x , j = 1 and j = 3, i.e., to the 1st and 3rd column of A:

Ax =

2 37 2 0 0

, and x is a vertex. Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 18 / 91

Linear programming The geometry of LP

A characterization of vertices

Pr

Recommended