mathematical and algorithmic foundations linear ... linear programming introduction 1 linear...

Download Mathematical and Algorithmic Foundations Linear ... Linear programming Introduction 1 Linear programming

Post on 05-Jul-2020

1 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Adavnced Algorithms Lectures

    Mathematical and Algorithmic Foundations Linear Programming and Matchings

    Paul G. Spirakis

    Department of Computer Science University of Patras and Liverpool

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 1 / 91

  • Outline

    1 Linear programming

    2 Bipartite matching

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 2 / 91

  • Linear programming Introduction

    1 Linear programming Introduction The geometry of LP Feasibility of LP Duality Solving a linear program

    2 Bipartite matching

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 3 / 91

  • Linear programming Introduction

    Linear programs

    A linear program (LP) is the problem of optimizing (i.e., minimizing or maximizing) a linear objective function subject to linear equality or inequality constraints.

    In standard form, it is expressed as:

    minimize ∑n

    j=1 cjxj (objective function)

    subject to: ∑n

    j=1 aijxj = bi , i = 1, . . . ,m (constraints)

    xj ≥ 0 , j = 1, . . . , n (non-negativity constraints)

    where aij , bi , cj are given for all i , j .

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 4 / 91

  • Linear programming Introduction

    Expressing LP using matrices

    A linear program is expressed more conveniently using matrices:

    min cT x subject to

    { Ax = b x ≥ 0

    where x = (xj)j∈{1,...,n} ∈ Rn×1 b = (bi )i∈{1,...,m} ∈ Rm×1 c = (cj)j∈{1,...,n} ∈ Rn×1 A = (aij)i∈{1,...,m},j∈{1,...,n} ∈ Rm×n

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 5 / 91

  • Linear programming Introduction

    Basic terminology

    If x satisfies Ax = b, x ≥ 0, then x is feasible. An LP is feasible if there exists a feasible solution, otherwise it is said to be infeasible.

    An optimal solution x∗ is a feasible solution such that

    cT x∗ = min{cT x : Ax = b, x ≥ 0} .

    An LP is unbounded (from below) if for all λ ∈ R, there exists a feasible x∗ such that

    cT x∗ ≤ λ .

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 6 / 91

  • Linear programming Introduction

    Equivalent forms

    An LP can take several forms:

    We might be maximizing instead of minimizing;

    We might have a combination of equality and inequality constraints;

    Some variables may be restricted to be non-positive instead of non-negative, or be unrestricted in sign.

    Two forms are said to be equivalent if they have the same set of optimal solutions or are both infeasible or both unbounded.

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 7 / 91

  • Linear programming Introduction

    Equivalent forms

    A maximization problem can be expressed as a minimization problem:

    max cT x ⇔ min −cT x

    An equality can be represented as a pair of inequalities:

    aTi = bi ⇔ {

    aTi x ≤ bi aTi x ≥ bi

    ⇔ {

    aTi x ≤ bi −aTi x ≤ −bi

    By adding a slack variable, an inequality can be represented as a combination of equality and non-negativity constraints:

    aTi x ≤ bi ⇔ aTi x + si = bi , si ≥ 0

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 8 / 91

  • Linear programming Introduction

    Equivalent forms

    Non-positivity constraints can be expressed as non-negativity constraints: To express xj ≤ 0, replace xj everywhere with −yj and impose the condition yj ≥ 0. If xj is unrestricted in sign, i.e., non-positive or non-negative, replace everywhere xj by x

    + j − x

    − j , adding the constraints x

    + j , x

    − j ≥ 0.

    In general, an inequality can be represented using a combination of equality and non-negativity constraints, and vice versa.

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 9 / 91

  • Linear programming Introduction

    Canonical and standard forms

    1 LP in canonical form:

    min{cT x : Ax ≥ b}

    2 LP in standard form:

    min{cT x : Ax = b, x ≥ 0}

    Using the rules described before, an LP in standard form may be written in canonical form, and vice versa.

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 10 / 91

  • Linear programming Introduction

    Example

    The optimal solution of the following LP is (4, 2) of cost 2:

    min x2 s.t. x1 ≥ 2

    3x1 − x2 ≥ 0 x1 + x2 ≥ 6

    −x1 + 2x2 ≥ 0

    x 2

    x 1

    -x 1 +2x

    2 ≥ 0

    x 1 ≥ 2

    (4,2) x 1 +x

    2 ≥ 6

    3x 1 -x 2 ≥ 0

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 11 / 91

  • Linear programming Introduction

    Example

    x 2

    x 1

    -x 1 +2x

    2 ≥ 0

    x 1 ≥ 2

    (4,2) x 1 +x

    2 ≥ 6

    3x 1 -x 2 ≥ 0

    If we were maximizing x2 instead of minimizing under some feasible region, the resulting LP would be unbounded.

    For any objective function for which the LP is bounded, there seems to exist an optimal solution which is a “corner” of the feasible section.

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 12 / 91

  • Linear programming The geometry of LP

    Vertices

    Consider an LP in standard form:

    min{cT x : Ax = b, x ≥ 0}

    Let P = {x : Ax = b, x ≥ 0} ⊆ Rn.

    Definition

    x is a vertex of P if @y 6= 0 such that x + y ∈ P and x − y ∈ P.

    Theorem

    Assume min{cT x : x ∈ P} is finite. Then, for all x ∈ P, there exists a vertex x ′ ∈ P such that

    cT x ′ ≤ cT x .

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 13 / 91

  • Linear programming The geometry of LP

    Vertices

    Proof.

    Recall that P = {x : Ax = b, x ≥ 0} ⊆ Rn. If x is a vertex then take x ′ = x .

    If x is not a vertex then, by definition, there exists y 6= 0 such that x + y ∈ P and x − y ∈ P. Then A(x + y) = b and A(x − y) = b, implying that Ay = 0. W.l.o.g. assume cT y ≤ 0 (take either y or −y). If cT y = 0 choose y such that ∃j : yj < 0. Since y 6= 0 and cT y = cT (−y) = 0, this must be true for either y or −y .

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 14 / 91

  • Linear programming The geometry of LP

    Vertices

    Proof (continued). Consider x + λy , λ > 0. Since cT y ≤ 0,

    cT (x + λy) = cT x + λcT y ≤ cT x .

    Case 1: ∃j : yj < 0. As λ increases, component j decreases until x + λy is no longer feasible.

    Choose the largest λ such that x + λy ≥ 0, i.e.,

    λ = min j :yj

  • Linear programming The geometry of LP

    Vertices

    Proof (continued). Case 2: yj ≥ 0 for all j .

    By assumption, cT y < 0.

    Moreover,

    A(x + λy) = Ax + λAy = Ax = b and

    x + λy ≥ x ≥ 0 .

    But cT (x + λy) = cT x + λcT y → −∞ as λ→∞, implying LP is unbounded, a contradiction.

    Case 1 can happen at most n times (the number of components of x). By induction on the number of nonzero components of x , we obtain a vertex x ′. �

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 16 / 91

  • Linear programming The geometry of LP

    Vertices and optimal solutions

    Corollary

    If min{cT x : Ax = b, x ≥ 0} is finite, there exists an optimal solution x∗ which is a vertex.

    Proof. Suppose not. Take an optimal solution. By the previous theorem, there exists a vertex costing no more than the optimal. This vertex must be optimal as well! �

    Corollary

    If P = {x : Ax = b, x ≥ 0} 6= ∅ (i.e., if the LP is feasible), then P has a vertex.

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 17 / 91

  • Linear programming The geometry of LP

    Vertices and optimal solutions

    Corollary

    If min{cT x : Ax = b, x ≥ 0} is finite, there exists an optimal solution x∗ which is a vertex.

    Proof. Suppose not. Take an optimal solution. By the previous theorem, there exists a vertex costing no more than the optimal. This vertex must be optimal as well! �

    Corollary

    If P = {x : Ax = b, x ≥ 0} 6= ∅ (i.e., if the LP is feasible), then P has a vertex.

    Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 17 / 91

  • Linear programming The geometry of LP

    A characterization of vertices

    Theorem

    Let P = {x : Ax = b, x ≥ 0}. For x ∈ P, let Ax be a submatrix of A corresponding to j such that xj > 0. Then x is a vertex iff Ax has linearly independent columns (i.e., Ax has full column rank).

    Example: Let

    A =

     2 1 3 07 3 2 1 0 2 0 5

     , x = 

    2 0 1 0

     . Then Ax corresponds to the positive components of x , j = 1 and j = 3, i.e., to the 1st and 3rd column of A:

    Ax =

     2 37 2 0 0

     , and x is a vertex. Paul G. Spirakis (U. Patras and Liverpool) Foundations -LP- Matcings 18 / 91

  • Linear programming The geometry of LP

    A characterization of vertices

    Pr

Recommended

View more >