introduction to approximation algorithms

Approximation Algorithms

Jhoirene B Clemente

Algorithms and Complexity LabDepartment of Computer Science

University of the Philippines Diliman

October 14, 2014

50

ApproximationAlgorithms

Jhoirene B Clemente

Optimization Problems

Hard CombinatorialOptimization ProblemsClique

Independent Set Problem

Vertex Cover

Approaches in SolvingHard Problems

ApproximationAlgorithmsVertex Cover

Traveling Salesman Problem

References

CS 397October 14, 2014

Overview

1. Combinatorial Optimization Problems2. NP-Hard Problems

2.1 Clique2.2 Independent Set Problem2.3 Vertex Cover2.4 Traveling Salesman Problem

3. Approaches in Solving Hard Problems4. Approximation Algorithms5. Approximable Problems6. Inapproximable Problems

50


Jhoirene B Clemente

3 Optimization Problems



Vertex Cover




References


Optimization Problems[Papadimitriou and Steiglitz, 1998]

Definition (Instance of an Optimization Problem)An instance of an optimization problem is a pair (F , c), where Fis any set, the domain of feasible points; c is the cost function, amapping

c : F → R

The problem is to find an f ∈ F for which

c(f ) ≤ c(y) ∀ y ∈ F

Such a point f is called a globally optimal solution to the giveninstance, or, when no confusion can arise, simply an optimalsolution.

Definition (Optimization Problem)An optimization problem is a set of instances of an optimizationproblem.

50


Jhoirene B Clemente




Vertex Cover




References


Optimization Problems[Papadimitriou and Steiglitz, 1998]

Two categories1. with continuous variables, where we look for a set of real

numbers or a function2. with discrete variables, which we call combinatorial, where

we look for an object from a finite, or possibly countablyinfinite set, typically an integer, set, permutation, or graph.

50


Jhoirene B Clemente




Vertex Cover




References


Combinatorial Optimization Problem

Definition (Combinatorial Optimization Problem[Papadimitriou and Steiglitz, 1998])An optimization problem Π = (DΠ,RΠ, costΠ, goalΠ) consists of1. A set of valid instances DΠ. Let I ∈ DΠ, denote an input

instance.2. Each I ∈ DΠ has a set of feasible solutions, RΠ(I ).3. Objective function, costΠ, that assigns a nonnegative

rational number to each pair (I ,SOL), where I is an instanceand SOL is a feasible solution to I.

4. Either minimization or maximization problem:goalΠ ∈ {min,max}.

50


Jhoirene B Clemente


6 Hard CombinatorialOptimization ProblemsClique


Vertex Cover




References


NP-Hard Problems

1. Clique Problem2. Independent Set Problem3. Vertex Cover Problem4. Traveling Salesman Problem

50


Jhoirene B Clemente


Hard CombinatorialOptimization Problems

7 Clique


Vertex Cover




References


Max Clique Problem

Definition (Max Clique Problem)Given a complete weighted graph, find the largest clique

TheoremMax Clique is NP-hard [Garey and Johnson, 1979].

PropositionThe decision variant of MAX-SAT is NP-Complete[Garey and Johnson, 1979].

50


Jhoirene B Clemente



8 Clique


Vertex Cover




References


Max Clique Reduction from SAT

Example

(a ∨ b ∨ c) ∧ (b ∨ c ∨ d) ∧ (a ∨ c ∨ d) ∧ (a ∨ b ∨ d)

50


Jhoirene B Clemente



9 Clique


Vertex Cover




References


Max Clique Reduction from SAT

50


Jhoirene B Clemente



10 Independent Set Problem

Vertex Cover




References


Independent Set

Definition (Max Independent Set Problem)Given a complete weighted graph, find the largest independent set

TheoremIndependent Set Problem is NP-hard [Garey and Johnson, 1979].

50


Jhoirene B Clemente




11 Vertex Cover




References


Vertex Cover Problem

Definition (Min Vertex Cover Problem)Given a complete weighted graph, find the minimum vertex cover.

TheoremVertex Cover Problem is NP-hard [Garey and Johnson, 1979].

50


Jhoirene B Clemente




12 Vertex Cover




References


Traveling Salesman Problem (TSP)

DefinitionINPUT: Edge weighted Graph G = (V ,E)OUTPUT: Minimum cost Hamiltonian Cycle.

50


Jhoirene B Clemente




Vertex Cover

13 Approaches in SolvingHard Problems



References


Approaches in Solving NP-hard Problems

I Exact algorithm always obtain the optimal solutionI Approximation algorithm settle for good enough solutions.

Goodness of solution is guaranteed and measured usingapproximation ratio.

I Heuristic algorithms produce solutions, which are notguaranteed to be close to the optimum. The performance ofheuristics is often evaluated empirically.

50


Jhoirene B Clemente




Vertex Cover

14 Approaches in SolvingHard Problems



References


Approaches in Solving NP-hard Problems

Design technique have a well specified structure that evenprovides a framework for possible implementations

I Dynamic ProgrammingI Greedy SchemaI Divide and conquerI Branch and BoundI Local searchI . . .

Concepts formulate ideas and rough frameworks about how toattack hard algorithmic problems .

I Approximation AlgorithmsI Parameterized AlgorithmsI Randomized AlgorithmsI . . .

50


Jhoirene B Clemente




Vertex Cover


15 ApproximationAlgorithmsVertex Cover


References


Approximation Algorithms

Definition (Approximation Algorithm[Williamson and Shmoy, 2010] )An ρ-approximation algorithm for an optimization problem is apolynomial-time algorithm that for all instances of the problemproduces a solution whose value is within a factor of ρ of thevalue of an optimal solution.Given an problem instance I with an optimal solution Opt(I ), i.e.the cost function cost(Opt(I )) is minimum/maximum.

I An algorithm for a minimization problem is calledρ-approximative algorithm for some ρ > 1, if the algorithmobtains a maximum cost of ρ · cost(Opt(I )), for any inputinstance I .

I An algorithm for a maximization problem is calledρ-approximative algorithm, for some ρ < 1, if the algorithmobtains a minimum cost of ρ · cost(Opt(I )), for any inputinstance I .

50


Jhoirene B Clemente




Vertex Cover




References


Approximation Ratio

I Minimization, ρ > 1

cost(Opt(I )) ≤ cost(SOL) ≤ ρ cost(Opt(I ))

I Maximization, ρ < 1

ρ cost(Opt(I )) ≤ cost(SOL) ≤ cost(Opt(I ))

50


Jhoirene B Clemente




Vertex Cover




References


Approximation Ratio

I Additive Approximation Algorithms:

SOL ≤ OPT + c

I Constant Approximation Algorithms:

SOL ≤ c ·OPT

I Logarithmic Approximation Algorithms:

SOL = O(log n) ·OPT

I Polynomial Approximation Algorithms:

SOL = O(nc) ·OPT ,

where c ≤ 1

50


Jhoirene B Clemente




Vertex Cover



18 Vertex Cover


References


Example: Vertex Cover Problem

Definition (Vertex Cover [Vazirani, 2001] )Given a graph G = (V ,E), a vertex cover is a subset C ⊂ Vsuch that every edge has at least one end point incident at C .

Definition (Minimum Vertex Cover Problem)Given a complete weighted graph G = (V ,E), find a minimumcardinality vertex cover C .

50


Jhoirene B Clemente




Vertex Cover



19 Vertex Cover


References


Maximal Matching

Definition (Matchings)Given a graph G = (V ,E), a subset M ⊂ E is called a matchingif no two edges in M are adjacent in G.

Figure : (a) Maximal matching (b) Perfect matching

50


Jhoirene B Clemente




Vertex Cover



20 Vertex Cover


References


2-Approximation Algorithm

1. Find a maximal matching in G, and output the set ofmatched vertices.

Algorithm 1: 2-Approximation Algorithm for minimum vectorcover problem

50


Jhoirene B Clemente




Vertex Cover



21 Vertex Cover


References



TheoremAlgorithm 1 is a 2-approximation algorithm.

Proof.Let M be a maximal matching in G = (V ,E) and OPT be theminimum vertex cover in G, then

|M | ≤ |OPT |

The cover picked by the algorithm has cardinality |SOL| ≤ 2 · |M |.

|SOL| ≤ 2 · |OPT |

50


Jhoirene B Clemente




Vertex Cover



22 Vertex Cover


References



Example:

50


Jhoirene B Clemente




Vertex Cover



23 Traveling Salesman Problem

References


Solving Traveling Salesman Problem

INPUT: Edge weighted graph1. Compute a Minimum Spanning Tree for G2. Select a vertex r ∈ V (G) to be the root vertex3. Let L be the list of vertices visited in a preorder walk

OUTPUT: Hamiltonian Cycle that visitsA preorder tree walk recursively visits every vertex in thetree, listing a vertex when it is first encountered , beforeany of its children are visited.

50


Jhoirene B Clemente




Vertex Cover




References


Computing the Minimum Spanning Tree

50


Jhoirene B Clemente




Vertex Cover




References



50


Jhoirene B Clemente




Vertex Cover




References


Preorder

50


Jhoirene B Clemente




Vertex Cover




References


Hamiltonian Cycle from the Preorder Walk

50


Jhoirene B Clemente




Vertex Cover




References


Hamiltonian Cycle from the Preorder Walk

4 + 4 + 10 + 15 + 10 + 16 = 59

50


Jhoirene B Clemente




Vertex Cover




References


Total tour cost vs. the optimal cost

Let H ∗ denote an optimal Hamiltonian tour in G. Let T be theminimum spanning tree of G.

c(T ) ≤ c(H ∗)

Let W be the full length cost of the preorder walk.

c(W ) = 2c(T )

Let H be the output Hamiltonian Cycle from the algorithm

c(W ) ≥ c(H )

Thenc(H ) ≤ 2c(H ∗)

Approximation ratio (ρ) is 2

50


Jhoirene B Clemente




Vertex Cover




References


Approximable Problems [Vazirani, 2001]

Definition (APX)I An abbreviation for “Approximable", is the set of NP

optimization problems that allow polynomial-timeapproximation algorithms with approximation ratio boundedby a constant.

I Problems in this class have efficient algorithms that can findan answer within some fixed percentage of the optimalanswer.

50


Jhoirene B Clemente




Vertex Cover




References


Polynomial-time Approximation Schemes

Definition (PTAS [Aaronson et al., 2008])The subclass of NPO problems that admit an approximationscheme in the following sense.For any ε > 0, there is a polynomial-time algorithm that isguaranteed to find a solution whose cost is within a 1 + ε factorof the optimum cost. Contains FPTAS, and is contained in APX.

Definition (FPTAS [Aaronson et al., 2008] )The subclass of NPO problems that admit an approximationscheme in the following sense.For any ε > 0, there is an algorithm that is guaranteed to find asolution whose cost is within a 1 + ε factor of the optimum cost.Furthermore, the running time of the algorithm is polynomial in n(the size of the problem) and in 1/ε.

50


Jhoirene B Clemente




Vertex Cover




References


Approximation Ratio of well known HardProblems

1. FPTAS: Bin Packing Problem2. PTAS: Makespan Scheduling Problem3. APX:

3.1 Min Steiner Tree Problem (1.55 Approximable) [Robins,2005]3.2 Min Metric TSP (3/2 Approximable) [Christofides,1977]3.3 Max SAT (0.77 Approximable) [Asano,1997]3.4 Vertex Cover (2 Approximable) [Vazirani,2001]

4. MAX SNP4.1 Independent Set Problem4.2 Clique Problem4.3 Travelling Salesman Problem

50


Jhoirene B Clemente




Vertex Cover




References


Complexity Classes [Ausiello et al., 2011]

FPTAS ( PTAS ( APX ( NPO

50


Jhoirene B Clemente




Vertex Cover




References


Inapproximable Problems

Many problems have polynomial-time approximation schemes.However, there exists a class of problems that is not so easy[Williamson and Shmoy, 2010]. This class is called MAX SNP.

TheoremFor any MAXSNP-hard problem, there does not exist apolynomial-time approximation scheme, unless P = NP[Williamson and Shmoy, 2010].

TheoremIf P 6= NP, then for any constant ρ ≥ 1, there is nopolynomial-time approximation algorithm with approximationratio ρ for the general travelling salesman problem.

50


Jhoirene B Clemente




Vertex Cover




50 References


References I

Aaronson, S., Kuperberg, G., and Granade, C. (2008).The complexity zoo.

Ausiello, G., Bonifaci, V., and Escoffier, B. (2011).Complexity and approximation in reoptimization.In Computability in Context: Computation and Logic in the Real World,volume 2, pages 101–129.

Garey, M. R. and Johnson, D. S. (1979).Computers and Intractability: A Guide to the Theory of NP-Completeness.

Papadimitriou, C. and Steiglitz, K. (1998).Combinatorial optimization: algorithms and complexity.

Vazirani, V. V. (2001).Approximation algorithms, volume 1997.

Williamson, D. and Shmoy, D. (2010).The Design of Approximation Algorithms.

introduction to approximation algorithms

Science

set of instances

countablyinfinite set

set of realnumbers

set of feasible solutions

independent set problem2

independent set problem3

set of valid instances

pair f