# Introduction to Algorithms - University of Florida ? Introduction to Algorithms ... Algorithms . My

Post on 27-Jun-2018

213 views

TRANSCRIPT

Introduction to Algorithms NP-Completeness and Approximation Algorithms My T. Thai mythai@cise.ufl.edu 2 Why Approximation Algorithms Problems that we cannot find an optimal solution in a polynomial time Eg: Set Cover, Bin Packing Need to find a near-optimal solution: Heuristic Approximation algorithms: This gives us a guarantee approximation ratio My T. Thai mythai@cise.ufl.edu 3 Why important Your advisers/bosses give you a computationally hard problem. Here are two scenarios: No knowledge about approximation: Spend a few months looking for an optimal solution Come to their office and confess that you cannot do it Get fired Knowledge about approximation: My T. Thai mythai@cise.ufl.edu 4 Knowledge about approximation Show your boss that this is a NP-complete (NP-hard) problem There does not exist any polynomial time algorithm to find an exact solution Propose a good algorithm (either heuristic or approximation) to find a near-optimal solution Better yet, prove the approximation ratio My T. Thai mythai@cise.ufl.edu 5 Techniques A variety of techniques to design and analyze many approximation algorithms for computationally hard problems: Combinatorial algorithms: Greedy Techniques, Independent System, Submodular Function Linear Programming based algorithms Semidefinite Programming based algorithms In this class, we just introduce a few examples My T. Thai mythai@cise.ufl.edu 6 Combinatorial Optimization The study of finding the best object from within some finite space of objects, eg: Shortest path: Given a graph with edge costs and a pair of nodes, find the shortest path (least costs) between them Traveling salesman: Given a complete graph with nonnegative edge costs, find a minimum cost cycle visiting every vertex exactly once Maximum Network Lifetime: Given a wireless sensor networks and a set of targets, find a schedule of these sensors to maximize network lifetime My T. Thai mythai@cise.ufl.edu 7 In P or not in P? Informal Definitions: The class P consists of those problems that are solvable in polynomial time, i.e. O(nk) for some constant k where n is the size of the input. The class NP consists of those problems that are verifiable in polynomial time: Given a certificate of a solution, then we can verify that the certificate is correct in polynomial time My T. Thai mythai@cise.ufl.edu 8 In P or not in P: Examples In P: Shortest path Minimum Spanning Tree Not in P (NP): Vertex Cover Traveling salesman Minimum Connected Dominating Set My T. Thai mythai@cise.ufl.edu 9 NP-completeness (NPC) A problem is in the class NPC if it is in NP and is as hard as any problem in NP My T. Thai mythai@cise.ufl.edu 10 What is hard Decision Problems: Only consider YES-NO Decision version of TSP: Given n cities, is there a TSP tour of length at most L? Why Decision Problem? What relation between the optimization problem and its decision? Decision is not harder than that of its optimization problem If the decision is hard, then the optimization problem should be hard My T. Thai mythai@cise.ufl.edu 11 NP-complete and NP-hard A language L is NP-complete if: 1. L is in NP, and 2. For every L in NP, L can be polynomial-time reducible to L My T. Thai mythai@cise.ufl.edu 12 Approximation Algorithms An algorithm that returns near-optimal solutions in polynomial time Approximation Ratio (n): Define: C* as a optimal solution and C is the solution produced by the approximation algorithm max (C/C*, C*/C) My T. Thai mythai@cise.ufl.edu 13 Approximation Algorithms (cont) PTAS (Polynomial Time Approximation Scheme): A (1 + )-approximation algorithm for a NP-hard optimization where its running time is bounded by a polynomial in the size of instance I FPTAS (Fully PTAS): The same as above + time is bounded by a polynomial in both the size of instance I and 1/ My T. Thai mythai@cise.ufl.edu 14 A Dilemma! We cannot find C*, how can we compare C to C*? How can we design an algorithm so that we can compare C to C* It is the objective of this course!!! My T. Thai mythai@cise.ufl.edu 15 Vertex Cover Definition: An Example 'at incident endpoint oneleastat has , edgeevery for iff ofcover vertex acalled is 'subset a ,),(graph undirectedan Given VeEeGVVEVGMy T. Thai mythai@cise.ufl.edu 16 Vertex Cover Problem Definition: Given an undirected graph G=(V,E), find a vertex cover with minimum size (has the least number of vertices) This is sometimes called cardinality vertex cover More generalization: Given an undirected graph G=(V,E), and a cost function on vertices c: V Q+ Find a minimum cost vertex cover My T. Thai mythai@cise.ufl.edu 17 How to solve it Matching: A set M of edges in a graph G is called a matching of G if no two edges in set M have an endpoint in common Example: My T. Thai mythai@cise.ufl.edu 18 How to solve it (cont) Maximum Matching: A matching of G with the greatest number of edges Maximal Matching: A matching which is not contained in any larger matching Note: Any maximum matching is certainly maximal, but not the reverse My T. Thai mythai@cise.ufl.edu 19 Main Observation No vertex can cover two edges of a matching The size of every vertex cover is at least the size of every matching: |M| |C| |M| = |C| indicates the optimality Possible Solution: Using Maximal matching to find Minimum vertex cover My T. Thai mythai@cise.ufl.edu 20 An Algorithm Alg 1: Find a maximal matching in G and output the set of matched vertices Theorem: The algorithm 1 is a factor 2-approximation algorithm. Proof: optCMCoptMCopt2|| Therefore,||2|| and ||:have We1. algorithm from obtainedcover vertex ofset a be Let solution. optimalan of size a be Let My T. Thai mythai@cise.ufl.edu 21 Can Alg1 be improved? Q1: Can the approximation ratio of Alg 1 be improved by a better analysis? Q2: Can we design a better approximation algorithm using the similar technique (maximal matching)? Q3: Is there any other lower bounding method that can lead to a better approximation algorithm? My T. Thai mythai@cise.ufl.edu 22 Answers A1: No by considering the complete bipartite graphs Kn,n A2: No by considering the complete graph Kn where n is odd. |M| = (n-1)/2 whereas opt = n -1 My T. Thai mythai@cise.ufl.edu 23 Answers (cont) A3: Currently a central open problem Yes, we can obtain a better one by using the semidefinite programming Generalization vertex Cover Can we still able to design a 2-approximation algorithm? Homework Excersice My T. Thai mythai@cise.ufl.edu 24 Set Cover Definition: Given a universe U of n elements, a collection of subsets of U, S = {S1, , Sm}, and a cost function c: S Q+, find a minimum cost subcollection C of S that covers all elements of U. Example: U = {1, 2, 3, 4, 5} S1 = {1, 2, 3}, S2 = {2,3}, S3 = {4, 5}, S4 = {1, 2, 4} c1 = c2 = c3 = c4 = 1 Solution C = {S1, S3} If the cost is uniform, then the set cover problem asks us to find a subcollection covering all elements of U with the minimum size. My T. Thai mythai@cise.ufl.edu 25 An Example My T. Thai mythai@cise.ufl.edu 26 INSTANCE: Given a universe U of n elements, a collection of subsets of U, S = {S1, , Sm}, and a positive integer b QUESTION: Is there a , |C| b, such that (Note: The subcollection {Si | } satisfying the above condition is called a set cover of U NP-completeness Theorem: Set Cover (SC) is NP-complete Proof: My T. Thai mythai@cise.ufl.edu 27 Proof (cont) First we need to show that SC is in NP. Given a collection of sets C, it is easy to verify that if |C| b and the union of all sets listed in C does include all elements in U. To complete the proof, we need to show that Vertex Cover (VC) p Set Cover (SC) Given an instance C of VC (an undirected graph G=(V,E) and a positive integer j), we need to construct an instance C of SC in polynomial time such that C is satisfiable iff C is satisfiable. My T. Thai mythai@cise.ufl.edu 28 Proof (cont) Construction: Let U = E. We will define n elements of U and a collection S as follows: Note: Each edge corresponds to each element in U and each vertex corresponds to each set in S. Label all vertices in V from 1 to n. Let Si be the set of all edges that incident to vertex i. Finally, let b = j. This construction is in poly-time with respect to the size of VC instance. My T. Thai mythai@cise.ufl.edu 29 VERTEX-COVER p SET-COVER one set for every vertex, containing the edges it covers VC one element for every edge VC SC My T. Thai mythai@cise.ufl.edu 30 Proof (cont) Now, we need to prove that C is satisfiable iff C is. That is, we need to show that if the original instance of VC is a yes instance iff the constructed instance of SC is a yes instance. () Suppose G has a vertex cover of size at most j, called C. By our construction, C corresponds to a collection C of subsets of U. Since b = j, |C| b. Plus, C covers all elements in U since C covers all edges in G. To see this, consider any element of U. Such an element is an edge in G. Since C is a vertex cover, at least endpoint of this edge is in C. My T. Thai mythai@cise.ufl.edu 31 () Suppose there is a set cover C of size at most b in our constructed instance. Since each set in C is associated with a vertex in G, let C be the set of these vertices. Then |C| = |C| b = j. Plus, C is a vertex cover of G since C is a set cover. To see this, consider any edge e. Since e is in U, C must contain at least one set that includes e. By our construction, the only set that include e correspond to nodes that are endpoints of e. Thus C must contain at least one endpoints of e. My T. Thai mythai@cise.ufl.edu 32 Solutions Algorithm 1: (in the case of uniform cost) 1: C = empty 2: while U is not empty 3: pick a set Si such that Si covers the most elements in U 4: remove the new covered elements from U 5: C = C union Si 6: return C My T. Thai mythai@cise.ufl.edu 33 Solutions (cont) In the case of non-uniform cost Similar method. At each iteration, instead of picking a set Si such that Si covers the most uncovered elements, we will pick a set Si whose cost-effectiveness is smallest, where is defined as: Questions: Why choose smallest ? Why define as above ||/)( USSc iiMy T. Thai mythai@cise.ufl.edu 34 Solutions (cont) Algorithm 2: (in the case of non-uniform cost) 1: C = empty 2: while U is not empty 3: pick a set Si such that Si has the smallest 4: for each new covered elements e in U 5: set price(e) = 6: remove the new covered elements from U 7: C = C union Si 8: return C My T. Thai mythai@cise.ufl.edu 35 Approximation Ratio Analysis Let ek, k = 1n, be the elements of U in the order in which they were covered by Alg 2. We have: Lemma 1: Proof: Let Uj be the set the remaining set U at iteration j. That is, Uj contains all the uncovered elements at iteration j. At any iteration j, the leftover sets of the optimal solution can cover the remaining elements at a cost of at most opt. (Why?) solution optimal theofcost theis where)1/()(price },,...,1{each For optknoptenk kMy T. Thai mythai@cise.ufl.edu 36 Proof of Lemma 1 (cont) Thus, among these leftover sets, there must exist one set where its value is at most opt/|Uj|. Let ek be the element covered at this iteration, |Uj| n k + 1. Since ek was covered by the most cost-effective set in this iteration, we have: price(ek) opt/|Uj| opt/(n-k+1) My T. Thai mythai@cise.ufl.edu 37 Approximation Ratio Theorem 1: The set cover obtained form Alg 2 (also Alg 1) has a factor of Hn where Hn is a harmonic series Hn = 1 + 1/2 + + 1/n Proof: It follows directly from Lemma 1 Examples of NP-complete problems Independent set - independent set: a set of vertices in the graph with no edges between each pair of nodes. - given a graph G=(V,E), find the largest independent set - reduction from vertex cover: smallest vertex cover S largest independent set V/S Independent Set Independent set - if G has a vertex cover S, then V \ S is an independent set proof: consider two nodes in V \ S: if there is an edge connecting them, then one of them must be in S, which means one of them is not in V \ S - if G has an independent set I, then V \ I is a vertex cover proof: consider one edge in G: if it is not covered by any node in V \ I, then its two end vertices must be both in I, which means I is not an independent set Steiner Tree Steiner tree - given a graph G=(V,E), and a subset C of V - find the minimum tree to connect each vertex in C - reduction Vertex cover for Graph G Steiner tree for graph G vertex (u,v)edge(u,v)-uedge(u,v)-vG (only edges with distance 1 are shown) G Steiner Tree Construction - G is a complete graph - for every node u in G, create a node u in G - for every edge (u,v) in G, create a node (u,v) in G - in G, every node (u,v) is connected to u and v with distance 1 - in G, every node u and v is connected with distance 1 - other edges in G are of distance 2 vertex (u,v)edge(u,v)-uedge(u,v)-vG (only edges with distance 1 are shown) G Sketch of Proof: - in the Steiner tree problem for G, choose C to be the set of all nodes (u,v) - G has a minimum Steiner tree of cost m+k-1 iff G has a minimum vertex cover of size k vertex (u,v)edge(u,v)-uedge(u,v)-vG (only edges with distance 1 are shown) G Summary of some NPc problems SAT 3SAT Vertex cover Independent set Set cover Graph coloring Maximum clique size Minimum Steiner tree Hamiltonian cycle Maximum cut find more NP-complete problems in http://en.wikipedia.org/wiki/List_of_NP-complete_problems Approximation Algorithms Bin Packing My T. Thai mythai@cise.ufl.edu 45 Definition My T. Thai mythai@cise.ufl.edu 46 Theorem My T. Thai mythai@cise.ufl.edu 47 An Easy Solution: First Fit My T. Thai mythai@cise.ufl.edu 48 Asymptotic PTAS My T. Thai mythai@cise.ufl.edu 49 My T. Thai mythai@cise.ufl.edu 50 Approximation Algorithms k-Center My T. Thai mythai@cise.ufl.edu 52 Definition My T. Thai mythai@cise.ufl.edu 53 2-Approx My T. Thai mythai@cise.ufl.edu 54 My T. Thai mythai@cise.ufl.edu 55 My T. Thai mythai@cise.ufl.edu 56 Analysis My T. Thai mythai@cise.ufl.edu 57 Obtain Better Ratio? My T. Thai mythai@cise.ufl.edu 58 Weighted k-Center My T. Thai mythai@cise.ufl.edu 59 Notations My T. Thai mythai@cise.ufl.edu 60 Algorithm My T. Thai mythai@cise.ufl.edu 61 Algorithm 2 My T. Thai mythai@cise.ufl.edu 62 Analysis Theorem: Alg 2 has an approx ratio of 3. Approximation Algorithms Multiway Cut and k-Cut My T. Thai mythai@cise.ufl.edu 63 Definition My T. Thai mythai@cise.ufl.edu 64 Example My T. Thai mythai@cise.ufl.edu 65 My T. Thai mythai@cise.ufl.edu 66 Can you prove it??? Approximation My T. Thai mythai@cise.ufl.edu 67 Analysis My T. Thai mythai@cise.ufl.edu 68 Minimum k-cut Problem My T. Thai mythai@cise.ufl.edu 69 Example My T. Thai mythai@cise.ufl.edu 70 Construct a Gomory-Hu Tree My T. Thai mythai@cise.ufl.edu 71 My T. Thai mythai@cise.ufl.edu 72 Properties of Gomory-Hu Tree wrt k-cut My T. Thai mythai@cise.ufl.edu 73 Approximation Algorithm My T. Thai mythai@cise.ufl.edu 74 Analysis My T. Thai mythai@cise.ufl.edu 75

Recommended

Introduction to Exponential Time Algorithms - sminaire ? Introduction to Exponential Time Algorithms

Introduction to Algorithms - Massachusetts Institute ? Day 1 Introduction to Algorithms L1.2 Welcome