Introduction to Algorithms - University of Florida ?· Introduction to Algorithms ... Algorithms . My…

Download Introduction to Algorithms - University of Florida ?· Introduction to Algorithms ... Algorithms . My…

Post on 27-Jun-2018




1 download

Embed Size (px)


<ul><li><p>Introduction to Algorithms </p><p>NP-Completeness and Approximation </p><p>Algorithms </p></li><li><p>My T. Thai </p><p> </p><p>2 </p><p>Why Approximation Algorithms </p><p> Problems that we cannot find an optimal </p><p>solution in a polynomial time </p><p> Eg: Set Cover, Bin Packing </p><p> Need to find a near-optimal solution: </p><p> Heuristic </p><p> Approximation algorithms: </p><p> This gives us a guarantee approximation ratio </p></li><li><p>My T. Thai </p><p> </p><p>3 </p><p>Why important </p><p> Your advisers/bosses give you a </p><p>computationally hard problem. Here are two </p><p>scenarios: </p><p> No knowledge about approximation: </p><p> Spend a few months looking for an optimal solution </p><p> Come to their office and confess that you cannot do it </p><p> Get fired </p><p> Knowledge about approximation: </p></li><li><p>My T. Thai </p><p> </p><p>4 </p><p> Knowledge about approximation </p><p> Show your boss that this is a NP-complete (NP-</p><p>hard) problem </p><p> There does not exist any polynomial time algorithm </p><p>to find an exact solution </p><p> Propose a good algorithm (either heuristic or </p><p>approximation) to find a near-optimal solution </p><p> Better yet, prove the approximation ratio </p></li><li><p>My T. Thai </p><p> </p><p>5 </p><p>Techniques </p><p> A variety of techniques to design and analyze </p><p>many approximation algorithms for </p><p>computationally hard problems: </p><p> Combinatorial algorithms: </p><p> Greedy Techniques, Independent System, Submodular </p><p>Function </p><p> Linear Programming based algorithms </p><p> Semidefinite Programming based algorithms </p><p> In this class, we just introduce a few examples </p></li><li><p>My T. Thai </p><p> </p><p>6 </p><p>Combinatorial Optimization </p><p> The study of finding the best object from </p><p>within some finite space of objects, eg: </p><p> Shortest path: Given a graph with edge costs and a </p><p>pair of nodes, find the shortest path (least costs) </p><p>between them </p><p> Traveling salesman: Given a complete graph with </p><p>nonnegative edge costs, find a minimum cost cycle </p><p>visiting every vertex exactly once </p><p> Maximum Network Lifetime: Given a wireless </p><p>sensor networks and a set of targets, find a schedule </p><p>of these sensors to maximize network lifetime </p></li><li><p>My T. Thai </p><p> </p><p>7 </p><p>In P or not in P? </p><p>Informal Definitions: </p><p> The class P consists of those problems that are </p><p>solvable in polynomial time, i.e. O(nk) for some </p><p>constant k where n is the size of the input. </p><p> The class NP consists of those problems that </p><p>are verifiable in polynomial time: </p><p> Given a certificate of a solution, then we can verify </p><p>that the certificate is correct in polynomial time </p></li><li><p>My T. Thai </p><p> </p><p>8 </p><p>In P or not in P: Examples </p><p> In P: </p><p> Shortest path </p><p> Minimum Spanning Tree </p><p> Not in P (NP): </p><p> Vertex Cover </p><p> Traveling salesman </p><p> Minimum Connected Dominating Set </p></li><li><p>My T. Thai </p><p> </p><p>9 </p><p>NP-completeness (NPC) </p><p> A problem is in the class NPC if it is in NP and </p><p>is as hard as any problem in NP </p></li><li><p>My T. Thai </p><p> </p><p>10 </p><p>What is hard </p><p> Decision Problems: Only consider YES-NO </p><p> Decision version of TSP: Given n cities, is there a </p><p>TSP tour of length at most L? </p><p> Why Decision Problem? What relation between </p><p>the optimization problem and its decision? </p><p> Decision is not harder than that of its </p><p>optimization problem </p><p> If the decision is hard, then the optimization </p><p>problem should be hard </p></li><li><p>My T. Thai </p><p> </p><p>11 </p><p>NP-complete and NP-hard </p><p>A language L is NP-complete if: </p><p>1. L is in NP, and </p><p>2. For every L in NP, L can be polynomial-time </p><p>reducible to L </p></li><li><p>My T. Thai </p><p> </p><p>12 </p><p>Approximation Algorithms </p><p> An algorithm that returns near-optimal </p><p>solutions in polynomial time </p><p> Approximation Ratio (n): </p><p> Define: C* as a optimal solution and C is the </p><p>solution produced by the approximation algorithm </p><p> max (C/C*, C*/C) </p></li><li><p>My T. Thai </p><p> </p><p>13 </p><p>Approximation Algorithms (cont) </p><p> PTAS (Polynomial Time Approximation </p><p>Scheme): A (1 + )-approximation algorithm </p><p>for a NP-hard optimization where its running </p><p>time is bounded by a polynomial in the size of </p><p>instance I </p><p> FPTAS (Fully PTAS): The same as above + </p><p>time is bounded by a polynomial in both the </p><p>size of instance I and 1/ </p></li><li><p>My T. Thai </p><p> </p><p>14 </p><p>A Dilemma! </p><p> We cannot find C*, how can we compare C to </p><p>C*? </p><p> How can we design an algorithm so that we can </p><p>compare C to C* </p><p>It is the objective of this course!!! </p></li><li><p>My T. Thai </p><p> </p><p>15 </p><p>Vertex Cover </p><p> Definition: </p><p> An Example </p><p>'at incident endpoint one</p><p>leastat has , edgeevery for iff ofcover vertex a</p><p>called is 'subset a ,),(graph undirectedan Given </p><p>V</p><p>eEeG</p><p>VVEVG</p></li><li><p>My T. Thai </p><p> </p><p>16 </p><p>Vertex Cover Problem </p><p> Definition: </p><p> Given an undirected graph G=(V,E), find a vertex </p><p>cover with minimum size (has the least number of </p><p>vertices) </p><p> This is sometimes called cardinality vertex cover </p><p> More generalization: </p><p> Given an undirected graph G=(V,E), and a cost </p><p>function on vertices c: V Q+ </p><p> Find a minimum cost vertex cover </p></li><li><p>My T. Thai </p><p> </p><p>17 </p><p>How to solve it </p><p> Matching: </p><p> A set M of edges in a graph G is called a matching </p><p>of G if no two edges in set M have an endpoint in </p><p>common </p><p> Example: </p></li><li><p>My T. Thai </p><p> </p><p>18 </p><p>How to solve it (cont) </p><p> Maximum Matching: </p><p> A matching of G with the greatest number of edges </p><p> Maximal Matching: </p><p> A matching which is not contained in any larger </p><p>matching </p><p> Note: Any maximum matching is certainly </p><p>maximal, but not the reverse </p></li><li><p>My T. Thai </p><p> </p><p>19 </p><p>Main Observation </p><p> No vertex can cover two edges of a matching </p><p> The size of every vertex cover is at least the </p><p>size of every matching: |M| |C| </p><p> |M| = |C| indicates the optimality </p><p> Possible Solution: Using Maximal matching to </p><p>find Minimum vertex cover </p></li><li><p>My T. Thai </p><p> </p><p>20 </p><p>An Algorithm </p><p> Alg 1: Find a maximal matching in G and output the </p><p>set of matched vertices </p><p> Theorem: The algorithm 1 is a factor 2-approximation </p><p>algorithm. </p><p> Proof: </p><p> optC</p><p>MCoptM</p><p>Copt</p><p>2|| Therefore,</p><p>||2|| and ||</p><p>:have We1. algorithm from obtainedcover vertex </p><p>ofset a be Let solution. optimalan of size a be Let </p></li><li><p>My T. Thai </p><p> </p><p>21 </p><p>Can Alg1 be improved? </p><p> Q1: Can the approximation ratio of Alg 1 be </p><p>improved by a better analysis? </p><p> Q2: Can we design a better approximation </p><p>algorithm using the similar technique (maximal </p><p>matching)? </p><p> Q3: Is there any other lower bounding method </p><p>that can lead to a better approximation </p><p>algorithm? </p></li><li><p>My T. Thai </p><p> </p><p>22 </p><p>Answers </p><p> A1: No by considering the complete bipartite </p><p>graphs Kn,n </p><p> A2: No by considering the complete graph Kn </p><p>where n is odd. </p><p> |M| = (n-1)/2 whereas opt = n -1 </p></li><li><p>My T. Thai </p><p> </p><p>23 </p><p>Answers (cont) </p><p> A3: </p><p> Currently a central open problem </p><p> Yes, we can obtain a better one by using the </p><p>semidefinite programming </p><p> Generalization vertex Cover </p><p> Can we still able to design a 2-approximation </p><p>algorithm? </p><p> Homework Excersice </p></li><li><p>My T. Thai </p><p> </p><p>24 </p><p>Set Cover </p><p> Definition: Given a universe U of n elements, a collection of subsets of U, S = {S1, , Sm}, and a cost function c: S Q+, find a minimum cost subcollection C of S that covers all elements of U. </p><p> Example: </p><p> U = {1, 2, 3, 4, 5} </p><p> S1 = {1, 2, 3}, S2 = {2,3}, S3 = {4, 5}, S4 = {1, 2, 4} </p><p> c1 = c2 = c3 = c4 = 1 Solution C = {S1, S3} </p><p> If the cost is uniform, then the set cover problem asks us to find a subcollection covering all elements of U with the minimum size. </p></li><li><p>My T. Thai </p><p> </p><p>25 </p><p>An Example </p></li><li><p>My T. Thai </p><p> </p><p>26 </p><p>INSTANCE: Given a universe U of n elements, a collection of subsets of U, S = {S1, , Sm}, and a positive integer b </p><p>QUESTION: Is there a , |C| b, such that </p><p>(Note: The subcollection {Si | } satisfying the above condition is called a set cover of U </p><p>NP-completeness </p><p> Theorem: Set Cover (SC) is NP-complete </p><p> Proof: </p></li><li><p>My T. Thai </p><p> </p><p>27 </p><p>Proof (cont) </p><p> First we need to show that SC is in NP. Given a collection of sets C, it is easy to verify that if |C| b and the union of all sets listed in C does include all elements in U. </p><p> To complete the proof, we need to show that Vertex Cover (VC) p Set Cover (SC) </p><p>Given an instance C of VC (an undirected graph G=(V,E) and a positive integer j), we need to construct an instance C of SC in polynomial time such that C is satisfiable iff C is satisfiable. </p></li><li><p>My T. Thai </p><p> </p><p>28 </p><p>Proof (cont) </p><p>Construction: Let U = E. We will define n elements of U and a collection S as follows: </p><p>Note: Each edge corresponds to each element in U and each vertex corresponds to each set in S. </p><p>Label all vertices in V from 1 to n. Let Si be the set of all edges that incident to vertex i. Finally, let b = j. This construction is in poly-time with respect to the size of VC instance. </p></li><li><p>My T. Thai </p><p> </p><p>29 </p><p>VERTEX-COVER p SET-COVER </p><p>one set for every vertex, containing the edges it covers </p><p>VC </p><p>one element for every edge </p><p>VC SC </p></li><li><p>My T. Thai </p><p> </p><p>30 </p><p>Proof (cont) </p><p>Now, we need to prove that C is satisfiable iff C is. </p><p>That is, we need to show that if the original instance </p><p>of VC is a yes instance iff the constructed instance of </p><p>SC is a yes instance. </p><p> () Suppose G has a vertex cover of size at most j, called C. By our construction, C corresponds to a collection C of </p><p>subsets of U. Since b = j, |C| b. Plus, C covers all </p><p>elements in U since C covers all edges in G. To see this, </p><p>consider any element of U. Such an element is an edge in </p><p>G. Since C is a vertex cover, at least endpoint of this edge is </p><p>in C. </p></li><li><p>My T. Thai </p><p> </p><p>31 </p><p> () Suppose there is a set cover C of size at most b in our constructed instance. Since each set in C is associated with a vertex in G, let C be the set of these vertices. Then |C| = |C| b = j. Plus, C is a vertex cover of G since C is a set cover. To see this, consider any edge e. Since e is in U, C must contain at least one set that includes e. By our construction, the only set that include e correspond to nodes that are endpoints of e. Thus C must contain at least one endpoints of e. </p></li><li><p>My T. Thai </p><p> </p><p>32 </p><p>Solutions </p><p>Algorithm 1: (in the case of uniform cost) </p><p>1: C = empty </p><p>2: while U is not empty </p><p>3: pick a set Si such that Si covers the most </p><p>elements in U </p><p>4: remove the new covered elements from U </p><p>5: C = C union Si </p><p>6: return C </p></li><li><p>My T. Thai </p><p> </p><p>33 </p><p>Solutions (cont) </p><p> In the case of non-uniform cost </p><p> Similar method. At each iteration, instead of picking a </p><p>set Si such that Si covers the most uncovered elements, </p><p>we will pick a set Si whose cost-effectiveness is </p><p>smallest, where is defined as: </p><p> Questions: Why choose smallest ? Why define as </p><p>above </p><p>||/)( USSc ii</p></li><li><p>My T. Thai </p><p> </p><p>34 </p><p>Solutions (cont) </p><p>Algorithm 2: (in the case of non-uniform cost) </p><p>1: C = empty </p><p>2: while U is not empty </p><p>3: pick a set Si such that Si has the smallest </p><p>4: for each new covered elements e in U </p><p>5: set price(e) = </p><p>6: remove the new covered elements from U </p><p>7: C = C union Si </p><p>8: return C </p></li><li><p>My T. Thai </p><p> </p><p>35 </p><p>Approximation Ratio Analysis </p><p>Let ek, k = 1n, be the elements of U in the order in which </p><p>they were covered by Alg 2. We have: </p><p> Lemma 1: </p><p> Proof: Let Uj be the set the remaining set U at iteration j. </p><p>That is, Uj contains all the uncovered elements at </p><p>iteration j. At any iteration j, the leftover sets of the </p><p>optimal solution can cover the remaining elements at a </p><p>cost of at most opt. (Why?) </p><p>solution optimal theofcost theis where</p><p>)1/()(price },,...,1{each For </p><p>opt</p><p>knoptenk k</p></li><li><p>My T. Thai </p><p> </p><p>36 </p><p>Proof of Lemma 1 (cont) </p><p> Thus, among these leftover sets, there must exist </p><p>one set where its value is at most opt/|Uj|. Let </p><p>ek be the element covered at this iteration, |Uj| </p><p>n k + 1. Since ek was covered by the most </p><p>cost-effective set in this iteration, we have: </p><p> price(ek) opt/|Uj| opt/(n-k+1) </p></li><li><p>My T. Thai </p><p> </p><p>37 </p><p>Approximation Ratio </p><p> Theorem 1: The set cover obtained form Alg 2 </p><p>(also Alg 1) has a factor of Hn where Hn is a </p><p>harmonic series Hn = 1 + 1/2 + + 1/n </p><p> Proof: It follows directly from Lemma 1 </p></li><li><p>Examples of NP-complete problems </p><p> Independent set </p><p> - independent set: a set of vertices in the graph with no </p><p>edges between each pair of nodes. </p><p> - given a graph G=(V,E), find the largest independent set </p><p> - reduction from vertex cover: </p><p>smallest vertex cover S largest independent set V/S </p></li><li><p>Independent Set </p><p> Independent set </p><p> - if G has a vertex cover S, then V \ S is an independent set </p><p> proof: consider two nodes in V \ S: </p><p> if there is an edge connecting them, then one of them must be in </p><p>S, which means one of them is not in V \ S </p><p> - if G has an independent set I, then V \ I is a vertex cover </p><p> proof: consider one edge in G: </p><p> if it is not covered by any node in V \ I, then its two end vertices </p><p>must be both in I, which means I is not an independent set </p></li><li><p>Steiner Tree </p><p> Steiner tree </p><p> - given a graph G=(V,E), and a subset C of V </p><p> - find the minimum tree to connect each vertex in C </p><p> - reduction </p><p>Vertex cover for Graph G Steiner tree for graph G </p><p>vertex </p><p>(u,v)</p><p>edge</p><p>(u,v)-u</p><p>edge</p><p>(u,v)-v</p><p>G (only edges with distance 1 are shown) </p><p>G </p></li><li><p>Steiner Tree </p><p> Construction </p><p> - G is a complete graph </p><p> - for every node u in G, create a node u in G </p><p> - for every edge (u,v) in G, create a node (u,v) in G </p><p> - in G, every node (u,v) is connected to u and v with distance 1 </p><p> - in G, every node u and v is connected with distance 1 </p><p> - other edges in G are of distance 2 vertex (u,v)</p><p>edge</p><p>(u,v)-u</p><p>edge</p><p>(u,v)-v</p><p>G (only edges with distance 1 are shown) </p><p>G </p></li><li><p> Sketch of Proof: </p><p> - in the Steiner tree problem for G, choose C to be the set of all </p><p>nodes (u,v) </p><p> - G has a minimum Steiner tree of cost m+k-1 iff G has a minimum </p><p>vertex cover of size k </p><p>vertex </p><p>(u,v)</p><p>edge</p><p>(u,v)-u</p><p>edge</p><p>(u,v)-v</p><p>G (only edges with distance 1 are shown) </p><p>G </p></li><li><p>Summary of some NPc problems </p><p>SAT </p><p>3SAT </p><p>Vertex cover </p><p>Independent set </p><p>Set cover </p><p>Graph coloring </p><p>Maximum clique size </p><p>Minimum Steiner tree </p><p>Hamiltonian cycle </p><p>Maximum cut </p><p>find more NP-complete problems in </p></li><li><p>Approximation Algorithms </p><p>Bin Packing </p></li><li><p>My T. Thai </p><p> </p><p>45 </p><p>Definition </p></li><li><p>My T. Thai </p><p> </p><p>46 </p><p>Theorem </p></li><li><p>My T. Thai </p><p> </p><p>47 </p><p>An Easy Solution: First Fit </p></li><li><p>My T. Thai </p><p> </p><p>48 </p><p>Asymptotic PTAS </p></li><li><p>My T. Thai </p><p> </p><p>49 </p></li><li><p>My T. Thai </p><p> </p><p>50 </p></li><li><p>Approximation Algorithms </p><p>k-Center </p></li><li><p>My T. Thai </p><p> </p><p>52 </p><p>Definition </p></li><li><p>My T. Thai </p><p> </p><p>53 </p><p>2-Approx </p></li><li><p>My T. Thai </p><p> </p><p>54 </p></li><li><p>My T. Thai </p><p> </p><p>55 </p></li><li><p>My T. Thai </p><p> </p><p>56 </p><p>Analysis </p></li><li><p>My T. Thai </p><p> </p><p>57 </p><p>Obtain Better Ratio? </p></li><li><p>My T. Thai </p><p> </p><p>58 </p><p>Weighted k-Center </p></li><li><p>My T. Thai </p><p> </p><p>59 </p><p>Notations </p></li><li><p>My T. Thai </p><p> </p><p>60 </p><p>Algorithm </p></li><li><p>My T. Thai </p><p> </p><p>61 </p><p>Algorithm 2 </p></li><li><p>My T. Thai </p><p> </p><p>62 </p><p>Analysis </p><p> Theorem: Alg 2 has an approx ratio of 3. </p></li><li><p>Approximation Algorithms </p><p>Multiway Cut and k-Cu...</p></li></ul>


View more >