complexity classes - university of...

Complexity Classes

Complexity Classes

What is a problem?

Given: a set of variables, a set of values that are possible for those variables, a set of constraints.Question: does there exist an assignment of values to the variables that satisfies the constraints?

Complexity Classes

What is a problem?


What happened to objective functions?

Complexity Classes

What is a problem?


What is the size of the input? A minimal encoding under a given alphabet, usually {0,1} Example: the number 25 could be written like this 11111111111111111111111110 or it could be written like this: 11001

Complexity Classes

What does it mean for a problem to be solved in polynomial time?

There exists an algorithm whose running time on an instance of size n is bounded by nk for some constant k.

Complexity Classes



Consider the problem of determining whether a numberis prime. An algorithm considered earlier takes time n0.5

Is this a polynomial time algorithm?

Complexity Classes



Consider the problem of determining whether a numberis prime. An algorithm considered earlier takes time n0.5

Is this a polynomial time algorithm? No Why? Length of input: log(n) = L Running time: n0.5 = (2log(n))0.5 = 20.5L

Complexity Classes



Consider the problem of finding the greatest numberin a list of n numbers. An algorithm considered earlier takes time bounded by n Is this a polynomial time algorithm?

Complexity Classes



Consider the problem of finding the greatest numberin a list of n numbers. An algorithm considered earlier takes time bounded by n Is this a polynomial time algorithm? Yes Why? Length of input: nlog(n) = L Running time: n = L/log(n) < L

Complexity Classes

Where do optimization problems fit in?

Optimization problem: find an assignment that Satisfies constraints and for which some objective Function is minimum (or maximum)

Optimization problem revised: the objective function becomes a constraint and the question is does there exist an assignment for which the objective function is less than k?

Complexity Classes

Where do optimization problems fit in?

Optimization problem: find an assignment that Satisfies constraints and for which some objective Function is minimum (or maximum)

Optimization problem revised: the objective function becomes a constraint and the question is does there exist an assignment for which the objective function is less than k?

Example: Min Cost Network – Old: Given graph G with edge weights, find a spanning tree of least total weight New: Given graph G with edges weights and a number k, does there exist a spanning tree of total weight ≤ k?

Complexity Classes

How does this help anything?

There is always an upper bound on the objective function Example: Min Cost Network – no greater than n max(w

i)

Proceed with binary search for the answer starting at the maximum value.

For Min Cost Network: result is obtained in log(n max(wi))

reruns of the algorithm. Total running time: nlog(n)log(n max(w

i))

= nlog(n)log(n) + nlog(n)log( max(wi))

Length of input: nlog(max(wi)) = L

Total running time: nlog2(n) + Llog(n) ≤ Llog2(L) + 2L Conclude: polynomial complexity is preserved

Complexity Classes

A problem instance: Some sort of structure (e.g. Graph, Boolean formula, list)

A problem: Some constraints that must hold for any structure of some specific type. Is there an assignment of values to the variables of a given instance obeying the specified structure of a problem for which all constraints hold?

Worstcase running time: An upper bound on the number of operations needed for an algorithm to solve a problem as a function of input length, assuming the most efficient encoding.

Complexity Classes

The complexity class P: A problem X (variables, constraints, yes/no result) is in complexity class P if and only if there is an algorithm which takes an instance of X as input, always correctly answers yes/no depending on whether the constraints can be satisfied, and has a worstcase running time that is bounded from above by a polynomial on the length of the instance.

Complexity Classes

Examples of problems in class P: Minimum Cost Network (greedy) Integer Deadline Scheduling (greedy) Single source shortest path (greedy) Huffman tree encoding (greedy) System of linear equality constraints (Gaussian elimination) System of linear constraints and objective function (Lin Prog) Biconnected components of a graph (depth first search) Graph connectedness (depth first search) Maximum matching in a graph (Edmond’s algorithm) Horn satisfiability (binary constraint propagation) 2SAT (binary constraint propagation)

Not an example: Knapsack: O(nm) dynamic programming not poly time

Complexity Classes

Reasonable statement of tractability: A problem is considered tractable if it is a member of complexity class P

Complexity Classes

Suppose God were helping us: Suppose God gives us a solution and all we have to do is check that the solution satisfies all the constraints. If we can make the check in polynomial time, the problem is said to be nondeterministic polynomial time solvable or in the class NP (all bets off for nonsolution)

Complexity Classes

Suppose God were helping us: Suppose God gives us a solution and all we have to do is check that the solution satisfies all the constraints. If we can make the check in polynomial time, the problem is said to be nondeterministic polynomial time solvable or in the class NP (all bets off for nonsolution)

Example: Satisfiability: does assignment 000100 satisfy?

Vertex cover: is 1,3,5,7 a vertex cover of size 4?

2

3

4

5

1

2

3

4

56

7

Complexity Classes

What does it mean to be the “hardest” problem in the world?

YX

1

2

3

4

56

7

∃ a vertex cover ofsize 5?

T = O(nk) algorithmto translate X to Y

∃ assignment of values tov

0... that causes formula

to evaluate to 1?YES iff YES

Complexity Classes

YX


∃ a O(nk) algorithm for Y → ∃ a O(nk) algorithm for X

Hence X is no harder than Y


Complexity Classes

YX


∃ a O(nk) algorithm for X → ∃ a O(nk) algorithm for Y

But Y can be harder than X


Complexity Classes

Y

X1

NPcomplete: the hardest problems in the world (for us, anyway) A problem is NPcomplete if it is a member of NP and there is a polynomial time translation from every NP problem to it

X2

X3 . . .

Complexity Classes

Examples of problems in class NPcomplete: Steiner Tree Integer Linear Programming Bin Packing Most Scheduling Problems Most Transportation Problems Satisfiability Deadlock Avoidance Production Planning Most Motion Planning Problems Traveling Salesman (Hamiltonian Cycle) Dynamic Storage Allocation

Not an example: Eulerian Path – visit every edge exactly once

Complexity Classes

coNP: A problem whose complement is in NP

Example: Satisfiability: given a CNF formula f, does there not exist an assignment that satisfies f ? Minimum Cost Network: given graph with edge weights is there no spanning network of weight k?

Complexity Classes



Observe: Since there is a polytime algorithm for MCN, just reverse the answer to solve the complementary problem

Complexity Classes



Observe: Since there is a polytime algorithm for MCN, just reverse the answer to solve the complementary problem

But for Satisfiability, we cannot rely on God for an answer so determining that f is unsatisfiable may be really hard

Complexity Classes

coNPcomplete: A problem is coNPcomplete if it is a member of coNP and there is a polynomial time translation from every coNP problem to it

Y

X1

X2

X3 . . .

Complexity ClassesWhat we know: A problem that is in P is also in coNP A single problem in P that is NPcomplete means P=NP

Complexity ClassesWhat we know: A problem that is in P is also in coNP A single problem in P that is NPcomplete means P=NP If P=NP then all hell breaks loose (a name goes up in lights)

Complexity ClassesWhat we know: A problem that is in P is also in coNP A single problem in P that is NPcomplete means P=NP If P=NP then all hell breaks loose (a name goes up in lights) Either the coNPcomplete and NPcomplete classes are equal or disjoint (no overlap)


What we do not know: Whether P=NP Whether graph isomorphism is in P or NPcomplete Whether factoring a large number is NPcomplete or in P


What we do not know: Whether P=NP Whether graph isomorphism is in P or NPcomplete Whether factoring a large number is NPcomplete or in P

What we think: P ≠ NP coNP ≠ NP coNPcomplete ≠ NPcomplete

Complexity ClassesWhat was the first problem shown to be NPcomplete? Hint: first public presentation in Cleveland, Ohio, 1971

Complexity ClassesWhat was the first problem shown to be NPcomplete? Correct: Satisfiability


How was it done?


How was it done? Rough idea: All we have to do is provide a circuit that outputs 1 if and only if the original problem has a solution that is, we need to express the quick “check” of the constraints in propositional logic


How was it done? Rough idea: All we have to do is provide a circuit that outputs 1 if and only if the original problem has a solution that is, we need to express the quick “check” of the constraints in propositional logic

1

2

3

k =1 Vertex cover


How was it done? Rough idea: All we have to do is provide a circuit that outputs 1 if and only if the original problem has a solution

1

2

3

∨ ∨

1 23

∨∨ ∨

1 2 3

∧∧∧

Make sure only one vertex is in the cover

Make sure alledges are covered

k =1

Complexity ClassesEven if the circuit uses gates other than ‘and’ and ‘or’any circuit can be expressed as an equivalent CNF formula

How so? 1st create a formula digraph – vertices are operators, edges connect to evaluation of nesting

Complexity ClassesEven if the circuit uses gates other than ‘and’ and ‘or’any circuit can be expressed as an equivalent CNF formula

How so? Then create a new variable foreach operator vertex.

vx10

vx9

vx6vx8

vx4vx7

vx5

vx3vx2vx1

Complexity ClassesFinally, add clause sets for all new variables according to this table:

Complexity ClassesFor this example we get the following:

Note: there are at most 4 clauses for each vertex – hence thisis a polynomial time translation from any Boolean expressionto CNFNote: no such translation exists to DNF

Complexity ClassesIf you have one NPcomplete problem you have them all

Y Z

If there is a polynomial time translation from Y to Z andY is NPcomplete

Complexity ClassesIf you have one NPcomplete problem you have them all

Y

X1

X2

X3 . . .

Z

If there is a polynomial time translation from Y to Z andY is NPcomplete then all NPproblems polytime translate to Z through Y

Complexity ClassesSimple transformation: Show 3CNF is NPcomplete

Any assignment that satisfies (1) can be extended to satisfy (2):if the left clause of (2) cannot be satisfied by the literal assignments in (1), then set vx1 to 1 otherwise, set it to 0.Any assignment not satisfying (1) also does not satisfy (2)

For any length clause, repeat the above until clause width is 3.It is not possible using this reduction to get equivalence with 2 literal clauses.

Complexity ClassesHarder transformation: Show Vertex Cover is NPcomplete Transformation from 3SAT:

v1

v2

v3

v4

v5

v6

k = 2(# clauses) + number variables 14 for this example


v1

v2

v3

v4

v5

v6

Any satisfying assignment: There is a cover of size k = 2(# clauses) + number variables


v1

v2

v3

v4

v5

v6

Any nonsatisfying assignment: There is no cover of size k = 2(# clauses) + number variables

Complexity ClassesHarder still: Show Hamiltonian Cycle is NPcomplete Transformation from Vertex Cover Hamiltonian Cycle: Given undirected graph G = (V,E), does there exist a cycle in G that passes through all vertices in G?


For each edge ⟨u,v⟩ ∈ E create an “edge gadget”:

u

v



u

v

1st way a Hamiltonian cycle can go through this gadget corresponds to v in the cover and not u



u

v

2nd way a Hamiltonian cycle can go through this gadget corresponds to u in the cover and not v



u

v

3rd way a Hamiltonian cycle can go through this gadget corresponds to v and u both in the cover

Complexity Classes1

2

3

4

56

7

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

⟨4,5⟩

⟨5,6⟩

⟨3,4⟩

⟨2,3⟩

⟨1,2⟩

Complexity Classes1

2

3

4

56

7

5

5

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

⟨4,5⟩

⟨5,6⟩

⟨3,4⟩

⟨2,3⟩

⟨1,2⟩

Complexity Classes1

2

3

4

56

7

75

21

3

4

6

5

617

43

2

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

⟨4,5⟩

⟨5,6⟩

⟨3,4⟩

⟨2,3⟩

⟨1,2⟩

Complexity Classes1

2

3

4

56

7

...

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

Complexity Classes1

2

3

4

56

7

...

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

7

Complexity Classes1

2

3

4

56

7

...

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

7

⟨4,5⟩

⟨5,6⟩

Complexity Classes1

2

3

4

56

7

...

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

⟨4,5⟩

⟨5,6⟩

75

Complexity Classes1

2

3

4

56

7

...

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

⟨4,5⟩

⟨5,6⟩

75

⟨3,4⟩

⟨2,3⟩

Complexity Classes1

2

3

4

56

7

...

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

⟨4,5⟩

⟨5,6⟩

⟨3,4⟩

⟨2,3⟩

⟨5,7⟩

753

Complexity Classes1

2

3

4

56

7

...

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

⟨4,5⟩

⟨5,6⟩

⟨3,4⟩

⟨2,3⟩

⟨5,7⟩

753

⟨1,2⟩

Complexity Classes1

2

3

4

56

7

...

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

⟨4,5⟩

⟨5,6⟩

⟨3,4⟩

⟨2,3⟩

⟨1,2⟩

7531

Complexity Classes1

2

3

4

56

7

...

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

⟨4,5⟩

⟨5,6⟩

⟨3,4⟩

⟨2,3⟩

⟨1,2⟩⟨1,2⟩

7531

Complexity Classes1

2

3

4

56

7

⟨5,7⟩

⟨6,7⟩⟨2,7⟩

⟨1,7⟩

⟨4,5⟩

⟨5,6⟩

⟨3,4⟩

⟨2,3⟩

⟨1,2⟩

7531

Complexity ClassesApple pie simple: Show Traveling Salesman is NPcomplete Transformation from Hamiltonian Cycle Traveling Salesman: Given undirected graph G = (V,E), and a weight function w

G: E → N

+ mapping edges to positive integers, does there exist a Hamiltonian cycle in G whose total weight is B?

1

2

3

4

56

7

Complexity Classes

1

2

3

4

56

7

Create a complete graph H that has a number of vertices equal to G. Labelthe vertices of H exactly as vertices arelabeled in G. For all edges ⟨x,y⟩ in G let w

H(⟨x,y⟩) = 1 for the corresponding

edge in H.

G:

H:

11

11

11

11

1

1

2

3

4

56

7

Complexity Classes

1

2

3

4

56

7



edge in H. For every other edge e in H, assign w(e) = number of vertices+1. LetB = number of vertices.

G:

H:

88

8

8

1

2

3

4

56

7

Complexity Classes



edge in H. Let n be the number of verticesin G and H. For every other edge e in H, assign w(e) = n+1. Let B = n.

Any Hamiltonian Cycle in H of total weight n corresponds to a HC in G.

G:

H:

1

2

3

4

56

7

11

1

11

11

1

2

3

4

56

7

Complexity Classes



edge in H. Let n be the number of verticesin G and H. For every other edge e in H, assign w(e) = n+1. Let B = n.

Any Hamiltonian Cycle in H of total weight n corresponds to a HC in G.

Any Hamiltonian Cycle in H of weight other than n must include an edge that does not exist in G

G:

H:

1

2

3

4

56

7

11

111

1

8

Complexity Classes

Shortest Path – two verticesIs there a simple path from a to bin G(V,E) having length B or less?

Longest path – two verticesIs there a simple path from a to bin G(V,E) having length B or more?

Edge Cover Is there a subset of edges of sizeno greater than k in G(V,E) that covers all vertices in G?

Vertex Cover Is there a subset of vertices of sizeno greater than k in G(V,E) that covers all edges in G?

Transitive ReductionIs there a set of directed edges A onvertices in G(V, E) of size no greater than k s.t. G(V, A) has paths between all vertices iff G does?

Minimum Equivalent DigraphIs there a subset E’ of dir. edgesin G(V,E) of size no greater than ksuch that G’(V,E’) has paths between all vertices iff G does?

Intree SchedulingProcesses: unit time, partial order with at most one successor. Is there a schedule that meets all deadlines?

Outtree SchedulingProcesses: unit time, partial order with at most one predecessor. Is there a schedule that meets all deadlines?

P NPcomplete

complexity classes - university of...

Documents