1

NP-Completeness

Algorithms and Networks 2015/2016Hans L. Bodlaender

Johan M. M. van Rooij

Today and next week

Complexity of computational problems Formal notion of computations NP-completeness

Why is it relevant? What is it exactly? How is it proven?

P vs. NP Some animals from the complexity zoo

INTRODUCTIONNP-Completeness – Algorithms and Networks

Easy Problems / Hard Problems

Finding the shortest simple path between vertices v and w in a given graph.

Determine if there is an Euler tour in a given graph.

Testing 2-colourability.

Satisfiability when each clause has two literals.

Finding the longest simple path between vertices v and w in a given graph.

Determine if there is a Hamiltonian circuit in a given graph.

Testing 3-colourability.

Satisfiability when each clause has three literals.

Fast and Slow

Algorithms whose running time is polynomial in input size.

Algorithms whose running time is exponential in input size.

Or worse…

Or in between???

EXPTIME

Many problems appear not to have a polynomial time algorithm.

For a few, we can proof that each algorithm needs exponential time: EXPTIME hardness, in particular generalized games

(Generalized Go – Japanese rules, Generalized Chess) For most, including many important and interesting

problems, we cannot.

NP-Completeness

Theory shows relations and explains behavior of many combinatorial problems

From many fields: Logic Graphs, networks, logistics, scheduling Databases Compiler optimization Graphics …

We need formalisation!

Formal notion of: Problem instance. Decision problem. Computation. Running time.

ABSTRACT AND FORMAL PROBLEMS

NP-Completeness – Algorithms and Networks

Different versions of problems

Decision problems. Answer is yes or no.

Optimization problems. Answer is a number representing an objective value.

Construction problems. Answer is some object (set of vertices, function, …).

Counting problems. How many objects of some kind exists?

Focus ondecision problems

Abstract versus formal problems

Abstract: Talks about graphs, logic formulas, applications, …

Formal: Inputs are sets of strings over a finite alphabet.

Abstract and formal problem instances

Computers work with bit strings. Problems are described using objects:

G is a graph, … Given a logic formula F, … Is there a clique C of size at least k, ...

To get a formal instance, we must map objects to bit strings. Or another encoding like Gödel numbers...

Abstract and formal problem instances

1

23

4 5

6

7 8

0 1 1 0 0 0 1 0

1 0 1 0 0 0 0 0

1 1 0 1 1 1 0 0

0 0 1 0 1 0 0 0

0 0 1 1 0 0 0 0

0 0 1 0 0 0 1 0

1 0 0 0 0 1 0 1

0 0 0 0 0 0 1 0

Abstract decision problems

Abstract decision problem: Set of instances I.

Objects like graphs, integers, logic formulas, etc. Subset of I: instances where the answer to the problem is YES.

Formal problem encoding

Encoding: mapping of set of instances I to bitstrings in {0,1}*.

Formal (decision) problem: Subset of {0,1}*.

By using an encoding, an abstract decision problem maps to a concrete decision problem.

Language of a problem Decision problem as a language:

Set of all Yes-instances.

P AND NPNP-Completeness – Algorithms and Networks

Solving a formal problem

An algorithm solves a formal problem: Decides if string in {0,1}* belongs to the subset.

Algorithm uses polynomial time, if there is a polynomial p such that on inputs of length n the algorithm uses at most p(n) computation steps. Size of input x is denoted |x|.

Time: Deterministic. Worst case.

Computation steps. Depend on model of computation. Turing machine. RAM-model with word size.

Complexity Class P

Formal Class of languages L, for

which there exists a deterministic Turing Machine deciding whether s L, using running time O( p(|s|) ) for some polynomial p.

Abstract Class of decision problems

that have polynomial time algorithms solving them.

Using P for abstract problems

Abstract problem (with encoding) is in P, if the resulting formal problem is in P. In practice: encoding and corresponding concrete problem is

assumed very implicitly. When considering polynomial time algorithms, encoding

mostly does not matter. If we can transform encodings in polynomial time to each

other. Details: see e.g., chapter 34 of Introduction to Algorithms.

Verification algorithm

Verification algorithm has two arguments: Problem input, Certificate (“solution”).

Answers “yes” or “no”.

The algorithm checks if the 2nd argument is a certificate for the first argument for the studied problem.

The language verified by the verification algorithm A is {s | there is an c with A(s,c)=Yes}.

Complexity Class NP

Two equivalent definitions of NP: Class of languages L, for which there exists a Non-

Deterministic Turing Machine deciding whether i L, using running time O(p(|i|)).

Class of languages L, for which there exists a Deterministic Turing Machine verifying whether i L, using a polynomial sized certificate c, and using running time O(p(|i|)) This definition we use: details follow.

NP

Problems with polynomial time verification algorithm and polynomial size certificates.

Problem L belongs to the class NP, if there exists a 2-argument algorithm A, with: A runs in polynomial time. There is a constant d such that for each i, i is a yes-instance to

L, if and only if there is a certificate c with:• A(i,c) = true.• |c| = O(|i|d).

Many problems are in NP

Examples: Hamiltonian Path, Maximum Independent Set, Satisfiability, Vertex Cover, …

Al of these have trivial certificates (set of vertices, truth assignment, …)

In NP (not trivial): Integer Linear Program.

P Í NP

If A decides L in polynomial time, then as verification algorithm, compute: B(i,c) = A(i). “We do not need a certificate” !

Famous open problem: P = NP ?? Or not??

NP-COMPLETENESS AND THE COOK-LEVIN THEOREM

NP-Completeness – Algorithms and Networks

Reducibility

Language L1 is polynomial time reducible to language L2 (or: L1 £P L2 ), if there exists a polynomial time computable function f : {0,1}* ® {0,1}* such that: For all x Î {0,1}*: x Î L1, if and only if, f(x) Î L2.

Lemma: If L1 £P L2 then: if L2 Î P, then L1 Î P.

Proof-idea: run an algorithm for L2 on f(i) for input i forproblem L1.

Lemma: If L1 £P L2 then: if L2 Î NP, then L1 Î NP.

NP-completeness

A language L is NP-complete, if:1. L Î NP.2. For every L’ Î NP: L’ £P L.

A language L is NP-hard, if:3. For every L’ Î NP: L’ £P L.

NP-hardness sometimes also used as term for problems that are not a decision problem, and for problems that are ‘harder than NP’.

What does it mean to beNP-Complete?

Evidence that it is (very probably) hard to find an algorithm that solves the problem. Always. Exactly. In polynomial time.

CNF-Satisfiability

Input: Expression over Boolean variables in conjunctive normal form.

Question: Is the expression satisfiable?I.e., can we give each variable a value (true or false) such that the expression becomes true? CNF: “and” of clauses; each clause “or” of variables or

negations (xi or not(xj))

Cook-Levin theorem

Satisfiability is NP-complete. Most well known is Cook’s proof, using Turing machine

characterization of NP.

Brief sketch of ideas in proof of Cook-Levin theorem

Suppose problem Q is in NP. Build a Turing Machine that:

Gets on an input tape a string (i, c), with c a certificate of size O(|i|d).

Runs the verification algorithm A in O(|i|q) time. d, q constants.

Make a logic formula that expresses “A halts in an accepting state on (i,c)” Variables:

• a(t,r,s): true if at time t, the rth symbol on the tape is symbol s.• b(t,a): true if the machine is at time t at state a.• c(t,r): true if at time t, the reading head at the tape is at position r.

Lots of clauses that express that the variable act like a Turing Machine tells them to do.

PROVING THAT PROBLEMS ARE NP-COMPLETE

NP-Completeness – Algorithms and Networks

Proving problems NP-complete

Lemma1. Let L’ £P L and let L’ be NP-complete. Then L is NP-hard.

2. Let L’ £P L and let L’ be NP-complete, and L Î NP. Then L is NP-complete.

3-SAT

3-Sat is the special case of CNF-Satisfiability where each clause has exactly three literals.

Lemma: CNF-Satisfiability ·P 3-Sat Clauses with one or two literals:

Use two extra variables p and q. Replace (x or y) by (x or y or p) and (x or y or :p). Similarly, replace a 1-literal clause by 4 clauses.

Clauses with more than three literals: For (l1 or l2 or … lr) add new variable t and take as replacement

clauses (l1 or l2 or t) and (:t or l3 or … or lr). Repeat this until the last clause has size three.

3-SAT is NP-complete

Proof:1. Membership in NP (almost trivial).2. Reduction (last slide).

3-Sat is important starting problem for many NP-completeness proofs.

Clique

Clique Given: graph G=(V,E), integer k. Question: does G have a clique with at least k vertices?

Clique is NP-complete. In NP … easy! NP-hardness: using 3-sat.

Reduction for Clique

One vertex per literal per clause.

Edges between vertices in different clauses, except edges between xi and :xi.

If m clauses, look for clique of size m.

x1

x1

x2

:x3

:x2 x3

…

Clause: {x1,:x2,x3}

Clause: {x1,x2,:x3}

Correctness

Lemma: There is a satisfying truth assignment, if and only if, there is a clique with m vertices.

! : Select from each clause the true literal. The corresponding vertices form a clique with m vertices.

à : Set variable xi to true, if a vertex representing xi is in the clique, otherwise set it to false. This is a satisfying truth assignment: The clique must contain one vertex from each 3 vertices

representing a clause. It cannot contain a vertex representing xi and a vertex

representing :xi.

Independent set

Independent set: set of vertices W Í V, such that for all v,w Î W: {v,w} Ï E.

Independent set problem: Given: graph G, integer k. Question: Does G have an independent set of size at least k?

Independent set is NP-complete.

Independent set is NP-complete

In NP. NP-hard: transform from Clique. W is a clique in G, if and only if, W is an independent set in

the complement of G (there is an edge in Gc iff. there is no edge in G).

How do I write down this proof?

Theorem: Independent Set is NP-complete.Proof: The problem belongs to NP:

As certificates, we use subsets of vertices of the input graph; We can check in polynomial time for a given subset of vertices

that it is an independent set and that its size is at least k. To show NP-hardness, we use a reduction from Clique.

Let (G,k) be an instance of the clique problem. Transform this to instance (Gc,k) of the independent set

problem with Gc the complement of G. As G has a clique with k vertices, if and only if, Gc has an

independent set with k vertices, this is a correct transformation.

The transformation can clearly be carried out in polynomial time.

Writing an NP-Completeness proof

State the theorem. Proof starts (or ends) with showing that problem belongs to NP.

Sometimes (e.g., in journal paper: trivial). Only if it is. Otherwise, explain the encoding of certificates and how / that they can

be checked in polynomial time. State which known NP-complete problem you transform from.

Careful: do not go in the wrong direction. Explain the transformation. Give the proof: instance to original problem is Yes-instance, if and

only if, transformed instance is Yes-instance for the known NP-complete problem. Remember: you need to proof this in two directions.

Phrase (or prove if not trivial): transformation can be carried out in polynomial time, hence problem is NP-complete.

QED.

Some comments on ‘proofs’by former students

Things I do not want to see in your ‘proofs’: If problem P would return true, then … Input/output of a problem. There is no polynomial time algorithm for P, thus there is no

polynomial time algorithm for Q. “Certificate to Hamilton cycle is a set of vertices.”

Terminology on reductions with problems/instances. A reduction reduces one problem to another problem. Yet the reduction transforms an instance of problem A into an

instance of problem B. After your reduction always prove that:

Your instance is a Yes-instance, if and only if, transformed instance is a Yes-instance.

Or if instance is a Yes-instance then transformed instance is a Yes-instance, and if instance is a No-instance then transformed instance is a No-instance.

Vertex Cover

Set of vertices W Í V with for all {x,y} Î E: xÎW or yÎW. Vertex Cover problem:

Given G, find vertex cover of minimum size.

= vertex cover

Vertex cover is NP-Complete

In NP. NP-hard: reduction from independent set. W is a vertex cover in G, if and only if, V-W is an

independent set in G.

3-Colouring

3-Colouring Given: Graph G=(V,E) Question: Can we colour the

vertices with 3 colours, such that for all edges {v,w} in E, the colour of v differs from the colour of w.

3-colouring is NP-Complete.

1 2

3

21

3

2

Proof of NP-Completeness of 3-Colouring

In NP: should be clear. Transformation from 3-sat:

We build a graph in 3 steps:1. Take a clique with 3

vertices True, False, C.2. Take two adjacent vertices

for each variable.

T F

C

x1 :x1 x2 :x2

We name the colours:

T, F, C

Continue

3. For each clause {l1,l2,l3}, take the following gadget:

T

C

l1 l2

l3

NP-Completeness of 3-ColouringWrap-up

One can show: If the formula is satisfiable, then there is a 3-colouring of G. If there is a 3-colouring of G, then the formula is satisfiable.

In both cases, the intuition is that a literal vertex is coloured T (true), if and only if, we take it to be true in the formula.

That the transformation takes polynomial time.

THREE APPROACHES TO PROVE NP-COMPLETENESS

NP-Completeness – Algorithms and Networks

Approaches to proving NP-Hardness

Three approaches to prove NP-hardness:1) Restriction2) Local replacement3) Component Design

Approach 1: Restriction

Take the problem. Add a restriction to the set of instances.

NOT to the problem definition! Show that this is a known NP-complete problem.

Restriction: Weighted Vertex Cover

Weighted vertex cover Given: Graph G=(V,E), for each vertex v Î V, a positive integer

weight w(v), integer k. Question: Does G have a vertex cover of total weight at most k?

NP-Complete. In NP. NP-Hardness: reduction from Vertex Cover to Weighted Vertex

Cover (set all weights to 1).

This is the easiest form of reduction.

Restriction: Knapsack

Subset-sum (assume we know it is NP-complete) Given: set of positive integers S, integer t. Question: Is there a subset of S with total sum t?

Knapsack Given: Set S of items, each with integer value v and integer

weight w, integers W and V. Question: is there a subset of S of weight no more than W, with

total value at least V?

NP-complete In NP. NP-hardness: set all weights equal to their values (Subset

sum).

Approach 2: Local replacement

Form an instance of our problem by Taking an instance of a known NP-Complete problem. Making some change “everywhere”. Such that we get an equivalent instance, but now of the

problem we want to show NP-hard.

Examples of Local Replacement

We saw: 3-Satisfiability (from Satisfiability). Clique (from 3-Satisfiability). Independent Set (from Clique). Vertex Cover (from Independent Set).

We will see: Travelling Salesman Problem (from Hamiltonian Cycle). Planar 3-Colouring (from 3-Colouring).

NP-Completeness of Travelling Salesman Problem by local replacement

In NP. Reduction from Hamiltonian Circuit:

Take city for each vertex. Take cost(i,j) = 1 if {i,j} Ï E. Take cost(i,j) = 0, if {i,j} Î E. G has HC, if and only if, there is a TSP-tour of length 0.

Remark TSP variant with triangle inequality:

Use weights 2 and 1. G has HC, if and only if, there is a TSP-tour of length n.

NP-Completeness of Planar 3-Colouring by Local Replacement

Planar 3-colouring Given: Planar graph G=(V,E). Question: Can we colour the vertices of G with 3 colours, such

that adjacent vertices have different colours?

Plan: transform from 3-colouring. Take arbitrary graph G. Draw G on the plane. So, we possibly get some crossings. Replace the crossings by something clever.

Clever reduction

Garey et al. found the following:

c d

a

b

c

a

d

b

Property of the gadget

If a and b have the same colour, we cannot colour the gadget with 3 colours.

If c and d have the same colour, we cannot colour the gadget with 3 colours.

Otherwise, we can.

... Some additional details (basically, repeat the step) when an edge has more than one crossing …

Approach 3: Component design

Component design (gadgeteering).

Build (often complicated) parts of an instance with certain properties. Often we call these parts components, gadgets, or widgets.

Glue them together in such a way that the proof works.

Examples: 3-Colouring (from 3-Satisfiability). Hamiltonian Circuit (from Vertex Cover).

Hamiltonian circuit

Given: Graph G Question: does G have a

simple cycle that contains all vertices?

NP-Completeness ofHamiltonian Circuit

Hamiltonian Circuit is in NP. Vertex Cover £P Hamiltonian Circuit: complicated proof

(component design) Widgets. Selector vertices. Given a graph G and an integer k, we construct a graph H,

such that H has a HC, if and only if G has a VC of size k.

Widget

For each edge {u,v} we have a widget Wuv [u,v,1]

[u,v,6] [v,u,6]

[v,u,1]

Only possible ways to visit all vertices in widget

Selector vertices

We have k selector vertices s1, …, sk. These will represent the vertices selected for the vertex

cover.

Connecting the widgets

For each vertex v we connect the widgets of the edges {v,w}.

Suppose v has neighbors x1, …, xr: add edges {[v,x1,6],

[v,x2,1]}, {[v,x2,6],[v,x3,1]},

…, {[v,xr-1,6],[v,xr,1]}.

Connecting the selector vertices to the widgets

Each selector vertex is attached to the first neighbor widget of each vertex, i.e. to vertex [v,x1,1] and to the last neighbor widget [v,xr,6]

si

Vertex in examplehas degree 2

Correctness of reduction

Lemma: G has a vertex cover of size (at most) k, if and only if H has a Hamiltonian circuit.

The reduction takes polynomial time. So, we can conclude that Hamiltonian Circuit is NP-

complete.

WEAK AND STRONG NP-COMPLETENESS

NP-Completeness – Algorithms and Networks

Problems with numbers

Strong NP-complete: Problem is NP-complete if numbers are given in unary.

Weak NP-complete: Problem is NP-complete if numbers are given in binary, but

polynomial time solvable when numbers are given in unary.

Examples

Subset-sum Given: set of positive integers S, integer t. Question: Is there a subset of S with total sum t?

Subset-sum is Weak NP-complete. Solvable using dynamic programming in O(nt) time. That is, solvable in pseudo-polynomial time.

3-Partition Given: set of positive integers S. Question: can we partition S into sets of exactly 3 elements

each, such that each has the same sum (t)? 3-Partition is Strong NP-hard.

t must be the sum of S divided by the number of groups (|S|/3). Starting point for many reductions.

Remark

Easily made mistake: reductions from subset sum that create exponentially large instances.

Subgraph Isomorphism for maximum degree 2 graphs. Given: Graphs G and H, such that each vertex in G and H has

degree at most 2. Question: Is G a subgraph of H?

NP-hardness proof can be done with 3-Partition.

Subgraph isomorphism for forests of paths

Input: Forest F =(V,E), forest H = (W,F) max degree 2. Question: Is there an injection f: W -> V, such that for all

{v,w} in F, {f(v),f(w)} is in E, i.e., is H isomorphic to a subgraph of F?

Theorem: Subgraph isomorphism for forests with maximum degree two is NP-complete.

Proof

In NP. NP-complete: transform from 3-partition. Suppose we have numbers a1, …, a3m with total sum mt. F is forest with m paths, each with t vertices. H is forest with 3m paths, with lengths a1, …, a3m.

Only correct because 3-Partition is strongly NP-Complete. NP-Complete when input is given in unary!

Similar proof from 2-Partition would be WRONG! 2-Partition: Subset sum with t = total sum / 2.

SOME DISCUSSIONNP-Completeness – Algorithms and Networks

How to prove NP-completeness

Is there an easy proof by restriction? Is there a problem that “looks like our problem” – try to do

local replacement. Try component design:

Good starting problems are: 3-Sat; special cases of 3-Sat; 3-Partition; subset sum; simple graph problems (like clique, Independent Set, Vertex Cover, Colouring, 3-Colouring)

… puzzle …

Discussion

Is P ¹ NP? (who thinks so?) www.claymath.org/prizeproblems/pvsnp.htm

One of the millennium problems.

Why so hard to prove? What to do with problems that are NP-hard? Other complexity notions…

NP

P

NP-complete

P vs NP is hard to prove

P = NP? Hard to design poly algorithm... Current mathematical knowledge does not suffice to prove

P != NP: “Natural Proofs” can not separate P from NP (Razborov &

Rudich, 1993) PA

= NPA, but PB != NPB for some oracles A and B, so diagonalisation can not separate P from NP (Baker, Gill, & Solovay, 1975)

NP-Completeness is not the end

Almost every interesting real-life problem is NP-Complete. Scheduling, planning, ...

NP-Completeness is a starting point to think further not an excuse to call a problem impossible to solve.

Theoretical approaches: Fixed parameter tractability, exact exponential-time

algorithms, special graph classes / restricted instances, ... Practical approaches:

Integer Linear Programming, Constraint Programming, Local search, Heuristics, ....

Often: finding subproblems that are not NP-Hard helps a lot.

SOME ANIMALS FROM THE COMPLEXITY ZOO

NP-Completeness – Algorithms and Networks

Much more complexity classes

In Theoretical Computer Science, a large number of other complexity classes have been defined.

Here, we give an informal introduction to a few of the more important ones.

There is much, much, much more…

coNP

Complement of a class: switch “yes” and “no”. Polynomial time verifiable certificates of no instances. coNP: complement of problems in NP, e.g.:

NOT-HAMILTONIAN Given: Graph G. Question: Does G NOT have a Hamiltonian circuit.UNSATISFIABLE Given: Boolean formula in CNF. Question: Do all truth assignments to the variable make the

formula false?

All problems in P are in coNP. NP-Complete problems are most probably not in coNP.

A more natural example for NP and coNP

Integer factorisation: Given: integer n, integer m. Question: does n have a prime factor less than m?

In NP - Yes-certificate: Two numbers i, j with ij = n, i < m.

In coNP - No-certificate: List of all prime factors of n. Verify that these are primes (prime testing is in P – 2002). Verify that when multiplied we have n.

Problem in NP and coNP. Also in P? Maybe not – for cryptography sake we hope not!

PSPACE

All decision problems solvable in polynomial space. Unknown: is P=PSPACE? Savitch, 1970: PSPACE = NPSPACE

NPSPACE: solvable with non-deterministic program in polynomial space.

PSPACE-complete, e.g., Generalized Tic-Tac-Toe, generalized Reversi, Some puzzles that need an exponential number of moves. Quantified Boolean formula’s (QBF):

423214321 xxxxxxxxx

EXPTIME

Decision problems that can be solved in exponential time. P is unequal EXPTIME (Stearns, Hartmanis, 1965). EXPTIME complete problems:

Generalized chess, generalized checkers, generalized go (Japanese drawing rule).

Given a Turing Machine M and integer k, does M halt after k steps?

And a few more

NEXPTIME: non-deterministic exponential time. EXPSPACE = NEXPSPACE: exponential space.

Graph Isomorphism

Discussed in another lecture. Given two graphs, are they isomorphic? In NP, not known to be NP-complete; not known to be in P. Several problems are equally hard: Isomorphism-complete.

NC

NC: “Nicks class”, after Nick Pippinger. Talks about the time to solve a problem with a parallel

machine. Model: we have a polynomial number of processors, that

use the same memory. Variants depending on what happens when processors try to

read or write the same memory location simultaneously.

NC: decision problems that can be solved with a PRAM (Parallel Random Access Machine) with polynomial number of processors in polylogarithmic time. O( (log n)d ) for some constant d.

Unknown: P=NC?

Counting: #P

#P (“Sharp-P”). Problems that output a number. The precise definition will not be given here. Think as:

“what is the number of certificates for this instance”, with polynomial checking of certificates.

#P-complete e.g.: Number of satisfying truth assignments of 3SAT-formula. Number of Hamiltonian circuits in a graph. Number of perfect matchings in a given graph.

LSPACE or L

Problems can be solved with only logarithmic extra space: You can read the input as often as you want You may use only O(log n) extra memory

• E.g.: Q(1) pointers to your input

NL: non-deterministic logspace...

More

Oracles. The Polynomial Time Hierarchy. UP: unique solutions.

Questions?