P, NP and more

• Goal: transfer all n disks from peg A to peg C• Rules:

– move one disk at a time– never place larger disk above smaller one

• Recursive solution:– transfer n 1 disks from A to B– move largest disk from A to C– transfer n 1 disks from B to C

• Total number of moves:– T(n) 2T(n 1) 1

Towers of Hanoi

Towers of Hanoi (2)

• Recurrence relation:T(n) 2 T(n 1) 1T(1) 1

• Solution by unfolding:T(n) =2 (2 T(n - 2) + 1) + 1 = = 4 T(n - 2) + 2 + 1 = = 4 (2 T(n - 3) + 1) + 2 + 1 = = 8 T(n - 3) + 4 + 2 + 1 = ... = 2i T(n - i) + 2i-1 +2i-2 +...+21 +20

• the expansion stops when i n 1T(n) = 2n – 1 + 2n – 2 + 2n – 3 + ... + 21 + 20

Towers of Hanoi (3)

• This is a geometric sum, so that we haveT(n) 2n 1 O(2n)

• The running time of this algorithm is exponential (kn) rather than polynomial (nk), where n is input size.

• Good or bad news?– the Tibetans were confronted with a tower problem

of 64 rings...– Assuming the priests move one ring per second, it

would take ~585 billion years to complete the process!

Traveling Salesman Problem• Given a set of cities, distances between all pairs of cities, and a

bound B, does there exist a tour (sequence of cities to visit) that returns to the start and requires at most distance B to be traveled?

• TSP is in NP:– given a candidate solution (a tour), add up all the distances and check if

total is at most B

Last Time

• The two theorems we proved showed an important distinction.– The difference between a single and multi-tape TM is at most a

square (or polynomial) difference.– Moving to a nondeterministic TM gave an exponential speedup.

Polynomial Time

• From the perspective of time complexity, polynomial differences are considered small, whereas exponential differences are large.

– Exponential functions do grow incredibly fast; they grow much faster than any polynomial function.

– However, different polynomials grow much faster than others. In an algorithms course, you would be crazy to suggest that they are equivalent:

• O(n log n) sorting algorithms are much better than• O(n2) algorithms.• O(n) and O(n2) are radically different.

• Nonetheless, there are some good reasons for assuming polynomial-time equivalence.

Polynomial Time Algorithms

• What’s about?– Halting problem

• Is your program slow or is in in infinite loop? (Can compiler help you?)

– Chess (40 to 200 moves ~ 400 positions)• Very bushy trees (even with alpha-beta pruning)

Not P

Reasonable vs. Unreasonable Processing 1 elements takes 0.01 sec

Growth rates

1

1E+10

1E+20

1E+30

1E+40

2 4 8 16 32 64 128 256 512 1024

5n

n^3

n^5

1.2^n

2^n

n^n Number of microseconds

Reasonable vs. Unreasonable

• ”Good”, reasonable algorithms– algorithms bound by a polynomial function nk

– Tractable problems

• ”Bad”, unreasonable algorithms– algorithms whose running time is above nk

– Intractable problems

intractable

problemstractable problems

problems not admitting reasonable

algorithms

problems admitting reasonable (polynomial-

time) algorithms

Background

• Exponential-time algorithms arise when we solve problems by exhaustively searching a space of possible solutions using brute force search.

• Polynomial-time algorithms require something other than brute force.

• All reasonable computational models are polynomially-time equivalent.

• So if we view all polynomial complexity algorithms as equivalent, then the specific computational model doesn't matter.

Definition of the class P

• P is the class of languages that are decidable in polynomial time on a deterministic single-tape TM:

• That is:

The importance of P

• P plays a central role in the theory of computation because:– P is invariant over all models of computation that are– polynomially equivalent to a single-tape DTM.– P roughly corresponds to the class of problems that

are realistically solvable on a computer.– Take this with a grain of salt.

• Some polynomial-time algorithms are bad: O(n10000) or 100000000000n2.

• Exponential algorithms may be okay for small n.

Verifying a Candidate Solution

• Suppose that you are organizing housing accommodations for a group of four hundred university students. Space is limited and only one hundred of the students will receive places in the dormitory. To complicate matters, the Dean has provided you with a list of pairs of incompatible students, and requested that no pair from this list appear in your final choice.

• Solving a problem: find possible people-room distribution?

• Verifying a candidate solution: given a possible people-room distribution, verify whether it is correct?

Verifying a Candidate Solution vs. Solving a Problem

• Intuitively it seems much harder (more time consuming) in some cases to solve a problem from scratch than to verify that a candidate solution actually solves the problem.

• If there are many candidate solutions to check, then even if each individual one is quick to check, overall it can take a long time

Verifying a Candidate Solution

• Many practical problems in computer science, math, operations research, engineering, etc. are polynomial time verifiable but have no known polynomial time algorithm– Wikipedia lists problems in computational geometry, graph

theory, network design, scheduling, databases, program optimization and more

Decision problems

• Are such long running times linked to the size of the solution of an algorithm?– No! To show that, we in the following consider only

TRUE/FALSE or yes/no problems – decision problems

• We can usually transform an optimization problem into an easier decision problem:– Optimization problem O : “Find a shortest path between vertices

u and v in a graph G.” – Related decision problem D : “Determine if there is a path

between vertices u and v in a graph G shorter than k.”– If we have an easy way to solve O, we can use it to solve D.– If we show that D is hard, we know that O must be also hard.

The Class P

Why is it reasonable to consider all problems in P as tractable (i.e., "easy")? What about n5000?

• If is not in P, then it certainly is not easy• Polynomial time is mathematically convenient for

defining a class– closed under composition

• Model independent notion– poly time reductions between various formal models

• In practice, exponent is often small

The Class NP

• First, NP does not stand for not-P!!• NP is the class of problems for which a

candidate solution can be verified in polynomial time (nondeterministic solution in poly time)

• P is a subset of NP

More definitions

• P. Problems that can be solved in polynomial time. ("P" stands for polynomial.) These problems have formed the main material of this course.

• NP. This stands for "nondeterministic polynomial time" where nondeterministic is just a fancy way of talking about guessing a solution. – A problem is in NP if you can quickly (in polynomial time) test

whether a solution is correct (without worrying about how hard it might be to find the solution). Problems in NP are still relatively easy: if only we could guess the right solution, we could then quickly test it.

• NP does not stand for "non-polynomial". There are many complexity classes that are much harder than NP.

P vs. NP

• Although polynomial time verifiability seems like a weaker condition than polynomial time solvability, no one has been able to prove that it is weaker (describes a larger class of problems)

• So it is unknown whether P = NPif 'yes'-answers to a 'yes'-or-'no'-question can be verified

"quickly" (in polynomial time), can the answers themselves also

be computed quickly?

– one of seven millennium prize problems– if you solve it you will get $1,000,000

http://www.claymath.org/millennium/P_vs_NP/

P and NP

all problems

P

NPor

all problems

P=NP

The Main Question• If P=NP, then:

– Efficient algorithms for TSP, etc.– Cryptography is impossible on conventional machines– Modern banking system will collapse

The Main Question

• Yes? No?– Thousands of researchers have spent four decades in search of

polynomial algorithms for many fundamental NP-complete problems without success

– Consensus opinion: P NP

• But maybe yes, since:– No success in proving P NP either

The Big Picture

• Summarizing: it is not known whether NP problems are tractable or intractable

• But, there exist provably intractable problems– Even worse – there exist problems with running times

unimaginably worse than exponential!

• More bad news: there are provably noncomputable (undecidable) problems– There are no (and there will not ever be!!!) algorithms to solve

these problems

Why should we care?

• Difficult problems come up all the time. Knowing they're hard lets you stop beating your head against a wall trying to solve them, and do something better:

– Use a heuristic (solving a reasonable fraction of the common cases)– Solve the problem approximately instead of exactly. – Use an exponential time solution anyway. If you really have to solve the

problem exactly, you can settle down to writing an exponential time algorithm and stop worrying about finding a better solution.

– Choose a better abstraction. The NP-complete abstract problem you're trying to solve presumably comes from ignoring some of the seemingly unimportant details of a more complicated real world problem. Perhaps some of those details shouldn't have been ignored, and make the difference between what you can and can't solve.

P = NP Question

• Open question since about 1970• Great theoretical interest• Great practical importance:

– If your problem is NP-complete, then don't waste time looking for an efficient algorithm

– Instead look for efficient approximations, heuristics, etc.