Everything you always wanted to know about theoretical computer science

(but were afraid to ask)

Dimitris AchlioptasMicrosoft Research

Turing’s Machine

• Alphabet Σ, state space K

• f : K×Σ → K×Σ×{←,→,?} ×{Halt,Yes,No}

• Language: L µ Σ* is decided by ML

• Polynomial-time Robustness

Any reasonable attempt to model mathematically computer algorithms and their performance is bound to end up with a model of computation and associated time cost that is equivalent to Turing machines within a polynomial.

Church’s Thesis

Any reasonable attempt to model mathematically computer algorithms and their performance is bound to end up with a model of computation and associated time cost that is equivalent to Turing machines within a polynomial.

Church’s Thesis

Turing’s Idea

Programmable Hardware:

Describe f on the leftmost part of the tape.

Polynomial Time• N vertices• As,t distance from s to t

Find a spanning cycle of minimum length

Polynomial Time

• Edge lengths are written in binary. Lists vs. Matrices

• Assume M could decide if OPT<B in polynomial time– Do binary search to determine OPT – Probe edges one-by-one, replacing their cost with OPT+1

• N vertices• As,t distance from s to t

Find a spanning cycle of minimum length

P vs. NP

Polynomial Time PTIME = [k nk

L is in P if there exists a Turing Machine M which for every x, decides if x is in L in a polynomial number of steps.

P vs. NP

Polynomial Time PTIME = [k nk

L is in P if there exists a Turing Machine M which for every x, decides if x is in L in polynomially many steps.

Non-Deterministic Polynomial Time

L is in NP if there exists a Turing Machine M s.t. for every x • If x is in L then there exists w s.t. M(x,w)→“Yes” in PTIME.• If x is not in L then there is no such w.

Complexity

• Is there a spanning cycle of cost < C ? NP• Is there a spanning tree of cost < C ? P• Is there no spanning tree of cost < C ? P • Is there no spanning cycle of cost < C ? coNP

NPcoNP

P

Perebor

• 50-60s: Many optimization problems appear amenable only to brute force, i.e. (near-)exhaustive enumeration.

• Edmonds 1966: “The classes of problems which are respectively known and not known tohave good algorithms are of great theoretical interest […] I conjecture

thatthere is no good algorithm for the traveling salesman problem. My

reasonsare the same as for any mathematical conjecture: (1) It’s a legitimate mathematical possibility, and (2) I do not know.”

• Hartmanis, Stearns 1965:

“On the computational Complexity of Algorithms”, AMS Transactions

Input for Problem B

Output for Problem B

Algorithm for Problem B

Reductionfrom B to A

Algorithmfor B

x R(x) Yes/No

Reducibility

Input for Problem B

Output for Problem B

Algorithm for Problem B

Reductionfrom B to A

Algorithmfor B

x R(x) Yes/No

Shortest Spanning Path from s to t

Shortest Spanning Cycle

Reducibility

NP-completeness

A problem Π is NP-hard if every

problem in NP has a polynomial-time reduction to Π.

If Π is in ΝP and NP-hard then Π is NP-complete.

Moral: At least as hard as any other problem in NP

Cook’s Theorem

Satisfiability is NP-Complete

Idea:

Let variable Xp,t denote the content of tape-cell p after t steps

• By now thousands of problems have been shown NP-complete

• NP-completeness has migrated to other sciences, e.g. economics

Integer Linear Programming:

Ax·bIs there a satisfying vector x where all xi are non-negative INTEGERS?

NPcoNP

P TSP

PRIMALITY

More hard problems

Approximation

• Vertex Cover: Find a smallest set of vertices that touches every edge.– Algorithm: If an edge is still untouched, add

both its endpoints to the set.

Approximation

• Vertex Cover: Find a smallest set of vertices that touches every edge.– Algorithm: If an edge is still untouched, add

both its endpoints to the set.

• Max Cut: Partition the vertices of the graph to maximize number of cut edges.– Algorithm: Partition randomly.

Greed

Problem: Find a spanning tree of minimum length

Greed

Problem: Find a spanning tree of minimum length

Algorithm: Always add the cheapest allowable edge

Greed

Problem: Find a spanning tree of minimum length

Algorithm: Always add the cheapest allowable edge

Proof:

The cheapest edge emanating from each tree is always good.

Duality

Max-Flow = Min-Cut

Linear Programming:

Ax·bIs there a satisfying vector where all xi are non-negative rationals?

Linear Programming

Linear Programming:

Ax·bIs there a satisfying vector where all xi are non-negative rationals?

Linear Programming

An integer matrix A is Totally Unimodular (TUM) if every square, nonsingular submatrix B has det(B)=±1

Linear Programming:

Ax·bIs there a satisfying vector where all xi are non-negative rationals?

Linear Programming

An integer matrix A is Totally Unimodular (TUM) if every square, nonsingular submatrix B has det(B)=±1

If A is TUM then the LP solution is integral!

Linear Programming II

• Simplex method:– Most known rules can be made to take an

exponential number of steps.

• Ellipsoid algorithm:– Number of steps depends on size of integers

Is there an algorithm that solves LPs with m equations in n variables in Poly(m,n) steps?

Semidefinite Programming

Arrange n unit vectors so that their inner products satisfy a system of

linear inequalities.

Approximability

MAX CUT

• Assign a unit vector vs to each vertex s.• Maximize the sum of (1-vsvt)wst over all vertices• Take a random hyperplane through the origin.• Get an 0.87… approximation

Vertex Cover

• Conjecture: Cannot beat stupid algorithm!• Theorem: Need at least 1.36..*OPT

• Tool: NP = PCP(O(logn), O(1)).

Non-worst-case Inputs

• Random graphs

• Random pointsets

• Random formulas

Non-worst-case Inputs

• Random graphs

• Random pointsets

• Random formulas

Theoretical CS ProbabilisticCombinatorics

Non-worst-case Inputs

• Random graphs

• Random pointsets

• Random formulas

Theoretical CS ProbabilisticCombinatorics

Physics