Complexity Theory

Lecture 1

Lecturer: Moni Naor

Computational Complexity Theory• Study the resources needed to solve computational problems

– Computer time– Computer memory– Communication– Parallelism– Randomness– …

• Identify problems that are infeasible to compute by any reasonable machine

• Taxonomy: classify problems into classes with similar properties wrt the resource requirements– Help find the most efficient algorithm for a problem

Goal of Complexity Theory

• Given a computational problem – determine the best method of solving it

• For a given computational setting – determine which computation tasks can be executed

• State of the art: very far from these goals– Cannot provide unconditional interesting lower bounds

for almost any interesting problem– But:…

Complexity, Algorithms and Cryptography

• Complexity Theory has developed a taxonomy allowing to determine for many problems

Cryptography Algorithms

Complexity

Brief History of Complexity Theory• Logical foundations• 1930s: Computability Theory:

– Kurt Gödel– Alan Turing– Alonzo Church

• 1940s 1950s: work on circuit complexity (Shannon)

• 1960s: first papers dealing explicitly with complexity issues:– Definitions of polynomial time (Cobham, Edmonds, Peterson)– Time hierarchies, Speedup Theorems, Abstract Complexity Measures

• Early 1970s: formulation of the P=NP? question, NP-Completeness an industry (Cook, Karp, Levin)

• Since then – an explosion

1956 Gödel’s letter to von Neumann with the P=NP question

Developments in the Soviet Union

What will we learn in this course

• Time and space hierarchies • Non-determinism and NP-Completeness• Relativization (oracles)• Alternation• Space Complexity• Randomness• Interactive Proofs• PCP• Complexity of counting • Concrete Complexity models:

– Communication Complexity – Pebbling– Branching Programs

• Circuit Complexity

P, RP, BPP BQPNP, co-NP, PH, #P, PSPACE,

IP, AM, PCPNC, AC, L, NL

Course InformationComplexity Theory - Winter 2004/5

Instructor: Moni NaorGrader: Asaf Nussbaum When: Thursdays, 14:00--17:00 (3 points)Where: Ziskind 1

• Course web page: www.wisdom.weizmann.ac.il/~naor/COURSE/complexity.html

• Prerequisites: familiarity with algorithms, data structures, probability theory, and linear algebra, at an undergraduate level; a basic course in computability is assumed.

• Requirements:– Homework: There will be around ten homework assignments and a final test.

• Homework assignments should be turned in on time (usually two weeks after they are given)! • Try and do as many problems from each set.• You may discuss the problems with other students, but the write-up MUST be individual.• You may look up any material, but should cite any external source.

– Exam : The exam will be in class.

BibliographyNo textbook for the course. A lot of background and relevant material is available in • Christos Papadimitriou , Computational Complexity , Cambridge, 1994 • Michael Sipser, Introduction to the Theory of Computation, 1997 • Garey and Johnson. Computers and Intractability: A Guide to the Theory of NP-

Completeness. New York: W. H. Freeman, 1979.

Two online courses close in nature to our course:

• Lecturers: Steven Rudich and Avrim Blum, University: CMU • Lecturer: Luca Trevisan, Computational Complexity, University : UC Berkeley.

Things you should already know

• Turing Machine– One-tape vs. Multi-tape– Defined by (,Q,)

finite alphabet including blank symbol• State space including qstart, qaccept and qreject

• transition function X Q X Q X {left,right}

– A configuration is represented as a string of symbols from [ Q 1 2 … q … m

– Multi-tape machine has • Read-only input tape• Write-only output tape• Several working tapes

Things you should already know

• Recognition or decision of language L by a TM M– if M accepts every string in L and rejects all strings not in L– A language L is recursive or decidable if there is a TM M which

recognizes it• Acceptance of language L by a TM M

– if M accepts every string in L and does not accept strings not in L• does not necessarily halt on all inputs• A language L is recursive enumerable (r.e.) if there is a TM M which

accepts it

Turing Machines are general

Truing Machines can simulate many other models and machines

• RAM – Random Access Machines– Java..

• Circuits – if the circuit itself is uniform• Various logical systems• Turing machines: the language

Lu ={<M,x>|M accepts x} is r.e. – there is a (universal) TM accepting it

Simulation are pretty efficient Counterexample: quantum

Time and SpaceLet f:N N by any function.Time: L 2 TIME(f(n)) if there is a (multi-tape) TM which recognizes

L and uses no more than f(|x|) steps on input x Space: L 2 SPACE(f(n)) if there is a (multi-tape) TM which

recognizes L and uses no more than f(|x|) cells of its work tapes during the computation on input x– Cell is used if it scanned by the tape head

Time and space can be defined for function computation as well

No point in accuracy in the time functionTheorem (Linear Speedup): for any L 2 TIME(f(n)) and >0 there

is an equivalent TM M’ requiring only f(n)+n steps Conclusion: lets talk about O(f(n))

Proper complexity functions

• f(n) is a proper complexity function if– f(n)≥ f(n-1) – There is a TM which on input x outputs a string of length exactly f(|x|) while

running it time O(|x|+f(|x|)) and space O(f(|x|))

• Exercise: show that if f(n) and g(n) are proper complexity functions, then so are – f(n)+g(n)– f(n)g(n)

Clocked Simulation Theorem: for every proper complexity function fThere exists a TM Mf running in time f3(|Mx|) that on input (M,x) accepts iff

M(x) accepts within f(|x|) steps

Can improve f3(n) to f(n) log f(n)

Hierarchy Theorems and Diagonalization• Diagonalization – proof technique invented by Cantor to prove

the uncountability of the real numbers

Theorem (Time Hierarchy): for any proper complexity functions f(n) and g(n) ≥ f(n) log(f(n))

TIME(f(n)) is properly contained in TIME(g(2n)) Proof: Recall Mf. Consider

L ={<M|Mf(MM) rejects} We have L2TIME(g(2n)) but L is not in TIME(f(n))

Corollary: TIME(nk) is properly contained in TIME(nk+1)

Communication Complexity

Alice

x2X

Boby2Y

Let f:X x Y Z

Input is split between two participantsWant to compute outcome: z=f(x,y)

while exchanging as few bits as possible

A protocol is defined by the communication tree

z0 z1 z2 z3 z4 z5 z6 z7 ...

Alice: 0

Bob: 1

Alice: 0

Bob: 1

z5

A ProtocolA protocol P over domain X x Y with range Z is a binary tree where

– Each internal node v is labeled with either • av:X {0,1} or• bv:Y {0,1}

– Each leaf is labeled with an element z 2 Z• The value of protocol P on input (x,y) is the label of the leaf reached by

starting from the root and walking down the tree. • At each internal node labeled av walk

– left if av(x)=0

– right if av(x)=1• At each internal node labeled bv walk

– left if bv(y)=0

– right if bv(y)=1

– The cost of protocol P on input (x,y) is the length of the path taken on input (x,y) – The cost of protocol P is the maximum path length

Motivation for studying communication complexity

• Originally for studying VLSI questions• Connection with Turing Machines• Data structures and the cell probe model• Boolean circuit depth• …

Communication Complexity of a function

• For a function f:X x Y Z the (deterministic) communication complexity of f (D(f)) is the minimum cost of protocol P over all protocols that compute f

Observation: For any function f:X x Y Z D(f) ≤ log |X| + log |Z|

Example: let x,y µ {1,…,n} and let f(x,y)=max{x [ y} Then D(f) · 2 log n

Median

let x,y µ {1,…,n} and let MED(x,y) be the median of the multiset x [ y If the size is even then element ranked |x[ y|/2

Claim: D(MED) is O(log2 n)protocol idea: do a binary search on the value, each party

reporting how many are above te current guess

Homework: D(MED) is O(log n)protocol idea: each party proposes a candidate

See which one is larger - no need to repeat bits

Combinatorial Rectangles• A combinatorial rectangle in X x Y is a subset R µ X x Y such that R=

A x B for some A µ X and B µ Y Proposition: R µ X x Y is a combinatorial rectangle iff (x1,y1) 2 R and

(x2,y2) 2 R implies that (x1,y2) 2 R

For Protocol P and node v let Rv be the set of inputs (x,y) reaching v

Claim: For any protocol P and node v the set Rv is a combinatorial rectangle

Claim: For any given the transcript of an exchange between Alice an Bob possible (but not x and y) possible to determine z=f(x,y)

Fooling Sets• For f:X x Y Z a subset R µ X x Y is f-monochromatic if f is fixed

on R• Observation: any protocol P induces a partition of X x Y into f-

monochromatic rectangles. The number of rectangles is the number of leaves in P

• A set Sµ X x Y is a fooling set for f if there exists a z 2 Z where– For every (x,y) 2 S, f(x,y)=z – For every distinct (x1,y1), (x2,y2) 2 S either

• f(x1,y2)≠z or• f(x2,y1)≠z

Property: no two elements of a fooling set S can be in the same monochromatic rectangle

Lemma: if f has a fooling set of size t, then D(f) ≥ log2 t

ApplicationsEquality: Alice and Bob each hold x,y 2 {0,1}n

– want to decide whether x=y or not.• Fooling set for Equality

S={(w,w)|w 2 {0,1}n } Conclusion: D(Equality) ¸ n

Disjointness: let x,y µ {1,…,n} and let– DISJ(x,y)=1 if |x y|¸ 1 and – DISJ(x,y)=0 otherwise

• Fooling set for Disjointness S={(A,comp(A))|A µ {1,…,n} }

Conclusion: D(DISJ) ¸ n

Applications to Time-Space lower bounds in Turing Machines

Let Palindromes={xxR| x 2 *}• Palindromes 2 TIME(n)• Palindromes 2 SPACE(log n)But what about simultaneous linear time and log space?

Lemma: let f: {0,1}n x {0,1}n {0,1} be a function with communication complexity D(f) . For any multi-tape TM running in time T(n) and space S(n) accepting all string

{x0ny| |y|=|x|=n, f(x,y)=1}and rejecting all strings

{x0ny| |y|=|x|=n, f(x,y)=0}

We have D(f) = O(T(n) S(n)/n) Proof: joint simulation of the TM by Alice and Bob.

To get time-space lower bound on Palindromes use EqualityConsider {x0nxR| x 2 *, |x|=n}

Rank lower bound

• For f:X x Y {0,1} let Mf tbe the |X| x |Y| matrix where entry f(x,y) has value f(x,y). The rank of f, rank(f), is the rank of Mf over the reals.

Theorem: For any f:X x Y {0,1} D(f) ≥ log2 rank(f)

Proof: For any protocol P Mf =ℓ leaf Mℓ Mℓ is the matrix corresponding to the rectangle of leaf ℓ

Examples: – Equality MEquality is the identity matrix – rank is 2n

Inner Product• Let x,y 2 {0,1}n

IP(x,y)= i=1n xi yi mod 2

What is rank(MIP)?

Let N=MIP MIP. Then entry (x,y) in N is z 2 {0,1}

n <x,z> <y,z> which is # z’s where <x,z>=<y,z>=1

– 2n-2 if x≠y– 2n-1 if x=y– 0 if x or y is 0Hence rank(N) ¸ 2n-1. Since rank(MIP MIP) · min{rank(MIP},rank(MIP}}

rank(MIP} ¸ 2n-1 and D(IP) ¸ n

Non-determinism and covers • Can a protocol tree be very unbalanced?

– Technique for balancing: given a protocol with t leaves there is a protocol with communication complexity O(log t)

Is the monotone rectangles lower bound tight?• Consider instead of

partition into monochromatic rectangles a cover by monochromatic rectangles

The rectangles are not necessary disjointFor z 2 Z a z-cover handles only the inputs (x,y) where f(z,y)=zThis corresponds to non-deterministic complexity

A non-deterministic protocol for verifying that f(x,y)=z:Alice: Guess a rectangle R intersecting row x, send name to BobBob: verify that R intersects column y tell Alice Accept only if Bob approve Complexity: log # of z-rectangles