circuit complexity meets the theory of randomness

Eric AllenderRutgers University

Circuit Complexity meets the Theory of Randomness

SUNY Buffalo, November 11, 2010

Eric Allender: Circuit Complexity meets the Theory of Randomness < 2 >

Today’s Goal:

To raise awareness of the tight connection between two fields:– Circuit Complexity– Kolmogorov Complexity (the theory of

randomness)

And to show that this is useful. More generally, to spread the news about

some exciting developments in the field of derandomization.


Derandomization

Probabilistic algorithm

Regular input Random bits b1,b2,…

Where do these random bits come

from?


Derandomization


Regular input Random bits b1,b2,…

Random bits are: Expensive

Low-quality Hard to find

Possibly non-existent!


Derandomization


Regular input PseudoRandom bits b1,b2,…

seed

Generator g


A Jewel of Derandomization [Impagliazzo, Wigderson, 1997]: If there is a

problem computable in time 2n that requires circuits of size 2εn, then there is a “good” generator that takes seeds of length O(log n).

Thus, for example, if your favorite NP-complete problem requires big circuits (as we expect), then any probabilistic algorithm can be simulated deterministically with modest slow-down.

(Run the generator on all of the poly-many seeds, and take the majority vote.)




Stated another way (and introducing some notation), a very plausible hypothesis implies P = BPP.




The proof is deep, and incorporates clever algorithms, deep combinatorics, and innovations in coding theory.

…and it provides some great tools that shed insight into the nature of randomness.


Randomness Which of these sequences did I obtain by

flipping a coin? 0101010101010101010101010101010 0110100111111100111000100101100 1101010001011100111111011001010 1/π =0.3183098861837906715377675267450 Each of these sequences is equally likely, in the sense of

probability theory – which does not provide us with a way to

talk about “randomness”.


Kolmogorov Complexity

C(x) = min{|d| : U(d) = x}– U is a “universal” Turing machine

Important property– Invariance: The choice of the universal

Turing machine U is unimportant (up to an additive constant).

x is random if C(x) ≥ |x|. CA(x) = min{|d| : UA(d) = x} Let’s make a connection between Kolmogorov

complexity and circuit complexity.


Circuit Complexity Let D be a circuit of AND and OR gates (with

negations at the inputs). Size(D) = # of wires in D.

Size(f) = min{Size(D) : D computes f} We may allow oracle gates for a set A, along

with AND and OR gates. SizeA(f) = min{Size(D) : DA computes f}


Oracle Gates An “oracle gate” for B in a circuit:

B


K-complexity ≈ Circuit Complexity

There are some obvious similarities in the definitions.

C(x) = min{|d| : U(d) = x} Size(f) = min{Size(D) : D computes f}



There are some obvious similarities in the definitions. What are some differences?

A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.

– Given any string x, let fx be the function whose truth table is the string of length 2log|x|, padded out with 0’s, and define Size(x) to be Size(fx).





A more fundamental difference:– C(x) is not computable; Size(x) is.


Lightning Review: Computability Here’s all we’ll need to know about

computability:– The halting problem, everyone’s favorite

uncomputable problem:– H = {(i,x) : Mi halts on input x}– Every r.e. problem is poly-time many-one

reducible to H.– One such r.e. problem: {x : C(x) < |x|}. – There is no time-bounded many-reduction in

other direction, but there is a (large) time t Turing reduction (and hence C is not computable).





A more fundamental difference:– C(x) is not computable; Size(x) is.

In fact, there is a fascinating history, regarding the complexity of the Size function.


MCSP MCSP = {(x,i) : Size(x) < i}. MCSP is in NP, but is not known to be NP-

complete. Some history: NP-completeness was

discovered independently by in the 70s by Cook (in North America) and Levin (in Russia).

Levin delayed publishing his results, because he wanted to show that MCSP was NP-complete, thinking that this was the more important problem.


MCSP MCSP = {(x,i) : Size(x) < i}. Why was Levin so interested in MCSP? In the USSR in the 70’s, there was great

interest in problems requiring “perebor”, or “brute-force search”. For various reasons, MCSP was a focal point of this interest.

It was recognized at the time that this was “similar in flavor” to questions about resource-bounded Kolmogorov complexity, but there were no theorems along this line.


MCSP MCSP = {(x,i) : Size(x) < i}. 3 decades later, what do we know about

MCSP? If it’s complete under the “usual” poly-time

many-one reductions, then BPP = P. MCSP is not believed to be in P. We showed:– Factoring is in BPPMCSP.– Every cryptographically-secure one-way

function can be inverted in PMCSP/poly.


So how can K-complexity and Circuit complexity be the same?

C(x) ≈ SizeH(x), where H is the halting problem.

For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for fx, where fx(i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable.

For the other direction, let SizeH(fx) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate DH and produce fx.


So how can K-complexity and Circuit complexity be the same?

C(x) ≈ SizeH(x), where H is the halting problem.

So there is a connection between C(x) and Size(x) …

…but is it useful? First, let’s look at decidable versions of

Kolmogorov complexity.


Time-Bounded Kolmogorov Complexity

The usual definition: Ct(x) = min{|d| : U(d) = x in time t(|x|)}. Problems with this definition– No invariance! If U and U’ are different

universal Turing machines, CtU and Ct

U’ have no clear relationship.

– (One can bound CtU by Ct’

U’ for t’ slightly larger than t – but nothing can be done for t’=t.)

No nice connection to circuit complexity!



Levin’s definition: Kt(x) = min{|d|+log t : U(d) = x in time t}. Invariance holds! If U and U’ are different

universal Turing machines, KtU(x) and KtU’(x) are within log |x| of each other.

And, there’s a connection to Circuit Complexity:– Let A be complete for E = Dtime(2O(n)). Then

Kt(x) ≈ SizeA(x).



Levin’s definition: Kt(x|φ) = min{|d|+log t : U(d,φ)=x in time t}. Why log t?– This gives an optimal search order for NP

search problems.– This algorithm finds satisfying assignments

in poly time, if any algorithm does:– On input φ, search through assignments y in

order of increasing Kt(y|φ).



Levin’s definition: Kt(x) = min{|d|+log t : U(d) = x in time t}. Why log t?– This gives an optimal search order for NP

search problems.– Adding t instead of log t would give every

string complexity ≥ |x|. …So let’s look for a sensible way to allow the

run-time to be much smaller.


Revised Kolmogorov Complexity C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff

b is the i-th bit of x} (where bit # i+1 of x is *).– This is identical to the original definition.

Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t}.– The new and old definitions are within O(log

|x|) of each other. Define KT(x) = min{|d|+t : for all i ≤ |x| + 1,

U(d,i,b) = 1 iff b is the i-th bit of x, in time t}.


Kolmogorov Complexity is Circuit Complexity

C(x) ≈ SizeH(x). Kt(x) ≈ SizeE(x). KT(x) ≈ Size(x). Other measures of complexity can be

captured in this way, too:– Branching Program Size ≈ KB(x) = min{|d|

+2s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s}.


Kolmogorov Complexity is Circuit Complexity

C(x) ≈ SizeH(x). Kt(x) ≈ SizeE(x). KT(x) ≈ Size(x). Other measures of complexity can be

captured in this way, too:– Formula Size ≈ KF(x) = min{|d|

+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t}, for an alternating Turing machine U.


…but is this interesting?

The result that Factoring is in BPPMCSP was first proved by observing that, in PMCSP, one can accept a large set of strings having large KT complexity: RKT = {x : KT(x) ≥ |x|}.

(Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPPT for any test T that distinguishes truly random strings from pseudorandom strings. RKT is just such a test T.


Given a hard f, g is good


Regular input PseudoRandom bits b1,b2,…

seed

Generator g


This idea has many variants.

Consider RKT, RKt, and RC. RKT is in coNP, and not known to be coNP hard. RC is not hard for NP under poly-time many-one

reductions, unless P=NP.– How about more powerful reductions?– Is there anything interesting that we could compute

quickly if C were computable, that we can’t already compute quickly?

– Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in

But RC is undecidable! Perhaps H is in P relative to RC?



Consider RKT, RKt, and RC. RKT is in coNP, and not known to be coNP hard. RC is not hard for NP under poly-time many-one

reductions, unless P=NP.– How about more powerful reductions? We show:

– PSPACE is in P relative to RC.

– NEXP is in NP relative to RC.

– Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in poly time, relative to RC [BFNV].

But RC is undecidable! Perhaps H is in P relative to RC?


Relationship between H and RC

Perhaps H is in P relative to RC? This is still open. It is known that there is a

computable time bound t such that H is in DTime(t) relative to RC [Kummer].

…but the bound t depends on the choice of U in the definition of C(x).

We also showed: H is in P/poly relative to RC. Thus, in some sense, there is an “efficient”

reduction of H to RC.



Consider RKT, RKt, and RC. What about RKt? RKt, is not hard for NP under poly-time many-

one reductions, unless E=NE.– How about more powerful reductions?

– We show: EXP = NP(RKt).

– …and RKt is complete for EXP under P/poly reductions.

– Open if RKt is in P!


A Very Different Complete Set

RKt, is not hard for NP under poly-time many-one reductions, unless E=NE.

And it’s provably not complete for EXP under poly-time many-one reductions.

Yet it is complete under P/poly reductions and under NP-Turing reductions.

First example of a “natural” complete set that’s complete only using more powerful reductions.


A Very Different Complete Set

RKt, is not hard for NP under poly-time many-one reductions, unless E=NE.

And it’s provably not complete for EXP under poly-time many-one reductions.

(Sketch): Assume that RKt is complete under poly-time many-one reductions, and let A be a language over 1* in EXP that’s not in P. (Such sets A exist.) Let f reduce A to RKt. Thus f(1n) is in RKt iff 1n is in A. But Kt(f(1n)) < O(log n), and thus the only way it can be in RKt is if f(1n) is short (O(log n) bits). Thus A is in P, contrary to assumption.


Playing the Game in PSPACE We can also define a space-bounded measure

KS, such that Kt(x) < KS(x) < KT(x). RKS is complete for PSPACE under BPP

reductions (and, in fact, even under “ZPP” reductions, which implies completeness under NP reductions and P/poly reductions).

[This makes use of an even stronger form of the Impagliazzo-Wigderson generator, which relies on the “hard” function being “downward self-reducible and random self-reducible”.]


Playing the Game Beyond EXP For essentially any “large” DTIME, NTIME, or

DSPACE complexity class C, one can show that the problem of computing (or approximating) the SIZEA function (where A is the standard complete set for C) is complete for C under P/poly reductions.

However, we obtain completeness under uniform notions of reducibility (such as NP- or ZPP- reducibility) only inside NEXP.

Is this inherent? (E.g., is the inclusion “NEXP is contained in NP(RC)” optimal?)


Some Speculation Is this inherent? (E.g., is the inclusion “NEXP

is contained in NP(RC)” optimal?) If so, then this could lead to characterizations

of complexity classes in terms of efficient reductions to the (undecidable) set RC.

…which would sufficiently strange, that it would certainly have to be useful!


Some Speculation Is this inherent? (E.g., is the inclusion “NEXP

is contained in NP(RC)” optimal?) If so, then this could lead to characterizations

of complexity classes in terms of efficient reductions to the (undecidable) set RC.

New development: evidence that complexity classes can be characterized in terms of reductions to RC (or RK).

We know: NEXP is in NP relative to RK (no matter which univ. TM we use to define K(x)).

No class bigger than EXPSPACE has this property.


Things to Remember There has been impressive progress in

derandomization, so that now most people in the field would conjecture that BPP = P.

The tools of derandomization can be viewed through the lens of Kolmogorov complexity, because of the close connection that exists between circuit complexity and Kolmogorov complexity.


Things to Remember This has led to new insights in complexity

theory, such as:– Clarification of the complexity of Levin’s Kt

function.– Presentation of fundamentally different

classes of complete sets for PSPACE and EXP.

– Clarification of the complexity of MCSP. It has also led to new insights in Kolmogorov

complexity (such as the efficient reduction of H to RC).


A Sampling of Open Questions Is MCSP complete for NP under P/poly

reductions? (The analogous sets in PSPACE, EXP, NEXP, etc. are complete under P/poly reductions.)

Is MCSP “complete” in some sense for cryptography? (I.e., can one show that secure crypto is possible ↔ MCSP is not in P/poly? The → direction is known. How about the ← direction? Can one build a crypto scheme based on MCSP?)


A Sampling of Open Questions Can one prove (unconditionally) that RKt is not

in P? (I know of no reason why this should be hard to prove.)– Note in this regard, that EXP = ZPP iff there

is a “dense” set in P that contains no strings of Kt complexity < √n.

Can one show that RKt is not complete for EXP under poly-time Turing reductions?


A Sampling of Open Questions Can one improve the inclusions:

– PSPACE is in P relative to RC.

– NEXP is in NP relative to RC. Can one show that H is not poly-time Turing

reducible to RC?

circuit complexity meets the theory of randomness

Documents