Invariance Principles
in Theoretical Computer Science
Ryan ’Donnell
Carnegie Mellon University
O
1. Describe some TCS results requiring
variants of the Central Limit Theorem.
Talk Outline
2. Show a flexible proof of the CLT
with error bounds.
3. Open problems and an advertisement.
1. Describe some TCS results requiring
variants of the Central Limit Theorem.
Talk Outline
2. Show a flexible proof of the CLT
with error bounds.
3. Open problems and an advertisement.
Linear Threshold Functions
Linear Threshold Functions
Learning Theory [O-Servedio’08]
Thm: Can learn LTFs f in poly(n) time,
just from correlations E[f(x)xi].
Key:
when all |ci| ≤ ϵ.
Property Testing [Matulef-O-Rubinfeld-Servedio’09]
Thm: Can test if is
ϵ-close to an LTF with poly(1/ϵ) queries.
Key:
when all |ci| ≤ ϵ.
Derandomization [Meka-Zuckerman’10]
Thm: PRG for LTFs with seed
length O(log(n) log(1/ϵ)).
Key:
even when xi’s not fully independent.
Multidimensional CLT?
when all small compared to
For
Derandomization+ [Gopalan-O-Wu-Zuckerman’10]
Thm: PRG for “functions of O(1) LTFs”
with seed length O(log(n) log(1/ϵ)).
Key: Derandomized multidimensional CLT.
Property Testing+ [Blais-O’10]
Thm: Testing if is a
Majority of k bits needs kΩ(1) queries.
Key:
assuming E[Xi] = E[Yi], Var[Xi] = Var[Yi],
and some other conditions.
(actually, a multidimensional version)
Social Choice,Inapproximability [Mossel-O-Oleszkiewicz’05]
Thm: a) Among voting schemes where no
voter has unduly large influence,
Majority is most robust to noise.
b) Max-Cut is UG-hard to .878-approx.
Key: If P is a low-deg. multilin. polynomial,
assuming P has “small coeffs. on each coord.”
1. Describe some TCS results requiring
variants of the Central Limit Theorem.
Talk Outline
2. Show a flexible proof of the CLT
with error bounds.
3. Open problems and an advertisement.
Gaussians
Standard Gaussian: G ~ N(0,1). Mean 0, Var 1.
a + bG also a “Gaussian”: N(a,b2)
Sum of independent Gaussians is Gaussian:
If G ~ N(a,b2), H ~ N(c,d2) are independent,
then G + H ~ N(a+c,b2+d2).
Anti-concentration: Pr[ G ∈ [u−ϵ, u+ϵ] ] ≤ O(ϵ).
X1, X2, X3, … independent, ident. distrib.,
mean 0, variance σ2,
Central Limit Theorem (CLT)
CLT with error bounds
X1 + · · · + Xnis “close to” N(0,1),
assuming Xi is not too wacky.
X1, X2, …, Xn independent, ident. distrib.,
mean 0, variance 1/n,
wacky:
Niceness of random variables
Say E[X] = 0, stddev[X] = σ.
eg: ±1. N(0,1). Unif on [-a,a].
not nice:
def: (≥ σ).
“def”: X is “nice” if
Niceness of random variables
Say E[X] = 0, stddev[X] = σ.
eg: ±1. N(0,1). Unif on [-a,a].
not nice:
def: (≥ σ).
def: X is “C-nice” if
Y “ϵ-close” to Z:
Berry-Esseen Theorem
X1, X2, …, Xn independent, ident. distrib.,
mean 0, variance 1/n,
X1 + · · · + Xnis ϵ-close to N(0,1),
assuming Xi is C-nice, where
[Shevtsova’07]: .7056
General Case
X1, X2, …, Xn independent, ident. distrib.,
mean 0,
X1 + · · · + Xnis ϵ-close to N(0,1),
assuming Xi is C-nice,
Berry-Esseen: How to prove?
1. “Characteristic functions”
2. “Stein’s method”
3. “Replacement” = think like a cryptographer
X1, X2, …, Xn indep., mean 0,
S = X1 + · · · + XnG ~ N(0,1).ϵ-close to
Indistinguishability of random variables
S “ϵ-close” to G:
Indistinguishability of random variables
S “ϵ-close” to G:
u
Indistinguishability of random variables
S “ϵ-close” to G:
ut
Indistinguishability of random variables
S “ϵ-close” to G:
Replacement method
S “ϵ-close” to G:
uδ
Replacement method
X1, X2, …, Xn indep., mean 0,
S = X1 + · · · + Xn
G ~ N(0,1)
For smooth
Replacement method
X1, X2, …, Xn indep., mean 0,
G = G1 + · · · + Gn
For smooth
S = X1 + · · · + Xn
Hybrid argument
X1, X2, …, Xn indep., mean 0,
SY = Y1 + · · · + Yn
For smooth
SX = X1 + · · · + Xn
Invariance principle
Y1, Y2, …, Yn Var[Xi] = Var[Yi] =
Hybrid argument
Def: Zi = Y1 + · · · + Yi + Xi+1 + · · · + Xn
SX = Z0, SY = Zn
X1, X2, …, Xn, Y1, Y2, …, Yn, independent,
matching means and variances.
SX = X1 + · · · + Xn SY = Y1 + · · · + Ynvs.
Hybrid argument
Zi = Y1 + · · · + Yi + Xi+1 + · · · + Xn
Goal:
X1, X2, …, Xn, Y1, Y2, …, Yn, independent,
matching means and variances.
Zi = Y1 + · · · + Yi + Xi+1 + · · · + Xn
Zi = Y1 + · · · + Yi + Xi+1 + · · · + Xn
where U = Y1 + · · · + Yi−1 + Xi+1 + · · · + Xn.
Note: U, Xi, Yi independent.
Goal:
−
=
by indep. and matching means/variances!
∴
Variant Berry-Esseen: Say
If X1, X2, …, Xn & Y1, Y2, …, Yn indep.
and have matching means/variances, then
Usual Berry-Esseen:
If X1, X2, …, Xn indep., mean 0,
uδ
Hack
Usual Berry-Esseen:
If X1, X2, …, Xn indep., mean 0,
Variant Berry-Esseen
+ Hack
Usual Berry-Esseen
except with error O(ϵ1/4)
Extensions are easy!
Vector-valued version:
Use multidimensional Taylor theorem.
Derandomized version:
If X1, …, Xm C-nice, 3-wise indep., then
X1+···+ Xm is O(C)-nice.
Higher-degree version:
X1, …, Xm C-nice, indep., Q is a deg.-d poly.,
then Q(X1, …, Xm) is O(C)d-nice.
1. Describe some TCS results requiring
variants of the Central Limit Theorem.
Talk Outline
2. Show a flexible proof of the CLT
with error bounds.
3. Open problems, advertisement, anecdote?
Open problems
1. Recover usual Berry-Esseen via the
Replacement method.
2. Vector-valued: Get correct dependence
on test sets K. (Gaussian surface area?)
3. Higher-degree: improve (?) the
exponential dependence on degree d.
4. Find more applications in TCS.
Do you like LTFs and PTFs?
Do you like probability and geometry?
Oct. 21-22 (“just before FOCS”) workshopat the Princeton Intractability Center:
Analysis and Geometry of Boolean Threshold Functions
Diakonikolas! Kane! Meka! Rubinfeld! Servedio! Shpilka! Vempala! And more!
http://intractability.princeton.edu/blog/2010/08/workshop-ltfptf/