a 3-query pcp over integers a.k.a solving sparse linear systems prasad raghavendra venkatesan...
TRANSCRIPT
A 3-Query PCP over integersa.k.a
Solving Sparse Linear Systems
Prasad Raghavendra
Venkatesan Guruswami
Linear Equations
Given a system of linear equations over reals,Find a solution.
.Easy,
Use Gaussian elimination.
Noise?
Given a set of linear equations for which there is a solution satisfying 99% of the equations,
What is the best solution that can be efficiently found?
Can we atleast satisfy 1% of the equations?
10 years ago [Håstad STOC97, JACM 01]
For any prime p, ε > 0, given a set of linear equations modulo p , it is NP-hard to distinguish between:
• (1 – ε) – fraction of the equations can be satisfied.• 1/p + ε – fraction of the equations can be satisfied.
All equations are of the form Xi + Xj = Xk + c (mod p)
X1 + X2 = X3 + 10 (mod p)X1 + X3 = X5 + 17 (mod p)X9 + X4 = X3 + 23 (mod p)X11 + X2 = X31 + 1 (mod p)X1 + X2 = X7 - 1 (mod p)X1 + X3 = X8 + p-10 (mod p)……..……..X9 + X7 = X3 + p/2 (mod p)X5 + X2 = X7 + 10 (mod p)
It is a 3-Query Probabilistically Checkable Proof system for NP
Just have to read values of 3 variables to check an equation.
Håstad’s 3-Query PCP[STOC97, JACM 01]
Can be verified by 3 queries
Reals?
NP-hard[Guruswami-Raghavendra 06, Feldman-Gopalan-Khot-Ponnuswami 06]For any ε,δ > 0, Given a set of linear equations over reals, it is NP-hard to distinguish between the following two cases:• There is a solution that satisfies 1 – ε fraction of the equations.• No solution satisfies more than δ fraction of the equations.
Unlike Hastad’s result, equations are not sparse
Sparse Equations?
• Solving sparse systems of equations important for many applications.
• In the spirit of PCP theorem..
• Sparse equations have important connections to PCPs, linearity testing, Unique Games conjecture.
Sparse Equations over Reals
For any ε,δ > 0, Given a set of sparse linear equations, it is NP-hard to distinguish between:
• (1 – ε) – fraction of the equations can be satisfied.• δ – fraction of the equations can be satisfied. X1 + X2 = X3 + 10
X1 + X3 = X5 + 17…X9 + X4 = X3 + 23 X2 + X6 = X7 + 27
Some fixed constant accuracy, say ±1
Label Cover Problem
U, V : set of verticesE : set of edges{1,2… R} : set of labels πe: constraint on edge e
An assignment A satisfies an edge e = (u,v) E if
πe (A(u)) = A(v)
123..R
123..Rπe
U V
u
v
Find an assignment A that satisfies maximum number of edges
3
π e (3)=2
7
5
2
3
3
1
4
1
2
5
6
Label Cover with Long Codes
πe
7
5
2
3
3
1
4
1
2
5
6
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
Write Long Codes of the Labels instead of the labels itself
Long Code
A long code over a finite field F is a function:
Gi : F X F … X F XF F
Gi(x1 , x2, … xn ) = xi
• n different long codes.• Long code over Fp represented by a table of pn values.
• Linear Function.
Extending Hastad’s result to integers
πe
7
5
2
3
3
1
4
1
2
5
6
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0 X1
X2
G2 (x1 , x2 ) = x2
A Long Code over integers is an infinite object.
Use long code over integers
Just Truncate the long code!
Core Problem
4 4 4 4 43 3 3 1 32 2 1 2 21 1 1 1 10 0 0 0 0 X1
X2
0 1 2 3 40 1 2 3 40 1 2 3 40 1 2 3 40 1 2 3 4 X1
X2
G2 (x1 , x2 ) = x2 G1 (x1 , x2 ) = x1
?=
Given two supposed long codes, query 3 locations and test if they are close to some long code
If test succeeds, must decode a small set of possible labels
Linearity Testing
Given a function from an group G1 to group G2 (both abelian)
A : G1 -> G2
Pick x,y uniformly at random from G1
Test if
A(x) + A(y) = A(x+y)
[Blum-Luby-Rubinfeld] With G1 = {0,1}n ,G2 = {0,1}, if A is δ- far from linear function, then the test rejects with probability at least δ
Derandomized Linearity Testing
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
For sufficiently large primes p,
Linearity testing on truncated long code
= Testing modulo p
This should imply a derandomized linearity test.Total randomness used independent of the prime.
p
Fourier Analysis
• One Fourier coefficient corresponding to every linear function
Pω(x) = ω•x for ω =(ω1 , ω2 ,… ωR ) in FpR
Â(ω) measures similarity with Pω(x) = ω•x
Â(ω) = E[ A(x)e-iω•x ]
Hastad’s Decoding
Pick a large Fourier coefficient Â(ω) of the long code, randomly pick a nonzero coordinate ωi
Decode to label iNot too many large Fourier Coefficients
Parseval’s Identity
Obstacle
The distribution is not uniform, so a Fourier coefficient that appears is
There could be exponentially many large Fourier coefficients!
Decoding Labels
Pick a large Fourier coefficient AP(ω) , randomly pick one of its large coordinate ωi
Assign label i to the vertex
•All large Fourier coefficients in the same cluster, will yield the same label with high probability.
• There are very few clusters, so there are very few possible choices
Conclusion
• Sparse linear equations over real numbers are hard to solve even with little noise.
• In a weak sense, complete Derandomization of linearity testing is possible.
• Two variable linear equations over reals?
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
πAB
Randomly pick a vector x = (x1,x2,.. xR)
Definex o π = (xπ(1), xπ(2) , xπ(3) ….xπ(R))
Test if a(x o π) = b(x)
Testing an EdgeI will just assign 0
to everything!
Testing an Edge
Randomly pick a vector x = (x1,x2,.. xR)
Definex o π = (xπ(1), xπ(2) , xπ(3) ….xπ(R))
A(x o π) = B(x)
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
πAB
For a long code a,
a(x + 1 ) = a(x) + 1
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0 X1
X2
G2 (x1 , x2 ) = x2
A(x o π – t11) + t1 = B(x – t21) + t2
I will give something that does not look
linear at all
Testing an Edge
Randomly pick a vector x = (x1,x2,.. xR)
Definex o π = (xπ(1), xπ(2) , xπ(3) ….xπ(R))
Randomly pick y = (y1,y2,.. yR)
Test ifa(x o π + y) – a(y) = b(x) + c
Long Code is a linear function!
a(x o π + y ) – a(y) = a(x o π)
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
πAB A(x) = (x1 + x2 + ...xR)/R
(ε,δ) – concentrated distribution
All Fourier coefficients of P that are 2πδ away from origin are bounded by ε
4πδ ε
Examples• Epsilon Biased Spaces over [0,1]n are (ε, ½) – concentrated.
• Epsilon Biased Spaces over Fp are (ε, 1/p) – concentrated. [BenSasson-Sudan-Vadhan-Widgerson] use Epsilon biased spaces to derandomize low degree tests(including linearity)
•Any sufficiently slowly decaying probability distribution over integers.
Hardness of Label Cover
There exists γ > 0 such thatGiven a label cover instance Г =(U,V,E,R,π), it is
NP-hard to distinguish between :• Г is completely satisfiable• No assignment satisfies more than 1/Rγ
fraction of the edges.
[Raz 98]
Testing an Edge
For a function π : [1,2,.. R] -> [1,2..R]
A vector x = (x1,x2,.. xR) Definex o π = (xπ(1), xπ(2) , xπ(3) ….xπ(R))
a(x o π) = b(x)
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
πeab
1
2
1
2
πe
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0
X1
X2
A (x1 , x2 ) = x2
0 1 2 3 40 1 2 3 40 1 2 3 40 1 2 3 40 1 2 3 4
X1
X2
B (x1 , x2 ) = x1
a(2, 4) = b(4,2)
A Linear Equation on Long code symbols
Hastad’s 3 Query PCP
Randomly pick a vector xDefine
x o π = (xπ(1), xπ(2) , xπ(3) ….xπ(R))
Randomly pick yPerturb each coordinate of x o π + y independently with
probability ε. To perturb just change the value to anything else in Fp
Test ifa(x o π + y+μ) – a(y) = b(x) + c
Long codes/Dictator functions are stable against noise in the coordinates
Arithmetization
Define A(x) = ωa(x) = e2πia(x)/p
B(y) = ωb(y) = e2πib(y)/p
Then : a(x o π + y+μ) – a(y) - b(x) = 0if and only if
1/p ∑ (A(y)B(x) A(x o π + y+μ) )j = 1
Soundness Argument
• As Linearity is tested, a(x) must have some similarity to a linear function. There have to be large Fourier coefficients Â(ω)
• As we force a(x + 1) = a(x) + 1 the function a(x) is not similar to constant function. Thus, there are some nonzero ω with large Â(ω)
• There are large Â(ω) with ω having few non-zero labels.
Obstacles
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0 X1
X2
G2 (x1 , x2 ) = x2
Truncated region no more a group.
For a constant fraction of x and y, (x+y) is outside the region.
There could be exponentially many (in the dimension of space) large Fourier coefficients.
Modified TestP, P’ be decaying and (ε,δ)- concentrated distributions
Pick x from distribution PPick y from distribution P’
Perturb each coordinate of x o π + y independently with
probability ε. To perturb, just change the value by a random number < M
Test ifA(x o π + y+μ) – A(y) = B(x) + c
P’
P
P much more flatter than P’
Fourier Analysis (continued)
Inverse Fourier Transform
Not too many large Fourier Coefficients
Parseval’s Identity
Properties
• For a long code a,
a(x + 1 ) = a(x) + 1
• Long codes/Dictator functions are stable against noise in the coordinates.
4 4 4 4 43 3 3 3 32 2 2 2 21 1 1 1 10 0 0 0 0 X1
X2
0 1 2 3 40 1 2 3 40 1 2 3 40 1 2 3 40 1 2 3 4 X1
X2
G2 (x1 , x2 ) = x2
G1 (x1 , x2 ) = x1
Two dimensional long codes over F5