Locally Decodable Codes from Locally Decodable Codes from Nice Subsets of Finite Fields andNice Subsets of Finite Fields and

Prime Factors of Mersenne Numbers Prime Factors of Mersenne Numbers

Kiran KedlayaKiran Kedlaya Sergey YekhaninSergey YekhaninMITMIT Microsoft ResearchMicrosoft Research

An InequalityAn Inequality

Error Correcting CodesError Correcting Codes

In classical error correcting codes decoder needs to In classical error correcting codes decoder needs to process the whole (corrupted) codeword to recover process the whole (corrupted) codeword to recover even a single bit of the original message!even a single bit of the original message!

00 00 11 00 …… 00 11 11

00 11 …… 00 11

00 11 11 00 …… 00 00 11

n bit message

N bit codewordAdversarial

noise

Decoder processes the (corrupted) codeword

Locally Decodable CodesLocally Decodable Codes

Definition: Definition: A code C encoding n bits to N bits is called k-LDC if given a (linearly) corrupted codeword one can recover any particular bit of the message (w.h.p.) by reading only k randomly chosen bits.

00 00 11 00 …… 00 11 11

00 11 …… 00 11

00 11 11 00 …… 00 00 11

n bit message

N bit codewordAdversarial

noise

Decoder reads only k bits

Codes with sub-linear decoding complexity!

Locally Decodable CodesLocally Decodable Codes

• Example: There is a Example: There is a 22-query LDC of length Exp(n).-query LDC of length Exp(n).

• Major question: Major question:

What is the length of optimal What is the length of optimal kk-query LDCs?-query LDCs?

• Applications: Applications:

– Cryptography (private information retrieval).Cryptography (private information retrieval).

– Worst-case to average case reductions.Worst-case to average case reductions.

– Fault tolerant computation.Fault tolerant computation.

– Data transmission / storage.Data transmission / storage.

LDCs: progress in boundsLDCs: progress in bounds

• 2-query: Tight bound - Exp(n) [KdW].

• 3-query: Lower bound: - ΩΩ(n(n2 2 / log log n) [W]. / log log n) [W].

Upper bounds:- Exp(nExp(n1/21/2) [BIK]. (P) [BIK]. (Polynomial interpolation.)olynomial interpolation.)

- - Exp(nExp(n1/t1/t), where 2), where 2tt-1 is prime-1 is prime [Y]. [Y]. ((Point removal method.)Point removal method.) Exp(nExp(n1/32,582,6571/32,582,657) - unconditionally.) - unconditionally. Exp(nExp(no(1)o(1)) - if there exist infinitely many Mersenne primes.) - if there exist infinitely many Mersenne primes.

• Goal: Obtain constant-query LDCs of length Exp(nExp(no(1)o(1)) ) unconditionally.unconditionally.

Mersenne primes

Primes

This workThis work

We undertake an in-depth study of the point removal We undertake an in-depth study of the point removal method of [Y] to answer two questions: method of [Y] to answer two questions:

• Are Mersenne primes essential to the method?Are Mersenne primes essential to the method?

• Has the method been pushed to its limit?Has the method been pushed to its limit?

Heart of the point removal methodHeart of the point removal method

• Definition: Definition: A set S A set S F Fqq is is t t - combinatorially nice if ….- combinatorially nice if ….

• Definition: Definition: A set S A set S F Fqq is is k k - algebraically nice if ….- algebraically nice if ….

• Theorem: Theorem: If for some FIf for some Fqq there exists S there exists S F Fqq such that: such that:

- S is tt-combinatorially nice and-combinatorially nice and

- S is - S is kk-algebraically nice; -algebraically nice;

then there exist then there exist kk-query LDCs of length Exp(n-query LDCs of length Exp(n1/1/tt).).

Lemma: Lemma: Let pLet p == 22tt-1 be a Mersenne prime; then S-1 be a Mersenne prime; then S = = {1,2,4,…,2{1,2,4,…,2tt-1-1} } in Fin Fpp is is tt-combinatorially nice and -combinatorially nice and 33-algebraically nice.-algebraically nice.

Are Mersenne primes essential?Are Mersenne primes essential?

Answer:Answer: No. No.

Mersenne numbers with large prime Mersenne numbers with large prime factors are good enough!factors are good enough!

Theorem: Theorem: Let Let > 0. If P(2> 0. If P(2tt-1) > (2-1) > (2tt-1)-1) = p; then = p; then

{1,2,…,2{1,2,…,2tt-1-1} } F Fpp is is tt-comb. nice and -comb. nice and kk(()-algebr. nice; thus)-algebr. nice; thus

exist exist kk(() – query LDCs of length ) – query LDCs of length Exp(nExp(n1/1/tt).).

Notation: Notation: P(P(mm) = the largest prime factor of ) = the largest prime factor of mm..

Primes

Large prime factors of Mersenne numbers

Mersenne primes

Has the method been pushed to its limit?Has the method been pushed to its limit?

Answer: Yes. Answer: Yes. Unless we progress on some old number theory questions. Unless we progress on some old number theory questions. Primes that are somewhat large factors of Mersenne numbers Primes that are somewhat large factors of Mersenne numbers are necessary!are necessary!

Theorem: Theorem: If for infinitely many If for infinitely many tt there is an F there is an Fqq and S and S F Fqq that is that is kk--algebraically nice and algebraically nice and tt-combinatorially nice; then infinitely often:-combinatorially nice; then infinitely often:

P(2P(2tt-1) > (-1) > ( tt // 22 ))1+11+1 // ((kk-2)-2)..

The largest function f(The largest function f(tt) for that P(2) for that P(2tt-1) -1) >> f( f(tt) unconditionally ) unconditionally infinitely often is: f(infinitely often is: f(tt) =) = t t log log22 tt / log log / log log tt. [Stewart]. [Stewart]

LDCs and factors of Mersenne numbersLDCs and factors of Mersenne numbers

P(2t-1) = 2t-1

P(2t-1) > (2(2tt-1)-1)

P(2t-1) > (( tt // 22 ))1+11+1 //

(k-2)(k-2)

P(2t-1) > t logg22 t / log log t

Sufficient

Necessary

Known

Goal: Obtain constant-query codes of subexponential length.

About the proofAbout the proof

• Mersenne numbers with large prime factors Mersenne numbers with large prime factors yield nice subsets.yield nice subsets.

• Nice subsets of finite fields yield Mersenne Nice subsets of finite fields yield Mersenne numbers with somewhat large prime factors.numbers with somewhat large prime factors.

(We will see a piece of the second (We will see a piece of the second proof.)proof.)

Nice subsets to large factors of Mersenne numbersNice subsets to large factors of Mersenne numbers

Claim: Claim: 33-algebraically nice subsets of prime fields yield -algebraically nice subsets of prime fields yield large prime factors of Mersenne numbers.large prime factors of Mersenne numbers.

Theorem: Suppose S Theorem: Suppose S F Fpp is is 33-algebraically nice; then -algebraically nice; then

- p | 22tt-1;-1;

- p - p >> 0.75 0.75 tt22..

Proof: two stepsProof: two steps

• S S F Fpp is is 33-algebraically nice; -algebraically nice;

then there exist then there exist 11 22 33 in C in Cpp such that: such that: 11 + + 22 + + 33 = = 0.0.

• There exist There exist 11 22 33 in C in Cpp such that: such that: 11 + + 22 + + 33 = = 0;0;

then then p | 22tt-1 and p-1 and p > > 0.25 0.25 tt22..

Notation: CNotation: Cp p - the set of p-th roots of unity in F- the set of p-th roots of unity in F22..

(We will go over the second step.)(We will go over the second step.)

Proof of the second step - IProof of the second step - I

Lemma: There exist Lemma: There exist 11 22 33 in C in Cpp such that: such that: 11 + + 22 + + 33 = 0; = 0;

then then p | 22tt-1 and p > 0.25 -1 and p > 0.25 tt22..

Proof:Proof:

• Let Let tt be the smallest such that C be the smallest such that Cp p F F2 2 . .

• p | 2p | 2tt-1; -1;

• Elements of Elements of CCpp \\ {1} are proper elements of {1} are proper elements of FF22 i.e., i.e.,

for for in C in Cpp \\ {1}, and f(x) in {1}, and f(x) in FF22[x], deg f < t: f([x], deg f < t: f() = 0.) = 0.

t

t

F2

Cp

t

Proof of the second step - IIProof of the second step - II

Proof (continued):Proof (continued):

• Let Let ii denote elements of C denote elements of Cpp.. 11 + + 22 + + 33 = 0; yields = 0; yields 44 = 1 + = 1 + 55..

– 44= = 22-1.-1.11 ; ; 55= = 22

-1.-1.33

• Fix Fix in C in Cpp such that (1+ such that (1+ ) is in C) is in Cpp..• Consider the set Z={Consider the set Z={a a (1(1 + + ))bb | a,b in [0 ,…, | a,b in [0 ,…, tt/2-1]}./2-1]}. a a (1(1 + + ))bb c c (1(1 + + ))dd else we would have: f( else we would have: f()) = = 0, where 0, where

deg fdeg f < < tt..

Thus, |Z| = (Thus, |Z| = (tt // 2)2)2 2 and hence pand hence p > > ((tt // 2)2)22 . .

Conclusions:Conclusions:

• Summary:Summary: Further progress on upper bounds for LDCs via point Further progress on upper bounds for LDCs via point removal method is tied to progress on lower bounds for removal method is tied to progress on lower bounds for prime factors of Mersenne numbers.prime factors of Mersenne numbers.

• Hopes:Hopes: – Progress in number theory problems.Progress in number theory problems.

– Broader generalizations of the method. (finite rings?) Broader generalizations of the method. (finite rings?)