efficientlydecodingreedmullercodes fromrandomerrorsbenleevolk/pdf/rm_slides.pdf · errors:...

Efficiently Decoding Reed Muller CodesFrom Random Errors

Ben Lee Volk

Joint work with

Ramprasad SaptharishiAmir Shpilka

Tel Aviv University

A game!Given the truth-table of a polynomial f ∈ F[x1, . . . , xm] of degree≤ r, with 1/2− o(1) of the entries flipped, recover f efficiently.

1 0 1 1 0 0 01 0 1 1 0 0 0 1 1


0 1 0 0 1 1 11 0 1 1 0 0 0 1 1


0 1 0 0 1 1 11 0 1 1 0 0 0 1 1

AdversarialErrors: Impossible.


0 1 0 0 1 1 11 0 1 1 0 0 0 1 1

AdversarialErrors: Impossible.RandomErrors?(each coordinate flipped independently with probability 1/2− o(1))


0 1 0 0 1 1 11 0 1 1 0 0 0 1 1


Theorem: There is an efficient algorithm to recover f ,even for r = o(

pm).


0 1 0 0 1 1 11 0 1 1 0 0 0 1 1


Theorem: There is an efficient algorithm to recover f ,even for r = o(

pm).

This talk is about decoding Reed-Muller codes fromrandom errors.

Reed-Muller Codes: RM(m, r)• Messages: (coefficient vectors of) degree ≤ r polynomials

f ∈ F2[x1, . . . , xm]

• Encoding: evaluations over Fm2

f (x1, x2, x3, x4) = x1 x2 + x3 x47→

0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 0

A linear code, with• Block Length: 2m := n

• Distance: 2m−r (lightest codeword: x1 x2 · · · x r )• Dimension:

�m0

�+�m

1

�+ · · ·+ �mr � := � m≤r

�• Rate: dimension/block length =

� m≤r

�/2m


f ∈ F2[x1, . . . , xm]


f (x1, x2, x3, x4) = x1 x2 + x3 x47→

0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 0

A linear code, with

• Block Length: 2m := n


�m0

�+�m

1

�+ · · ·+ �mr � := � m≤r


� m≤r

�/2m


f ∈ F2[x1, . . . , xm]


f (x1, x2, x3, x4) = x1 x2 + x3 x47→

0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 0



�m0

�+�m

1

�+ · · ·+ �mr � := � m≤r


� m≤r

�/2m


f ∈ F2[x1, . . . , xm]


f (x1, x2, x3, x4) = x1 x2 + x3 x47→

0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 0


• Distance: 2m−r (lightest codeword: x1 x2 · · · x r )

• Dimension:�m

0

�+�m

1

�+ · · ·+ �mr � := � m≤r


� m≤r

�/2m


f ∈ F2[x1, . . . , xm]


f (x1, x2, x3, x4) = x1 x2 + x3 x47→

0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 0



�m0

�+�m

1

�+ · · ·+ �mr � := � m≤r

�

• Rate: dimension/block length =� m≤r

�/2m


f ∈ F2[x1, . . . , xm]


f (x1, x2, x3, x4) = x1 x2 + x3 x47→

0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 0



�m0

�+�m

1

�+ · · ·+ �mr � := � m≤r


� m≤r

�/2m

More Properties of Reed-Muller CodesGenerator Matrix: evaluation matrix of deg≤ r monomials:

v ∈ Fm2

M

:= E(m, r)M(v)

(Every codeword is spanned by the rows.)

Linear codes can be also defined by their parity check matrix: amatrix H whose kernel is the code.Duality: RM(m, r)⊥ = RM(m, m− r − 1).=⇒ parity check matrix of RM(m, r) is E(m, m− r − 1).


v ∈ Fm2

M

:= E(m, r)M(v)


Linear codes can be also defined by their parity check matrix: amatrix H whose kernel is the code.

Duality: RM(m, r)⊥ = RM(m, m− r − 1).=⇒ parity check matrix of RM(m, r) is E(m, m− r − 1).


v ∈ Fm2

M

:= E(m, r)M(v)


Linear codes can be also defined by their parity check matrix: amatrix H whose kernel is the code.Duality: RM(m, r)⊥ = RM(m, m− r − 1).

=⇒ parity check matrix of RM(m, r) is E(m, m− r − 1).


v ∈ Fm2

M

:= E(m, r)M(v)


Linear codes can be also defined by their parity check matrix: amatrix H whose kernel is the code.Duality: RM(m, r)⊥ = RM(m, m− r − 1).=⇒ parity check matrix of RM(m, r) is E(m, m− r − 1).

Decoding Reed-Muller CodesWorstCaseErrors: Up to d/2 (d is minimal distance).

d d/2

(algorithm by Reed54)

ListDecoding: max radius with constant # of words

Gopalan-Klivans-Zuckerman08,Bhowmick-Lovett15:List decoding radius = d.


d d/2

(algorithm by Reed54)ListDecoding: max radius with constant # of words

Gopalan-Klivans-Zuckerman08,Bhowmick-Lovett15:List decoding radius = d.


d d/2

(algorithm by Reed54)ListDecoding: max radius with constant # of words

d Gopalan-Klivans-Zuckerman08,Bhowmick-Lovett15:List decoding radius = d.

Decoding Reed-Muller Codes: Average Case• Reed-Muller Codes are not very good with respect to

worst-case errors (can’t have constant rate and mindistance at the same time)

• What about the Shannon model of random corruptions?• Standard model in coding theory with recent breakthroughs

in the last few years (e.g. Arıkan’s polar codes)• An ongoing research endeavor: how do Reed-Muller

perform in Shannon’s random error model?



• What about the Shannon model of random corruptions?

• Standard model in coding theory with recent breakthroughsin the last few years (e.g. Arıkan’s polar codes)

• An ongoing research endeavor: how do Reed-Mullerperform in Shannon’s random error model?




in the last few years (e.g. Arıkan’s polar codes)

• An ongoing research endeavor: how do Reed-Mullerperform in Shannon’s random error model?




in the last few years (e.g. Arıkan’s polar codes)• An ongoing research endeavor: how do Reed-Muller

perform in Shannon’s random error model?

Models for random corruptions (channels)Binary Erasure Channel — BEC(p)

Each bit independently replaced by ’?’ with probability p

0 0 1 1 0



0 0 1 1 00 0 1



0 0 1 1 0? ? ?



0 0 1 1 0? ? ?

Binary Symmetric Channel — BSC(p)Each bit independently flipped with probability p



0 0 1 1 0? ? ?


0 0 1 1 00 0 1



0 0 1 1 0? ? ?


0 0 1 1 01 1 0



0 0 1 1 0? ? ?


0 0 1 1 01 1 0

(almost) equiv: fixed number t = pn of random errors



0 0 1 1 0? ? ?


0 0 1 1 01 1 0

(almost) equiv: fixed number t = pn of random errors

Shannon48: max rate that enables decoding (w.h.p.) is 1− p (forBEC) and 1−H(p) (for BSC). Codes achieving bound calledcapacity achieving.

Decoding Erasures• How many random erasures can RM(m, m− r − 1) tolerate?

• Fact: for any linear code, a subset S ⊆ [n] of erasures isuniquely decodable ⇐⇒ columns indexed by S inparity-check matrix are linearly independentdecoding algo: solve system of linear equations

• Reminder: PCM of RM(m, m− r − 1) is E(m, r)

• =⇒ can’t correct more than� m≤r

�erasures

(also follows from Shannon’s theorem)• Question: Can we correct (1− o(1))

� m≤r

�erasures?

• if yes, RM(m, m− r − 1) achieves capacity for the BEC

Decoding Erasures• How many random erasures can RM(m, m− r − 1) tolerate?• Fact: for any linear code, a subset S ⊆ [n] of erasures is

uniquely decodable ⇐⇒ columns indexed by S inparity-check matrix are linearly independentdecoding algo: solve system of linear equations

• Reminder: PCM of RM(m, m− r − 1) is E(m, r)

• =⇒ can’t correct more than� m≤r

�erasures

(also follows from Shannon’s theorem)• Question: Can we correct (1− o(1))

� m≤r

�erasures?

• if yes, RM(m, m− r − 1) achieves capacity for the BEC

Reed-Muller Codes and the BECFor v ∈ Fm

2 , let vr = the column indexed by v in E(m, r)(all evals of monoms of deg≤ r on v )

What’s the max t such that forrandomly picked u1, . . . ,ut ∈ Fm

2 ,ur

1, . . . ,urt are linearly independent?

E(m, r)




2 ,ur


E(m, r)

If t = (1− o(1))� m≤r

�, RM(m, m− r −1) achieves capacity for BEC.




2 ,ur


E(m, r)

If t = (1− o(1))� m≤r

�, RM(m, m− r −1) achieves capacity for BEC.

sanity check: r = 1 is good

Reed-Muller Codes and the BECMainQuestion: Does RM(m, r) achieve capacity for BEC?

Abbe-Shpilka-Wigderson15: YES if r = o(m) orr = m− o(p

m/ log m).Kumar-Pfister15, Kudekar-Mondelli-Şaşoğlu-Urbanke15:YES if rate is constant (i.e. r = m/2±O(

pm)).

m/20 m

o(m) o(p

m/ log m)O(p

m)

OpenProblem: Prove for every degree r.

Reed-Muller Codes and the BECMainQuestion: Does RM(m, r) achieve capacity for BEC?Abbe-Shpilka-Wigderson15: YES if r = o(m) orr = m− o(p

m/ log m).

Kumar-Pfister15, Kudekar-Mondelli-Şaşoğlu-Urbanke15:YES if rate is constant (i.e. r = m/2±O(

pm)).

m/20 m

o(m) o(p

m/ log m)O(p

m)


Reed-Muller Codes and the BECMainQuestion: Does RM(m, r) achieve capacity for BEC?Abbe-Shpilka-Wigderson15: YES if r = o(m) orr = m− o(p

m/ log m).Kumar-Pfister15, Kudekar-Mondelli-Şaşoğlu-Urbanke15:YES if rate is constant (i.e. r = m/2±O(

pm)).

m/20 m

o(m) o(p

m/ log m)O(p

m)


Decoding Erasures to Decoding ErrorsHow is this relevant for error correction?

Theorem: (ASW) Any erasure pattern correctable from erasuresin RM(m, m− r − 1) is correctable from errors inRM(m, m− 2r − 2).Thistalk: There is an efficient algorithm that corrects fromerrors, in RM(m, m− 2r − 2), any pattern which is correctablefrom erasures in RM(m, m− r − 1).(+ extensions to larger alphabets and other codes)

m/20 m

o(m) o(p

m/ log m)O(p

m)

Decoding Erasures to Decoding ErrorsHow is this relevant for error correction?Theorem: (ASW) Any erasure pattern correctable from erasuresin RM(m, m− r − 1) is correctable from errors inRM(m, m− 2r − 2).

Thistalk: There is an efficient algorithm that corrects fromerrors, in RM(m, m− 2r − 2), any pattern which is correctablefrom erasures in RM(m, m− r − 1).(+ extensions to larger alphabets and other codes)

m/20 m

o(m) o(p

m/ log m)O(p

m)

Decoding Erasures to Decoding ErrorsHow is this relevant for error correction?Theorem: (ASW) Any erasure pattern correctable from erasuresin RM(m, m− r − 1) is correctable from errors inRM(m, m− 2r − 2).Thistalk: There is an efficient algorithm that corrects fromerrors, in RM(m, m− 2r − 2), any pattern which is correctablefrom erasures in RM(m, m− r − 1).

(+ extensions to larger alphabets and other codes)

m/20 m

o(m) o(p

m/ log m)O(p

m)

Decoding Erasures to Decoding ErrorsHow is this relevant for error correction?Theorem: (ASW) Any erasure pattern correctable from erasuresin RM(m, m− r − 1) is correctable from errors inRM(m, m− 2r − 2).Thistalk: There is an efficient algorithm that corrects fromerrors, in RM(m, m− 2r − 2), any pattern which is correctablefrom erasures in RM(m, m− r − 1).(+ extensions to larger alphabets and other codes)

m/20 m

o(m) o(p

m/ log m)O(p

m)

Decoding Errors in RM CodesCorollary#1: (low-rate) efficient decoding algo for (1

2 − o(1))nrandom errors in RM(m, o(

pm)) (min distance is 2m−pm) .

Corollary#2: (high-rate) efficient decoding algo for(1− o(1))� m≤r

�random errors in RM(m, m− r) if r = o(

pm/ log m)

(min distance is 2r ) .Corollary#3: If RM(m,

�1+ρ

2

�m) achieves capacity, efficient

decoding algo for 2h�

1−ρ2

�m random errors in RM(m,ρm).

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

ρ

# of

erro

rs c

orre

cted

1−ρh� 1−ρ

2

�






pm/ log m)

(min distance is 2r ) .

Corollary#3: If RM(m,�

1+ρ2



1−ρ2


0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

ρ

# of

erro

rs c

orre

cted

1−ρh� 1−ρ

2

�






pm/ log m)

(min distance is 2r ) .Corollary#3: If RM(m,

�1+ρ

2



1−ρ2


0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

ρ

# of

erro

rs c

orre

cted

1−ρh� 1−ρ

2

�

Proof IdeaGoal: decode in RM(m, m− 2r − 2) every pattern which iscorrectable from erasures in RM(m, m− r − 1).

0 0 1 1 01 1 0

recall: {u1, . . . ,ut} correctable from erasures iff {ur1, . . . ,ur

t } arelinearly independent.

E(m, r)

Dual PolynomialsFact: If {ur

1, . . . ,urt } lin. indep., ∃ polys { f1, . . . , ft} of deg≤ r

such that

fi(u j) =

¨1 i = j

0 i ̸= j.

Proof: —— ur1 ——...

—— urt ——

· |fi|

= ei

Solve this system for fi.

Our approach would be to find those polynomials.

Algorithm: Solve a system of Linear EquationsU = {u1, . . . ,ut} ∈ Fm

2 (unknown) set of error locations.

MainClaim: For every v ∈ {0,1}m, v ∈ U ⇐⇒ the followinglinear system, in unknown coeffs of degree r poly f , is solvable:∀ monomial M , deg M ≤ r:

t∑i=1

f (ui) = f (v) = 1 ( f non-trivial)t∑

i=1( f ·M)(ui) = M(v) (v spanned by U )

t∑i=1( f ·M · (xℓ + vℓ + 1))(ui) = M(v)

(v spanned by vectors in U that agree with its ℓ-th coordinate)If U lin. indep. and v= ui ∈ U , fi is a solution. Conversely, ifsolvable and U lin. indep., can show v ∈ U .



MainClaim: For every v ∈ {0,1}m, v ∈ U ⇐⇒ the followinglinear system, in unknown coeffs of degree r poly f , is solvable:

∀ monomial M , deg M ≤ r:t∑

i=1f (ui) = f (v) = 1 ( f non-trivial)

t∑i=1( f ·M)(ui) = M(v) (v spanned by U )

t∑i=1( f ·M · (xℓ + vℓ + 1))(ui) = M(v)





t∑i=1



t∑i=1( f ·M · (xℓ + vℓ + 1))(ui) = M(v)





t∑i=1

f (ui) = f (v) = 1 ( f non-trivial)

t∑i=1( f ·M)(ui) = M(v) (v spanned by U )

t∑i=1( f ·M · (xℓ + vℓ + 1))(ui) = M(v)





t∑i=1



t∑i=1( f ·M · (xℓ + vℓ + 1))(ui) = M(v)





t∑i=1



t∑i=1( f ·M · (xℓ + vℓ + 1))(ui) = M(v)

(v spanned by vectors in U that agree with its ℓ-th coordinate)

If U lin. indep. and v= ui ∈ U , fi is a solution. Conversely, ifsolvable and U lin. indep., can show v ∈ U .




t∑i=1



t∑i=1( f ·M · (xℓ + vℓ + 1))(ui) = M(v)


Setting up system of EquationsEvery coefficient in the system is of the form

∑ti=1 g(ui) for poly g

of degree ≤ 2r + 1.

How to compute the coefficients?

Input to the algo: y= c+ e with c ∈ RM(m, m− 2r − 2), and echaracteristic vector of U = {u1, . . . ,ut}.Syndrome of y: E(m, 2r + 1) · y= E(m, 2r + 1) · e.(recall: E(m, 2r + 1) is PCM of RM(m, m− 2r − 2) )

Corollary: The syndrome of y is a� m≤2r+1

�long vector α, where

αM =∑t

i=1 M(ui), for every monom M , deg M ≤ 2r + 1.



of degree ≤ 2r + 1.How to compute the coefficients?




αM =∑t





Input to the algo: y= c+ e with c ∈ RM(m, m− 2r − 2), and echaracteristic vector of U = {u1, . . . ,ut}.

Syndrome of y: E(m, 2r + 1) · y= E(m, 2r + 1) · e.(recall: E(m, 2r + 1) is PCM of RM(m, m− 2r − 2) )



αM =∑t








αM =∑t


Back to BECThe algorithm works whenever ur

1, . . . ,urt lin. indep., which

happens (w.h.p.) whenever RM(m, m− r − 1) achieves capacity.

Restating the main question: for which values of r, ur1, . . . ,ur

t arelinearly independent with high probability for t = (1− o(1))

� m≤r

�?

m/20 m

o(m) o(p

m/ log m)O(p

m)

SummaryDecoding algo for RM far beyond the minimal distance, both forhigh-rate and low-rate regimes.

OpenProblems:• Prove RM achieves capacity for BEC for all degrees

• RM for BSC: much less is known! (ASW proved it achievescapacity for small rates)

m/20 m


OpenProblems:• Prove RM achieves capacity for BEC for all degrees• RM for BSC: much less is known! (ASW proved it achieves

capacity for small rates)

m/20 m


OpenProblems:• Prove RM achieves capacity for BEC for all degrees• RM for BSC: much less is known! (ASW proved it achieves

capacity for small rates)

m/20 m

Thank You

efficientlydecodingreedmullercodes fromrandomerrorsbenleevolk/pdf/rm_slides.pdf · errors:...

Documents