codes al-fec pour le canal à effacements : codes ldpc-staircase
TRANSCRIPT
1
Codes AL-FEC pour le canal à effacements : codes
LDPC-Staircase et Raptor
Vincent Roca (Inria, France)
4MMCSR – Codage et sécurité des réseaux 12 février 2016
❍ Copyright © Inria 2016
❍ license❍ Work distributed under Creative Commons (CC) -
Attribution-Noncommercial-No Derivative Works 2.0 http://creativecommons.org/licenses/by-nc-nd/2.0/fr/deed.en_US
2
Outline 1. the erasure channel 2. AL-FEC codes 3. what is a good AL-FEC code? The performance
metrics 4. example: LDPC-Staircase 5. example: Raptor(Q)™
3
The erasure channel ! erasure channel ❍ definition:
≠ BSC (binary symmetric) and AWGN channels…
❍ the integrity assumption is a strong hypothesis ❍ a received symbol is 100% guaranteed error free
❍ largely simplifies decoding: ❍ belief propagation decoding ⇒ iterative decoding ❍ maximum likelihood decoding ⇒ Gaussian elimination ❍ more details to come…
4
0 0
1 1
erased! a symbol either arrives to the destination, without any error… … or is erased and never received
The erasure channel… (cont’) ! where do we find erasure channels? ❍ the Internet is intrinsically an erasure channel
❍ an IP datagram is either received or erased ❍ usually caused by a router congestion, or a routing error
❍ because of Ethernet CRC or TCP/UDP/IP checksum verifications ❍ when the underlying layers failed to recover/identify errors,
the “packet” is eventually discarded by higher layers checks ❍ certain MAC layers can ask for retransmissions, but it's
usually not the case
5
The erasure channel… (cont’) ! where do we find erasure channels… (cont) ❍ due to an intermittent connection model
❍ caused by the environment (obstacle in wireless comm.) • a mobile terminals receives a subset of the packets sent
❍ caused by the application (e.g., if it crashes) • packets sent during the off-period are lost
❍ these situations are easily recovered with point-to-point connections
• TCP will do the job if the disconnection is short• if a new connection is needed, it’s sufficient to remember
where the download stopped and ask the sender to continue from that point
❍ …but it’s quite different if the content is broadcast/multicast • the same content is sent to 100,000s of receivers!
6
The erasure channel… (cont’) ! where do we find erasure channels… (cont) ❍ with distributed storage of a content (e.g., a file), when
a storage point fails ❍ RAID disks ❍ network-wide distributed storage, where a client selects a
subset of the storage node, each of them having a chunk of the content (source or repair data)
7
Quite different from PHY-layer FEC ! completely different assumptions!
❍ information bits coming from the upper layers, once encoded, form a codeword
8
PHY channel
introduces errors
PHY-level FEC
information bits (e.g. 8 bits)
<1,0,0,1,1,0,1,1>
codeword (e.g. 12 bits)
<1,0,0,1,1,0,1,1|0,1,0,0>
PHY-level FEC
information bits <1,0,0,1,1,0,1,1>
erroneous codeword <1,0,0,0,1,1,1,1,0,1,0,0>
information bits coded bits
modified bits
FEC encoding
FEC decoding
Outline 1. the erasure channel 2. AL-FEC codes 3. what is a good AL-FEC code? The performance
metrics 4. example: LDPC-Staircase 5. example: Raptor(Q)™
9
AL-FEC code definitions ! AL-FEC are codes for the erasure channel ❍ only! ❍ we always assume there’s no error in what we receive
❍ codes and codecs are not the same
10
Be careful: codes ≠ codecs
specifies the way repair symbols are generated
implementation of the encoder and decoder
AL-FEC code definitions… (cont’) ! the symbol is the data unit manipulated by the AL-
FEC codec ❍ usually a symbol is carried in a UDP/IP datagram
� either received or erased❍ NB: sometimes there are several symbols per UDP/IP datagram
• goal is to artificially increase the # of symbols in an object • useful since LDPC/Raptor codes perform better when there are
more than a few 1,000s of symbols ❍ let's keep it simple from now on… one symbol per packet
❍ initially k source symbols, n encoding symbols after FEC encoding
11
AL-FEC code definitions… (cont’) ❍ example:
❍ source object, of size 5 kB, segmented into 5 source symbols of size 1 kB each, to which FEC encoding adds 4 repair symbols, also of size 1 kB. Source symbols are part of the 5+4 encoding symbols (systematic FEC code).
12
FEC
EN
CO
DIN
G
source object
n-k=4 repair symbols
k=5 source symbols
n=9 encoding symbols
UDP/IP datagram UDP/IP datagram
…
AL-FEC code definitions… (cont’) ! the code rate is a key parameter
❍ close to 1 ⇒ little/no redundancy❍ close to 0 ⇒ high amount of redundancy
❍ examples: • code_rate = 0.5 means that there are as many redundancy
symbols as source symbols• code_rate = 2/3 means that n=1.5*k, i.e. we add 50% of
redundancy
13
€
code_ rate =kn
=before_encodingafter _encoding
AL-FEC code definitions… (cont’) ! systematic codes ❍ codes for which the k source symbols are part of the n
encoding symbols ❍ see previous example…
❍ preferred to non-systematic codes because: ❍ without loss, no decoding is needed
• save processing❍ a receiver that does not implement the FEC code can still
use source symbols (backward compatibility) • of critical importance
14
AL-FEC code definitions… (cont’) ! ideal or MDS (maximum distance separation)
codes ❍ a code for which decoding is always possible after
receiving any set of k symbols among n possible ❍ example: Reed-Solomon codes over GF(2m), with m=4, 8 or
16 for instance
❍ it’s an optimum code in terms of erasure recovery… ❍ one cannot find something better
❍ … which does not mean it’s necessarily the best possible code for a given use case ❍ there are constraints ❍ example: RS over GF(28) have strict limits k ≤ n ≤ 255
15
An example of use ! trivial example with a (9; 5) code
16
ENC
OD
ING
DEC
OD
ING
source object decoded object
n-k=4 repair symbols
k=5 source symbols
n=9 encoding symbols
UDP/IP datagrams
erasure channel
An example of use… (cont’) ! more complex example (larger object)
17
source object (size=15 000
bytes)
step 1: segment into
source symbols (size 1000 bytes)
step 2: partition into blocks, given the maximum k the AL-FEC
encoder supports
(e.g. kmax=8)
total of 15 source symbols
bloc
k 1:
k=8
src
sym
. bl
ock
2: k
=7 s
rc
ENC
OD
ING
EN
CO
DIN
G
step 3:
n=12 encoding symbols
n=11 enc. symb.
An example of use… (cont’) ! broadcast of a digital library to vehicles, using
FLUTE/ALC in carousel mode, over a long period
18
FEC encoding
multimedia content carrousel
reception… multimedia content
FEC decoding
FLUTE/UDP broadcast transmission to a multicast group
AL-FEC differ from PHY-layer FEC ! AL-FEC codes ❍ usually implemented in higher communication layers
(rather than in the PHY layer), e.g.: ❍ within the application ❍ within the transport system (between RTP and UDP for
streaming flows, in FLUTE/ALC for filecasting applications) ❍ within the MAC layer (e.g., in DVB-H/MPE-FEC, or in DVB-SH/
MPE-IFEC)
❍ hence their name “Application Level-FEC” (AL-FEC) ❍ also called “Upper Layer-FEC” (UL-FEC) by those who work
at the PHY layer
19
Outline 1. the erasure channel 2. AL-FEC codes 3. what is a good AL-FEC code? The
performance metrics 4. example: LDPC-Staircase 5. example: Raptor(Q)™
33
Performance metrics
! how can we define good AL-FEC codes? � define performance metrics
34
Performance metrics ! metric 1: decoding overhead as a measure of the
erasure recovery capabilities ❍ major performance criteria since many AL-FEC are not MDS ❍ measured as the overhead ratio:
• e.g. overhead=0.63% means that 0.63% of symbols in addition to k are needed for decoding to succeed
❍ or as the raw overhead (number of extra symbols):
❍ derivatives: average overhead, 99% margin above (resp. below) the average, etc.
35
decoding_overhead _ ratio = #_of _ symbols_ required _ for _ decodingk
−1
decoding_overhead = #_of _ symbols_ required _ for _ decoding− k
Performance metrics… (cont’) ! metric 3: decoding failure probability = f(number
of symbols received) ❍ similar to metric 2 but as a function of the number of
symbols actually received • the curve is an average over a large number of experiments,
for a given {code rate, loss rate}
37
number of symbols received
decoding limit, depending on block size
decoding impossible
(too few symbols…)
decoding failure probability
10-3
10-2
10-1
1
slope should be as vertical as possible…
fall should happen as soon as possible… decoding
OK with high proba…
k
Performance metrics… (cont’) ! metric 5: encoding and decoding speed
❍ to appreciate the algorithmic complexity • more reliable than theoretical algorithmic complexity analyzes
that ignore major aspects (e.g., memory access/cache behavior/implementation details/…)
❍ there is an appropriate balance to find between erasure recovery and complexity
• some algorithms are faster but lead to lower erasure recovery capabilities, and vice-versa
❍ sometimes decoding complexity is the key • e.g., lightweight portable device
❍ sometimes encoding complexity is the key • e.g., deep space (autonomous) probe
39
Performance metrics… (cont’) ! metric 6: required memory during encoding and
decoding ❍ even if RAM is used (rather than costly chipset memory), it
should be used with caution ❍ e.g., if data can fit in the CPU L2 cache, it’s great
❍ high impact of the decoding algorithm
40
Performance metrics… (cont’) ! performances depend on many parameters: ❍ block size
❍ impacts the decoding overhead ❍ some codes (e.g., LDPC and Raptor) are good asymptotically
but sub-optimal with “small blocks”
❍ symbol size ❍ impacts speed and memory requirements
❍ decoding algorithm ❍ e.g., iterative decoding is fast, but leads to sub-optimal
overhead results
41
good AL-FEC = good code + good codec ( + good lobbying !)
Outline 1. the erasure channel 2. AL-FEC codes 3. what is a good AL-FEC code? The performance
metrics 4. example: LDPC-Staircase 5. example: Raptor(Q)™
42
LDPC codes ! in short ❍ “Low Density Parity Check” (LDPC)
❍ linear block codes ❍ discovered by Gallager in the 60’s ❍ re-discovered in mid-90s
❍ original codes were extremely costly to encode ❍ generator matrix (G) creation requires a matrix inversion ❍ but we use high performance variants
❍ in the remaining we only consider binary codes
44
LDPC-Staircase codes ! LDPC-staircase codes (RFC 5170)
❍ a particular class of binary LDPC codes ❍ A.K.A. “Repeat Accumulate” codes ❍ a simple structure that defines a set of linear equations ❍ IETF RFC 5170 http://datatracker.ietf.org/doc/rfc5170/
45
S1 ⊕ S2 ⊕ S3 ⊕ P2 ⊕ P3 = 0 S2 ⊕ S4 ⊕ S5 ⊕ P3 ⊕ P4 = 0 S1 ⊕ S2 ⊕ S3 ⊕ S5⊕ P4 ⊕ P5 = 0
S3 ⊕ S4 ⊕ P1 = 0
linear system source symbols parity symbols
0 1 0 1 0 1 0 0 0 0 1 0 0 1 1 1 1 0 0 0 1 1 1 0 0 0 1 1 0 0 0 0 1 1 1 0 0 1 1 0 1 1 1 0 1 0 0 0 1 1
S1 S2 S3 S4 S5 P1 P2 P3 P4 P5
N1 “1”s per column (e.g. 3)
S1 ⊕ S4 ⊕ S5 ⊕ P1 ⊕ P2 = 0
parity check matrix (H)
LDPC-Staircase codes… (cont’) ! encoding ❍ it’s trivial "
❍ produce repair symbols in sequence: P1, then P2, then P3…
❍ guaranties a high encoding speed ❍ example: CR=2/3, N1=7, symbol size=1024 bytes ❍ Samsung Galaxy S2 smartphone, ARM Cortex A9, 1.2GHz
46
0
200
400
600
800
1000
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
aver
age
bitra
te (M
bps)
block size (k)
Samsung Galaxy SII
500-700 Mbps "
LDPC-Staircase codes… (cont’) ! decoding (much more complex) ❍ solve a system of binary linear equations ❍ scheme 1: ITerative decoding (IT) ❍ scheme 2: Structured Gaussian Elimination (SGE)
• SGE is an old optimized way of using GE over sparse systems, midway between IT and regular GE
❍ start with (1), finish with (2) if in bad reception conditions
47
Raptor RaptorQ RS+LDPCclip SD HD clip SD HD clip SD HD
Decoding speed (Mbps)bad channel: L/R=5%/5% 203.6 216.9 218.6 173.8 178.1 172.9 326.8 343.3 292.4bad channel: L/R=20%/20% 190.1 175.5 198.6 172.4 177.3 174.9 245.9 185.4 177.8good channel: L/R=20%/5% Not avail. Not avail. Not avail. 193.6 201.4 189.6 368.5 439.7 322.0Maximum memory requirements (MB)all channels 6.8 to 6.9 21.2 to 21.3 21.9 to 22.2 6.7 to 7.0 21.0 to 21.9 22.4 to 23.4 4.8 to 6.2 13.8 to 18.6 25.3 to 30.3
TABLE III. DECODING SPEED AND MAXIMUM MEMORY REQUIREMENTS FOR 3GPP-EMBMS DOWNLOAD USE-CASES (SOURCES: [20], [21]).
Raptor RaptorQ RS+LDPC1 sec. 2 sec. 4 sec. 1 sec. 2 sec. 4 sec. 1 sec. 2 sec. 4 sec.
Decoding speed (Mbps)bad channel: L/R=5%/5%, 3km/h 92.6 109.5 169.1 204.2 230.2 215.5 217.4 189.7 260.4bad channel: L/R=20%/20%, 3km/h 70.6 72.3 94.5 200.4 210.9 219.0 187.6 142.6 235.3bad channel: L/R=20%/20%, 120km/h 84.3 90.4 143.4 207.1 220.7 227.9 164.0 137.4 185.7good channel: L/R=20%/5%, 120km/h Not Avail. Not Avail. Not Avail. 239.7 237.0 240.9 220.6 185.7 266.0Decoding latency (sec.)all channels Not avail. Not avail. Not avail. 1.8 to 4.3 5.0 to 7.9 11.7 to 17.4 2.0 to 4.0 7.4 to 10.4 10.6 to 14.2
TABLE IV. DECODING SPEED AND LATENCY FOR 3GPP-EMBMS STREAMING USE-CASES (SOURCES: [20], [21]).
by IETF, LDPC-Staircase codes are part of the ISDB-TmmJapanese video push standard, and our proposal based on thesecodes has been ranked first (1 more vote than the RaptorQproposal, 31 vs. 30) after more than one year of competitionin the context of the 3GPP-eMBMS competition. It was alsorecognized that the RS+LDPC proposal offers several benefitsover Raptor codes.
In this paper we detail the code internals, some techniqueswe used to design high speed decoders, and provide extensiveperformance analyses. We show that these codes, in spiteof their simplicity, achieve excellent results. The RS+LDPC-Staircase combination is therefore a very good choice, withmany advantages in terms of openness, availability of freecodecs, simplicity of the specifications, excellent erasure re-covery performance that are either ideal or in a 1% marginof ideal codes, and very high decoding speeds, even onsmartphones. There are also situations where our approach is,from our point of view, IPR free, which can be an advantagein some situations.
REFERENCES
[1] ETSI, “Evaluation of mbms fec enhancements (final report),” Apr.2013, 3GPP TR 26.947 version 11.0.0 Release 11. [Online]. Available:http://www.3gpp.org/ftp/Specs/html-info/26947.htm
[2] T. Paila, R. Walsh, M. Luby, V. Roca, and R. Lehtonen, “FLUTE -File Delivery over Unidirectional Transport,” Nov. 2012, IETF Requestfor Comments, RFC 6726 (Standards Track) (obsoletes RFC 3926).[Online]. Available: http://www.ietf.org/rfc/rfc6726.txt
[3] D. MacKay, Information Theory, Inference and Learning Algorithms.Cambridge University Press, ISBN: 0521642981, 2003.
[4] V. Roca, C. Neumann, and D. Furodet, “Low Density Parity Check(LDPC) Staircase and Triangle Forward Error Correction (FEC)schemes,” Jun. 2008, IETF Request for Comments, RFC 5170(Standards Track). [Online]. Available: http://www.ietf.org/rfc/rfc5170.txt
[5] V. V. Zyablov and M. S. Pinsker, “Decoding complexity of low-densitycodes for transmission in a channel with erasures,” Probl. PeredachiInf., vol. 48, pp. 18–28, 1974.
[6] M. Cunche and V. Roca, “Optimizing the error recovery capabilitiesof LDPC-staircase codes featuring a Gaussian Elimination decodingscheme,” in 10th IEEE International Workshop on Signal Processing forSpace Communications (SPSC’08), Rhodes Island, Greece, Oct. 2008.
[7] E. Paolini, M. Varrella, M. Chiani, B. Matuz, and G. Liva, “Low-complexity ldpc codes with near-optimum performance over the bec,” inAdvanced Satellite Mobile Systems, 2008. ASMS 2008. 4th, aug. 2008.
[8] I. S. Reed and G. Solomon, “Polynomial codes over certain finite fields,”Journal of the Society for Industrial and Applied Mathematics, vol. 8,no. 2, pp. 300–304, 1960.
[9] J. Lacan, V. Roca, J. Peltotalo, and S. Peltotalo, Reed-SolomonForward Error Correction (FEC) Schemes, Apr. 2009, IETF Requestfor Comments, RFC 5510 (Standards Track). [Online]. Available:http://www.ietf.org/rfc/rfc5510.txt
[10] A. Shokrollahi and M. Luby, “Raptor codes,” Foundations and Trends inCommunications and Information Theory, vol. 6, no. 3-4, pp. 213–322,May 2011.
[11] M. Luby, A. Shokrollahi, M. Watson, and T. Stockhammer, “RaptorForward Error Correction Scheme for Object Delivery,” Oct. 2007,IETF Request for Comments, RFC 5053 (Standards Track). [Online].Available: http://www.ietf.org/rfc/rfc5053.txt
[12] M. Luby, A. Shokrollahi, M. Watson, T. Stockhammer, and L. Minder,“RaptorQ Forward Error Correction Scheme for Object Delivery,”IETF Request for Comments, RFC 6330 (Standards Track), Aug.2011. [Online]. Available: http://www.ietf.org/rfc/rfc6330.txt
[13] V. Roca, M. Cunche, and J. Lacan, “Simple Low-Density ParityCheck (LDPC) Staircase Forward Error Correction (FEC) Scheme forFECFRAME,” Dec. 2012, IETF Request for Comments, RFC 6816(Standards Track). [Online]. Available: http://www.ietf.org/rfc/rfc6816.txt
[14] V. Roca, M. Cunche, J. Lacan, A. Bouabdallah, and K. Matsuzono,Simple Reed-Solomon Forward Error Correction (FEC) Scheme forFECFRAME, Feb. 2013, IETF Request for Comments, RFC 6865(Standards Track). [Online]. Available: http://www.ietf.org/rfc/rfc6865.txt
[15] A. Yamada, H. Matsuoka, T. Ohya, R. Kitahara, J. Hagiwara, andT. Morizumi, in IEEE International Symposium on Broadband Mul-timedia Systems and Broadcasting (BMSB’11), Jun. 2011.
[16] B. A. LaMacchia and A. M. Odlyzko, “Solving large sparse linearsystems over finite fields,” in Advances in Cryptology (Crypto’90),LNCS 537, Springer-Verlag, 1991.
[17] C. Pomerance and J. W. Smith, “Reduction of huge, sparse matricesover finite fields via created catastrophes,” Experimental Mathematics,Vol. 1, No. 2, 1992.
[18] E. Paolini, B. Matuz, G. Liva, and M. Chiani, “Pivoting algorithms formaximum likelihood decoding of LDPC codes over erasure channels,”IEEE Global Telecommunications Conference (GLOBECOM’09), 2009.
[19] C. Thiennot, C. Seyrat, V. Roca, and J. Detchart, “Proposal for acandidate for emm-efec work item,” May 2012, Expway, DocumentS4-120731, 3GPP TSG-SA4 meeting 69, Erlangen, Germany.
[20] “Performance verification results for MBMS EFEC candidates,” Jan.2013, Huawei Technologies France, Document S4-130061, 3GPP TSG-SA4 meeting 72, Valencia, Spain. [Online]. Available: http://www.3gpp.org/ftp/tsg sa/WG4 CODEC/TSGS4 72/Docs/S4-130061.zip
[21] “Emm-efec, selection of the fec,” Nov. 2012, document S4-121367, 3GPP TSG-SA4 meeting 71, Bratislava, Slovakia. [Online].Available: http://www.3gpp.org/ftp/tsg sa/WG4 CODEC/TSGS4 71/Docs/S4-121367.zip
LDPC-Staircase codes… (cont’) ! decoding… (cont’)
❍ CR=2/3, N1=7, symbol size=1024 bytes ❍ Samsung Galaxy S2 smartphone, ARM Cortex A9, 1.2GHz
48
0
200
400
600
800
1000
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
aver
age
bitra
te (M
bps)
block size (k)
IT decoding (5% loss rate)ML decoding (33% loss rate)
IT+SGE decoding (bad rx conditions)
IT only decoding (good rx conditions)
> 600 Mbps "
150 - 550 Mbps "
LDPC-Staircase codes… (cont’) ! erasure recovery performance are close to that of
Raptor for medium to large blocks ❍ it’s good enough, no need to go any further!
49
Pr = 1: cannot decode
Pr << 1: high decoding probability
Parameters Average overhead (i.e. Prfail = 0.5) Overhead for Prfail 10�4
CR = 0.9k=180 0.902%, i.e. 1.623 add. symbols 6.667%, i.e. 12 add. symbols, Prfail = 3.8 ⇤ 10�5
k=256 0.638%, i.e. 1.633 add. symbols 5.469%, i.e. 14 add. symbols, Prfail = 6.2 ⇤ 10�5
k=1024 0.168%, i.e. 1.720 add. symbols 1.367%, i.e. 14 add. symbols, Prfail = 7.9 ⇤ 10�5
k=4096 0.051%, i.e. 2.089 add. symbols 0.366%, i.e. 15 add. symbols, Prfail = 6.4 ⇤ 10�5
k=8192 0.030%, i.e. 2.474 add. symbols 0,183%, i.e. 15 add. symbols, Prfail = 8.4 ⇤ 10�5
CR = 2/3k=180 0.971%, i.e. 1.748 add. symbols 8.333%, i.e. 15 add. symbols, Prfail = 7.6 ⇤ 10�5
k=256 0.706%, i.e. 1.807 add. symbols 5.859%, i.e. 15 add. symbols, Prfail = 5.9 ⇤ 10�5
k=1024 0.238%, i.e. 1.026 add. symbols 1.465%, i.e. 15 add. symbols, Prfail = 8.2 ⇤ 10�5
k=4096 0.120%, i.e. 4.915 add. symbols 0.464%, i.e. 19 add. symbols, Prfail = 7.1 ⇤ 10�5
k=8192 0.101%, i.e. 8.258 add. symbols 0,281%, i.e. 23 add. symbols, Prfail = 9, 3 ⇤ 10�5
TABLE I. LDPC-STAIRCASE ERASURE RECOVERY PERFORMANCE, ML DECODING, N1=7 AND G=1. NOTE THAT USE-CASES OF 3GPP TESTSINVOLVING SMALL BLOCKS ARE ADDRESSED EITHER WITH REED-SOLOMON CODES OR WITH LDPC-STAIRCASE CODES WITH G > 1.
0
1
2
3
4
5
6
7
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
De
cod
ing
ove
rhe
ad
(%
)
k (number of symbols)
Decoding overhead for LDPC-Staircase, N1=7 and CR 2/3
average overheadoverhead for Pr_fail < 10e-4
(a) Overhead to reach Pfail < 10�4 as a function of k
1e-05
0.0001
0.001
0.01
0.1
1
10
1020 1030 1040 1050 1060 0
50000
100000
150000
200000
250000
De
cod
ing
fa
ilure
pro
ba
bili
tiy
Nu
mb
er
of
sam
ple
s
Number of received symbols
Histogram (total of 1000000 iter.)LDPC-Staircase N1=7 CR=2/3 k=1024
(b) Pfail = f(overhead), k = 1024
Fig. 2. LDPC-Staircase performance, with CR = 2/3, N1 = 7, G = 1.
means a 5.8% overhead is needed to reach Prfail < 10�4
(Table I), then using by G = 4 times more symbols, eachof them of size 1/4 the packet payload size, this overhead isreduced to 1.4%. It is an efficient way to better use these codes,at the price of a slightly increased processing load (section VIshows decoding remains fast with such values for k).
However this technique has limits, and our 3GPP-eMBMSproposal relies on Reed-Solomon over GF(28) codes for smallblocks (e.g. when k � 170 in case of a code rate 2/3)and LDPC-Staircase above. Reed-Solomon being MDS, weachieve a zero overhead whenever they can be used.
This is not the case for Raptor which is supposed to be usedin all situations. Therefore Raptor suffers from bad recoverycapabilities with very small blocks, even if values up to G =10 are used to mitigate the problem.
C. LDPC-Staircase used in a Non-Systematic Way
LDPC-Staircase codes can also be used in a non systematicway, by generating a sufficiently large number of repair sym-bols (if CR < 1/2, the number of repair symbols is higher thanthat of source symbols) and by sending only repair symbols.Erasure recovery performance under ML decoding is in thatcase excellent. With k = 1024, CR = 0.4821, N1 = 7 andG = 1, an overhead of 14 symbols is sufficient to achievePrfail < 10�4, which is even slightly better than the samecode used in a systematic way.
D. Comparison with Raptor/RaptorQ codes and 3GPP-eMBMS Performance Results
1e-05
0.0001
0.001
0.01
0.1
1
1000 1010 1020 1030 1040
Deco
din
g failu
re p
robabili
tiy
Number of received symbols
LDPC-Staircase N1=7 CR=2/3 k=1000Raptor CR=2/3 k=1000
Fig. 3. LDPC-Staircase vs. Raptor perf., k = 1024, CR = 2/3 and G = 1.
We have compared these results with that of Raptor, i.e.a binary code that only involves XOR symbol operations(like LDPC-Staircase). Figure 3 shows that both codes, whenused classically (i.e. G = 1), provide the same level ofperformance for k = 1024, CR = 2/3, G = 1. The difference(e.g. 1 additional symbol to reach the same Prfail value) isunnoticeable to the end user.
However with small blocks, Raptor is largely penalized.The artificial increase of the number of symbols by choosinga smaller symbol size and putting several symbols per packet(i.e. G > 1) is not sufficient. Our use of Reed-Solomon codes(section V-B) solves this issue as shown in [19].
The comparison with RaptorQ, a non-binary code, is less infavor of LDPC codes. This is caused by the use of operationsin the GF(28) finite field into the encoding/decoding process.It adds some complexity (well managed however, the majorityof symbol operations remaining XOR sums) but improves theerasure recovery performance by an order of magnitude.
Conclusions: LDPC-Staircase vs. Raptor ! achievements ❍ an IETF standard, RFC 5170
❍ enables interoperable, independent implementations
❍ LDPC-Staircase codes are included in the ISDB-Tmm Japanese standard for mobile multimedia
❍ with a good codec they achieve: ❍ recovery performances close to ideal ❍ high speed encoding and decoding, even on smartphones ❍ and high flexibility
• adjust the “recovery capabilities” / “processing load” trade-off
50
Concl.: LDPC-Staircase vs. Raptor… (cont’) ! during the 3GPP/eMBMS contest we showed that
in “representative use-cases” ❍ RS+LDPC-Staircase outperform Raptor™ codes " ❍ have performance that approach RaptorQ™ codes "
! morality:
❍ illustrates the need for
❍ high potential codes ❍ high quality codecs
❍ both aspects are critical in practice
51
in spite of their simplicity, LDPC-staircase codes are high-performance codes
Outline 1. the erasure channel 2. AL-FEC codes 3. what is a good AL-FEC code? The performance
metrics 4. example: LDPC-Staircase 5. example: Raptor(Q)™
52
Raptor™ codes ! history ❍ at the beginning were Tornado codes (Luby et al., 1997)
❍ the great idea is the discovery that irregular LDGM codes largely improve the performance of iterative decoding
❍ Tornado = cascade of irregular LDGM + many patents ❍ our LDPC codes perform as well and are patent free "
❍ then LT (Luby Transform) codes (1999) ❍ it’s just a simple evolution of irregular LDGM codes, so as to
enable small code rates + many patents ❍ but the erasure recovery capabilities are not good enough
❍ finally, with the help of Amin Shokrollahi, were Raptor codes (2001-now) ❍ a two stage encoding, taking advantage of LT codes ❍ plus optimized decoding techniques + many patents
53
Raptor™ codes… (cont’) ! the theory
❍ A. Shokrollahi, “Raptor Codes”, IEEE Trans. on Information Theory, Volume 52, Number 6, 2006.
! the practice ❍ all the details of the codes found in 3GPP and DVB systems
are in the IETF RFC5053 (Raptor) and RFC6330 (RaptorQ) http://www.ietf.org/rfc/rfc5053.txt
❍ significantly different from the theory! ❍ some refinements (e.g., predefined source block sizes) can
be found in other standardization docs
! the patents ❍ search any patent database site with Luby, Shokrollahi, or
Raptor keywords… E.g., in: • http://ep.espacenet.com/
54
Raptor™ encoding ! two steps encoding ❍ precoding: generate Intermediate Symbols (IS) ❍ LT encoding: produce encoding symbols from IS
! this is a systematic encoding ❍ source are part of encoding symbols ❍ …thanks to a specific way of producing IS
55
LT encoder ! discovered by Michael Luby
❍ who very humbly gave his name: Luby Transform (LT) ❍ theory rooted on work on the Soliton distribution, optimized
for IT decoding
! enables the generation of a potentially infinite number of encoding symbols
❍ unlimited only in theory
! each repair symbol is the XOR of a different number of source symbols ❍ irregular scheme: the number is not constant
❍ a small number is the XOR of a high number of source symbols
❍ while most of them are the XOR of a very small number (2) 56
LT encoder as per RFC5053 ! example of LT encoding: ❍ each column corresponds to an IS
(input) ❍ each row corresponds to an
encoding symbol (output) ❍ each row contains a “.” for each IS
considered during the encoding of a specific encoding symbol
❍ … it’s a sparse matrix
57
Precoding ! LT is fine, but it’s not systematic
❍ systematic codes are required in practice for situations where backward compatibility and/or low encoding overhead are needed
❍ hence a “trick”: do a pre-coding in such a way that the following LT encoding produces source symbols in the set of encoding symbols
58
LT LT
k src symbols
same k src symbols
add. repair symbols
L (>k) intermediate
symbols
Idea:
Precoding… (cont’) ! “LT decoding” creates k equations between source
symbols and IS… ! … in order to have an invertible system, since there
are L>k IS, we add L-k additional equations ❍ L = k + S + H ❍ S symbols are generated from an LDPC encoding of K
first IS ❍ H “half” symbols are generated from the remaining k+S
other IS
❍ in practice L is only slightly higher than k:
59
K" S" H" L"200" 23" 10" 233"
500" 41" 12" 553"
1000" 59" 13" 1072"
2000" 89" 14" 2103"
Precoding… (cont’) ! But… ❍ these L equations do not directly enable the generation
of the L IS � let’s put that in a matrix form ❍ A = matrix composed of the L equations ❍ D = (0 .. 0; source vector)
❍ D is a vector composed of L-k null symbols plus k source symbols
❍ C = (intermediate symbols) ❍ then: D = A*C ❍ or:
❍ encoding requires computing A-1 ❍ Raptor specifications are such that for any k in {4; 8192},
A-1 exists (this is not the case by default!) 60
C = A-1 * D
A and A-1 matrices
61
=
GLDPCT
GHalfT
GLTT
Id
Id 0
C
S
L
H
KD
0
N
S+H
0 + k source symbols
A matrix L intermediate symbols
A and A-1 matrices… (cont’)
62
GLDPC IS 0
GHalf ,,IH
GLT
A (theory) A (example) A-1 (example)
Raptor™ encoding (final step) ! when the L IS are generated, a
simple LT encoding suffices to produce the encoding symbols ❍ of course, there’s no need to re-
compute the source symbols!
63
Raptor™ decoding ! final goal ❍ rebuild the missing source symbols (from C’[0] to C’[k-1])
from the set of N available encoding symbols (D’[0] to D’[N-1])
! a two step decoding ❍ LT decoding to reconstruct the full set of IS (hard) ❍ LT encoding to reconstruct missing source symb. (trivial)
64
Raptor decoding D’[0], … ,D’[N-1] C’[0], … ,C’[k-1]
N available encoding symbols (received) source symbols
Raptor™ decoding… (cont’) ! step 1: the hard part… ❍ rebuild L intermediate symbols from the N encoding
symbols available… ❍ … taking advantage of the L-k extra relationships (null
symbols) ❍ otherwise the decoding overhead would be poor as N ≥ L > k
❍ it’s a matter of solving a linear system Ax = B, which can be done in many different ways
65
LT decoding D’[0], … ,D’[N-1] C[0], … ,C[L-1]
N available encoding symbols (received) L intermediate symbols
Raptor™ decoding… (cont’) ! step 1 (cont’): ❍ Gaussian Elimination (GE) is fine but costly ❍ ITerative decoding (IT) is fast, but sub-optimum
❍ there’s a midway solution: Structured Gaussian Elimination ❍ that’s the key to the problem, and this is not Raptor specific,
it can be applied to any sparse linear system, e.g. LDPC-Staircase codes
❍ B. A. LaMacchia and A. M. Odlyzko. “Solving Large Sparse Linear Systems over Finite Fields”, Advances in Cryptology (Crypto’90), 1991.
❍ C. Pomerance and J. W. Smith. “Reduction of huge, sparse matrices over finite fields via created catastrophes”, Experimental Mathematics, Vol. 1, No. 2, 1992.
66