chapter 5: linear block codes - université de limogesoutline basic principles linear block coding...

OutlineBasic principles

Linear Block Coding

Chapter 5: Linear Block Codes

Vahid Meghdadi

University of [email protected]

Vahid Meghdadi Chapter 5: Linear Block Codes


Linear Block Coding

Basic principlesIntroductionBlock codesError detectionError correction

Linear Block Coding



Linear Block Coding

IntroductionBlock codesError detectionError correction

Introduction

I Transmission through noisy channel

I Transmission errors can occur, 1’s become 0’s and 0’s become1’s

I To correct the errors, some redundancy bits are added to theinformation sequence, at the receiver the correlation isexploited to locate transmission errors

I Here, only binary transmission is considered.

I In this chapter we try to optimally add this redundancy to theinformation bits.



Linear Block Coding


Block codes (1/3)

Definition of block coding:

I A message m is a binary sequence of size k : m ∈ Ak

I To each message m correspond a codeword which is a binarysequence c of size n : n > k .



Linear Block Coding


Block codes (2/3)

I The code space contains 2n points but only 2k of them arevalid codewords.

I A code must be a one-to-one relation (injection)

Example:m ∈ {00, 01, 10, 11}

c ∈ {000, 011, 101, 110}

This is a one parity check code. 001,010,100,111 /∈ C



Linear Block Coding


Block code (3/3)

Definition:The rate of a code is defined as k/n. For example for the previouscode n = 3, k = 2 so the rate is 2/3.



Linear Block Coding


Linear codes

Definition:A binary code is linear if the following condition is satisfied:

∀c1 ∈ C , ∀c2 ∈ C ⇒ c1 + c2 ∈ C

Example: With the previous example

011 + 101 = 110 ∈ C

011 + 110 = 101 ∈ C

101 + 110 = 110 ∈ C



Linear Block Coding


Hamming distance and Hamming weight (1/2)

Definition :Hamming weight of a binary sequence is defined as the number of1 in the sequence.

w([101]) = 2 w([11101]) = 4

Definition:Hamming distance between two binary sequences v and w is thenumber of places where they differ.Example

v = [110101]w = [110011]

⇒ d(v,w) = 2



Linear Block Coding


Hamming distance and Hamming weight (2/2)

Property 1: d(v,w) + d(w, x) ≥ d(v, x) (triangle inequality)

Property 2: d(v,w) = w(v + w)

Example:v = [110101] w = [110011]

v + w = [000110] d(v,w) = w(v + w) = 2



Linear Block Coding


Minimum distance of block code

dmin = minv,w{d(v,w) : v,w ∈ C and v 6= w}

If the code is linear:

dmin = min{w(v + w) : v,w ∈ C and v 6= w}= min{w(x) : x ∈ C and x 6= 0}= wmin

Theorem: The minimum distance of a linear block code is equal tothe minimum weight of its non zero codeword.



Linear Block Coding


Good codes

A good code is one whose minimum distance is large.For the previous example dmin = 2:

{00, 01, 10, 11} ⇒ {000, 011, 101, 110}



Linear Block Coding


Error detection (1/3)

I Noisy channel introduces an error-pattern: c is sent, r isreceived where r = c + e

I w(e) =the number of errors that occur during thetransmission

I e is called error pattern

I For a code with dmin as minimum distance, any two distinctcodewords differ in at least dmin places.

I No error pattern of weight dmin − 1 or less can change onecodeword into another.



Linear Block Coding



I Any error pattern of size at most dmin − 1 can be detected.

I Some error patterns of size dmin or larger can be detected.

Example:{000, 011, 101, 110} ∈ C

For this code dmin = 2:

I any one error is detected

I but any 3 errors can be detected as well.



Linear Block Coding



I For c + e ∈ C, and e 6= 0, the error cannot be detected.Because the code is linear, only for e as a codeword thisconstraint is satisfied.

I So, for a (n, k) code, there are exactly 2n − 2k error patternsthat can be detected.

Example: for the previous code, all the following error pattern canbe detected:

{[001], [010], [100], [111]}

For this code, n = 3, k = 2, 2n − 2k = 4



Linear Block Coding


Error Detection Capacity

I The error detection fails only when the error pattern isidentical to a non-zero codeword, otherwise c + e ∈ C and theerror is not detected.

Definition: Weight distribution

I Let Ai be the number of codewords of weight i ,

I A0,A1, · · · ,An are called the weight distribution of the code.



Linear Block Coding


Code example

For this example:A0 = 1, A1 = A2 = 0, A3 = 7, A4 = 7, A5 = A6 = 0, A7 = 1



Linear Block Coding


Undetected Error Probability

Definition : Pu(E ) probability of undetected error

Pu(E ) = prob(e ∈ C)

In a BSC channel with p as the error transmission probability

Pu(E ) =n∑

i=1

Aipi (1− p)n−i



Linear Block Coding


Example

Considering previous code (7,4)

Pu(E ) = 7p3(1− p)4 + 7p4(1− p)3 + p7

For example if p = 0.01 then Pu(E ) = 7× 10−6

1-p

1-p

p

p0 0

1 1



Linear Block Coding


Error correction 1/2

How many errors can be corrected ?Let t be a positive number such that 2t + 1 ≤ dmin < 2t + 2Let v be the transmitted and r the received sequences, and w anycodeword. We can write:

d(v, r) + d(w, r) ≥ d(v,w)

Suppose that t ′ errors occur: d(v, r) = t ′, and for any v,w ∈ C

d(w, v) ≥ dmin ≥ 2t + 1

So d(w, r) ≥ 2t + 1− t ′. If t ′ ≤ t then:

d(w, r) > t

Conclusion: For any error pattern of t or fewer errors, r is closer toreal codeword v than any other codeword. So any t errors or fewercan be corrected.



Linear Block Coding


Error correction 2/2

Number of vectors (n-tuples) around a codeword with a distance of

u is:

(nu

).

Definition: Hamming Sphere of radius r is the number of vectorsaround a codeword with a Hamming distance up to t:

V (n, t) =t∑

j=0

(nj

)



Linear Block Coding


Conclusion

I A block code with minimum distance dmin guaranteescorrecting all error patterns of t = b(dmin − 1)/2c or fewer.

I t = b(dmin − 1)/2c is called error correcting capability of thecode.

I Some patterns of t + 1 or more can be corrected as well.I In general, for a t-error-correcting (n, k) linear code, 2n−k

error patterns can be corrected.I For a t-error-correcting (n, k) linear code and using an optimal

error correcting algorithm at the receiver, the probability thatthe receiver make a bad correction is upper bounded by

P(E ) <n∑

i=t+1

(ni

)pi (1− p)n−i



Linear Block Coding

Basic principlesIntroductionBlock codesError detectionError correction

Linear Block Coding



Linear Block Coding

Introduction

I Given n and k , how the code can be designed ?

I How an information sequence can be mapped to thecorresponding code word ?

I We are interested in systematic design.



Linear Block Coding

Code subspace

I An (n, k) linear code is a k-dimensional subspace of the vectorspace of all the binary n-tuples, so it is possible to find klinearly independent code words g0, g1, · · · , gk−1 to span thisspace.

I So any code word can be written as a linear combination ofthese base vectors:

c = m0g0 + m1g1 + · · ·+ mk−1gk−1



Linear Block Coding

Generator matrix

We can arrange these k linearly independent code words (vectors)as the rows of a k × n matrix as follows:

G =

g0

g1...

gk−1

=

g00 g01 · · · g0,n−1

g10 g11 · · · g1,n−1...

......

...gk−1,0 gk−1,1 · · · gk−1,n−1

k×n

c = mG



Linear Block Coding

Equivalent code

I One can obtain the same code (same codewords) but with adifferent generator matrix. For example, we can exchange anytwo rows of G.

I The only difference is that we obtain another mapping frominformation sequences to codewords.

I We can replace each row by a linear combination of that rowwith another row.

I All the codes obtained are equivalent.



Linear Block Coding

Systematic codes

Definition:

I If in all the codewords we can find exactly the correspondinginformation sequence, the code is called systematic.

It is convenient to group all these bits either at the end or at thebeginning of the code word.

I In this case the generator matrix can be divided into two submatrices [P|I ].



Linear Block Coding

Systematic codes

G =

g0

g1...

gk−1

=

p00 p01 · · · p0,n−1

p10 p11 · · · p1,n−1...

......

...pk−1,0 pk−1,1 · · · pk−1,n−1

1 0 · · · 00 1 · · · 0...

......

...0 0 · · · 1

G = [Pn×n−k Ik×k ]

So for a given message m the codeword will bec = mG = [mP m]1×n. The first part of the codeword is theparity bits and the second is the systematic part of the codeword.Note:Any linear code can be transformed into systematic code.



Linear Block Coding

Example

Example: u = [1101] and

G =

1 1 1 1 0 0 01 0 1 0 1 0 01 1 0 0 0 1 00 1 1 0 0 0 1

So the codeword will be: [0011101]. The codeword is the sum ofthe first, second and forth lines of the G.The last four bits are the systematic part of the codeword.



Linear Block Coding

Parity check matrix

For any k × n matrix G with k linearly independent rows, thereexists an (n − k)× n matrix H with (n − k) linearly independentrows such that any vector in the row space of G is orthogonal tothe rows of H and vice versa. So we can say:

I An n-tuple c is a codeword generated by G if and only ifcHT = 0

Matrix H is called Parity check matrix.

I The rows of H span a code space called dual code of G.

I GHT = 0



Linear Block Coding

Parity check matrix of systematic codes

For the systematic code G = [Pn×n−k Ik×k ], the parity checkmatrix is simply H = [I(n−k)×(n−k) PT

(n−k)×k ]. (Why?)Exercise: For the following generator matrix, give the generatormatrix of its dual code.

G =

1 1 1 1 0 0 01 0 1 0 1 0 01 1 0 0 0 1 00 1 1 0 0 0 1

Note: A non systematic linear code can be transformed intosystematic and then the parity check matrix can be calculated.This parity check matrix can also be used for the original code.



Linear Block Coding

Minimum distance of the code

Theorem: The minimum distance of a code is equal to thesmallest positive number of columns of H which are linearlyindependent. That is, all combination of dmin − 1 columns of H arelinearly independent, so there is some set of dmin columns whichare linearly dependent.



Linear Block Coding

Singleton bound

For an (n, k) linear code

dmin ≤ n − k + 1

Proof: the rank(H) = n − k so any n − k + 1 columns of H mustbe linearly dependent (the row rank of a matrix is equal to itscolumn rank). So dmin cannot be larger than n − k + 1 (accordingto the previous theorem).Definition: A code for which dmin = n − k + 1 is called”maximum distance separable” code.



Linear Block Coding

Systematic Error Detection

I An n-tuple vector r is a codeword iff

rHT = 0

I The product S = rHT is a (1× n− k) vector and is called thesyndrome of r.

I If the syndrome is not zero, a transmission error is declared.



Linear Block Coding

Systematic Error Correction

Maximum likelihood error correction:

c = arg minc∈C

dH(c, r)

Should we test all the possibilities ?For a (256,200) binary code, there are 2200 = 1.6× 1060

possibilities to be verified !Solution: Syndrome decoding which decreases the number ofpossibilies.



Linear Block Coding

Syndrome Decoding

r is the received sequencer = c + eS = rHT = (c + e)HT = cHT + eHT = eHT

Conclusion: The syndrome is not a function of the transmittedcodeword but a function of error pattern. So we can construct onlya matrix of all possible error pattern with corresponding syndrome.



Linear Block Coding

Syndrome Table Construction

I There are 2n possible received vectors

I There are 2k valid codewords

I There are 2n−k possible syndromes

First we generate all the error patterns of length 1, and calculatethe corresponding syndrome and put all of them into a matrix.Then, we increase the error pattern weight and we do the samething as before.Each time if the new syndrome had already been saved, it will bethrown away and we continue with the next error pattern.When all the 2k syndromes are found, the table is complete.For the previous example (256,200) the size of the matrix is256 = 7.2× 1016, which is still too big.



Linear Block Coding

Exercise

H =

1 0 0 0 0 1 10 1 0 0 1 0 10 0 1 0 1 1 00 0 0 1 1 1 1

It is a (7,3) code withdmin = 4. Complete thesyndrome table which has 16rows:

Error pattern Syndrome

0000000 00001000000 1000

......

0000001 1101

11000001010000

......

1110000 1110



Linear Block Coding

Exercise

Suppose r = [0, 0, 1, 1, 0, 1, 1], calculate the syndrome and thencorrect it in maximum likelihood sense. Use the same H as theprevious example.



Linear Block Coding

Hamming Code

Definition: If the syndrome table becomes complete for allw(e) ≤ t the code is called perfect.For example for the code (7, 4) with t = 1, the syndrome table iscompleted for all the error patterns of weight 1. These codes witht = 1 are called Hamming codes.For Hamming codes, there are 2n−k − 1 non-zero syndromes. Thisnumber must be equal to the number of error patterns of weight 1.It means that 2n−k − 1 = n. Calling the number of paritiesm = n − k , we can write for a Hamming code that:

k = 2m −m − 1

So the valid Hamming codes are:(3, 1), (7, 4), (15, 11), ..., (2m − 1, 2m −m − 1).



Linear Block Coding

Error Detection Performance Example

As we saw before the probability that a received sequence is takenfor a codeword by false is given by:

Pu(E ) =n∑

i=1

Aipi (1− p)n−i

For a (7,4) code the weight distribution is given by[1, 0, 0, 7, 7, 0, 0, 1] and for a (15,11) code it is[1, 0, 0, 35, 105, 168, 280, 435, 435, 280, 168, 105, 35, 0, 0, 1]. Usinga BSC channel derived from a BPSK system withp = Q(

√2REb/N0), we can draw the probability that a received

sequence is taken for a codeword erroneously.



Linear Block Coding

Pu curves

I R = 4/7I Eb/N0 = 6dB,

p = 0.0165I Eb/N0 = 10dB,

p = 3.6× 10−4

I R = 11/15I Eb/N0 = 6dB,

p = 0.0078I Eb/N0 = 10dB,

p = 6.41× 10−50 2 4 6 8 10

10−12

10−10

10−8

10−6

10−4

10−2

100

SNR(dB)

Pu

(15,11)(7,4)

It means that at 10 dB of Eb/N0, from 1011 received sequences,just one codeword error is not detected (it is not related to the bitsbut to the codewords).



Linear Block Coding

Error correction probability for perfect codes

Up to t errors are corrected in a perfect code wheret = b(dmin − 1)/2c. So the probability that a received sequence goout of Hamming sphere of radius t can be calculated easily. It is infact the probability of failure for a correcting receiver.

P(F ) = 1−t∑

j=0

(nj

)pj(1− p)n−j



Linear Block Coding

Error correction probability for perfect codes

0 1 2 3 4 5 6 7 8 9 1010

−7

10−6

10−5

10−4

10−3

10−2

10−1

100

Eb/N

0(dB)

P(F

)

(15,11)(7,4)

Note that this is not the bit error rate probability but word errorrate probability.



Linear Block Coding

Soft Decision Decoding

I A BPSK modulation is used to send the codeword (−√

Ec ,+√

Ec)

I The Hamming distance of two codewords is related to theirEuclidean distance by the relation: dE = 2

√EcdH

I With a (n, k , dmin) block code, and an AWGN with σ2 = N0/2, atransmitted codeword will be a point in n-dimensional space.

I Ec and Eb are related by Ec = REb where R = n/k is the code rate.

I Suppose that there are on the average K codewords at the distancedminfrom a codeword, the probability of block decoding (by theunion bound) is

P(E ) ≈ KQ

(√2RdminEb

N0

)I Neglecting K , the asymptotic coding gain in comparison with

uncoded system is Rdmin.



Linear Block Coding

Modifications to Linear Codes: Extension

An (n + 1, k , d + 1) code can be constructed from an (n, k , d)code by:

1. Add to the H a last all-zero column

2. Add to the new H a last all-one row

3. Use linear operation to obtain an equivalent systematic code

4. Find the generator matrix G

Exercise: Give the systematic generator matrix of (8, 4, 4) from thefollowing (7,4,3) code:

H =

1 0 0 1 0 1 10 1 0 1 1 1 00 0 1 0 1 1 1



Linear Block Coding

Modifications to Linear Codes: Puncturing

A code is punctured by deleting one column in the H matrix. So a(n, k) code is transformed into an (n − 1, k) code. The minimumdistance of the code can be decreased by one.


chapter 5: linear block codes - université de limogesoutline basic principles linear block coding...

Documents