Introduction to Error-Correcting CodingSession 01-02Course: Computational Group TheoryYear: 2014

Bina Nusantara University*Introduction to Error-Correcting Coding Communication Channels Error Detection Error Correction Hamming Distance Nearest Neighbour Decoding

What is Coding Theory ?

Coding theory is concerned with successfully transmitting data through a noisy channel and correcting errors in corrupted messages. It is of central importance for many applications in computer science or engineering.Bina Nusantara University*

A Simple ExampleA set of fruits {apple, banana, cherry, grape} are encoded into 2-digit binary string as follows:apple 00, banana 01, cherry 10, grape 11Suppose the message apple, which is encoded as 00, is transmitted over a noisy channel. Then the message is distorted and received as 01. The receiver may not realize that the message was corrupted.Bina Nusantara University*

Error DetectionIn order to anticipate error, we will use additional encoder and decoder, called as channel encoder and decoder.

In channel encoder, we usually add some redundancy or parity such that, if the message is corrupted, the channel decoder can detect the errors that occur in the transmission.

Bina Nusantara University*

Example of Error DetectionIn the previous example, we perform the channel encoding by introducing a redundancy of 1 bit as follows:apple 00 000,banana 01 011,cherry 10 101, grape 11 110

If we send the mesage apple (encoded as 000), and suppose that one error occurs in the transmission. So the received message will be 100, 010 or 001. None of that three possibilities is among our encoded messages. Thus, the channel decoder can detect that the received message contains some error. Bina Nusantara University*

Error CorrectionSuppose that, in our previous example, the message 000 (apple) is received as 100. The channel decoder can know that 100 is not the correct message. But can it know the correct message? Remember that the receiver doesnt know the correct message.

If it is assumed that there is only one error, the receiver can decode 100 into 000 (apple, correct) or 101 (cherry, wrong) or 110 (grape, wrong). So, this method can only detect error, but cannot correct the error.Bina Nusantara University*

Example of Error CorrectionExtending the previous example, we perform the channel encoding by introducing more redundancy as follows:apple 00 00000,banana 01 01111,cherry 10 10110, grape 11 11001

Suppose that the message apple (00000) is transmitted over a noisy channel and there is one error in the first digit so that the received message is 10000. The receiver knows that it contains error. What is the correct message according to the receiver?Bina Nusantara University*

Example of Error Correction (Cont.)

If we compares the received message 10000 with all possibilities of encoded message, then we will get 1 difference with 00000 (apple), 5 differences with 01111 (banana), 2 differences with 10110 (cherry) and 2 differences with 11001 (grape). Thus, the receiver can conclude that the correct message is 00000 (apple) because it has the smallest number of differences.Bina Nusantara University*

Basic Definitions in Coding TheoryBina Nusantara University*

ExamplesConsider the set of binary alphabet A={0,1}. All sets below are binary block code: is a (2,3)-code is a (3,5)-code is a (5,4)-codeA code over {0,1,2} is called a ternary code. Some examples are given below: is a ternary (3,3)-code is a ternary (4,5)-codeBina Nusantara University*

Hamming DistanceBecause the receiver doesnt know the correct message, so when any error occur, the receiver will correct it to the most similar one. In other words, it will be corrected to a message which has fewest number of differences.Therefore, we will introduce the distance between two encoded message, called as Hamming distance, as the number of their digits differences. Distance between two words x and y (of same length) is denoted by d(x,y).Example:Distance between 00110 and 10101 is 3, because they have 3 differences in the first, forth and last digits. We write d(00110,10101)=3.Bina Nusantara University*

Nearest Neighbour DecodingThe most used principle in channel decoding is decode/correct the corrupted message into a message that has fewest differences (lowest Hamming distance) with the received message. This principle is called nearest neighbour decoding.

Nearest neighbour decoding works just in two steps: first, compute the Hamming distance between the received message with all possible encoded message, and second, choose a message which has lowest Hamming distance and conclude that it is the correct message.Bina Nusantara University*

ExampleLet C={000000, 010101, 101010,111111} be a binary [6,4]-code. Suppose that 111010 is the received word. We compute the distance between that word and all codewords in C:d(111010,000000)=4d(111010,010101)=5d(111010,101010)=1d(111010,111111)=2We see that the most similar codeword to 111010 is 101010, with only 1 digit difference. By nearest neighbour decoding principle, 111010 is corrected to 101010. 101010 is claimed as the sent codeword.Bina Nusantara University*

Distance of a CodeHow many error can detected by a block code? And how many of them can be corrected? Answer of these questions are related to Hamming distance.Definition:Let C be a block code. The (minimum) distance of C, denoted by d(C), is the smallest possible distance between two different codewords in C, i.e.

Theorem: An [n,M]-block code whose distance is equal to d is said to be an [n,M,d]-code. It can detect at mostd-1 errors and correct at most errors.Bina Nusantara University*

ExampleLet C={000000,000111,111000,111111} be a binary [6,4]-code. We compute the distance of C:d(000000,000111)=3d(111000,111111)=3d(000000,111000)=3d(000111,111111)=3d(000000,111111)=6d(000111,111000)=6Thus, d(C)=3 and C is a [6,4,3]-code.Because d(C)=3, the block code C can detect up to 2 errors and correct 1 error. If only one error occurs, the message can be corrected to the true codeword. If two errors occur, the errors will be detected, but not always corrected. If more than two errors occur, they may be undetected.Bina Nusantara University*

Reference

San Ling and Chaoping Xing (2004). Coding Theory, A First Course. Cambridge University Press. ISBN: 0-511-18637-1Bina Nusantara University*