exercise in the previous class consider the following code c. determine the weight distribution of c...
TRANSCRIPT
exercise in the previous class
Consider the following code C.determine the weight distribution of Ccompute the “three” probabilities (p. 7), and draw a graph
1
.
10001110
01001101
00101011
00010111
H
We want to construct a cyclic code with n = 7, k = 4, and m =3.confirm that G(x) = x3 + x2 + 1 can be a generator polynomialencode 0110decide if 0001100 is a correct codeword or not
Answers: http://apal.naist.jp/~kaji/lecture/
today’s class
soft-decision decodingmake use of “more information” from the channel
convolutional codegood for soft-decision decoding
Shannon’s channel coding theorem
error correcting codes of the next generation
2
the channel and modulation
We considered “digital channels”.At the physical-layer,
almost all channels are continuous.digital channel=modulator + continuous channel + demodulator
3
modulator( 変調器 )
demodulator( 復調器 )
a naive demodulator translates the waveform to 0 or 1 the risk of possible errors
0010 0010
0010 0010
continuous(analogue)
“more informative” demodulator
From the viewpoint of error correction, the waveform contains more information than the binary output of the demodulator.
demodulators with multi-level output can help error correction.
4
0
1
definitely 0maybe 0
definitely 1maybe 1
to make use of this multi-level demodulator,the decoding algorithm must be able to handle multi-level inputs
hard-decision vs. soft-decision
hard-decision decodingthe input to the decoder is binary (0 or 1)decoding algorithms discussed so far are hard-decision type
soft-decision decodingthe input to the decoder can have three or more levelsthe “check matrix and syndrome” approach does not work
more powerful, BUT, more complicated
5
formalization of the soft-decision decoding
outputs of the demodulator0+ (definitely 0), 0- (maybe 0), 1- (maybe 1), 1+ (definitely 1)
code C = {00000, 01011, 10101, 11110}for a received vector 0- 0+ 1+ 0- 1-,
find a codeword which minimizes the penalty .
6
received0+
0-
1-
1+
penalty of “0”
0123
penalty of “1”
3210
(hard-decision ... penalty = Hamming distance)
0- 0+ 1+ 0- 1-
0 0 0 0 01 0 3 1 2 7
r
c0+ + + + =
1 0 1 0 12 0 0 1 1 4+ + + + =
c2
=
=
algorithms for the soft-decision decoding
We just formalized the problem... how can we solve it?by exhaustive search?
... not practical for codes with many codewordsby matrix operation?
... yet another formalization which is difficult to solveby approximation?
... yes, this is one practical approach anyway...
design special codes;whose soft-decision decoding is not “too difficult”
convolutional code ( 畳み込み符号 )
7
convolutional codes
the codes we studied so far... block codesa block of k-bit data is encoded to a codeword of length nthe encoding is done independently for block to block
convolutional codesencoding is done in a bit-by-bit mannerprevious inputs are stored in shift-registers in the encoder, and affects future encoding
8
input data
combinatorial logicencoder outputs
encoding of a convolutional code
at the beginning, the contents of registers are all 0when a data bit is given, the encoder outputs several bits,
and the contents of registers are shifted by one-bitafter encoding, give 0’s until all registers hold 0
9
r3 r2 r1
encoder exampleconstraint length = 3 ( = # of registers)the output is constrained by three previous input bits
encoding example (1)
to encode 1101...
10
0 0 0 1
11 0 0 1 1
10
0 1 1 0
01 1 1 0 1
10
give additional 0’s to push-out 1’s in the register...
encoding example (2)
11
1 0 1 0
00 0 1 0 0
00
1 1 0 1
10
1 0 0 0
01
the output is 11 10 01 10 00 00 01
encoder as a finite-state machine
constraint length k the encoder has 2k internal states
12
r2 r1
inpu
t
outp
ut
s0
s1 s2
s3
00
11
11
0001
1001
10
01
the encoder is a finite-state machinewhose initial state is s0
makes one transition for each input bitreturns to s0 after all data bits are provided
internal state = (r2, r1)s0=(0, 0), s1=(0, 1), s2=(1, 0), s3=(1, 1)
at the receiver’s end
the receiver knows...the definition of the encoder (finite-state machine)the encoder starts and ends at the state s0
the transmitted sequence which can be corrupted by errors
13
encoder receiver
01001... 01100...
errors
to correct errors = to estimate the “real” transition of the encoder... estimation on a hidden Markov model (HMM)
trellis diagram
a trellis diagram is obtained by...expanding the transition of the encoder to the time axis
14
possible encoding sequences (of length 5)= the set of paths connecting s0,0 and s5,0
the transmitted sequence = the path which is the most-likely to the received
sequence= the path with the minimum penalty
s0
s1
s2 expansion
s0,0 s5,0 s0s1s2
time 0 1 2 3 4 5
trellis
Viterbi algorithm
given a received sequence...the demodulator defines penalties for symbols at each positionthe penalties are assigned to edges of the trellis diagramfind the path with the minimum penalty using a good algorithm
Viterbi algorithmthe Dijkstra algorithm for HMMrecursive width-first search
15
Andrew Viterbi1935-
s0,0
pA
pB
qA
qB
the minimum penaltyof this state ismin (pA+qA, pB+qB)
soft-decision decoding for convolutional codes
the complexity of Viterbi algorithm ≈ the size of the trellis diagram
for convolutional codes (with constraint length k):the size of trellis ≈ 2k×data length ... manageablewe can extract 100% performance of the code
for block codes:the size of trellis ≈ 2data length ... too largeit is difficult to extract the full performance
16
block code good performance,but difficult to use
performance
convolutionalcode
moderated performance,full power available
summary of the convolutional codes
advantage:the encoder is realized by a shift-register (like as cyclic codes)soft-decision decoding is practically realizable
disadvantage:no good algorithm for constructing good codes
code design = the wire connection of register outputsdesign in the “trial-and-error” manner with computer
17
channel capacity
mutual information I(X; Y):X and Y: the input and output of the channelthe average information of X given by Ydepends on the statistical behavior of the input X
channel capacity = max I(X; Y)... the maximum performance achievable by the channel
18
X Y
computation of the channel capacity
the channel capacity of BSC with bit error probability p:if the input is X = {0, 1} with P(0) = q, P(1) = 1 – q, then
19
𝐼 ( 𝑋 ;𝑌 )=𝑝 log𝑝+(1−𝑝 ) log (1−𝑝)− (𝑝+𝑞−2𝑝𝑞) log (𝑝+𝑞−2𝑝𝑞 )− (1−𝑝−𝑞+2𝑝𝑞) log (1−𝑝−𝑞+2𝑝𝑞 )
the maximum of I(X; Y) is given when q = 0.5, with
𝐶=max 𝐼 (𝑋 ;𝑌 )
0 1.0
1.0
0.5
0.5p
C
¿𝑝 log𝑝+(1−𝑝 ) log (1−𝑝 )+1
Shannon’s channel coding theorem
code rate R = log2(# of codewords) / (code length)
(= k / n for an (n, k) binary linear code)
Shannon’s channel coding theorem:consider a communication channel with capacity C;
a code with rate R ≤ C, with which error at the receiver → 0no such code exists if the code rate R > C
The theorem says that such a code exists at “some place”.... How can we reach there?
20
investigation towards the Shannon’s limit
by mid 1990s ... “combine several codes” approach
concatenated code ( 連接符号 )apply multiple error correcting codes sequentiallytypically; Reed-Solomon + convolutional codes
21
encoder 1 encoder 2
decoder 1 decoder 2
channelouter code(RS code)
inner code(conv. code)
product code
product code ( 積符号 )2D code with more powerful codes applied for row/column
22
paritiesof code 1
parities of code 2
data bits0 1 01 1 01 0 0
1 10 11 0
1 0 10 1 0
1 00 1
0 1 00 1 01 0 1
1 00 11 0
0 0 01 1 01 0 1decoding
of code 2decodingof code 1
possible problem: in the decoding of code 1,the received information is not
considered
idea for the breakthrough
let the decoder see two inputs:the result of the previous stage + received information
feed-back the decoding result of code 1, and try decode code 2
Exchange the decoding results between two decoders iteratively.二台の復号器間で,復号結果を繰り返し交換する
23
paritiesof code 1
parities of code 2
data bits0 1 01 1 01 0 0
1 10 11 0
1 0 10 1 0
1 00 1
0 1 00 1 01 0 1
1 00 11 0
0 0 01 1 01 0 1decoding
of code 2decodingof code 1
the iterative decoding
idea: product code + iterative decoding
24
decoder for code 1
decoder for code 2
receivedsequence
decoding result
the decoder is modified to have two inputstwo soft-value inputs and one soft-value outputtrellis-based maximum a-posteriori decoding
the result of one decoder helps the other decoder more # of iteration, more reliable result
Turbo code: encoding
Turbo code:Add two sets of parities for one set of data bitsUse convolutional codes for the simplicity of decoding
25
encode: code 1
encode: code 2interleaver
data bits codedsequence
(bit reorder)
data bits parity: code 1 parity: code 2coded sequence
Turbo code: decoding
Each decoder sends its estimation to the other decoder.Experiments shows that...
code length must be sufficiently long (> thousands bits)the # of iterations can be small (≈ 10 to 20 iterations)
26
decoder: code 1
decoder: code 2
received sequencedecodingresult
II–1 I
I interleaver I–1 de-interleaver
performance of Turbo codes
The performance is much betterthan other known codes.
There is some “saturation ( 飽和 )”of the performance improvement.(error-floor)
27
C. Berrou et al.: Near Shannon Limit Error-Correcting Coding and Decoding: Turbo Codes, ICC 93, pp. 1064-1070, 1993.
LDPC code
The study of Turbo codes revealed that “the length is the power”.don’t bother too much on mathematical propertiespursue long and easily decodable codes
LDPC code (Low Density Parity Check code)linear block code with very sparse check matrix
almost all components are 0, with small # of 1discovered by Gallager in 1962, but forgotten for long yearsMacKay’s rediscovery in 1999
28
Robert Gallager1931-
David MacKay1967-
the decoding of LDPC codes
An LDPC code is a usual linear block code,but its sparse check matrix allows belief-propagation
algorithmto work efficiently and effectively.
29p3 = p1 p4 p6 + q1 q4 p6 + q1 p4 q6 + p1 q4 q6
Tanner Graph
1 1 1 0 1 0 01 0 1 1 0 1 00 1 1 1 0 0 1
H =
prob. of 0prob. of 1 (= 1 – pi)
p1 p3 p4 p6q1 q3 q4 q6
LDPC vs. Turbo codes
performance:both codes show excellent performancethe decoding complexities are “almost linear”LDPC shows “more mild” error-floor phenomenon
realization:O(n) encoder for Turbo, O(n2) encoder for LDPC
code design:LDPC has more varieties and strategies
30
summary
soft-decision decodingretrieve more information from the channel
convolutional codegood for soft-decision decoding
Shannon’s channel coding theoremwhat we can and we cannot
error correcting codes of the next generation
31
exercise
give proof for the discussion in p.19
32