feedback with carry shift registers synthesis with the euclidean algorithm

910 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004

Feedback With Carry Shift Registers Synthesis With theEuclidean Algorithm

François Arnault, Thiery P. Berger, and Abdelkadar Necer

Abstract—Feedback with carry shift registers (FCSR) were introducedby Goresky and Klapper in [3]. They are very similar to classical linearfeedback shift registers (LFSR) used in many pseudorandom generators.The main difference is the fact that the elementary additions are not addi-tions modulo 2 but with propagation of carries. The mathematical modelsfor LFSR are equivalently linear recurring sequences over GF (2) or ra-tional series in the set GF (2)[[ ]]. For FCSR, the “good” model is the oneof rational 2-adic numbers (cf. [5], [6]). It is well known, that the seriesgenerated by a LFSR can be synthesized by either the Berlekamp–Masseyalgorithm for binary linear recurring sequences or the extended Euclideanalgorithm in the set GF (2)[ ] of binary polynomials (cf. [11], [13]). In [5],Goresky and Klapper give an algorithm for the FCSR synthesis. This algo-rithm is similar to those of Berlekamp–Massey and is based on De Wegerand Mahler’s rational approximation theory (cf. [10], [14]).In this correspondence, we prove that it is possible to synthesize the

FCSR with the extended Euclidean algorithm in the ring of integers.This algorithm is clearly equivalent to the previous one, however, it issimpler to understand, to implement, and to prove. Our algorithm is stillvalid in the case of -adic integers where is a positive integer. We alsogive a near-adaptative version of this algorithm.

Index Terms—Feedback shift registers, pseudorandom generators, ra-tional approximation, 2-adic expansion.

I. GENERATION OF EVENTUALLY PERIODIC BINARY SEQUENCES WITH

FEEDBACK WITH CARRY SHIFT REGISTERS

In this section, we recall some definitions and basic properties of2-adic numbers and the link between eventually periodic binary se-quences and rational numbers. Then we explain how this approachleads to the realization of circuits which generate these sequences: thefeedback with carry shift registers (FCSR), introduced in [4]. Finally,using the notion of 2-adic complexity of an eventually periodic se-quence, we discuss the problem of 2-adic synthesis of such a sequenceand the difference with the problem of best rational 2-adic approxima-tion of any sequence.

For most details on 2-adic approximations and FCSR circuits thereader could refer to [3]–[5], [7].

A. Eventually Periodic Binary Sequences and 2-Adic Integers

First, we will recall briefly some basic properties of 2-adic numbers.For a more theoretical approach, we refer the reader to [9].

A 2-adic integer is formally a power series s = 1

n=0sn2

n, sn 2f0; 1g. Clearly, such a series does not always converge, relatively tousual topology, however, it can be considered as a formal object. Butthis series always converges if we consider the 2-adic topology. The setof 2-adic integers is denoted by 2.

Addition and multiplication in 2 can be performed by reportingcarries to higher order term, i.e., 2n + 2n = 2n+1 for all n 2 . Ifthere exists an integer N such that sn = 0 for all n � N , then s is apositive integer.

Manuscript received December 31, 2002; revised November 14, 2003. Thematerial in this correspondence was presented at the IEEE International Sym-posium on Information Theory, Lausanne, Switzerland, June/July 2002.

The authors are with UFR des Sciences de Limoges, 87060 Limoges Cedex,France (e-mail: [email protected]; [email protected]; [email protected]).

Communicated by A. M. Klapper, Associate Editor for Sequences.Digital Object Identifier 10.1109/TIT.2004.826651

An important remark is the fact that �1 = 1

n=02n, which is

easy to verify by computing 1 + 1

n=02n = 0. This fact allows us

to compute the opposite of a 2-adic integer very easily: if s = 2n +1

i=n+1si2

i, for some n 2 , then �s = 2n + 1

i=n+1(1� si)2

i.In particular, this implies that s is a negative integer if and only if thereexists an integer N such that sn = 1 for all n � N . Moreover, everyodd integer q has an inverse in 2 which can be computed by the for-mula q�1 = 1

n=0q0n, where q = 1 � q0.

The following theorem gives a complete characterization of eventu-ally periodic 2-adic binary sequences in terms of 2-adic integers (see[5] for the proof).

Theorem 1: Let S = (sn)n2 be a binary sequence and S2 =1

n=0sn2

n be the associated 2-adic integer. The sequence S is even-tually periodic if and only if there exist two numbers p and q in , qodd, such that S2 = p=q. Moreover, S is strictly periodic if and onlyif pq � 0 and jpj < jqj.

An important point is that the period of the rational number p=q isknown (since Gauss, cf. [5]).

Theorem 2: Let S be an eventually periodic binary sequence, letS2 = p=q, with q odd and p and q coprime, be the corresponding 2-adicnumber in its rational representation. The period of S is the order of 2modulo q, i.e., the smallest integer t such that 2t � 1 (mod q).

B. Generation of Periodic Binary Sequences Using FCSR

The 2-adic division p=q can be easily performed by a Galois ar-chitecture using FCSF. This corresponds to the classical division fol-lowing the increasing powers of 2. For the sake of simplicity, we willonly consider p � 0 and odd q = 1 � q0 < 0. If pq > 0, it iseasy to compute �p=q and then to obtain p=q by the formula �s =2n + 1

i=n+1(1 � si)2

i.For instance, the following circuit gives the coefficients of the 2-adic

expansion of �13=19 = (1 + 4 + 8)=(1� (4 + 16)):

In this diagram, boxes represent cells (or stages), whose content isa bit and which are controlled by a clock. At each cycle of the clock,each cell outputs the bit value present at input during the precedingcycle. The symbol denotes addition with carry, i.e., corresponds tothe following scheme:

For more details on the generation of periodic sequences with linearfeedback shift registers (LFSR) or FCSR circuits, the reader can referto [3]–[5] and [1]. In particular, it is interesting to understand why theGalois architecture, corresponding to the representation of periodic se-quences as quotients of integers or polynomials is more convenient thanthe classical Fibonacci architecture, corresponding to the linear recur-ring representation of periodic sequences.

0018-9448/04$20.00 © 2004 IEEE

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004 911

C. Rational Approximation and 2-Adic Synthesis of a BinaryEventually Periodic Sequence

In the sequel, k denotes a positive integer and if S is a binarysequence, we identify this sequence with the corresponding 2-adicnumber S = 1

i=0si2

i. We also denote Sk the finite sequence(s0; . . . ; sk�1) and the integer k�1

i=0si2

i. Following the classicaldefinition of the linear complexity of a binary eventually periodicsequence it is possible, in a way close to [5], to define the 2-adiccomplexity of such a sequence.

Definition 1: The 2-adic complexity of a binary eventually periodicsequence is the length (i.e., the number of cells) of the smallest FCSRgenerating S.

Remark 1: Let S be a binary sequence. If S = p=q with p and qcoprime integers, then the 2-adic (or FCSR) complexity �2 of S canbe defined as the maximum of bit lengths of jpj and jqj (cf. [6]).

The problem of 2-adic synthesis of a binary eventually periodic se-quence S of 2-adic complexity �2 is as follows.

• Given an eventually periodic sequence S (or Sk for some in-teger k), find two integers p and q satisfying S = p=q andmax(bitlength(jpj);bitlength(jqj)) = �2.

Now, we consider a binary sequence S not necessarily eventuallyperiodic.

Definition 2: Let S be a 2-adic integer and k 2 . We say that arational number p=q, with q odd, is an approximation of order k of Sif the first k terms in the 2-adic expansions of S and p=q are equal, i.e.,2k divides (S � p=q) in the ring 2.

In other words, p=q is an approximation of order k of S if and onlyif qSk � p mod 2k and q odd as defined by De Weger in [14].

The problem of the best (2-adic) rational approximation of order kof a sequence S is as follows.

• Given Sk and k, find two integers p and q, q odd, satisfyingqSk � p mod 2k and minimizing max(jpj; jqj).

Remark 2: Let k 2 . The problems of the best (2-adic) rationalapproximation of order k of a sequence S and the problem of 2-adicsynthesis of a binary eventually periodic sequence are clearly related,but they are distinct.

The solution of synthesis of a binary eventually periodic sequence isunique up to the sign p=q = (�p)=(�q).

The solution of the best rational approximation of S at the order kis not always unique: for example, if k = 4 and S4 = 9, (�5=3) and(�3=5) are two minimal approximations of S.

Let S be an eventually periodic sequence of 2-adic complexity �2.Suppose that p=q is a solution to the synthesis problem for S, then it isthe unique solution for the best rational approximation problem for allk satisfying k � 2�2 + 1 (see Corollary 1).

A first consequence of these facts is that an algorithm returning abest rational approximation of order k solves the problem of synthesisif it is applied to an integer k greater than 2�2 + 1. The reciprocalproblem is not so easy to solve.

Now, we want to focus on the practical solution of the synthesisproblem: let S be a binary sequence. We suppose that we can haveaccess to the value of Sk for any k.

We know that this sequence is eventually periodic for various rea-sons. For example, it is generated by a finite-state automaton (com-bining LFSR, FCSR, nonlinear circuits, etc.). We suppose that this se-quence has a “small” 2-adic complexity �2. There are two distinct sit-uations.

1) The 2-adic complexity �2 is known. In that case, a rational ap-proximation algorithm applied to S at the order 2�2 + 1 givesthe solution of the problem of 2-adic synthesis of S.

2) The 2-adic complexity �2 is not known. In that case, it is notpossible to prove that any output is the solution of the synthesisproblem. The final decision of acceptance or rejection of anoutput of the approximation problem is then based on subjectivecriteria.

• An upper bound � for the 2-adic complexity �2 is esti-mated. We have to apply our algorithm up to the order2� + 1.

• For a sufficiently large integer k, the size of the rationalapproximation of the sequence S is significatively smallerthan those expected for a random sequence.

According to Rueppel ([12]), the expected linear complexity of arandom sequence of length k is k

2. Experiments suggest that this result

is also true for 2-adic complexity.

II. EXTENDED EUCLIDEAN ALGORITHM FOR SOLVING THE FCSRSYNTHESIS PROBLEM

In [5], Klapper and Goresky gave an analog of the Berlekamp–Massey algorithm to recover the FCSR structure. Their algorithm isbased on the De Weger and Mahler’s rational approximation theory.As for the linear case, the knowledge of 2�2 + 1 consecutive termsof S allows to recover p and q in

O(�2+"

2 )(O(�22 ln �2 ln ln�2))

operations.In [7], Klapper and Xu extend their analyses to registers based on

�-adic elements where � is an element of a domainR and they developa general rational approximation for �-adic numbers.

Now we give another algorithm to recover p and q from 2�2 + 1consecutive terms of S. It is the adaptation of the classical extendedEuclidean algorithm for decoding Bose–Chaudhuri–Hocquenghem(BCH) codes (cf. [13, p. 362]). Moreover, with a slight modification ofthis algorithm, it is possible to find the best rational approximation ofany binary sequence for a fixed order. This algorithm will be extendedto g-adic numbers in Section IV.

Klapper and Goresky claimed for their algorithm a complexity inO(T 2 lnT ln lnT ), where T = 2n + 2, for synthesizing an FCSR ofsize n. For this, they assumed the underlying use of the Schönhage–Strassen algorithm for fast multiplication and division.

But the Euclidean algorithm is known to have a quadratic cost (see[8]). Our algorithm inherits this quadratic complexity O(T 2). Thisdoes not even require other multiplication/division algorithm than clas-sical schoolbook quadratic algorithm.

A. Rational Approximation and Lattices

In this subsection, we will first recall some classical results on lat-tices. Then we will describe the approximation lattice of a sequence.

1) Lattices: Let be the field of real numbers. We will considerlattices in 2. By definition, they are sets of the form

L = fxU + yV j x; y 2 g

whereU = (r1; v1) and V = (r2; v2) are linearly independent vectorsin 2. A set L 2 2 is a lattice if and only if it contains two linearlyindependent vectors and satisfies the property

U 0; V 0 2 L and �; � 2 =)�U 0 + �V 0 2 L:


In other terms, L is a free -module of rank 2. Let us recall here somefacts about lattices we will need later. The determinant of L (or itsvolume) is the absolute value jr1v2 � r2v1j of the determinant of thebasis (U; V ). This absolute value is independent of the basis (U; V ).We denote det(L) = jr1v2 � r2v1j. We have the following lemma.

Lemma 1: LetL be a lattice in 2. Two elementsU 0 = (r01; v0

1) andV 0 = (r02; v

0

2) of L form a basis of L if and only if jr01v02 � r02v0

1j =det(L).

Proof: It is a consequence of the formula of change of basis.

Note that this emma implies [5, Lemma 10.3].

Conventions andNotations:We will be concerned with lattices thatare contained in 2 and called sublattices of 2. Let U = (r; v) be anelement of 2. By convention, we say that U is positive if rv > 0, andnegative if rv < 0 (note that elements with r = 0 or v = 0 are neitherpositive nor negative). Let jU j be the “supremum norm” on 2, i.e.,the maximum of jrj and jvj. If L is a sublattice of 2, then it has twosuccessive minima for this norm

M1: a minimal element of L n f(0;0)gM2: a minimal element of L n M1:

Remember that M1 or M2 are not necessarily unique.

Lemma 2: Let L be a sublattice of 2. Let M1 = (m1; n1) andM2 = (m2; n2) be successive minima of L. Then they do not have thesame sign, i.e., m1n1m2n2 � 0.

Proof: Suppose that m1n1m2n2 > 0. Replacing eventually M1

by �M1 (or M2 by �M2), we can suppose that n1n2 > 0, and thenm1m2 > 0. We deduce jM2 �M1j = max(jm2 �m1j; jn2 � n1j)

jM2 �M1j < max(jm2j; jm1j; jn2j; jn1j)jM2 �M1j < max(jM1j; jM2j):

This contradicts the minimality of M2. So m1n1m2n2 � 0.

Proposition 1: Let L be a sublattice of 2, and M1;M2 be succes-sive minima of L. Then (M1;M2) is a basis for L.

Proof: Let U = (r; v) 2 L. We claim that U is a -combinationof M1 and M2. If U 2 M1 then there is nothing to prove. Else, sincejU j � jM2j � jM1j, we can reduce U using one of Mi (i = 1; 2),with the same sign (or with no sign): we replace U by U � "Mi where" = �1, such that the coordinates of U and "Mi have the same sign.Then we repeat the operation (U U � "Mi). The supremum normofU decreases until the sign of one (and only one) of the coordinates (ror v) of U changes. Then we reduce this vector by using the other Mi.Since jU j decreases while jU j � jM2j, the process can be continueduntil U 2 M1.

Definition 3: Let L be a sublattice of 2. The basis (M1;M2) of Lformed by successive minima is called a minimal basis of L.

Proposition 2: Let L be a sublattice of 2. If U1 and U2 are twolinearly independent elements of L then a Hadamard-like inequality(for the supremum norm) holds

det(L)

2� jU1j � jU2j:

Proof: Recall that the elements U1 and U2 said to be linearlyindependent if aU1 + bU2 = (0; 0), with a and b in , implies a =b = 0. Let U1 = (r1; v1) and U2 = (r2; v2) in L. We know thatj det(U1; U2)j = jr1v2 � r2v1j � det(L). So, we have

2 � jU1j � jU2j � jr1v2j + jr2v1j � jr1v2 � r2v1j � det(L):

Remarks :1) The result of Proposition 2 can also be obtained by the classical

Hadamard inequality and the fact that N(U) � p2 � jU j where N isthe Euclidean norm.

2) Note also that the equality holds only if jr1j= jv2j and jv1j= jv2j(and U1, U2 are linearly independent).

Lemma 3: Let L be a sublattice of 2. Let U1 = (r1; v1) and U2 =(r2; v2) be two linearly independent elements in L such that jUij <det(L) for i = 1; 2. Then U1 and U2 are of opposite sign (i.e.,

r1v1r2v2 < 0) and form a basis for L.Proof: Suppose that r1v1r2v2 � 0. Then we have

det(L)�jr1v2�r2v1j= jr1v2j�jr2v1j �jU1j � jU2j<det(L):

This is a contradiction. Hence, the vectorsU1 andU2 are of oppositesign and we have

jr1v2 � r2v1j = jr1v2j + jr2v1j � 2 � jU1j � jU2j < 2 � j det(L)j:Since jr1v2�r2v1j is a multiple of det(L), the equality holds and thenU1 and U2 form a basis for L (cf. Lemma 1)

Theorem 3: Let L be a sublattice of 2 and M1;M2 be successiveminima of L. If U1 and U2 are two linearly independent vectors inL such that jUij < det(L) for i = 1; 2 then U1 = �M1 andU2 = �M2 or U1 = �M2 and U2 = �M1.

Proof: Lemma 3 shows that vectors U1 and U2 are of oppositesigns. We have also jM1j < det(L) and M1 cannot be of oppo-site signs with both U1 and U2. Then M1 is necessarily in U1 or inU2. Suppose that M1 is in U1. Then, minimality of jM1j implies

U1 = �M1. Now the “opposite sign argument” can be applied to M2

and then M2 is in U1 or in U2. Since M1 and M2 are linearly inde-pendent, we have M2 2 U2 and, finally, M2 = �U2.

2) The Approximation Lattice: We give here some properties of ap-proximations of a 2-adic number by a rational number and the linkbetween rational approximation and some lattices. For more detailsconcerning theory of rational approximation the reader can refer to [2,p. 116].

Let S be a 2-adic integer and k 2 . Following De Weger [14], weconsider Lk(S) the kth approximation lattice defined by

Lk(S) = (r; v) 2 2vS � r mod 2k :

We have the following lemma.

Lemma 4: The set Lk(S) is a sublattice of 2 of determinant 2k.Proof: The set Lk(S) is obviously a lattice. The remaining part

follows from Lemma 1, the fact that (2k; 0), and (Sk; 1) are evidentlyin Lk(S) and if (r1; v1) and (r2; v2) are in Lk(S) then

r1v2 � r2v1 � 0 (mod 2k):

Using Lemmas 1 and 4 we obtain the following.

Lemma 5: Let k 2 �, S be a 2-adic integer, and Lk(S) be thekth approximation lattice of S. Two elements (r1; v1) and (r2; v2) ofLk(S) form a basis of Lk(S) if and only if jr1v2 � r2v1j = 2k .

We have also the next technical results which will be used later.

Lemma 6: Let k 2 �, S be a 2-adic integer, and Lk(S) be the kthapproximation lattice of S. Let U1 = (r1; v1) and U2 = (r2; v2) betwo linearly independent elements of Lk(S) such that: jU1j � 2i andjU2j < 2j for some real numbers i and j. Then i+ j > k � 1.

Proof: Since U1 and U2 are linearly independent, we havejr1v2 � r2v1j � 2k . But

jr1v2 � r2v1j � jr1v2j + jr2v1j < 2i2j + 2j2i = 2i+j+1

and then i+ j > k � 1.


Corollary 1: Let k 2 �, S be a binary sequence, and p=q be arational approximation of S at the order k. Let n = b(k � 1)=2c.If max(jpj; jqj) is strictly less than 2n, then this approximation is theunique best approximation of S of order k. Moreover, if there existsanother element U = (r; v) in Lk(S) such that jU j < 2n, then U =�2i(p; q) for some i � 0.

Proof: These results are direct consequences of Lemma 6.

The lattice Lk(S) can be partitioned into two parts

L0k(S) = f(r; v) 2 2 j v eveng = 2Lk�1(S) and

L1k(S) = f(r; v) 2 2 j v oddg = Lk(S) n L

0k(S):

The set L1k(S) is the set of rational approximations of S to order k.

A best rational approximation is then a minimal element of L1k(S) for

the ”supremum norm” on 2.The problem of finding the smallest FCSR generating the k first bits

of S is exactly the problem of finding a minimal element of L1k(S).

In [5], Klapper and Goresky gave an algorithm analogous to thatof Berlekamp–Massey for LFSR [11]. This algorithm computes re-cursively the minimal elements of L1

k(S) and L0k(S) from those of

L1k�1(S) andL0

k�1(S). In our work, we propose an algorithm based onthe extended Euclidean algorithm to compute directly the minimal ele-ment ofL1

k(S). In fact, with a small modification, our algorithm solvesthe problem of finding a minimal basis of Lk(S) for the supremumnorm. Then, applying the following theorem, this gives us a minimalelement of L1

k(S).

Theorem 4: Let k 2 �, S be a 2-adic integer, and Lk(S) be thekth approximation lattice of S. Let (M1;M2) be a minimal basis ofLk(S). Then either M1 or M2 is a minimal element of L1

k(S).Proof: The set L1

k(S) is not empty since (Sk; 1) 2 L1k(S). So

each base of Lk(S) contains an element of L1k(S). If M1 2 L1

k(S)then we have the result. Else, M1 � L0

k(S) and then M2 2 L1k(S).

B. Synthesis of an FCSR Generator With the Extended EuclideanAlgorithm

Let a and b be positive integers. The extended Euclidean algorithmcomputes the gcd d and the associated Bézout coefficients of a and b.It uses the formulas

r0 = a

u0 = 1

v0 = 0

r1 = b

u1 = 0

v1 = 1

and for i � 1;

ri+1 = ri�1 � qiriui+1 = ui�1 � qiuivi+1 = vi�1 � qivi

until rt = 0 for some t 2 , where ri�1 = qiri+ri+1 is the Euclideandivision of ri�1 by ri.

The sequence (ui; vi; ri)0�i�t is called the Bézout sequence of aand b.

We recall here some elementary facts about the extended Euclideanalgorithm.

Proposition 3: With the above mentioned notation, we have the fol-lowing properties.

1) For all i 2 f0; . . . ; tg, uia + vib = ri.

2) For all i 2 f0; . . . ; t� 1g, uivi+1 � ui+1vi = (�1)i.

3) The sequence ri is (strictly) decreasing and its terms are positive.

4) If i � 0 mod 2 then ui � 0 and vi � 0. If i � 1 mod 2 thenui � 0 and vi � 0.

5) The sequences (juij)i�1 and (jvij)i�0 are increasing.

6) For all i 2 f0; . . . ; t � 1g, jui+1rij � b, jvi+1rij � a;juiri+1j � b and jviri+1j � a.

Let S = (sn) be a binary sequence k 2 �, a = 2k , andb = k�1

i=0 si2i. Let (ui; vi; ri)0�i�t be their Bézout sequence. Using

Property 3) of Proposition 3, we see that there exists a unique positiveinteger i such that ri � 2k=2 < ri�1.

Definition 4: The output of the extended Euclidean algorithm for aand b (EEA (a; b) for short) is by definition the pair (ri; vi), where i isthe unique integer such that ri � 2k=2 < ri�1.

Now, we can give the following theorem which allows us to synthe-size an eventually periodic binary sequence.

Theorem 5: Let n 2 �. Suppose that a sequence S = p=q isgenerated by an FCSR generator of size less or equal than n. Let kbe an integer greater or equal to 2n + 1. The output of the extendedEuclidean algorithm with Sk and 2k in input is �(p; q).

Proof: By hypothesis, we have an approximation p=q of S withnorm less than or equal to 2n. The output of EEA U = (r; v) satisfiesjU j � 2n. From Corollary 1, there exists an integer i � 0 such thatU = �2i(p; q). Moreover, let u and w be the integers given by therelations u2k + vSk = r and w2k + qSk = p. Clearly, we deduceu = �2iw. However, from Property 2) of Proposition 3, u and v arecoprime. This implies i = 0 and U = �(p; q).

From these results, we deduce the following algorithm:

Synthesis Algorithm “EEAapprox”Input: A positive integer �, and the k = 2�+ 1 first terms of a binarysequence S.Output:

• Two integers p, q of “size” (i.e., bit length) less or equal to� suchthat p=q is an approximation of S to the order k if they exist.

• FALSE if no such integers exist.

k := 2� + 1(r0; u0; v0) := (2k; 1; 0)

(r1; u1; v1) :=k�1i=0 si2

i; 0; 1

While r1 > 2k=2 Do(s; t) := Euclidean quotient s and remainder t of r0

divided by r1(u2; v2) := (u0 � su1; v0 � sv1)(r0; u0; v0) := (r1; u1; v1)(r1; u1; v1) := (t; u2; v2)

End WhileIf max(jr1j; jv1j) � 2(k�1)=2 and v1 odd Then

Output (r1; v1) (as (p; q)).Else

Output FALSE

(the 2-adic complexity of S is greater than �).

C. Remarks About the Nonadaptative Nature of the SynthesisAlgorithm

Our algorithm is asymptotically faster than previous ones whichare based on the Berlekamp–Massey algorithm. However, one can beconcerned with its nonadaptative nature. If a better approximation isneeded, then the previous approximation is no longer useful and theentire algorithm must be restarted from the beginning. The algorithmsbased on the Berlekamp–Massey algorithms are adaptative: thesealgorithms operate by continuously updating the previous rationalapproximation with each newly discovered bit of the binary sequence.

Two remarks can be made about this concern.1) Our algorithm can be made adaptative while keeping a quadratic

asymptotic cost as follows (however, the cost increase can be notice-able in practice). Let EEAapprox (S; n) denote the computation of an


approximation of the sequence S to the order n using the extended Eu-clidean algorithm. Then consider the following loop:

For m from 1 to M do(p; q) := EEAapprox (S; 2m)

End for

Let c be a constant such that the cost of EEAapprox (S;n) isbounded by cn2. The total cost of the loop is bounded by

c

M

m=1

4m = 4Mc

M�1

m=0

4�m � 4Mc

1

m=0

4�m =4

34Mc:

So we have an adaptative algorithm which computes an approximationto the order n in time (4=3)cn2.

The resulting algorithm is more theoretical than practical. However,it induces some very efficient practical strategies as we shall see in thesequel.

2) The second remark is that if an approximation of the sequenceS to the order n has been computed by the extended Euclidean algo-rithm, then one can quickly check if this approximation is also validto the order n0 > n using the following procedure. Remember thatEEA (Sn; 2

n) computes not only r and v but also u such that vSn +u2n = r.

r; v; u := EEA (Sn; 2n)

n1 := nWhile n1 < n0 repeat

If u � vsn is odd thenoutput“Approximation r=v is only valid up to n1.”Stop

Else u := (u � vsn )=2n1 := n1 + 1

End ifEnd Whileoutput “Approximation r=v is valid up to n0.”

To show the validity of this procedure, consider the callEEA (Sn; 2

n) that has computed un, vn, rn satisfying

un2n + vnSn = rn:

As Sn+1 = Sn + 2nsn, we have

(un � vnsn)2n + vnSn+1 = rn:

If un � vnsn is even, then we can write un+12n+1 + vnSn+1 = rn,with un+1 = (un � vnsn)=2.

This shows that the approximation rn=vn of S is still valid to theorder n+ 1. The argument can be iterated for higher orders n1, whileun � vn sn remains even.

Note that at each step, the operations are simply additions of integersof size less or equal to n (i.e., the bit length of these numbers is less orequal to n). For example, if we compute the EEAapprox algorithm fora given n and verify the solution until n0 = 2n, the cost is c

4n02 for

the extended Euclidean algorithm part and n additions of numbers ofsize n which needs n2 = n

4bit operations. It remains clearly smaller

than the cost cn02 of the direct calculation of EEAapprox for n0.Now, we will discuss the interest of our algorithm in a practical

problem of FCSR as described in Section I-C.

• If the 2-adic complexity �2 of the sequence S is known, it is suf-ficient to apply EEAapprox for n = 2�2+1. The nonadaptativenature of this algorithm is not a problem.

• If the 2-adic complexity of the sequence is probably �2 andupper-bounded by �, a better approach is possible.Apply EEAapprox to n = 2�2 + 1.If it returns FALSE, then replace n by another n significativelygreater than the first one.If it returns a pair (p; q), use the verification procedure up to �or until it returns FALSE.

• If we use another stop test, for example, the size of the FCSR pro-duced at the order nmust be less than n=4 (the expected value fora random sequence is n=2), we use the same method: ComputeEEAapprox for a well chosen n.If it returns FALSE, then increase the size of n (something between2n and n2).If it returns a pair (p; q) which do not satisfy the stop test, thenuse the verification procedure until this test is satisfied or the pro-cedure returns FALSE.

III. THE EXTENDED EUCLIDEAN ALGORITHM AND THE BEST

APPROXIMATION PROBLEM

In this section, we first gives a lattice formulations of properties ofthe extended Euclidean algorithm. Then, we show that it is possible toderive the construction of a minimal basis of Lk(S) from this algo-rithm.

A. Lattice Formulation of the Extended Euclidean Algorithm

The extended Euclidean algorithm described before can be viewedas a way to reduce a particular basis of the lattice Lk(S) giving a newbasis whose components are short vectors for the supremum norm.

Suppose we know a base (U0; U1) of a sublatticeL of 2, withU0 =(r0; v0) and U1 = (r1; v1). Without loss of generality, assume thatjU1j � jU0j and that r0 and r1 are nonnegative. Assume, moreover,that v0v1 � 0 and r0 6= 0. The extended Euclidean algorithm givesthe following sequence of elements of L:

Ui+1 = Ui�1 � bri�1=ric � Ui; for i � 1 and while Ui > 0:

Proposition 4:

1) Every pair (Ui; Ui+1) is a basis for L.2) The ri are nonnegative and strictly decreasing.3) All products vivi+1 are nonnegative.4) The jvij are increasing.5) We have jrivi+1j + jri+1vij = j detLj.6) There exists an i such that jUij � j detLj.7) If we obtain jUij < j detLj=2 then Ui = �M1.8) If we have jUij; jUi+1j � j detLj then (Ui; Ui+1) is a re-

duced basis, equal to (�M1;�M2) or (�M2;�M1).Proof:

1) and 3) These two points are clear by recurrence.2) We have in fact ri+1 = ri�1 mod ri.4) This is a consequence of 3) because vi+1 = vi�1�bri�1=ricvi.5) By the preceding points, jrivi+1j+ jri+1vij = jrivi+1�ri+1vij

and the determinant of a basis for L is an invariant.6) By 2), there exists a first i such that ri � j detLj. If i � 1

then, by 5), jvij � j detLj.7) If Ui satisfies this inequality, then also M1. Now, Proposition 2

applies so Ui and M1 are colinear. Since they both belong to latticebasis of L, we get the result.

8) This is a consequence of Theorem 3 and the point (6) above.

B. Construction of a Minimal Basis of Lk(s)

Theorem 5 shows that an FCSR generator of size n can be synthe-sized from the knowledge of 2n+1 bits using the extended Euclideanalgorithm.


By a slight modification of the extended Euclidean algorithm, it ispossible to construct a minimal basis of Lk(S).

Lemma 7: Let k 2 �, S be a binary sequence (or equivalently a2-adic integer), and Sk = S mod 2k . Let M = (ri; vi) be the outputof the extended Euclidean algorithm for the inputs Sk and 2k. Thereexists a minimal basis (M1;M2) of Lk(S) such that M = M1 or M2.

Proof: This is a consequence of Theorem 3.

Theorem 6: Let S be a binary sequence (or 2-adic integer), k 2 �,and Sk = S mod 2k . Let M = (ri; vi) be the output of the extendedEuclidean algorithm for latticeLk(S), with inputsSk and 2k, andN =(ri�1; vi�1) be the vector found just before the last step. Then thereexists an integer a such that (M;N+aM) is a minimal basis ofLk(S).Moreover, a is one of the four integers immediately less than or greaterthan (ri�1 � vi�1)=(ri � vi) or �(ri�1 + vi�1)=(ri + vi)

Proof: From Lemma 7, there existsM 0 2 Lk such that (M;M 0)is a minimal basis for Lk. As (M;N) is also a basis, there are integersa and b such that M 0 = aM + bN . The changing basis matrix from(M;N) to (M;M 0) is

1 0

a b

and must have �1 determinant, so b = �1. Changing M 0 to �M 0

if necessary, we can assume that b = 1. Then M 0 = aM + N . Letr0 = ri�1 + ari and v0 = vi�1 + avi so that M 0 = (r0; v0). Theinteger a is such that max(jr0j; jv0j) is minimal. It is easy to check thatthis integer a is one of the four integers given above.

Now we can give the algorithm based on these results, which in factcomputes a minimal basis of L1

k(S).

Minimal basis algorithmInput: A positive integer k and a binary sequence S.Output : a minimal basis (M1;M2) of Lk(S).

(r0; u0; v0) := (2k; 1; 0)(r1; u1; v1) := ( k�1

i=0 si2i; 0; 1)

While r1 > 2k=2 Do(s; t) := Euclidean quotient s and remainder t of r0

divided by r1(u2; v2) := (u0 � su1; v0 � sv1)(r0; u0; v0) := (r1; u1; v1)(r1; u1; v1) := (t; u2; v2)

End WhileLet M1 := (r0; v0) (as (p1; q1)). Compute the four integers a1, a2,a3, a4 adjacent to the quotients (r0 � v0)=(r1 � v1) and �(r0 +v0)=(r1 + v1).

Compute the four vectors (r0j ; v0j) := (r0 � ajr1; v0 � ajv1) for j =

1; 2; 3; 4.Set (r0; v0) := (r0j ; v

0j) with the j minimizing max(jr0j; jv0j).

Set M2 := (r0; v0) (as (p2; q2)).Return (M1;M2)

Remark about the best approximation of a sequence S to theorder k

Let (M1;M2) be the output of the minimal-basis algorithm (MBA)for S and k in input. From Theorem 4, it is very easy to derive a bestrational approximation of S to the order k, since it is one of M1 andM2. Moreover, it is possible to improve the MBA algorithm for thisspecific use: If jM1j is less than 2(k�1)=2 and q1 odd, it is a best rationalapproximation. In that case, the second part of the MBA algorithm isnot needed. Since the complexity of the extended Euclidean algorithm

isO(k2), the complexity of this best rational approximation algorithmis clearly O(k2).

IV. GENERALIZATION TO g-ADIC APPROXIMATION

In [15], Xu and Klapper gave a generalization of 2-adic sequencesto g-sequences, where g is any integer greater than 2.

We recall briefly the results needed in this section. Let g be an in-teger greater or equal to 2. A g-adic integer is a power series S =1n=0 sng

n with sn 2 f0; 1; . . . ; g � 1g. The set of g-adic integeris denoted by g . As for the 2-adic case, additions and multiplicationsare performed by reporting carries to the higher term.

An element S = (sn)n�0 of f0; . . . ; g � 1g is called g-sequenceand can be identified with the g-adic integer 1

n=0 sngn.

If q is a natural integer coprime to g, it has an inverse in g . Analogsof Theorem 1 and 2 hold for g-sequences.

Moreover, it is possible to construct FCSR circuits for generatingsuch eventually periodic g-sequences. It is possible to deduce a Galoisarchitecture similar to that described in Section I. This naturally leadsto the definition of g-FCSR (or FCSR for short) complexity of an even-tually periodic g-sequence.

Definition 5: Let S be a g-sequence. The g-adic complexity of ag-sequence S is the length (i.e., the number of cells) of the smallestg-FCSR generating S.

If S = p=q, with p and q coprime, the g-FCSR complexity of Sis the maximum of g-lengths of jpj and jqj, where the g-length of aninteger is the number of symbols in its g-expansion. For example, ifp = 1 + 3 � 6 + 5� 62, the 6-length of p is 3.

Now, we will examine the use of the extended Euclidean and MBAalgorithms for g-sequences.

A. Lattices

Let k 2 �. As before, the kth approximation lattice of a g-adicinteger S is defined by

Lk(S) = (r; v) 2 2 j vS � r(mod gk) :

Its determinant is gk (follow the proof of Lemma 4). Also, Lemmas 1and 6, as well as Corollary 1 remain true replacing powers of 2 by thecorresponding powers of g. But we need to be more careful definingL0k(S) and L1

k(S)

L0k(S) = f(r; v) 2 2 j gcd(v; g) > 1g and

L1k(S) = f(r; v) 2 2 j gcd(v; g) = 1g:

The set L1k(S) is the set of rational approximations of S at the order

k. The main difference is the fact that L0k(S) is not a sublattice of

Lk(S) as soon as g is not a power of a prime.Also, Theorem 4 does not remain true. The problem is that (M1;M2)

can be a minimal basis ofLk(S) but with neitherM1 norM2 inL1k(S).

Note that Lemmas 3, 2, Propositions 1, 2, and Theorem 3 are alsotrue if we replace 2 with g.

B. Synthesis of a g-FCSR Generator

Suppose now that a g-sequence S is generated by a g-FCSR of sizen, i.e., S = p=q, p and q coprime, max(jpj; jqj) < gn. Let k be aninteger greater than or equal to 2n + 1. Set

Sk =

k�1

i=0

sigi = S(mod gk):

The output of the extended Euclidean algorithm withSk and gk in inputproduces an element U = (r; v) of size less than gk=2. From unicity ofsuch an element inLk(S), we deduce the g-adic version of Theorem 5.


Theorem 7: Let n 2 �. Suppose that a g-sequence S = p=q isgenerated by an FCSR generator of size less than or equal to n (p andq are coprime). Let k be an integer greater than or equal to 2n+1. Theoutput of the extended Euclidean algorithm with Sk and 2k in input is�(p; q).

Proof: It is similar to the proof of Theorem 5. The output U isof form U = �(p; q) where � 2 . There are two integers u and wsuch that ugk + vSk = r and wgk + qSk = p. We deduce u = ��w.However, from Property 2 of Proposition 3, u and v are coprime. Thisimplies � = �1 and U = �(p; q).

If S is a g-sequence of g-FCSR complexity �g , it is possible to re-cover its structure from the knowledge of 2�g + 1 consecutive termsof S. For comparison, the algorithm given in [15] needs the knowledgeof 9�g + 30 terms of S (cf. [15, Theorem 3]).

C. Construction of a Minimal Basis and the Best RationalApproximation Problem

It is easy to check that the MBA described in Section III-B holds forany g without changes.

It is always possible to obtain a minimal basis (M1;M2) of Lk(S).If either q1 or q2 is coprime to g, then it produces a minimal elementof L1

k(S), and then a best rational approximation of S to the order k.Theorem 4 does not remain true, but there are partial results in that

case.

Lemma 8: Let (M1;M2) be a minimal basis of Lk(S), M1 =(p1; q1), and M2 = (p2; q2). The integers q1 and q2 are coprime.

Proof: The element U = (Sk; 1) is in Lk(S). There are twointegers � and � such that U = �M1+�M2. This implies 1 = �q1+�q2 so q1 and q2 are coprime.

Corollary 2: If g is a power of a prime � (i.e., g = �i, i > 0), theneither M1 or M2 is a minimal element of L1

k(S) and is a best rationalapproximation of S to the order k.

Proof: Clearly, either q1 or q2 is coprime to � and then coprimeto g.

In the general case, the problem is as follows.

• Given M1 and M2 (a minimal basis of Lk(S)), find two integers� and � such that q = �q1 + �q2 is coprime to g and jM j =j�M1 + �M2j is minimal.

The construction of an algorithm solving this problem in nonexpo-nential time is quite difficult. However, it is possible for some particularcases.

Lemma 9: SetM 0

1 =M1+M2 andM 0

2 =M1�M2 and chooseMamong M 0

1 and M 0

2 minimizing the norm jM j. Then M is a minimalelement of Lk(S) among those which are not a multiple of M1 or M2.Moreover, (M1;M) and (M2;M) are both basis of Lk(S).

Proof: Let M3 be a minimal element of Lk(S) among those thatare not multiple of M1 or M2. We want to prove that M3 is either M 0

1

or M 0

2.Since M1 and M2 are of opposite sign, either M1 or M2 has the

same sign as M3. Suppose that it is M1 (exchange the values of M1

and M2 if not).Let M 0

3 = M3 �M1. Since M3 and M1 have the same sign andjM3j � jM1j we obtain jM 0

3j < jM3j From the conditions of mini-mality on M3 and the fact that it is not a multiple of M1 we deducethat M 0

3 is a multiple of M2.Let � be the nonzero integer such thatM 0

3 = �M2. This givesM3 =M1 + �M2.

The problem is now to find the integer � 6= 0 minimizing jM3j =jM1+�M2j. This problem looks like those solved in Theorem 6. Since

(M1;M2) is a minimal basis, the minimum of jM1+�M2j is obtainedfor � = 0. It is easy to check that the next minima for jM1 + �M2jare one of the integers immediately less than or greater than this value� = 0. This leads to � = �1 and M3 equals either M 0

1 or M 0

2.The fact that (M1;M) and (M2;M) are two basis ofLk(S) is clear:

(M1;M2) is a basis, M2 = M 0

1 �M1 = �M 0

2 +M1, and M1 =M 0

1 �M2 =M 0

2 +M2.

For a practical implementation, it is interesting to note that this min-imum M3 is one of the four vectors (r0j ; v

0

j) computed in the MBAalgorithm and needs no more computation.

If at least one element M1, M2, M = min(M1 +M2;M1 �M2)is in L1

k(S), then one of these three elements is a minimal element ofL1

k(S) and provides a best rational approximation of S to the order k.If g has at most two distinct primes in its factorization, this result is

always true.

Proposition 5: Suppose that g = �i1�i2

. Let (M1;M2) be theoutput of the MBA algorithm for S and k. One of the four elementsM1, M2, M1 +M2, or M1 �M2 is a minimal element of L1

k(S) andprovides a best rational approximation of S to the order k.

Proof: If q1 or q2 is coprime to g, the result holds. If they arenot, remember that they are coprime to each other. Suppose that �1divides q1 and �2 divides q2, then �1 and �2 do not divide q1+ q2 norq1�q2. So, both the integers q1+q2 and q1�q2 are coprime to g, andM = min(M1 +M2;M1 �M2) is a minimal element of L1

k(S).

However, this method cannot be generalized to more than two primefactors. For example, if g = 30 it is possible to construct some g-se-quences such that, for some values of k, the output of the MBA algo-rithm provides a basis (M1;M2) such that M1, M2, M1 +M2, andM1 �M2 are not in L1

k(S).

ACKNOWLEDGMENT

The authors would like to thank the referees for their comments andsuggestions, particularly for the generalization of our results on g-se-quences.

REFERENCES

[1] T. P. Berger, F. Arnault, and A. Necer, “A new class of stream cipherscombining LFSR and FCSR architectures,” in Proc. Indocrypt’02 (Lec-ture Notes in Computer Science), vol. 2551, Hyderabad, India, Dec.2002, pp. 22–33.

[2] J. von zur Gathen and J. Gerhard, Modern Computer Algebra. Cam-bridge, U.K.: Cambridge Univ. Press, 1999.

[3] A. Klapper and M. Goresky, “2-adic shift registers, fast software en-cryption,” in Proc. 1993 Cambridge Security Workshop (Lecture Notesin Computer Science), vol. 809, Cambridge, U.K, 1994, pp. 174–178.

[4] , “Cryptanalysis based on 2-adic rational approximation,” inAdvances in Cryptology, Crypto’95 (Lecture Notes in ComputerScience). Berlin, Germany: Springer-Verlag, 1995, vol. 963, pp.262–274.

[5] , “Feedback shift registers, 2-adic span, and combiners withmemory,” J. Cryptol., vol. 10, pp. 11–147, 1997.

[6] M. Goresky and A. M. Klapper, “Fibonacci and Galois representationof feedback with carry shift registers,” IEEE Trans. Inform. Theory, vol.48, pp. 2826–2836, Nov. 2002.

[7] A. Klapper and J. Xu, “Register synthesis for algebraic feedback shiftregisters based on non primes,” preprint, 2002.

[8] D. E. Knuth, “The art of computer programming,” in SeminumericalAlgorithms. Reading, MA: Addison-Wesley, 1981, vol. 2.

[9] N. Koblitz, p-adic Numbers p-adic Analysis and Zeta-Func-tions. Berlin, Germany: Springer-Verlag, 1997.

[10] K. Mahler, “On a geometrical representation of p-adic numbers,” Ann.Math., vol. 41, pp. 8–56, 1940.

[11] J. L. Massey, “Shift register synthesis and BCH decoding,” IEEE Trans.Inform. Theory, vol. IT-15, pp. 122–127, Jan. 1969.


[12] R. A. Rueppel, “Linear complexity and random sequences,” in Proc.Eurocrypt’85 (Lecture Notes in Computer Science). Berlin, Germany:Springer-Verlag, 1985, vol. 219, pp. 167–188.

[13] F. J. MacWilliams and N. J. A. Sloane, The Theory of Error CorrectingCodes. Amsterdam, The Netherlands: North-Holland, 1986.

[14] B. M. M. de Weger, “Approximation lattices of p -adic numbers,” J.Num. Theory, vol. 24, pp. 70–88, 1986.

[15] J. Xu and A. Klapper, “Feedback with carry shift registers over z=(n),”in Proc. SETA’98. New York: Springer-Verlag, 1998.

The Kullback–Leibler Divergence Rate BetweenMarkov Sources

Ziad Rached, Student Member, IEEE,Fady Alajaji, Senior Member, IEEE, and

L. Lorne Campbell, Life Fellow, IEEE

Abstract—In this work, we provide a computable expression for theKullback–Leibler divergence rate lim ( ) betweentwo time-invariant finite-alphabet Markov sources of arbitrary order andarbitrary initial distributions described by the probability distributions

and , respectively. We illustrate it numerically and examine itsrate of convergence. The main tools used to obtain the Kullback–Leiblerdivergence rate and its rate of convergence are the theory of nonnegativematrices and Perron–Frobenius theory. Similarly, we provide a formulafor the Shannon entropy rate lim ( ) of Markov sources andexamine its rate of convergence.

Index Terms—Classifcation, decision theory, Kullback–Leibler diver-gence rate, nonnegative matrices, pattern recognition, Perron–Frobeniustheory, rate of convergence, Shannon entropy rate, time-invariant Markovsources.

I. INTRODUCTION

Let fX1; X2; . . .g be a first-order time-invariant Markov source withfinite-alphabetX = f1; . . . ;Mg. Consider the following two differentprobability laws for this source. Under the first law

PrfX1= ig =: pi and PrfXk+1=jjXk= ig =: pij ; i; j 2 X

so that

p(n)(in) :=PrfX1 = i1; . . . ; Xn = ing

= pi pi i � � � pi i ; i1; . . . ; in 2 X

while under the second law, the initial probabilities are qi, the transitionprobabilities are qij , and the n-tuple probabilities are q(n). Let p =(p1; . . . ; pM) and q = (q1; . . . ; qM) denote the initial distributionsunder p(n) and q(n), respectively.

Manuscript received July 19, 2001; revised November 13, 2003. This workwas supported in part by the Natural Sciences and Engineering ResearchCouncil of Canada. The material in this correspondence was presented in partat the Conference on Information Sciences and Systems, Princeton, NJ, March2002.

Z. Rached was with the Department of Mathematics and Statistics, Queen’sUniversity, Kingston, ON K7L 3N6, Canada. He is now with the Department ofMathematics and Statistics, Notre Dame University, Zouk Mosbeh, Keserouan,P. O. Box 72 Zouk Mikael, Lebanon.

F. Alajaji and L. L. Campbell are with the Department of Mathematics andStatistics, Queen’s University, Kingston, ON K7L 3N6, Canada (e-mail: [email protected]).

Communicated by I. E. Telatar, Associate Editor for Shannon Theory.Digital Object Identifier 10.1109/TIT.2004.826687

The Kullback–Leibler divergence [13] between two distributions pand q defined on X is given by

D(pkq) =i2X

pi logpi

qi

where the base of the logarithm is arbitrary. The application of the Kull-back–Leibler divergence can be found in many areas such as approx-imation of probability distributions [3], [12], signal processing [10],[11], [5], pattern recognition [1], [2], etc.

One natural direction for further studies is the investigation of theKullback–Leibler divergence rate

limn!1

1

nD p

(n)kq(n)

between two probability distributions p(n) and q(n) defined on Xn,where

D(p(n)kq(n)) =i 2X

p(n)(in) log

p(n)(in)

q(n)(in)

for sources with memory. In earlier work, Gray [8] proved that the Kull-back–Leibler divergence rate exists between a stationary source p(n)

and a time-invariant Markov source q(n). This result can also be foundin [18, p. 27]. In [14], the authors noted that the Kullback–Leibler di-vergence rate between ergodic Markov sources exists. In [17], Shieldspresented two examples for non-Markovian sources for which the Kull-back–Leibler divergence rate does not exist. Finally, in [5], Do providesan upper bound for the Kullback–Leibler divergence rate between sta-tionary hidden Markov sources. To the best of our knowledge, theseare the only results available in the literature about the existence and/orcomputation of the Kullback–Leibler divergence rate between sourceswith memory.

Here, we provide an explicit computable expression for the Kull-back–Leibler divergence rate between two arbitrary time-invariant (notnecessarily stationary, irreducible) finite-alphabet Markov sources.This expression, which is proved in a straightforward manner usingresults from the theory of nonnegative matrices and Perron–Frobeniustheory, has a readily usable form, making it appealing for variousanalytical studies and applications involving the divergence rate forsystems with memory.

The rest of this work is organized as follows. Preliminaries aboutthe theory of nonnegative matrices are first briefly presented in Sec-tion II. In Section III, an explicit formula for the divergence rate be-tween arbitrary time-invariant finite-alphabet Markov sources is de-rived and its rate of convergence is investigated. A similar study forthe expression and convergence rate of the Shannon entropy rate oftime-invariant (nonstationary in general) Markov sources is briefly ad-dressed in Section IV. Numerical examples are presented in Section V,and conclusions are stated in Section VI.

II. PRELIMINARIES

Matrices and vectors are positive if all their components are positiveand nonnegative if all their components are nonnegative. Throughout,A denotes an M �M nonnegative matrix with elements aij . The ijthelement of Am is denoted by a(m)

ij .

We write i ! j if a(m)ij > 0 for some positive integer m, and we

write i 6! j if a(m)ij = 0 for every positive integer m. We say that i

and j communicate and write i $ j if i ! j and j ! i. If i ! j

but j 6! i for some index j, then the index i is called inessential (ortransient); otherwise, it is called essential (or recurrent). Thus, if i isessential, i ! j implies i $ j, and there is at least one j such thati ! j.

0018-9448/04$20.00 © 2004 IEEE

feedback with carry shift registers synthesis with the euclidean algorithm

Documents