convergence of discrete-time cellular neural networks with interlaced block-sequential dynamics

INTERNATIONAL JOURNAL OF CIRCUIT THEORY AND APPLICATIONS

Int. J. Circ. ¹heor. Appl., 27, 367}373 (1999)

CONVERGENCE OF DISCRETE-TIME CELLULAR NEURALNETWORKS WITH INTERLACED BLOCK-SEQUENTIAL

DYNAMICS

R. PERFETTI

Istituto di Elettronica, Universita% di Perugia, via G. Duranti 93, 06125 Perugia, Italy

SUMMARY

Some interlaced block-sequential modes of operation are introduced for discrete-time cellular neural networks(DTCNN), and the corresponding convergence conditions are investigated. It is proved that DTCNNs, under someblock-sequential updating rules, result to be convergent when the feedback templates satisfy some restrictions rathermilder than reciprocity or dominance, as required in synchronous mode. Moreover, the set of "xed points of the networkresults to be independent of the particular updating rule adopted. The drawback of desynchronization is a reduced speedof convergence, which however is tolerable in the usual case when the neighbourhood radius is small. Copyright ( 1999John Wiley & Sons, Ltd.

KEY WORDS: discrete-time cellular neural networks; block-sequential dynamics

1. INTRODUCTION

Discrete-time cellular neural networks (DTCNNs) have been introduced as a promising paradigm for imageprocessing and pattern recognition, well suited for analogue or digital discrete-time implementation.1~3

When a threshold-type nonlinearity is assumed, a DTCNN represents a sort of cellular automata machine,whose dynamic behaviour is determined by the speci"c templates and by the updating rule adopted. Inreal-time image processing applications, where speed is a main problem, synchronous operation is desirablewhere the cells are updated all at once. However, DTCNNs can fail to converge to a steady-state pattern ifsynchronous operation is adopted, unless some signi"cant restrictions are imposed on the template values(dominance condition).1

Oscillations can be avoided with reciprocal templates, using a sequential or block-sequential operation. InReference 4 it is shown how oscillations are absent in reciprocal DTCNNs if a particular block-sequentialdynamics is used, namely only non-interacting cells are updated simultaneously. This result will be extendedhere to the more general class of non-reciprocal DTCNNs, with the unique template restriction of positiveself-feedback.

Parallel updating of non-interacting cells requires (r#1)2 time steps to complete the cell grid updatingcycle, where r is the neighbourhood radius. As a consequence the convergence speed to steady state isconsiderably slower than in full synchronous operation. As shown in this paper, it is possible to reacha higher degree of parallelism, while preserving convergence, using di!erent updating rules. Some updatingrules will be proposed which guarantee the absence of oscillations under milder conditions than thoseconsidered in Reference 1. These conditions apply to a wide class of reciprocal as well as non-reciprocaltemplates, and allow to complete the cell grid updating cycle within only (r#1) or (2r#1) time-steps.Finally, it is observed that the particular updating rule does not change the set of "xed points of the network.These issues are important from the viewpoint of practical implementation of DTCNNs, since: (1) in some

*Correspondence to: Dr R. Perfetti, Institute of Electronics, University of Perugia, via G. Duranti 93, 06125 Perugia, Italy

CCC 0098}9886/99/040367}07$17.50 Received 10 November 1997Copyright ( 1999 John Wiley & Sons, Ltd. Revised 2 July 1998

applications, cloning templates are required, which are neither reciprocal nor dominant; (2) in discrete-timeimplementations of CNNs desynchronization of cells can be required by the limited degree of parallelism ofthe hardware.

2. PRELIMINARIES

We assume a DTCNN de"ned on a rectangular M]N cell grid CG. C(i, j) denotes the cell at the intersectionof row i and column j. N

r(i, j) denotes the r-neighbourhood of cell C (i, j):

Nr(i, j)"MC(k, l): Di!kD) r, D j!lD)rN. (1)

De,nition 1. Two cells C(i, j) and C(k, l) are said non-interacting if C(k, l)NNr(i, j).

If two cells C(i, j) and C(k, l) are non-interacting, then the state of C(i, j) at time t#1 does not depend onthe state of C(k, m) at time t. Obviously, the term non-interacting refers to a single time step. Over longer timesall the cells are interacting.

De,nition 2. A subset of cells pLCG, is said non-interacting if every pair of cells in p are non-interacting:

pWNr(i, j)"C(i, j) ∀C(i, j)3p

Let B"M!1, #1N. In forthcoming notation boldface letters denote points in BM]N"M!1, #1NM]N. Let

x3BM]N denote the state of the DTCNN, where xij

is the state of cell C(i, j). Let de"ne the input to cell C(i, j)as follows:

uij"(A * x )

ij" +

C(k,l)|Nr(i,j)

A (k!i, l!j)xkl (2)

1)i)M, 1)j)N, where A3R(2r`1)](2r`1) represents the feedback cloning template . In the followingwe will refer to the de"nition of template entries shown in Figure 1.

Let us consider the block-sequential dynamics described by the following equations:

xij(t#1)"G

sgn[uij(t)]

xij(t)

if C(i, j)3)(t#1)LCG

otherwise(3)

where )(t#1) is a subset of the cell grid. The sequence ) (t), t"0, 1, 2,2, de"nes the particular updatingsequence or updating rule. If each )(t) coincides with a single cell we have complete asynchronous operation.If )(t) coincides with the cell grid we have complete synchronization.

Figure 1. Template coe$cients for r"2. The remaining values are not relevant to the convergence properties proven in the paper. Notethat for r"1 all the coe$cients are involved in the stability criteria

368 R. PERFETTI

Copyright ( 1999 John Wiley & Sons, Ltd. Int. J. Circ. ¹heor. Appl., 27, 367}373 (1999)

The sign function is de"ned as follows:

sgn(u)"G#1

!1

if u*0

if u(0(4)

De,nition 3. The DTCNN dynamics is said convergent if &t0: x

ij(t)"x

ij(t0), for every i, j, and for every

t*t0.

Remark. In the following, we consider a DTCNN without external inputs, since the presence of constantexternal inputs does not a!ect the convergence of the dynamic behaviour.

Remark. It is worth noting that the set of "xed points of the DTCNN is de"ned by the following necessaryand su$cient conditions5:

xij(A*x)

ij'0 (5a)

orxij(A*x)

ij"0, x

ij#1 (5b)

for every i, j.

Hence, the set of "xed points is uniquely determined by the cloning template and does not depend on theparticular updating rule. As shown in Reference 5 some properties concerning the attractivity of "xed pointsare not in#uenced by the updating rule. Hence, given a particular template, it is possible to select theappropriate updating sequence )(t) in order to guarantee a convergent behaviour, without modifying the setof desired "xed points, and, to some extent, their basins of attraction.

3. NON-INTERACTING UPDATING RULE

In non-interacting mode of operation, the subsets )(t) in (3) are a non-interacting subsets of cells. For r"1there are four non-interacting subsets of cells, ! (1), ! (2), ! (3), ! (4) as shown in Figure 2. The non-interactingupdating rule consists in the sequential updating of the four subsets in arbitrary order. In the following, thecells in each of the four subsets are relabelled sequentially. The states of the cells in the four subsets will bedenoted by the vectors x(1), x(2), x(3), x(4). The DTCNN under non-interacting mode of operation can beconsidered a generalization of Bidirectional Associative Memories proposed by Kosko.6

The following theorem generalizes a result given in Reference 4, where all the cell interactions wereassumed reciprocal.

¹heorem 1. ¹he D¹CNN with non-interacting updating rule is convergent for arbitrary templates, providedthat a

0'0.

Proof. To simplify the formalism, the proof is given for r"1. The extension to the case r'1 isstraightforward. For r"1 there are four subsets of non-interacting cells (see Figure 2). Let us consider thefollowing Liapunov function:

<(x(1), x(2), x(3), x(4))

"!

1

2+i

a0(x(1)

i)2!

1

2+i

a0(x(2)

i)2!

1

2+i

a0(x(3)

i)2!

1

2+i

a0(x(4)

i)2

!+i

+j

A(1,2)ij

x(1)i

x(2)j!+

i

+j

A(1,3)ij

x(1)i

x(3)j

!+i

+j

A(1,4)ij

x(1)i

x(4)j!+

i

+j

A(2,4)ij

x(2)i

x(4)j

!+i

+j

A(2,3)ij

x(2)i

x(3)j!+

i

+j

A(3,4)ij

x(3)i

x(4)j

(6)

NETWORKS WITH INTERLACED BLOCK-SEQUENTIAL DYNAMICS 369


Figure 2. The four subsets of non-interacting cells considered in non-interacting dynamics, in the case r"1. They are denoted withblack, white, grey and hatched cells

where A(1,2)ij

denotes the weight of the connection from the jth cell in !(2), to the ith cell in !(1). A(1,3)ij

denotesthe weight of the connection from the jth cell in !(3), to the ith cell in !(1). The remaining coe$cients havea similar meaning. Indices i, j span the number of cells in each subset !(i). Assume that the subset !(1) isupdated synchronously at time t. The corresponding variation of the Liapunov function (6) is

*<(x(1), x(2), x(3), x(4) )

"!

1

2+i

a0(*x(1)

i)2!+

i

*x(1)i Aa0x(1)

i#+

j

A(1,2)ij

x(2)j#+

j

A(1,3)ij

x(3)j

#+j

A(1,4)ij

x(4)j B (7)

The "rst term is non-positive provided that a0'0. Taking into account (2), the second term can be rewritten

as

!+i

*x(1)i

u(1)i

where u(1)i

is the input to the ith cell in !(1). If u(1)i*0 (u(1)

i(0) then Dx(1)

i*0 (Dx(1)

i)0). Then *<)0. The

equality sign applies only when the state of all the cells remains unchanged. Since < is bounded from below,the system converges to a stable state where < holds constant.

Remark. It is worth noting that a slightly di!erent result would be obtained if we consider unchanged thecell output when the cell input is zero. In this case the convergence is guaranteed if a

0*0.

4. INTERACTING UPDATING RULES

Note that the periodicity of the updating sequence ) (t) is not required in the proof of Theorem 1. However, inpractice, it is convenient that non-interacting subsets are updated sequentially and cyclically, so that thewhole cell grid is updated in the minimum number of time steps. In general, there are (r#1)2 non-interactingsubsets in a CNN with r-neighbourhood. Hence, the number of time steps necessary to update the wholenetwork increases rapidly with r. To reduce the updating time of the whole network it is necessary to updateinteracting cells simultaneously. In the following, we consider four distinct interacting updating rules:row-wise updating, column-wise updating, diagonal updating, and "nally chess-board updating (applicableonly in the case r"1).

370 R. PERFETTI


Figure 3. (a) Subsets of non-interacting rows for r"1, and (b) for r"2. These subsets are updated sequentially. Arrows denote theextension of the interactions among the cells. (c) The three subsets of non-interacting diagonals for r"1. (d) Chess-board dynamics: grey

and white cells are updated alternately

Row-wise updating is shown in Figures 3(a) and (b). In Figure 3(a) the case r"1 is shown. Thereare two subsets of rows, namely even-numbered rows and odd-numbered rows. Arrows describethe extension of the interactions among the cells. The two subset of rows are updated alternately.In each time step are updated synchronously all the cells of a subset. The cells updated synchronouslyinteract along the rows, but not in other directions. In Figure 3(b), the case r"2 is shown. In thiscase even-numbered and odd-numbered rows do interact. So we have to decompose the cell gridinto three subsets of non-interacting rows, shown as grey, white and hatched in the "gure. In general,the cell grid is decomposed into (r#1) subsets !(k), each of them consisting of non-interacting rows.The cells of each subset are updated simultaneously according to (3). The di!erent subsets are updatedsequentially. The updating cycle is completed in r#1 time steps. Column-wise updating is the sameas row-wise updating, ex-changing the term row with column. Diagonal updating is shown in Figure 3(c)for r"1. In this case, the cell grid is decomposed into three subsets, each composed of alternatingdiagonals, shown with di!erent colours. The cells updated synchronously interact only along one of thetwo diagonals. In general it requires 2r#1 time steps to update all the cells in the array. Finally, chess-board updating consists in the alternate updating of the two subsets shown in Figure 3(d). In thiscase cells updated synchronously interact along both the diagonals. This rule can be adopted onlyfor r"1.



De,nition 4. With reference to Figure 1, the template is said row-symmetric if ai"a

~i, for all i.

A row-symmetric template is said row-dominant if

a0'2 +

i;0

DaiD (8)

The template is said column-symmetric if bi"b

~i, for all i.

A column-symmetric template is said column-dominant if

a0'2 +

i;0

DbiD (9)

The template is said diagonally-symmetric if ci"c

~ifor all i, or d

i"d

~ifor all i.

A diagonally symmetric template is said diagonally dominant if

a0'2 +

i;0

DciD or a

0'2 +

i;0

DdiD (10)

respectively.The 3]3 template is cross-symmetric if c

1"c

~1and d

1"d

~1.

A cross-symmetric template is said cross-dominant if a0'2(Dc

1D#Dd

1D).

Remark. The above restrictions on the template values are rather milder than those required by reciprocityor dominance. Namely, a reciprocal template is, at the same time, row-symmetric, column-symmetric, anddiagonally symmetric with respect to both diagonals.

In a dominant template the self-feedback is greater than the sum of the absolute values of the remainingtemplate coe$cients. A dominant template is, at the same time, row-dominant, column-dominant andcross-dominant.

¹heorem 2. Assume a row-wise updating of the cell-grid. If the template is row-dominant then the D¹CNNdynamics is convergent.

Proof. The proof is given for r"1. The extension for r'1 is straightforward. For r"1 there are twosubsets of non-interacting rows (Figure (3a). Let x(1), x(2) denote the states of the two subsets of rows !(1), !(2).Let consider the following Liapunov function:

<(x(1) , x(2))"!

1

2+i

+j

A(1,1)ij

x(1)i

x(1)j!

1

2+i

+j

A(2,2)ij

x(2)i

x(2)j!+

i

+j

A(1,2)ij

x(1)i

x(2)j

(11)

where x(1)i

denotes the state of ith cell in !(1), and x(2)i

denotes the state of jth cell in !(2). A(1,1)ij

denotes theweight of the connection from the jth cell to the ith cell in !(1). A(2,2)

ijdenotes the weight of the connection

from the jth cell to the ith cell in !(2). A(1,2)ij

denotes the weight of the connection from the jth cell in !(1) to theith cell in !(2).

Assuming synchronous updating of cells in the rows of !(1), and taking into account that A(1,1)ij

"A(1,1)ji

forevery i, j, since the template is row-symmetric, it is

*<(x(1), x(2))"!

1

2+i

+j

*x(1)i

A(1)ji

x(1)j!

1

2+i

+j

*x(1)i

A(1,1)ij

x(1)j

!

1

2+i

+j

*x(1)i

*x(1)j

A(1,1)ij

!+i

*x(1)i

+j

A(1,2)ij

x(2)j

"!+i

*x(1)i A+

j

A(1,1)ij

x(1)j#+

j

A(1,2)ij

x(2)j B!

1

2+i

+j

*x(1)i

*x(1)j

A(1,1)ij

(12)

372 R. PERFETTI


The "rst term in the last expression can be rewritten as !+k

*x(1)k

u(1)k

where u(1)k

is the input to kth cell in!(1). If u(1)

k*0 (u(1)

k(0) then *x(1)

k*0 (*x(1)

k)0). The second term is a quadratic function which is positive

de"nite provided that the template is row-dominant. In fact, the rows of matrix A(1,1) have a0on the diagonal

and the remaining non-zero elements are ai, a

~i, i"1, 2,2 . Then *<)0. The equality sign applies only

when the state of the cells remains unchanged. Since < is bounded from below, the system converges toa stable state where < holds constant.

¹heorem 3. Assume a column-wise updating of the cell grid. If the template is column-dominant then theD¹CNN dynamics is convergent.

¹heorem 4. Assume a diagonal updating of the cell grid. If the template is diagonally dominant then theD¹CNN dynamics is convergent.

The proofs of Theorems 3 and 4 are similar to that of Theorem 2.

¹heorem 5. Assume r"1 and the chess-board updating of the cell grid shown in Figure 3(d). If the template iscross-dominant then the dynamics is convergent.

Proof. The proof can be carried out as for Theorem 2. The o!-diagonal coe$cients of the quadraticfunction are the elements c

1and d

1.

Remark. If we consider unchanged the cell state when the cell input is zero, the above results applyreplacing the sign'with*in De"nition 4.

CONCLUSIONS

Using non-interacting updating every template with non-negative self-feedback guarantees a convergentbehaviour. However, it requires to decompose the updating cycle into (r#1)2 time steps. Then, severalinteracting updating rules with a higher degree of synchronization among the cells have been proposed. Thecorresponding stability conditions have been derived. They apply to templates which are neither symmetricnor dominant. Given a speci"c feedback template, if it satis"es one or more of the conditions above, we canselect the appropriate updating rule preserving the convergence and requiring less time steps to complete theupdating cycle (r#1 or 2r#1 time steps) with respect to non-interacting dynamics.

REFERENCES

1. H. Harrer and J. A. Nossek, &Discrete-time cellular neural networks', Int. J. Cir. ¹heor. Appl., 20, 453}467 (1992).2. H. Harrer, Discrete-¹ime Cellular Neural Networks, Verlag Shaker, Aachen, 1992.3. H. Harrer, J. A. Nossek, and R. Stelzl, &An analog implementation of discrete-time cellular neural networks', IEEE ¹rans. Neural

Networks, 3, 466}476 (1992).4. R. Perfetti, &On the convergence of reciprocal discrete-time cellular neural networks', IEEE ¹rans. Circuits and Systems2 I: Fundam.

¹heor. Appl., CAS 40, 286}287 (1993).5. R. Perfetti, &Some properties of the attractors of discrete-time cellular neural networks', Int. J. Circ. ¹heor. Appl., 23, 485}499 (1995).6. B. Kosko, &Bidirectional associative memories', IEEE ¹rans. Systems Man Cybernet., SMC-18 (1) pp. 49}60 (1998).



convergence of discrete-time cellular neural networks with interlaced block-sequential dynamics

Documents