# On Stability of Cellular Neural Networks

Post on 02-Aug-2016

213 views

Embed Size (px)

TRANSCRIPT

<ul><li><p>Journal of VLSI Signal Processing 23, 429435 (1999)c 1999 Kluwer Academic Publishers. Manufactured in The Netherlands.</p><p>On Stability of Cellular Neural Networks</p><p>PIER PAOLO-CIVALLERI AND MARCO GILLIDipartimento di Elettronica, Politecnico di Torino, Corso Duca degli Abruzzi 24, I-10129 Torino, Italy</p><p>Abstract. The main results about stability of cellular neural networks (CNNs) are reviewed. Some of themare extended and reformulated, with the purpose of providing to the CNN designer simple criteria for checkingthe stability properties. A particular emphasis is given to the conditions for the stability of CNNs described byspace-invariant templates.</p><p>1. Introduction</p><p>Cellular neural networks (CNNs) are analog dynamicprocessors, suitable for solving all the computationalproblems that can be formulated in terms of local in-teractions among signals placed on a regular structure[13]. The fundamental property that distinguishes aCNN from a general neural network is the local con-nectivity, that has allowed the realization of severalhigh-speed complex VLSI chips [412].</p><p>From a mathematical point of view a CNN can beconsidered as a dynamical system described by a largeset of nonlinear differential equations, that may exhibitseveral kinds of attractors: equilibrium points, limitcycles and also non-periodic attractors (like for exam-ple a chaotic behavior) [1315]. A design techniquefor CNNs would require to determine, for given sets ofparameters, all the attractors of the network and theirdomains of attraction. This is practically impossiblefor actual CNNs, composed of far more than 2 or 3cells [1317]. In most applications (like for examplein image-processing tasks) it is required that the CNNbe stable, i.e. that after a transient all trajectories tendto a constant value.</p><p>In this paper we review the main stability resultsavailable in the literature and reformulate and extendsome of them, with the purpose of giving to the CNNdesigner simple and practical tools for checking CNNstability. We restrict our attention to the original modelof CNN, introduced in [1].</p><p>The manuscript is organized as follows. In Section IIwe review the mathematical model of a CNN (witha particular emphasis on CNNs described by space-</p><p>invariant templates, that represent the most commoncase in applications). In Sections IIIV we discuss thevarious types of stability of a CNN: complete stabil-ity in Section III, stability almost everywhere in Sec-tion IV and global asymptotic stability in Section V.We rigorously define each kind of stability in relationwith its impact on applications; then we provide a setof theorems, that represent useful and simple criteriafor checking CNN stability.</p><p>2. Mathematical Model of a CNN</p><p>We consider CNNs composed by N M cells arrangedon a regular grid. We denote the position of a cell withtwo indexes .i; j/ with the assumption that cell .0; 0/is located in the lower left corner and cell .N ;M/ islocated in the upper right corner.</p><p>Such a CNN is described by the following normal-ized state equations</p><p>Pvx.i j/ D vx.i j/ CX</p><p>.kl/2Nr .i j/QAi j;klvy.kl/</p><p>CX</p><p>.kl/2Nr .i j/QBi j;klvu.kl/ C Ii j (1)</p><p>where vx.i j/ and vu.i j/ represent the state-voltage andthe input voltage of cell .i; j/; vy.i j/ is the output volt-age, defined through a bounded monotonic Lipschitzfunction f ./,</p><p>vy.i j/ D fvx.i j/</p><p> (2)</p></li><li><p>430 Paolo-Civalleri and Gilli</p><p>In most applications f ./ has the following piecewiselinear expression:</p><p>f vx.i j/ D 12flflvx.i j/ C 1flfl flflvx.i j/ 1flfl (3)Finally Nr .i; j/ is the neighborhood of order r of cell.i; j/ defined in [1]; QA and QB are linear templates andIi j is the bias term.</p><p>A CNN is said to be space-invariant if the two tem-plates do not depend on cell .i; j/. In such a case thetemplates are described by two .2rC1/ .2rC1/ realmatrices OA and OB, satisfying QAi j;kl D OAki;l j , QBi j;kl DOBki;l j .</p><p>The state Eq. (1), using the commutative property ofthe convolution, can be written in the simplified form:</p><p>Pvx.i j/ D vx.i j/ CX</p><p>jpjr;jqjrOApqvy.iCp; jCq/</p><p>CX</p><p>jpjr;jqjrOBpqvu.iCp; jCq/ C Ii j (4)</p><p>An alternative and useful expression for the state equa-tion of a CNN is obtained by ordering the cells in someway (e.g. by rows or by columns) and by repacking thestate, the input and the output variables into the vectorsx , u and y. If we call .i; j/ the invertible ordering,we have:</p><p>x.i; j/ D vx.i j/; u.i; j/ D vu.i j/; y.i; j/ D vy.i j/ (5)</p><p>The state equations can be written in the following com-pact form:</p><p>Px D x C Ay C Bu C QI (6)</p><p>where matrices A and B are obtained through the tem-plates ( QA or OA) and ( QB or OB), as explained in [18] andQI.i; j/ D Ii j .</p><p>Hereafter we assume that the inputs vu.i j/ (and there-fore vector u) are constant and hence that the CNN isan autonomous dynamical system.</p><p>3. Completely Stable CNNs</p><p>In image processing applications the input image isloaded either as the initial state x0 or as a constant in-put u. The output image is obtained through the timeevolution of the network and is extracted from the out-put voltage y. In order to ensure a proper working of</p><p>the network it is needed that the CNN be completelystable or stable almost everywhere (i.e. stable with theexception of a set of initial conditions of measure zero).A rigorous definition of the complete stability of an au-tonomous system is reported below:</p><p>Definition 1. An autonomous dynamical system, de-scribed by the state equation:</p><p>Px D f .x/; x 2 Rn; f : Rn ! Rn (7)</p><p>is said to be completely stable (or convergent) if foreach initial condition x0 2 Rn</p><p>limt!1 x.t; x0/ D const (8)</p><p>where x.t; x0/ is the trajectory starting from x0.The following properties hold for a convergent</p><p>system:</p><p> each trajectory converges towards an equilibriumpoint; almost all trajectories converge towards a stable equi-</p><p>librium points, whereas those starting from a set ofinitial conditions of measure zero converge towardssaddle points of various indexes; the domains of attraction of the stable equilibrium</p><p>points are separated by the stable manifolds of thesaddle points [19].</p><p>The main results concerning the complete stabilityof CNNs are summarized in the theorems stated below.</p><p>Theorem 1. A sufficient condition for the completestability of a CNN described by state equations .6/;with the output function .3/; is that the matrix A besymmetric.</p><p>Proof: It is based on the introduction of suitable Lya-punov functions. The fundamental ideas are reported in[1]. A rigorous proof can be found in [21], by assumingthat the diagonal matrix D, introduced in Theorem 1,be the identity matrix. 2</p><p>Theorem 2. A sufficient condition for the completestability of a CNN described by state equations .6/;with a differentiable and invertible output functionf ./; is that the matrix A be symmetric and that allthe equilibrium points are isolated.</p><p>Proof: Reported in [20, Theorem 1]. 2</p></li><li><p>On Stability of Cellular Neural Networks 431</p><p>Theorem 3. A sufficient condition for the completestability of a CNN described by state equations .6/;with the output function .3/; is that there exists a posi-tive diagonal matrix D, such that the product D A is asymmetric matrix.</p><p>Proof: See [21, Theorem 1]. 2</p><p>Theorem 4. A sufficient condition for the completestability of a CNN described by state equations .6/;with a differentiable and invertible output functionf ./; is that there exists a positive diagonal matrix D;such that the product D A is a symmetric matrix andthat all the equilibrium points are isolated.</p><p>Proof: See [20, Corollary 1]. 2</p><p>Note that the condition given in [20, Corollary 1],(i.e. the existence of two diagonal matrices D1 and D2,such that D1 AD2 be symmetric, and D1 D2 positive def-inite) is equivalent to that of Theorem 4 above. In fact,if D1 AD2 is symmetric, then also D12 D1 AD2 D</p><p>12 D</p><p>D12 D1 A is symmetric. Hence there exists a positivediagonal matrix D D D12 D1, such that D A is symmet-ric, that is exactly the condition of stability providedby Theorem 4 above.</p><p>Note also that the proof of [21, Theorem 1] canbe easily extended to CNNs with bounded monotonicLipschitz output functions (that not necessarily are dif-ferentiable and invertible), without requiring that all theequilibrium points are isolated. Therefore Theorems 14 above can be replaced by the following theorem.</p><p>Theorem 5. A sufficient condition for the completestability of a CNN described by state equations .6/;with a bounded monotonic Lipschitz output functionf ./; is that there exists a positive diagonal matrix D;such that the product D A is a symmetric matrix.</p><p>Proof: It is sufficient to observe that the time-derivative of the output vector y.t/ can be written as:</p><p>Py.t/ D N .x/ Px.t/; N .x/ D diagfnh.xh/g (9)where nh.xh/ is equal to f 0.xh/ 0 if the derivativeexists and is not defined if the derivative does not exist.Then the proof can be carried out as shown in [21,Theorem 1], by simply noting that the inequality (12)of [21] holds because f ./ is a bounded monotonicLipschitz function and that the Lyapunov function V .t/(4) introduced in [21] is bounded, due to the fact thatf ./ is assumed to be bounded. 2</p><p>As we have already pointed out, in most of the appli-cations CNNs are defined by space invariant templatesOA and OB. As a consequence it is important to express</p><p>the stability conditions reported above in terms of thetemplate elements. This is trivial for Theorems 1 and2 (the symmetry of matrix A implies the symmetry ofthe OA template, OApq D OAqp). For what concerns The-orems 35, the following result holds:</p><p>Theorem 6. If a CNN is described by a space-invariant template OA such that:</p><p> two couples of entries satisfy OArs OAr;s > 0; OAtwOAt;w > 0 with rw ts 6D 0I all the other couples of entries different from zero</p><p>satisfy:OAnmOAn;m</p><p>D OArsOAr;s</p><p>fi OAtwOAt;w</p><p>fl(10)</p><p>with</p><p>fi D wn tmrw ts ; fl D</p><p>rm snrw ts (11)</p><p>then:</p><p> there exists a positive diagonal matrix D such thatD A is symmetricI the CNN is completely stable.</p><p>Proof: See [21, Theorem 2]. 2</p><p>Another class of completely stable networks havebeen studied in [22] and [23]. In order to understand themain results there reported, the following definitionsand lemmas are needed [22]:</p><p>Definition 2. Let P a matrix with positive diagonalelements. The comparison matrix S of P is defined assii D pii and si j D jpi j j; i 6D j .</p><p>Definition 3. A matrix with positive diagonal and non-positive off-diagonal elements is called a non-singularM-matrix if and only if all its eigenvalues have a posi-tive real part or all its principal minors are positive.</p><p>Definition 4. A matrix P with positive diagonal ele-ments is said to be strictly diagonal dominant if:</p><p>8i; pii >Xj 6Dijpi j j (12)</p></li><li><p>432 Paolo-Civalleri and Gilli</p><p>Lemma 1. Let P a matrix with positive diagonal ele-ments. There exists a positive diagonal matrix D suchthat PD is strictly diagonal dominant if and only if thecomparison matrix of P is a nonsingular M-matrix.Proof: See [22, Lemma 1]. 2</p><p>With the help of the above definitions, the main sta-bility results of [22, 23] can be expressed as follows:</p><p>Theorem 7. A sufficient condition for the completestability of a CNN is that the comparison matrix of AIis a nonsingular M-matrix .or equivalently that thereexists a positive diagonal matrix D such that .A I /Dis strictly diagonally dominant/.</p><p>Proof: See [23, Theorem 5] for the proof of thefirst part and [22, Lemma 1] for the equivalentcondition. 2</p><p>As mentioned before, for practical purposes, itis important to translate the stability condition ofTheorem 7, in terms of a set of constraints among thetemplate elements of a space-invariant network. Wereport here an original result, that represents the exten-sion of Lemma 5 of [23].</p><p>Theorem 8. A sufficient condition for the completestability of a CNN is that there exist two positive con-stants k1 and k2; such that the space invariant templateOA satisfies the relation:</p><p>OA0;0 1 >X</p><p>.p;q/6D.0;0/k p1 k</p><p>q2 j OA pqj (13)</p><p>Proof: We show that if (13) is satisfied then there ex-ists a positive diagonal matrix D such that .A I /D isstrictly diagonally dominant and therefore the stabil-ity derives through Theorem 7. We call the invert-ible ordering among the cells, such that A.i; j/; .l;k/ DQAi j;kl D OAki;l j . Condition (13) implies that for each</p><p>cell .i; j/ the following relation is satisfied:</p><p>A.i; j/; .i; j/ 1 >pDmin.Ni;r/X</p><p>pDmax.1i;r/;p 6D0</p><p>qDmin.M j;r/XqDmax.1 j;r/;q 6D0</p><p>k p1 kq2flflA.i; j/; .iCp; jCq/flfl (14)</p><p>Let us introduce now a positive diagonal matrix D,</p><p>whose entries are defined as:</p><p>8.i; j/ : D.iC1; j/ D k1 D.i; j/; D.i; jC1/ D k2 D.i; j/(15)</p><p>By substituting (15) in (14) one obtains that for each.i; j/,</p><p>A.i; j/; .i; j/ 1</p><p>D.i; j/ ></p><p>pDmin.Ni;r/XpDmax.1i;r/;p 6D0</p><p>qDmin.M j;r/X</p><p>qDmax.1 j;r/;q 6D0</p><p>flflA.i; j/; .iCp; jCq/flflD.iCp; jCq/(16)</p><p>This implies that there exists a positive diagonal ma-trix D, such that that .A I /D is strictly diagonallydominant. Theorem 7 ensures the complete stability ofthe network. 2</p><p>Finally the last class of completely stable CNNs isthat described by acyclic templates [24]. We reporthere a slight extension of the result presented in [24,Theorems 6, 7].</p><p>Theorem 9. A sufficient condition for the completestability of a CNN is that the template QAi j;kl be acyclic;i.e. satisfies the following conditions</p><p>QAi j;kl D</p><p>8>:0 .l > j OR l < j/0 l D j AND .k > i OR k < i/arbitrary otherwise</p><p>In the space-invariant case the corresponding condi-tions on the OA template becomes:</p><p>OApq D</p><p>8>:0 .q > 0 OR q < 0/0 q D 0 AND .p > 0 OR p < 0/arbitrary otherwise</p><p>Proof: it is given in [24, Theorem 6] for the first case.In the other three case the proof can be carried out in asimilar way. Note that the assumption Ai j;i j > 1 of [24,Theorem 6] is not necessary for proving the completestability. 2</p><p>4. CNNs Stable Almost Everywhere</p><p>Stability almost everywhere is a property easier toprove than complete stability, but that from a practical</p></li><li><p>On Stability of Cellular Neural Networks 433</p><p>point of view is equivalent. The formal definition isreported here:</p><p>Definition 5. An autonomous dynamical system, de-scribed by the state equation:</p><p>Px D f .x/; x 2 Rn; f : Rn ! Rn (17)</p><p>is said to be stable (or convergent) almost everywhereif for each initial condition x0 2 Rn (with at most theexception of a set of measure zero)</p><p>limt!1 x.t; x0/ D const (18)</p><p>where x.t; x0/ is the trajectory starting from x0.</p><p>The fundamental result is based on the studies oncooperative systems presented in [25] and is reportedin [26] and [24].</p><p>Theorem 10. A sufficient condition for the stabil-ity almost everywhere of a CNN; described by stateEq. .1/with a differentiable and strictly monotonic out-put function f ./; is that:</p><p> the template elements satisfy:</p><p>QAi j;kl 0 .i; j/ 6D .k; l/</p><p> the template is cell-linking; i.e. it gives rise to anirreducible matrix A.</p><p>Proof: See [26, Theorem 1]. We remark that the cell-linking property of a template can be checked through[24, Theorem 2]. 2</p><p>For space-invariant templates the results ofTheorem 10 can be simplified and extended via suit-able state transformations. We report here two stabilityconditions, that represent an equivalent formula...</p></li></ul>