202 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—II: EXPRESS BRIEFS, VOL. 53, NO. 3, MARCH 2006

Invariance Principle and Complete Stability forCellular Neural Networks

Xuemei Li, Chaoqun Ma, and Lihong Huang

Abstract—In applications of classification of patterns, image pro-cessing, associative memories etc, the complete stability of cellularneural networks (CNNs) plays an important role. Invariance prin-ciples based on the Lyapunov functions and functionals are stillthe most advantageous theory to analyze the complete stability.However, one difficulty in applying classical invariance principlesto the complete stability is to prove that the largest invariant setconsists of equilibrium points. In this paper, we present one invari-ance principle to analyze the complete stability. We can avoid thedifficulty of proving that the largest invariant set is constituted ofequilibrium points in discussing some sufficient condition for com-plete stability of CNNs by using this invariance principle.

Index Terms—Cellular neural network (CNN), complete sta-bility, delay, invariance principle.

I. INTRODUCTION

CELLULAR neural networks (CNNs), introduced by Chuaand Yang in 1988 [3], is one of the most popular models in

the literature of artificial neural networks. CNN shares the bestfeatures: local interconnection feature making it to be easily re-alizable for the very large-scale integration (VLSI) implemen-tation either as planar or as multilayer structures and real-timecontinuous and high speed parallel signal processing feature. Byordering the cells, in the case where cells are linear interactionsand the input of each cell is constant, the state equation of CNNis described by the first-order nonlinear differential system [3]

(1)

and the state equation of a delayed CNN (DCNN) can be writtenas follows [7]:

(2)

where the input vector is a constant vector,and are the state vector and the output vector,respectively. is a positive diagonal matrix,

, and , are called feedback matrixand delayed feedback matrix respectively, the time delay is anonnegative constant, and

(3)

Manuscript received January 11, 2005. This work was supported by the NNSFof China (10371034, 70471030). This paper was recommended by AssociateEditor C. W. Wu.

X. Li and C. Ma are with Department of Mathematics, Hunan Normal Uni-versity, Hunan 410081, China (e-mail: Liu3080@sohu.com).

L. Huang is with College of Mathematics and Econometrics, Hunan Univer-sity, Hunan 410082, China (e-mail: lhhuang@mail.hunu.edu.cn).

Digital Object Identifier 10.1109/TCSII.2005.857086

with the output function given byfor .

The delays in electronic neural networks are usually timevarying, due to the finite switching speed of amplifiers and faultsin the electrical circuit. Hence, a more general form of stateequation of a DCNN is

(4)

which includes the special models (1) and (2) that have beenstudied in many papers.

CNNs have very important applications in different areas [4],[14], [15]. The most famous applications are image processingand pattern recognition. In these applications, a CNN possessesa lot equilibrium points (i.e., stationary states) and must becompletely stable (a neural network is said to be completelystable, if each trajectory converges to one of these equilibriumpoints). It is important to discuss conditions ensuring completestability of networks. So far, some sufficient conditions forcomplete stability of CNNs without delays and with delayshave been obtained [2], [3], [5]–[7], [11], [12], [16], [18]. It isgenerally difficult to establish the complete stability. For thesimplest case where the feedback matrix is symmetric and thedelayed feedback matrix is zero matrix, there are four papers[3], [6], [12], [18] to complete the proof of the complete sta-bility. General approaches used for the analysis of the completestability are invariance principles, including the LaSalle invari-ance principle for finite dimensional differential equations, andthese invariance principles based on the Lyapunov functionalmethod and the Lyapunov function method together with Razu-mikhin-type techniques for functional differential equations[8], [9], [13]. However, one often finds basic drawbacks inappling these classical invariance principles to CNNs becausethe output function is piecewise linear and it is difficult to provethat the largest invariant set consists of equilibrium points (see[3], [6], [12], [18]). In this paper, utilizing the special feature ofCNN models, we obtain an invariance principle for completestability of CNNs. According to this invariance principle, thelargest invariant set only consists of equilibrium points. Thus,we overcome the difficulty of proving that the largest invariantset is the set of equilibrium points. As will be illustrated, itis easy to prove complete stability of CNNs by using thisinvariance principle. In Section II, we present the invarianceprinciple and we give some sufficient conditions on completestability of CNNs by using the invariance principle in SectionIII.

In the following, we introduce some definitions and notations.

1057-7130/$20.00 © 2006 IEEE

LI AND HUANG: INVARIANCE PRINCIPLE AND COMPLETE STABILITY FOR CNNs 203

Let and let denote thereal -dimensional vector space. For a given and forany given real denotes a convenient norm anddenotes the norm. denotes the Banach space of continuousfunctions mapping into , where is a given realnumber, with the norm for .If is continuous, then is defined by

for and so that. For a given matrix denotes the transposi-

tion of , and . (or )for symmetric matrices and means that is positivesemidefinite (or positive definite). representsan by diagonal matrix with diagonal entries . denotesthe identity matrix.

Let denote a solution of (4) with the initial func-tion at . We will write if .A solution of (4) is said to be convergent ifexists and is an equilibrium point of (4). A solution of (4)is said to be quasi-convergent if for every such that

exists, then this limit is an equilibrium point of(4). The network described by (4) is said to be completely stableif every solution (or trajectory) of (4) is convergent.

We note that If is a solution of (4), is piece-wise linear and possibly is not differentiable with re-spect to (when ), but its right-hand deriva-tive does exist, which is denoted by . By using

, it follows thator 0 (see [11]).

Let iff , and iff. Set . Then

(5)

Let be continuous. Along the solutionof (4) through , upper right-hand

derivative of is defined by

If we emphasize the dependence on (4), we write .In Section III, we will require some algebraic knowledge (see

[1], [10]). Let be a real square matrix. is said to be Lya-punov diagonally stable if there exists a positive diagonal ma-trix such that is positive definite. is said tobe stable if has all its eigenvalues with positive real part. issaid to be D-stable if is stable for every positive diagonalmatrix . with nonnegative diagonal and nonpositive off-di-agonal elements is called a nonsingular M-matrix if all of itseigenvalues have positive real parts. The comparison matrixof with positive diagonal elements is defined as and

.Lemma 1: Let be a Lyapunov diagonally stable matrix.

Then is D-stable.

Lemma 2: Let the comparison matrix of is a nonsingularM-matrix. Then there exists a positive diagonal matrix

such that

II. INVARIANCE PRINCIPLE

By the differential meanvalue theorem, it is easy to prove thefollowing lemma.

Lemma 3: Assume that such thatis continuous and bounded and exists on . If

, then .The main result in this section is the following invariance

principle, which represents that the largest invariant subset ofonly consists of equilibrium points.

Theorem 1: Assume that the delays aredifferentiable and there exist positive constants , and suchthat the delays satisfy

Assume that is a continuous functional, andare positive constants, and . If there is a continuous

functional such that

and along any solution of (4),

then every solution of (4) is quasi-convergent, furthermore, inthe case where all equilibria of (4) are isolated, (4) is completelystable.

Proof: Denote. By the boundedness of solutions of (4) and the hy-

potheses, we obtain that is monotonic and boundedfunction of . Hence

Due to the equivalence between norms in , we also have, i.e., . Next, we will

prove . From (4), one can derive

Using integration by parts, for can be expressed as

204 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—II: EXPRESS BRIEFS, VOL. 53, NO. 3, MARCH 2006

where , and. Let . Then

(6)

where is some positive constant. In order to prove, we only need to prove that the later two terms of (6)

belong to . We now distinguish two cases.Case I: . Let . By the Holder in-

equality, it implies that

and, similarly

Therefore, .Case II: . Similar to the above processing except using

the Holder inequality, we also have .In summary, . As is uniformly con-

tinuous on , we obtain

By using andLemma 3, we have

(7)

By the boundedness of , assume that withas . (4), and (7) imply

i.e., is an equilibrium point of (4). Thus, is quasi-conver-gent. In the case where all equilibria are isolated, is con-vergent. The proof of the theorem is complete.

Corollary 1: If in Theorem 1 is equal to 1, then (4) is com-pletely stable whether or not all equilibrium points of (4) areisolated.

Proof: From the proof of Theorem, it follows that. Hence, for any positive , there is

such that for . Thus,

By Cauchy convergence principle on limit existence, it impliesthat exists and this limit is an equilibrium pointbecause of .

In the special case and , we obtain a completestability result of CNN (1).

Corollary 2: Let and be positive constants and .If there is a continuous function such that alongany solution of (1)

then every solution of (1) is quasi-convergent, furthermore, inthe case where all equilibria of (1) are isolated or , (1) iscompletely stable.

III. PARTICULAR CASES

We present two results below to illustrate the significance ofTheorem 1 and Corollary 2.

Consider DCNN (4) with , and, i.e.,

(8)

where satisfies

for some constants and .Theorem 2: If there is a positive diagonal matrix such that

and are symmetric matrices, and ,then every solution of (8) is quasi-convergent, and (8) is com-pletely stable in the case where all equilibria are isolated.

Proof: Let . By (5), (8) implies that

(9)

LI AND HUANG: INVARIANCE PRINCIPLE AND COMPLETE STABILITY FOR CNNs 205

Consider the Lyapunov functional

where is a positive function with its positive continuousderivative on , which will be determined in the followingproof. The Lyapunov functional above is similar to that pro-posed in [7]. Noting that is a symmetric matrix, and using(9) and (5), by a calculating procedure similar to that in [7] or[5], along solutions of (8), satisfies

(10)

where

and is a symmetric matrix. Set , thenis the following matrix:

Clearly, is a principal submatrix of . In the following,we show that there exists with and

for such that the symmetric matrix ispositive definite, or equivalently, the matrix is positivedefinite. Indeed, for any with

Hence, in order that is positive definite, we only need

(11)

(12)

By the hypothesis which is symmetric, (11) can berewritten as

Set . If the inequality

(13)

holds, then is positive definite. Therefore, is Lya-punov diagonally stable, and by Lemma 1, is D-stable. Thus,since is symmetric, we obtain that is positive definite,i.e., (11) holds if (13) holds. Hence, if we can find such a func-tion such that (12), and (13) satisfy, is positive defi-nite. We assume

where and to ensure the continuity of on. Clearly, always satisfies (12). If there exists

such that

(14)

holds, we can take such that

Note that (14) is equivalent to

Thus, the only constraint implies that, which must be compatible with (13). We obtain

that is the assumption . Thus, we have provedthat for is positive definite. By (10), thereexists a positive constant such that

Moreover, we have

By Theorem 1, Theorem 2 is implied.Remark 1: The complete stability of (8) in the case where

is a constant function , has been studied in [5]and [7]. It was proved that (8) with is completelystable under conditions which and issymmetric (in [5]), or and and aresymmetric with some positive diagonal matrix (in [7]). In

206 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—II: EXPRESS BRIEFS, VOL. 53, NO. 3, MARCH 2006

such case, in Theorem 2, the constrain on the delayand the delayed feedback matrix changes into ,which is weaker than that in [5] and [7].

Theorem 3: If the comparison matrix of is a nonsin-gular M-matrix, then (1) is completely stable.

Proof: By Lemma 2, there exists a positive diagonal ma-trix such that

Let

By (5) and (1), we have or. Consider the

Lyapunov function

As the right-hand derivative of exists, alongsolutions of (1),

By Corollary 2, (1) is completely stable.Remark 2: In [17], Takahashi and Chua have discussed the

complete stability of (1) with . They proved that if thecomparison matrix of is a nonsingular M-matrix, then(1) with is completely stable by using the convergencetheorem of the Gauss-Seidel method, which is an iterative tech-nique to solve linear algebraic equations. Their proof is very

complex. The proof here is simpler by using the invariance prin-ciple in this paper.

Remark 3: The invariance principle in this paper can sim-plify the procedure proving the complete stability of CNNs insome papers, such as [5], [11], [12], and [16] like that in Theo-rems 2 and 3.

REFERENCES

[1] A. Berman and R. J. Plemmons, Nonnegative Matrices in the Mathemat-ical Sciences. New York: Academic, 1979.

[2] L. O. Chua and C. W. Wu, “On the universe of stable cellular neuralnetworks,” Int. J. Circuit Theory Appl., vol. 20, pp. 497–517, 1992.

[3] L. O. Chua and L. Yang, “Cellular neural networks: Theory,” IEEETrans. Circuits Syst. I, Fundam. Theory Appl., vol. 35, no. 12, pp.1257–1272, Dec. 1988.

[4] , “Cellular neural networks: Applications,” IEEE Trans. CircuitsSyst. I, Fundam. Theory Appl., vol. 35, no. 12, pp. l273–1290, Dec. 1988.

[5] P. P. Civalleri, M. Gilli, and L. Pandolfi, “On stability of cellular neuralnetworks with delay,” IEEE Trans. Circuits Syst. I, Fundam. TheoryAppl., vol. 40, no. 1, pp. l57–165, Jan. 1993.

[6] M. Forti and A. Tesi, “A new method to analyze complete stabilityof PWL cellular neural networks,” Int. J. Bifurc. Chaos, vol. 11, pp.655–676, 2001.

[7] M. Gilli, “Stability of cellular neural networks and delayed cellularneural networks with nonpositive templates and nonmonotonic outputfunctions,” IEEE Trans. Circuits Syst. I, Fundam. Theory Appl., vol. 41,no. 5, pp. 518–528, May 1994.

[8] J. R. Haddock and J. Terjeki, “Liapunov-Razumikhin functions and aninvariance principle for functional differential equations,” J. Different.Eq., vol. 48, pp. 95–122, 1983.

[9] J. K. Hale and S. M. V. Lunel, Introduction to Functional DifferentialEquations. New York: Springer-Verlag, 1993.

[10] D. Hershkowitz, “Recent directions in matrix stability,” Linear AlgebraAppl., vol. 171, pp. 161–186, 1992.

[11] X. Li, L. Huang, and J. Wu, “Further results on the stability of delayedcellular neural networks,” IEEE Trans. Circuits Syst. I, Fundam. TheoryAppl., vol. 50, no. 8, pp. 1239–1242, Aug. 2003.

[12] S. Lin and C. Shih, “Complete stability for standard cellular neural net-works,” Int. J. Bifurc. Chaos, vol. 9, pp. 909–918, 1999.

[13] H. M. Rodrigues, L. F. C. Alberto, and N. G. Bretas, “On the invarianceprinciple, generalizations and applications to synchronization,” IEEETrans. Circuits Syst. I, Fundam. Theory Appl., vol. 47, no. 7, pp.730–739, Jul. 2000.

[14] T. Roska and J. Vandewalle, Cellular Neural Networks. New York:Wiley, 1993.

[15] , Cellular Neural Networks. New York: Wiley, 1995.[16] N. Takahashi, “A new sufficient condition for complete stability of cel-

lular neural networks with delay,” IEEE Trans. Circuits Syst. I, Fundam.Theory Appl., vol. 47, no. 7, pp. 793–799, Jul. 2000.

[17] N. Takahashi and L. O. Chua, “On the complete stability of nonsym-metric cellular neural networks,” IEEE Trans. Circuits Syst. I, Fundam.Theory Appl., vol. 45, no. 7, pp. 754–758, Jul. 1998.

[18] C. W. Wu and L. O. Chua, “A more rigorous proof of complete sta-bility of cellular neural networks,” IEEE Trans. Circuits Syst. I, Fundam.Theory Appl., vol. 44, no. 3, pp. 370–371, Mar. 1997.