PHYSICAL REVIEW E 89, 042903 (2014)

Restricted feedback control in discrete-time dynamical systems with memory

Kathryn G. Workman,1 Shuang Zhao,1 and John W. Cain1,2

1Department of Mathematics and Computer Science, University of Richmond, 28 Westhampton Way, Richmond, Virginia 23173, USA2Department of Mathematics, Harvard University, One Oxford Street, Cambridge, Massachusetts 02138, USA

(Received 18 November 2013; revised manuscript received 20 February 2014; published 4 April 2014)

When an equilibrium state of a physical or biological system suffers a loss of stability (e.g., via a bifurcation), itmay be both possible and desirable to stabilize the equilibrium via closed-loop feedback control. Significant efforthas been devoted towards using such control to prevent oscillatory or chaotic behavior in dynamical systems, bothcontinuous-time and discrete-time. Regarding control in discrete-time systems, most prior attempts to stabilizeunstable equilibria require that the system be perturbed once during each time step. However, there are examplesof systems for which this is neither feasible nor possible. In this paper, we analyze a restricted feedback controlmethod for discrete-time systems (restricted in the sense that the controller’s perturbations may be applied onlyin every other time step). We apply our theoretical analysis to a specific example from cardiac electrophysiologyin which this sort of restricted feedback control is especially relevant. The example is a useful test case for thetheory, and one for which an experimental setup is rather straightforward.

DOI: 10.1103/PhysRevE.89.042903 PACS number(s): 05.45.Gg, 87.19.Hh

I. INTRODUCTION

Closed-loop feedback can be an effective way to controlinstabilities in nonlinear dynamical systems. The underlyingprinciple of feedback control is this: By applying smallperturbations to an accessible system parameter, it may bepossible to convert an unstable state of a system into astable, attracting state, or to prevent undesirable bifurcations,oscillations, or chaos. Some readers may be familiar with theclassical control problem of stabilizing an inverted pendulum[1] mounted to a moving cart, i.e., “balancing a broomstick onthe palm of your hand.” There are two distinctions betweenthat problem and the ones that we shall analyze in this article.First, and for emphasis, we consider feedback control appliedvia perturbations to a parameter, not directly to the state vari-able(s). For the inverted pendulum example, moving the cartwould directly influence the angle θ of the pendulum relative tothe upward vertical (a state variable), as well as its derivative.Second, our focus is on discrete-time dynamical systems.We shall use notation such as xn to denote the state of thesystem at time n, a nonnegative integer. Mappings of the formxn+1 = f (xn; μ), where f is a smooth function and μ is the“accessible system parameter,” will be of particular interest.

In the early 1990s, Ott, Grebogi, and Yorke (OGY)[2] proposed one of the first feedback control techniquesfor preventing periodic and aperiodic behavior in physicalsystems. Their method for stabilizing an unstable equilibriumstate x∗ was to perturb some system parameter by an amountproportional to xn − x∗, the difference between the mostrecent state of the system and the equilibrium being targetedfor stabilization. While this idea makes sense intuitivelyand can be an effective method for chaos control, it hasone notable drawback: The OGY method requires advanceknowledge of the unstable state x∗, a state which might bedifficult or impossible to find in many applications. Pyragas[3] proposed an alternative, namely that the perturbations bechosen proportional to xn − xn−1, the difference between thetwo most recent states of the system. Applied to a mapping ofthe type mentioned above, such a method takes the form

xn+1 = f (xn; μ + γ (xn − xn−1)), (1)

where the proportionality constant γ is commonly referredto as the feedback gain. Many subsequent studies (see, forexample, [4–8]) have implemented Pyragas’ technique andminor variants thereof; following the authors of [7], we shallrefer to Pyragas’ method as time-delay autosynchronization(TDAS). TDAS and some of its generalizations have beenanalyzed mathematically in the aforementioned studies withseveral goals in mind, such as (i) determining the method’spotential for successful stabilization of unstable states and (ii)optimizing its use in experimental setups.

In this paper, we adapt the TDAS method to the case inwhich control can be applied only in every other cycle, sothat the controller operates in an ON-OFF pattern. Thereare systems for which this sort of constraint is natural,and throughout the paper we shall refer frequently to onesuch example from cardiac electrophysiology. In that context,the discrete-time model is appropriate due to the inherentlydiscrete nature of the beating of the heart, and we shall useTDAS to prevent a period-doubling bifurcation associatedwith the onset of an arrhythmia known as T-wave alternans.Our mathematical analysis of the restricted TDAS methodadapts and extends previous work [9,10] in two ways. First,we analyze control in a discrete-time system with short-termmemory in the sense that the system depends upon more thanone previous state. Second, we analyze a variant of restrictedTDAS in which the perturbations applied by the controllermust compete with impulsive forcing that is native to thesystem itself. In doing so, the example we have in mind isagain from cardiac electrophysiology: competition betweenthe heart’s native electrical stimuli and the stimuli applied viaa pacer electrode. Our simulations suggest that, by automatinga pacer electrode with an algorithm that does not require theelectrode to fire a stimulus in every beat, it is still possible toterminate the abnormal alternans rhythm. Using less energy toachieve the same results may offer advantages in the form ofprolonged battery life for an implantable pacemaker device.

The remainder of the paper is organized as follows. InSecs. II A and II B, linear stability analyses are used topredict parameter regimes in which the (unrestricted) TDASand (restricted) ON-OFF TDAS algorithms could stabilize

1539-3755/2014/89(4)/042903(9) 042903-1 ©2014 American Physical Society

KATHRYN G. WORKMAN, SHUANG ZHAO, AND JOHN W. CAIN PHYSICAL REVIEW E 89, 042903 (2014)

unstable equilibria. Section II C introduces a specific problemfrom cardiac electrophysiology: feedback control of alternansvia perturbations to the timing of electrical stimuli. For reasonsexplained in that section, such perturbations need not beapplied during every heartbeat, only in every other beat. Aslight variant of the ON-OFF TDAS method is introducedto simulate competition between the heart’s native stimuliand the “artificial” stimuli supplied by a pacer electrode. InSec. III, we analyze the (unrestricted) TDAS and ON-OFFTDAS as applied to a memoryless restitution mapping, andcharacterize the parameter regime in which those methodsare predicted to succeed. Section IV contains similar analysesapplied to a restitution mapping with memory, in which casethe mathematics is more tedious but equally straightforward.Going beyond the local stability analysis, we compute thebasins of attraction of fixed points stabilized by ON-OFFTDAS, thereby providing information about the robustnessof control. Those results are presented in Sec. IV C, and arequite encouraging. The final section summarizes our findings,highlights the advantages and shortcomings of restrictedfeedback control, and relates problems for further study.

II. TDAS CONTROL

A. Unrestricted TDAS

Here, we recall the control domain (defined below) forTDAS as applied to a simple, one-dimensional mapping; fora similar derivation, see [11]. Let x∗ denote an unstable fixedpoint of the mapping

xn+1 = f (xn; μ∗), (2)

that is, x∗ = f (x∗; μ∗). Suppose that we wish to stabilize x∗ byapplying perturbations to μ∗ during each iteration, replacingμ∗ by μ∗ + εn. TDAS chooses the perturbations εn accordingto the rule

εn = γ (xn − xn−1), (3)

where γ is a proportionality constant called the feedback gain.In other words, TDAS perturbs the parameter μ by an amountproportional to the difference between the two most recentstates of the system.

Combining (2) and (3) results in a higher-dimensionalmapping

xn+1 = f (xn; μ∗ + γ (xn − xn−1)),

for which x∗ is still a fixed point. Introducing an auxiliary vari-able zn = xn−1 allows us to write this higher-order recurrenceas the first-order system

xn+1 = f (xn; μ∗ + γ (xn − zn)),

zn+1 = xn.(4)

The vector (x∗,x∗) is a fixed point of this system, and theprimary goal of applying TDAS is to choose the feedback gainγ in such a way that the fixed point is stabilized. During theprocess of linearizing this system about the fixed point, onenaturally encounters the Jacobian matrix

J =[D1 + γD2 −γD2

1 0

], (5)

where D1 = D1f (x∗; μ∗) and D2 = D2f (x∗; μ∗) are deriva-tives of f with respect to its first and second arguments,respectively, evaluated at the fixed point. Note that |D1| � 1since the fixed point was assumed to be unstable. Botheigenvalues of a 2 × 2 matrix J have modulus less than 1if and only if the trace and determinant obey the inequalities

(i) det J < 1, (ii) trJ − det J < 1, (iii) trJ + det J > −1.

(6)

Applied to (5) and assuming that D2 > 0, these three condi-tions require

(i) γ <1

D2, (ii) D1 < 1, (iii) γ >

−1 − D1

2D2. (7)

The inequalities (7) define the control domain for unrestrictedTDAS, the set of γ for which the unstable fixed point may bestabilized by feedback control.

To plot the control domain, suppose that D1 < −1 so thatthe reason for instability is due to alternation of growingamplitude, i.e., the sort of instability that one associates withperiod-doubling bifurcations. In particular, the second inequal-ity in (7) is automatically satisfied. Figure 1(a) illustratesthe control domain (shaded region) under these assumptions,rendered as a plot of γD2 versus D1. In the figure, we haveopted not to extend the shading beyond D1 = −1 since x∗

D 1D 1

D 2 D 2

−1 0

00−1

−1

11

2

−30−1−3

γγ (a) (b)

FIG. 1. (a) Unrestricted TDAS control domain defined by inequalities (7), assuming D1 < −1 and D2 > 0. (b) Restricted TDAS controldomain defined by inequalities (8) under the same assumptions.

042903-2

RESTRICTED FEEDBACK CONTROL IN DISCRETE-TIME . . . PHYSICAL REVIEW E 89, 042903 (2014)

would be stable if |D1| < 1. In fact, x∗ would remain stableeven in the presence of “anticontrol” (negative γ ), providedthat γ still obeys inequality (7)(iii).

B. Restricted TDAS

As before, suppose that x∗ is an unstable fixed point ofthe mapping xn+1 = f (xn; μ∗), and that we wish to stabilizex∗ by appropriate perturbations of μ∗. If the perturbations arerestricted in that they are applied only at every other iteration,we obtain a system

xn+1 = f (xn; μ∗), (control off)

xn+2 = f (xn+1; μ∗ + γ (xn+1 − xn)). (control on)

Written as a single equation,

xn+2 = f [f (xn; μ∗); μ∗ + γ (f (xn; μ∗) − xn)].

Linearizing about x∗, one obtains

xn+2 − x∗ ≈ [D2

1 + γD2(D1 − 1)](xn − x∗),

where D1 and D2 denote derivatives of f with respect to itsfirst and second arguments, respectively, measured at the point(x∗; μ∗). It follows that the fixed point x∗ can be stabilized ifγ is chosen such that

− 1 < D21 + γD2(D1 − 1) < 1. (8)

As in the previous section, suppose that D1 < −1 andD2 > 0. The control domain described by the inequalities(8) is depicted in Fig. 1(b) as a plot of γD2 versus D1.(A similar figure appears in [10].) The lower boundaryγ = −(1 + D1) is a line of slope −1; by comparison, thecorresponding boundary for unrestricted TDAS derived in theprevious section is a line of slope −1/2. Notice that the upperboundary γ = −(1 + D2

1)/(D1 − 1) asymptotes to the lowerboundary as D1 → −∞ because the stabilization of x∗ shouldbecome increasingly difficult as the magnitude of D1 increases.Interestingly, the control domain for restricted ON-OFF TDASis unbounded, unlike the control domain for unrestricted TDASshown in Fig. 1(a).

C. Example: Two models of cardiac restitution

When a cardiac cell is repeatedly stimulated, or paced,by a sequence of impulsive electrical stimuli, each stimulustypically elicits a response known as an action potential. Ona single-cell level, an action potential is a prolonged elevationof the potential v across the cell’s membrane, following astimulus. Action potential duration (APD) can be measuredrelative to some threshold voltage slightly above the restingpotential; a precise definition is not important for our purposes.Suppose that pacing is periodic with period B, and let An

denote the APD following the nth stimulus. For large B (slowpacing), cells normally phase lock into a response in which thesequence {An} is roughly constant, say A∗. If the pacing periodB is varied quasistatically, this steady-state APD value A∗ alsovaries, a feature of cardiac tissue known as APD restitution.Usually, decreasing B causes A∗ to decrease because rapidheart rates cause tissue fatigue, preventing cells from stayingin an excited state (i.e., an action potential) for quite as long.If B becomes critically low, A∗ may lose stability, resulting

in an abnormal alternation of APD: alternans, illustrated inthe top panel of Fig. 3. Alternans can set a substrate for theformation of potentially deadly arrhythmias; for details see,for example, [12–15]. In Secs. III and IV, we shall explore thefeedback control of alternans using a variant of the restrictedTDAS method described above.

In the language of nonlinear dynamics, the simplest modelfor APD restitution is presented as a one-dimensional mapping

An+1 = f (An; B). (9)

The onset of alternans can be interpreted as a period-doubling[16] or border-collision [17] bifurcation that occurs as B isdecreased. If alternans occurs for some pacing period B and A∗denotes the associated [unstable] fixed point of (9) satisfyingA∗ = f (A∗; B), it is desirable to stabilize A∗ by feedbackcontrol. A specific example [5] of a restitution function f thatwas fit to experimental data from bullfrogs is given by

f (An; B) = Amax − βe−(B−An)/τ , (10)

where Amax = 392 ms, β = 525 ms, and τ = 40 ms. Forthis mapping, the bifurcation to alternans occurs when B ≈455 ms; for a bifurcation diagram of (9) and (10), see [18].

While one-dimensional restitution models of the form (9)may be appropriate in some settings, An+1 is influenced notonly by An, but also by a variety of other factors. For thisreason, we shall also analyze TDAS as applied to a memorymodel [19]

An+1 = (1 − αMn+1)G(B − An),

Mn+1 = [1 − (1 − Mn)e−An/τ ]e−(B−An)/τ ,(11)

where

G(x) = Amin + E

1 + e−(x−C)/D.

The “memory” variable M “charges up” (accumulates) duringeach action potential and decays during the resting periodbetween consecutive action potentials, both with time constantτ . During rapid pacing (small B), the increase in M thenmodulates APD via the factor αMn+1, where α > 0 is aparameter. The function G is an alternative functional formto the restitution function f in (10), sigmoidal as opposedto exponential. The parameters Amin, E, C, and D (allmeasured in milliseconds) merely translate and dilate G in thehorizontal and vertical directions. When numerical simulationsare required, we use the parameter values

α = 0.2, τ = 1000 ms, Amin = 88 ms,(12)

C = 280 ms, D = 28 ms, E = 170 or 250 ms.

We remark that these are not the same parameter choicesthat appeared originally in [19], and that we have listed twodifferent choices for E. Our purpose in doing so is to amplifythe alternans and enlarge the interval of B over which it occurs,thereby allowing us to test the robustness of feedback control.Figure 2 shows the bifurcation diagrams for (11) with theparameters listed above. Note that by increasing the parameterE from 170 to 250 ms, alternans can occur over a wider rangeof B values, and with larger amplitude.

Variants of unrestricted TDAS have been previouslyanalyzed in the context of controlling alternans; see, for

042903-3

KATHRYN G. WORKMAN, SHUANG ZHAO, AND JOHN W. CAIN PHYSICAL REVIEW E 89, 042903 (2014)

(b)(a)350

300 400 500 600 300 400 500 600 700

150

50

AP

D (

ms)

350

250

150

50

250

AP

D (

ms)

B (ms) B (ms)

FIG. 2. Bifurcation diagrams for (11) with parameters chosen from (12). (a) Bifurcation diagram for E = 170 ms. (b) Bifurcation diagramfor E = 250 ms. (Note the wider horizontal scale.)

example, [4,18,20–23]. Regarding the application of TDASto a restitution mapping, there is an important physiologicalconsideration to take into account. Perturbing B itself iscomplicated because B represents the period of the nativeelectrical stimuli that are regulated by the autonomic nervoussystem. If an implanted electrode were to supply an artificialstimulus that stimulus would either (a) elicit an action potentialif the cells were sufficiently well rested or (b) fail to elicitan action potential if the cells had been recently excitedby one of the native stimuli. In principle, one only has thefreedom to select the timing of the artificial stimuli, not thenative ones, and there is no need to fire an artificial stimulusif a native stimulus has just been supplied. Referring toFig. 3(a), suppose that alternans occurs if the native pacingperiod is B as is shown. In order for an artificial stimulusto elicit an action potential, it would need to preempt anative stimulus. Figure 3(b) illustrates how this could beaccomplished. Following a short action potential durationAn, the cell would have plenty of rest prior to the nextaction potential. Suppose that an artificial stimulus is appliedearly, preempting the (n + 1)st native stimulus (see figure),and effectively adjusting B by εn = γ (An − An−1) < 0 inaccordance with (3). After An+1, an artificial stimulus is notapplied: The controller is turned OFF because the TDAScomputed perturbation γ (An+1 − An) may well be positive,potentially requiring the artificial stimulus to occur right after

ε nB+ ε n

A n

A n−1 A n+1A n A n+2

B−

alternans(control off)

B B B

von-off after

v

TDAS control

t

t

(b)

(a)

FIG. 3. Schematic action potentials in a periodically stimulatedcardiac cell. (a) Alternans in the absence of control. (b) Introductionof TDAS control in an ON-OFF pattern after the nth action potential.Native stimuli are indicated by bold dots, and the artificial stimulusis indicated by a hollow circle.

a native stimulus. The interval between the nth and (n + 1)ststimuli would be B + εn, while the interval between the(n + 1)st and (n + 2)nd stimuli would be B − εn.

III. MEMORYLESS RESTITUTION MAPPINGS: ANALYSIS

In this section, we assume that B is such that alternanswould occur in the mapping (9) in the absence of feedbackcontrol, A∗ denotes the (unstable) fixed point of the mapping,and D1 = D1f (A∗; B) represents a slope of a restitutionfunction at the unstable fixed point. According to the sta-bility criterion for fixed points of one-dimensional mappings,D1 � −1 in order for alternans to occur in (9).

A. Control domain for unrestricted TDAS

With TDAS control, the one-dimensional mapping (9) isreplaced with a two-dimensional mapping

An+1 = f (An; B + γ (An − An−1)), (13)

which may be written as a system of the form (4) withAn = xn, An−1 = zn, and B = μ∗. Instead of rendering thecontrol domain by plotting γD2f (A∗; B) versus D1f (A∗; B)as in Fig. 1(a), in this context it is more natural to convert toa plot of γ versus B. That way, given a heart rate for whichalternans occurs, one is able to predict the range of feedbackgains γ for which TDAS may suppress alternans. Figure 4(a)shows this alternate rendering of the control domain forthe mapping (9) with the specific restitution function (10).Regarding the horizontal scale, the bifurcation to alternansoccurs at B ≈ 455 ms, and D1f (A∗; B) = −3 when B ≈ 331ms. As Fig. 1(a) predicted, TDAS can never stabilize A∗ whenthe magnitude of D1f (A∗; B) exceeds 3.

B. Control domain for restricted TDAS

In the preceding section, we were able to recycle thecalculations in Sec. II A to explore how unrestricted TDASwould be applied to a memoryless cardiac restitution mapping.A simple conversion also allowed us to rerender the generalcontrol domain from Fig. 1(a) in a more physiologicallymeaningful way [Fig. 4(a)]. To analyze the restricted ON-OFFTDAS applied to the restitution mapping, we cannot recyclethe material in Sec. II B in the same way because thosecalculations were performed without accounting for the unique

042903-4

RESTRICTED FEEDBACK CONTROL IN DISCRETE-TIME . . . PHYSICAL REVIEW E 89, 042903 (2014)

1

0

311 455B (ms)

γ (i)

(ii)

(iii)

455387

1

1.25

γ

0

B (ms)

(a) (b)

FIG. 4. (a) Control domain (shaded region) for unrestricted TDAS applied to the memoryless restitution mapping (9) and (10). Althoughthe axes have been converted to γ versus B, this is merely a reproduction of the control domain in Fig. 1(a) for a specific mapping. (b) Controldomain (shaded region) for restricted ON-OFF TDAS applied to the memoryless restitution mapping (9) and (10). The boundary curves arelabeled according to (18). As explained in Sec. III B, this is not a reproduction of Fig. 1(b).

cardiac-specific restriction that was mentioned at the end ofSec. II C [see also Fig. 3(b)]. However, adapting the analysisis rather straightforward as we shall now demonstrate.

Referring to Fig. 3(b), the ON-OFF TDAS control algo-rithm would take the following form:

[An+1

An+2

]=

[f (An; B + εn)

f (An+1; B − εn)

]

=[

f (An; B + εn)f (f (An; B + εn); B − εn)

], (14)

where εn = γ (An − An−1) is chosen according to the TDASrule (3). Inserting this expression for εn into the right-handside of the preceding equation, the system can be written inthe form

[An+1

An+2

]= �

[An−1

An

]=

[�1(An−1,An)�2(An−1,An)

]. (15)

That is, each subsequent pair of APD values can be calculatedfrom the preceding pair. Linearizing (15) about its fixed point(i.e., the vector whose components are both A∗), one finds thatthe associated Jacobian matrix is

J = D�(A∗,A∗)

=[ −γD2 D1 + γD2

−γD1D2 + γD2 D1(D1 + γD2) − γD2

], (16)

where D1 = D1f (A∗; B) and D2 = D2f (A∗; B). Applied tothe restitution mapping (9) and (10), the Jacobian matrixsimplifies because D2 = −D1:

J =[

γD1 (1 − γ )D1

−γD1(1 − D1) D1(γ − (γ − 1)D1)

]. (17)

To stabilize the fixed point, the trace 2γD1 − (γ − 1)D21

and determinant γD21 of the matrix (17) must satisfy the

inequalities (6) as follows:

(i) γ <1

D21

, (ii) γ >D2

1 − 1

2(D2

1 − D1) , (iii) γ <

D21 + 1

−2D1.

(18)

The control domain described by these three inequalities isshown in Fig. 4(b); observe that inequality (iii) is redundant,as it is uniformly less restrictive than inequality (i). Notsurprisingly, the control domain for this restricted version ofTDAS in which control is ON in every other beat is a bit smallerthan the control domain for unrestricted TDAS [Fig. 4(a)].In particular, linear stability analysis predicts that ON-OFFTDAS could never completely suppress alternans if D1 < −2.

To test the above theory, we applied both unrestricted andON-OFF TDAS to the one-dimensional restitution mapping(9) and (10). Sample results are illustrated in Fig. 5: after20 beats of (uncontrolled) alternans, we apply unrestrictedTDAS [Fig. 5(a)] and ON-OFF TDAS [Fig. 5(b)] using a

(ms)

400

240

320

400

320

2400 20 40 0 20 40

(b)(a)

nn

begin

TDAS control

begin ON-OFFTDAS controlA unrestricted

n

FIG. 5. Time series for the memoryless model (9) and (10) for B = 440 ms. During the first 20 beats, no feedback control is applied andalternans occurs. During the next 20 beats, (a) unrestricted TDAS and (b) restricted TDAS are applied with γ = 0.2.

042903-5

KATHRYN G. WORKMAN, SHUANG ZHAO, AND JOHN W. CAIN PHYSICAL REVIEW E 89, 042903 (2014)

feedback gain of γ = 0.2. In both figures, B = 440 ms, whichcorresponds to D1 ≈ −1.2. Because (B,γ ) = (440,0.2) liesin the intersection of the control domains shown in the twopanels of Fig. 4, the above theory predicts that the fixed point isstabilized regardless of whether control is applied in every beator every other beat. This is precisely what is shown in Fig. 5: thesequence {An} converges to A∗ in each case, with a slightlylonger transient if ON-OFF TDAS is used. We remark thatfor γ = 0.15, similar results are observed (not pictured here),but that the transient is longer for the unrestricted TDAS. Ofcourse, there are (B,γ ) pairs for which only the unrestrictedTDAS can stabilize A∗, and (B,γ ) pairs for which neitherversion of TDAS can succeed.

IV. RESTITUTION MAPPING WITH MEMORY: ANALYSIS

Applying TDAS to a memory model such as (11), eitherthe unrestricted or ON-OFF version, is equally straightforwardbut more tedious. Again, B is the accessible parameter thatwe perturb according to (3). Suppose that alternans occursif the native pacing period is B, and let (A∗,M∗) denotethe associated (unstable) fixed point of the uncontrolledsystem (11). (Note: We shall use row and column vectorsinterchangeably, with notational convenience in mind.)

A. Control domain for unrestricted TDAS

With (unrestricted) TDAS, the model equations (11) takethe form

(a) An+1 = (1 − αMn+1)G[B + γ (An − zn) − An],

(b) Mn+1 = [1 − (1 − Mn)e−An/τ ]e−[B+γ (An−zn)−An]/τ , (19)

(c) zn+1 = An,

where, as in the analysis of (13), we have introduced anauxiliary variable zn = An−1 to write the equation as asystem of one-dimensional mappings. Note that the Mn+1

appearing on the right-hand side of (19a) can be replacedwith the entire right-hand side of (19b), upon which the vector(An+1,Mn+1,zn+1) is expressed as a function of the vector(An,Mn,zn). The fixed point of (19) that we wish to stabilizeis (A∗,M∗,A∗), and the range of γ for which stabilization ispossible is obtained by characterizing the magnitudes of theeigenvalues of the 3 × 3 Jacobian matrix J at the fixed point.Using the Jury stability test, the eigenvalues of J are the rootsof the polynomial

p(λ) = λ3 + a1λ2 + a2λ + a3 = 0, (20)

where a1 = −traceJ , a2 = 12 [(traceJ )2 − trace(J 2)], and

a3 = − det J . All eigenvalues have modulus strictly less than1 (implying stability) if and only if

(i) p(1) > 0, (ii) p(−1) < 0,

(iii) 3 + a1 − a2 − 3a3 > 0, (21)

(iv) 1 + a1a3 − a2 − a23 > 0.

These requirements produce four inequalities that γ mustsatisfy in order for (unrestricted) TDAS to succeed. Ratherthan plotting the control domain defined by these inequalities(similar plots are shown in [22]), we provide the analogous

plot for the novel case of restricted ON-OFF TDAS in the nextsection.

B. Control domain for restricted TDAS

Referring to Fig. 3(b), applying ON-OFF TDAS control tothe model (11) yields

(a) An+1 = (1 − αMn+1)G[B + γ (An − An−1) − An],

(b) Mn+1 = [1 − (1 − Mn)e−An/τ ]e−[B+γ (An−An−1)−An]/τ ,

(c) An+2 = (1 − αMn+2)G[B − γ (An − An−1) − An+1],

(d) Mn+2 = [1 − (1 − Mn+1)e−An+1/τ ]e−[B−γ (An−An−1)−An+1]/τ .

(22)

Note that the right-hand sides of all four of these equationscan be written in terms of the variables An−1, An, and Mn. Forinstance, the right-hand side of (22b) can be substituted forthe Mn+1 appearing in (22a) and (22d). Hence, the vector(An+1,Mn+1,An+2,Mn+2) may be expressed as a function of the vector consisting of the two previous pairs ofiterates:⎡

⎢⎣An+1

Mn+1

An+2

Mn+2

⎤⎥⎦ =

⎡⎢⎣

An−1

Mn−1

An

Mn

⎤⎥⎦ =

⎡⎢⎣

1(An−1,Mn−1,An,Mn)2(An−1,Mn−1,An,Mn)3(An−1,Mn−1,An,Mn)4(An−1,Mn−1,An,Mn)

⎤⎥⎦ .

(23)

Because (A∗,M∗) was a fixed point of the uncontrolledmapping (11), it follows that (A∗,M∗,A∗,M∗) is a fixedpoint of (23). It would appear that one must characterize thedependence of the eigenvalues of a 4 × 4 Jacobian matrix onthe parameter γ to determine the control domain. However,because the right-hand side of (22) actually does not involveMn−1, each entry in the second column of the Jacobianmatrix J = D(A∗,M∗,A∗,M∗) is zero. Disregarding thezero eigenvalue, the other three eigenvalues determine thestability of the fixed point and, because they are roots of acubic polynomial, we may apply the stability test (21) to thatpolynomial. Again, we obtain four inequalities that γ mustsatisfy to stabilize the fixed point.

We shall not write out these inequalities, as they are quitelong and not especially illuminating. Instead, we shall plot thesolution of the inequalities (i.e., the control domain) usingthe two specific parameter sets in (12) above. We remark thatthe control domains for these two parameter sets are represen-tative of what one sees generally for parameter sets capableof producing alternans in (11). Figure 6 shows the controldomain for the two different choices of the parameter E. Theshaded region in Fig. 6(a) indicates the set of (B,γ ) pairsfor which restricted TDAS control can successfully suppressalternans by stabilizing the fixed point of (23). Importantly,there are choices of γ which bridge across all values of B. Theimplication is that, according to our linear stability analysis,each such γ could potentially terminate alternans regardlessof the underlying pacing period B. By contrast, note thatthe analogous region in Fig. 6(b) indicates that there are B

for which no choice of γ could possibly stabilize the fixedpoint. The differences between these two control domains canbe understood by referring to their corresponding bifurcation

042903-6

RESTRICTED FEEDBACK CONTROL IN DISCRETE-TIME . . . PHYSICAL REVIEW E 89, 042903 (2014)

B (ms) B (ms)

300 4500.0

0.5γ

1.0

500 700

γ

3000.0

0.5

1.0(a) (b)

600

FIG. 6. (a) Control domain for (22) using the parameters in (12) and E = 170 ms. (b) Same as Panel (a) except that E = 250 ms.

diagrams in Fig. 2. For E = 250 ms, there are pacing periodsB for which the amplitude of alternans is substantially higherthan in the case E = 170 ms. Certainly, it makes sense thatcontrol should be harder to achieve in the more severe case inwhich E = 250 ms.

Figure 7 shows a bifurcation diagram for (22), the memorymodel with ON-OFF control, using a feedback gain ofγ = 0.25 and the usual parameter sets (12). Figure 7(a) showsthat for E = 170 ms, the fixed point A∗ is stabilized for all B

and alternans is completely eliminated. Figure 7(b) shows thatfor E = 250 ms, there is still a window of B for which alternanscannot be eliminated by ON-OFF TDAS. Still, in comparisonto the corresponding bifurcation diagram in the absence ofcontrol [Fig. 2(b)], both the size of the alternans windowand the amplitude of alternans are substantially reduced.Incidentally, we selected γ = 0.25 under the guidance of thetheoretical control domain in Fig. 6, and it is encouraging thatthe results in Fig. 7 appear consistent with what the theorypredicts.

C. Basins of attraction

The preceding local stability analyses are useful in predict-ing how an ON-OFF controller device should be automatedto achieve stabilization. Because those results are merelylocal, to test the robustness of ON-OFF TDAS, we calculatednumerically the basins of attraction of the stabilized fixedpoints both for the memoryless restitution mapping (10) and

for the memory model (11). The numerical experiments with(9) and (10) were performed as follows.

(1) For each choice of B, compute the fixed point thatsatisfies A∗ = f (A∗; B). Then select a fixed choice for γ .

(2) Choose some separation δ > 0 between the initial APDand the fixed point APD. Set A−1 = A∗ + δ, and then generateA0 = f (A−1; B) by performing one iteration of the mappingwithout control.

(3) Starting from the initial pair (A−1,A0), use ON-OFFTDAS control (14) to generate the pairs (A1,A2), (A3,A4), upto (A499,A500).

(4) If |A500 − A∗| < 10−1, regard A−1 = A∗ + δ as be-longing to the basin of attraction for A∗. Record the point(B,δ). Repeat this process for various choices of B and δ.We considered δ as large as 200 ms, a very large discrepancybetween the initial APD and the fixed point APD for thesemapping models. Simulations for the memory model (11)were performed in a similar manner, choosing initial APDvalues at various distances δ from A∗. Regarding the initialconditions on the memory variable M , in every simulationwe set M−1 = 1.1M∗, ten percent larger than the fixed pointvalue of M , and generated M0 by performing one iteration ofthe uncontrolled mapping. Subsequent pairs of iterates wereobtained recursively from (22).

Some of our results are shown in Fig. 8, rendered as plotsof δ versus B as opposed to A∗ + δ versus B. In each panel,the basins are shaded. The panels in the first row correspond toa relatively weak feedback gain γ = 0.1, and the panels in the

B (ms) B (ms)

(b)(a)350

300 500 600 300 400 500 600 700

150

50

AP

D (

ms)

350

250

150

50

250

AP

D (

ms)

400

FIG. 7. Bifurcation diagrams for (22) with the usual parameters (12) and γ = 0.25. Compare to Fig. 2, the corresponding bifurcationdiagrams without control (i.e., γ = 0). (a) For E = 170 ms, the fixed point A∗ is stabilized regardless of B, and alternans is eliminated. (b) ForE = 250 ms, there is a window of B for which alternans cannot be eliminated even though its amplitude is substantially reduced.

042903-7

KATHRYN G. WORKMAN, SHUANG ZHAO, AND JOHN W. CAIN PHYSICAL REVIEW E 89, 042903 (2014)

B (ms) B (ms) B (ms)

100

200

0

100

200

0

100

200

0

100

200

0

100

200

0

100

200

0

(ms)

δ(m

s)δ

355 405 455

355 405 455

300 500 700 300 500 700

300 500 700300 500 700

(f)(e)(d)

(a) (c)(b)

FIG. 8. Basins of attraction (shaded) of fixed points stabilized by ON-OFF TDAS. The vertical axis shows δ, the separation between theinitial APD and the fixed point A∗. (a) Memoryless mapping (10), γ = 0.1. (b) Memory model (11), γ = 0.1 and E = 170 ms. (c) Memorymodel (11), γ = 0.1 and E = 250 ms. Panels (d)–(f) are similar, except that γ = 0.25.

second row correspond to a stronger feedback gain γ = 0.25.The panels in the first column are for the memoryless mapping(10), and the horizontal axis is stopped at B = 455 ms wherethe bifurcation to alternans occurs. The panels in the secondcolumn are for the memory model (11) with E = 170 ms, andthe ones in the last column are for the memory model (11)with E = 250 ms.

There are several important observations regarding Fig. 8.First, by comparing the small-γ panels in Figs. 8(a) to 8(c)with their moderate-γ counterparts in Figs. 8(d) to 8(f), thebasins of attraction are considerably larger in the latter case.In these simulations, the choice γ = 0.25 was nearly optimalfor the purpose of creating large basins of attraction. Increasingγ much beyond 0.3 causes the basins to retreat, and choosingvery large γ has a destabilizing effect. Next, the basins ofattraction are surprisingly large for the systems in these threeexamples. Figure 8(e) shows that ON-OFF TDAS alwaysstabilizes A∗ regardless of B, even if the initial APD iterateis a whopping 200 ms away from A∗. Apart from Fig. 8(d),ON-OFF TDAS either stabilizes A∗ for every initial separationδ up to 200 ms, or for none at all. The observation that varyingB can cause an abrupt transition from an empty basin ofattraction (apart from A∗ itself) to a huge basin of attractioncame as a surprise to us, and we were skeptical at first.However, further numerical experiments showing time seriesof An for various γ and δ confirmed our findings.

Of all of the panels in Fig. 8, only Fig. 8(d) was alignedwith our expectations. As the amplitude of alternans increasesand the magnitude of the Floquet multiplier D1 = D1f (A∗; B)increases further above 1, the repulsiveness of the fixed pointbecomes stronger. When the stabilization of A∗ is possible, thebasin of attraction ought to shrink as the repulsiveness of thefixed point increases. Importantly, general mapping modelsmight have multiple fixed points (unlike the restitution modelexamples studied here), which could lead to smaller basins ofattraction.

V. DISCUSSION AND CONCLUDING REMARKS

We have provided careful stability analyses of a feedbackcontrol method for stabilizing unstable fixed points of discrete-time systems. The method is a restricted version of a previouslyreported technique known as TDAS [3,11]; it is restricted inthe sense that the system cannot be perturbed during eachtime step. When applied to two examples of cardiac restitutionmapping models, our local stability analyses predict how thefeedback gain should be chosen so as to prevent unwantedalternation of action potential duration. For the restitutionmappings that we analyzed, it is encouraging that whenrestricted TDAS successfully stabilizes a fixed point, the basinof attraction of the fixed point is substantially larger than onemight rightfully expect.

There are potential advantages to applying feedback controlin an ON-OFF pattern. First, and perhaps most importantly, isthe possibility of stabilizing a fixed point with less energyusage than would be required if perturbations were appliedduring every iteration. In the cardiac example, this couldimply longer battery life for an implanted medical devicecapable of delivering electrical stimuli. Second, when onecompares the two panels of Fig. 1, it is potentially useful thatthe control domain for restricted ON-OFF TDAS is actuallyunbounded, unlike the control domain for TDAS. However,this observation could be misleading for two reasons: (i) thequantities D1 and D2 are usually related to one another (e.g.,D1 = −D2 for the memoryless cardiac restitution mapping)and (ii) in the restricted case, the upper and lower boundariesof the control domain merge as D1 decreases, imposing a tightrestriction on γ that could render such control impractical.

One might wonder about generalizations of the restrictedfeedback control method described in this paper. For instance,what would happen if perturbations could only be applied inevery third cycle, so that the controller fires in an ON-OFF-OFF pattern? Because the upper boundaries of the controldomains shown in Fig. 4 are γ = −1/D1 [Fig. 4(a)] and

042903-8

RESTRICTED FEEDBACK CONTROL IN DISCRETE-TIME . . . PHYSICAL REVIEW E 89, 042903 (2014)

γ = 1/D21 [Fig. 4(b)], it seems natural to conjecture that

the upper boundary of the corresponding domain for ON-OFF-OFF control would be γ = −1/D3

1 . Certainly, the lessfrequently that feedback control is turned ON, the smallerthe control domain should become (see also the figure in[10], which supports this). We did not perform the analysisof ON-OFF-OFF TDAS control because, for the purpose ofcontrolling a period-doubling bifurcation, it seems unnaturalto implement control every third beat to stop a period-2response.

Finally, we should mention that (unrestricted) TDAS hasbeen used to successfully control alternans experimentally [5].

To our knowledge, the restricted version of TDAS describedin this paper has not been used to control alternans or toprevent the onset of similar instabilities in other systems.We hope that our mathematical analysis will serve as aguide to those seeking to stabilize a variety of physical orbiological systems for which control need not be applied all thetime.

ACKNOWLEDGMENT

The support of a University of Richmond Summer ResearchFellowship is gratefully acknowledged.

[1] M. Landry, S. A. Campbell, K. Morris, and C. O. Aguilar,SIAM J. Appl. Dyn. Syst. 4, 333 (2005).

[2] E. Ott, C. Grebogi, and J. A. Yorke, Phys. Rev. Lett. 64, 1196(1990).

[3] K. Pyragas, Phys. Lett. A 170, 421 (1992).[4] B. Echebarria and A. Karma, Chaos 12, 923 (2002).[5] G. M. Hall and D. J. Gauthier, Phys. Rev. Lett. 88, 198102

(2002).[6] K. Pyragas, Phys. Lett. A 206, 323 (1995).[7] J. E. S. Socolar and D. J. Gauthier, Phys. Rev. E 57, 6589 (1998).[8] D. W. Sukow, M. E. Bleich, D. J. Gauthier, and J. E. S. Socolar,

Chaos 7, 560 (1997).[9] K. Hall and D. J. Christini, Phys. Rev. E 63, 046204

(2001).[10] D. J. Gauthier and J. E. S. Socolar, Phys. Rev. Lett. 79, 4938

(1997).[11] J. E. S. Socolar, D. W. Sukow, and D. J. Gauthier, Phys. Rev. E

50, 3245 (1994).[12] D. R. Adam, J. M. Smith, S. Akselrod, S. Nyberg, A. O. Powell,

and R. J. Cohen, J. Electrocardiol. 17, 209 (1984).[13] J. M. Pastore, S. D. Girouard, K. R. Laurita, F. G. Akar, and

S. Rosenbaum, Circulation 99, 1385 (1999).

[14] D. S. Rosenbaum, L. E. Jackson, J. M. Smith, H. Garan, J. N.Ruskin, and R. J. Cohen, N. Engl. J. Med. 330, 235 (1994).

[15] J. M. Smith, E. A. Clancy, R. Valeri, J. N. Ruskin, and R. J.Cohen, Circulation 77, 110 (1988).

[16] M. R. Guevara, G. Ward, A. Shrier, and L. Glass, in Proceed-ings of the 11th Computers in Cardiology Conference (IEEEComputer Society, Los Angeles, 1984), p. 167.

[17] C. M. Berger, X. Zhao, D. G. Schaeffer, H. M. Dobrovolny,W. Krassowska, and D. J. Gauthier, Phys. Rev. Lett. 99, 058101(2007).

[18] C. M. Berger, J. W. Cain, J. E. S. Socolar, and D. J. Gauthier,Phys. Rev. E 76, 041917 (2007).

[19] J. J. Fox, E. Bodenschatz, and R. F. Gilmour, Jr., Phys. Rev. Lett.89, 138101 (2002).

[20] K. Hall, D. J. Christini, M. Tremblay, J. J. Collins, L. Glass, andJ. Billette, Phys. Rev. Lett. 78, 4518 (1997).

[21] P. N. Jordan and D. J. Christini, J. Cardiovasc. Electrophysiol.15, 1177 (2004).

[22] E. G. Tolkacheva, M. M. Romeo, and D. J. Gauthier, Physica D194, 385 (2004).

[23] E. G. Tolkacheva, M. M. Romeo, M. Guerraty, and D. J.Gauthier, Phys. Rev. E 69, 031904 (2004).

042903-9