Kuramoto Model with Levy Noise

Dale Roberts Alex KalloniatisANU DSTO

An applied problem: the OODA loop

The phrase OODA Loop refers to the decision cycle of observe, orient,decide, and act, developed by military strategist and USAF Colonel JohnBoyd.

An applied problem: the OODA loop

The phrase OODA Loop refers to the decision cycle of observe, orient,decide, and act, developed by military strategist and USAF Colonel JohnBoyd.

Kuramoto model of coupled oscillators

The (classic) Kuramoto model is given by the coupled system of ODEs

dθωjdt

(t) = ωj −K

N

N∑i=1

Aij sin(θξj (t)− θξi (t)

)(KM)

for j = 1, 2, . . . ,N where:

I K is a real parameter,

I {ωi}i=1,...,N are natural rotation frequencies,

I the interaction topology is modelled by a graph G = (V ,E ) withA = [Aij ] the adjacency matrix and N = |V |.

Classically, G is a complete graph, i.e., Aij = 1 for i 6= j and 0 otherwise.

Hundreds of applications ranging from biological synchronization andrythmic phenomena, to engineering, etc.

Synchronisation

DefinitionA solution θ : R+ → Tn to the coupled oscillator model (KM) achievesphase synchronisation if all phases θi (t) become identical as t →∞.

Kuramoto (1975, 1984) showed that synchronisation occurs in (KM) forthe complete graph if the coupling gain K exceeds a certain thresholdKcritical function of the distribution of the natural frequencies ωi .

Order parameter

The order parameter r given by

r(t) =1

N

∣∣∣ N∑i=1

e iθi (t)∣∣∣ (OP)

is a standard metric for synchronisation.

5 10 15 20 25 30t

-3

-2

-1

1

2

3

θi(t)

0 5 10 15 20 25 30t

0.2

0.4

0.6

0.8

1.0r(t)

Network topologies

Complete Ring Erdös-Renyi Barabasi-Albert

Lots of interest in physics journals about how (KM) behaves on randomnetworks: Zhang & Xiao (2014), Medvedev (2014), Esfahani et al.(2012), Kalloniatis (2010), Gomez-Gardenes et al. (2007), Ping &Zhang, Kocarev & Amato (2005), Moreno & Pacheco (2004), Wang &Chen (2002), . . .

BA easier to synchronise than ER

!36,40". However, in these studies at least two parameters#clustering and average path length$ vary along the studiedfamily of networks. This paired evolution, although yieldingan interesting interplay between the two topological param-eters, makes it difficult to distinguish what effects were dueto one or other factors. Here, we would like to address firstwhat is the influence of heterogeneity, keeping the number ofdegrees of freedom to a minimum for the comparison to bemeaningful. The family of networks used in the present sec-tion are comparable in their clustering, average distance, andcorrelations so that the only difference relies on the degreedistribution, ranging from a Poissonian type to a scale-freedistribution. Later on in this paper, we will relax these con-straints and study networks in which the main topologicalfeature is given at the mesoscopic scale—i.e., networks withcommunity structure.

Therefore, let us first scrutinize and compare the synchro-nization patterns in Erdös-Rényi #ER$ and scale-free #SF$networks. For this purpose we make use of the model pro-posed in !45", which allows a smooth interpolation betweenthese two extremal topologies. Besides, we introduce a pa-rameter to characterize the synchronization paths to unraveltheir differences. The results reveal that the synchronizabilityof these networks does depend on the coupling between unitsand, hence, that general statements about their synchroniz-ability are eventually misleading. Moreover, we show thateven in the incoherent solution r=0, the system is self-organizing towards synchronization. We will analyze in de-tail how this self-organization is attained.

The first numerical study about the onset of synchroniza-tion of Kuramoto oscillators in SF networks !39" revealedthe great propensity of SF networks to synchronization,which is revealed by a nonzero but very small critical value!c !46". Besides, it was observed that at the synchronizedstate r=1, hubs are extremely robust to perturbations sincethe recovery time of a node as a function of its degree fol-lows a power law with exponent −1. However, how do SFnetworks compare with homogeneous networks and what arethe roots of the different behaviors observed?

We first concentrate on global synchronization for theKuramoto model, Eq. #4$. For this we follow the evolution ofthe order parameter r, Eq. #2$, as ! increases, to capture theglobal coherence of the synchronization in networks. We willperform this analysis on the family of networks generatedwith the model introduced in !45". This model generates aone-parameter family of networks labeled by "! !0,1". Theparameter " measures the degree of heterogeneity of the finalnetworks so that "=0 corresponds to the heterogeneousBarabási-Albert #BA$ network and "=1 to homogeneous ERgraphs. For intermediate values of " one obtains networksthat have been grown, combining both preferential attach-ment and homogeneous random linking so that each mecha-nism is chosen with probabilities #1−"$ and ", respectively.It is worth stressing that the growth mechanism preserves thetotal number of links, Nl, and nodes, N, for a proper com-parison between different values of ". Specifically, assumingthe final size of the network to be N, the network is built upstarting from a fully connected core of m0 nodes and a setS#0$ of N−m0 unconnected nodes. Then, at each time step, anew node #not selected before$ is chosen from S#0$ and

linked to m other nodes. Each of the m links is attached withprobability " to a randomly chosen node #avoiding self-connections$ from the whole set of N−1 remaining nodesand with probability #1−"$ following a linear preferentialattachment strategy !47". After repeating this process N−m0times, networks interpolating between the limiting cases ofER #"=1$ and SF #"=0$ topologies are generated !45". Fur-thermore, with this procedure, the degree of heterogeneity ofthe grown networks varies smoothly between the two limit-ing cases.

The curves r#!$ for several network topologies rangingfrom ER to SF are shown in Fig. 1. We have performedextensive numerical simulations of Eq. #4$ for each networksubstrate starting from !=0 and increasing it up to !=0.4with #!=0.02. A large number #at least 500$ of differentnetwork realizations and initial conditions were consideredfor every value of ! in order to obtain an accurate phasediagram. The natural frequencies $i and the initial values of%i were randomly drawn from a uniform distribution in theinterval #−1/2 ,1 /2$ and #−& ,&$, respectively.

Figure 1 reveals the differences in the critical behavior asa function of the substrate heterogeneity. The global coher-ence of the synchronized state, represented by r, shows thatthe onset of synchronization first occurs for SF networks. Asthe network substrate becomes more homogeneous the criti-cal point !c shifts to larger values and the system seems to beless synchronizable. On the other hand, it is also clear thatthe route to complete synchronization, r=1, is faster for ho-mogeneous networks. That is, when !'!c#"$ the growthrate of r increases with ". To inspect in depth the criticalparameters of the system dynamics we perform a finite-sizescaling #FSS$ analysis. This allows us to determine with pre-cision the curve !c#"$ and study the critical behavior nearthe synchronization transition. We assume a scaling relationof the form

r = N−(f„N)#! − !c$… , #5$

where f#x$ is as usual a universal scaling function boundedas x→ ±* and ( and ) are critical exponents to be deter-mined. The detailed analysis performed for both SF and ER

1

0.8

0.6

0.4

0.2

00 0.2 0.4 0.6 0.8 1

r

λ

α=0.0α=0.2α=0.4α=0.6α=0.8α=1.0

1

0.8

0.6

0.4

0.2

00 0.05 0.1 0.15 0.2

FIG. 1. Global synchronization curves r#!$ for different networktopologies labeled by " #"=0 corresponds to the BA limit and "=1 to ER graphs$. The inset shows the region where the onset ofsynchronization takes place. The network sizes are N=104 and %k&=6 #Nl=3+104$ and were generated using the model introduced in!45".

SYNCHRONIZABILITY DETERMINED BY COUPLING… PHYSICAL REVIEW E 75, 066106 #2007$

066106-3

Figure : Global synchronisation as graph varies from BA (α = 0) to ER (α = 1)for N = 10, 000. λ ≈ K/N. Figure from Gomez-Gardenes et al. (2007).

Kcritical(BA) < Kcritical(ER) < Kcritical(SW )

Kuramoto model with Gaussian noise

Gaussian noise can be added to the Kuramoto model, giving the coupledsystem of SDEs

dθξj (t) = ξjdt− K

N

N∑i=1

Aij sin(θξj (t)− θξi (t)

)dt + σdWj(t) (KMG)

for j = 1, . . . ,N where

I {Wj(·)}j=1,...,N is a family of independent standard Brownianmotions that models the thermal noise,

I ξ = {ξj}j=1,...,N is a family of IID random variables that models thedisorder.

The stochastic evolution is considered once a realisation of the disordervariables ξ is chosen: the disorder is of quenched type.

Path in Gaussian case

5 10 15 20 25 30t

-3

-2

-1

1

2

3

θi(t)

0 5 10 15 20 25 30t

0.2

0.4

0.6

0.8

1.0r(t)

Gaussian noise: Robustness and stochastic synchronisation

NOISE-INDUCED SYNCHRONIZATION IN SMALL WORLD . . . PHYSICAL REVIEW E 86, 036204 (2012)

node, we rewire each edge randomly with probability p.Choosing 0.005 ! p ! 0.05, this process converts the initialregular network to a complex network with a small mean pathlength and a large clustering coefficient, characteristics of SWnetworks.

Starting from a randomly distributed initial phase θi(0)(which is selected from a box distribution in the interval[−π,π ]), the set of coupled differential equations, (2), isintegrated from t = 0 to a given time t with the time stepdt , using the Euler method. This method enables us tocomputeθi(t), and to determine the synchrony among theoscillators at any time we define the complex order parameter

reiψ = 1N

N!

j=1

eiθj (t), (3)

where 0 " r(t) " 1 indicates the degree of synchronization inthe network and ψ is the phase of the order parameter.

Figure 1 shows the temporal variations of r(t) in thethree types of networks with N = 1000 and ⟨k⟩ = 10. Toobtain these plots, the time step is set to dt = 0.01 andfive realizations of the initial phase distribution are taken fora fixed network of each type. The rewiring probability forconstructing a WS network from a regular one is chosen to bep = 0.04. As shown, the oscillators on ER and Barabasi-Albertnetworks immediately reach a fully synchronized state (r = 1)irrespective of the initial conditions. However, in the case ofa WS network, they go more slowly toward the steady states,which are highly dependent on the initial phase distributionsin such a way that r(∞) reaches several values between 0and 1. These results show that, in contrast to ER and SFnetworks, the structure of steady states of the Kuramoto modelof SW networks can be more complex. In what follows, wediscuss that the sensibility of dynamics to initial conditions isindeed inherited by SW networks from their regular-networkparents. In a regular network, the ratio of nearest neighborconnections to network size (k/N) determines the numberof stable solutions. It has been shown that for k/N < 0.34,different initial conditions lead to different final states [24].

It is easy to show that the stable stationary solutions ofEqs. (2) have to satisfy the conditions

N!

i=1

sin θi =N!

i=1

cos θi = 0, (4)

provided that the phase difference between any two adjacentoscillators is less than π/2 (i.e, $θij = θi − θj < π/2 if aij =1). These solutions can be put into two categories: (i) fullysynchronized states with r = 1 ($θij = 0 for any i,j ) and(ii) phase-locked states with a regular arrangement of phasesaround the phase circle with nonzero phase difference $θ , forwhich r = 0.

The phase-locked states represent helical-wave phase mod-ulations and their number depends on N and k. For instance, inthe case of N = 1000 and k = 10, there are 10 such states withnearest neighbor phase differences $θα

nn = 2π/λα , in whichλα = 20, 25, 40, 50, 100, 125, 200, 250, 500, 1000 are thewavelengths of the helical states indicated by α = 1, 2, . . . ,10,respectively.

The stationary phase configuration of all nodes, correspond-ing to the initial conditions in Fig. 1, are plotted in Fig. 2 forboth the regular network and its offspring WS network. Thisplot corresponds to helical patterns with phase differencesλ = 1000 and 50, denoted in Fig. 1 by indexes b and e,respectively. Figure 2 shows that rewiring a regular networkwith a phase-locked state deforms its helical pattern to aninhomogeneous state in the subsequent WS one. Therefore, aWS network possesses various stable stationary states whosenumber equals the number of helical patterns in the parentregular network.

The local structure of the steady state can be better clarifiedby the correlation matrix D, defined as [25]:

Dij = lim$t→∞

1$t

" tr+$t

tr

cos(θi(t) − θj (t))dt, (5)

where tr is the time needed to reach a stationary state. Thematrix element −1 " Dij " 1 is a measure of coherency be-tween each pair of nodes. In the case of full synchrony betweeni and j (θi = θj ) the correlation matrix element is Dij = 1, and

FIG. 4. (Color online) Stationary order parameter versus reduced noise intensity for the four network types. Left: Regular, ER, SF, andfully synchronized states of WS. Right: Four phase-locked states of WS corresponding to states represented in Fig. 3. The number of nodesand mean degree for the three networks are N = 1000 and ⟨k⟩ = 10.

036204-3

Figure : Comparison of r∞ for increasing σ in (KMG). Left figure fromKhobasht et al. (2008) and right figure from Esfahani et al. (2012).

Stochastic synchronisation can be observed: noise can sometimes helpsynchronisation.

Questions

What happens when the perturbing noise is heavier-tailed?

Kcritical(BA) < Kcritical(ER)?

Levy process

Let L = (Lt)t≥0 be a Levy process with canonical triplet (γ, σ2,Π). Thusthe characteristic function of L is given by Ee iθLt = etΨ(θ), where

Ψ(θ) = iθγ − 12σ

2θ2 +

∫R

(e iθx − 1− iθx1{|x|<1})Π(dx), for θ ∈ R.

We look at two explicit cases:

I Stable process (0, 0,Π) with: (Subexponential Class)

Π(dx) =1

x1+αdx, x > 0.

I Tempered stable (γ, 0,Π) with: (Convolution Equivalent Class)

Π(dx) =e−λx

x1+αdx, x > 0,

and γ = (1− α)−1 −∫ 1

0(1− e−λx) dx

xα .

Kuramoto model with Levy noise

Levy noise can be added to the Kuramoto model, giving the coupledsystem of SDEs

dθξj (t) = ξjdt− K

N

N∑i=1

Aij sin(θξj (t)− θξi (t)

)dt + σdLj(t) (1)

for j = 1, . . . ,N where

I {Lj(·)}j=1,...,N is a family of independent Levy processes.

I We can vary parameters σ, α, and λ

Simulation

Look at graphs of size N = 1000, 1000 paths up to time T = 60.0 with afine time step (5000 steps), average over:

I Initial conditions θ0 ∼ Uniform(T)

I Disorder (natural frequency) ξ = 0

I Graph G (sample of size 30 from each class: ER and BA)

Simulation of:

I Stable 0 < α < 1 and 1 < α < 2 (exact)

I Tempered stable 0 < α < 1 (exact)

I Tempered stable 1 < α < 2 (approximate depending on param c)See survey by Kawai and Masuda

Parallelised over 1000 cpus, C++ and MPI.

Numerical results

Figure : r∞ for BA (pink) vs. ER (cyan) as α and σ varies for λ = 0.001.

Numerical results

1 2 3 4 5 6

0.2

0.4

0.6

0.8

1.0

α=0.5

1 2 3 4 5 6

0.2

0.4

0.6

0.8

1.0

α=1.1

1 2 3 4 5 6

0.2

0.4

0.6

0.8

1.0

α=1.5

1 2 3 4 5 6

0.2

0.4

0.6

0.8

1.0

α=2

Figure : Synchronisation for BA (blue) vs ER (red) as σ increases.

Analytic results

Analytic results are very hard (impossible?) to obtain for random G andadding driving noise as well.

I Linearise around a stable point to get tail probabilities

I Estimates for average exit time from a stable point in special casesof graphs in the small noise limit (Freidlin-Wentzell type result)

I If the system escapes a stable point (in finite time), how does it doit? Asymptotically as “well depth” u →∞.

How does it exit?

Linearised around a stable point, we can try to understand how the noisepushes the system out of synchronisation

Since L0 = 0, and conditional on τ(u) < T we have Lt > u for somet < T , in order to get a limit it is natural to scale L by a factor of u.Thus, setting

L(u)t =

Lt

u, 0 ≤ t ≤ T , (2)

we will investigate the limiting behavior of

L(L(u)|τ(u) < T ) as u →∞, (3)

where L denotes the law of the process.

How does it exit? Roughly linearly...

Suppose Π+(dx) is absolutely continuous for sufficiently large x and

Π+(dx)/dx ∼ βxα−1e−λx as x →∞ for some β > 0, α > 0, r ∈ R.

Theorem (Griffin and Roberts (2014))Let R be the degenerate process defined by

R(t) = tT−1, 0 ≤ t ≤ T ,

and let || · ||∞ denote the sup norm on D[0,T ]. If α > 0, then for anyδ > 0,

P(||L(u) − R||∞ > δ | τ(u) < T )→ 0. (4)

With heavier tails, you see a sudden large jump away from stable point...

AMSI Internship

AMSI Internship with DSTOFY15/16

PhD student who has just submitted?

Australian citizen

Thank you

Subexponential and Convolution Equivalent

A distribution F on R with tail F = 1− F belongs to the class L(α),α ≥ 0, if

limu→∞

F (u + x)

F (u)= e−αx , for x ∈ (−∞,∞).

Further, F belongs to the class S(α), α ≥ 0, if in addition

limu→∞

F 2∗(u)

F (u)exists and is finite, (5)

where F 2∗ = F ∗ F . When F ∈ S(α),

δFα :=

∫R

eαxF (x) <∞, (6)

and the limit in (5) is given by 2δFα . Distributions in S(0) are calledsubexponential, and those in S(α) with α > 0, are called convolutionequivalent of index α.

Subexponential and Convolution Equivalent Levy Processes

For any Levy measure Π, let Π+(·) = Π(· ∩ (0,∞)) andΠ+(u) = Π+((u,∞)). Then we say

Π ∈ S(α) iff F ∈ S(α) where F (u) =Π+(u)

Π+(1)∧ 1, α ≥ 0. (7)

Equivalently, Π+(1) may be replaced with Π+(a) for any a > 0 by closureof S(α) under tail equivalence. Thus Π ∈ S(α) depends only on thepositive tail of Π, and so to emphasize this we will write Π+ ∈ S(α)

instead of Π ∈ S(α). Condition (7) is also equivalent to Xt ∈ S(α) for all(some) t > 0.