Dynamical network motifs:

building blocks of complex dynamics in biological networks

Valentin ZhigulinDepartment of Physics, Caltech, and

Institute for Nonlinear Science, UCSD

Spatio-temporal dynamics in biological networks Periodic oscillations in cell-cycle regulatory network

Periodic rhythms in the brain

Chaotic neural activity in models of cortical networks

Chaotic dynamics of populations’ sizes in food webs

Chaos in chemical reactions

Challenges for understanding of these dynamics Strong influence of networks’ structure on their dynamic

It induces long term, connectivity-dependent spatio-temporal correlations which present formidable problem for theoretical treatment

These correlations are hard to deal with because connectivity is in general not symmetric, hence dynamics is non-Hamiltonian

Dynamical mean field theory may allow one to solve such a problem in the limit of infinite-size networks [Sompolinsky et al, Phys.Rev.Lett. 61 (1988) 259-262]

However, DMF theory is not applicable to the study of realistic networks with non-uniform connectivity and a relatively small size

Questions

How can we understand the influence of networks’ structure on their dynamics?

Can we predict dynamics in networks from the topology of their connectivity?

Simple dynamical model

k

j

xF

ix

xxFxdt

dx

ik

ij

i

i

ijjij

ikkikii

i

nodeby excitation ofstrength

nodeby inhibition ofstrength

nstimulatio external ofstrength

tynonlineari sigmoid )(

node of activity''

)(

Hopfield (‘attractor’) networks

Symmetric connectivity → fixed point attractors Memories (patterns) are stored in synaptic weights Current paradigm for the models of ‘working memory’

jiijjiij ;

There is no spatio-temporal dynamics in the model

Networks with random connectivity For most biological networks exact connectivity is not

known

As a null hypothesis, let us first consider dynamics in networks with random (non-symmetric) connectivity

Spatio-temporal dynamics is now possible

Depending on connectivity, periodic, chaotic and fixed point attractors can be observed in such networks

Further simplifications of the model

graph theofmatrix adjacency

inhibition ofstrength 5

field excitatorymean 1

tynonlineari sigmoid )(

node of activity''

)(

ij

i

ijjiji

i

A

xF

ix

xAFxdt

dx

Dynamics in a simple circuit

Single, input-dependent attractor Robust, reproducible dynamics Fast convergence regardless of

initial conditions

00.2

0.40.6

0.8

00.2

0.40.6

0.8

00.20.40.6

0.8

00.2

0.40.6

0.8

10 20 30 40 50

0.2

0.4

0.6

0.8

time

20 40 60 80 100

0.2

0.4

0.6

0.8

1

10 20 30 40 50

0.2

0.4

0.6

0.8

1

20 40 60 80 100

0.2

0.4

0.6

0.8

1

LLE ≈ 0

LLE > 0

LLE < 0

Studying dynamics in large random networks Consider a network of N nodes with some probability p of node-to-

node connections

For each p generate an ensemble of ~104·p networks with random connections (~ pN2 links)

Simulate dynamics in each networks for 100 random initial conditions to account for possibility of multiple attractors in the network

In each simulation calculate LLE and thus classify each network as having chaotic (at least one LLE>0), periodic (at least one LLE≈0 and no LLE>0) or fixed point (all LLEs<0) dynamics

Calculate F (fraction of an ensemble for each type of dynamics) as a function of p

0.00 0.02 0.04 0.06 0.08 0.10

0.0

0.2

0.4

0.6

0.8

1.0

chaotic

F

p

periodic

Dynamical transition in the ensemble

Similar transitions had been observed in models of genetic networks [Glass and Hill, Europhys. Lett. 41 599] and ‘balanced’ neural networks [van Vreeswijk and Sompolinsky, Science 274 1724]

0.00 0.02 0.04 0.06 0.08 0.10

0.0

0.2

0.4

0.6

0.8

1.0

dynamicchaotic

F

p

periodic

Hypothesis about the nature of the transition As more and more links are added to the network,

structures with non-trivial dynamics start to form

At first, subnetworks with periodic dynamics and then subnetworks with chaotic dynamics appear

The transition may be interpreted as a proliferation of dynamical motifs – smallest dynamical subnetworks

Testing the hypothesis Strategy:

Identify dynamical motifs - minimal subnetworks with non-trivial dynamics Estimate their abundance in large random networks

Roadblocks: Number of all possible directed networks growth with their size n as ~2n

2

Rest of the network can influence motifs’ dynamics

Simplifications: We can estimate the number of active elements in the rest of the network and make

sure that they do not suppress motif’s dynamics Number of non-isomorphic directed networks grows much slower:

Since the probability to find a motif with l links in a random network is proportional to pl, we are only interested in motifs with small number of links

N 3 4 5 6 7 8

NAll 512. 65536. 3.36 107 6.87 1010 5.63 1014 1.84 1019

NNI 16. 218. 9608. 1.54 106 8.82 108 1.79 1012

Motifs with periodic dynamics (LLE≈0)

3 nodes, 3 links -

4 nodes, 5 links -

Motifs with chaotic dynamics (LLE>0) 5 nodes

9 links

8 nodes11 links

6 nodes10 links

7 nodes10 links

Appearance of dynamical motifs in random networks

Appearance of dynamical motifs II

Prediction of the transition in random networks

0.00 0.02 0.04 0.06 0.08 0.10

0.0

0.2

0.4

0.6

0.8

1.0

periodic motifs

dynamic networks

chaotic networks

F

p

periodic networks

Appearance of chaotic motifs

0.00 0.02 0.04 0.06 0.08 0.10

0.0

0.2

0.4

0.6

0.8

1.0

dynamic networks

chaotic networks

periodic networks

chaotic motifs

periodic motifs

F

p

1. Under-sampling2. Over-counting

How to avoid chaos ?

Dynamics in many real networks are not chaotic

Networks with connectivity that minimizes the number of chaotic motifs would avoid chaos

For example, brains are not wired randomly, but have spatial structure and distance-dependent connectivity

Spatial structure of the network may help to avoid chaotic dynamics

2D model of a spatially distributed network

2.5

5

7.5

10

2.5

5

7.5

10

0

0.2

0.4

0.6

0.8

2.5

5

7.5

10

)/exp(~)( 22 ijij ddp

Dynamics in the 2D model

Only 1% of networks with λ=2 exhibit chaotic dynamics

99% of networks with λ=10 exhibit chaotic dynamics

Calculations show that chaotic motifs are absent in networks with local connectivity (λ=2) and present in non-local networks (λ=10)

Hence local clustering of connections plays an important role in defining dynamical properties of a network

Number of motifs in spatial networks

Computations in a model of a cortical microcircuit

)/exp(~)( 22 ijij ddp

Maass, Natschläger, Markram, Neural Comp., 2002nscomputatio useful2

ncomputatio no chaos, 8

dynamics no 1

Take-home message

Calculations of abundance of dynamical motifs in networks with

different structures allows one to study and control dynamics in

these networks by choosing connectivity that maximizes the

probability of motifs with desirable dynamics and minimizes

probability of motifs with unacceptable dynamics.

This approach can be viewed as one of the ways to solve an

inverse problem of inferring network connectivity from its

dynamics.

Acknowledgements

Misha Rabinovich (INLS UCSD) Gilles Laurent (CNS & Biology, Caltech) Michael Cross (Physics, Caltech)

Ramon Huerta (INLS UCSD) Mitya Chklovskii (Cold Spring Harbor Laboratory) Brendan McKay (Australian National University)