bartnett - information flow in dynamic system supplement

8/13/2019 Bartnett - Information Flow in Dynamic System Supplement

1/6

Information flow in a kinetic Ising model peaks in the disordered phase: Supplemental

material

Lionel Barnett,1,Joseph T. Lizier,2, Michael Harre,3,Anil K. Seth,1,and Terry Bossomaier4,1 Sackler Centre for Consciousness Science, School of Informatics, University of Sussex, Brighton BN1 9QJ, UK

2 CSIRO Computational Informatics, PO Box 76, Epping, NSW 1710, Australia3 School of Civil Engineering, The University of Sydney, Sydney NSW 2006, Australia

4 Centre for Research in Complex Systems, Charles Sturt University, Panorama Ave, Bathurst NSW 2795, Australia(Dated: September 12, 2013)

1: CALCULATION OF PAIRWISE MUTUAL INFORMATION

It is clear from homogeneity that Si has the same distribution for any site i and, similarly, that Si, Sj have thesame jointdistribution for any pair of neighbouring sites i, j. Thus we haveIpw = I(Si : Sj) = 2H(Si) H(Si, Sj)for any fixed choice of lattice neighbours i, j (we note, though, that in sample the lattice-averaged form will yield amore efficient estimator). Firstly, H(Si) =

plogp where p P(Si = ) and the sum is over spins =1.

Firstly, we show thatp is as given in[1], eq. 6. In the calculations that follow, we make frequent use of the identity

(, ) = 12

(1 + ) (1)

for spins , = 1. We have

P(Si = ) =s

P(S= s) P( Si = | S= s) conditioning on S

=s

(s)(si, )

=s

(s) 12(1 +si) by (1)

= 12

(1 + Si) 12 (1 +M) asN

as required. Next we show that p is also as in [1], eq. 6. We have H(Si, Sj) =

,plogp where

p P(Si = , Sj = ), andP(Si = , Sj =

) =s

P(S= s) P( Si = , Sj = | S= s)

=s

(s)(si, )(sj , )

= 14

s

(s)(1 +si+sj+ sisj)

= 14

(1 + Si + Sj + SiSj) 14 [1 + (+)M 12 U] as N

as required. In the last step, we useSiSj 12U as N , which follows fromU= 1NH(S). Ipw is thus as in[1], eq. 5. Note that forT < Tc the sign of

Mdoes not affect the result; i.e. Ipw is invariant to the direction in which

symmetry breaks (this applies to the other information measures too).

2: CALCULATION OF PAIRWISE TRANSFER ENTROPY

We start by proving the following lemma: for arbitrary lattice neighboursi, j,

SiPi(S) 0 (2)SiSjPi(S) 0 (3)

SjPi(S) 0 for T Tc only . (4)


2/6

2

Let +i {s|si = +1} and i {s|si = 1}. Then

SiPi(S) =s

(s)siPi(s)

= 1

Z

s+

i

eH(s)Pi(s) 1Z

s

i

eH(s)Pi(s)

Now we note that as s runs through +

i , si runs through

i

= 1

Z

s+

i

eH(s)Pi(s) 1Z

s+

i

eH(si)Pi(s

i)

= 1

Z

s+

i

eH(s)

1

1 +eHi(s) e[H(s)+Hi(s)] 1

1 +eHi(s)

using Hi(s

i) = Hi(s)

= 1

Z

s+

i

eH(s)

1

1 +eHi(s) e

Hi(s)

1 +eHi(s)

= 0

proving (2). A similar argument works for (3). IfT

Tc then since symmetry is unbroken, for each equilibrium state

there is a corresponding equilibrium state with all spins reversed. For spin-reversed states, Hi(s), and hence Pi(s),is unchanged, so that sjPi(s) has the opposite sign; (4) thus follows.

By homogeneity, the joint distribution of Si(t), Si(t 1), Sj(t 1) is the same for any fixed pair of neighbour-ing sites i, j and we haveTpw = H( Si(t) | Si(t 1)) H( Si(t) |Si(t 1), Sj(t 1)). Firstly, H( Si(t) | Si(t 1)) =pp|logp|, where we define p| P( Si(t) = | Si(t 1) =). In the calculations that follow, wemake use of the following explicit expression for the Markov transition probabilities in the Glauber kinetic model:

P(s|s) =

1 1N

kPk(s) s= s

1NPj(s) s

= sj

0 otherwise

. (5)

We have

P(Si(t) =, Si(t 1) =)

=s,s

P( Si(t) =, Si(t 1) = | S(t) = s, S(t 1) = s) P(S(t) = s,S(t 1) = s)

=s

(s)(si, )s

(si, )P(s|s)

=s

(s)(si, )

(si, )1 1

N

j

Pj(s)

+

j

(sji , )

1

NPj(s)

by (5)

=s

(s)(si, )(si, ) 1N

j

(si, ) (s

ji , )

Pj(s)

Note that the term in square brackets vanishes unless j = i

=s

(s)(si, )

(si,

) 1N

(si,

) (sii, )

Pi(s)

=s

(s)(si, )

(si,

) 1N

siPi(s)

sincesii = si

=s

(s)(si, )(si, ) 1

N s

(s)(si, )Pi(s)


3/6

3

=(, )s

(s)(si, ) 1N

s

(s)12

(1 +si)Pi(s)

=(, )p 1N

12

(Pi(S) + SiPi(S))

=(, )p 1N

q since by (2)SiPi(S) vanishes ,

withqas in[1], eq. 11. So

p| =

1 1N

q

p =

1

N

q

p =

. (6)

Next, H( Si(t) | Si(t 1), Sj(t 1)) =

,p

p|logp| , where we define p| P( Si(t) =

| Si(t 1) =, Sj(t 1) = ), and we may calculate along the same lines as above (we omit details)that

p| =

1 1N

q

p=

1N q

p=

(7)

withq again as in[1], eq. 11. Now, working to O1N

,

Tpw =

p

p|logp|+,

p

p|logp|

=

pp|logp|+ p|logp|

+,

pp|logp|+ p|logp|

= p 1

1

N

q

p log1

1

N

q

p+ 1

N

q

plog1

N

q

p

+,

p

1 1

N

q

p

log

1 1

N

q

p

+

1

N

q

plog

1

N

q

p

= 1N

q

log

q

p log N 1

+ 1

N

,

q

log

q

p log N 1

+ O

1

N2

= 1N

qlog q

p+

1

N

,

qlog q

p+ O

1

N2

as N , where in the penultimate step we use log(1 + x/N) = x/N+ O1/N2 as N and in the last stepwe use the identityq

q, which follows directly from[1], eq. 11, so that the (log N+ 1) terms cancel. Thus in

the thermodynamic limit, we obtain[1], eq. 10.

3: CALCULATION OF GLOBAL TRANSFER ENTROPY

Once again by homogeneity we haveTgl = H( Si(t) | Si(t 1)) H( Si(t) | S(t 1)) for any fixed site i. The firstterm has been calculated above and for the second term H( Si(t) | S(t 1)) =

s

(s)

pi(|s)logpi(|s)


4/6

4

wherepi(|s) P( Si(t) = | S(t 1) = s). We have

P( Si(t) = | S(t 1) = s) =s

P( Si(t) = | S(t 1) = s, S(t) = s) P(S(t) = s | S(t 1) = s)

=s

(si, )P(s|s) again, s= s or s= sj for somej

=(si, )

1 1

Nj

Pj(s)

+j

(sj

i , )1

NPj(s)

=(si, ) 1

N

j

(si,

) (sji , )

Pj(s)

=(si, ) 1

N

(si,

) (sii, )

Pi(s)

=(si, ) 1

NsiPi(s) ,

so

pi(

|s) =

1 1N

Pi(s) = si

1N

Pi(s) = si. (8)

By an argument analogous to that for the pairwise case,

Tgl = 1N

q

log

q

p log N 1

+ 1

N

s

(s)Pi(s)[log Pi(s) log N 1] + O

1

N2

= 1N

qlog q

p+

1

NPi(S)log Pi(S) + O

1

N2

as N , where in the last step we uses

(s)Pi(s) =Pi(s)= 2q, so that again the (log N+ 1) terms cancel.Thus in the thermodynamic limit we obtain[1], eq. 13.

4: GRADIENT OF MUTUAL INFORMATION MEASURES AT CRITICALITY

In the thermodynamic limitM 0 for c, so that

plogp is constant with respect to and p =14(1 12 U). Thus from[1], eqs. 5, 8 we may calculate that up to a constant

Ipw = 12

(1 + 12U) log(1 + 1

2U) + 1

2(1 1

2U) log(1 1

2U) (9)

1

NIgl = (U F) (10)

For convenience we change to the variable x 2, and denote partial differentiation with respect to x by a prime.FromU=

(F) we find

Ipw = 14

log

1 + 1

2U

1 12U

U (11)1

NIgl = 12xU (12)

We want to evaluate these quantities as x xc from below, where xc 2c = log

2 + 1

. We thus set x = xc and let 0 from above. Setting 2sinh x

cosh2 xwe have ([1], TABLE I)

U= coth x

1 +2

( sinh x 1) K()

(13)


5/6

5

where

K() /20

d1 2 sin2

(14)

is the complete elliptic integral of the first kind [2]. Working to O(), we may calculate

sinh x= 1

2+ O2 (15)

cosh x= 2 + O2 (16)tanh x= 1

2 1

2+ O

2

(17)

coth x=

2 ++ O

2

(18)

and to O

2

= 1 2 + O3 (19)First we evaluateU as x xc from below. From (13) we have

U=

(

2 +)1 2

2

K1

2+ O2 (20)NowK

1 2 logarithmically as 0 [3], so thatK1 2 0 andU 2 as x xc from below. Thusfrom (11) and (12) we see that both Ipw and

1NI

gl 12xcU as xxc from below. From (13)a straightforward

calculation yields

U= 1sinh x cosh x

U 8

1

cosh2 xK() +

4

( sinh x 1)2sinh x

K() (21)

Now

K() = 1

(1 2) E() 1

K() (22)

[2] where

E() /20

1 2 sin2 d (23)

is the complete elliptic integral of the second kind [2]. Some algebra yields

U= 1sinh x cosh x

U+ 4

( sinh x 1)2(1 2)sinh x E()

2

coth2 x K() (24)

UsingE(1) = 1 [2], we find

U

1 +

4

4

K() (25)

as 0. ButK() logarithmically as 1, soU which implies Ipw

, 1

N

Igl

+ as c from

below and finally, since

= T2

T, we have

IpwT

, 1

N

IglT

logarithmically as T Tc from above.

Electronic address: [email protected] Electronic address: [email protected] Electronic address: [email protected]
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]


6/6

6

Electronic address: [email protected] Electronic address: [email protected]

[1] L. Barnett, T. Bossomaier, J. T. Lizier, M. Harre, and A. K. Seth, Phys. Rev. Lett.[vol], [pp] (2013).[2] L. M. Milne-Thomson, in Handbook of Mathematical Functions With Formulas, Graphs, and Mathematical Tables (9th ed.),

edited by M. Abramowitz and I. A. Stegun (Dover Publications, New York, 1972), chap. 17.[3] D. Karp and S. M. Sitnik, J. Comput. Appl. Math. 205 , 186 (2007).
mailto:[email protected]:[email protected]:[email protected]:[email protected]

bartnett - information flow in dynamic system supplement

Documents