are powerlaws useful?
Post on 25-Dec-2014
404 Views
Preview:
DESCRIPTION
TRANSCRIPT
Are Power Laws Useful?
Michael P.H. Stumpf
Theoretical Systems Biology Group
07/03/2013
Are Power Laws Useful? Michael P.H. Stumpf 1 of 28
Outline
Power Laws: Ubiquituous, Universal, Useful?
Critical Phenomena and Scaling Behaviour
Empirical Power Laws
Simple Models of Network Evolution
More Normal Than Normal
Summary
Are Power Laws Useful? Michael P.H. Stumpf 2 of 28
What Do We Mean By A Powerlaw?
Power Law Relationships
Y ∝ Xλ
log(X )
log(Y )
log(X )
log(p(X ))
Are Power Laws Useful? Michael P.H. Stumpf Power Laws: Ubiquituous, Universal, Useful? 3 of 28
What Do We Mean By A Powerlaw?
Power Law Relationships
Y ∝ Xλ
log(X )
log(Y )
log(X )
log(p(X ))
Are Power Laws Useful? Michael P.H. Stumpf Power Laws: Ubiquituous, Universal, Useful? 3 of 28
What Do We Mean By A Powerlaw?
Power Law Relationships
Y ∝ Xλ
log(X )
log(Y )
log(X )
log(p(X ))
Are Power Laws Useful? Michael P.H. Stumpf Power Laws: Ubiquituous, Universal, Useful? 3 of 28
Power Laws as Physical Scaling Relationships
Laminar Flow
R
x
r
Universality in FlowFor sufficiently low Reynolds number the velocity profile is universalfor all pipes and fluids:
v(r)v(0)
=
(1 −
r2
R2
)
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 4 of 28
Power Laws as Physical Scaling Relationships
Laminar Flow
R
x
r
Universality in FlowFor sufficiently low Reynolds number the velocity profile is universalfor all pipes and fluids:
v(r)v(0)
=
(1 −
r2
R2
)Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 4 of 28
Power Laws as Physical Scaling Relationships
1D Josephson-Junction Array in insulating and super conducting phaseSondhi et al., Rev.Mod.Phys. 69:315 (1997).
As the critical temperature is approached from above, the correlationlength increases as some power of the reduced temperature,
ξ ∝(
T − Tc
Tc
)−ν
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 4 of 28
Making Sense of Power Laws in Physics
The Ising Model
↓↓↑↓↑↑
↑↑↓↓↓↓
↑↑↓↓↑↑
↓↓↓↑↓↓
↑↓↑↓↓↓
↑↓↓↑↑↓
T > TC
↑↑↑↑
↑
↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑
↑↑
↓↓
↓
T ≈ TC
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
T < TC
Critical Exponents
For θC = T−TCTC
we have for any macroscopic variable F (θC) in thevicinity of the critical point θC ≈ 0,
F (θC) = const.× θ−λC .
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28
Making Sense of Power Laws in Physics
The Ising Model
↑↓↓↑↓↓
↓↑↓↑↑↓
↓↓↓↑↓↑
↓↑↓↑↓↑
↓↑↓↓↑↓
↓↓↓↑↑↑
T > TC
↑↑↑↑
↑
↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑
↑↑
↓↓
↓
T ≈ TC
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
T < TC
Critical Exponents
For θC = T−TCTC
we have for any macroscopic variable F (θC) in thevicinity of the critical point θC ≈ 0,
F (θC) = const.× θ−λC .
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28
Making Sense of Power Laws in Physics
Renormalization Group Theory
↑↑↑↑
↑
↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑
↑↑
↓↓
↓
↑
↑
↑ ↑
↑
↑ ↑
↑
↓↑ ↑
Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:
Z =∑[Si ]
exp (−βH[Si ]) =∑[S ′α]
exp(−βH ′[S ′
α])= Z ′
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28
Making Sense of Power Laws in Physics
Renormalization Group Theory
↑↑↑↑
↑
↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑
↑↑
↓↓
↓
↑
↑
↑ ↑
↑
↑ ↑
↑
↓↑ ↑
Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:
Z =∑[Si ]
exp (−βH[Si ]) =∑[S ′α]
exp(−βH ′[S ′
α])= Z ′
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28
Making Sense of Power Laws in Physics
Renormalization Group Theory
↑↑↑↑
↑
↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑
↑↑
↓↓
↓
↑
↑
↑ ↑
↑
↑ ↑
↑
↓↑ ↑
Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:
Z =∑[Si ]
exp (−βH[Si ]) =∑[S ′α]
exp(−βH ′[S ′
α])= Z ′
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28
Making Sense of Power Laws in Physics
Renormalization Group Theory
↑↑↑↑
↑
↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑
↑↑
↓↓
↓
↑
↑
↑ ↑
↑
↑ ↑
↑
↓↑ ↑
Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:
Z =∑[Si ]
exp (−βH[Si ]) =∑[S ′α]
exp(−βH ′[S ′
α])= Z ′
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28
Making Sense of Power Laws in Physics
Renormalization Group Theory
↑↑↑↑
↑
↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑
↑↑
↓↓
↓
↑
↑
↑ ↑
↑
↑ ↑
↑
↓↑ ↑
Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:
Z =∑[Si ]
exp (−βH[Si ]) =∑[S ′α]
exp(−βH ′[S ′
α])= Z ′
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28
Making Sense of Power Laws in Physics
Renormalization Group Theory
↑↑↑↑
↑
↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑
↑↑
↓↓
↓
↑
↑
↑ ↑
↑
↑ ↑
↑
↓
↑ ↑
Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:
Z =∑[Si ]
exp (−βH[Si ]) =∑[S ′α]
exp(−βH ′[S ′
α])= Z ′
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28
Making Sense of Power Laws in Physics
Renormalization Group Theory
↑↑↑↑
↑
↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑
↑↑
↓↓
↓
↑
↑
↑ ↑
↑
↑ ↑
↑
↓
↑ ↑
Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:
Z =∑[Si ]
exp (−βH[Si ]) =∑[S ′α]
exp(−βH ′[S ′
α])= Z ′
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28
Making Sense of Power Laws in Physics
Renormalization Group Theory
↑↑↑↑
↑
↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑↑↑↑
↑↑↑
↑↑
↓↓
↓
↑
↑
↑ ↑
↑
↑ ↑
↑
↓↑ ↑
Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:
Z =∑[Si ]
exp (−βH[Si ]) =∑[S ′α]
exp(−βH ′[S ′
α])= Z ′
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28
Behaviour Around Critical Points
The fixed points of the RGT define the possible macroscopic states ofthe system, e.g. ferromagnet vs. paramagnet.Of particular interest are the non-trivial fixed points, 0 < Tc, J <∞,which mark the occurrence of a phase transition.
In this case changing the scale does not change the physics. For ourspin-system, for example, there is long-range correlation whichextends beyond the lattice spacing.
Critical PointsAt the critical point the correlation length diverges as a power law
ξ ∝ θ−νc
Note that knowledge of the critical point does not tell us necessarilywhat the state of the system on either side is.
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 6 of 28
Behaviour Around Critical Points
The fixed points of the RGT define the possible macroscopic states ofthe system, e.g. ferromagnet vs. paramagnet.Of particular interest are the non-trivial fixed points, 0 < Tc, J <∞,which mark the occurrence of a phase transition.In this case changing the scale does not change the physics. For ourspin-system, for example, there is long-range correlation whichextends beyond the lattice spacing.
Critical PointsAt the critical point the correlation length diverges as a power law
ξ ∝ θ−νc
Note that knowledge of the critical point does not tell us necessarilywhat the state of the system on either side is.
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 6 of 28
Behaviour Around Critical Points
The fixed points of the RGT define the possible macroscopic states ofthe system, e.g. ferromagnet vs. paramagnet.Of particular interest are the non-trivial fixed points, 0 < Tc, J <∞,which mark the occurrence of a phase transition.In this case changing the scale does not change the physics. For ourspin-system, for example, there is long-range correlation whichextends beyond the lattice spacing.
Critical PointsAt the critical point the correlation length diverges as a power law
ξ ∝ θ−νc
Note that knowledge of the critical point does not tell us necessarilywhat the state of the system on either side is.
Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 6 of 28
Heureka!
Mason Porter, http://www.quickmeme.com/meme/3sqh80/
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 7 of 28
Simple Scaling Laws
West et al., PNAS, 99:2473 (2002).May & Stumpf, Science, 290:2084 (2000).
Plausible Theories and Simple ModelsEven when simple, plausible physical arguments can be put forward
In an area that is twice as large, we can accommodate fourtimes as many species, N ∝ A2
the empirical results are better described by other phenomenologicaldistributions.
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 8 of 28
Simple Scaling Laws
West et al., PNAS, 99:2473 (2002).May & Stumpf, Science, 290:2084 (2000).
Plausible Theories and Simple ModelsEven when simple, plausible physical arguments can be put forward
In an area that is twice as large, we can accommodate fourtimes as many species, N ∝ A2
the empirical results are better described by other phenomenologicaldistributions.
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 8 of 28
Illusions of Invariance
We consider off-spring weaning weight, w , and maternal weight, m,and seek to understand w/m. If this ratio is invariant then log(w)plotted against log(m) should have a regression slope of 1.0.
The Inverse is not true! A slope of 1.0 does not imply invariance.
For example, consider the model where log(w) = log(m) + log(c)where log(c) is a non-normally distributed error, ε, c ∼ U[0, 1].We then have
R2 =Var[log(w)] − Var[log(c)]
Var[log(w)]=
Var[log(m)]
Var[log(m)] + Var[log(c)]
Nee et al., Science, 309:1236 (2005).
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 9 of 28
Illusions of Invariance
We consider off-spring weaning weight, w , and maternal weight, m,and seek to understand w/m. If this ratio is invariant then log(w)plotted against log(m) should have a regression slope of 1.0.
The Inverse is not true! A slope of 1.0 does not imply invariance.
For example, consider the model where log(w) = log(m) + log(c)where log(c) is a non-normally distributed error, ε, c ∼ U[0, 1].We then have
R2 =Var[log(w)] − Var[log(c)]
Var[log(w)]=
Var[log(m)]
Var[log(m)] + Var[log(c)]
Nee et al., Science, 309:1236 (2005).
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 9 of 28
Illusions of Invariance
We consider off-spring weaning weight, w , and maternal weight, m,and seek to understand w/m. If this ratio is invariant then log(w)plotted against log(m) should have a regression slope of 1.0.
The Inverse is not true! A slope of 1.0 does not imply invariance.
For example, consider the model where log(w) = log(m) + log(c)where log(c) is a non-normally distributed error, ε, c ∼ U[0, 1].
We then have
R2 =Var[log(w)] − Var[log(c)]
Var[log(w)]=
Var[log(m)]
Var[log(m)] + Var[log(c)]
Nee et al., Science, 309:1236 (2005).
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 9 of 28
Illusions of Invariance
We consider off-spring weaning weight, w , and maternal weight, m,and seek to understand w/m. If this ratio is invariant then log(w)plotted against log(m) should have a regression slope of 1.0.
The Inverse is not true! A slope of 1.0 does not imply invariance.
For example, consider the model where log(w) = log(m) + log(c)where log(c) is a non-normally distributed error, ε, c ∼ U[0, 1].We then have
R2 =Var[log(w)] − Var[log(c)]
Var[log(w)]=
Var[log(m)]
Var[log(m)] + Var[log(c)]
Nee et al., Science, 309:1236 (2005).
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 9 of 28
Illusions of Invariance
We consider off-spring weaning weight, w , and maternal weight, m,and seek to understand w/m. If this ratio is invariant then log(w)plotted against log(m) should have a regression slope of 1.0.
The Inverse is not true! A slope of 1.0 does not imply invariance.
For example, consider the model where log(w) = log(m) + log(c)where log(c) is a non-normally distributed error, ε, c ∼ U[0, 1].We then have
R2 =Var[log(w)] − Var[log(c)]
Var[log(w)]=
Var[log(m)]
Var[log(m)] + Var[log(c)]
Nee et al., Science, 309:1236 (2005).
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 9 of 28
Scale-Free Networks
The Beginnings
A:
Actor collaboration; B: WWW; C: Power grid
Barabasi & Albert, Science, 286:510 (1999).
Ever since many real-worldnetworks have been“discovered” to be
scale-free. Partial alignment of Human and Fly protein-interaction network (PIN).
What Are “Scale-Free” Networks?Typically this means that the degree distributions is scale-free, i.e.
Pr(αk)Pr(k)
= const. ∀k .
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 10 of 28
Scale-Free Networks
The Beginnings
A:
Actor collaboration; B: WWW; C: Power grid
Barabasi & Albert, Science, 286:510 (1999).
Ever since many real-worldnetworks have been“discovered” to be
scale-free. Partial alignment of Human and Fly protein-interaction network (PIN).
What Are “Scale-Free” Networks?Typically this means that the degree distributions is scale-free, i.e.
Pr(αk)Pr(k)
= const. ∀k .
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 10 of 28
Are Networks Scale-Free?
Saccharomyces cerevisiae PIN
If D = {d1, . . . , dn} is the empirical degreedistribution and Prm(k |θ) the probability toobserve degree k for model m ∈M, with{M1, . . . , Mq}, and parameter θ then thelikelihood is given by
Lm(θm) =
n∏i=1
Prm(k |θ).
This allows us to compare different models inlight of the data (using e.g. the AIC or BIC toenforce parsimony).
So far, no network has been found to bescale-free when proper statistical analysis wasapplied.
Stumpf & Ingram, Europhys. Lett. 71:152 (2005); Tanaka et al., FEBS Lett. 579:5140 (2005); Khanin & Wit, J.Comp.Biol.13:810 (2006).
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 11 of 28
Are Networks Scale-Free?
Saccharomyces cerevisiae PIN
If D = {d1, . . . , dn} is the empirical degreedistribution and Prm(k |θ) the probability toobserve degree k for model m ∈M, with{M1, . . . , Mq}, and parameter θ then thelikelihood is given by
Lm(θm) =
n∏i=1
Prm(k |θ).
This allows us to compare different models inlight of the data (using e.g. the AIC or BIC toenforce parsimony).
So far, no network has been found to bescale-free when proper statistical analysis wasapplied.
Stumpf & Ingram, Europhys. Lett. 71:152 (2005); Tanaka et al., FEBS Lett. 579:5140 (2005); Khanin & Wit, J.Comp.Biol.13:810 (2006).
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 11 of 28
Are Networks Scale-Free?
Saccharomyces cerevisiae PIN
If D = {d1, . . . , dn} is the empirical degreedistribution and Prm(k |θ) the probability toobserve degree k for model m ∈M, with{M1, . . . , Mq}, and parameter θ then thelikelihood is given by
Lm(θm) =
n∏i=1
Prm(k |θ).
This allows us to compare different models inlight of the data (using e.g. the AIC or BIC toenforce parsimony).
So far, no network has been found to bescale-free when proper statistical analysis wasapplied.
Stumpf & Ingram, Europhys. Lett. 71:152 (2005); Tanaka et al., FEBS Lett. 579:5140 (2005); Khanin & Wit, J.Comp.Biol.13:810 (2006).
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 11 of 28
Real Networks Are Scale Rich
Tanaka, Phys.Rev.Lett. 94:168101 (2005).
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 12 of 28
Other Statistical Challenges
Incomplete and Noisy Data
Sub-nets of scalefree networks arenot scale-free.Stumpf et al., PNAS 102:4221,(2005);Wiuf & Stumpf, Proc.Roy.Soc. A462:1181 (2006).
Truncated Power Laws
Pr(k) ∝ k−λ for klow < k < khigh
It is hard to see what is gained by this: the statistical power is lowerthan that of other mixture models, and the elegance of theinterpretation is no longer given.
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 13 of 28
Other Statistical Challenges
Incomplete and Noisy Data
Sub-nets of scalefree networks arenot scale-free.Stumpf et al., PNAS 102:4221,(2005);Wiuf & Stumpf, Proc.Roy.Soc. A462:1181 (2006).
Truncated Power Laws
Pr(k) ∝ k−λ for klow < k < khigh
It is hard to see what is gained by this: the statistical power is lowerthan that of other mixture models, and the elegance of theinterpretation is no longer given.
Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 13 of 28
Evolving Networks
α
δ
δ
γ
γ
α
δ
δAre Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 14 of 28
Evolving Networks
Model-Based Evolutionary Analysis• For sequence data we use models of nucleotide substitution in
order to infer phylogenies in a likelihood or Bayesian framework.• None of these models — even the general time-reversible model
— are particularly realistic; but by allowing for complicating factorse.g. rate variation we capture much of the variability observedacross a phylogenetic panel.
• Modes of network evolution will be even more complicated andexhibit high levels of contingency; moreover the structure andfunction of different parts of the network will be intricately linked.
• Nevertheless we believe that modelling the processes underlyingthe evolution of networks can provide useful insights; in particularwe can study how functionality is distributed across groups ofgenes.
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 14 of 28
Network Evolution Models
(a) Duplication attachment (b) Duplication attachmentwith complimentarity
(c) Linear preferentialattachment
wi
wj
(d) General scale-free
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 15 of 28
ABC on Networks
Summarizing Networks• Data are noisy and incomplete.• We can simulate models of network
evolution, but this does not allow us tocalculate likelihoods for all but verytrivial models.
• There is also no sufficient statistic thatwould allow us to summarize networks,so ABC approaches require somethought.
• Many possible summary statistics ofnetworks are expensive to calculate.
Full likelihood: Wiuf et al., PNAS (2006).
ABC: Ratman et al., PLoS Comp.Biol. (2008).
Stumpf & Wiuf, J. Roy. Soc. Interface (2010).
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 16 of 28
Graph Spectrum
a
b
c
d e0 1 1 1 01 0 1 1 01 1 0 0 01 1 0 0 10 0 0 1 0
a b c d e
abcde
A =
Graph SpectraGiven a graph G comprised of a set of nodes N and edges (i, j) ∈ Ewith i, j ∈ N, the adjacency matrix, A, of the graph is defined by
ai,j =
{1 if (i, j) ∈ E ,
0 otherwise.
The eigenvalues, λ, of this matrix provide one way of defining thegraph spectrum.
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 17 of 28
Spectral Distances
A simple distance measure between graphs having adjacencymatrices A and B, known as the edit distance, is to count the numberof edges that are not shared by both graphs,
D(A, B) =∑
i,j
(ai,j − bi,j)2.
However for unlabelled graphs we require some mapping h fromi ∈ NA to i ′ ∈ NB that minimizes the distance
D(A, B) > D ′h(A, B) =
∑i,j
(ai,j − bh(i),h(j))2,
Given a spectrum (which is relatively cheap to compute) we have
D ′(A, B) =∑
l
(λ(α)l − λ
(β)l
)2
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 18 of 28
Spectral Distances
A simple distance measure between graphs having adjacencymatrices A and B, known as the edit distance, is to count the numberof edges that are not shared by both graphs,
D(A, B) =∑
i,j
(ai,j − bi,j)2.
However for unlabelled graphs we require some mapping h fromi ∈ NA to i ′ ∈ NB that minimizes the distance
D(A, B) > D ′h(A, B) =
∑i,j
(ai,j − bh(i),h(j))2,
Given a spectrum (which is relatively cheap to compute) we have
D ′(A, B) =∑
l
(λ(α)l − λ
(β)l
)2
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 18 of 28
Spectral Distances
A simple distance measure between graphs having adjacencymatrices A and B, known as the edit distance, is to count the numberof edges that are not shared by both graphs,
D(A, B) =∑
i,j
(ai,j − bi,j)2.
However for unlabelled graphs we require some mapping h fromi ∈ NA to i ′ ∈ NB that minimizes the distance
D(A, B) > D ′h(A, B) =
∑i,j
(ai,j − bh(i),h(j))2,
Given a spectrum (which is relatively cheap to compute) we have
D ′(A, B) =∑
l
(λ(α)l − λ
(β)l
)2
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 18 of 28
ABC using Graph Spectra
For an observed network, N, and a simulated network, Sθ, we use thedistance between the spectra
D ′(N, Sθ) =∑
l
(λ(N)l − λ
(S)l
)2,
in our ABC SMC procedure. Note that this distance is a close lowerbound on the distance between the raw data; we therefore do nothave to bother with summary statistics.Also, calculating graph spectra costs as much as calculating otherO(N3) statistics (such as all shortest paths, the network diameter orthe within-reach distribution).Thorne & Stumpf, J.Roy.Soc. Interface, 9:2653 (2012).
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 19 of 28
Protein Interaction Network Data
Species Proteins Interactions Genome size Sampling fraction
S.cerevisiae 5035 22118 6532 0.77
D. melanogaster 7506 22871 14076 0.53
H. pylori 715 1423 1589 0.45
E. coli 1888 7008 5416 0.35
Model
Mod
el p
roba
bilit
y
0.0
0.1
0.2
0.3
0.4
0.5
DA DAC LPA SF DACL DACR
Organism
S.cerevisae
D.melanogaster
H.pylori
E.coli
Model Selection• Inference here was based on all
the data, not summarystatistics.
• Duplication models receive thestrongest support from the data.
• Several models receive supportand no model is chosenunambiguously.
Thorne & Stumpf, J.Roy.Soc. Interface, 9:2653 (2012).
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 20 of 28
Protein Interaction Network Data
Species Proteins Interactions Genome size Sampling fraction
S.cerevisiae 5035 22118 6532 0.77
D. melanogaster 7506 22871 14076 0.53
H. pylori 715 1423 1589 0.45
E. coli 1888 7008 5416 0.35
Model
Mod
el p
roba
bilit
y
0.0
0.1
0.2
0.3
0.4
0.5
DA DAC LPA SF DACL DACR
Organism
S.cerevisae
D.melanogaster
H.pylori
E.coli
Model Selection• Inference here was based on all
the data, not summarystatistics.
• Duplication models receive thestrongest support from the data.
• Several models receive supportand no model is chosenunambiguously.
Thorne & Stumpf, J.Roy.Soc. Interface, 9:2653 (2012).
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 20 of 28
Protein Interaction Network Data
Species Proteins Interactions Genome size Sampling fraction
S.cerevisiae 5035 22118 6532 0.77
D. melanogaster 7506 22871 14076 0.53
H. pylori 715 1423 1589 0.45
E. coli 1888 7008 5416 0.35
Model
Mod
el p
roba
bilit
y
0.0
0.1
0.2
0.3
0.4
0.5
DA DAC LPA SF DACL DACR
Organism
S.cerevisae
D.melanogaster
H.pylori
E.coli
Model Selection• Inference here was based on all
the data, not summarystatistics.
• Duplication models receive thestrongest support from the data.
• Several models receive supportand no model is chosenunambiguously.
Thorne & Stumpf, J.Roy.Soc. Interface, 9:2653 (2012).
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 20 of 28
PIN Model Evolution
Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 21 of 28
Power Laws As Phenomenological Models
We have seen above that power laws emerge naturally in the contextof continuous phase transitions; but we have also seen that they offerat best limited insights for finite systems.Why do they nevertheless appear so often?
Revisiting the Renormalization GroupLet f (X ) be a probability distribution. Then the RGT acting on it isdefined as
Taf (X = x) := |a|∫∞−∞ f (ax − s)f (s)ds.
Taf (X ) is the pdf of the random variable
Y =X1 + X2
awith X1, X2 ∼ f (X ).
Here a controls the qualitative properties of the transformation.Calvo et al., J.Stat.Phys. 141:409 (2010).
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 22 of 28
Power Laws As Phenomenological Models
We have seen above that power laws emerge naturally in the contextof continuous phase transitions; but we have also seen that they offerat best limited insights for finite systems.Why do they nevertheless appear so often?
Revisiting the Renormalization GroupLet f (X ) be a probability distribution. Then the RGT acting on it isdefined as
Taf (X = x) := |a|∫∞−∞ f (ax − s)f (s)ds.
Taf (X ) is the pdf of the random variable
Y =X1 + X2
awith X1, X2 ∼ f (X ).
Here a controls the qualitative properties of the transformation.Calvo et al., J.Stat.Phys. 141:409 (2010).
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 22 of 28
Renormalization Group Transformation of PDFs
Here we are after the fixed points, f0, of the RGT,
Taf0(x) = f0(x)
When all moments of f (X ) exist and are finite then we can determinethe moments of the transformed distribution,
Eg [Y n] = ETaf [Y n] =1an
n∑i=0
(ni
)Ef[X i]Ef
[X n−i]
Some Fixed Points of the RGT
a <√
2 a =√
2 a >√
2
limm→∞ET m
a f0 [x2] =∞ Ef0 [x
2n] = Ef0 [x2])n (2n)!
n!2n limm→∞ET m
a f0 [x2] = 0
— N(0, 1) δ(x)
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 23 of 28
Renormalization Group Transformation of PDFs
Here we are after the fixed points, f0, of the RGT,
Taf0(x) = f0(x)
When all moments of f (X ) exist and are finite then we can determinethe moments of the transformed distribution,
Eg [Y n] = ETaf [Y n] =1an
n∑i=0
(ni
)Ef[X i]Ef
[X n−i]
Some Fixed Points of the RGT
a <√
2 a =√
2 a >√
2
limm→∞ET m
a f0 [x2] =∞ Ef0 [x
2n] = Ef0 [x2])n (2n)!
n!2n limm→∞ET m
a f0 [x2] = 0
— N(0, 1) δ(x)
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 23 of 28
Renormalization Group Transformation of PDFs
Here we are after the fixed points, f0, of the RGT,
Taf0(x) = f0(x)
When all moments of f (X ) exist and are finite then we can determinethe moments of the transformed distribution,
Eg [Y n] = ETaf [Y n] =1an
n∑i=0
(ni
)Ef[X i]Ef
[X n−i]
Some Fixed Points of the RGT
a <√
2 a =√
2 a >√
2
limm→∞ET m
a f0 [x2] =∞ Ef0 [x
2n] = Ef0 [x2])n (2n)!
n!2n limm→∞ET m
a f0 [x2] = 0
— N(0, 1) δ(x)
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 23 of 28
Renormalization Group Transformation of PDFs
Here we are after the fixed points, f0, of the RGT,
Taf0(x) = f0(x)
When all moments of f (X ) exist and are finite then we can determinethe moments of the transformed distribution,
Eg [Y n] = ETaf [Y n] =1an
n∑i=0
(ni
)Ef[X i]Ef
[X n−i]
Some Fixed Points of the RGT
a <√
2 a =√
2 a >√
2
limm→∞ET m
a f0 [x2] =∞ Ef0 [x
2n] = Ef0 [x2])n (2n)!
n!2n limm→∞ET m
a f0 [x2] = 0
— N(0, 1) δ(x)
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 23 of 28
Central Limit Theorems
The conventional central limit theorem emerges as the fixed point ofthe RGT for distributions where all moments are finite.
Now we look at the characteristics function, φ(t) = Ef [eitX ] and obtainfor the fixed points, φ0(t),
ϕ0(t/a)2 = ϕ0(t)
General Fixed PointsThe general fixed points of the RGT are given by the Levy-stable laws,
ϕ0(t) = Sα,A)(k) := exp(−A|t |αθ(t) − A|t |αθ(−k)
)with |a| = 21/α, A the complex conjugate of A and θ(x) the Heavisidestep function.For α = 2 we recover the Gaussian, and for all α < 2 we obtain theheavy-tailed stable laws. For α < 1 the mean is infinite.
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 24 of 28
Central Limit Theorems
The conventional central limit theorem emerges as the fixed point ofthe RGT for distributions where all moments are finite.Now we look at the characteristics function, φ(t) = Ef [eitX ] and obtainfor the fixed points, φ0(t),
ϕ0(t/a)2 = ϕ0(t)
General Fixed PointsThe general fixed points of the RGT are given by the Levy-stable laws,
ϕ0(t) = Sα,A)(k) := exp(−A|t |αθ(t) − A|t |αθ(−k)
)with |a| = 21/α, A the complex conjugate of A and θ(x) the Heavisidestep function.For α = 2 we recover the Gaussian, and for all α < 2 we obtain theheavy-tailed stable laws. For α < 1 the mean is infinite.
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 24 of 28
Central Limit Theorems
The conventional central limit theorem emerges as the fixed point ofthe RGT for distributions where all moments are finite.Now we look at the characteristics function, φ(t) = Ef [eitX ] and obtainfor the fixed points, φ0(t),
ϕ0(t/a)2 = ϕ0(t)
General Fixed PointsThe general fixed points of the RGT are given by the Levy-stable laws,
ϕ0(t) = Sα,A)(k) := exp(−A|t |αθ(t) − A|t |αθ(−k)
)with |a| = 21/α, A the complex conjugate of A and θ(x) the Heavisidestep function.For α = 2 we recover the Gaussian, and for all α < 2 we obtain theheavy-tailed stable laws. For α < 1 the mean is infinite.
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 24 of 28
Power Law vs. Gaussian Distribution
From the above we see that the conventional CLT is in fact a veryspecial case of a much more general form of the CLT.Thus we would expect fat-tailed distributions (by whichever sensibledefinition) to occur frequently.
Gaussian Distributions are stable under aggregation andmarginalization.
Scaling Distributions are stable under aggregation, mixture,maximisation and marginalization.
More Normal Than NormalIn this sense scaling or fat-tailed distributions should be expected tooccur very frequently. For low-variability data we obtain the Gaussianas a fixed point.Willinger et al., Proceedings of the 2004 Winter Simulation Conference, 130 (2004).
It is thus also easy to come up with simple mechanisms that give riseto dispersed data.
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 25 of 28
Power Law vs. Gaussian Distribution
From the above we see that the conventional CLT is in fact a veryspecial case of a much more general form of the CLT.Thus we would expect fat-tailed distributions (by whichever sensibledefinition) to occur frequently.Gaussian Distributions are stable under aggregation and
marginalization.
Scaling Distributions are stable under aggregation, mixture,maximisation and marginalization.
More Normal Than NormalIn this sense scaling or fat-tailed distributions should be expected tooccur very frequently. For low-variability data we obtain the Gaussianas a fixed point.Willinger et al., Proceedings of the 2004 Winter Simulation Conference, 130 (2004).
It is thus also easy to come up with simple mechanisms that give riseto dispersed data.
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 25 of 28
Power Law vs. Gaussian Distribution
From the above we see that the conventional CLT is in fact a veryspecial case of a much more general form of the CLT.Thus we would expect fat-tailed distributions (by whichever sensibledefinition) to occur frequently.Gaussian Distributions are stable under aggregation and
marginalization.Scaling Distributions are stable under aggregation, mixture,
maximisation and marginalization.
More Normal Than NormalIn this sense scaling or fat-tailed distributions should be expected tooccur very frequently. For low-variability data we obtain the Gaussianas a fixed point.Willinger et al., Proceedings of the 2004 Winter Simulation Conference, 130 (2004).
It is thus also easy to come up with simple mechanisms that give riseto dispersed data.
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 25 of 28
Power Law vs. Gaussian Distribution
From the above we see that the conventional CLT is in fact a veryspecial case of a much more general form of the CLT.Thus we would expect fat-tailed distributions (by whichever sensibledefinition) to occur frequently.Gaussian Distributions are stable under aggregation and
marginalization.Scaling Distributions are stable under aggregation, mixture,
maximisation and marginalization.
More Normal Than NormalIn this sense scaling or fat-tailed distributions should be expected tooccur very frequently. For low-variability data we obtain the Gaussianas a fixed point.Willinger et al., Proceedings of the 2004 Winter Simulation Conference, 130 (2004).
It is thus also easy to come up with simple mechanisms that give riseto dispersed data.
Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 25 of 28
Power Laws, Evidence, Usefulness
MechanisticSophistication
Statistical Support
AllometricScaling
Zipf’s Law
Internet
C. elegansnervoussystem
S. cerevisiae PINStumpf & Porter, Science, 335:665 (2012).
Are Power Laws Useful? Michael P.H. Stumpf Summary 26 of 28
How to Check if You Have a Power Law in Your Data?
Y ∝ X−λ YES or NO ?
1. Would a power law relationship offer profound new insights?
◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?
2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?
3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?
4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?
A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.
Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28
How to Check if You Have a Power Law in Your Data?
Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?
◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?
2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?
3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?
4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?
A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.
Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28
How to Check if You Have a Power Law in Your Data?
Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.
◦ Could you just have a very dispersed random variable, Y?2. Does it extend over at least three orders of magnitude in both
variables (sanity check)?3. Does the power law relationship hold up in comparison to other
distributions (log-normal, stretched exponential, negativebinomial)?
4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?
A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.
Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28
How to Check if You Have a Power Law in Your Data?
Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?
2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?
3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?
4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?
A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.
Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28
How to Check if You Have a Power Law in Your Data?
Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?
2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?
3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?
4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?
A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.
Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28
How to Check if You Have a Power Law in Your Data?
Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?
2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?
3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?
4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?
A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.
Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28
How to Check if You Have a Power Law in Your Data?
Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?
2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?
3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?
4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?
A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.
Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28
How to Check if You Have a Power Law in Your Data?
Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?
2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?
3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?
4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?
A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.
Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28
Acknowledgements
• Imperial CollegeLondon◦ Thomas Thorne◦ William Kelly
• OxfordUniversity◦ Robert May◦ Mason Porter
• KopenhagenUniversity◦ Carsten Wiuf
Theoretical Systems Biology Group
www.theosysbio.bio.ic.ac.uk
Are Power Laws Useful? Michael P.H. Stumpf Summary 28 of 28
top related