Balancing Fact and Formula in the Science of Complex Systems: The

Example of 1/f Spectra

Nick Watkins

NickWatkins@mykolab.com

• Slides 1-24 were given as an invited talk on 19th December 2014 at the Fall AGU in San Francisco, beginning the Session on Intermittency and Dynamical Complexity in Space Plasmas from the Sun to Interplanetary and Planetary Environments.

• I have also included the spares prepared in case of questions and being developed for future talks.

Thanks

• Tom, Giuseppe and Marius for inviting me

• Holger Kantz for hosting me in Dresden since September 2013

• Co-authors Tim Graves, Christian Franzke, Bobby Gramacy, Scott Osprey and Paulo Davini

• Discussions with all of the above and Sandra Chapman, Eli Barkai, Igor Sokolov, Rainer Klages and Aleksei Chechkin among others

Theme

Hurst effect

Will today distinguish three things often taken as same

• Observed growth of range in time series: “Hurst effect”

Theme

1/f

Hurst effect

Will today distinguish three things often taken as same

• Observed growth of range in time series: “Hurst effect”

• Observation of a singularity at zero in Fourier spectra: “1/f”

Theme

(S)LRD

1/f

Hurst effect

Will today distinguish three things often taken as same • Observed growth of range

in time series: “Hurst effect”

• Observation of a singularity at zero in Fourier spectra: “1/f”

• The long range dependence seen in stationary 1/f case: (S)LRD.

• Using 1/f as a diagnostic of LRD assumes stationarity

Fact: Anomalous growth of range

Hurst Effect

Hurst, Nature, 1957

“I heard about the … Nile … in '64, ... the variance doesn't draw like time span as you take bigger and bigger integration intervals; it goes like time to a certain power different from one. … Hurst …was getting results that were incomprehensible”. – Mandelbrot, 1998

Formula: Long Range Dependence

(S)LRD

Hurst Effect

• Mandelbrot, van Ness, and Wallis, 1965-69

• First [history in Graves et al, arXiv, 2014a] demonstration that Hurst effect could be explained by stationary long range dependent process

• Model, fractional Gaussian noise [see also Kolmogorov’s“Wiener Spiral”], had singular spectral density at lowest frequencies.

'( ) ~ S f f

The 1/f “paradox”If spectral density '( )

then i) it is singular as

and ii) if we define an autocorrelation

function via ( ) ( ) ( )

and use Wiener-Khinchine theorem to

get from Fourier transform of

~

0

S f f

f

x t x t

S

falls off as power law, and

'( )

then

summed lags "blow up"

its

( )

f

Ionosphere

Magnetosphere

1 12 2

, 22fBm: ( ) ( ) ( )~ ( )

H

HR

HX t t s s dL s

Infinite range memory kernel

Gaussian

Fractional motions and noises

0 100 200 300 400 500 600 700 800 900 1000-18

-16

-14

-12

-10

-8

-6

-4

-2

0

2Fractional Brownian motion, H=0.7

1 12 2

, 22fBm: ( ) ( ) ( )~ ( )

H

HR

HX t t s s dL s

Build a nonstationary, self similar walk … (used wfbm in Matlab)

fractional motion

2 1 H

Fractional motions and noises

0 100 200 300 400 500 600 700 800 900 1000-18

-16

-14

-12

-10

-8

-6

-4

-2

0

2Fractional Brownian motion, H=0.7

1 12 2

, 22fBm: ( ) ( ) ( )~ ( )

H

HR

HX t t s s dL s

Build a nonstationary, self similar walk … (used wfbm in Matlab)

0 100 200 300 400 500 600 700 800 900 1000-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1Fractional Gaussian noise

fractional motion

Then differentiate to give a stationary LRD noise

fractional noise

2 1 H 2 1 H

Ionosphere

Magnetosphere

[…], if infinite dependence is necessary it does not mean that IBM's details of ten years ago influence IBM today, because there's no mechanism within IBM for this dependence. However, IBM is not alone. The River Nile is [not] alone. They're just one-dimensional corners of immensely big systems. The behaviour of IBM stock ten years ago does not influence its stock today through IBM, but IBM the enormous corporation has changed the environment very strongly. The way its price varied, went up or went up and fluctuated, had discontinuities, had effects upon all kinds of other quantities, and they in turn affect us. –Mandelbrot, interviewed in 1998 by B. Sapoval for Web of Stories

One resolution of 1/f paradox

In modern fractional Langevin models fGn is noise term e.g. reviews of Metzler et al, PCCP, 2014; Watkins GRL, 2013.

Ionosphere

Magnetosphere

[…], if infinite dependence is necessary it does not mean that IBM's details of ten years ago influence IBM today, because there's no mechanism within IBM for this dependence. However, IBM is not alone. The River Nile is [not] alone. They're just one-dimensional corners of immensely big systems. The behaviour of IBM stock ten years ago does not influence its stock today through IBM, but IBM the enormous corporation has changed the environment very strongly. The way its price varied, went up or went up and fluctuated, had discontinuities, had effects upon all kinds of other quantities, and they in turn affect us. –Mandelbrot, interviewed in 1998 by B. Sapoval for Web of Stories

One resolution of 1/f paradox

In modern fractional Langevin models fGn is noise term e.g. reviews of Metzler et al, PCCP, 2014; Watkins GRL, 2013.

• Resolution of apparent paradox is that world as a whole is Markovian, the memory is a consequence of looking at a piece of it. Generalises the Mori-Zwanzig approach.

Often see 1/f spectra and heavy tails

Ionosphere

Magnetosphere

Ground-based magnetometerssense ionosphericcurrents

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

x 105

0

500

1000

1500

2000

2500

3000

1978 AE data: threshold percentile=99

Time, t [minutes]

AE

[nano T

esla

]

AE data

threshold

Slow 1/f region in AE power spectrum

Tsurutani et al, GRL, 1991

See also Hush et al, PosterSH53A-4209 this afternoon

Moscone South

Joint inference on LRD and heavy tails.

Tim Graves’ PhD developed Bayesian method: tested on α-stable ARFIMA(0,d,0)

where heavy tails & LRD co-exist, see Graves et al, arXiv, submitted CSDA 2014b;

& my talk from AGU, 2012 on Slideshare.

1.5 0.15 d

Test on Synthetic ARFIMA data

Other models for “1/f”

Ionosphere

Magnetosphere

Selecta H

Selecta N

1/f without (S)LRD

(S)LRD

1/f

Hurst effect

• Before (S)LRD models, Mandelbrot [1963-67] had proposed other 1/f models which were not stationary LRD in same sense as fGn.

• Solved 1/f paradox by a different route. Still little known in the geosciences [but see Klemes, WRR, 1974].

Ionosphere

Magnetosphere

• Abrupt state changes• Fat distributions of switching times: “Levy” (E[t^2] = ∞) case.

The conditional spectrum:

Magnetosphere

Mandelbrot 1967 reviewed in N2, Selecta, 1999

The conditional spectrum:

Magnetosphere

Mandelbrot 1967 reviewed in N2, Selecta, 1999

• “Numerical … 1/f … spectrum … need not … estimate … Wiener-Khinchine spectrum”. Instead “depends on conditioning length T”. Unlike stationary LRD model, singularity is an artefact.

Hurst effect from state changes

Ionosphere

Magnetosphere

Franzke et al, in review, Sci. Rep., 2014

• Interestingly, the classic Lorenz 63 model (columns 1 and 2) can generate Hurst effect in some measures, such as DFA, even without long tailed waiting times between regime shifts. Confirms that Hurst effect is easier to generate than 1/f or full blown S(LRD).

Formula versus fact

“Like the ear, the eye is very

sensitive to features that the

spectrum does not reflect. Seen

side by side, different 1/f noises,

Gaussian [i.e. fGn], dustborne [i.e.

fractional renewal] and multifractal,

obviously differ from one another”-Mandelbrot, Selecta N, 1999.

“Nothing can be more fatal to

progress than a too confident

reliance on mathematical symbols;

for the student is only too apt to …

consider the formula and not the

fact as the physical reality”.

Thomson (Kelvin) & Tait, 1890

edition.

Summary• The 1/f paradox: The "1/f" spectral shape seen

throughout physics, & why it seems paradoxical. • Many faces of 1/f: Mandelbrot promoted two

resolutions of paradox in the mid 60s: stationary long range dependence (S)LRD , or what he dubbed “conditionally stationary” renewal models. History is in Graves et al, arXiv, 2014a. One “formula” consistent with different “facts”.

• Ways forward ? Mandelbrot’s Selecta (Vols. N &H) urged use of our eyes as well as formalism, and I’ve advertised two new results in this spirit: Bayesian method for choosing between (S)LRD models, Graves et al, arXiv, 2014b, and work on a dynamical origin for Hurst effect in the Lorenz model (Franzke et al, Scientific Reports, in review, 2014 ).

SPARE SLIDES

THE ROAD TO MULTIFRACTALS

So what were the other models ?

• Additive models extending fGn like fractional hyperbolic model of Mandelbrot & Wallis [1969].

• Multiplicative, multifractal models exhibiting volatility bunching as well as 1/f spectra and fat tails-1972 (turbulence), 1990s (finance).

• And the class he referred to as “dustborne”: the least known of his papers, from 1965-67, though closely related to the Alternating Fractional Renewal Process, the CTRW and modern work on weak ergodicity breaking.

Multiplicative multifractal cascades

Many systems have aggregation, but not by an additive route. Classic example is turbulence.

One indicator: lognormal or stretched exponential pdf …

Selecta Volume N1999

Multifractals and volatility clustering

another: correlations between the absolute value of the time series- or here, in ionospheric data, the first differences.

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

-600

-400

-200

0

200

400

600

incre

ments

, r

First differences of AE index January-June 1979

-100 -80 -60 -40 -20 0 20 40 60 80 100-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

lag

acf

AE data: acf of returns

-100 -80 -60 -40 -20 0 20 40 60 80 100-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

lag

acf

AE data: acf of squared returns

CONDITIONAL STATIONARITY, NON-ERGODICTY ETC

The infrared catastrophe as mirage:

Ionosphere

Magnetosphere

Reviewed in N2, Selecta, 1999

The infrared catastrophe as mirage:

Ionosphere

Magnetosphere

• Rather than representing a true singularity at lowest frequencies, as seen in stationary LRD, in this case he described the infrared catastrophe as a ``mirage“ arising from fact that here “measured

square Fourier moduli do not estimate a Wiener-Khinchinedensity”. Same formula changes meaning because of different facts

Reviewed in N2, Selecta, 1999

Ionosphere

Magnetosphere

(Fifth Berkeley Symposium onProbability, 1965)

Non-ergodicity:

Ionosphere

Magnetosphere

• M 1967 illustrated this first in case of single jump, infinite interval

Non-ergodicity:

Ionosphere

Magnetosphere

• Mandelbrot 1967 then discussed case of many state changes with power law waiting times

FRACTIONAL STABLE MODELS

Distinct from fBm and fGN:

Ionosphere

Magnetosphere

Mandelbrot 1967 was prepared in the same period as Mandelbrot and van Ness on fBm and fGn, which it cites as ``to be published". In it contrast is clearly drawn between the conditionally stationary, non-Gaussian renewal process as a 1/f model and his stationary, Gaussian (fGn) model:

LRD and its controversies

Ionosphere

Magnetosphere

• Mandelbrot [1965]; M and Van Ness[1968] proposed use of fractional Brownian motion. Non stationary self similar model which generalises Wiener process, has spectral index between -1 and -3.

• … and its derivative, fractional Gaussian noise, which is stationary, and long range dependent.

• Unlike the stable amplitude distribution we just saw, the power spectra of fBm and fGn are singular at zero frequency. In Bm (and fBm)this is a result of its nonstationarity … [Selecta, N2, 1999]

Ionosphere

MagnetosphereMandelbrot & Wallis [1969] first attempt to unify long range memory kernel of fGn with heavy tailed amplitude fluctuations - called it “fractional hyperbolic” model because of its power law tails.

Anticipated the versatile linear fractional stable noises, but it didn’t satisfy him completely for problems he was looking at.

Additive fractional stable class

HISTORY

A neglected paper … ?

Ionosphere

Magnetosphere

Mandelbrot 1967 received far less attention than either papers on heavy tails in finance in early 1960s or the series with Van Ness and Wallis in 1968-69 on stationary fractional Gaussian models for LRD. Only about 20 citations in its first 20 years !

Was apparently unknown to Vit Klemes [Water Resources Research, 1974], who essentially reinvented it to criticise fBm. Still seems a relatively little known paper. Not cited by Beran et al [2013], and while listed in the citations of Beran [1994] I can’t find in the text. Some exceptions, e.g. Grigolini et al, Physica A, 2009.

Although he revisited the paper with new commentary in Selecta Volume N [1999] dealing with multifractals and 1/f noise, Mandelbrot neglected to mention it explicitly in his popular and historical accounts of genesis of LRD such as Mandelbrot and Hudson [2008].

Why ? Because it wasn’t as popular as fBm/fGn ? Or because it wasn’t as “beautiful” ?

… Whose time has come ?

Ionosphere

Magnetosphere

• Should we pay more attention to this class of models ? As the AFRP [Lowen and Teich’s book], and as models for weak ergodicity breaking [c.f. Niemann et al, 2013], we already are ! Also links to the CTRW.

• Particular value of looking back nearly 50 years to how Mandelbrot saw these models is to see how they fit into “the panorama of grid-bound self-affine variability” as he later put it [Selecta, 1999, N1].

Helps link maths and physics, the formula & the fact.And inform future work.

Ionosphere

Magnetosphere

And so my argument has always [sic] been that each of these causal chains is totally incomprehensible in detail, probably exponentially decaying. There are so many of them that a very strong dependence may be perfectly compatible. Now I would like to mention that this is precisely the reason why infinite dependence exists, for example, in physics. In a magnet- because two parts far away have very minor dependence along any path of actual dependence. There are so many different paths that they all combine to create a global structure. In other words, there is no global structure in one dimension, but there's one in two and three dimensions etc. for magnets -the basis of Onsager's work and the whole theory. And in economics there is nothing comparable to these calculations, but the intuition of what they represent is the same – BBM, op cit

A critical phenomenon ?