nonlinear estimators and time-embedding
Post on 01-Nov-2014
629 Views
Preview:
DESCRIPTION
TRANSCRIPT
Nonlinear estimatorsand time embedding
Raul Vicente raulvicente@mpih-frankfurt.mpg.de
FIASFrankfurt, 08-08-2007
OUTLINE
Introduction to nonlinear systems
2
Phase space methods
Exponents and dimensions
Interdependence measures
Take-home messages
OUTLINE
Introduction to nonlinear systems
3
Definition
Why nonlinear methods?
Linear techniques
Phase space methods
Exponents and dimensions
Interdependence measures
Take-home messages
INTRODUCTIONDefinition
A nonlinear system is one whose behavior can‘t be expressed asa sum of the behaviors of its parts. In technical terms, the behaviorof nonlinear systems is not subject to the principle of superposition.
4
„Nonlinear“ is a very popular word in (neuro)science but what does it really mean?
The brain as a whole is a nonlinear „device“
Ex: our perception can be more than the sum of responsesto individual stimulus Surface completion
INTRODUCTIONDefinition
A nonlinear system is one whose behavior can‘t be expressed asa sum of the behaviors of its parts. In technical terms, the behaviorof nonlinear systems is not subject to the principle of superposition.
4
„Nonlinear“ is a very popular word in (neuro)science but what does it really mean?
Individual neurons are also nonlinear
Excitable cells with all or none responses Double the input does not
mean double the output
Nonlinear frequency response
INTRODUCTIONDefinition
A nonlinear system is one whose behavior can‘t be expressed asa sum of the behaviors of its parts. In technical terms, the behaviorof nonlinear systems is not subject to the principle of superposition.
4
„Nonlinear“ is a very popular word in (neuro)science but what does it really mean?
Individual neurons are also nonlinear
Double the input does not mean double the output
The brain as a whole is a nonlinear „device“
Ex: our perception can be more than the sum of responsesto individual stimulus
INTRODUCTIONWhy nonlinear methods?
5
„The study of non-linear physics is like the study of non-elephant biology“Unknown
Neuronal activity is highly nonlinear
Nonlinear features will be present in the recorded neurophysiological data
From neuronal action potentials (spikes) to integrated activity (EEG, MEG, fMRI)
Linear techniques might fail to capture key information
Nonlinear indices: measure complexity of EEG, monitoring depth of anaesthesia, studies of epilepsy, detection of interdependence, etc...
INTRODUCTIONLinear techniques
6
Linear systems always need irregular inputs to produce bounded irregular signals
Linear SystemLinear System
Most simple system which produces nonperiodic (interesting) signals is a linear stochastic process
...,Sn-1, Sn, Sn+1,...
Measurement of state Sn at time n of such a process p(s) probability dist.
Information about p(s) can be inferred from the time series:
1
1 N
nn
s sN
2
2
1
1
1
N
nn
s sN
INTRODUCTIONLinear techniques
7
Linear methods interpret all regular structure in a data set, such as a dominantfrequency, as linear correlations (time or frequency domain)
2
2 2
1 n nn n
s s sc s s s s
Autocorrelation at lag
Sn
Sn
-
Sn
Sn
-
Sn
Sn
-0c 0c 0c
Periodic signalStochastic processChaotic system
Periodic autocorrelationDecaying autocorrelationExponential decay ?
INTRODUCTIONLinear techniques
8
Cross-correlation function: measures the linear correlation between two variablesX and Y as a function of their delay time ()
1
1( ) ( ) ( )
N
XYk
C x k y kN
Cross-correlation at lag
( ) 0XYC ( ) 0XYC ( ) 0XYC
tendency to have similar values with the same signtendency to have similar values with opposite signsuggest lack of linear interdependence
EEG time series recorded from the two hemispheres in a rat
X´=X4
Y´=Y4
rxy = 0.63 rxy = 0.25
that maximizes this function
estimator delay between signals
INTRODUCTIONLinear techniques
8
Cross-correlation function: measures the linear correlation between two variablesX and Y as a function of their delay time ()
1
1( ) ( ) ( )
N
XYk
C x k y kN
Cross-correlation at lag
( ) 0XYC ( ) 0XYC ( ) 0XYC
tendency to have similar values with the same signtendency to have similar values with opposite signsuggest lack of linear interdependence
that maximizes this function
estimator delay between signals
Cross-correlogram histogram is also used to reveal the temporal coherence in the firing of neurons
MT neurons in visual cortex of a macaque monkey
INTRODUCTIONLinear techniques
9
Coherence: measures the linear correlation between two signals as a function ofthe frequency
2( )
( )( ) ( )
XYXY
XX YY
S fK f
S f S f Coherence at frequency f
0( ) 0XYK f
0( ) 1XYK f activities of the signals in this frequency are linearly independentmaximum linear correlation for this frequency
In forming an estimate of coherence, it is always essential to simulate ensemble averaging. EEGand MEG signals are subdivided in epochs or forevent-related data spectra are averaged over trials
( ( ))XYFFT C
INTRODUCTIONLinear techniques
10
Prediction: we have a sequence of measuraments sn, n = 1,...,N and we want to predict the outcome of the following measurement, sN+1
Linear prediction1
1
m
n j n m jj
s a s
minimising the error
2
1 1
N
n nn m
s s
In-sample Out-of-sample
ja
INTRODUCTIONLinear techniques
11
Causality: (Nobert Wiener in 1956) for two simultaneously measured signals, if one can predict the first signal better by incorporating the past information from the second signal than using only information from the first one, then the second signal can be called causal to the first one
Predicting the future of X improves when incorporating the information about the past of Y → Y is causal to XX1
Y1
Time
X2
In neurophysiology, a question of great interest is whether there exists a causal relation between twobrain regions. Inferring causality from the time delay in the cross-correlation is not always straighforward
INTRODUCTIONLinear techniques
12
Causality: (Nobert Wiener in 1956) for two simultaneously measured signals, if one can predict the first signal better by incorporating the past information from the second signal than using only information from the first one, then the second signal can be called causal to the first one
X1
Y1
Time
X2
Granger just applied this definition in the context of linear stochastic models. If X is influencing Y, then adding the past values of the first variable to the regression of the second one will improve its prediction error.
Univariate fitting Bivariate fitting
Prediction performance: is assessed by the variances of the prediction errors
Granger causality
OUTLINE
Introduction to nonlinear systems
13
The concept of phase space
Attractor reconstruction
Time embedding
Application: nonlinear predictor
Phase space methods
Exponents and dimensions
Interdependence measures
Take-home messages
Phase spaceThe concept
14
Phase space: is a space in which all possible states of a system are represented, with each possible state of the system corresponding to one unique point in the phase space
Ex: the Fitzhugh-Nagumo model is a two-dimensional simplification of the Hodgkin-Huxley model of spike generation
In a phase space, every degree of freedom or parameter of the system is represented as an axis of a multidimensional space. Ex: dim(HH) = 4
- membrane potential- recovery variable
(V, W)V(t)
W(t)
For deterministic systems (no noise), the system state at time t consists of all information needed to uniquely determine the future system states for times > t
Phase spaceAttractor reconstruction
15
Attractor: a set of points in phase space such that for "many" choices of initial point the system will evolve towards them. It is a set to which the system evolves after a long enough time
Attractor setA pointA curve
A manifold
BehaviorConstantPeriodic
Possibly chaotic
Van der Pol limit cycle attractor
Lorenz attractor
Strange attractors produce chaotic behavior.Nonlinear systems: irregular dynamics without invoking noise!
Phase spaceAttractor reconstruction
16
In general (and especially in biological systems) it is impossible to access all relevant variables of a system. Ex: usually in electrophysiology we just measure membrane voltage.
[ ( ), ( ), ( 2 )]tX x t x t T x t T
These vectors constructed from a single variable play a role similar to [x(t),y(t),z(t)]
How from a single measured quantity can onereconstruct the original attractor?
?
Phase spaceTime embedding
17
In general (and especially in biological systems) it is impossible to access all relevant variables of a system. Ex: usually in electrophysiology we just measure membrane voltage.
[ ( ), ( ), ( 2 )]tX x t x t T x t T
( )( , ( ), ( ), ( ));
( )( , ( ), ( ), ( ));
( )( , ( ), ( ), ( ));
dx tf t x t y t z t
dtdy t
g t x t y t z tdt
dz th t x t y t z t
dt
3 2( ) ( ) ( )( , ( ))
d x t d x t dx ta b p t x t
dt dt dt
2( ) ( )[ ( ), , ]
dx t d x tx t
dt dt2
( ) ( ) ( ) 2 ( ) ( 2 )[ ( ), , ]
x t x t T x t x t T x t Tx t
T T
Time delay embedding
[ ( ), ( ), ( 2 ),..., ( )]tX x t x t T x t T x t mT
m > 2DF
T ~ first zero autocorrelation
Phase spaceApplication: nonlinear predictor
18
- A signal does not change is easy to predict: take the last observation as a forecast for the next one
Depending on the type of signals the power of predictability and the best strategy changes:
- A periodic system is also easy one observed for a full cycle
- For independent random numbers the best prediction is the mean value
- Interesting signals are not periodic but contain some kind of structure which can be exploited to obtain better predictions
If the source of predictability are linear correlations in time: next observations will be given approximately by a linear combination of preceding observations
1 01
m
n j n m jj
s a a s
What if I know that my series is nonlinear?
Phase spaceApplication: nonlinear predictor
19
For nonlinear deterministic systems all future states are unambiguosly determinedby specifying its present state. Nonlinear correlations can be exploited with new techniques
Lorenz method of analogues:Prediction for the future state XN+1
Look for recorded states close to the one we want to predictPredict the average of the next states of the past neighbors
Current state
Neighbors
[ ( ), ( ), ( 2 )]tX x t x t T x t T
Next values of
( )
1
( )n N
N n n nX U XN
X XU X
Better prediction for short time scales than linear predictors
Predicted state
OUTLINE
Introduction to nonlinear systems
20
Sensibility to initial conditions: Lyapunov exp.
Self-similarity: correlation dimension
Phase space methods
Exponents and dimensions
Interdependence measures
Take-home messages
Exponents and dimensionsSensibility to initial conditions
21
The most striking feature of chaos is the long-term unpredictability of the futuredespite a deterministic time evolution.
The cause is the inherent instability of the solutions, reflected in their sensitivedependence on initial conditions.
Amplification of errors since nearby trajectories separate exponentially fast. How fast it is measured by the Lyapunov exponent .
( ) (0)exp( )t t
type of motion maximal Lyap. exp.
stable fixed point < 0
limit cycle = 0
chaos 0 < < ∞
noise = ∞
The inverse of the maximal Lyap. Exp.defines the time beyond which predictability is impossible.
Exponents and dimensionsSensibility to initial conditions
22
The most striking feature of chaos is the long-term unpredictability of the futuredespite a deterministic time evolution.
The cause is the inherent instability of the solutions, reflected in their sensitivedependence on initial conditions.
( ) (0)exp( )t t
Maximal Lyapunov exponent from time series:
- Delay embedding- Compute the average diverging rate:
0
0 001 ( )
1 1( ) ln
( )n n
N
n n n nn X U Xn
S n X XN U X
- The slope of S(n) is an estimate of the maximal LE
Exponents and dimensionsCorrelation dimension
23
Strange attractors with fractal dimension are typical of chaotic systems. Non integerdimensions are assigned to geometrical objects which exhibit self-similarity and structure on all length scales.
Box-counting dimension:
( ) , 0.FDN r r r
For time series:
- Delay embedding- Compute the correlation sum: 1 1
2( ) ( )
( 1)
N N
i ji j i
C r rN N
X X
0
ln ( )lim
lnr
C rD
r
Kaplan-Yorke conjecturerelates D to Lyapunov spectra
Exponents and dimensionsApplications
24
Nonlinear statistics such exponents, dimensions, prediction errors, etc., can becomputed to characterize non-trivial differences in signals (EEG) between different stages (brain states: sleep/rest, eyes open/closed).
Word of caution: such quantities are used to compare data from similar situations!
Ex: ECG series taken during exercise are more noisy due to sweat of patient skin.
The different noise levels at rest and exercising can affect the former nonlinear estimators and erroneusly conclude a higher complexity of the heart duringexercise just because of sweat on the skin.
OUTLINE
Introduction to nonlinear systems
25
Synchronization
Phase space methods
Exponents and dimensions
Interdependence measures
Take-home messages
Interdependence measuresSynchronization
24
Synchronization is the dynamical process by which two or more oscillators adjust their rhythms due to their weak interaction.
A universal phenomenon found everywhere:
Mechanical systems (pendula, London’s bridge, …)
Electrical generators (power grids, Josephson junctions, …)
Life sciences (biological clocks, firing neurons, pacemaker cells, …)
Chemical reactions (Belousov-Zhabotinsky)
...
Synchronization refers to the way in which coupled elements, due to their dynamics, communicate and exhibit collective behavior.
In large populations of oscillators synchronization can be understood as a self-organization process.
Without a master o leader the individuals spontaneously tend to oscillate in synchrony.
Neural synchronization is one of the most promising mechanism to underlay the flexible formation of cell assemblies and thus bind the information processed at different areas.
Interdependence measuresSynchronization hallmarks
25
Before coupling: f
F=F2-F1=0f2f1
After coupling: F
in-phase
Higher order: F2/F1=q/p
anti-phase
Phase shift is fixed:
Phase can be extracted from data by several techniques:
• Hilbert transform• Wavelet transform• Poincare map
__________________________Frequency locking___________________________
____________________________Phase locking______________________________
Interdependence measuresSynchronization solutions
Information theory: mutual information, entropies,... 26
Different types of synchronization capture different relationships betweenthe signals x1(t) and x2(t) of two interacting systems:
Classical: adjustment of rhythms in periodic oscillators.
Identical: coincidence of outputs due to their coupling, x1(t)=x2(t).
Generalized: captures a more general relationship like x1(t)=F(x2(t)).
Phase: expresses the regime where the phase difference between two irregular oscillators is bounded but their amplitudes are uncorrelated.
Lag: accounts for relation between two systems when compared atdifferent times such as x1(t)=x2(t-).
Noise-induced: synchronization induced by a common noise source.
OUTLINE
Introduction to nonlinear systems
27
Phase space methods
Exponents and dimensions
Interdependence measures
Take-home messages
Take-home messagesLinear vs nonlinear
28
- Linear techniques are much well understood and rigorous.
- Linear and nonlinear estimates may assess different characteristics of the signals.
- Complementary approaches to the analysis of temporal series.
Even though most of systems in Nature are nonlinear do not underestimate linear methods
Take-home messagesPhase space methods
29
- Attractor reconstruction is a powerful technique to recover the topological structureof an attractor given a scalar time series.
- Useful complexity quantifiers of the signal can be computed after the reconstruction.
- Word of caution in their use.
Take-home messagesDo it yourself... with a little help
29
- TISEAN is a very complete software package for nonlinear time series analysis
- „Nonlinear time series analysis“ by Holger Kantz and Thomas Schreiber, Cambridge University Press.
top related