IMA, November 2004

I. G. Kevrekidis -C. W. Gear and several other good people-

Department of Chemical Engineering, PACM & MathematicsPrinceton University, Princeton, NJ 08544

Equation-free Modeling For Complex Systems

orEnabling Microscopic Simulators to perform

System Level Tasksor

Solving Differential Equations Without the Equations

OrA Systems Approach to Multiscale Computations

-1 0 1

No news – drift to Center

SELL BUYNeutral BullishBearish

Bad News Arrives

jump left --Good News Arrives

jump right +

Bad Newsarrival rate(Poisson)

Good Newsarrival rate(Poisson)

If moves to extreme rightagent buys and becomes

neutral

If moves to extreme leftagent sells and becomes

neutral

Sell rate increasesbad news

arrival rate

Buy rate increasesgood newsarrival rate

Influence of other investors

Buyrate

Sellrate

- or lemmings falling off a cliff

SIMULATION RESULTS50000 agents, ε+=0.075, ε - = - 0.072, v0

+=v0-=20

Open loop response, g = 42

SIMULATION RESULTS50000 agents, ε+=0.075, ε - =-0.072, v0

+ = v0- =20

Open loop response, g = 46.5

“simple”

“complicated” “complex”

J. Ottino, NWU

Multiscale / Complex System Modeling

“Textbook” engineering modeling:macroscopic behavior through macroscopic models(e.g. conservation equations augmented by closures)

Alternative (and increasingly frequent) modeling situation:• Models

– at a FINE / ATOMISTIC / STOCHASTIC level– MD, KMC, BD, LB (also CPMD…)

• Desired Behavior– At a COARSER, Macroscopic Level– E.g. Conservation equations, flow, reaction-diffusion, elasticity

• Seek a bridge– Between Microscopic/Stochastic Simulation– And “Traditional, Continuum” Numerical Analysis– When closed macroscopic equations are not available in closed form

What I will tell you:

Solve the equations WITHOUT writing them down.

Write “software wrappers” around “fine level” microscopic codes

Top level: all algorithms we know and love (e.g. AUTO)Bottom level: MD, kMC, LB, BD, heterogeneous/ discrete media,

CPMD, hybridINTERFACE:

Trade Function Evaluationfor “on demand” experimentaton and estimation

“Equation Free” (motivated by “matrix free iterative linear algebra”)Algorithms (coarse integration, patch dynamics, coarse RPM…)Tasks (stability/ bifurcation, control, optimization, dynamic renormalization)Examples (LB, KMC, BD, MD), and some nebulous thoughts

Think of the microscopic simulator AS AN EXPERIMENTThat you can set up and run at will

)(xfx

dt

dEquations

0x

0t

1x2x

1t 2t

Experimental ExperimentalistLegacy Code

Computational Experimentalist

0t

0x

1t

1x

0t

0x

1t

1x

Estimate d/dt

Look at the experiment & RESTART IT

IN EFFECT, Do Forward Euler not with the EQUATION, but with the (computational) EXPERIMENT

Determinism (in a practical way)

Program

…..subroutine

fx

f(x)

RESTRICTION - a many-one mapping from a high-dimensional description (such as a collection of particles in Monte Carlo simulations) to a low-dimensional description - such as a finite element approximation to a distribution of the particles.

LIFTING - a one-many mapping from low- to high-dimensional descriptions.

We do the step-by-step simulation in the high-dimensional description.

We do the macroscopic tasks in the low-dimensional description.

SIMULATION RESULTS50000 agents, ε+=0.075, ε - = - 0.072, v0

+=v0-=20

Open loop response, g = 42

Coarse projective integration: Accelerating things

Simulation results at g = 35, 200,000 agents

x = x(ΙCDF)

0 1

x = x(ΙCDF)

0 1

[α1,…αns](k) [α1,…αns](k+1) [α1,…αns](k+2+M)

MicroscopicSimulator

x = x(ΙCDF)

0 1

x = x(ΙCDF)

0 1 0 1

x = x(ΙCDF)

0 1

x = x(ΙCDF)

MicroscopicSimulator

x = x(ΙCDF)

0 1

x = x(ΙCDF)

0 1

[α1,…αns](k+2)Projection

Ru

n f

or 5

x03

t.u

.P

roj e

c t f

or5x

0.3

t.u

.

Multiscale Modeling Challenges:

Time

Space

Val

ue

Proposal: detailed modeling in small spatial boxes with interpolation between boxes - the “gap-tooth scheme”

How to improve the spatial technology?

xxxt vuuuu Viscous Burgers equation:kMC Realization

Initial Conditions 1

Integrate 1

Restrict 1 Initial Conditions 2

Integrate 2

Restrict 2

Project

Lift

Boxes

Start from macroscopic description Space

TimeSolution

Combined patch-dynamics scheme

THE CONCEPT: What else can I do with an integration code ?

Have equation

Write Simulation

)(xfx

( )x t

Do Newton on 0 )(xx

T

Do Newton

Compile

x

)(xΦ

)( εxΦ

Also

εΦ D

x)(kx

)(xf

)( 1kx )(kx)(kx

)(xx

x

)( tx

Estimate

matrix-vector product

Matrix freeiterative linear algebra

The World

CG, GMRES Newton-Krylov

“Coarse” Bifurcations (PNAS 2000, CChE 2002)

BifurcationResults

Coarse Bifurcation CodeRPM-based

Parameter

coarse IC PDE-basedTimestepper

MicroscopicTimestepper

…{Microscopic IC’s…

Local equilibrium assumption e.g Maxwellian distribution

}

Averaging in timeAnd/or space and or nr. of realizations and filtering

LIFT RESTRICT M

30 35 40 45 500.05

0.1

0.15

0.2

0.25

0.3

0 ( )m x

g

The Bifurcation Diagram

35 40 45 50

0.8

1

1.2

1.4

1.6

1.8

2

g

1

-1 0 10

100

200

300

400

-1 0 10

100

200

300

400 ) ,x( gH 0)()xx(α 11 Sgg

S

T

)( 01 xx

αS

gg

Δ

)( 01

Tracing the branch with arc-length continuation)(x )( tx

T 0),( gxx

The Bifurcation Diagram50000 agents, g=35, ε+=0.075, ε-=-0.072, v0

+=v0-=20

Open loop response. From unstable to stable markets

The Bifurcation Diagram50000 agents, g=35, ε+=0.075, ε - = -0.072, v0

+ = v0- = 20

Open loop response. Blow up

STABILIZING UNSTABLE M*****S

Feedback controller design

We consider the problem of stabilizing an equilibrium x*, p* of a dynamical system of the form

))( ),(((t)tptxfx , f : RN x RRN where f and hence x* is characterized of uncertainty

To do this the dynamic feedback control law is implemented: ( )* K x D wp p

Where w is a M-dimensional variable that satisfies w x D w

Choose matrices K, D such that the closed loop system is stable

At steady state: 0 0w , x *p p and the system is stabilized in it’s“unknown” steady state

*, x x*p p

In the case under study the control variable is the exogenous arrival frequency of “negative” information vex

- and the controlled variables the coefficients of the orthogonal polynomials used for the approximation of the ICDF

STABILIZING UNSTABLE MARKETS

Control variable: the exogenous arrival frequency of “negative” information vex-

Coarse Self-Similar Solutions:

Dynamic Renormalization using Coarse Timesteppers.

An analogy: problems with traveling solutions (translational invariance)Move along with the solution – it appears steady

(u)Lt

u

(v)x

v(v) Lc

t

v

Original Equation

Transformed equation

(template-based reduction/reconstruction, Rowley and Marsden, 2000)

problems with focusing or collapsing solutions (scale invariance)explode along with the solution - it appears steady

Templates: Aronson, Betelu and Kevrekidis, 2001

Original Equation

Transformed equation

(u)Lt

u

Rowley, Kevrekidis, Marsden and Lust, 2003)

Discrete time, lift –run – restrict:

Coarse renormalization flow integration / bifurcation analysis

)())(( vLv

vvGt

v

Formulation( ); ( ( )) ( ( )), a b

t x x yx xu D u D Bf A B D f y yA A

1

If ( , ) ( ) ( , ( ))( )

( )y y

a bs

B AD w w w ywB

xu x s B s w sA s

A B

A

If ( , ) ( )

1

( )y y

xu x t

a

s Us

D U U yU

b

BB

AA

/lim

/lim

Compare Eqns (2) (3):

(1)

(5)

(2)

(3)

(4)

Eqns (1) (5) determines and

Time evolution of an initial rectangular density profile

by the 1d diffusion equation

Dynamically renormalizedRegular

Schematic view of the dynamically renormalized coarse timestepper

1D glass compaction model

Simulation Method

• 100000 particles at given density placed in 1D simulation box with periodic boundary condition.

• Particles interact through hard-core potential.• Monte Carlo random walk is performed in each

step.• Once a gap of unit size opened up between two

adjacent particles, an additional block will deposit.

• CDF of inter-particle space is recorded.

Glassy dynamics

Dynamics in linear time Dynamics in logarithmic time

)ln(~1

t

Eq. 7 StinchcombePRL 88, 125701 (2002)

Self-Similar Distribution

Dynamic Rescaling Fixed Point Calculation at =0.9

GMRES Fixed Point Calculation

Dynamics With Limited Distribution and Adjusted t0

bttk

)ln(1

0

Reverse Projective Integration – a sequence of outer integration steps backward; based on forward steps + estimation

We are studying the accuracy and stability of these methods

12

3

Reverse Integration: a little forward, and then a lot backward !

Forward and Backward Integration

Forward Integration Backward Integration

NLDbyMDofH2OinCNT

Water Confinement in Carbon Nanotube Molecular Channels

Classical Molecular Dynamics (MD)

- Flexible CNT (L = 13.5 Å; R = 8.1

Å)

- Graphite parameters

- 1034 TIP3P water molecules

- AMBER 6.0

- Particle-mesh Ewald electrostatics

- Gerhard Hummer

G. Hummer, J. C. Rasaiah, and J. P. Noworyta, Nature 414, 188-190 (2001)

Model system for studying H20 transport in channel pores of membrane proteins

Filling-Emptying Transition of H2O in CNT

G. Hummer, J. C. Rasaiah, and J. P. Noworyta, Nature 414, 188-190 (2001)

CNT

Water

Filling-Emptying Transition of H2O in CNT

“Equation Free” Approach

Analyse effect of CNT-Water Interactions on the H20 Occupancy

G. Hummer, J. C. Rasaiah, and J. P. Noworyta, Nature 414, 188-190 (2001)

.

.

.

The Heart of the Method – The “Timestepper”

M

ii tNN

MN

10 ;0,;

1, ;0,; 0 tNN

“Coarse” Initial Condition

50 Copies

“FREE”

MD

Choose Parameters

Constrained MD

NN

N

NNN

Ddt

d

G

Tk

D

dt

d

B

2var

xk Xk+1

N0 N

Nvar

.

.

.

“Lifting”

• Constrain for = 15 ps and sample configurations: > 10 ps• Initialize replicas: velocities from Maxwell-Boltzmann distribution

= 1 ps(Coarse “fine”)

0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.000

1

2

3

4

5

6

7

N*

Fully Empty States

Fully Filled States

Partially Filled States

Bifurcation Diagram: N vs

“Vestibule”

“Vestibule”

Alanine DipeptideIn 700 tip3p waters

w/ Gerhard Hummer, NIDDK / J.Chem.Phys. 03

The waters The dipeptide and the Ramachandran plot

Parametrizing nonlinear mainfoldsGiven: (noisy) data in space, unordered, in high dimension. Want: discover meaningful parameters.Euclidean distances between any two points usually not meaningful.Euclidean distances between very close points is meaningful. Use local Euclidean distances and glue them to find paths between any two points. Diffusion distance: Compute ALL paths between A and B and take a weighted average (long paths count less, but there are more of them). This is stable under noise and behaves well with bottlenecks. It is a diffusion process.

-0.5 0 0.5 1 1.5 2 2.5

-1.5

-1

-0.5

0

0.5

A

B

1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8

-1.8

-1.6

-1.4

-1.2

-1

-0.8

-0.6

A

B

geod.dist(A,B)≈geod.dist(B,C), diff.dist(A,B)>>diff.dist(B,C)

Slowly communicating states:

Graphs, Laplacian Eigefunctions, Embeddings, Heat ParametrizationWe want to compute diffusion distances, and then deduce parametrization from those.Data Vertices of graphLocal distances Conductivity of edges Want to solve an heat/electrical network equation: look at Laplacian on graph, compute eigenfunctions

The eigenfunctions (long time behaviour) can be used to compute diffusion distances:

and to parametrize, mapping the set into R

1.2 1.4 1.6 1.8 2 2.2 2.4

-1.6

-1.4

-1.2

-1

-0.8

-0.6

0

1

2

x 10-4

A

B

Color proportional to diffusion distance, related to how much heat flows between A and B in a certain time

Slowly communicating states are mapped by the eigefunctions into far away points.

A B

C** Belkin, Nyogi, LafonComputations: - Order n, nlog(n) via eigenfunctions- Order nlog(n) via diffusion wavelets (full multiscale organization)All depend on decay of eigenvalues, i.e. whether there is separation of time scales

n

Alanine Dipeptide Data

12-atom dipeptide fragments in water. Long time coarse molecular dynamics by projective integration.Data consists of the coordinates of the atoms in the molecule.

The eigenfunctions as parameters

They do a good job at parametrizing the set of configurations of the molecule during the simulation. In here we can see the image of the trajectories in the coordinates given by the top 3 eigenfunctions, each point is a configuration, red points are from the first (comprehensive) simulation, blue points from second (mostly stuck in the wells) simulation.

Just from the gap in the spectrum, one can see the set of configurations embeds very well in 4 dimensions.

0 2 4 6 8 10 12 14 16 18 20-0.01

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07Eigenvalues of Laplacian on the set of structures

Big gap

4 almost disconnected components corresponding to potential wells.

Potential wells Transitions

The parametrization is dynamically significant if the equation of motion (differential/stochastic) are local and smooth (modulo small random perturbations).

Rare Events: Escaping from stabilityOpen loop response, g = 45.8

Histogram of escape times

( x(t=0)=0 )

0 50 100 150 200 250 3000

10

20

30

40

50

60

70

80

180 runs

Recursive Projection Method (RPM)

• Recursively identifies subspace of slow eigenmodes, P

Subspace P of few slow eigenmodes

Subspace Q =I-P

Reconstruct solution:u = p+q = PN(p,q)+QF

Pica

rdite

ratio

ns Newtoniterations

• Treats timstepping routine, as a “black-box”

– Timestepper evaluates un+1= F(un)Initial state un

TimesteppingLegacy Code

Convergence?

Final state uf

F(un)

YES

Picard iteration

NO

• Substitutes pure Picard iteration with

–Newton method in P–Picard iteration in Q = I-P

• Reconstructs solution u from sum of the projectors P and Q onto subspace P and its orthogonal complement Q, respectively:

–u = PN(p,q) + QF

F.P.I.

•Isothermal operation•Modeling Equations (Nilchan & Pantelides)

Step 1 : Pressurisation

Step 2: Depressurisation

Rapid Pressure Swing Adsorption1-Bed 2-Step Periodic Adsorption Process

t=0 to T/2

Ci(z=0)=PfYf/(RTf)

P(z=0)=Pf

z=0

z=L

t= T/2 to T

P(z=0)=Pw

0)0(

zz

Ci

)(

)1(180

)(

3

2

2

1

2

2

iiiii

b

b

p

n

ii

ii

iib

it

qpmkt

q

d

v

z

P

CRT

Pz

CD

z

vC

t

q

t

C

Mass balance in ads. bed

Darcy’s law

Rate of ads.

RPSA simulation results

0

0.2

0.4

0.6

0.8

1 02

46

810

12

30

35

40

45

50

55

60

65

70

(sec) Time(m) Axial

C1

(mol

e/k g

-1)

0 500 1000 1500 2000 2500 3000 35005

10

15

20

25

30

35

40

45

50

160 180 200 220 240 260 28020

22

24

26

28

30

32

34

COARSE INTEGRATION

Coming full circleNo equations ?

Isn’t that a little medieval ? Equations = “Understanding”, right ?

AGAIN matrix free iterative linear algebra

A x = b

PRECONDITIONING, B A x = B b

B approximate inverse of A

Use “the best equation you have”

to precondition equation-free computations.

With enough initialization authority:

equation free laboratory experiments

Computer-Aided Analysisof Nonlinear Problems in Transport Phenomena

Robert A. Brown, L. E. Scriven and William J. Silliman

in HOLMES, P.J., New Approaches to Nonlinear Problems in Dynamics, 1980

ABSTRACT The nonlinear partial differential equations of mass, momentum, energy, Species and charge transport…. can be solved in terms of functions of limited differentiability,no more than the physics warrants, rather than the analytic functions of classical analysis…….. basis sets consisting of low-order polynomials. …. systematically generating andanalyzing solutions by fast computers employing modern matrix techniques.

….. nonlinear algebraic equations by the Newton-Raphson method. … The Newton-Raphsontechnique is greatly preferred because the Jacobian of the solution is a treasure trove, not onlyfor continuation, but also for analysing stability of solutions, for detecting bifurcations ofsolution families, and for computing asymptotic estimates of the effects, on any solution, ofsmall changes in parameters, boundary conditions, and boundary shape……

In what we do, not only the analysis, but the equations themselves are obtained on thecomputer, from short experiments with an alternative, microscopic description.

EXISTING WORK Coarse kMC of catalytic surface reactions Coarse instabilities in gas-liquid multiphase flows (LB)Coarse rheology of nematic liquid crystals (BD)Homogenization / Effective Equations for random media Coarse Optimal Paths / Coarse Optimization and Control (LB, kMC)Coarse Molecular Dynamics (w/NIH, MD, CPMD)Dislocation motion w/ diffusing impurities (kMC)Legacy Simulators-Pressure Swing Adsorption, (gPROMS)Coarse Computational Epidemiology

Influenza A virus evolution, (kMC)Coarse granular flow ( MD)Structural Transitions in Metal Crystals (MS, MD)Coupled Oscillator Fields / Computational Neuroscience (w/ NIH; kMC)Equilibrium Complex Fluids-Micelle Formation, (MC)Bifurcations and Coarse Bifurcations in Renormalization FlowsAgent-based modeling in environmental and social sciences (kMC)Coarse modeling of transport in model turbulent flows (SDEs)Growth experiments – KPZ type equations (SDEs, kMC)