Applications of Extended Ensemble Monte Carlo

Yukito IBA

The Institute of Statistical Mathematics, Tokyo, Japan

Extended Ensemble MCMC

A Generic Name which indicates:

Parallel Tempering,

Simulated Tempering,

Multicanonical Sampling,

Wang-Landau, …

Umbrella Sampling Valleau and Torrie1970s

Contents1. Basic Algorithms

Parallel Tempering .vs Multicanonical

2. Exact Calculation with soft Constraints

Lattice Protein / Counting Tables

3. Rare Events and Large Deviations

Communication Channels

Chaotic Dynamical Systems

Basic Algorithms

Parallel Tempering

Multicanonical Monte Carlo

References in physics• Iba (2001) Extended Ensemble Monte Carlo Int. J. Mod. Phys. C12 p.623. A draft version will be found at http://arxiv.org/abs/cond-mat/0012323

• Landau and Binder (2005) A Guide to Monte Carlo Simulations in Statistical Physics

(2nd ed. , Cambridge)

• A number of preprints will be found in Los Alamos Arxiv on the web.

# This slide is added after the talk

Slow mixing by multimodal dist.

××

××××

Bridging

fast mixinghigh temperature

slow mixinglow temperature

Path Sampling

1.Facilitate Mixing2.Calculate Normalizing Constant (“free energy”)

In Physics: from 2. to 1.1970s 1990s

“Path Sampling” Gelman and Meng (1998)

stress 2. but 1. is also important

Parallel Tempering

a.k.a. Replica Exchange MC

Metropolis Coupled MCMC

Simulate Many “Replica”s in Parallel

MCMC in a Product Space

Geyer (1991), Kimura and Taki (1991)

Hukushima and Nemoto (1996)

Iba(1993, in Japanese)

Examples

Gibbs Distributions with different temperatures

Any Family parameterized by

a hyperparameter

Exchange of Replicas

K=4

Accept/Reject Exchange

Calculate Metropolis Ratio

Generate a Uniform Random Number

in [0,1) and accept exchange

iff

Detailed Balance in Extended Space

Combined Distribution

Multicanonical Monte Carlo

　　　sufficient statistics

sufficient statisticssufficient statistics

Exponential Family

Energy not Expectation

Berg et al. (1991,1992)

Density of States

　　　　　　　　　　　　　　　The number of which satisfy

Multicanonical Sampling

Weight and Marginal Distribution Original (Gibbs) Multicanonical Random

flat marginal distribution

Scanning broad range of Scanning broad range of E E

Reweighting

Formally, for arbitrary 　　　　 it holds.

Practically, is required,

else the variance diverges in a large system.

Q.　 How can we do without knowledge on D(E)Ans． Estimate D(E) in the preliminary runs

k th simulation

Simplest Method : Entropic Sampling

in

Estimation of Density of States

55

k=1k=1 22

44 1010

k=15k=1514141111

30000 MCS30000 MCS33

(Ising Model on a random net)(Ising Model on a random net)

Estimation of D(E)

• Histogram

• Piecewise Linear

• Fitting, Kernel Density Estimation ..

• Wang-Landau

• Flat Histogram

Entropic Sampling

Original Multicanonical

Continuous Cases D(E)dE : Non-trivial Task

Parallel Tempering / Multicanonical

parallel tempering combined distributionsimulated tempering mixture distribution

to approximate

disorderedordered

Potts model (2-dim, q=10 states)

Phase Coexistence/ 1st order transition

parameter (Inverse Temperature) changes

sufficient statistics (Energy) jumps

water and ice coexists

disordereddisorderedorderedordered

Potts model (2-dim, q=10 states)

bridging by multicanoncal constructionbridging by multicanoncal construction

Comparison

@ Simple Liquids , Potts Models .. Multicanonical seems better than Parallel Tempering

@ But, for more difficult cases ?

ex. Ising Model with three spin Interaction

Soft Constraints

Lattice Protein

Counting Tables

The results on Lattice Protein are taken from joint workswith G Chikenji (Nagoya Univ) and Macoto Kikuchi (Osaka Univ)

Some examples are also taken from the other worksby Kikuchi and coworkers.

Lattice Protein Model

Motivation

Simplest Models of Protein

Lattice Protein :

Prototype of “Protein-like molecules”

Ising Model :

Prototype of “Magnets”

Lattice Protein (2-dim HP)

FIXEDsequence of

conformation of chain STOCHASTIC VARIABLE

SELF AVOIDING(SELF OVERLAP is not allowed)

IMPORTANT!

and corresponds to 2-types of amino acids (H and P)

E(X)= - the number of

Energy (HP model)

in x

the energy of conformation x is defined as

Examples

Here we do not count the pairs neighboring on the chainbut it is not essential because the difference is const.

E=0

E= -1

MCMC

Slow Mixing

Even Non-Ergodicity with local moves

Chikenji et al. (1999)Phys. Rev. Lett. 83 pp.1886-1889

Bastolla et al. (1998) Proteins 32 pp. 52-66

Multicanonical

Multicanonical w.r.t. E only

NOT SUFFUCIENT

Self-Avoiding condition is essential

Soft Constraint

Self-Avoiding condition is essential

Soft Constraint

is the number ofmonomers that occupy the site i

Multi Self-Overlap Sampling

Multi Self-Overlap Ensemble

Bivariate Density of States

in the (E,V) plane

E

V (self-overlap)

Samples withare used for the calculationof the averages

EXACT !!

V=0 large V

E

Generation of Paths by softening of constraints

Comparison with multicanonical with hard self-avoiding constraint

conventional(hard constraint)

proposed(soft constraint)

switching between

three groups ofminimum energy

statesof a sequence

optimization

optimization (polymer pairs)

Nakanishi and Kikuchi (2006)J.Phys.Soc.Jpn. 75 pp.064803 / q-bio/0603024

double peaks

An Advantage of the methodis that it can usefor the sampling at any temperature as well as optimization

3-dim

Yue and Dill (1995) Proc. Nat. Acad. Sci. 92 pp.146-150

Another Sequence

non monotonic changeof the structure

Chikenji and Kikuchi (2000)Proc. Nat. Acad. Sci 97

pp.14273 - 14277

Related WorksSelf-Avoiding Walk without interaction / Univariate Extension Vorontsov-Velyaminov et al. : J.Phys.Chem.,100,1153-1158 (1996)

Lattice Protein but not exact / Soft-Constraint without control Shakhnovich et al. Physical Review Letters 67 1665 (1991)

Continuous homopolymer -- Relax “core”Liu and Berne J Chem Phys 99 6071 (1993)

See References in Extended Ensemble Monte Carlo, Int J Phys C 12 623-656 (2001)but esp. for continuous cases, there seems more in these five years

Counting Tables

４ ９ ２

３ ５ ７

８ １ ６

Pinn et al. (1998)Counting Magic Squares

Soft Constraints+

Parallel Tempering

Sampling by MCMC

Multiple Maxima

　　　 Parallel Tempering

Normalization Constant

　

calculated by Path sampling (thermodynamic integration)

Latin square (3x3)

For For each column, any given number column, any given number appears once and only once once and only once

For each raw, any given For each raw, any given number appears once and only once once and only once

　 Latin square (26x26)

# This sample is taken from the web.# This sample is taken from the web.

Counting Latin Squares

• 6

• 10

• 11

410000 MCS x 27 replicas

510000 MCS x 49 replicas

510000 MCS x 49 replicas

other 3 trials

Counting Tables

Soft Constraints + Extended Ensemble MC

“Quick and Dirty” ways of calculating the number of tables that satisfy given constraints.

It may not be optimal for a special case,

but no case-by-case tricks, no mathematics,

and no brain is required.

Rare Events and Large Deviations

Communication Channels #1

Chaotic Dynamical Systems #2

# 1 Part of joint works with Koji Hukushima (Tokyo Univ).

# 2 Part of joint works with Tatsuo Yanagita (Hokkaido Univ). (The result shown here is mostly due to him )

Applications of MCMCStatistical Physics (1953 ~ )

Statistical Inference (1970s,1980s, 1990~)

Solution to any problem on

sampling & counting

estimation of large deviation

generation of rare events

Noisy Communication Channel

prior encoded & degraded

decode distance (bit errors)by Viterbi, loopy BP,

MCMC

Distribution of Bit Errors

Kronecker delta

tails of the distribution is not easy to estimate

Introduction of MCMC

Sampling noise in channels by the MCMC

Given an error-correcting code

Some patterns of noise are very harmful

difficult to correct

Some patterns of noise are safe

easy to correct

NOT sampling from the posterior

Multicanonical Strategy

MCMC sampling of

Broad distribution of

Broad distribution of distance

and

Multicanonical Sampling

MCMC Sampling and

with the weight

Estimated by the iteration of preliminary runsexactly what we want,

but can be ..

flat marginal distribution

Scanning broad range of bit errors

Enable efficient calculation of the tails of the distribution

(large deviation)

Example

Convolutional Code

Binary Symmetric Channel Fix the number of noise (flipped bits)

Viterbi decoding

SimplificationIn this case

is independent of Set

Binary Symmetric Channel Fix the number of noise (flipped bits)

sum over the possible positions of the noise

Simulation

the number of bit errors

difficult tocalculate by simple

sampling

Correlated Channels

It will be useful for the study of error-correcting code in a correlated channel.

Without assuming models of correlation

in the channel we can sample relevant

correlation patterns.

Rare events in Dynamical Systems

Deterministic ChaosDoll et al. (1994), Kurchan et al. (2005)

Sasa, Hayashi, Kawasaki .. (2005 ~)

(Mostly) Stochastic DynamicsChandler Group

Frenkel et al.

and more …

Transition Path Sampling

Stagger and Step Method Sweet, Nusse, and Yorke (2001)

Sampling Initial Condition

Sampling initial condition of

Chaotic dynamical systemsRare Events

Double Pendulum

control and stop the pendulumone of the three positions

Unstable fixed points

energy dissipation (friction) is assumedi.e., no time reversal sym.

T is max time

Definition of artificial “energy”

stop = zero velocity

stopping position

penalty tolong time

Metropolis step

Integrate Equation of Motion Integrate Equation of Motion andand

Simulate TrajectorySimulate Trajectory

Perturb Initial StatePerturb Initial State Evaluate “Energy”Evaluate “Energy”

Reject or AcceptReject or Accept

for given Tfor given Tfor given Tfor given T

Parallel Tempering

An animation by Yanagita is shown in the talk, but might not be seen on the web.

Summary

Extended Ensemble + Soft Constraint strategy gives simple solutions to a number of difficult problems

The use of MCMC should not be restricted to the standard ones in Physics and Bayesian Statistics.

To explore new applications of MCMC extended ensemble MC will play an essential role.

END