Parallel ComputationalComplexity and Statistical

Physics

Jon MachtaUniversity of Massachusetts Amherst

Supported by the National Science Foundation

Outline

• Parallel computing and computational complexity

• Complexity of models in statistical physical

• Physical complexity and computational complexity

Collaborators

• Ray Greenlaw, Armstrong Atlantic University• Cris Moore, University of New Mexico, SFI

• Ken Moriarty, UMass• Xuenan Li, UMass

• Dan Tillberg, UMass• Ben Machta, Brown University

Computational Complexity•How do computational resources scale with the size ofthe problem?

TimeHardware

•Equivalent models of computation.Turing machineParallel random access machineBoolean circuit familyFormal logic

Boolean Circuit Family

NOR NOR

NOR NOR NOR

NOR NOR

INPUT INPUT INPUT

OUTPUT OUTPUT

width

depth

•Gates evaluated one levelat a time from input tooutput with no feedback.•One hardwired circuit foreach problem size.•Primary resources

Depth=number of levels, DcWidth=maximum numberof gates in a levelWork=total number of gates

Parallel ComputingAdding n numbers can be carried out in O(log n) steps

using O(n) processors.

+X1 X2

+X3 X4

+X5 X6

+X7 X8

+ +

+

ΣXi

log n

Connected components of a graph can be found inO(log2n) steps using n2 processors.

Parallel Random AccessMachine

1 2 3 m

Controller

Global Memory

Processors

PRAM•Each processor runs thesame program but has adistinct label

•Each processorcommunicates with anymemory cell in a singletime step.

•Primary resources:Parallel time ~ depth

Number of processors~width

Complexity Classes andP-completeness

•P is the class of feasible problems: solvable withpolynomial work.•NC is the class of problems efficiently solved inparallel (polylog depth and polynomial work, NC ⊆ P).

•Are there feasible problems that cannot be solvedefficiently in parallel (P≠NC)?

•P-complete problems are the hardest problems in P tosolve in parallel. It is believed they are inherentlysequential: not solvable in polylog depth.•The Circuit Value Problem is P-complete.

NOR NOR

NOR NOR NOR

NOR

TRUE FALSE TRUE

?

Statistical Physics

The study of the emergent properties of manyparticle system using probabilistic methods.Objects of study are statistical ensembles ofsystem states or histories:

•Equilibrium states--Gibbs distribution•Non-equilibrium states--stochastic dynamics

Computational statistical physics: sample theseensembles.

Sampling Complexity

•Monte Carlo simulationsconvert random bits intodescriptions of a typicalsystem states.•What is the depth of theshallowest feasible circuit(running time of the fastestPRAM program) thatgenerates typical states?

NOR NOR

NOR NOR NOR

NOR NOR

Typical System State

depthRandom Bits

Depth is a property of systems in statistical physics

Diffusion Limited Aggregation

•Particles added one at a timewith sticking probabilitiesgiven by the solution ofLaplace’s equation.•Self-organized fractal object

df=1.715… (2D)•Physical systems:

Fluid flow in porous mediaElectrodepositionBacterial colonies

Witten and Sander, PRL 47, 1400 (1981)

Random Walk Dynamics for DLA

#1#3#2 #1 #3

#2

Parallel dynamics ignoresinterference between 1 and 3

Sequential dynamics

The Problem with Parallelizing DLA

Depth of DLATheorem: Determining the shape of an aggregate from therandom walks of the constituent particles is a P-completeproblem.Proof sketch: Reduce the Circuit Value Problem to DLA dynamics.

Caveats:1. P≠NC not proven2. Average case may be easier than worst case3. Alternative dynamics may be easier than random walk dynamics

a b c

input 1 input 2

power

output

d

132

tentative cluster, step N tentative cluster, step N+1

1. Start with seed particle at the origin and N walk trajectories2. In parallel move all particles along their trajectories to tentative sticking

points on tentative cluster, which is initially the seed particle at the origin.3. New tentative cluster obtained by removing all particles that interfere

with earlier particles.4. Continue until all particles are correctly placed.

Parallel Algorithm for DLAD. Tillberg and JM, PRE 69, 051403 (2004)

2 1

Efficiency of the Algorithm•DLA is a tree whosestructural depth , Ds scalesas the radius of the cluster.•The running time, T of thealgorithm is asymptoticallyproportional to the structuraldepth.

Ds/T

T

Internal DLAParticles start at the origin, random walk and stick where theyfirst leaves the cluster.

•Shape approaches a circle with logarithmic fluctuations.•P-completeness proof fails.

Parallel Algorithm for IDLA1. Start with seed particle at the origin and N walk trajectories2. Place particles at expected positions along their trajectories.3. Iteratively move particles until holes and multiple occupancies are

eliminated

Average parallel timepolylogarthmic orpossibly a small powerin N.

Cluster of 2500 particles made in 6 parallel steps.

1

2 4

3 5

6

C. Moore and JM, J. Stat. Phys. 99, 661 (2000)

Hierarchy of Depth vs Size

Mandelbrot percolation DLA

constant polylog polynomial

T>Tc Ising T=Tc IsingEden growth

Invasion percolation

Eden growthInvasion percolationScale free networksBallistic depositionBak-Sneppen model

Internal DLA

What is Physical Complexity?

• “I shall not today attempt further to define the kindsof material I understand to be embraced with thatshorthand description. ... But I know it when I see it.”– Justice Potter Stewart on pornography

< ≤ <<

History and Complexity

• The emergence of a complex system fromsimple initial conditions requires a long history.

• History can be quantified in terms of thecomputational complexity of simulating statesof the system.

–Charles Bennett

Depth is the appropriate measure to quantify history.

Properties of Depth• Defined for any system in the framework of

statistical physics.• For a system of nearly independent subsystems

given by the maximum over the subsystems.• Greatest at the boundary between order and

disorder.• Systems that solve (P-hard) computational

problems are deep.

…and physical complexity

+ ≈

Depth of a system of nearly independentsubsystems given by the maximum overthe subsystems.

≈Depth is intensive (independent ofsize) for homogeneous systemswith short range correlations.

Follows immediately from parallelism

Depth is greatest at the boundary betweenorder and disorder.

In equilibrium systems with continuousphase transitions, such as the Ising model,depth is greatest at the critical point,separating order and disorder (criticalslowing down).

<< <<

Conclusions

Parallel computational complexity theory providesinteresting perspectives on model systems in statisticalphysics.Depth, defined as the minimum number of parallel stepsneeded to simulate a system, is a prerequisite for physicalcomplexity and has many of its properties.