gravitySimulatogravitySimulatorrBeyond the Million Body Beyond the Million Body ProblemProblem

Collaborators:Collaborators: Rainer Spurzem (Heidelberg)Rainer Spurzem (Heidelberg)Peter Berczik (Heidelberg/Kiev)Peter Berczik (Heidelberg/Kiev)Simon Portegies Zwart (Amsterdam)Simon Portegies Zwart (Amsterdam)Alessia Gualandris (Amsterdam)Alessia Gualandris (Amsterdam)Hans-Peter Bischof (RIT)Hans-Peter Bischof (RIT)

Stefan Harfst and David MerrittStefan Harfst and David MerrittRochester Institute of TechnologyRochester Institute of Technology

Modelling Dense Stellar SystemsModelling Dense Stellar Systems

one approach: direct N-body simulationsone approach: direct N-body simulations exact but very compute-intensive exact but very compute-intensive ~O~O((NN22)) many problems require large Nmany problems require large N

e.g. the evolution of binary Black holese.g. the evolution of binary Black holes ““empty losscone” is artificially repopulated by empty losscone” is artificially repopulated by

two-body scattering unless two-body scattering unless N N > 10> 1066

How to deal with large How to deal with large NN

A standard SupercomputerA standard Supercomputer Special-purpose hardwareSpecial-purpose hardware

GRAvity PipEline (GRAPE)GRAvity PipEline (GRAPE)

(J. Makino, T. Fukushige)

Customed-designed pipelines for force calculations

Very fast (~1 TFlops) Limited particle numbers

(< 1/4 million) Cost: ~$50K + extras (GRAPE-6)

The GRAPE clusterThe GRAPE cluster

mini-GRAPEs(GRAPE-6A) N < 131,072

GRAPE clusterGRAPE clusterRIT’s gravitySimulator isoperational since Feb 2005 32 dual 3GHz-Xeon nodes 32 GRAPE-6A’s 14 Tbyte RAID low-latency Infiniband

interconnects (10Gbps) Speed: 4 TFlops N up to 4 Million particles Cost: $0.5x106

Funding: NSF/NASA/RIT Next largest:

24 nodes (University of Tokyo) soon 32 nodes (Heidelberg)

The Code and PerformanceThe Code and Performance new parallel direct-summation new parallel direct-summation

codecode fourth-order Hermite fourth-order Hermite

integratorintegrator individual, block time stepsindividual, block time steps

achieves best performanceachieves best performance for small particle numbers for small particle numbers

communication dominatescommunication dominates efficiencies are between 60% efficiencies are between 60%

(many processors) and 90% (many processors) and 90% (few processors)(few processors)

For details see posterFor details see poster

GRAPE

PC

• store local particles

• select active particles

• collect all active particles

• compute local force and sum over all nodes

Visualization of N-Body SimulationsVisualization of N-Body Simulations

new software new software package “Spiegel”package “Spiegel” GUI to plot N-body GUI to plot N-body

data and make data and make moviesmovies

See Poster for detailsSee Poster for details

in collaboration with Hans-Peter Bischof (RIT)