Application Projects in VIOLA
The integration of applications with high communication demands is a crucial element of thevertically integrated approach of VIOLA. These demands include guaranteed high bandwidthas well as quality-of-service requirements between dynamically changing communicationpartners. In the first phase, four application projects covering visualisation and Grid-Computing in several scientific disciplines have been selected for funding. An additionalinfrastructure project is responsible for setting up an UNICORE-based Compute-Grid for theapplications and to develop and integrate additional middleware components, which areneeded by the applications but are not yet available. This presentation gives an overview ofgoals of these projects and the results achieved so far.
Page 127.04.2005, Th.Eickermann, NIC , Research Centre Jülich
Overview about applicationprojects in VIOLA
April 27th, 2005
Thomas EickermannJohn von Neumann Institute for Computing
Research Centre Jü[email protected]
Page 228.04.2005, Th.Eickermann, NIC , Research Centre Jülich
Content
– Introduction
– Application projects– VIOLA-Support– MetaTrace– TechSim– AMG-OPT– KoDaVis
– Status and Outlook
– Offline Demo
Page 328.04.2005, Th.Eickermann, NIC , Research Centre Jülich
Application Projects: Partners & Contributors
– NIC, Research Centre JülichU.Detert, T.Düssel, M.-A.Hermanns, W.Frings, L.Kirtchakova, M.Rambadt, K.Stollwerk,H.Zilken, …
– Fraunhofer Institute for Media CommunicationS.Conrad, …
– Fraunhofer Institute for Algorithms and Scientific ComputingH.Grund, A.Krechel, A.Leventi-Peetz, H.Schwichtenberg,S.Springstubbe, B.Steckel,O.Waeldrich, G.Winter, K.Wolf, W.Ziegler, …
– Research Centre caesarR.Backofen, N.Botkin, K.B.Duong, A.Goepfert, O.Pykhteev, V.Turova, S.Vey, A.Voigt,A.Zollorsch, …
– Fachhochschule Bonn-Rhein-SiegR.Berrendorf, J.Seidel, …
– Technical University of Aachen (RWTH)I.Assenmacher, T.Kuhlen, [N.Berr, B.ierbaum, R.Finocchiaro, K.Klaus, M.Poeppe, …]
Page 428.04.2005, Th.Eickermann, NIC , Research Centre Jülich
Project structure: Vertical Integration
optical fiber infrastructure
Network components (SDH/Ethernet Switches/Router)
Control Plane(Bandwidth and QoS reservation, user-network-interface)
Grid-Middleware and -Services
(Secure access, Scheduling, Communication, I/O, Tools)
Environmental Sciences:
Simulation / Visualization
Material Science & Engineering:
Technical Simulations
N
etw
orks
Mid
dlew
are
App
licat
ions
Dis
sem
inat
ion
/ Co-
oper
atio
n
Page 528.04.2005, Th.Eickermann, NIC , Research Centre Jülich
VIOLA Applications - Rationale
– Networker's perspective
From laboratory experiment to deployment in NREN:– Evaluation of network technology in close-to-production
environment– Involve users from leading scientific institutions in the local region– Applications with a broad spectrum of requirements
– Users perspective:
Early access to next-generation technology may be competitiveadvantage:
– Deployment in X-WiN in 2006+– Evaluation of Grid-System by means of applications
Page 628.04.2005, Th.Eickermann, NIC , Research Centre Jülich
Application Areas & Ressources
– Complex distributed scientific simulations– Bundling power of PC-cluster at partner sites:
CPUs, memory, I/O capabilities– Load sharing: use an appropriate free resource
– Visualization– Access to huge data repositories in the network– Control and steer running computer simulations– Collaborate in distributed teams
RWTH Aachen
Forschungszentrum Jülich
FH Bonn-Rhein-SiegSankt Augustin
Universität Bonn
caesarBonn
Fraunhofer Gesell.Sankt Augustin
– 4 Intel/AMD Linux-Clusters:– FZ Jülich– Caesar– Fraunhofer SCAI– FH Bonn-Rhein-Sieg
– High-End Visualisation / VR:– RWTH Aachen– FZ Jülich– Fraunhofer IMK
Page 728.04.2005, Th.Eickermann, NIC , Research Centre Jülich
VIOLA-Support: Provisioning of Infrastructure
optical fiber infrastructure
Network components (SDH/Ethernet Switches/Router)
Control Plane(Bandwidth and QoS reservation, user-network-interface)
Grid-Middleware and -Services
(Secure access, Scheduling, Communication, I/O, Tools)
Environmental Sciences:
Simulation / Visualization
Material Science & Engineering:
Technical Simulations
N
etw
orks
Mid
dlew
are
App
licat
ions
Dis
sem
inat
ion
/ Co-
oper
atio
n
Page 828.04.2005, Th.Eickermann, NIC , Research Centre Jülich
VIOLA-Support:Application Support and Grid-Integration
– Partners: FZJ/ZAM, FhG/SCAI, FH BRS, caesar
– Task:provide a production-quality environment for the users– Secure and seamless access to the parallel computers of the project
partners– Allow coordinated reservations of CPUs, I/O-Servers and network
bandwidth– Leverage network bandwidth for applications via standard APIs (MPI)– Transparent access to data stored anywhere in the testbed
Page 928.04.2005, Th.Eickermann, NIC , Research Centre Jülich
VIOLA-Support:Application Support and Grid-Integration
– Actions:– Deploy a UNICORE-Grid at the partner sites– develop a meta-scheduler based on Global Grid Forum standards,
capable of managing compute and network resources– port, enhance MetaMPICH (RWTH) and integrate it into UNICORE– Adapt and integrate the coupling-tool MpCCI into the VIOLA-
environment (UNICORE) [in sub-project TechSim]– extend MetaMPICH by a high-performance parallel-I/O-component
(MPI-IO) for distributed remote file-servers and memory
– Prototypes of all components are available– As required by additional applications:
attach additional computers to the testbed– JUMP (FZJ‘s 9 TeraFLOPS IBM Supercomputer)– SUN SMPs (SCAI‘s Cluster)
Page 1028.04.2005, Th.Eickermann, NIC , Research Centre Jülich
VIOLA-Applications: Using the Infrastructure
optical fiber infrastructure
Network components (SDH/Ethernet Switches/Router)
Control Plane(Bandwidth and QoS reservation, user-network-interface)
Grid-Middleware and -Services
(Secure access, Scheduling, Communication, I/O, Tools)
Environmental Sciences:
Simulation / Visualization
Material Science & Engineering:
Technical Simulations
N
etw
orks
Mid
dlew
are
App
licat
ions
Dis
sem
inat
ion
/ Co-
oper
atio
n
Page 1128.04.2005, Th.Eickermann, NIC , Research Centre Jülich
MetaTrace: Simulation of pollutant transportin groundwater with distributed SMP-Clusters
– Partners: FZJ/ZAM, FH BRS– Environmental research in Jülich: distribution of chemicals in the
soil– experiments in a test field– simulations on parallel computers
– Optimal distribution of work to several PC-clusters:– calculation of water-flow– distribution and chemical reactions of pollutants– exchange of intermediate results: 1 GigaByte in 1 second
– Goal:simulate moreaccurate modelsin less time
Page 1228.04.2005, Th.Eickermann, NIC , Research Centre Jülich
AMG-OPT: Grid-enabled technologyto accelerate simulations
– Partners: FhG/SCAI, FZJ/ZAM– Development of algorithms at the Fraunhofer-Institut SCAI:
– optimal numerical solvers for special large linear systems ofequations
– e.g. equations with 100.000.000 unknowns in simulations ofmechanical structures or liquids
– A stress-test for the VIOLA-network:– bandwidth and latency are just sufficient
to solve such equations efficientlydistributed over a network
– Goal:– SAMGP-Software as a building block
for simulation-programs in the Grid
Page 1328.04.2005, Th.Eickermann, NIC , Research Centre Jülich
TechSim: Distributed Simulationof complex technological Systems
– Partners: caesar, FhG/SCAI– “Different kinds of physics” often interact in technological systems:
– makes modeling and simulation difficult,– is well suited for distribution to several computers
– Two forward-looking applications from caesar in VIOLA
simulation of crystal growth of 300mm silicon-wafers for microchip-production : process-optimization
(uses coupling software MpCCI from SCAI)
biosensors (liquid film on piezo-crystal)detect smallest amounts of organic molecules:3D-simulations should lead to improvements
Page 1428.04.2005, Th.Eickermann, NIC , Research Centre Jülich
KoDaVis:Making Atmospheric Processes visible
– Partners: FZJ/ZAM, RWTH Aachen, FhG/IMK, FH BRS– Joint project from FZ Jülich (ICG) and MPI Hamburg
– simulations of physical and chemical processes in the atmospherehelp to understand the effect of human activities on the climate: datasets ~ 1000 GigaByte
– visualization of data needed to gain insight into processes
– VIOLA:– high bandwidth allows for
interactive access to the dataover the network
– short signal delays enablecollaboration in distributedteams:
– common view of the data– video-conference support
– Demo at the end of the presentation !
Page 1528.04.2005, Th.Eickermann, NIC , Research Centre Jülich
Application Requirements
MiddlewareHardwareBandwidthLatency
Application
ViSTAAVS/ExpressVISIT
1 Cluster
VR / Visualisation
8 Gbit/s25 msec
QoS (Video)
KoDaVis
UNICOREMetaMPI
2 Clusters8 Gbit/s< 1 ms
AMG-OPT
UNICOREMetaMPIMpCCI
2-3 Clusters2 … 8 Gbit/sTechSim
UNICOREMetaMPIVISIT
2-5 Clusters8 Gbit/s(< 0.1 msec)
MetaTrace
Page 1628.04.2005, Th.Eickermann, NIC , Research Centre Jülich
Status & Current work
– Initial version of middleware is deployed
– Integration of new middleware components in progress
– Application projects have finished preparatory workin time - until network testbed had been set up
– First “runs” over the network have been madein line with project plan
Applications are beginning to stress network & middlewareStability and functionality of all components are rapidly improving
– Next steps:– Functional and performance improvements of middleware and application– Integration of new hardware (Cray XD1, Sun Cluster)– Inclusion of additional applications & partners (Erlangen, Garching, …)
Page 1728.04.2005, Th.Eickermann, NIC , Research Centre Jülich
KoDaVis – Use Case Scenario
DataBroker
Visualisation Client 1Linux-Workstation
AVS/Express
Visualisation Client 2Powerwall
ViSTA
InteractionServer
File Server A(parallel, caching)
Data Sets: a,b,c, …
File Server X(parallel, caching)
Data Sets: x,y,z, …
Page 1828.04.2005, Th.Eickermann, NIC , Research Centre Jülich
KoDaVis – Demo Setup
DataBroker
Visualisation Client 2Powerwall
ViSTA
InteractionServer
File Server A(parallel, caching)
Data Sets: a,b,c, …
File Server X(parallel, caching)
Data Sets: x,y,z, …
Visualisation Client 1Linux-Workstation
AVS/Express
1 Gbit/s Ethernet