application projects in viola - dfn · application projects in viola the integration of...

Application Projects in VIOLA

The integration of applications with high communication demands is a crucial element of thevertically integrated approach of VIOLA. These demands include guaranteed high bandwidthas well as quality-of-service requirements between dynamically changing communicationpartners. In the first phase, four application projects covering visualisation and Grid-Computing in several scientific disciplines have been selected for funding. An additionalinfrastructure project is responsible for setting up an UNICORE-based Compute-Grid for theapplications and to develop and integrate additional middleware components, which areneeded by the applications but are not yet available. This presentation gives an overview ofgoals of these projects and the results achieved so far.

04.2005, Th.Eickermann, NIC , Research Centre Jülich

Overview about applicationprojects in VIOLA

April 27th, 2005

Thomas EickermannJohn von Neumann Institute for Computing

Research Centre Jü[email protected]


Content

– Introduction

– Application projects– VIOLA-Support– MetaTrace– TechSim– AMG-OPT– KoDaVis

– Status and Outlook

– Offline Demo


Application Projects: Partners & Contributors

– NIC, Research Centre JülichU.Detert, T.Düssel, M.-A.Hermanns, W.Frings, L.Kirtchakova, M.Rambadt, K.Stollwerk,H.Zilken, …

– Fraunhofer Institute for Media CommunicationS.Conrad, …

– Fraunhofer Institute for Algorithms and Scientific ComputingH.Grund, A.Krechel, A.Leventi-Peetz, H.Schwichtenberg,S.Springstubbe, B.Steckel,O.Waeldrich, G.Winter, K.Wolf, W.Ziegler, …

– Research Centre caesarR.Backofen, N.Botkin, K.B.Duong, A.Goepfert, O.Pykhteev, V.Turova, S.Vey, A.Voigt,A.Zollorsch, …

– Fachhochschule Bonn-Rhein-SiegR.Berrendorf, J.Seidel, …

– Technical University of Aachen (RWTH)I.Assenmacher, T.Kuhlen, [N.Berr, B.ierbaum, R.Finocchiaro, K.Klaus, M.Poeppe, …]


Project structure: Vertical Integration

optical fiber infrastructure

Network components (SDH/Ethernet Switches/Router)

Control Plane(Bandwidth and QoS reservation, user-network-interface)

Grid-Middleware and -Services

(Secure access, Scheduling, Communication, I/O, Tools)

Environmental Sciences:

Simulation / Visualization

Material Science & Engineering:

Technical Simulations

N

etw

orks

Mid

dlew

are

App

licat

ions

Dis

sem

inat

ion

/ Co-

oper

atio

n


VIOLA Applications - Rationale

– Networker's perspective

From laboratory experiment to deployment in NREN:– Evaluation of network technology in close-to-production

environment– Involve users from leading scientific institutions in the local region– Applications with a broad spectrum of requirements

– Users perspective:

Early access to next-generation technology may be competitiveadvantage:

– Deployment in X-WiN in 2006+– Evaluation of Grid-System by means of applications


Application Areas & Ressources

– Complex distributed scientific simulations– Bundling power of PC-cluster at partner sites:

CPUs, memory, I/O capabilities– Load sharing: use an appropriate free resource

– Visualization– Access to huge data repositories in the network– Control and steer running computer simulations– Collaborate in distributed teams

RWTH Aachen

Forschungszentrum Jülich

FH Bonn-Rhein-SiegSankt Augustin

Universität Bonn

caesarBonn

Fraunhofer Gesell.Sankt Augustin

– 4 Intel/AMD Linux-Clusters:– FZ Jülich– Caesar– Fraunhofer SCAI– FH Bonn-Rhein-Sieg

– High-End Visualisation / VR:– RWTH Aachen– FZ Jülich– Fraunhofer IMK


VIOLA-Support: Provisioning of Infrastructure










N

etw

orks

Mid

dlew

are

App

licat

ions

Dis

sem

inat

ion

/ Co-

oper

atio

n


VIOLA-Support:Application Support and Grid-Integration

– Partners: FZJ/ZAM, FhG/SCAI, FH BRS, caesar

– Task:provide a production-quality environment for the users– Secure and seamless access to the parallel computers of the project

partners– Allow coordinated reservations of CPUs, I/O-Servers and network

bandwidth– Leverage network bandwidth for applications via standard APIs (MPI)– Transparent access to data stored anywhere in the testbed


VIOLA-Support:Application Support and Grid-Integration

– Actions:– Deploy a UNICORE-Grid at the partner sites– develop a meta-scheduler based on Global Grid Forum standards,

capable of managing compute and network resources– port, enhance MetaMPICH (RWTH) and integrate it into UNICORE– Adapt and integrate the coupling-tool MpCCI into the VIOLA-

environment (UNICORE) [in sub-project TechSim]– extend MetaMPICH by a high-performance parallel-I/O-component

(MPI-IO) for distributed remote file-servers and memory

– Prototypes of all components are available– As required by additional applications:

attach additional computers to the testbed– JUMP (FZJ‘s 9 TeraFLOPS IBM Supercomputer)– SUN SMPs (SCAI‘s Cluster)


VIOLA-Applications: Using the Infrastructure










N

etw

orks

Mid

dlew

are

App

licat

ions

Dis

sem

inat

ion

/ Co-

oper

atio

n


MetaTrace: Simulation of pollutant transportin groundwater with distributed SMP-Clusters

– Partners: FZJ/ZAM, FH BRS– Environmental research in Jülich: distribution of chemicals in the

soil– experiments in a test field– simulations on parallel computers

– Optimal distribution of work to several PC-clusters:– calculation of water-flow– distribution and chemical reactions of pollutants– exchange of intermediate results: 1 GigaByte in 1 second

– Goal:simulate moreaccurate modelsin less time


AMG-OPT: Grid-enabled technologyto accelerate simulations

– Partners: FhG/SCAI, FZJ/ZAM– Development of algorithms at the Fraunhofer-Institut SCAI:

– optimal numerical solvers for special large linear systems ofequations

– e.g. equations with 100.000.000 unknowns in simulations ofmechanical structures or liquids

– A stress-test for the VIOLA-network:– bandwidth and latency are just sufficient

to solve such equations efficientlydistributed over a network

– Goal:– SAMGP-Software as a building block

for simulation-programs in the Grid


TechSim: Distributed Simulationof complex technological Systems

– Partners: caesar, FhG/SCAI– “Different kinds of physics” often interact in technological systems:

– makes modeling and simulation difficult,– is well suited for distribution to several computers

– Two forward-looking applications from caesar in VIOLA

simulation of crystal growth of 300mm silicon-wafers for microchip-production : process-optimization

(uses coupling software MpCCI from SCAI)

biosensors (liquid film on piezo-crystal)detect smallest amounts of organic molecules:3D-simulations should lead to improvements


KoDaVis:Making Atmospheric Processes visible

– Partners: FZJ/ZAM, RWTH Aachen, FhG/IMK, FH BRS– Joint project from FZ Jülich (ICG) and MPI Hamburg

– simulations of physical and chemical processes in the atmospherehelp to understand the effect of human activities on the climate: datasets ~ 1000 GigaByte

– visualization of data needed to gain insight into processes

– VIOLA:– high bandwidth allows for

interactive access to the dataover the network

– short signal delays enablecollaboration in distributedteams:

– common view of the data– video-conference support

– Demo at the end of the presentation !


Application Requirements

MiddlewareHardwareBandwidthLatency

Application

ViSTAAVS/ExpressVISIT

1 Cluster

VR / Visualisation

8 Gbit/s25 msec

QoS (Video)

KoDaVis

UNICOREMetaMPI

2 Clusters8 Gbit/s< 1 ms

AMG-OPT

UNICOREMetaMPIMpCCI

2-3 Clusters2 … 8 Gbit/sTechSim

UNICOREMetaMPIVISIT

2-5 Clusters8 Gbit/s(< 0.1 msec)

MetaTrace


Status & Current work

– Initial version of middleware is deployed

– Integration of new middleware components in progress

– Application projects have finished preparatory workin time - until network testbed had been set up

– First “runs” over the network have been madein line with project plan

Applications are beginning to stress network & middlewareStability and functionality of all components are rapidly improving

– Next steps:– Functional and performance improvements of middleware and application– Integration of new hardware (Cray XD1, Sun Cluster)– Inclusion of additional applications & partners (Erlangen, Garching, …)


KoDaVis – Use Case Scenario

DataBroker

Visualisation Client 1Linux-Workstation

AVS/Express

Visualisation Client 2Powerwall

ViSTA

InteractionServer

File Server A(parallel, caching)

Data Sets: a,b,c, …

File Server X(parallel, caching)

Data Sets: x,y,z, …


KoDaVis – Demo Setup

DataBroker

Visualisation Client 2Powerwall

ViSTA

InteractionServer

File Server A(parallel, caching)

Data Sets: a,b,c, …

File Server X(parallel, caching)

Data Sets: x,y,z, …

Visualisation Client 1Linux-Workstation

AVS/Express

1 Gbit/s Ethernet

application projects in viola - dfn · application projects in viola the integration of...

Documents