using the vl-e proof of concept environment connecting users to the e-science infrastructure david...
TRANSCRIPT
Using the VL-EProof of Concept Environment
Connecting Users to the e-Science Infrastructure
David Groep, NIKHEF
Virtual Laboratory for e-Science (NL)
• To boost e-Science by – the creation of an e-Science environment – and doing research on methodologies
• To carry out concerted research – along the complete e-Science technology
chain, – ranging from applications to networking, – focused on new methodologies and reusable
components.
Grid ServicesHarness multi-domain distributed
resources
XXXXXXXXVL-e Application Oriented Services
Food Informatics
Dutch Telescience
Medical Diagnosis &
Imaging
Bio-Informatics
Data Intensive Science/
Bio-Diversity
Virtual Laboratory for e-Science
VL-E in a nutshell
• Experiments become more complex– more than just coping with the data– Computer is integrated part of the experiment– support the experimental process end-to-end
Technology (push)
…Grid
Resource SharingWeb
Networks
Application Needs (pull)
Experiment validationPapers and associated dataProvenance meta-dataInformation modelingData/Resource Collection Access…
interpretation
Rationalization of the experiment and processes via protocols
processingprocessed data
conversion, filtering,analyses, simulation, …experiment
parameters/settings,algorithms,
intermediate results,…
Parameter settings,Callibrations,
Protocols…
software packages,algorithms
…
raw dataacquisition
sensors,amplifiers imaging devices,, …
presentationvisualization, animationinteractive exploration, …
MetadataMuch of this is lost when an experiment is completed.
The Experimental Process
Combining data sources
Key element for all users: Data Combination• From different organisations
– data ownership preserved– data correctness maintained by preventing
‘forks’
• Extracting common meaning– need for workflow definition and
ontologies in collaborative experiments
Combining data in Cognition Science
• Collaborative scientific research– Information sharing– Metadata modeling
• Allows for experiment validation– Independent confirmation
of results• Statistical methodologies
– Access to large collections of data and metadata
• Training– Train the next generation
using peer reviewed publications and the associated data
Combining Acquisition and Simulation
• Robert: kun je hier een mooi plaatje voor maken? Het lijkt me de goede plaats om ook in-silico experimenten even te noemen
Role of the Proof-of-Concept (PoC)
• Platform for user application development• Provisioning network & grid infrastructure
– stable releases of common tools – tested ‘external’ middleware– stable releases of internal developments
• Support for users & dissemination– infrastructure installations– end-user helpdesk– on-site aid in migration
Application development
NL-Grid production clusterCentral mass-storage
facilities+SURFnet
Initial computeplatform
Stable, reliable, testedCert. releases Grid MW & VL-
software
VL-e Proof of Concept EnvironmentVL-e Rapid Prototyping Environment
DAS-2, local resources
VL-e Certification Environment
NL-Grid FabricResearch Cluster
Test & Cert.Grid MW & VL-software
Compatibility
Flexible, test environment
Environments
Usage
Characteristics
Virtual Lab.rapid prototyping
(interactive simulation)
Flexible, ‘unstable’
Download RepositoryPoC InstallerCluster Tools
Developer CVSNightly builds
Unit tests
stable, tested releases
Integration testsFunctionality testsAdventurous application people
PoC Release n
Release Candidate n+1
Developers Heaven/Haven
Tagged Release Candidates
LCG2.x + SRB + LCG2.x + others
GT3.2 + *
externalmiddleware
products
Food Informatics
Dutch Telescience
Medical Diagnosis &
Imaging
Bio-Informatics
Data Intensive Science/
Bio-Diversity
Involving Users
• Training via tutorials on middleware– good attendance, but slow uptake later on
• On-site support in integration– good technology update, but people intensive
• User driven integration: application pull– rapid update, good attendance– requires an ICT scientist to work long-term
with the domain scientists to recognize and extract generic elements
Tutorials
• Grid, LCG2 tutorials • Hands-on event series
‘Grid Admin Nerd Group’
‘After Sales Service’• Documentation• User help-desk (by phone & mail)
User Experience: nice, but information quickly ‘lost’
On-site support
• EMUTD exampleMaurice to provide image & input
• Effective use of EDG/EGEE tools for job submission, SRB for data access
User experience:problem effectively solved!but with high manpower investment by PoC
Grid ServicesHarness multi-domain distributed resources
Managementof comm. & computing
Managementof comm. & computing
Managementof comm. & computing
Potential Genericpart Potential
Genericpart
Potential Genericpart
ApplicationSpecific
Part
ApplicationSpecific
Part
ApplicationSpecific
Part
Virtual Laboratory Application Oriented Services
App
lica
tion
pul
lApplication Pull VL-E methodology
Can we keep our users content?
• Take care of grid & generic aspects– collaboration community building & security– policy-constraint & dynamic resource sharing
• Software Integration– there are many tools already … ‘just integrate them’– but only wide deployment will show the weaknesses
• Make it work– consistent software engineering practices– hide changes lower layers by use of standard interfaces– Easy-to-use installers (PoC Installer, Quattor)– and teach us how to scale up to a grid service provider
http://www.vl-e.nl/