grid for cbm kilian schwarz, gsi. what is grid ? ● sharing of distributed resources within one...
Post on 18-Dec-2015
218 views
TRANSCRIPT
![Page 1: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/1.jpg)
Grid for CBM
Kilian Schwarz, GSI
![Page 2: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/2.jpg)
What is Grid ?
● Sharing of distributed resources within one Virtual Organisations !!!!
![Page 3: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/3.jpg)
70
538
27
4603637
55
22 87
10
Europa: 267 Institute, 4603 UserSonstige: 208 Institute, 1632 User
LHC Wissenschaftler weltweit
![Page 4: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/4.jpg)
Start of CBM Grid
● There are considerations to start a CBM Grid● Task: distributed MC production● Potential sites: 3 (Bergen, Dubna, GSI)● After positive experiences the Grid can be
enlarged to more sites and tasks, like distributed analysis
![Page 5: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/5.jpg)
requirements* Globus-style X509 user certificates
issued for CBM by GermanGrid CA
http://www.gridka.de
* How to get a certificate ?
at GSI: > . globuslogin
> grid-cert-request –cn “<surname> <name>”
certificate request file and private key will be stored in $HOME/.globus
The request file has to be signed (openssl) by the CA responsible person and mailed to GermanGrid CA
The certificate will be mailed back via e-mail
![Page 6: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/6.jpg)
GermanGrid CA
How to get a certificate in detail:
See http://wiki.gsi.de/Grid/DigitalCertificates
![Page 7: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/7.jpg)
requirements: CBM VO Server (one per VO)
additional sites:
- Bergen, Dubna
additional users:
- to be added
![Page 8: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/8.jpg)
Globus/LCG – creation of grid-mapfilenecessary for each site
● E.g. with gLite-security tools:
- adjust $GLITE_LOCATION/etc/glite-mkgridmap.conf
add: “group ldap://glite001.gsi.de:8389/o=cbm,dc=de,dc=de”
- Create grid-mapfile
$GLITE_LOCATION/sbin/glite-mkgridmap –output=/etc/grid-security/grid-mapfile
![Page 9: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/9.jpg)
user creation on each site (support of CBM VO)
• Each site has to create cbm-user-IDs onto which the Grid-users will be mapped:
EGEE/LCG: a certain number of POOL accounts, e.g. cbmvo00 – cbmvo10
Globus & AliEn: one production user: via this userID the jobs will be submitted. E.g. cbmprod
![Page 10: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/10.jpg)
CBM software environment
● To be able to send real CBM jobs to the Grid, the participating sites have to
* Install the CBM software and prepare the environment
* Or the job has to bring it’s own environment (static links)
![Page 11: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/11.jpg)
Agreement on common Grid middleware
• basically, the possibilities are:• - Globus• - NorduGrid• - LCG-2• - AliEn• - gLite (EGEE)• - gLite (AliEn)
![Page 12: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/12.jpg)
LHC Computing Grid ProjectFundamental Goal of the LCG
To help the experiments’ computing projects
Phase 1 – 2002-05prepare and deploy the environment for LHC computing
Phase 2 – 2006-08acquire, build and operate the LHC computing service
SC2 – Software & Computing CommitteeSC2 includes the four experiments, Tier 1 Regional CentresSC2 identifies common solutions and sets requirements for the project
PEB – Project Execution BoardPEB manages the implementation
organising projects, work packagescoordinating between the Regional Centres
![Page 13: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/13.jpg)
EDG Middleware Architecture
Collective Services
Information &
Monitoring
Replica Manager
Grid Scheduler
Local Application Local Database
Underlying Grid Services
Computing Element Services
Authorization Authentication and Accounting
Replica Catalog
Storage Element Services
SQL Database Services
Fabric services
ConfigurationManagement
Node Installation &Management
Monitoringand
Fault Tolerance
Resource Management
Fabric StorageManagement
Grid
Fabric
Local Computing
Grid Grid Application Layer
Data Management
Job Management
Metadata Management
Service Index
APPLICATIONS
GLOBUSCondorG
(via VDT)
M / W
![Page 14: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/14.jpg)
Dubna (JINR): LCG-2 site
![Page 15: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/15.jpg)
Dubna (JINR): LCG-2 siteLCG-test mostly successful
![Page 16: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/16.jpg)
JINR (LCG-2 site: job-submit)
![Page 17: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/17.jpg)
Timeline
o After only 2 years of development, we have deployed a distributed computing environment which meets the needs of Alice experiment Simulation & Reconstruction Event mixing Analysis
o Using Open Source components (representing 99% of the code), internet standards (SOAP,XML, PKI…) and scripting language (perl) was the key element that alllowed quick prototyping and very fast development cycles
2001 2002 2003 2004 2005
First production (distributed simulation)
10% DC (analysis)
P. Buncic, CERN
![Page 18: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/18.jpg)
Building AliEn
SvOutPlaceObjectSvOutPlaceObject
SvOutPlaceObject
SvOutPlaceObject
P. Saiz, CERN
SvOutPlaceObject
SvOutPlaceObject
SvOutPlaceObject
SvOutPlaceObject
![Page 19: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/19.jpg)
AliEn Grid (ALICE VO):● 77 configured sites worldwide
![Page 20: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/20.jpg)
DC Monitoring: http://alien.cern.ch● Monalisa: http://aliens3.cern.ch:8080
![Page 21: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/21.jpg)
lxts05.gsi.de: AliEn client (PANDA VO)
![Page 22: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/22.jpg)
JINR and Bergen: AliEn sites
![Page 23: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/23.jpg)
JINR and Bergen: AliEn sites
![Page 24: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/24.jpg)
Grids and Open Standards
Incr
ease
d fu
nctio
nalit
y,st
anda
rdiz
atio
n
Time
Customsolutions
Open GridServices
Arch
GGF: OGSI, …(+ OASIS, W3C)
Multiple implementations,including Globus Toolkit
Web services
Globus Toolkit
Defacto standardsGGF: GridFTP, GSI
X.509,LDAP,FTP, …
App-specificServices
![Page 25: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/25.jpg)
Architecture Guiding Principles
● Lightweight (existing) services – Easily and quickly deployable– Use existing services where possible as
basis for re-engineering
● Interoperability– Allow for multiple implementations
● Resilience and Fault Tolerance
● Co-existence with deployed infrastructure– Run as an application (e.g. on LCG-2; Grid3)– Reduce requirements on site components
● Basically globus and SRM– Co-existence (and convergence) with LCG-2 and Grid3 are essential for the EGEE Grid
service
● Service oriented approach– WSRF still being standardized– No mature WSRF implementations exist to date, no clear picture about the impact of WSRF
hence: start with plain WS● WSRF compliance is not an immediate goal, but we follow the WSRF evolution● WS-I compliance is important
![Page 26: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/26.jpg)
Approach
● Exploit experience and components from existing projects
– AliEn, VDT, EDG, LCG, and others● Design team works out architecture
and design– Architecture: https://edms.cern.ch/document/476451– Design: https://edms.cern.ch/document/487871/
● Components are initially deployed on a prototype infrastructure– Small scale (CERN & Univ. Wisconsin)– Get user feedback on service semantics and interfaces
● After internal integration and testing components are delivered to SA1 and deployed on the pre-production service
EDGVDT . . .
LCG
EGEE
. . .AliEn
![Page 27: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/27.jpg)
gLite (AliEn)
* From now on used by ALICE for globally distributed analysis in connection with
• PROOF (at GSI: http://www-w2k.gsi.de/root/• PROOF at GSI )
![Page 28: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/28.jpg)
gLite (EGEE)
* Will replace LCG-2.X in near? future, but nobody has real experience with it
![Page 29: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/29.jpg)
summary (middlewares)● LCG-2: GSI and Dubna • - pro: large distribution, support• - contra: difficult to set up, no distributed analysis● AliEn: GSI, Dubna, Bergen
- pro: in production since 2001
- contra: unsecure future, no support
Globus 2: GSI, Dubna, Bergen?
- pro/contra: simple, but functioning (no RB, no FC, no support)
gLite/GT4: new on the market
- pro/contra: nobody has production experience (gLite)
![Page 30: Grid for CBM Kilian Schwarz, GSI. What is Grid ? ● Sharing of distributed resources within one Virtual Organisations !!!!](https://reader036.vdocuments.site/reader036/viewer/2022062421/56649d225503460f949f8d3f/html5/thumbnails/30.jpg)
lxg01-05.gsi.de● LCG test installation, visible in LCG –
preproduction testbed● Trying to port LCG to Debian Linux