from p-grade to sci-bus peter kacsuk, zoltan farkas and miklos kozlovszky mta sztaki - computer and...
TRANSCRIPT
From P-GRADE to SCI-BUS
Peter Kacsuk, Zoltan Farkas and Miklos Kozlovszky
MTA SZTAKI - Computer and Automation Research Institute of the Hungarian Academy of Sciences
www.lpds.sztaki.hu
2
Content
1. P-GRADE2. WS-PGRADE & gUSE in a nutshell3. Basic and new services4. Case studies5. SCI-BUS6. How to access
3
P-GRADE Portals around the globe
• National Grid portals (UK, Ireland, Croatia, Turkey, Spain, Belgium, Malaysia , Kazakhstan, Switzerland, Australia, ClGrid, etc.)
• HP-SEE/SEE-GRID-SCI, BalticGrid• US Open Science Grid, TeraGrid• IDGF (International Desktop Grid Federation)• SHIWA Simulation Platform• Economy-Grid, Swiss BioGrid, Bio és
Biomed Virtual Organisations, MathGrid, etc.
HP-SEE
4
P-GRADE Portal software family
P-GRADE Portal2.4
NGS P-GRADE portal
P-GRADE Portal2.5
P-GRADE Portal2.8
P-GRADE Portal2.9.1
WS-PGRADE Portal 3.1
WS-PGRADE Portal 3.2
GEMLCAGrid Legacy Code Arch.
GEMLCA, storage concepts
Basic concepts
2008
2009
2010
WS-PGRADE Portal 3.3
P-GRADE Portal2.10
Open source from 2008
stats:
2011
5
Motivations of creating gUSE
•To overcome (most of) the limitations of P-GRADE portal:
•To provide better modularity to replace any service•To improve scalability to millions of jobs•To enable advanced dataflow patterns•To interface with wider range of resources•To separate Application Developer view from Application User view
WS-PGRADE (Web Services Parallel Grid Runtime and Developer Environment)
and gUSE (Grid User Support Environment) architecture
6
gUSE and WS-PGRADE architecture
Graphical User Interface: WS-PGRADEGraphical User Interface: WS-PGRADE
WorkflowEngine
WorkflowEngine
Workflowstorage
Workflowstorage File
storage
Filestorage
Applicationrepository
Applicationrepository
LoggingLogging
gUSEinformation
system
gUSEinformation
system
SubmittersSubmitters
Gridsphere LIFERAY Large set of portlets
Autonomous Services: high level
middleware service layer
Resources: middleware service layer
Local resources, service grid VOs, Desktop Grid resources, Web services, Databases
Local resources, service grid VOs, Desktop Grid resources, Web services, Databases
gUSE
Meta-brokerMeta-broker SubmittersSubmittersSubmittersSubmitters
Filestorage
Filestorage
SubmittersSubmitters
7
WS-PGRADE/gUSE
•Creating complex workflow and parameter sweeps•Seamless access to various types of resources
•clusters, •service grids, •desktop grids, •databases.
•Scalable architecture•Advanced data-flows and workflow concept•Creating complex applications using embedded workflows, legacy codes•Comfort features
•Separated views for application developers and end-users•Community components from workflow repository
www.wspgrade.hu
8
WS-PGRADE in a nutshell
•General purpose, workflow-oriented portal. Supports the development and execution of workflow-based applications•Based on Liferay•Services supported by the portal:•New functionalities
•Web services•DB connectors•Embedded workflows•Job level PS•Conditional jobs•Recursive graph•Multi-generator•Multi-collector•CROSS product PS•DOT product PS
Simultaneous use of various clusters, service and desktop Grids at workflow level
Basic Services EGEE grids
(LCG2,GLite)
Globus grids
(GT2,GT4)
Desktop
grids
clusters
Job execution
File storage
Certificate management
Information system
Brokering
Job monitoring
Workflow & job visualization
9
WS-PGRADE in European and national projects•CancerGrid (EU FP6)•ProSim (UK)•EDGeS and EDGI (EU FP7)•MosGrid (D-Grid)•Swiss Proteomics Portal•HP-SEE (EU FP7)•SHIWA (EU FP7)•SCI-BUS (EU FP7)
10
The CancerGrid portal for anti-cancer drug research
Workflow development & configuration
Algorithms configuration
Workflow execution
Molecule database browser
Integrated components of CancerGrid portal
Structure viewer
11
Creating multi-workflow applications and executing them in the multi-middleware SHIWA VO
SHIWA Simulation Platform v1 (P-GRADE) and v2 (WS-PGRADE)
SHIWA RepositorySHIWA Portal
WF1
e-scientist
SHIWA Science Gateway
invoke WE
GEMLCA Service
WFn
WE1 WEp
search WF
GEMLCA Repositoryedit WF
WE + WF
WF1 WFm
GEMLCA with GIB
s1
s2
s3
s5
s2
s4
WF list
PGRADEWorkflow
engine
PGRADE Workflow
editor
gLite DCI
MOTEUR WE
GWES WE
Globus DCI
pre-deployed-WEs
MOTEUR WE
Kepler WE
Taverna WE
Triana WE
local cluster
ASKALON WE
SHIWA VO
ASKALON WE
GEMLCA admin
12
WS-PGRADE and SCI-BUS
•P-GRADE and WS-PGRADE has been developed in the framework of EU and Hungarian projects•This tradition will be followed in SCI-BUS•The major goals:•To further develop WS-PGRADE/gUSE•To develop customization methodology•To create production gateways both
•Generic-purpose gateways for NGIs•Customized gateways for at least 17 user communities
14
Portal Access
• Portal related information•www.wspgrade.hu
•Access to multi-grid environment•WS-PGRADE/gUSE portal •https://guse.sztaki.hu/liferay-portal-6.0.5/
•For application porting•GASUC (Grid Appl. Support Centre) success stories•http://www.lpds.sztaki.hu/gasuc/
•Interoperable workflow and DCI support •SHIWA Simulation platform
•http://www.shiwa-workflow.eu
GASUC
