nbcr science gateway: transparent access to remote resources through rich desktop clients nbcr...
TRANSCRIPT
![Page 1: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/1.jpg)
NBCR Science Gateway: Transparent Access to Remote NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop ClientsResources Through Rich Desktop Clients
Luca [email protected]
![Page 2: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/2.jpg)
OutlineOutline
• NBCR/PRAGMA
• NBCR SOA infrastructure– Opal Overview – Opal Toolkit GUI– Opal Integration into Existing Tools– Access to Multiple Resources (TG, PRAGMA)
![Page 3: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/3.jpg)
National Biomedical Computational ResourceNational Biomedical Computational Resource
• Funded by NIH (NCRR)
• Its mission is to conduct, catalyze, and enable biomedical research by harnessing, developing, and deploying forefront computational, information and grid technologies – Develop and deploy advanced
computational, data and visualization tools.– Integrate them in an advanced grid
framework.– Deliver and support this grid/Cyber-
infrastructure– Training (Summer Institute)
![Page 4: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/4.jpg)
• Funded by NIH (NCRR)
• Its mission is to conduct, catalyze, and enable biomedical research by harnessing, developing, and deploying forefront computational, information and grid technologies – Develop and deploy advanced
computational, data and visualization tools.– Integrate them in an advanced grid
framework.– Deliver and support this grid/Cyber-
infrastructure– Training (Summer Institute)
National Biomedical Computational ResourceNational Biomedical Computational Resource
![Page 5: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/5.jpg)
In the Pacific Rim and Globally 29 institutions
Strengthen Existing and Establish New Sustainable Collaborations
Work with Science Teams to Advance Grid Technologies and
Improve the Underlying Infrastructure
Pacific Rim Application and Grid Middleware Assembly (PRAGMA)
http://www.pragma-grid.net
A Practical Collaborative Framework
•Collaborations•Integration of existing applications•Deployment of a Grid infrastructure
![Page 6: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/6.jpg)
MotivationMotivation
• Integration of different tools (client side application, workflow tools, web portals, etc.)– Several user interfaces
• Seamless access to computational resources– Easy from the scientific user perspective
![Page 7: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/7.jpg)
Condor pool SGE Cluster PBS Cluster
Globus Globus Globus
Application Services Security Services (GAMA)
StateMgmt
Web server PMV/Vision Kepler
Big PictureBig Picture
![Page 8: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/8.jpg)
Opal ToolkitOpal Toolkit
• Opal allows application developers to publish command-line applications using Web services– Minimal deployment effort: no coding, only a simple
configuration file – plain web service, SOAP– Common interface: every application uses the same
WSDL (same stubs on the client side)– Simple APIs (Launch job, Query job status, Get job
outputs, Get output as Base64, Destroy job, etc.)
![Page 9: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/9.jpg)
Opal ToolkitOpal Toolkit
– Every application exposed with its own URL http://ws.nbcr.net/opal/services/Pdb2pqrService
– It takes care of data staging– It supports submission via:
• Fork• Globus GRAM• DRMAA• CSF4 (experimental)
– Run all the applications under a single user account– Used in NBCR, CAMERA, GLEON, among other
![Page 10: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/10.jpg)
Opal Toolkit / Client SideOpal Toolkit / Client Side
• Several clients APIs available: Java, Python, PERL.
• Command line generic client:
# java edu.sdsc.nbcr.opal.GenericServiceClient -l
http://localhost:8080/axis/services/PDB2PQRServicePort -r launchJob -a "-ipdb sample.pdb -h -opdb output.pdb“ -f etc/sample.pdb
• Too complex for beginner users • Graphical User Interfaces
![Page 11: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/11.jpg)
getAppMetadata
launchJob
getOutput…
PDB2PQR
getAppMetadata
launchJob
getOutput…
MEME
…Opal server
User
Vision
Kepler
User clients
Browser
![Page 12: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/12.jpg)
Opal GUIOpal GUI
• Main characteristics:– Working out of the box (no configuration)– Multiplatform -> Web interface– Shipped with Opal
• Key features:– Access Statistics – List of services– Simple submission form– Advanced submission form
![Page 13: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/13.jpg)
Access StatisticsAccess Statistics
Go to http://ws.nbcr.net
NBCR production server
![Page 14: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/14.jpg)
List of ServicesList of Services
![Page 15: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/15.jpg)
User has to input command line!
Too error prone.
Submission form should be customized on command line arguments
Simple Submission FormSimple Submission Form
![Page 16: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/16.jpg)
Advanced Submission FormAdvanced Submission Form
• When deploying a scientific application there is an optional part to describe input and output of the application
• It is a command line syntax description language
![Page 17: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/17.jpg)
Group 1
Group 2
Group 3
Group 4
Exclusive group
Input file
String
Exclusive enumeration
Flag
PDB2PQR Advanced Submission FormPDB2PQR Advanced Submission Form
![Page 18: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/18.jpg)
Vision Opal NodeVision Opal Node
![Page 19: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/19.jpg)
Kepler Opal Web Services ActorKepler Opal Web Services Actor
![Page 20: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/20.jpg)
Autodock ToolsAutodock Tools
![Page 21: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/21.jpg)
PBD2PQR Web SitePBD2PQR Web Site
![Page 22: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/22.jpg)
Access to Multiple ResourcesAccess to Multiple Resources
![Page 23: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/23.jpg)
CSF4CSF4
• Community Scheduler Framework:
– Open Source project and can be accessed at http://sourceforge.net/projects/gcsf
– Developed by Lab. of Distributed Computing and System Architecture, Jilin University, China
– It is a metascheduler framework hosted as an Execution Component in GT4 container
– It uses WSRF compliant services– It can submit jobs to Globus
![Page 24: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/24.jpg)
CSF4 Meta-Scheduler
Grid Site GT2
LSF
Grid Site GT2
PBS
Grid Site GT4
SGE
Grid Site GT2
Condor……
•GT2 (gatekeeper)
•GT4 (WS-GRAM)
CSF4 Typical DeploymentCSF4 Typical Deployment
![Page 25: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/25.jpg)
CSF4 New FeatureCSF4 New Feature
Users want to run applications
• Application based scheduling:
– CSF4 keeps a table of available applications
Application name Resources
namd tg1:/usr/local/namd
namd tg2:/usr/share/bin/namd
autodock pragma1:/usr/local/autodock
autodock pragma2:/usr/local/autodock
![Page 26: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/26.jpg)
CSF4 New FeatureCSF4 New Feature
Users want to run applications
• Application based scheduling:
– CSF4 keeps a table of available applications
Application name Resources
namd tg1:/usr/local/namd
namd tg2:/usr/share/bin/namd
autodock pragma1:/usr/local/autodock
autodock pragma2:/usr/local/autodock
Avian Flu Grid
![Page 27: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/27.jpg)
Opal CSF4 IntegrationOpal CSF4 Integration
• Opal plug-in for submission to CSF4
• Deployment architecture:
![Page 28: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/28.jpg)
Opal Enable Clients
App A App B App C
Opal Server
App A App B App C
CSF4
ClusterAappAappB
Globus GRAM
…
Globus GRAM
ClusterBappAappC
WS/SOAP
WSRF
(Pre/WS)-GRAM
![Page 29: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/29.jpg)
Future DirectionsFuture Directions
• Opal 2.0 will be released soon – Improved architecture (plug-in model)– URL support for input and output files– Embedded database (Hibernate)
• Support for other metascheduler (Gridway)• Opal GUI for Gridsphere (Portlet)
![Page 30: NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop Clients NBCR Science Gateway: Transparent Access to Remote Resources](https://reader030.vdocuments.site/reader030/viewer/2022032517/56649cb05503460f94973fc7/html5/thumbnails/30.jpg)
AcknowledgmentsAcknowledgments
• Wilfred Li, Peter Arzberg (NBCR vision)• Sriram Krishnan (Opal)• Xiaohui Wei (CFS4)• Many other NBCR developers...
• More information, downloads, documentation:– http://nbcr.net – http://nbcr.net/software/opal/