august 2007 leveraging campus authentication to access the teragrid - or - partnering with campuses...
TRANSCRIPT
August 2007
Leveraging Campus Authentication to Access the TeraGrid
- OR -Partnering with Campuses to Broaden
Participation in TeraGrid
Scott Lathrop
TeraGrid Director Education, Outreach and Training
Tom Barton
University of Chicago
www.teragrid.org
August 2007
11 Resource Providers, One Facility
August 2007
TeraGrid Vision
TeraGrid will create integrated, persistent, and pioneering computational resources that will significantly improve our nation’s ability and capacity to gain new insights into our most challenging research questions and societal problems.
This vision requires an integrated approach to the scientific workflow including obtaining access, application development and execution, data analysis, collaboration and data management.
August 2007
TeraGrid Architectural Model
ComputeService
VizService
DataService
TeraGrid Infrastructure Network, Accounting, …
POPS
Help
August 2007
TeraGrid Resources• Computing - over 250 Tflops today and growing
– 500 Tflop system comes on-line in January at TACC– U Tennessee system to come on-line in 2008– Visualization - Remote visualization servers and software
• Data – Allocation of data storage facilities – Over 100 Scientific Data Collections
• Access– Over 20 Science Gateways– Shibboleth testbed to facilitate access– Central allocations mechanism
• Support and Services– Centralized help desk for all resource providers– Advanced Support for TeraGrid Applications (ASTA)– Education and training events and resources
August 2007
Account Management via TeraGrid User Portal
August 2007
Requesting Allocations of Time
• TeraGrid resources are provided for free to academic researchers and educators
• Development Allocations Committee (DAC) for start-up accounts up to 30,000 hours of time are requests processed in two weeks - start-up and courses
• Medium Resource Allocations Committee (MRAC) for requests of up to 500,000 hours of time are reviewed four times a year
• Large Resource Allocations Committee (LRAC) for requests of over 500,000 hours of time are reviewed twice a year
August 2007
25
50
75
100
125
150
175
200
225
250
275
J F MA MJ J A S O N D J F MA MJ J A S ON D J F MA MJ J A S O N D J F MA MJ
2004 2005 2006 2007
NUs (millions)
Specific
Roaming
TeraGrid Usage
33% Annual Growth
Specific Allocations Roaming Allocations
200
100
Normalized Units (millions)
TeraGrid currently delivers an average of 420,000 cpu-hours per day -> ~21,000 CPUs DC Dave Hart ([email protected])
August 2007
Science GatewaysBroadening Participation in TeraGrid
• Increasing investment by communities in their own cyberinfrastructure, but heterogeneous:
• Resources• Users – from expert to K-12• Software stacks, policies
• Science Gateways– Provide “TeraGrid Inside”
capabilities– Leverage community investment
• Three common forms:– Web-based Portals – Application programs running on
users' machines but accessing services in TeraGrid
– Coordinated access points enabling users to move seamlessly between TeraGrid and other grids.
Technical Approach
Biomedical and Biology, Building Biomedical Communities
OGCE Portletswith ContainerOGCE Portletswith Container
Apache JetspeedInternal ServicesApache JetspeedInternal Services
ServiceAPI
ServiceAPI
GridProtocols
GridServiceStubs
GridServiceStubs
RemoteContentServices
RemoteContentServices
RemoteContentServersHTTP
GridService
sLocalPortal
Services
LocalPortal
Services
Grid Resources
Open Source Tools
Build standard portals to meet the domain requirements of the biology communitiesDevelop federated databases to be replicated and shared across TeraGrid
Workflow Composer
Source: Dennis Gannon ([email protected])
August 2007
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
“HPC University”• Advance researchers’ HPC skills
– Catalog of live and self-paced training– Schedule series of training courses– Gap analysis of materials to drive development
• Work with educators to enhance the curriculum– Search catalog of HPC resources– Schedule workshops for curricular development– Leverage good work of others
• Offer Student Research Experiences– Enroll in HPC internship opportunities– Offer Student Competitions
• Publish Science and Education Impact– Publish transformative Science Highlights– Publish education resources to NSDL-CSERD
August 2007
CI Days• Working with campuses to take a leadership role applying
CI to accelerate scientific discovery• Assist in catalyzing campus-wide discussions and planning • Collaboration of Open Science Grid, Internet 2, National
Lamda Rail, EDUCAUSE, Minority Serving Institution Cyberinfrastructure Empowerment Coalition, TeraGrid, and local and regional organizations
http://cidays.org
August 2007
Campus Champions Program
• Training program for campus representatives• Campus advocate for TeraGrid and CI resources• TeraGrid ombudsman for local users• Quick start-up accounts managed by campus
representative• Direct contact with TeraGrid staff for quick problem
resolution
We’re looking for campuses interested in joining!
August 2007
ScienceGateway
Scaling the TeraGrid Community
ResourceProvider TGCDB
Grant Programs
uiduid
O(10) Gateways
O(10) Resource Providers
O(1000) PIs
O(10) Programs
O(10000) Users
project
August 2007
And now a few words from Tom….
August 2007
Q&A
• What are campuses doing to provide Shibboleth access to the desktops of the users?
• What are the needs of the user community?• How is the community benefiting from single sign-on
capabilities today?• Anticipating TG putting the TGUP and POPs online
as a Shibboleth SP, would campuses consider that a carrot that would help convince them to become IdPs?
• Are campuses in a position to provide persistent identifiers and contact information about their faculty and grad students via Shibboleth?
August 2007
For More Information
www.teragrid.org www.computationalscience.org www.s-education.org
www.nsdl.orgcserd.nsdl.org
www.nsf.gov/oci/http://cidays.org
August 2007
Account management
• Central process for getting/managing allocation– NSF Allocations process
• Central database keeps track of TeraGrid user accounts at all sites– no uid or username alignment across sites
• Also keeps track of User’s Grid Identities– X.509 DNs– Both TG-issued and from external CAs– Pushes out to all sites
• All users have a TG username and password– Exposed via Kerberos 5 domain and MyProxy online-CA
• TeraGrid User Portal
August 2007
TeraGrid Access
• Traditional interactive SSH login via Site authn• Grid (PKI) SSO SSH interactive login
– Short-lived PKI credentials issues via MyProxy and User’s TG username & password
– Hides site-specific identity details from user
• Grid Services– Globus job submission, GridFTP, etc.
• Science Gateways/Web Portals– Have own user databases– Tied to community accounts and allocations on TG sites– Give constrained, domain-specific interface
August 2007
Ultimate Id Federation Goals and Testbed
• Allow scaling of TeraGrid to O(10k)+ users• Get TeraGrid out of identity management game to
allow this• Leverage existing campus identity management• Allowing servicing of existing VO’s
– Attribute-based authorization
• Allow for incident response– Blocking and/or contacting problematic users
• Testbed to evaluate how Shibboleth, GridShib and other tools can achieve this– NCSA, Purdue
August 2007
Testbed Thrusts
• Three thrusts…• One: Java-based Grid-enabled SSH and MyProxy
client• Build on work from UK NGS
– http://www.grid-support.ac.uk/files/gsissh/
• Allow user to do Grid-based SSH SSO with no Grid client installation– Just vanilla Java– Using TeraGrid username and password
• This is working:– http://grid.ncsa.uiuc.edu/gsi-sshterm/
August 2007
Testbed Thrusts
• Two: Shibboleth-based TeraGrid Access• Using GridShib-CA to access existing TeraGrid account
– In Shibboleth terms, a Shibboleth SP that issues short-lived Grid credentials
• Allows user to connect to TeraGrid using their local campus authentication
• Integrated with Java GSI-SSH client to allow for zero-client install SSH access
• Currently doing bi-lateral Shibboleth peering– eventually InCommon– Requires ePPN from IdP
• Friendly user mode– One time registration of Shibboleth-based X.509 DN– http://gridshib-ca.ncsa.uiuc.edu/
August 2007
Testbed Thrusts
• Three: Attribute-based authorization from Science Gateways
• Allow Science Gateways to push VO attributes to TeraGrid sites
• Could be passed from user’s Idp or generated locally
• In development.
August 2007
Overview of TG Allocations Process
• Potential PI makes a proposal– Via Partnership Online Proposal System (POPS)– Can be for combination of compute, storage, and advanced
consulting (ASTA)
• Proposal is reviewed– Startup proposals (DACS) in real-time– Medium and Large by committees (MRAC, LRAC)
• Successful PI gets login on one or more resource provider sites
• TeraGrid User Portal provides means of administering allocation– http://portal.teragrid.org
• Details: http://www.ci-partnership.org/Allocations/
August 2007
How can Campuses help in this process?
August 2007
PI Requirements
• PI must be a researcher or educator at a U.S. academic or non-profit research institution–Students may not be PIs but can be added to PI’s allocation
August 2007
Creating a POPS Account…
August 2007
TeraGrid User Portal SSO
• TG User Portal is being integrated with back-end resources to provide single interface to resources
August 2007
What Does the Community Need?
• Do you have users currently using Shibboleth?• What are they using it for and what has been their
experience?• How can Shibboleth access to TeraGrid resources bedst
enhance their research and education efforts?
August 2007
Next Steps and Issues
• TeraGrid is applying for InCommon membership as a service provider–TeraGrid User Portal as Shibboleth SP
• Open issues:–Level of Assurance for PIs/users–Incident Response: responsibilities of campuses when something goes wrong
August 2007
TeraGrid User Community
Quarter Ending
Gateways
Dave Hart ([email protected])
Growth Target
August 2007
Use ModalityUse ModalityCommunity SizeCommunity Size
(est. number of (est. number of people/projects)people/projects)
Batch Computing on Individual Resources 850
Exploratory and Application Porting 650
Workflow, Ensemble, and Parameter Sweep 160
Science Gateway Access 100
Remote Interactive Steering and Visualization 35
Tightly-Coupled Distributed Computation 10
TeraGrid Usage Modes in CY2006
Grid
-y U
sers
August 2007
Coupled Simulation: Full Body Arterial Tree Simulation
Karniadakis (Brown)
Virtualized Resources, Ensembles:
FOAM Climate
Model
Liu (UWisc)
Sources: Ian Foster (UC/ANL), Mike Papka (UC/ANL), George Karniadakis (Brown). Images by UC/ANL.
Advanced Support for TeraGrid Applications
August 2007
TeraGrid Wide Initiatives (2007-9)
• Science Gateways–Completing first generation integrations–Tutorials, Documentation, Services–Develop “consulting” approach
• Software as Service/Service Oriented Architecture–Capability Kits and Service Directory–Investigate Service Hosting Capabilities/Need
• Operations–Improved Instrumentation, monitoring, testing
August 2007
TeraGrid Open Initiatives (2007-9)
• Campus Infrastructure Engagement–HPC University & Institutional Ambassadors–Client Software Kit/distribution–Followup on Shibboleth/inCommon testbed
• Open Science Grid Partnership (& EGEE)–Software stack alignment on Condor + Globus–Training/Education/Outreach
• Grid Interoperation Now (GIN)–Focus next on Information Services and joint use cases–Demand growing, but still tentative
• Commercial Service Provision–TG buys some internal project services now (e.g. Wiki,
surveymonkey)–Looking at Web, Mail, …
August 2007
TeraGrid Identity Federation Testbed Update
I2MMApril 25, 2007
VonWelch
NCSA/U. of Illinois
August 2007
TeraGrid Objectives
• DEEP Science: Enabling Petascale Science–Make Science More Productive through an integrated set of very-high capability resources
•Address key challenges prioritized by users
• WIDE Impact: Empowering Communities–Bring TeraGrid capabilities to the broad science community
•Partner with science community leaders - “Science Gateways”
• OPEN Infrastructure, OPEN Partnership–Provide a coordinated, general purpose, reliable set of services and resources
•Partner with campuses and facilities
August 2007
Gateways are Expanding• 10 initial projects as part of TG proposal• >20 Gateway projects today• No limit on how many gateways can use TG
resources– Prepare services and documentation so
developers can work independently
• Open Science Grid (OSG)• Special PRiority and Urgent Computing
Environment (SPRUCE)• National Virtual Observatory (NVO)• Linked Environments for Atmospheric
Discovery (LEAD)• Computational Chemistry Grid (GridChem)• Computational Science and Engineering
Online (CSE-Online)• GEON(GEOsciences Network)• Network for Earthquake Engineering
Simulation (NEES)• SCEC Earthworks Project• Network for Computational Nanotechnology
and nanoHUB• GIScience Gateway (GISolve)• Biology and Biomedicine Science Gateway• Open Life Sciences Gateway• The Telescience Project• Grid Analysis Environment (GAE)• Neutron Science Instrument Gateway• TeraGrid Visualization Gateway, ANL• BIRN• Gridblast Bioinformatics Gateway• Earth Systems Grid• Astrophysical Data Repository (Cornell)
August 2007
A Simple Use Case: TeraGrid Allocations Process
Von Welch
NCSA
August 2007
TeraGrid Overview
• Eleven site federation of Resource Providers– http://www.teragrid.org/– Each with own accounts, processes, policies, etc.– There exist both TeraGrid users and local, site-specific
users
• O(4K) TeraGrid users from wide variety of different sites– Most users not from TeraGrid sites– Almost all from U.S. campuses
• TeraGrid users have accounts on some/all sites– Each site has own local users as well– These are centrally managed