globus online for research data management
DESCRIPTION
This presentation is by Rachana Ananthakrishnan, Sr. Engagement Manager and Solutions Architect at the Computation Institute at The University of Chicago. It was given at the Great Plains Network Annual Meeting, on May 29, 2013. For more information on Globus Online, visit globusonline.org.TRANSCRIPT
globus online
Globus Online for Research Data Management
Rachana Ananthakrishnan Great Plains Network Annual Meeting 2013
We started with technology proven in many large-scale grids
GridFTP GRAM
MyProxy GSI-OpenSSH
…
1.2 PB of climate data delivered to 23,000 users
Typical of large, well funded research projects using GT
1.2 PB of climate data delivered to 23,000 users
GT provides robust infrastructure for the 1%
What about the 99%?
GT provides robust infrastructure for the 1%
What about the 99%?
BIG SCIENCE. Small labs
GT provides robust infrastructure for the 1%
globus online
Managing data should be easy …
Registry
Staging Store
Ingest Store
Analysis Store
Community Store
Archive Mirror
Ingest Store
Analysis Store
Community Store
Archive Mirror
Registry
… but it’s hard and frustrating!
Registry
Staging Store
Ingest Store
Analysis Store
Community Store
Archive Mirror
Ingest Store
Analysis Store
Community Store
Archive Mirror
Registry
Quota exceeded
!
Expired credentials
!
Network failed. Retry.
!
Permission denied
!
What is Globus Online?
Transfer and sharing of large data sets…
…with dropbox-like characteristics…
…directly from your own storage systems
We adopted SaaS approaches to transform the user experience
… for both researchers and resource owners/system
administrators
We started with reliable, secure, high-performance file transfer …
Data Source
Data Destination
User initiates transfer request
1
Globus Online moves and syncs files
2
Globus Online notifies user
3
… and then made it simple to share big data off existing storage systems
Data Source
User A selects file(s) to share, selects user or group, and sets permissions
1
Globus Online tracks shared files; no need to move files to cloud storage!
2
User B logs in to Globus Online and accesses
shared file
3
Log into Globus Online
Alternate Logins
Login using InCommon
InCommon Login
Source endpoint
Destination endpoint
Activation
Transfer data
Share data
Set permissions
Manage Groups
Interactive login to command line interface:
Running commands remotely:
Using CLI with gsissh:
Globus Online CLI
$ ssh [email protected]
$ ssh [email protected] <command>
$ gsissh [email protected] <command>
$ ssh [email protected] scp –r –s 3 -D \ nersc#dtn:~/myfile* mylaptop:~/projects/p1 Task ID: 4a3c471e-edef-11df-aa30-1231350018b1 $ _
Usage is accelerating
Early Adopters
• What is GCMU? – Globus Connect version for easily creating (sharable) endpoints
on multi-user storage servers – Packages a GridFTP server and MyProxy CA authentication
server, pre-configured for use with Globus Online
• Why GCMU? – Create transfer endpoints in minutes – Avoid complex GridFTP install
• To download: www.globusonline.org/gcmu
Globus Connect Multiuser (GCMU)
29
“We used GCMU to form a campus-‐wide GSI authenAcaAon service spanning mulAple servers. Now my users have a fast, easy way to get their data wherever it needs to go, and the setup process was trivial." -‐-‐University of Michigan
“As a resource admin, I've found GCMU an exceedingly useful tool.... With GCMU, seHng up a GridFTP server and handling authenAcaAon for mulAple users is easy." -‐-‐Oak Ridge Na8onal Lab
We are a non-profit service provider to the non-profit
research community
Our challenge:
Sustainability
We are a non-profit service provider to the non-profit
research community
Globus Online Provider Plans
Support ongoing operations
Offer value-added capabilities Engage more closely with users
• Endpoint operations management • Branded web sites • Alternate identity provider • Usage reporting • MSS optimizations • Multiple GridFTP servers per endpoint
Provider Plans offer…
Starting at $20k per year
End User Plans
• Basic: Free – File transfer and synchronization to/from
servers – Personal endpoints with Globus Connect – Access to shared endpoints created by others
• Plus: $7/month (or $70/year) – Create and manage shared endpoints – Peer-to-peer transfer and sharing
Globus Platform-as-a-Service
Globus Nexus (Identity, Group, Profile)
…
Sharing Service
Transfer Service
Dataset Services
Globus Toolkit
Glo
bus
Onl
ine
API
s
Glo
bus
Con
nect
Our research is supported by:
U.S. DEPARTMENT OF
ENERGY
Questions?
Contact: [email protected]
Providers: globusonline.org/provider-plans
www.globusonline.org