rcuk cloud workshop
TRANSCRIPT
Supporting research in the Cloud: Our experiences and future directionsSimon WoodmanDigital InstituteNewcastle University
Digital Institute• In 2001, NEReSC was formed to help researchers
– £17M funding • Bioinformatics, Neuroscience, Aging & Health, Chemical Engineering,
Transport, Video archiving– Core UK e-Science funding programme
• Became the Digital Institute– Similar remit but more diverse: Humanities, Medical Science, BIM– £40M funding
• Provided computing aspect of many research projects• Frequently same requirements
ResearchWorld-leading
Teaching Specialist
Skills& Knowledge
Engagement
Impact on Industry &
Public Sector
Innovation Cycle
Cloud Innovation CentreEvents, engagement &partnerships
EPSRC Centre for Doctoral training in Cloud Computing for Big DataDeveloping thenext generation of leaders
World-leading Research in Cloud computing & data analytics
Digital Institute
Having 100s of machines available to process data
doesn’t automatically help
Building a data management and processing platform
• An environment to store, manage and process data– Every project needed this, volumes growing– Open Source
• A platform that can operate in a number of different locations– Laptop– On Premise– On a cloud provider (Amazon and MS)
• An expandable system– API to connect other software– Data processing code can be added
• A platform for our academic research– Scalability, data management, provenance
Provenance
Features of e-SC• Data storage– Cloud: infinite scale
• Data processing– Best-of-breed open source tools (R / Octave)
• Provenance– Audit of everything performed
• Scalability– Easy to run at large scale
• External communications– Rich APIs
All our projects have significant storage, processing and
collaboration needs
Use of the platform
Movement monitoring
Collaboration
ModellingStroke rehabilitation
Care home design
Cap Ex v Op Ex
Service Provision v Research
One eScience Central to rule
them all.
Learning from Industry - DevOps
The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win. Gene Kim, Kevin Behr, George Spafford. ISBN 0988262509
Cloud Innovation Centre
• £1M funding from DCMS
• Offerings– Training– Architecture Reviews– Consultancy– KTPs– Cloud time
• £700k Private and Public Cloud infrastructure
Cloud Computing for Big Data CDT
• £7M funding from EPSRC• 60 Students • 5 years
• High level of industry engagement
Cloud Computing
Bayesian Statistics
Group Projects Internships
Thesis
Retreats
Professional Skills
Challenges• Past– Cost of on-boarding– Scaling
• Present– Licensing– Researchers doing Sys Admin
• Future–Manage the relationship with Central IT– Staff costs/time
