london 2015 - amazon s3 · amazon web services , novartis built a platform using 10,600 spot...
TRANSCRIPT
London 2015
Transforming Education in the Cloud
Mark Rowlands Solutions Architect EMEA Public Sector
How our Customers are using the Cloud to
Transform Education.
• Research. – Computational Chemistry/Biology, Molecular Biomechanics, Machine Learning,
– Scientific Databases, Natural Language Processing, Neuroimaging data analysis
– Genome modeling,
• Campus IT – University Websites , Student Databases, Library Management Systems
– Storage/Backup, Disaster Recovery.
• Education. – Learning/Course Management, Systems Student Lab Environments
– Massive Open Online Courses ( MOOCs)
Why are researchers using AWS today?
Reducing the time to Science. – Create HPC cluster for Research in minutes.
– Simple creation process using AWS API’s, Marketplace, or CF templates.
– Access to virtually unlimited compute and storage capacity on demand.
– Use Traditional HPC schedulers and cluster managers
– Community Projects.
=
The AWS MarketPlace
AWS.amazon.com/marketplace
Dozens of scientific
applications.
Almost all major
research-focused
operating systems
• University of Liège is a Belgian multi-
disciplinary research lab studying the brain’s
function and structure
• Needed a solution that would quickly and
reliably conduct image processing for
Parkinson’s research
• Found the NITRC computational environment
on AWS Marketplace, reducing time required
to process neuroimaging data by 85%
Using AWS Marketplace to Find Key Software, University of Liège
Accelerates Parkinson’s Disease Research
Reducing the time to Science.– Create HPC cluster for Research in minutes.
– Simple creation process using AWS API’s, AWS CLI Marketplace, or CF templates.
– Access to virtually unlimited compute and storage capacity on demand.
– Use Traditional HPC schedulers and cluster managers
– Community Projects.
Try our HPC AWS
CloudFormation-based demo
For more info
aws.amazon.com/hpc/resources
#cfncluster
10 minutes
Reducing the time to Science.– Create HPC cluster for Research in minutes.
– Simple creation process using AWS API’s, Marketplace, or CF templates.
– Access to virtually unlimited compute and storage capacity on demand.
– Use Traditional HPC schedulers and cluster managers
– Community Projects.
“Amazon Builds World’s
Fastest Nonexistent
Supercomputer”
- Wired Magizine
Jun 2014 Top
500 list
484.2 TFlop/s
26,496 cores in
a cluster of
EC2 C3
instances
Cluster compute instances
Intel® Xeon® processors
10 Gigabit Ethernet –
c4.8xlarge
36 vCPUs
2.9 GHz Intel Xeon
E5-2666 v3 Haswell
60GB RAM
Performance for tightly-coupled workloads
Network placement groups
Cluster instances deployed in a Placement Group
enjoy low latency, full bisection 10 Gbps bandwidth
• Partnering with Cycle Computing and
Amazon Web Services , Novartis built
a platform using 10,600 Spot
Instances (approximately 87,000
compute cores) in just under two
hours.
• Researchers were able to conduct 39
years of computational chemistry in 9
hours for a cost of $4,232. Out of the
10 million compounds screened, three
were successfully identified.
AWS & Cycle Computing Helps Novartis accelerate the virtual
screening of 10 million compounds against a common cancer
target in less than a week
Spot Instances are your friends!
Reducing the time to Science.– Create HPC cluster for Research in minutes.
– Simple creation process using AWS API’s, Marketplace, or CF templates.
– Access to virtually unlimited compute and storage capacity on demand.
– Use Traditional HPC schedulers and cluster managers
– Community Projects.
Data-Intensive Computing
The Square Kilometer Array will link 250,000 radio telescopes together, creating the
world’s most sensitive telescope. The SKA will generate zettabytes of raw data,
publishing exabytes annually over 30-40 years.
Researchers are using AWS to develop and test:
• Data processing pipelines
• Image visualization tools
• Exabyte-scale research data management
• Collaborative research environments
aws.amazon.com/solutions/case-studies/icrar/
Why are Researchers using AWS ? • Agility.
– Quickly deploy multiple Clusters running at the same time and match the architectures to the jobs.
– Use Autoscaling to automatically re-size clusters based on demand and policy.
• Cost.
– Cluster Compute or Cluster GPU servers on-demand without large capital investments
– Use spot with Autoscaling for jobs where time is not a primary concern.
Autoscaling with Spot instances
as-create-launch-config spotlc-5cents
--image-id ami-e565ba8c
--instance-type m1.small
--spot-price “0.05”
. . .
as-create-auto-scaling-group spotasg
--launch-configuration spotlc-5cents
--availability-zones “us-east-1a,us-
east-1b”
--max-size 16
--min-size 1
--desiredcapacity 3
AWS Command line Tool syntax
Harvard Medical School: simulation development
The combination of our approach to
biomedical computing and AWS
allowed us to focus our time and
energy on simulation development,
rather than technology, to get results
quickly. Without the benefits of AWS,
we certainly would not be as far along
as we are.
Dr. Peter Tonellato,
LPM, Center for Biomedical
Informatics, Harvard Medical School
”
“ • Leveraged EC2 spot instances in
workflows
• 1 day worth of effort resulted in
50% in cost savings
AWS Global Impact Initiatives for Science
• AWS Research Grants– Initiate & support development of cloud-enabled technologies.
• AWS Hosted Public Datasets– Large and globally significant datasets hosted and paid for by AWS for community use.
– Data can be quickly and easily processed with elastic computing resources in the
surrounding cloud.
• AWS SciCo Team– Dedicated team focusing on Scientific Computing & Research workloads.
– Ensuring the cloud is able to make a disruptive impact on science
How Campus IT is using AWS today
Campus IT use cases– Development and Test.
– University and Departmental Websites.
– Student Information System Software.
– Learning /Course Management Systems.
– Massive Open Online Courses MOOCs.
– Student Lab Environments.
– Storage and Data Collaboration.
– Back and Archival
– Disaster Recovery
Campus IT use cases
– Agility. – Global Cloud Infrastructure.
– Quickly innovate , experiment and iterate.
– Deploy new applications quickly & instantly Scale Up & Down.
AWS enables Campus IT.
• Add New Test/Dev Environment
• Add 50 Servers for Student Labs
• Add New DR Environment.
• Add 1,000 Servers for Enrollment
• Remove 1,000 Servers after Enrollment
• Deploy a 10,000 Core HPC Cluster
• Shut Down a 10,000 Core HPC Cluster
AWS:
Infrastructure in
Minutes
Oxford University Press : MyMaths scales to 9000 schools.
We are a small team of two
programmers and we don't actually
have a dedicated SysAdmin. I can
safely say that Amazon Web Services
has saved months of effort building
and automating infrastructure, AWS
makes my life a lot easier keeping the
stack running
Simon Engledew,
Senior Engineer
Oxford University Press
”
“ • MyMaths is used in 9000 schools around the globe
and in 85% of UK Secondary schools.
• In 2014 MyMaths migrated from a traditional privately
hosted platform to AWS.
• 4.5 million unique student accounts.
• Average weekday load +7million page views
equivalent to 40 million requests.
• Scales quickly to handle peaks
• Improvements in availability and response times.
Campus IT use cases
– Low Cost.– Pay as you go with no-upfront fees or long term commitments
– Efficiencies of Scale allow us to pass the cost saving on to customers.
– 48 price reduction since 2006.
• Gibraltar Area School District is a Wisconsin school district serving 681 students
• The district needed a cost-effective way to replace 8 campus servers without going to the voters for funding
• The 25% cost savings realized with AWS will be used to buy laptops for students and build out an on-campus wireless network
Gibraltar Area School District Cuts Costs by 25% with
AWS
How Educators are using AWS today
How Educators are using AWS today
Cit slide deck
Make waves slide deck
PLACE
London
Makewaves
Transforming the way we recognise, capture and communicate learning
Building a global badging community
Proxy Server
Mail Server
Web Server
Media Server
MSSQL Database Backup Server
Fragile
Hard to maintain &
expand
Expensive
Makewaves Architecture on Amazon
• S3 & Cloudfront: Cost effective, faster and more expandable
• EC2: Agile, scalable and reliable (& seasonal!)
• Mobile SDK: Developer effective, secure and flexible
• Dynamo DB & Aurora (our future): Fast, reliable and scalable
Web, mobile and cloud
Our pioneering Apps enable
mobile badging on the move
direct from the cloud
“The Makewaves app has
transformed teaching and
learning in school.”
LONDON
Location Name
Pat McCarthy Lecturer & Course Co-OrdinatorCloud ComputingCork Institute of Technology
Agenda
• Background To CIT Cloud Programmes• Focus of The Programmes - Upskilling• Cloud Architecture Module• Sample Student Project• Future Plans - Amazon AWS certification• Q & A
CIT - First Msc in Cloud Computing Delivered Online
CIT Cloud Computing Online Programmes
• Msc & Bsc in Cloud Computing focused on IT professionals in Industry
• Hdip in Cloud Computing & Cloud Software Development -conversion programmes(Govt Funded) focused on upskilling graduates for cloud computing roles.
Bsc in Cloud Computing - Cloud Architectures
Cloud Architecture Course Format
• Lectures online using Adobe Connect Software
• Meeting Room Cloud Architecture http://citcork.adobeconnect.com/ca
• Lecture Presentations, Case Studies on Cloud Architectures & Lab Exercises
• Interaction on the Blackboard (LMS) forum evaluated by the lecturer.
• Assessments include Cloud Architecture Project using Amazon Web Services and Course Exam
Course Content
Students gain the theoretical and practical skills in Cloud Computing by using Cloud Technology i.e AWS Cloud Architecture
• Cloud Architecture Reference Model [NIST]• Private, Public & Hybrid Cloud models• Cloud Data Center Environment AWS Console AWS cloud Amazon AMI
• Enterprise Architecture – TOGAF model• Infrastructure as a Service • Virtualization – Role of the Hypervisor AWS EC2 AWS VPC AWS S3
• PaaS– Application Development• SaaS - Applications & APIs• Cloud Security & Compliance
AWS CloudWatch AWS IAM
Challenges & Deliverables
• Module delivered to a mixed audience, undergraduates and graduates from different demographic groups.
• Location – country wide, flexibility of online delivery & greater accessibility
• Adobe Connect for Live Lectures supported by Blackboard LMS
• Amazon AWS Cloud Platform for Virtual Labs on Cloud Architecture
• Very positive results and experience from students – placement & employment from both Hdip and Bsc courses
Cloud Architecture Overview
AWS – The Cloud Learning Environment
Sample Student Project Assessment
• Cloud Migration
– Use the TOGAF model to propose Cloud Architecture
– Redesign using AWS Cloud Center Architecture
– Visio Design of new Cloud Data Center Architecture
– Create AMI instances to emulate new Architecture in a VPC
• Configure AWS
– Infrastructure (Storage, Network and Compute)
– Implement services such as AWS EC2, S3, EBS, VPC, IAM & CloudFrontetc
– Management & Monitoring Dashboards of the Virtual Infrastructure
Teaching Python using AWS
Cloud Computing with Python
• Students get to use boto library to write Python scripts to automate and manage AWS services
• Manage EC2 images (start, terminate, monitor attributes etc), allocate and manage S3 storage.
• Monitor performance of the EC2 images with AWS CloudWatch.
Future Plans For CIT & AWS
• Integrate AWS in other Cloud Computing Modules
• Cloud Modules offered as electives on undergraduate programmes.
• Look to add value to students with hands on skills – more attractive to employers
• CIT has strong history of offering professionally certified courses to students
• Incorporate AWS Solutions Architect Associate.
