aws apac webinar week - maintaining performance & availability while lowering cost with aws
TRANSCRIPT
aws.amazon.com/webinars/apac/webinar-week | #AWSWebinarWeek
Maintaining Performance & Availability While Lowering Costs with AWSDean SamuelsSolutions Architect Manager
v
What You’ll Get Out of This Session• A lower AWS bill• Maintain high performance while minimising cost• A more scalable, robust, dynamic architecture• More time to innovate• Real-world customer examples• Easy to implement
Pay as you Go, No Upfront Investment
Replace capital expenditure with variable expense
Low ongoing cost
Customers leverage our economies of scale
Easily Scale Up & Down
No need to guess capacity requirements and over-
provision
PRICEREDUCTIONS
Speed, agility and innovation
Infrastructure in minutes not weeks
Focus on business
Not undifferentiated heavy lifting
Global Reach
Go global in minutes and reach a global audience
Cloud Computing Benefits
You’re using AWS.You like it.
A couple assumptions…
But maybe you are spendingmore than you planned
Or you’d just like to spend less
v
Or in other words…..
Going From:
• Paying for what you
use
To:
• Paying for what you
need
What should you do??
1. Limit Resource Provisioning…
By Controlling Who Can Provision Resources…
Identity andAccess Management
Roles Permissions
2. Understand What is Deployed & What it Costs…
By Employing Tags…• Key (Attribute): 127 Unicode
characters
• Value (Detail/Description): 255 Unicode characters
• Tags per resource: 10 tags
Jane_Doe
v
…And Using The Different Types of Tags Appropriately
Resource Tags• Provide the ability to organize and search within and across resources• Filterable and Searchable• Do not appear in Detail Billing Report
Cost Allocation Tags• Provide the ability to map AWS charges to organizational attributes for accounting purposes• Information presented in Detailed Billing Report and Cost Explorer (must be explicitly selected)• Only available on certain services or limited to components within a service (e.g. S3 bucket but not objects)
Tag Key Examples
Cost Center Business Unit
Environ. Tier
Owner
Dept./ Group
Product / Application
Shutdown Time
Support ContactEndpoint
Backup
v
AWS Config Rules
• Set up rules to check configuration changes recorded• Use pre-built rules provided by AWS• Author custom rules using AWS Lambda • Invoked automatically for continuous assessment • Use dashboard for visualizing compliance and identifying offending
changes
Customer Example
Large Financial Services
Customer“No tags? No
instance”
3. Proactively Monitor Your Account Billing Usage…
By Using Detailed Billing Reports andEnabling Billing Alerts…
…And Configuring Simple…
…Or Advanced Alarms in CloudWatch…
…Alongside Cost Explorerhttp://amzn.to/1zHE2Fj
…Or Using an AWS Partner Tool…
Customer Example – Kellogg“CloudWatch helps our people make better decisions around the capacity they need, so that they can avoid waste…”
“We were never able to do that with our on-premises infrastructure. AWS breaks down usage and cost to such a granular level that we can identify which costs come from which department, like a toll model.”
4. Identify Idle Resources and Turn Off Unused Instances…
…Using Trusted Advisor…
Trusted Advisor
…Or Amazon CloudWatch to Monitor,Collect and Track Metrics…
Amazon CloudWatch
…Or (Again) Using an AWS Partner Tool…
v
Customer Case Studies
Pfizer’s high performance computing (HPC) software and systems for worldwide research and development (WRD) support large-scale data analysis, research projects, clinical analytics, and modeling. “We accomplished this by customizing the job scheduler in our HPC environment to recognize workload, and start and stop instances as needed to address the workflow”
RunE2E for Kenneth Cole leverage several AWS APIs for starting/stopping the system, increasing system capacity based on demand needs, and changing security profiles providing a low cost but scalable environment to run its business.
“Instance actions, such as create, stop, start, and terminate, can very easily be scheduled as an event. For example, now we can terminate all instances automatically after a training class, thereby eliminating the generation of unnecessary cost”
4. Design for Elasticity Rather Than Deploy for Peak…
v
By Picking the Right Instance Type…
Instance Type vCPU Mem(GiB)
Monthly Cost (OD)*
Ideal use case
m3.medium 1 3.75 $48.24 Always available, balanced
m4.large 2 8 $90.72 Latest generation, always available, balanced
c4.large 2 8 $79.20 Always available, compute
r3.large 2 15 $126.00 Always available, memory
g2.xlarge 8 15 $468.00 Always available, GPU
d2.xlarge 4 30.5 $496.80 Always available, storage
t2.medium 2 4 $37.44 Bursty workloads
* Assumes Amazon Linux Instances in US-East at 720 hours per month
Customer Example – KaplanWe quickly converted a hefty percentage of our systems to run under this EC2 Instance type, and have seen major cost reductions without affecting performance whatsoever.
For all instances built for Kaplan moving forward, the T2 is going to be the preferred option unless measured or load-tested computing needs demand otherwise.
Using Autoscaling….
Automatic resizing of compute clusters based on demand
Trigger autoscaling policy
Feature DetailsControl Define minimum and maximum
instance pool sizes and when scaling and cool down occurs.
Integrated to Amazon
CloudWatch
Use metrics gathered by CloudWatch to drive scaling.
Instance types Run Auto Scaling for On-Demand and Spot Instances.
AWS autoscaling create-autoscaling-group— Auto Scaling-group-name MyGroup— Launch-configuration-name MyConfig— Min size 4— Max size 200— Availability Zones us-west-2c
…And Adjusting for Utilization and Granularity
More smaller instances vs. less larger instances
29 m4.large @ $0.126/hr$2,630.88 / mo*
59 t2.medium @ $0.052/hr$2,203.20 / mo*
*Assumes Linux instances in US-East at 720 hours per month
v
Customer Case Studies
“We believe auto scaling greatly improves the availability of our services and provides an excellent means of optimizing our cloud costs.”
http://techblog.netflix.com/2012/01/auto-scaling-in-amazon-cloud.html
Siemens Healthcare Diagnostics helps advance human health through innovation. By using services such as ELB, Amazon SQS, Amazon EC2, Amazon RDS, Auto Scaling, Amazon ElastiCache, and AWS CloudTrail, Siemens has built a secure, HIPAA-compliant, and scalable platform on AWS.
The array of AWS products offers TicketLeap a potent solution. The company uses Amazon EC2, Amazon RDS, Amazon CloudFront, Amazon S3, ELB, Auto Scaling, and cloud monitoring services as the foundation for its platform.
v
5. Architect Your Workload with Cost in Mind
Select Reserved Instances for SteadyState Workloads…
Up to 60%+ Savings*
(and Capacity Reservation)
Commitment Level• 1 Year• 3 Year
AWS Services Offering RIs • Amazon EC2 • Amazon RDS• Amazon DynamoDB• Amazon Redshift• Amazon ElastiCache
* Dependent on specific AWS Service, size/type and Region
Or Spot Instances for Non-Stateful (Time Insensitive or Fault Tolerant) Workloads…
90%+ Savings!*
Minimum Commitment• Commit to 1 hour
Tradeoff• Potential for interruption
* Compared to On Demand price based on specific EC2 instance type, region and availability zone
…Or A Combination of All Three1. Start by using RIs for
known/steady-state workloads2. Set-up multiple autoscaling groups3. Autoscale first using Spot first4. If Spot instances are unavailable
(or priced too high), use On-Demand
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 240
2
4
6
8
10
/Spot Instances
On DemandSpotReserved Instance
v
Customer Case Studies
As one of the world’s largest dance fitness programs, Zumba Fitness needed a scalable and high performance website. When Zumba migrated its website to AWS in 2009, the company saved 39% on computing resources by using Amazon Reserved Instances. Zumba could also scale easily to serve up 7 TB of images to its customers each month.
Project that involved screening 10 million compounds against a common cancer target in less than a week. They calculated that it would take 50,000 cores and close to a $40 million investment if they wanted to run the experiment internally. The project ran across 10,600 Spot Instances (approximately 87,000 compute cores) and allowed Novartis to conduct 39 years of computational chemistry in 9 hours for a cost of $4,232. Out of the 10 million compounds screened, three promising compounds were successfully identified.
Leverage The Right Storage Classes…
AWS Cloud
Amazon Glacier
Gateway Appliance/ AWS Storage Gateway
Amazon S3
Block File
On-premises Data Center
Archive Backup Disaster Recovery
Amazon EBS
• Reduced redundancy storage class– 99.99% durability vs. 99.999999999%– Up to 20% savings– Everything that is easy to reproduce– Use Amazon SNS lost object
notifications• Amazon Glacier storage class
– Same 99.999999999% durability– 3 to 5 hours restore time– Up to 64% savings– Archiving, long-term backups, and old
data• Use S3 Lifecycle rules
– Archive objects to Glacier after x days– Delete objects after y daysAmazon EFS
…Alongside AWS Managed or ApplicationServices…
Elastic LoadBalancing
(ELB)
Amazon Relational Database
Service (RDS)
Amazon ElastiCache
Amazon DynamoDB
Amazon Simple Queue Service
(SQS)
Amazon Simple Email Service
(SES)
AmazonLambda
Amazon Elastic MapReduce
(EMR)
Amazon Simple Notification
Service (SNS)
Amazon Kinesis
Amazon Route 53
Amazon Redshift
To Offload Your Architecture…
+
Standard Setup
• 4 x Medium Instances$193
• AWS Data Transfer 1 TB$92
• Total = $285
Optimized• 1 x Medium Instance
$48• CloudFront Data 1 TB
$87• CloudFront Requests (10M)
$7.5• Total = $142.5
50%
6X
Cheaper
Faster
v
Customer Case Studies
Airbnb is a community marketplace for unique vacation spaces around the world. Airbnb benefits from the scalability, agility and reliability provided by Amazon Web Services, including Amazon EC2, Elastic Load Balancing, Amazon S3, Amazon EMR, Amazon CloudFront, Amazon CloudWatch, and Amazon RDS.
The Internet Movie Database (www.imdb.com) is one of the world’s most popular and authoritative sources for movie, TV and celebrity content with more than 100 million unique visitors per month. IMDb uses AWS and Amazon CloudFront to host search data, making the search experience the fastest possible by distributing the content physically close to IMDb's worldwide user base.
PBS, a private, non-profit corporation, provides content through television, the Web and mobile applications. PBS Interactive, the department responsible for PBS’ Internet and mobile presence, improved its video streaming performance by migrating to AWS to utilize the content delivery service Amazon CloudFront and increase its existing usage of Amazon S3.
v
Customer Case Studies
PlaceIQ, a location-based intelligence company in New York City, provides location intelligence for mobile advertising. The company uses Amazon ElastiCache and AWS to improve its web service response time by 83%.
Sanoma Games designs casual online gaming and fantasy sports leagues as part of the Sanoma diversified media group. The business unit recently closed its local datacenter in order to build a scalable, service-based architecture that can facilitate expansion into additional markets and gaming categories. Cloud management specialist Nordcloud was appointed to create and manage the new environment, which now includes Amazon RDS, Amazon EC2, Amazon S3, Amazon ElastiCache, and Amazon CloudWatch.
SpareFoot.com helps self-storage facilities market their extra space, and helps consumers make educated decisions on which storage unit is best for their needs. In order to manage a network of consumer facing sites, the company uses several AWS products—including Amazon RDS, Amazon ELB, Amazon S3, Amazon ElastiCache, Amazon CloudWatch, and Amazon CloudFront—to create a stable system and reduce time, costs, and labor.
Go Server-less!
Amazon Lambda
Amazon S3
Amazon SES
Amazon SNS
Amazon Cognito
Amazon Mobile Analytics
JavaScript SDK
High ScoresTable
Amazon Lambda
Web Server Authentication & User High Score Storage
Compare User High Score to Global Top 10
DynamoDB
UsersTable
DynamoDB
Notify Top10On New High Score
Obtain Top10Contacts & Send Messages
Send E-MailRecord Usage Statistics
v
7. And Remember to Iterate!
Architect
BuildMonitor
• New Instance Types/Families (e.g. m4s)
• New Technologies (e.g. Amazon RDS for Aurora, AWS Service Catalog)
New Prices
…And Use ConsolidatedBilling• Receive a single bill for all
charges incurred across all linked accounts‒ Share RI discounts‒ Combine tiering benefits / Volume
discounts
• View & manage linked accounts• Add additional accounts
Let’s RecapLimit Resource ProvisioningUnderstand What is Deployed and What it
CostsProactively Monitor Your Account Billing
UsageDesign for Elasticity Rather Than Deploy for
PeakIdentify Idle Resources and Turn Off
Unused InstancesArchitect Your Workloads with Cost in MindBe Cognizant of Technology Lifecycle
Updates
Online Labs | Training
Gain confidence and hands-on experience with AWS. Watch free
Instructional Videos and explore Self-Paced Labs
Instructor Led Classes
Learn how to design, deploy and operate highly available, cost-effective and secure
applications on AWS in courses led by qualified AWS instructors
Validate your technical expertise with AWS and use practice exams
to help you prepare for AWS Certification
AWS Certification
http://aws.amazon.com/training