Transcript
Page 1: Understanding cloud costs with analytics

Ali Khajeh-Hosseini

@AliKhajeh Nov 2014, Bristol University

®

®

Understanding cloud costs with analytics  

Page 2: Understanding cloud costs with analytics

What is a cloud?

Page 3: Understanding cloud costs with analytics

Server

Page 4: Understanding cloud costs with analytics

Rack

Page 5: Understanding cloud costs with analytics

Container

Page 6: Understanding cloud costs with analytics

Electricity, Water, Internet

2500 Servers

Page 7: Understanding cloud costs with analytics
Page 8: Understanding cloud costs with analytics
Page 9: Understanding cloud costs with analytics

Taken from one of Dave Cliff ’s talk on Cloud Computing

Page 10: Understanding cloud costs with analytics

a cloud

Page 11: Understanding cloud costs with analytics

Around 60 Staff

30 Megawatts = Perth (Scotland)!

38,500 m2 = 4 ftbl pitches

Microsoft has >1M servers h#p://web.archive.org/web/20120224062157/h#p://www.microso;.com/Presspass/emea/presscentre/pressreleases/February2012/23-­‐02DublinDataCentre.mspx  h#p://news.microso;.com/speeches/steve-­‐ballmer-­‐worldwide-­‐partner-­‐conference-­‐2013-­‐keynote/    

Scale of operations

Page 12: Understanding cloud costs with analytics

Many benefits - Elasticity & pay-per-use-pricing - On-demand computing ...

Why do companies care?

Page 13: Understanding cloud costs with analytics

Animoto’s crazy scale-out

h#p://blog.rightscale.com/2008/04/23/animoto-­‐facebook-­‐scale-­‐up/  

Scale-out from 50 to 4,000 servers in 3 days

Page 14: Understanding cloud costs with analytics

Capacity planning

Images  taken  from  h#p://aws.amazon.com/economics/    

Page 15: Understanding cloud costs with analytics

Zynga’s Z-Cloud

h#p://www.rightscale.com/customers/zynga-­‐grows-­‐to-­‐1-­‐social-­‐gaming-­‐site-­‐with-­‐rightscale.php  h#p://code.zynga.com/2012/02/the-­‐evoluNon-­‐of-­‐zcloud/  

Launch new games on public clouds Move to private cloud when usage patterns are understood

Page 16: Understanding cloud costs with analytics

There are different public cloud pricing models

Rackspace IBM SoftLayer HP …

Page 17: Understanding cloud costs with analytics

Different purchase options - On-Demand: fixed rate - Reserved: pay upfront, lower rate - Spot: Bid how much you would pay - Discounts and consolidated bills

Amazon Web Services

Page 18: Understanding cloud costs with analytics

up-front procurement to pay-per-use

Image  taken  from  h#p://aws.typepad.com/aws/2011/12/reserved-­‐instance-­‐opNons-­‐for-­‐amazon-­‐ec2.html  

AWS Reserved Instances

You have to specify: term (1/3 yr), utilization, availability zone, instance type and OS when buying a RI

Page 19: Understanding cloud costs with analytics

AWS Reserved Instances Marketplace

Buy and sell reservations

Page 20: Understanding cloud costs with analytics

AWS Spot Prices

Bid for an instance, it’ll get switched on if bid >= spot price. You pay spot price, AWS sets it.

h#p://www.cs.technion.ac.il/~ladypine/spotprice-­‐slides-­‐postmortem.pdf  

Page 21: Understanding cloud costs with analytics

Example: a MySQL Database AWS Relation Database Service (RDS) - Up-front reservation fee (optional) - CPU: $X/hour (based on instance type and cloud) - Storage: $X/GB/month - Transactions: $X/Millions of I/O ops - Provisioned IOPS: $X/IOPS/month (has other limitations) - Redundancy: double the CPU hourly rate - Data transfer: $X/GB (tiered across cloud)

Page 22: Understanding cloud costs with analytics

What the heck? I just wanted a simple database

Page 23: Understanding cloud costs with analytics

Google’s sustained usage model

Images  taken  from  h#ps://cloud.google.com/compute/pricing#sustained_use  

Automatically lower the price of your virtual machines when you use them to run sustained workloads

Page 24: Understanding cloud costs with analytics

Let’s play Tetris

Images  taken  from  h#p://googlecloudplaRorm.blogspot.co.uk/2014/04/introducing-­‐sustained-­‐use-­‐discounts.html    

Page 25: Understanding cloud costs with analytics

Cloud spending is growing rapidly

Cloud pricing is complicated… we scrape over 12,000 prices.

Need to monitor, allocate, forecast & optimize costs

The challenge

h#p://www.gartner.com/newsroom/id/2352816  

Page 26: Understanding cloud costs with analytics

Jan 2009: Started researching cloud cost forecasting in St Andrews

Dec ‘11: With Hassan, we launched PlanForCloud

Jul ‘12: Acquired by RightScale

Nov ‘13: Launched Private-Beta of Cloud Analytics

Mar ‘14: Cloud Analytics goes into Public-Beta

Oct ‘14: Cloud Analytics is Generally Available

The story so far…

Page 27: Understanding cloud costs with analytics

Connect to a cloud

We regularly collect data

Page 28: Understanding cloud costs with analytics

Slice and dice the data

Page 29: Understanding cloud costs with analytics

See historic trends

Page 30: Understanding cloud costs with analytics

Forecast costs based on switching clouds or

increased growth

Page 31: Understanding cloud costs with analytics

Who buys our products?

Enterprises like Sony, EA Games, PBS,

Pearson Publishing, Zynga…

Page 32: Understanding cloud costs with analytics

Some����������� ������������������  things����������� ������������������  to����������� ������������������  look����������� ������������������  for����������� ������������������  when����������� ������������������  looking����������� ������������������  for����������� ������������������  internships����������� ������������������  or����������� ������������������  jobs����������� ������������������  

Page 33: Understanding cloud costs with analytics

Do they have fun?

What’s their hiring process?

Do they invest in good tools?

What’s the team setup, and how often do they ship?

Are people given freedom to experiment? Friday afternoon

is our iTime

What’s the level of transparency and trust? Open by Default

What’s����������� ������������������  important?����������� ������������������  

Page 34: Understanding cloud costs with analytics

Edinburgh, Santa Barbara and remote…

Page 35: Understanding cloud costs with analytics

RightScale����������� ������������������  Employee����������� ������������������  Meetup����������� ������������������  (a����������� ������������������  week����������� ������������������  of����������� ������������������  fun����������� ������������������  in����������� ������������������  California)����������� ������������������  

Page 36: Understanding cloud costs with analytics

Dev tools

Invest����������� ������������������  in����������� ������������������  good����������� ������������������  tools����������� ������������������  

Page 37: Understanding cloud costs with analytics

Invest����������� ������������������  in����������� ������������������  design����������� ������������������  

Page 38: Understanding cloud costs with analytics

eng.rightscale.com

Invest����������� ������������������  in����������� ������������������  open����������� ������������������  source����������� ������������������  

Page 39: Understanding cloud costs with analytics

Invest����������� ������������������  in����������� ������������������  automation����������� ������������������  Meet����������� ������������������  Leeroy:����������� ������������������  our����������� ������������������  bot����������� ������������������  

Page 40: Understanding cloud costs with analytics

Ruby, Praxis (our own open source API framework), Go,

Erlang, JavaScript, AngularJS, Node.

MySQL, PostgreSQL, MongoDB, Cassandra, DynamoDB,

Redshift, Memcached, Redis, Nginx, HAProxy, Rsyslog.

AWS, Google Compute Engine, MS Azure, VMware…

Use����������� ������������������  a����������� ������������������  variety����������� ������������������  of����������� ������������������  tech����������� ������������������  

Page 41: Understanding cloud costs with analytics

RightScale.com/jobs


Top Related