mark marsiglio - autoscaling with ez in the cloud - a case study

Autoscaling eZ in the CloudThe Cloud is cool, but robots in the cloud are better

Mark Marsiglio, President/CEO, ThinkCreative16 Jun 2011

Friday, June 17, 2011

Client & Developer Expectations

Typical vs. Cloud System Design

Auto-Scaling Cloud

Economics


Project Goals

Be more “cloudy”

Serve more pages, faster

Reduce hosting costs

Increase platform flexibility

Automate as much as possible

Be fully redundant, recover faster

Get woken up less by downtime alerts


Client Needs

Predictable cost

Burst capacity

Uptime

Load speed

Managed service

Trust that it is “handled”

Security

Dev Needs

Separate development environments

Easy deployment

Uptime

System stability

Ability to recover from human error quickly

Fast


Client Site Client Site Client Site


Big, fast dedicated web

server 1

Local MySQL

Local File Sys

Local eZfind

Apache/PHP


backup tar file sent to s3 for storage


server 2

Local MySQL

Local File Sys

Local eZfind

Apache/PHP

Client Site

Client Site

Client Site



server 3

Local MySQL

Local File Sys

Local eZfind

Apache/PHP


Dedicated ServersTypical Hosting Platform, Our Original Approach

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site


RiskMany points of failure

Hard to restore

Slow download of backups

Heavy backup load

Inability to backup large sites (disk space)

Traffic surge can overwhelm

Capacity estimation

RewardPretty fast

Better than a shared server

Otherwise, not much


Dedicated Cloud ServersOur 1st Generation Cloud Platform

Snapshots Snapshots Snapshots



Big, fast dedicated Amazon

Instance server 1

Local MySQL

Local File Sys

Local eZfind

Apache/PHP



Instance server 2

Local MySQL

Local File Sys

Local eZfind

Apache/PHP


Instance server 3

Local MySQL

Local File Sys

Local eZfind

Apache/PHP

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site


For ClientsFor Clients

High quality backups

Better failure recovery

More problems vs. dedicated servers

Still no failover

For UsFor Us

Lower cost

Easy to create more instances

No automation

Higher failure rate

BadGood

vs. Original System


RiskMany points of failure

Kind of hard to restore

System design changes are hard

Large traffic surge can overwhelm

Miscalculation of capacity requires DNS hassle

RewardFast and frequent backup snapshots

Super-fast backup restoration

Easy to create new instances as needed (AMI)

Technically, it’s in the cloud


New Cloud Hosting PlatformAuto-scaling array of single-purpose servers


Big, fast web app server 1

OS, Apache/PHP, synchronized apache config





Big, fast web app server 4OS, caches,

synchronized apache config

Big, fast web app

server etcOS, caches,




Elastic Load Balancers

Snapshots

NFS Server

EBS RAIDsite data, extensions,kernel, ini and logs

Snapshots

eZFindData store

eZ Find/Forwarder

Snapshots

MySQL SlaveRightscale template, can be promoted

MySQL Master

Additional instancesas needed, based on

traffic load, time of day

SSL SSL SSL SSL

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client SiteClient Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Rightscript powered

auto-scaling and scheduled min sizes


For ClientsFor Clients

Less downtime

Failover systems

Dev/staging/production

Single point of failure in NFS filesystem

More expensive

For UsFor Us

Automatic scaling

Scripted instance launch

High tolerance of failure

Higher cost of operation

More complex arch.

BadGood

vs. 1st Gen System


Current Design NotesAmazon EC2, c1.medium, m1.large

Rightscale, scripted instance launching

Unique ELB for each SSL site

Approximately 5-7 servers running, 16gb/day, 30 req/sec

Array servers vote to scale, 2 new servers in ~4 mins

Array members first in-first out

Development/Staging/Production

Scripted deployment, version controlled


Current Design Limitations

NFS bottleneck

NFS single point of failure

Little CDN advantage

Unreliable sendmail email delivery

Limited data on per-client usage

No static IP for ELBs

And...


The Cloud is Falling!

Judgement DayApril 21, 2011


Single-cloud dependence

Planned for...

Instance failure

Availability zone failure

Data store failure

Network connectivity failure

Database failure

What if the entire AWS system fails?


Next Generation ImprovementsPlanned enhancements - next 3 months










Big, fast web app






Rightscript powered


SnapshotsSnapshots

eZFindData store

eZ Find

Snapshots

SQL Slave

SQL Cluster

SSL SSL SSL SSL

S3 Storage

Video CDN

Transcoding System

File CDN

Reporting/Analytics/

Log Analysis(Splunk)

Gluster

Gluster Bricks Postmark SMTP

Sendlabs SMTP

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

Client SiteClient Site

Client Site

Client Site

Client Site

Client Site

Client Site

Non-Client Site

Client Site

Client Site

Non-Client Site

Additional instancesas needed, based on

traffic load, time of day


Replicate the system: Dedicated client arrays, multiple regions, complete staging environments, load testing copies, etc

More cloud-only benefits









Big, fast web app






Rightscript powered


Snapshots

NFS

EBS RAIDsite data, extensions, kernel, ini and logs

Snapshots

eZFindData store

eZ Find

Snapshots

MySQL Slave

MySQL

Additional instancesas needed

SSL SSL SSL SSL










Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

S3 Storage

Video CDN

Transcoding System

File CDN











Big, fast web app






Rightscript powered


Snapshots

NFS


Snapshots

eZFindData store

eZ Find

Snapshots

MySQL Slave

MySQL


SSL SSL SSL SSL










Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

S3 Storage

Video CDN

Transcoding System

File CDN











Big, fast web app






Rightscript powered


Snapshots

NFS


Snapshots

eZFindData store

eZ Find

Snapshots

MySQL Slave

MySQL


SSL SSL SSL SSL










Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

S3 Storage

Video CDN

Transcoding System

File CDN











Big, fast web app






Rightscript powered


Snapshots

NFS


Snapshots

eZFindData store

eZ Find

Snapshots

MySQL Slave

MySQL


SSL SSL SSL SSL










Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

S3 Storage

Video CDN

Transcoding System

File CDN











Big, fast web app






Rightscript powered


Snapshots

NFS


Snapshots

eZFindData store

eZ Find

Snapshots

MySQL Slave

MySQL


SSL SSL SSL SSL










Client Site

Client Site

Client Site

Client Site

Client Site

Client Site

S3 Storage

Video CDN

Transcoding System

File CDN




Cloud EconomicsCost & Revenue Considerations


System development costsStaff research & development time

Maintenance of legacy systems

Pre-launch service subscriptions, monthly fees

Migration time/cost, upgrades

Expert consulting


New Ongoing CostsAWS hourly costs

AWS Backup storage

AWS bandwidth

Cloudfront CDN bandwidth

RightScale (scripted servers)

Pingdom (monitoring)

Postmark (SMTP)

Bits on the Run (Video)

DNS Made Easy (Dynamic DNS)

Github (version control)

About US$5,000/mo Total


New RevenueHigher monthly hosting fees (US$500-800/mo avg)

Much greater hosting capacity (unlimited sites?)

Sell hosting to other developers

Reduced concessions for downtime

Reduced management time, automation

Reduced legacy system costs


Thank you.

[email protected] Marsiglio, President/CEO, ThinkCreative

16 Jun 2011


mailto:[email protected]

mailto:[email protected]

mark marsiglio - autoscaling with ez in the cloud - a case study

Technology

client usageno static

fast dedicatedbig

dedicated cloud serversour

cloudthe cloud

sent backup

singlecloud dependenceplanned

frequent backup

muchheavy backup loadinability