Advanced Discussion Cloud Formation

Henry Huang

Quick SolutionTo Deploy One for Dev/QA

Problems• Automation Efforts• Service Learning• Partial Setup• Unique Resource• Resource Cleanup

Automation Efforts

Current Status• ? No bundle solution from

Ops

Wish List• Not to re-invent the wheel• Configuration-driven

Platform

Service LearningDescription• First have to know most service

details• Long Learning Curve

Wish List• No detail but more general

Partial SetupCurrent Status• Not fully support• Most used in the deployment

Wish List• De-couple the deployment

solution• Easy to setup a partial

environment

Unique ResourceDescription• Environment cannot be shared• Avoid naming conflicts

Wish List• Most resources can be unique for

each deployment (special cases)• Dev provide the required resources

and their relationship• Ops allocate and maintain the real

resources

Example

Resource Cleanup

Cloud Formation

It BringsProblems Cloud Formation

Automation Efforts JSON

Service Learning Configuration Template

Partial Setup AutoGen-Dependencies

Unique Resource Prefix + Suffix

Resource Cleanup Yes

Example• AdminPortal

Security Group ELB Launch Auto-Scaling Group R53

Template Anatomy• Format Version Declaration• Description Declaration• Parameters Declaration• Mappings Declaration• Resource Declaration• Properties Declaration• Function Declaration• Outputs Declaration

RecentlyI finished a prototype

One ClickService almost Ready

Inputs

Cassandra, Mongo, RDS existing• DNS• Security Group Name

• Role AMI should be ready first• Other pre-defined

configuration

Demo Time

LessonsCase IFor Production, seamless upgrade requires• Old version and new one co-

existing• Service data should keep

consistent SQS, S3 Database: Cassandra, Mongo,

RDS• Static network attribute should be

smoothly switched EIP, high-level DNS

LessonsCase IIConfiguration Format need to refine• Same value but with different

format• S3Cfg Tool need to support more

scalable scenarios Partial setup Different Layers

S3CFG Layers Layer1 Role Layer2 Machine Layer3 Auto-Scaling Group Layer4 Global

Layer 1: Role

Layer 2: Machine

Layer 3: Auto-scaling Group

ASG Setting• Region, Availability zones• Instance Numbers: Min/Max/Desired• ELB• Instance Tags• Scaling Policy

Launch Configuration• Spot or On-demand Instances• Image Id• Instance Type etc.

LessonsCase IIIIssues in Cloud Formation • Cannot clean up the resources in

some special scenarios (submitted and fixing) VPC RDS Security Group update

• S3 bucket cannot be cleaned due to the data existing

• Configuration Too Large More abstraction layers needed

One day,Drag and DropService just Born!

LessonsCase IVInstance Bootstrap not scalable/common• Too many restrictions

Configuration downloaded from S3

Bundled with Instance Tags• Not scalable enough

Not easy to modify commands to execute in instance booting

Should take the experiences from cloud-init

F A Q