how intuit leveraged aws opsworks as the engine of our paas (dmg305) | aws re:invent 2013

26
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. DMG305 - How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS Capen Brinkley and Rick Mendes, Intuit November 14, 2013

Upload: amazon-web-services

Post on 08-May-2015

1.776 views

Category:

Technology


0 download

DESCRIPTION

In this talk, the engineering team behind the Intuit PaaS takes you through the design of our shared PaaS and its integration with AWS OpsWorks. We give an overview of why we decided to build our own PaaS, why we chose OpsWorks as the engine, technical details of the implementation as well as all the challenges in building a shared runtime environment for different applications. Anyone interested in OpsWorks or building a PaaS should attend for key lessons from our journey.

TRANSCRIPT

Page 1: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

DMG305 - How Intuit Leveraged AWS

OpsWorks as the Engine of Our PaaS

Capen Brinkley and Rick Mendes, Intuit

November 14, 2013

Page 2: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Agenda

• How we got into AWS, and why we’ve decided

to do a PaaS

• Why we chose AWS OpsWorks as our engine

• How we use AWS OpsWorks to build our PaaS

• Q&A

Page 3: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Intuit’s Journey to AWS

Page 4: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Why PaaS?

Page 5: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

EC2

CloudFormation

S3

Auto Scaling

Chef

Simple Deploy

Heirloom

Page 6: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Application A Application B

Page 7: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Why AWS OpsWorks?

• Skillset and operational expertise:

• Chef

• AWS services

• Similar operational model

• More controls, compliance, and security than

Elastic Beanstalk could offer

Page 8: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Architecture

Page 9: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Layered

Page 10: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Layers

Intuit Abstraction Layer

AWS OpsWorks

EC2 IAM S3 ELB SQS

Page 11: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Abstraction Layer Benefits

Page 12: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Nomenclature

Intuit AWS OpsWorks

App Stack

Stack Layer

Instance Instance

Build App

Deployment Deployment

Page 13: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Simplicity

Abstraction Layer

• Create app API

• 3 parameters

• All parameters simple

strings

• Users only need to

provide name, app type

and region

AWS OpsWorks

• Create stack API

• 16 parameters

• Parameters mix of simple

strings and elements

• Users need AWS

knowledge to set

parameters

Page 14: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Simplicity – Create Inputs Abstraction Layer

{

"name": "My App",

"app_type": "rails",

"region": "us-west-2"

}

AWS OpsWorks {

"Attributes": {

"string" : "string"

},

"ConfigurationManager": {

"Name": "string",

"Version": "string"

},

"CustomCookbooksSource": {

"Password": "string",

"Revision": "string",

"SshKey": "string",

"Type": "string",

"Url": "string",

"Username": "string"

},

"CustomJson": "string",

"DefaultAvailabilityZone": "string",

"DefaultInstanceProfileArn": "string",

"DefaultOs": "string",

"DefaultRootDeviceType": "string",

"DefaultSshKeyName": "string",

"DefaultSubnetId": "string",

"HostnameTheme": "string",

"Name": "string",

"Region": "string",

"ServiceRoleArn": "string",

"UseCustomCookbooks": "boolean",

"VpcId": "string"

}

Page 15: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Flexibility – Multiple Clients

Page 16: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Behind the Curtain

Page 17: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Continuous Integration

Intuit Package Manager

Intuit Abstraction Layer

Page 18: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Intuit Abstraction Layer

User Vars User, System Vars

Custom JSON

App Instances

App Configuration Variables

Page 19: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Managing Instance Statuses Start/Stop

Intuit Abstraction Layer

Status Request

Info Manager

Transition

Page 20: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Collaborators

App

1

App

2 Dev Group A

(4 users)

Dev Group B (6 users)

QA Group A (2 users)

Page 21: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Technologies

• RESTful APIs

• Ruby on Rails

• AWS SDK Gem

• RDS MySQL

• Opscode Chef

Page 22: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Challenges

• Think differently about Chef code – Things like roles and data bags are not supported yet

• Polling for instance statuses – There are no asynchronous APIs or notifications at this time

• Multi-step deletion of apps – Instances cannot be deleted automatically

Page 23: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Looking Ahead

• Java Support – Should we replace our custom support with AWS OpsWorks

support

• Application Visibility – Access to application logs and application runtimes

• Additional Languages/Frameworks – Expand beyond Ruby and Java

Page 24: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013
Page 25: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Q & A

Page 26: How Intuit Leveraged AWS OpsWorks as the Engine of Our PaaS (DMG305) | AWS re:Invent 2013

Please give us your feedback on this

presentation

As a thank you, we will select prize

winners daily for completed surveys!

DMG305