aws re:invent 2016: from monolithic to microservices: evolving architecture patterns in the cloud...

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Emerson Loureiro, Sr. Software Engineer, Gilt

Derek Chiles, Tech Leader Team, AWS

November 30, 2016

From Monolithic to MicroservicesGilt’s Journey To Microservices on AWS

ARC305

What to Expect from the Session

• Why Microservices?

• Migration approaches and considerations

• Gilt’s journey to microservices on AWS

• Architecture patterns and best practices

Gilt

Gilt

Gilt

“How can we scale things up independently?”

“How can we arrange teams around strategic initiatives?”

“How can we make it easy and fast to push changes to

production?”

Gilt

Ruby on

Rails

Jobs

memcache

Postgres

Ruby on

Rails

Jobs

memcache

Postgres

Product

service

Order

service

Cart

service

…

Swift

(gilt.com)

City Swift

(giltcity.com)

Taste(gilttaste.com)

…Front-endJava services

Ruby on

Rails

Jobs

memcache

Postgres

Product

service

Java services

Web

PDP

Web

Search

Web

Homepage

…

gilt.comScala

microservices

Order

service

201520112007

…

Common MobilePersonali-

sationAdmin Data

Gilt

(2) “Department” accounts

Existing Data Centre

“Legacy VPC”

(1) Deploy to VPC Dual 10 Gb direct connect line, 2-ms latency.

Monolithic Microservices

• Simple deployments

• Inter-module refactoring

• Binary failure modes

• Vertical scaling

• Technology monoculture

• Partial deployments

• Strong module boundaries

• Graceful degradation

• Horizontal scaling

• Technology diversity

Organization & Ownership

Monolithic Organization

Organized on technology capabilities

UI Team

DBA Team

App Logic Team

Web Tier

App Tier

DB

Organizational Structure Application Architecture

Microservices OrganizationOrganized around business initiatives

Login

Registration

Order

Personalization

Accounts team

Mobile

Personalization team Mobile team

The Teams

• Ingredients

• Team size: typically 3-5 people

The “A” Team

The Teams

• Independent teams who fully own their services:

• Requirements

• Quality

• Deployment

• Source code

• Technology selection

Architecture

Monolithic Architecture

Load

Balancer

Account Service

Cart Service

Shipping Service

StoreFront UI

BrowserDatabase

Data Access

Service

Account

Database

Inventory

Database

Shipping

Database

Microservices Architecture

Load

BalancerStoreFront

UIBrowser

Account

Service

Cart Service

Shipping

Service

Load

Balancer

Load

Balancer

Load

Balancer

Microservices Architecture

Load

BalancerStoreFront

UIBrowser

Account

Database

Account

Service

Cart Service Inventory

Database

Shipping

Service

API

Gateway

Load

Balancer

Load

Balancer

Load

BalancerShipping

Database

Auto Scaling Group

Typical Service Stack at Gilt

ELB

DNS

Docker registry

…

Amazon CloudWatch Metrics

Amazon SNS Topic

EC2 Instance

Service “X” VM

EC2 Instance

Service “X” VMCloudWatch Log Group

Service Instance Types at Gilt

Decomposition

Strangling the Monolith

DatabaseStoreFront

UIBrowser

Cart

Service

Account

Service

User

Service

No New Features on the Monolith

DatabaseStoreFront

UIBrowser

New

Microservice

New DB

Cart

Service

Account

Service

User

Service

Anti-Corruption Layer for Integration

ACL

DatabaseStoreFront

UIBrowser

New

Microservice

New DB

Cart

Service

Account

Service

User

Service

Cart

Service

Account

Service

Start With a Simple Existing Service…

DatabaseStoreFront

UIBrowser

User

Microservice

User DB

New

Microservice

New DB

User

Service

Client

Account

Service

User

Service

Client

…Move on to More Complicated Services

Cart

Microservice

DatabaseStoreFront

UIBrowser

User

Microservice

User DB

New

Microservice

New DB

ACL

Cart DB

Cart

Service

…Keep Walking and…

Cart

Microservice

DatabaseStoreFront

UIBrowser

User

Microservice

User DB

New

Microservice

New DB

Account

Microservice

Cart DB Account DB

Account

Service

User

Service

Client

ACL

Cart

Service

ACL

…The Monolith is Eventually Gone

Cart

Microservice

StoreFront

UIBrowser

User

Microservice

User DB

New

Microservice

New DB

Account

Microservice

Cart DB Account DB

Data Management

Decentralized Data Stores

• Freedom of choice

• Low impact schema changes

• Independent scalabilityaccount-

svc

cart-

svc

user-

svc

Amazon

ElastiCacheAmazon

RDSAmazon

DynamoDB

RDS

Storage & DB options in AWS

Amazon

RDS

Amazon

DynamoDBAmazon

ElasticSearch

Amazon

S3Amazon

Kinesis

Amazon

ElastiCache

Cache NoSQL SQL SearchObject Streaming

Amazon

Redshift

Amazon

Glacier

Serv

ice

Cross-Service Transactions

1) Avoid them

2) Use an optimistic model

• Accept eventual consistency

• Design for idempotency

• Detect conflicts and resolve later

• Ignore conflicts

3) Use a pessimistic model

• Create a transaction manager or distributed locking service

• Build transaction logic into the services

• Rethink your design

Aggregation

usr svc

Pull model Push model

Data Aggregation

Application

usr svc

Data

Aggregation

Application

Pub/Sub Composite

Composite Data Service

usr account cart

account svc cart svc

account svc

cart svc

Pub Sub

usr svc

account svc

cart svc

Data

Aggregation

Application

Push

Pull

Aggregation

usr svc

Pull model Push model

Data Aggregation

Application

usr svc

Data

Aggregation

Application

Pub/Sub Composite

Composite Data Service

usr account cart

account svc cart svc

account svc

cart svc

Pub Sub

usr svc

account svc

cart svc

Data

Aggregation

Application

Push

Pull

Aggregation

usr svc

Pull model Push model

Data Aggregation

Application

usr svc

Data

Aggregation

Application

Pub/Sub Composite

Composite Data Service

usr account cart

account svc cart svc

account svc

cart svc

Pub Sub

usr svc

account svc

cart svc

Data

Aggregation

Application

Push

Pull

Aggregation

usr svc

Pull model Push model

Data Aggregation

Application

usr svc

Data

Aggregation

Application

Pub/Sub Composite

Composite Data Service

usr account cart

account svc cart svc

account svc

cart svc

Pub Sub

usr svc

account svc

cart svc

Data

Aggregation

Application

Push

Pull

API Discovery & Management

DiscoveryOption #1 – Convention-based naming

shoppingcart.gamma.example.com

DiscoveryOption #2: Dynamic Discovery

StoreFront UI

Account

Service

Service Registry

Cart Service

Cart Service2) Query

3) Request

1) Register

DiscoveryOption #2: Dynamic Discovery

API Management

• Enforce API consistency

• Monitor key metrics

• Implement read-through caching

• Provide authentication and authorization

• Meter and throttle

Amazon API Gateway

• Enforce API consistency

• Monitor key metrics

• Implement read-through caching

• Provide authentication and authorization

• Meter and throttle

Amazon API Gateway for Microservices

ClientAPI Gateway

Monolith

on EC2

Microservice 1

(Lambda)

Microservice 2

(ELB+EC2)

Microservice 3

(AWS Elastic Beanstalk)

DO DON’T

• Add new methods

• Add optional params

• Deprecate

• Negotiate disruptive

changes

• Delete or rename

• Add required params

• Remove required params

• Mandate disruptive

changes

API Version Management

Deployment

Continuous Delivery & Continuous Deployment

Integration &

perf tests

Build & unit

testsbeta Prod

Monolit

h

Timeline

Cart, UserCart,

Account

Sales,

Account,

User

On the monolith…

Continuous Delivery & Continuous Deployment

Create the right build pipeline for each service

Integration &

perf tests

Build & unit

testsbeta Produser-svc

Integration &

perf tests

Build & unit

testsbeta gamma Prodcart-svc

AWS

CodeDeploy

Elastic

BeanstalkJenkins TravisCI CircleCIAWS

CodePipeline

Deployment Pace

cart-svc

checkout-svc

Timeline

Rollback

Phased Rollout

Elastic Beanstalk Application

Phased Rollout at Gilt Under the HoodBlue-Green Deployments with Elastic Beanstalk

Environment (0.0.1)

Instance Instance Instance

Environment (0.0.2)

Instance Instance Instance

ELB

Elastic Beanstalk Application

Phased Rollout at Gilt Under the HoodBlue-Green Deployments with Elastic Beanstalk

Environment (0.0.1)

Instance Instance Instance

Environment (0.0.2)

Instance Instance Instance

ELB

Elastic Beanstalk Application

Phased Rollout at Gilt Under the HoodBlue-Green Deployments with Elastic Beanstalk

Environment (0.0.1)

Instance Instance Instance

Environment (0.0.2)

Instance Instance Instance

ELB

Elastic Beanstalk Application

Phased Rollout at Gilt Under the HoodBlue-Green Deployments with Elastic Beanstalk

Environment (0.0.1)

Instance Instance Instance

Environment (0.0.2)

Instance Instance Instance

ELB

Elastic Beanstalk Application

Phased Rollout at Gilt Under the HoodBlue-Green Deployments with Elastic Beanstalk

Environment (0.0.2)

Instance Instance Instance

ELB

Phased Rollout at Gilt Under the Hood

CodeDeploy Application

Canary Deployment Group Production Deployment GroupDevelopment Deployment Group

InstanceInstanceInstanceInstance

Production ELBDevelopment ELB

Gilt Production

Users

Production DNSDevelopment DNS

Testing in Production with CodeDeploy

Phased Rollout at Gilt Under the Hood

CodeDeploy Application

Canary Deployment Group Production Deployment GroupDevelopment Deployment Group

InstanceInstanceInstanceInstance

Production ELBDevelopment ELB

Gilt Production

Production DNS

UsersEngineers

Development DNS

Testing in Production with CodeDeploy

Promote Promote

Phased Rollout at Gilt Under the Hood

CodeDeploy Application

Canary Deployment Group Production Deployment GroupDevelopment Deployment Group

InstanceInstanceInstanceInstance

Production ELBDevelopment ELB

Gilt Production

UsersEngineers & QA

Production DNSDevelopment DNS

Testing in Production with CodeDeploy

Promote Promote

Dev, please

Monitoring & Logging

Monitoring

• Instance level: Memory usage, GC Cycles

• Service level: RPMs, Latency, Error rate

• Universe level: Endpoint response times

Monitoring at Gilt

Monitoring at Gilt

Remote calls response time

Submitted orders

CloudWatch

Logging

• Push your logs somewhere

S3 Elasticsearch CloudWatch Logs

Logging

• Push your logs somewhere

• Have visibility on the whole life cycle

Deployment => instance startup => application => termination

S3 Elasticsearch CloudWatch

Logging

• Push your logs somewhere

• Have visibility on the whole life cycle

Deployment => instance startup => application => termination

• Separate your logs by service and by instance

S3 Elasticsearch CloudWatch

Logging At Gilt

Service

EC2 Instance

Service “X” VM

EC2 Instance

Service “X” VM

CloudWatch Log Group

Log Stream

Log Stream

Errors!

Aggregation Layer

Log Aggregation

CloudWatch Log Group

Log Stream

Log Stream

CloudWatch Log Group

Log Stream

Log Stream

CloudWatch Log Group

Log Stream

Log Stream

CloudWatch Log Group

Log Stream

Log Stream

CloudWatch Log Group

Log Stream

Log Stream

CloudWatch Log Group

Log Stream

Log Stream

CloudWatch Log Group

Log Stream

Log Stream

CloudWatch Log Group

Log Stream

Log Stream

CloudWatch Log Group

Log Stream

Log Stream

CloudWatch Log Group

Log Stream

Log Stream

+ + + + + =

Cleanup Service

EC2 Instance

Service “X” VM

EC2 Instance

Service “X” VM

CloudWatch Log Group

Log Stream

Log Stream

Service

EC2 Instance

Service “X” VM

EC2 Instance

Service “X” VM

CloudWatch Log Group

Log Stream

Log Stream

Log Aggregation At Gilt

Service

EC2 Instance

Service “X” VM

EC2 Instance

Service “X” VM

CloudWatch Log Group

Log Stream

Log Stream

LambdaAWS Elasticsearch

Kibana

Lambda

Delete!

Index!

Timer

Good Citizenship

Service Consumer Tenets

• Design for failure

• Plan to be throttled

• Plan to retry (w/ exponential backoff)

• Degrade gracefully

• Cache when appropriate

Service Provider Tenets

• Publish standard metrics

• Publish logs for aggregation

• Be prepared to throttle

• Implementation details are private

• Maintain backwards compatibility

Standardize Logging

2016-09-02T15:03:24+00:00 ui-svc INFO [uuid-123] ...

2016-09-02T15:03:25+00:00 catalog-svc INFO [uuid-123] ...

2016-09-02T15:03:26+00:00 checkout-svc ERROR [uuid-123] ...

2016-09-02T15:03:27+00:00 payment-svc INFO [uuid-123] ...

2016-09-02T15:03:27+00:00 shipping-svc INFO [uuid-123] ...

ui-svc catalog-svc checkout-

svc

shipping-

svc

payment-

svc

request correlation id:

“uuid-123”correlation id:

“uuid-123”

Parallelize

Aggregate

Service

Service 1

Service 2

Service 3

Single request parallel request

Parallelize

for {

a <- callServiceA()

b <- callServiceB()

c <- callServiceC(a, b)

} yield c

val futureA = callServiceA()

val futureB = callServiceB()

for {

a <- futureA

b <- futureB

c <- callServiceC(a, b)

} yield c

Degrade Gracefully

checkout-svc

Downstream

Service 1

Downstream

Service 2

Downstream

Service 3

Single request

Request

response

Fast response

Cache Locally

Remote

Service

Application

Runtime

Local

Cache

Cache Externally

Remote

Service

Local

Service

External Cache

Use a Read-Through Cache

Local

Service

Cache

Remote

Service

API Gateway

Private Services

Putting It All Together

Public Services

Web front-end Mobile apps

Recap

• Organisation & Ownership

• Architecture & Decomposition

• Data Management

• Discovery

• API Management

• Deployment

• Logging and Monitoring

• Good citizenship

Thank you!Thank You!

Remember to complete

your evaluations!

Related Sessions