microservices and serverless for mega startups - devops il meetup

Microservices andServerless forMegaStartupsBoazZiniman,TechnicalEvangelistAmazonWebServices

@ziniman

DevOpsIsrael– December2017

About me

• Technical Evangelist at AWS

• Events & Meetsup

• Webinars

• Blogs

• Community

• Social Media

• A long history of Web Development, IT and Operations

• AWS Customer since 2009

Agenda

• Breaking the Monolith

• An Introduction to Microservices

• Who needs servers?

• Lambda and Serverless on AWS

• 5 Principals for running Microservices at scale

Migrating from Monolith

to Microservice

“The Monolith”

Challenges with monolithic software

Long Build/Test/Release

Cycles

(who broke the build?)

Operations

is a nightmare

(module X is failing,

who’s the owner?)

Difficult to

scale

New releases

take months

Long time to add

new features

Architecture is

hard to maintain

and evolve

Lack of innovation

Frustrated customers

Lack of agility

Monolith development lifecycle

releasetestbuild

delivery pipeline

app

(aka the “monolith”)developers

Too much software coupling

Shared libraries

Shared data

Evolving towards microservices

“IMG_1760” by Robert Couse-Baker. No alterations other than cropping. https://www.flickr.com/photos/29233640@N07/14859431605/

Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

“service-orientedarchitecturecomposed ofloosely coupled elementsthat havebounded contexts”

Adrian Cockcroft (VP of Cloud Architecture @ AWS, former Cloud Architect at Netflix)


Services communicate with

each other over the network



You can update the services

independently; updating one

service doesn’t require

changing any other services.



Self-contained; you can

update the code without

knowing anything about the

internals of other

microservicesAdrian Cockcroft (VP of Cloud Architecture @

AWS, former Cloud Architect at Netflix)

“Do one thing, and do it well”

“Swiss Army” by by Jim Pennucci. No alterations other than cropping. https://www.flickr.com/photos/pennuja/5363518281/


“Tools” by Tony Walmsley: No alterations other than cropping. https://www.flickr.com/photos/twalmsley/6825340663/


“Do one thing, and do it well”

= 50 million deployments a year

Thousands of teams

× Microservice architecture

× Continuous delivery

× Multiple environments

(5708 per hour, or every 0.63 second)

ServersHow will the application

handle server hardware failure?

How can I control

access from my servers?

When should I decide to

scale out my servers?

When should I decide to

scale up my servers?

What size servers are

right for my budget?

How much remaining

capacity do my servers have?

(AAHHHHHHHHH!!)

What is

serverless?

Build and run applications

without thinking about servers

Let’s take a look at the evolution of computing

Physical Servers

Datacenters

Virtual Servers

Datacenters

Virtual Servers

in the cloud

Each progressive step was better

Physical Servers

Datacenters

Virtual Servers

Datacenters

Virtual Servers

in the cloud

• Higher utilization

• Faster provisioning speed

• Improved uptime

• Disaster recovery

• Hardware independence

• Trade CAPEX for OPEX

• More scale

• Elastic resources

• Faster speed and agility

• Reduced maintenance

• Better availability and fault

tolerance

But there are still limitations

Physical Servers

Datacenters

Virtual Servers

Datacenters

• Trade CAPEX for OPEX

• More scale

• Elastic resources

• Faster speed and agility

• Reduced maintenance

• Better availability and fault

tolerance

• Still need to administer

virtual servers

• Still need to manage

capacity and utilization

• Still need to size

workloads

• Still need to manage

availability, fault tolerance

• Still expensive to run

intermittent jobs

Virtual Servers

in the cloud

Evolving

to serverlessSERVERLESS

Virtual servers

in the cloud

Physical servers

in datacenters

Virtual servers

in datacenters

No server is easier to manage than any server

All of these responsibilities

go away

Provisioning and utilization

Availability and fault tolerance

Scaling

Operations and management

Serverless with AWS Lambda

EVENT DRIVEN CONTINUOUS SCALING PAY BY USAGE

Deliver on demand, never pay for idle

No servers to provision

or manage

Scales with usage

Never pay for idle Availability and fault

tolerance built in

Benefits of Lambda and serverless compute

How it works?

Using AWS Lambda

Bring your own code

• Node.js, Java, Python,

C#

• Bring your own libraries

(even native ones)

Simple resource model

• Select power rating from

128 MB to 1.5 GB

• CPU and network

allocated proportionately

Flexible use

• Synchronous or

asynchronous

• Integrated with other

AWS services

Flexible authorization

• Securely grant access to

resources and VPCs

• Fine-grained control for

invoking your functions

…and GO 3 GB

Using AWS Lambda

Authoring functions

• WYSIWYG editor or

upload packaged .zip

• Third-party plugins

(Eclipse, Visual Studio)

Monitoring and logging

• Metrics for requests,

errors, and throttles

• Built-in logs to Amazon

CloudWatch Logs

Programming model

• Use processes, threads,

/tmp, sockets normally

• AWS SDK built in

(Python and Node.js)

Stateless

• Persist data using

external storage

• No affinity or access to

underlying infrastructure

Amazon S3 Amazon DynamoDB

Amazon Kinesis

AWS CloudFormation

AWS CloudTrail Amazon CloudWatch

Amazon Cognito

Amazon SNSAmazonSES

Cron events

DATA STORES ENDPOINTS

CONFIGURATION REPOSITORIES EVENT/MESSAGE SERVICES

Example event sources that trigger AWS Lambda

… and a few more with more on the way!

AWS CodeCommit

AmazonAPI Gateway

AmazonAlexa

AWS IoT AWS Step

Functions

Use cases

Common use cases

Web

Applications

• Static

websites

• Complex web

apps

• Packages for

Flask and

Express

Data

Processing

• Real time

• MapReduce

• Batch

Chatbots

• Powering

chatbot logic

Backends

• Apps &

services

• Mobile

• IoT

</></>

Amazon

Alexa

• Powering

voice-enabled

apps

• Alexa Skills Kit

IT

Automation

• Policy engines

• Extending

AWS services

• Infrastructure

management

Common use cases

Web Applications and Backends

Internet

Mobile Apps

Websites

Services

AWS Lambda function

AWS

API Gateway Cache

All publicly accessible endpoints

Amazon CloudWatch Monitoring

Amazon CloudFront

AmazonS3 AWS Lambda function

AWS Lambda function

Amazon DynamoDB

Web

Applications

• Static

websites

• Complex web

apps

• Packages for

Flask and

Express

Backends

• Apps &

services

• Mobile

• IoT

</></>

Common use cases

Web

Applications

• Static

websites

• Complex web

apps

• Packages for

Flask and

Express

Data

Processing

• Real time

• MapReduce

• Batch

Chatbots

• Powering

chatbot logic

Backends

• Apps &

services

• Mobile

• IoT

</></>

Amazon

Alexa

• Powering

voice-enabled

apps


IT

Automation

• Policy engines

• Extending

AWS services

• Infrastructure

management

Lambda + S3

Common use cases

Web

Applications

• Static

websites

• Complex web

apps

• Packages for

Flask and

Express

Data

Processing

• Real time

• MapReduce

• Batch

Chatbots

• Powering

chatbot logic

Backends

• Apps &

services

• Mobile

• IoT

</></>

Amazon

Alexa

• Powering

voice-enabled

apps


IT

Automation

• Policy engines

• Extending

AWS services

• Infrastructure

management

Customers innovating with serverless

@parallax

1. User visits

HTTPS

GET /

S3 with

CloudFront

2. CSS & JS

HTTPS

GET

*.js

*.css

S3 with

CloudFront

index.html

3. Lang detect

API Gateway

w/

Lambda

HTTPS

GET

/users/

country

HTTPS

POST

/users/

update

4. User fills details

API Gateway

w/

Lambda

Inserted into

DynamoDB

PAGE LIFECYCLE

5.

6.

5. FB Login

(optional)

Hits Facebook

Hosted Endpoint

via Facebook

Javascript SDK

6. YouTube

YouTube iframe

7. Start recording

API Gateway w/

Lambda

Responds with

S3 upload token

Uploads directly

to S3 bucket

over HTTPS

using token

8. Upload recording

HTTPS

GET

api.facebook.com

HTTPS

GET

youtube.com

{

“email”: “xyz”,

“profile_id”: 123

}

HTTPS

GET

/recordings/token

HTTPS

POST

bucket.s3.amazon/UID/

recording-X.mp3

PAGE LIFECYCLE

9. Submit details

API Gateway w/

Lambda

10. Artwork gen

HTTPS

POST

/users/generate_

artwork

API Gateway w/

Lambda.

Create image,

put on S3, return

S3 image URL

11. Artwork display

CloudFront w/ S3

HTTPS

GET

/domain.com/UID.png

AddThis.js is

populated with

the share texts,

and includes the

S3 URL

HTTP

GET

addthis.com/file.js

HTTPS

POST

/users/

update

{ url:

“domain.com/UID

.png” }

12. Social share 13. Social share

HTTPS

GET

https://twitter.com/intent/

tweet?text=XYZ

Directly hits the

social media

service

PAGE LIFECYCLE

Principle 1

Microservices only

rely on each other’s

public API

“Contracts” by NobMouse. No alterations other than cropping.

https://www.flickr.com/photos/nobmouse/4052848608/

Image used with permissions under Creative Commons

license 2.0, Attribution Generic License

(https://creativecommons.org/licenses/by/2.0/)

Microservice A Microservice B

public API

Principle 1: Microservices only rely on each

other’s public API

public API

public API




public API

public API public API

Nope!




storeRestaurant (id, name, cuisine)

storeRestaurant (id, name, cuisine)

storeRestaurant (id, name, arbitrary_metadata)

addReview (restaurantId, rating, comments)

storeRestaurant (id, name, arbitrary_metadata)

addReview (restaurantId, rating, comments)

Version 1.0.0

Version 1.1.0

Version 2.0.0

public API

Microservice A



Principle 2

Use the right tool for the

job

“Tools #2” by Juan Pablo Olmo. No alterations other than cropping.

https://www.flickr.com/photos/juanpol/1562101472/

Image used with permissions under Creative Commons license

2.0, Attribution Generic License


public API


Amazon

Elasticsearch

Service

RDS

Aurora

DynamoDB

public API

Principle 2: Use the right tool for the job

public API

RDS

Aurora


Amazon

Elasticsearch

Service

DynamoDB

public API

Principle 2: Use the right tool for the job

Principle 3

Secure Your Services

“security” by Dave Bleasdale. No alterations other than cropping.

https://www.flickr.com/photos/sidelong/3878741556/

Image used with permissions under Creative Commons license 2.0,

Attribution Generic License


Principle 3: Secure Your Services

• Defense-in-depth

• Network level (e.g. VPC, Security Groups, TLS)

• Server/container-level

• App-level

• IAM policies

• Gateway (“Front door”)

• API Throttling

• Authentication & Authorization

• Client-to-service, as well as service-to-service

• API Gateway: custom Lambda authorizers

• IAM-based Authentication

• Token-based auth (JWT tokens, OAuth 2.0)

• Secrets management

• S3 bucket policies + KMS + IAM

• Open-source tools (e.g. Vault, Keywhiz)

APIGateway

Principle 4

More than just

technology

transformation

“rowing on the river in Bedford” by Matthew Hunt.

No alterations other than cropping.

https://www.flickr.com/photos/mattphotos/19189529/

Image used with permissions under Creative Commons license 2.0,

Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

“Any organization that designs a system

will produce a design whose structure is a

copy of the organization’s communication

structure.”

Melvin E. Conway, 1967

Conway’s Law

Silo’d functional teams à silo’d application architectures

Image from Martin Fowler’s article on microservices, at

http://martinfowler.com/articles/microservices.html


Permission to reproduce: http://martinfowler.com/faq.html

Cross functional teams à self-contained services






(“Two-pizza teams” at Amazon)










(“Two-pizza teams” at Amazon)

Full ownership

Full accountability

Aligned incentives

Principle 5

Automate Everything

“Robot”byRobin Zebrowski.Noalterationsotherthancropping.https://www.flickr.com/photos/firepile/438134733/

Image usedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)§

Focused agile teams

delivery pipeline service

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

releasetestbuild

2-pizza team

Principle 6: Automate everything

AWS

CodeCommit

AWS

CodePipeline

AWS

CodeDeploy

EC2 ELBAuto

ScalingLambdaECS

DynamoDBRDS ElastiCache SQS SWF

SES SNS

API GatewayCloudWatch Cloud Trail

KinesisElastic

Beanstalk

It’s a journey…

Expect challenges along the way…

• Understanding of business domains

• Coordinating txns across multiple

services

• Eventual Consistency

• Service discovery

• Lots of moving parts requires increased

coordination

• Complexity of testing / deploying /

operating a distributed system

• Cultural transformation

Benefits of Microservices

Rapid

Build/Test/Release

Cycles

Clear ownership and

accountability

Easier to scale

each individual

microservice

New releases

take minutes

Short time to add

new features

Easier to

maintain and

evolve system

Faster innovation

Delighted customers

Increased agility

AWS resources:

• Microservices without the Servers

https://aws.amazon.com/blogs/compute/

microservices-without-the-servers

• Microservices with ECS:

https://aws.amazon.com/blogs/compute/using-amazon-

api-gateway-with-microservices-deployed-on-amazon-

ecs/

• Serverless Service Discovery:

https://aws.amazon.com/blogs/developer/

serverless-service-discovery-part-1-get-started/

• ECS Service Discovery:

https://aws.amazon.com/blogs/compute/

service-discovery-an-amazon-ecs-reference-

architecture/

• Serverless Webapp - Reference Architecture:

https://github.com/awslabs/lambda-refarch-webapp

• Zombie Microservices Workshop:

https://github.com/awslabs/aws-lambda-zombie-

workshop

Popular open-source tools:

• Serverless – http://serverless.com

• Apex - http://apex.run/

https://aws.amazon.com/devops/

Additional Resources

Boaz Ziniman@ziniman

microservices and serverless for mega startups - devops il meetup

Technology