Microservices andServerless forMegaStartupsBoazZiniman,TechnicalEvangelistAmazonWebServices
@ziniman
DevOpsIsrael– December2017
About me
• Technical Evangelist at AWS
• Events & Meetsup
• Webinars
• Blogs
• Community
• Social Media
• A long history of Web Development, IT and Operations
• AWS Customer since 2009
Agenda
• Breaking the Monolith
• An Introduction to Microservices
• Who needs servers?
• Lambda and Serverless on AWS
• 5 Principals for running Microservices at scale
Challenges with monolithic software
Long Build/Test/Release
Cycles
(who broke the build?)
Operations
is a nightmare
(module X is failing,
who’s the owner?)
Difficult to
scale
New releases
take months
Long time to add
new features
Architecture is
hard to maintain
and evolve
Lack of innovation
Frustrated customers
Lack of agility
Monolith development lifecycle
releasetestbuild
delivery pipeline
app
(aka the “monolith”)developers
Evolving towards microservices
“IMG_1760” by Robert Couse-Baker. No alterations other than cropping. https://www.flickr.com/photos/29233640@N07/14859431605/
Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)
“service-orientedarchitecturecomposed ofloosely coupled elementsthat havebounded contexts”
Adrian Cockcroft (VP of Cloud Architecture @ AWS, former Cloud Architect at Netflix)
“service-orientedarchitecturecomposed ofloosely coupled elementsthat havebounded contexts”
Services communicate with
each other over the network
Adrian Cockcroft (VP of Cloud Architecture @ AWS, former Cloud Architect at Netflix)
“service-orientedarchitecturecomposed ofloosely coupled elementsthat havebounded contexts”
You can update the services
independently; updating one
service doesn’t require
changing any other services.
Adrian Cockcroft (VP of Cloud Architecture @ AWS, former Cloud Architect at Netflix)
“service-orientedarchitecturecomposed ofloosely coupled elementsthat havebounded contexts”
Self-contained; you can
update the code without
knowing anything about the
internals of other
microservicesAdrian Cockcroft (VP of Cloud Architecture @
AWS, former Cloud Architect at Netflix)
“Do one thing, and do it well”
“Swiss Army” by by Jim Pennucci. No alterations other than cropping. https://www.flickr.com/photos/pennuja/5363518281/
Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)
“Tools” by Tony Walmsley: No alterations other than cropping. https://www.flickr.com/photos/twalmsley/6825340663/
Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)
“Do one thing, and do it well”
= 50 million deployments a year
Thousands of teams
× Microservice architecture
× Continuous delivery
× Multiple environments
(5708 per hour, or every 0.63 second)
ServersHow will the application
handle server hardware failure?
How can I control
access from my servers?
When should I decide to
scale out my servers?
When should I decide to
scale up my servers?
What size servers are
right for my budget?
How much remaining
capacity do my servers have?
(AAHHHHHHHHH!!)
Let’s take a look at the evolution of computing
Physical Servers
Datacenters
Virtual Servers
Datacenters
Virtual Servers
in the cloud
Each progressive step was better
Physical Servers
Datacenters
Virtual Servers
Datacenters
Virtual Servers
in the cloud
• Higher utilization
• Faster provisioning speed
• Improved uptime
• Disaster recovery
• Hardware independence
• Trade CAPEX for OPEX
• More scale
• Elastic resources
• Faster speed and agility
• Reduced maintenance
• Better availability and fault
tolerance
But there are still limitations
Physical Servers
Datacenters
Virtual Servers
Datacenters
• Trade CAPEX for OPEX
• More scale
• Elastic resources
• Faster speed and agility
• Reduced maintenance
• Better availability and fault
tolerance
• Still need to administer
virtual servers
• Still need to manage
capacity and utilization
• Still need to size
workloads
• Still need to manage
availability, fault tolerance
• Still expensive to run
intermittent jobs
Virtual Servers
in the cloud
Evolving
to serverlessSERVERLESS
Virtual servers
in the cloud
Physical servers
in datacenters
Virtual servers
in datacenters
No server is easier to manage than any server
All of these responsibilities
go away
Provisioning and utilization
Availability and fault tolerance
Scaling
Operations and management
No servers to provision
or manage
Scales with usage
Never pay for idle Availability and fault
tolerance built in
Benefits of Lambda and serverless compute
Using AWS Lambda
Bring your own code
• Node.js, Java, Python,
C#
• Bring your own libraries
(even native ones)
Simple resource model
• Select power rating from
128 MB to 1.5 GB
• CPU and network
allocated proportionately
Flexible use
• Synchronous or
asynchronous
• Integrated with other
AWS services
Flexible authorization
• Securely grant access to
resources and VPCs
• Fine-grained control for
invoking your functions
…and GO 3 GB
Using AWS Lambda
Authoring functions
• WYSIWYG editor or
upload packaged .zip
• Third-party plugins
(Eclipse, Visual Studio)
Monitoring and logging
• Metrics for requests,
errors, and throttles
• Built-in logs to Amazon
CloudWatch Logs
Programming model
• Use processes, threads,
/tmp, sockets normally
• AWS SDK built in
(Python and Node.js)
Stateless
• Persist data using
external storage
• No affinity or access to
underlying infrastructure
Amazon S3 Amazon DynamoDB
Amazon Kinesis
AWS CloudFormation
AWS CloudTrail Amazon CloudWatch
Amazon Cognito
Amazon SNSAmazonSES
Cron events
DATA STORES ENDPOINTS
CONFIGURATION REPOSITORIES EVENT/MESSAGE SERVICES
Example event sources that trigger AWS Lambda
… and a few more with more on the way!
AWS CodeCommit
AmazonAPI Gateway
AmazonAlexa
AWS IoT AWS Step
Functions
Common use cases
Web
Applications
• Static
websites
• Complex web
apps
• Packages for
Flask and
Express
Data
Processing
• Real time
• MapReduce
• Batch
Chatbots
• Powering
chatbot logic
Backends
• Apps &
services
• Mobile
• IoT
</></>
Amazon
Alexa
• Powering
voice-enabled
apps
• Alexa Skills Kit
IT
Automation
• Policy engines
• Extending
AWS services
• Infrastructure
management
Common use cases
Web Applications and Backends
Internet
Mobile Apps
Websites
Services
AWS Lambda function
AWS
API Gateway Cache
All publicly accessible endpoints
Amazon CloudWatch Monitoring
Amazon CloudFront
AmazonS3 AWS Lambda function
AWS Lambda function
Amazon DynamoDB
Web
Applications
• Static
websites
• Complex web
apps
• Packages for
Flask and
Express
Backends
• Apps &
services
• Mobile
• IoT
</></>
Common use cases
Web
Applications
• Static
websites
• Complex web
apps
• Packages for
Flask and
Express
Data
Processing
• Real time
• MapReduce
• Batch
Chatbots
• Powering
chatbot logic
Backends
• Apps &
services
• Mobile
• IoT
</></>
Amazon
Alexa
• Powering
voice-enabled
apps
• Alexa Skills Kit
IT
Automation
• Policy engines
• Extending
AWS services
• Infrastructure
management
Lambda + S3
Common use cases
Web
Applications
• Static
websites
• Complex web
apps
• Packages for
Flask and
Express
Data
Processing
• Real time
• MapReduce
• Batch
Chatbots
• Powering
chatbot logic
Backends
• Apps &
services
• Mobile
• IoT
</></>
Amazon
Alexa
• Powering
voice-enabled
apps
• Alexa Skills Kit
IT
Automation
• Policy engines
• Extending
AWS services
• Infrastructure
management
1. User visits
HTTPS
GET /
S3 with
CloudFront
2. CSS & JS
HTTPS
GET
*.js
*.css
S3 with
CloudFront
index.html
3. Lang detect
API Gateway
w/
Lambda
HTTPS
GET
/users/
country
HTTPS
POST
/users/
update
4. User fills details
API Gateway
w/
Lambda
Inserted into
DynamoDB
PAGE LIFECYCLE
5.
6.
5. FB Login
(optional)
Hits Facebook
Hosted Endpoint
via Facebook
Javascript SDK
6. YouTube
YouTube iframe
7. Start recording
API Gateway w/
Lambda
Responds with
S3 upload token
Uploads directly
to S3 bucket
over HTTPS
using token
8. Upload recording
HTTPS
GET
api.facebook.com
HTTPS
GET
youtube.com
{
“email”: “xyz”,
“profile_id”: 123
}
HTTPS
GET
/recordings/token
HTTPS
POST
bucket.s3.amazon/UID/
recording-X.mp3
PAGE LIFECYCLE
9. Submit details
API Gateway w/
Lambda
10. Artwork gen
HTTPS
POST
/users/generate_
artwork
API Gateway w/
Lambda.
Create image,
put on S3, return
S3 image URL
11. Artwork display
CloudFront w/ S3
HTTPS
GET
/domain.com/UID.png
AddThis.js is
populated with
the share texts,
and includes the
S3 URL
HTTP
GET
addthis.com/file.js
HTTPS
POST
/users/
update
{ url:
“domain.com/UID
.png” }
12. Social share 13. Social share
HTTPS
GET
https://twitter.com/intent/
tweet?text=XYZ
Directly hits the
social media
service
PAGE LIFECYCLE
Principle 1
Microservices only
rely on each other’s
public API
“Contracts” by NobMouse. No alterations other than cropping.
https://www.flickr.com/photos/nobmouse/4052848608/
Image used with permissions under Creative Commons
license 2.0, Attribution Generic License
(https://creativecommons.org/licenses/by/2.0/)
Microservice A Microservice B
public API
Principle 1: Microservices only rely on each
other’s public API
public API
public API
Microservice A Microservice B
Principle 1: Microservices only rely on each
other’s public API
public API
public API public API
Nope!
Microservice A Microservice B
Principle 1: Microservices only rely on each
other’s public API
storeRestaurant (id, name, cuisine)
storeRestaurant (id, name, cuisine)
storeRestaurant (id, name, arbitrary_metadata)
addReview (restaurantId, rating, comments)
storeRestaurant (id, name, arbitrary_metadata)
addReview (restaurantId, rating, comments)
Version 1.0.0
Version 1.1.0
Version 2.0.0
public API
Microservice A
Principle 1: Microservices only rely on each
other’s public API
Principle 2
Use the right tool for the
job
“Tools #2” by Juan Pablo Olmo. No alterations other than cropping.
https://www.flickr.com/photos/juanpol/1562101472/
Image used with permissions under Creative Commons license
2.0, Attribution Generic License
(https://creativecommons.org/licenses/by/2.0/)
public API
Microservice A Microservice B
Amazon
Elasticsearch
Service
RDS
Aurora
DynamoDB
public API
Principle 2: Use the right tool for the job
public API
RDS
Aurora
Microservice A Microservice B
Amazon
Elasticsearch
Service
DynamoDB
public API
Principle 2: Use the right tool for the job
Principle 3
Secure Your Services
“security” by Dave Bleasdale. No alterations other than cropping.
https://www.flickr.com/photos/sidelong/3878741556/
Image used with permissions under Creative Commons license 2.0,
Attribution Generic License
(https://creativecommons.org/licenses/by/2.0/)
Principle 3: Secure Your Services
• Defense-in-depth
• Network level (e.g. VPC, Security Groups, TLS)
• Server/container-level
• App-level
• IAM policies
• Gateway (“Front door”)
• API Throttling
• Authentication & Authorization
• Client-to-service, as well as service-to-service
• API Gateway: custom Lambda authorizers
• IAM-based Authentication
• Token-based auth (JWT tokens, OAuth 2.0)
• Secrets management
• S3 bucket policies + KMS + IAM
• Open-source tools (e.g. Vault, Keywhiz)
APIGateway
Principle 4
More than just
technology
transformation
“rowing on the river in Bedford” by Matthew Hunt.
No alterations other than cropping.
https://www.flickr.com/photos/mattphotos/19189529/
Image used with permissions under Creative Commons license 2.0,
Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)
“Any organization that designs a system
will produce a design whose structure is a
copy of the organization’s communication
structure.”
Melvin E. Conway, 1967
Conway’s Law
Silo’d functional teams à silo’d application architectures
Image from Martin Fowler’s article on microservices, at
http://martinfowler.com/articles/microservices.html
No alterations other than cropping.
Permission to reproduce: http://martinfowler.com/faq.html
Silo’d functional teams à silo’d application architectures
Image from Martin Fowler’s article on microservices, at
http://martinfowler.com/articles/microservices.html
No alterations other than cropping.
Permission to reproduce: http://martinfowler.com/faq.html
Cross functional teams à self-contained services
Image from Martin Fowler’s article on microservices, at
http://martinfowler.com/articles/microservices.html
No alterations other than cropping.
Permission to reproduce: http://martinfowler.com/faq.html
Cross functional teams à self-contained services
Image from Martin Fowler’s article on microservices, at
http://martinfowler.com/articles/microservices.html
No alterations other than cropping.
Permission to reproduce: http://martinfowler.com/faq.html
Cross functional teams à self-contained services
(“Two-pizza teams” at Amazon)
Image from Martin Fowler’s article on microservices, at
http://martinfowler.com/articles/microservices.html
No alterations other than cropping.
Permission to reproduce: http://martinfowler.com/faq.html
Image from Martin Fowler’s article on microservices, at
http://martinfowler.com/articles/microservices.html
No alterations other than cropping.
Permission to reproduce: http://martinfowler.com/faq.html
Cross functional teams à self-contained services
(“Two-pizza teams” at Amazon)
Full ownership
Full accountability
Aligned incentives
Principle 5
Automate Everything
“Robot”byRobin Zebrowski.Noalterationsotherthancropping.https://www.flickr.com/photos/firepile/438134733/
Image usedwithpermissionsunderCreativeCommonslicense2.0,AttributionGenericLicense(https://creativecommons.org/licenses/by/2.0/)§
Focused agile teams
delivery pipeline service
releasetestbuild
releasetestbuild
releasetestbuild
releasetestbuild
releasetestbuild
releasetestbuild
2-pizza team
Principle 6: Automate everything
AWS
CodeCommit
AWS
CodePipeline
AWS
CodeDeploy
EC2 ELBAuto
ScalingLambdaECS
DynamoDBRDS ElastiCache SQS SWF
SES SNS
API GatewayCloudWatch Cloud Trail
KinesisElastic
Beanstalk
It’s a journey…
Expect challenges along the way…
• Understanding of business domains
• Coordinating txns across multiple
services
• Eventual Consistency
• Service discovery
• Lots of moving parts requires increased
coordination
• Complexity of testing / deploying /
operating a distributed system
• Cultural transformation
Benefits of Microservices
Rapid
Build/Test/Release
Cycles
Clear ownership and
accountability
Easier to scale
each individual
microservice
New releases
take minutes
Short time to add
new features
Easier to
maintain and
evolve system
Faster innovation
Delighted customers
Increased agility
AWS resources:
• Microservices without the Servers
https://aws.amazon.com/blogs/compute/
microservices-without-the-servers
• Microservices with ECS:
https://aws.amazon.com/blogs/compute/using-amazon-
api-gateway-with-microservices-deployed-on-amazon-
ecs/
• Serverless Service Discovery:
https://aws.amazon.com/blogs/developer/
serverless-service-discovery-part-1-get-started/
• ECS Service Discovery:
https://aws.amazon.com/blogs/compute/
service-discovery-an-amazon-ecs-reference-
architecture/
• Serverless Webapp - Reference Architecture:
https://github.com/awslabs/lambda-refarch-webapp
• Zombie Microservices Workshop:
https://github.com/awslabs/aws-lambda-zombie-
workshop
Popular open-source tools:
• Serverless – http://serverless.com
• Apex - http://apex.run/
https://aws.amazon.com/devops/
Additional Resources