serverless: an update from the field -...
TRANSCRIPT
AMI Hours to Months Patching Multi-Threaded Multi-Task
Per VM Per Hour
The World Of Virtual Machines
Container File Minutes to Days Versioning Multi-Threaded Single-Task
Per VM Per Hour
The World Of Containers
Code Microseconds to Seconds
Versioning Single-Threaded Single-Task
Per Memory/Second Per Request
Free Tier
The World of Lambda Functions
Low Cost
SecuritySimplicity Reliability
Scalability Low Latency
S3: Simple By Name, Simple By Nature
Mobile Apps
Websites
Services
Internet
API Gateway Cache
API Gateway
CloudWatch
Lambda Functions
EC2 Endpoints
Any Other Endpoint
Amazon API Gateway
Credit: Gerry Cranham/Fox Photos/Getty Images http://www.telegraph.co.uk/travel/destinations/europe/united-kingdom/england/london/galleries/The-history-of-the-Tube-in-pictures-150-years-of-London-Underground/1939-ticket-examin/
Credit: Gerry Cranham/Fox Photos/Getty Images http://www.telegraph.co.uk/travel/destinations/europe/united-kingdom/england/london/galleries/The-history-of-the-Tube-in-pictures-150-years-of-London-Underground/1939-ticket-examin/
1939 London Underground
Name
Create New User
Submit
Check the “Name” syntax is right (only letters and spaces)
Check the “Email” syntax is right ([email protected])
Create the new user using provided Name and Email
User Interface Actions
Name
Create New User
Submit
Check the “Name” syntax is right (only letters and spaces)
Check the “Email” syntax is right ([email protected])
Create the new user using provided Name and Email
User Interface ActionsObservers
Target Action
Generate thumbnails
Keep another database in sync
Process streaming data
Services FunctionsPlatform
Target Action
KinesisStream
DynamoDBTable
S3Bucket
Procedural Programming“This is a function!”
Reactive Programming“This is data binding!”
area = length x widtharea = length x width
A B
C
D
E
F
Who I need to notify(output events)
What I need to know(input events)
What I need to do(internal logic)
Monolith
Service
Service
μ
μ
μ
space (distribution, mobility)
time(
conc
urre
ncy,
late
ncy)
μ
μ
μ
μ
Distributedin Time and Space
[Jonas Bonér]
A
C
I
D
tomic
onsistent
isolated
urable
A
C
I
D
ssociative
commutative
dempotent
istributed
ACID 2.0
A B
C
D
E
F
Event-Driven Interactions replacing Distributed Transactionsusing Eventual Consistency
Differentiate readsfrom writes (commands),
as in the Command QueryResponsibility Segregation
(CQRS) pattern
Customers Orders
Warehouse Deliveries
Payments
Catalog
Event-Driven Interactions replacing Distributed Transactionsusing Eventual Consistency
The Reactive Manifesto
Asynchronouscommunicationis not enforced
by serverless architectures
Responsive
ResilientElastic
Message Driven
AWSLambdaUser Client
Application
Back End for a Web (HTML/JavaScript) or Mobile AppCalling the Lambda API directly
Web or Mobile Back End
DynamoDBTable
S3Bucket
AmazonCognito
AWSLambda
AmazonCognito
User ClientApplication
OAuth JWT
AmazonAPI Gateway
(HTTPS)
Back End for a Web (HTML/JavaScript) or Mobile AppUsing an API Gateway for authentication and decoupling
Web or Mobile Back EndVia API Gateway
DynamoDBTable
S3Bucket
AWSLambdaUser Client
Application
Using WebSockets instead of Web API to talk to the Back End
AWS IoT(MQTT over
WebSockets)
WebSockets Back End
DynamoDBTable
S3Bucket
AmazonCognito
MQTT orDevice
Shadow
Possible optimisations depending on your data flowand your acceptable latency
DataSource
KinesisStream
KinesisFirehoseKinesis
Analytics(improve & aggregate)
AWSLambda(batch)
S3Bucket
AWSLambda
(file)
Data Streams Processing
AWSLambda(batch)
AmazonCognito
User ClientApplication
Buffering Requests to the Back End to reduce Lambda invocations
KinesisStream
Buffered Back EndFor Mobile
AmazonSNS
DynamoDBTable
S3Bucket
AWSLambda(batch)
User ClientApplication
Using WebSockets instead of Web API to talk to the Back EndBuffering Requests to reduce Lambda invocations
AWS IoT(MQTT over
WebSockets)KinesisStream
Buffered Back EndFor Everything
MQTT orDevice Shadow
AmazonCognito
DynamoDBTable
S3Bucket
User ClientApplication
Using WebSockets instead of Web API to talk to the Back EndBuffering Requests to reduce Lambda invocations
Storing all customer interactions for further analysis
AWS IoT(MQTT over
WebSockets)Kinesis
Firehose
Buffered Back EndSaving Interactions
MQTT orDevice Shadow
AmazonCognito
AWSLambda
(file)S3
Bucket
DynamoDBTable
AWSLambda
DynamoDBTable
User ClientApplication
AmazonAPI Gateway
(HTTPS)
S3 Bucket(Website HostingRouting Rules)
For example, to generate image thumbnails on the first request,S3 LifeCycle Rules can delete content after N days
Using Amazon S3 asa Dynamic CacheAmazon
Cognito
To receive events from everywhere on the Internet
Webhooks are Callbacks for the Web
AWSLambda
AmazonAPI Gateway
InternetService
HTTP(S) GET or POST to https://domain/SOURCE/RANDOM-HOOK
Webhook
Other AWSServices
To generate eventsfor repositories
that don’t supportAWS Lambda natively
LogMonitor
AWSLambda
AWSLambda
AWSLambda
AmazonSNS
KinesisStream
LogFilesExternal
Application
Log Monitor
CloudWatchLogs or Metrics
AWSLambda
S3 object auto compress
S3 bucket per prefix statistics
DynamoDB provisioned throughput dynamic scaling
DynamoDB item support for geohash
Service Discovery
. . .
Extending Services with Your Own Functionalities
Client Application
Get Content Index
Get Content and Thumbnails
Upload Content with Metadata
Update Content Metadata
Get Content Metadata
Files (Multimedia)
Database
Build Thumbnails
Update Content Index
Extract and Update Metadata
Event: new or
updated file
Event: new or updated
content metadata
Functions
! User
ResourcesEvent-drivendesign
MediaSharing App
Client Application
Get Content Index (S3 API)
Get Content and Thumbnails
(S3 API)
Upload Contentwith Metadata
(S3 API)
Update Content Metadata
(DynamoDB API)
Get Content Metadata (DynamoDB API)
S3 Bucket
DynamoDB Table
Build Thumbnails (Lambda function)
Update Content Index
(Lambda function)
Extract and Update Metadata
(Lambda function)
Event: new or
updated file
Event: new or updated
content metadata
! User
Choosingthe technology
MediaSharing App
Client Application
Get Content Index (S3 API)
Get Content and Thumbnails
(S3 API)
Upload Contentwith Metadata
(S3 API)
Update Content Metadata
(DynamoDB API)
Get Content Metadata(DynamoDB API)
S3 Bucket
DynamoDB Table
Build Thumbnails (Lambda function)
Update Content Index
(Lambda function)
Extract and Update Metadata
(Lambda function)
Event: new or
updated file
Event: new or updated
content metadata
! User
Amazon Cognito
Addingimplementation
details
MediaSharing App
Client Application
Get Content Index (S3 API)
Get Content and Thumbnails
(S3 API)
Upload Contentwith Metadata
(S3 API)
Update Content Metadata
(DynamoDB API)
Get Content Metadata(DynamoDB API)
S3 Bucket
DynamoDB Table
Build Thumbnails (Lambda function)
Update Content Index
(Lambda function)
Extract and Update Metadata
(Lambda function)
Event: new or
updated file
Event: new or updated
content metadata
! User
Amazon Cognito
From featureviewpoint…
MediaSharing App
Client Application
GET Object S3 API
PUT Object S3 API
UpdateItem DynamoDB API
GetItem DynamoDB API
S3 Bucket
DynamoDB Table
buildThumbnails (Lambda function)
updateContentIndex (Lambda function)
extractAndUpdateMetadata (Lambda function)
Event: new or
updated file
Event: new or updated
content metadata
! User
Amazon Cognito
Amazon S3
Amazon DynamoDB
AWS Lambda…To the technology
viewpoint
MediaSharing App
Client Application
GET Object S3 API
PUT Object S3 API
UpdateItem DynamoDB API
GetItem DynamoDB API
S3 Bucket
DynamoDB Table
buildThumbnails (Lambda function)
updateContentIndex (Lambda function)
extractAndUpdateMetadata (Lambda function)
Event: new or
updated file
Event: new or updated
content metadata
! User
Amazon Cognito
Amazon S3
Amazon DynamoDB
AWS LambdaSome eventsare correlated
MediaSharing App
Client Application
GET Object S3 API
PUT Object S3 API
UpdateItem DynamoDB API
GetItem DynamoDB API
S3 Bucket
DynamoDB Table
contentUpdated (Lambda function)
buildThumbnails extractAndUpdateMetadata
updateContentIndex (Lambda function)
Event: new or
updated file
! User
Amazon Cognito
Amazon S3 AWS Lambda
Amazon DynamoDB
Event: new or updated
content metadata
Some functionscan be tied together
MediaSharing App
Client Application
GET Object S3 API
PUT Object S3 API
UpdateItem DynamoDB API
GetItem DynamoDB API
S3 Bucket
DynamoDB Table
contentUpdated (Lambda function)
buildThumbnails extractAndUpdateMetadata
updateContentIndex (Lambda function)
Event: new or
updated file
! User
Amazon Cognito
Amazon S3 AWS Lambda
Amazon DynamoDB
Event: new or updated
content metadata
What if I want todelete content?
MediaSharing App
Client Application
GET Object S3 API
PUT Object S3 API
UpdateItem DynamoDB API
GetItem DynamoDB API
S3 Bucket
DynamoDB Table
contentUpdated (Lambda function)
buildThumbnails extractAndUpdateMetadata
deleteMetadata
updateContentIndex (Lambda function)
Event: new, updated,
or deleted file
! User
Amazon Cognito
Amazon S3 AWS Lambda
Amazon DynamoDB
DELETE Object S3 API
Event: new or updated
content metadata
REST to the rescue,the events flow isalmost unchanged
MediaSharing App
Client Application
GET Object S3 API
PUT Object S3 API
UpdateItem DynamoDB API
GetItem DynamoDB API
S3 Bucket
DynamoDB Table
contentUpdated (Lambda function)
buildThumbnails extractAndUpdateMetadata
deleteMetadata
updateContentIndex (Lambda function)
Event: new, updated,
or deleted file
! User
Amazon Cognito
Amazon S3 AWS Lambda
Amazon DynamoDB
DELETE Object S3 API
Event: new or updated
content metadata
From 8 functions to 2,using 5 native API calls
MediaSharing App
Devs can benefit from:
event-driven design
architectural patterns
integrating new products
architecture simplification
customization of the platform
Serverless Architectures are not just simplifying Ops