(mbl302) building scalable, serverless mobile & iot backends w/aws lambda
TRANSCRIPT
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ajay Nair, Senior Product Manager, AWS Lambda
Olivier Klein 奧樂凱, Solutions ArchitectOctober 2015
MBL302
Build Scalable, Serverless
Mobile & IoT Back Ends with AWS Lambda
What to Expect from the Session
• A brief introduction to AWS Lambda
• How to use Amazon Cognito & Amazon Mobile Analytics
• Build an Amazon API Gateway and AWS Lambda CRUD
back end with DynamoDB
• Leverage AWS Lambda to power an event-driven mobile
back end
• Push and alert mobile apps through Amazon SNS
• See how Easy Ten put this approach to work
AWS Lambda
COMPUTE SERVICE
EVENT DRIVEN
Run code
without thinking
about servers
Code only runs
when it needs to
run
AWS Lambda: Capabilities
BRING YOUR OWN CODE COMPUTE “POWER LEVELS”
FLEXIBLE INVOCATION PATHS GRANULAR PERMISSIONS CONTROL
Multiple ways to put Lambda to work
AWS
CloudFormation
custom
resources
… and the list will
continue to grow!
Amazon Echo
skillsAmazon SWF
tasks
Customized
notifications with
Amazon SNS
Amazon Cognito
triggers
Amazon S3
triggers
Amazon
Dynamo DB
triggers
Amazon
Kinesis
processors
Microservices
with API
Gateway
Back-end wish list
What it does
User administration
Content storage
Push notifications
Analyze user behavior
Custom app logic
How it behaves
Cost follows usage
Minimal undifferentiated heavy lifting
Iterative development
Reduced time to market
Instant scale
Reliable and secure
Amazon Cognito
Authenticate & sync
Amazon Mobile Analytics
Analyze user behavior
AWS Lambda
Run business logic
Amazon S3
Amazon DynamoDB
Store content
Store data
Amazon SNS mobile push
notifications
Send push notifications
Back-end architecture on AWS
Mobile SDK
Amazon API Gateway
Sample app: “Find-a-Like”
• Premise: Create a profile with interests
and get notified when like-minded users
are nearby
• Functionalities:
• Create a profile with interests and upload
content
• Track location continuously
• Notify when users with similar interests are
close by
• Log and analyze app usage
Let’s think in layers
Create profile, upload
content, and track usage
Track location and user
interests
Match and alert users
App-centric
“You”
Activity-centric
“What you do”
User base-centric
“Them & me”
1
2
3
Cognito
Mobile Analytics
SNS Mobile Push
Kinesis Recorder DynamoDB Mapper S3 Transfer Manager
SQS Client
AWS global infrastructure (regions, Availability Zones, edge locations)
Core building
block services
Mobile-optimized
connectors
Your mobile app
AWS Mobile SDK (iOS, Android, Unity, Xamarin)
Compute Storage Networking Analytics Databases
Integrated SDK
Lambda
AWS Mobile SDKs
Security model for AWS API calls
Mobile client
IAM PermissionsAWS Security
Token Service
1. Request token
2. Receive temporary
credentials
3. Sign API request
with temporary token
AWS service APIs
4. Make API request
against AWS service API
Authenticate your user: Amazon Cognito
• Generate temporary credentials
and enforce rotation to limit
credential lifetime
• Authenticate user through third-party
authentication provider
• Unique users across multiple
devices and identity providers
• Allows anonymous user access
• Enables security best practices
through IAM roles
Use Cognito for authentication on iOS
//Create and configure Cognito credentials provider
AWSCognitoCredentialsProvider *credentialsP = [AWSCognitoCredentialsProvidercredentialsWithRegionType:AWSRegionUSEast1 accountId:@"0123456789” identityPoolId:@”us-east-1:beeeeeef-beef-beef-beef-beef” unauthRoleArn:@"arn:aws:iam::0123456789:role/Unauth” authRoleArn:@"arn:aws:iam::0123456789:role/Auth"];
//Set Cognito as default credentials provider for all AWS service calls
AWSServiceConfiguration *configuration = [AWSServiceConfigurationconfigurationWithRegion:AWSRegionUSEast1 credentialsProvider:credentialsP];
[AWSServiceManager defaultServiceManager].defaultServiceConfiguration = configuration;
Create your profile: Cognito Sync
• Create your app profile and save it
locally in the Cognito data store
• Cognito will synchronize the data sets
across all your user’s devices
• Cognito data sets are key/value pairs
AWSCognito *syncClient = [AWSCognito defaultCognito];
AWSCognitoDataset *subs = [syncClientopenOrCreateDataset:@”UserProfile"];
[dataset setString:”Oli" forKey:@”name"];[dataset setString:”50km" forKey:@”interestRadius"];
[dataset synchronize];
Upload a profile picture: S3 Transfer Utility
• Amazon S3 to store and share UGC
directly from the mobile device
• S3 Transfer Utility provides:
• Ability to continue transferring data in
the background when your app is
not running
• Ability to upload binary data instead
of having to first save it as a file on
the device
Amazon S3
S3 Transfer Utility: iOS code
NSData *dataToUpload = // The data to upload
AWSS3TransferUtility *transferUtility = [AWSS3TransferUtility defaultS3TransferUtility];
[[transferUtility uploadData:dataToUpload
bucket:@"YourBucketName"
key:@"YourObjectKeyName"
contentType:@"text/plain"
expression:expression
completionHander:completionHandler] continueWithBlock:^id(AWSTask *task) {
if (task.result) {
AWSS3TransferUtilityUploadTask *uploadTask = task.result;
// Do something with uploadTask
}
}
Track app usage: Amazon Mobile Analytics
• Allows you to collect, visualize, and
understand your mobile app usage
• Scales seamlessly to billions of events
per day
• You retain full control and ownership
of the data
Amazon Mobile
Analytics
AWSMobileAnalytics *analytics = [AWSMobileAnalytics
mobileAnalyticsForAppId:@"yourAppId” identityPoolId: @"cognitoId"];
Let’s think in layers
Create profile, upload
content, and track usage
Track location and user
interests
Match and alert users
App-centric
“You”
Activity-centric
“What you do”
User base-centric
“Them & me”
1
2
3
“Location Tracker” and “Interest” microservice
Amazon
Lambda
Amazon API
Gateway
Amazon
DynamoDB
• /location
• /interests
• reportLocation()
• likeInterest()
• createInterest()
• listInterest()
Microservice
• location-table
• interest-table
Concepts first: Geohash
GeoHash is a lat/long
geocode system that
subdivides space into
buckets on a grid.
Can be numerical
(e.g.6093522776912656819)
Divide the planet earth
into six cells
(A,B,C,D,E,F) like the
six faces of a cube.
Divide each cell into
child cells, and divide
child cells into more
child cells. The red dot
here would thus be
A224.
Works with
DynamoDB!
How does it work?
Geo library for Amazon DynamoDB
• Java library to easily create and query
geospatial data in DynamoDB using GeoHashes
GeoPoint point = new GeoPoint(47.62, -122.34);
// find places 250m of Seattle’s Space NeedleQueryRadiusRequest request = new
QueryRadiusRequest(point, 250);
QueryRadiusResult result = geoDataManager.queryRadius(request);
https://github.com/awslabs/dynamodb-geo
Works with
Lambda!
Amazon API Gateway
• Fully managed and scalable RESTful
API gateway service
• Powered through our content
delivery network via our 53 global
edge locations
• Provides DDoS protection and
throttling capabilities
• Multiple API stages which you define
(e.g. dev, test, prod)
AWS Lambda
Amazon API
Gateway
Amazon
EC2
AWS API
On-prem
server
When to choose API Gateway vs. direct SDK?
• Amazon API Gateway adds an additional
layer between your mobile users and your
logic and data stores in order to:
• Allow back-end logic to be interchanged
without mobile app code modifications
• Ability to throttle individual users or requests
• Protect against DDoS attacks including
counterfeit requests (Layer 7) and SYN floods
(Layer 3)
• Provide a caching layer for your calls
Let’s think in layers
Create profile, upload
content, and track usage
Track location and user
interests
Match and alert users
App-centric
“You”
Activity-centric
“What you do”
User base-centric
“Them & me”
1
2
3
Find a proximity match based on interests
/location
REST API
Profile
(proximity
setting)
Interest table
AWS SDK call
reportLocation()
Invoke
findMatch()
DynamoDB Streams
GeoHash table
AWS SDK call
Interest tablelikeInterest()
/interest
DynamoDB Streams processor: findMatch()
exports.handler = function(event, context) {
// Process all the records in the stream
event.Records.forEach(function(record) {
var newLocation = record.dynamodb.NewImage.geohash.S;
if (findProximityMatch(newLocation)) {
// Found match!
}
});
context.succeed();
};
Amazon SNS mobile push notifications
• Amazon SNS is a fully
managed, cross-platform
mobile push intermediary
service
• Fully scalable to millions
of devices
• Allows you to create
topics (e.g. per geo,
interest, usage pattern,
etc.)
Amazon SNS
Apple APNS
Google GCM
Amazon ADM
Windows WNS and
MPNS
Baidu CP
Android phones and tablets
Apple iPhones and iPads
Kindle Fire devices
Android phones and tablets in China
iOS
Windows phone devices
Amazon
SNS
Found a match: Notify user!
AWS SDK call
findMatch()
DynamoDB Streams
GeoHash table
Interest table
Amazon SNS
Cognito Sync Trigger – AWS Lambda Code
exports.handler = function(event, context) {
if (event.eventType === 'SyncTrigger') {
event.datasetRecords.forEach(function(item) {
if (item.interestRadius.op == 'replace') {
// New interest radius set - process findMatch()
var params = {
FunctionName: 'findMatch',
InvocationType: 'Event', //makes it async
Payload: '{"user":'+ item.identityId +'}’};
lambda.invoke(params, function(err, data) {[..]});
}
}
}
context.succeed(event);
};
Let’s think in Layers
Create profile, upload
content and track usage
Track location and user
interests
Match and alert users
App-centric
“You”
Activity-centric
“What you do”
User base-centric
“Them & me”
1
2
3
Mobile AppMobile
SDKAmazon
API
Gateway
AWS
Lambda
Amazon
S3
Amazon
DynamoDB
Amazon
Cognito
Amazon Mobile
Analytics
Amazon
SNS
Final architecture
easy ten
Users have learned
170 000 000+new words
1 200 000+downloads
Mobile app that helps you learn
10 new, foreign words a day • Featured in 85+ countries
• Top 5 grossing apps overall (Russia)
• Top 8 grossing apps overall (Brazil)
Legacy approach
• Large monolithic application running on multiple
EC2 instances (expensive)
• Complex deployment process; single-line
modification required the whole project to
be redeployed
• Constant operational pain with DevOps team
• Unable to iterate quickly trying to balance
concerns over scalability and resiliency with new
features
Current approach
• Completely serverless microservice framework based
on Lambda, DynamoDB, Amazon Kinesis, and Cognito
• Full coverage of business requirements in this
architecture
• No dedicated DevOps, streamlined deployment
• Each engineer can build complete microservice
prototype from scratch in matter of hours
• AWS manages scalability, resiliency, and security for us
at lower costs
• Client AWS SDK instead of self-made solutions
Lambda consumer
S3
Mobile
Analytics
DynamoDB
SQS
Amazon
EMR
Amazon
Cognito
Amazon
KinesisMobile app
Lambda interface
S3 dump
DynamoDB log
Amazon
Redshift
Microservice Core
What’s next?
• API Gateway for more flexible integration
• DynamoDB Streams for data replication
across different regions and usage metrics
• SWF for complex multi-step tasks on
Lambda
Back-end wish list: ACHIEVED
What it does
User administration
Content storage
Push notifications
Analyze user behavior
Custom app logic
How it behaves
Cost follows usage
Minimal undifferentiated heavy lifting
Iterative development
Reduced time to market
Instant scale
Reliable
More to explore
• Test your app on AWS Device Farm
• Export Amazon Mobile Analytics data to Amazon
Redshift
• Customize your notifications with SNS +
Lambda
• Watermark your pictures using S3 + Lambda
Amazon
Device Farm
Amazon
Redshift
Next steps
1. Download the AWS Mobile SDK and create your first
AWS-backed iOS or Android app.
2. Go to console.aws.amazon.com/lambda and create
your first Lambda function. (The first 1M requests are
on us.)
3. Stay up-to-date with AWS Mobile and Lambda on the
Mobile blog and the Compute blog.
Thank you!Ajay Nair, Senior Product Manager, AWS Lambda
Olivier Klein 奧樂凱, Solutions Architect
Visit http://aws.amazon.com/lambda, the
AWS Compute blog, or the Lambda forum to
learn more and get started using Lambda.