김무현 | Solutions Architect

November 22, 2016

What to expect today ?

But first, what is a thing ?

What won’t we cover ?

Getting setup

http://bit.ly/2glLLpU

– Do not flash your edison they are already up-to-date

http://intel.ly/1QeDxu6

http://bit.ly/nopimonitorhttp://bit.ly/nopibeginner

Some Cool Things

Voice Controlled Drone

Some Cool Things

Voice Controlled Drone

Some Cool Things

Automatic Vehicle Accident Alert System using AWS IoT

Some Cool Things

Automatic Vehicle Accident Alert System using AWS IoT

Some Cool Things

Magic Mirror - Amazon Echo and AWS IoT

Some Cool Things

Magic Mirror - Amazon Echo and AWS IoT

Some Cool Things

AWS IoT Demo for Danbo

Some Cool Things

AWS IoT Demo for Danbo

Current IoT Ecosystem

API GatewayBuild, deploy and manage APIs.

• Great first entry-point for applications especially mobile• Presents a unified front for all your backend services• Direct integration with AWS Lambda• Integrated with Cognito for authentication• Protection from unwanted traffic• SDK generation, caching and throttling• Support swagger imports and exports

API Gateway

API Gateway & IoT Usage

Lambda

mobile app / device

API Gateway EC2 Instances

AWS IoT

Data Center

CognitoUser Identity and App Data Synchronization

• Cognito is a simple user-data synchronization and identity service that helps you securely manage and synchronize app data for your users across their mobile devices.

• Authenticate users with popular public identity providers(Amazon, Facebook, Google and any other OpenID Connect compatible identity provider) or support unauthenticated guest users and use the AWS Cloud to save and sync user data for their mobile.

• Amazon Cognito can integrate with your existing identity System.

Authentication

API Gateway, Cognito & IoT Usage

Lambda

mobile app / device

API Gateway EC2 Instances

AWS IoT

Data Center

Credentials

AWS Cognito

AWS LambdaServerless event driven compute service

• Bring your own code • Simple resource model• Flexible invocation paths• Permissions integrated with all services, Cognito/APIG• Completely stateless• Integration with AWS IoT makes this a key service

Lambda

Lambda & IoT Usage

Lambda

mobile app / device

API Gateway EC2 Instances

AWS IoT

Data Center

PythonNode.jsJava

Simple Notification Service (SNS)Push Notification Service

• Set up, operate, and send notifications • Publish messages from an application and immediately

deliver them to subscribers or other applications• Push messages to mobile devices• Supports notifications over multiple transport protocols

– Email,SMS,HTTP/HTTPS

SNS

SNS & AWS IoT

AWS IoT

AWSLambda

Amazon EC2

AmazonSNS

Simple Queue Service (SQS)Message Queue Service

• Managed and scalable message queue• Building block for distributed systems• Highly available and durable• Pay for what you use

SQS

SNS

SQS

Lambda

RulesKinesis

Firehose

Redshift

AWS IoT

Rules

SNSLambda

Analytics

SQS, SNS and AWS IoT

AWS IoT

KinesisReal-time Processing of Streaming Big Data

• Fully-managed service for real time processing of streaming data, at any scale

• Kinesis can continuously capture and store terabytes of data per hour from hundreds of thousands of sources

• Integrate with S3, Dynamo DB, and Amazon Redshift

• Build custom applications on top of Kinesis data

Streaming with Kinesis

Kinesis and AWS IoT

AWS IoT AWSLambda

Amazon Kinesis

Amazon Kinesis

DynamoDBPredictable and Scalable NoSQL Data Store

• Fast, fully-managed NoSQL Database Service• Capable of handling any amount of data• Durable and Highly Available• Fast predictable performance backed by SSD

storage• Can scale up to millions of IOPS• Simple and Cost Effective

DynamoDB

DynamoDB, Lambda and AWS IoT

AWS IoT AWSLambda

Amazon Kinesis

DynamoDB

Ecosystem Architecture

Amazon SQS

Credentials

IAM

DynamoDB

Lambda

mobile app / device

API Gateway

AWS CognitoEC2 Instances

Amazon SNS

RDS

AWS IoT

Deep Dive on AWS IoT

AWS IoT Overview

DEVICE SDKSet of client libraries to

connect, authenticate and exchange messages

DEVICE GATEWAYCommunicate with devices via

MQTT and HTTP

AUTHENTICATIONAUTHORIZATION

Secure with mutual authentication and encryption

RULES ENGINETransform messages based on rules and

route to AWS Services

AWS Services- - - - -

3P Services

DEVICE SHADOWPersistent thing state

during intermittent connections

APPLICATIONS

AWS IoT API

DEVICE REGISTRYIdentity and Management of

your things

DEVICE SDKSet of client libraries to

connect, authenticate and exchange messages

DEVICE GATEWAYCommunicate with devices via

MQTT and HTTP

AUTHENTICATIONSecure with mutual

authentication and encryption

RULES ENGINETransform messages based on rules and

route to AWS Services

AWS Services- - - - -

3P Services

SHADOWPersistent thing state

during intermittent connections

APPLICATIONS

AWS IoT API

REGISTRYIdentity and Management of

your things

AWS IoT Overview

What does it cost?

AWS IoT Telemetry

AdministrationActuators

Controlautomation

AWS IoT

Data storage& analytics

Sensors

Connected Farm

AWS IoT Telemetry & Analytics

AWS IoT Telemetry

DEVICE GATEWAYCommunicate with devices via

MQTT and HTTP

AUTHENTICATIONAUTHORIZATION

Secure with mutual authentication and encryption

RULES ENGINETransform messages based on rules and

route to AWS Services

AWS Services- - - - -

3P Services

1) Connect the devices

2) Send data

3) Collect the data

AWS IoTData storage& analytics

Sensors?

Single consumer (don’t do this)

AWS IoT instance database

PUBLISH sensors/123

PUBLISH sensors/456

SUBSCRIBE sensors/#

PUBLISH sensors/789

Don’t do this: scalability

AWS IoT instance

SUBSCRIBE sensors/#

Don’t do this: availability

AWS IoT instance

Don’t store analytical data in shadows

Sensors

DEVICE SHADOWS

1. AWS Services(Direct Integration)

Rules Engine

Actions

LambdaSNS SQS

S3 KinesisDDB RDS

Redshift

Glacier

EC2

3. External Endpoints(via Lambda and SNS)

Rules Engine connects AWS IoT to External Endpoints and AWS Services.

2. Rest of AWS(via Kinesis, Lambda, S3, and more)

AWS IoT Rules Engine

Solve the “where to put it” problem

Cloud Command & Control

Administration

AWS IoT

Data storage& analytics

Sensors

Connected Farm

Actuators

Controlautomation

Automated Sprinkler Service

Amazon Kinesis

Amazon Machine Learning

Amazon Redshift

RulesEngine

DeviceGateway

Sensor

Sprinkler

Amazon Kinesis–enabled app

Publish on/off to the sprinkler (don’t do this)

DeviceGateway

SprinklerControl

logic

SUBSCRIBEmacdonald/sprinkler-456

Publish on/off to the sprinkler (don’t do this)

DeviceGateway

SprinklerControl

logic

PUBLISHmacdonald/sprinkler-456{ "water": "on" }

Direct publishing: why not?

Sprinkler

Control logic

on

DeviceGateway

off

on

off

Direct publishing: why not?

So then what?

Device Shadows

ShadowState

Apps

offline

Device Shadows

Device Controller

reported state

desired state

Device Shadows

Device Controller

reported state

desired state

HTTP/RESTWebSocketsMQTT

AWS IoT Shadow - Simple Yet Powerful

{"state" : {

“desired" : {"lights": { "color": "RED" },"engine" : "ON"

},"reported" : {

"lights" : { "color": "GREEN" },"engine" : "ON"},"delta" : {

"lights" : { "color": "RED" }} },

"version" : 10}

Thing

Report its current state to one or multiple shadowsRetrieve its desired state from shadow

Mobile App

Set the desired state of a device Get the last reported state of the device Delete the shadow

Shadow

Shadow reports delta, desired and reported states along with metadata and version

Device Shadows and versioning

Sprinkler

Control logic

on (version=1)

off (version=2)

DeviceGateway

off (version=2)

on (version=1)

(old message ignored by device)

Mobile Control

Data storage& analytics

Sensors

Talking back to the sprinklers: manual override

Controlautomation

AWS IoT

AdministrationActuators

AWS IoT - Applications

DEVICE SHADOWPersistent thing state

during intermittent connections

APPLICATIONS

Using Cognito with IoT

DEVICE SHADOWPersistent thing state

during intermittent connections

APPLICATIONS

AMAZONCOGNITO PERMISSIONS APIs

Configure device and Cognito User permissions

end-user(farmer)

Best practice: Thing name prefixing

WebSockets

WebSockets

AWS IoT

Lifecycle Management

Actuators

Datastorage&analytics

Devicelifecyclemanagement

Controlautomation

AWSIoT

Sensors

Maintenance

1

Lifecycle workflow

Notify operator

1

Connected Disconnected Still disconnected?

AWS IoT Rules Engine & Amazon SNS

Push NotificationsApple APNS Endpoint, Google GCM Endpoint, Amazon ADM Endpoint, Windows WNS

Amazon SNS -> HTTP Endpoint (Or SMS or Email)Call HTTP based 3rd party endpoints through SNS with subscription and retry support

SNS

2

Detecting disconnects

DisconnectedConnected

Graceful disconnect

Crash

Back online

Handling lifecycle events

- Look up mobile push id for device owner- Send SNS mobile push

AWS Lambda Function

Delayed lifecycle events

Device Status Timesensor-123 connected 11:30…

- Double-check the status in DynamoDB- Send SNS push notification if still offline

- Store update device status in DynamoDB- If offline: enqueue an SQS message with

DelaySeconds

AWS Lambda Function

SQS Message (15 minutes later)

AmazonDynamoDB

Last Will and Testament

Key takeaways

New in 2016 – Q2!

Labs!

Patterns we’ll find in the labs

Problematic Single Service Pattern

AmazonDynamoDB

AWS IoT

AWS IoT

AmazonDynamoDB

AWSLambda

Amazon EC2

AmazonSNS

Fragmented Architecture

AWS IoT

AWS IoT

AmazonDynamoDB

AWSLambda

Amazon EC2

AmazonSNS

Amazon API Gateway

Where is my logic?

Where is my logic? Found.

Debugging Labs

Lab 1 – Getting Setup

Lab 2 – The SNS hook

AWS IoT AmazonSNS

AWSLambda

AmazonSQS

SNS

SQS

Lambda

RulesKinesis

Firehose

Redshift

AWS IoT

Rules

SNSLambda

Analytics

Lab 2 – The SNS hook – Expanded Look

AWS IoT

Lab 3 – Streaming data to Redshift

AWS IoTFirehose

Amazon Kinesis

Amazon Redshift

AmazonSNS

AWSLambda

Lab 4 – Websockets

Lab 5 – Thing Shadows

Lab 6 – Failure scenarios

AWS IoT IoTtopic

My LWT

My LWT

Publish Message

Lab 7 – API Gateway and Mobile C&C

Credentials IAM

mobile app / device

API Gateway

AWS Cognito

AWS IoTLambda

Lab 8 – Streaming data to DynamoDB

Lab 9 – Switch to Kinesis, critical events

AWS IoT AWSLambda

Amazon Kinesis