iot communication design - cool.ntu.edu.tw
TRANSCRIPT
IoT Communication DesignLecturer: Cheng-Jen Liu
Outline● Introdcution of REST
● Introduction of gRPC
● Introduction of MQTT
● Homework Description (5%)
Communication Pattern (1) -- Client Server Paradigm
● Simple communication but heavy overhead
● Stateless communication
● General used in web technologies
Communication Pattern (2) -- Peep-To-Peer Paradigm
● Flexible communication model (Peer can be either client/server)
● Stateful communication
● Hard to scalable
● Remote function call pattern
Communication Pattern (3) -- Publish Subscribe Paradigm
● Decouple communication parties with central message broker
● Scalable Desgin
● Flexible Communication
How Youtube Works (Discussion) ??● Web Service
○ Client Server Model
● Video Processing○ Peer-to-Peer Model
○ Pub-Sub Model
● Notification System○ Pub-Sub Model
Q & AIs there any specific design/communication pattern fit all the scenarios ?
There is no one-solution-fit-all approach that can fit all the scenarios.
Usually, a project will combine lots of different communication patterns
to realize the application goal.
Introduction of RESTREST (Rest API) is a common application interface used for softwares to export
their services to other softwares. It is not a protocol but an architectural style.
Generally, developers use REST to create website and export CRUD operations
to manipulate resource on the website.
Let’s build a simple web service with Django and RESTWe are going to use a python web backend framework, called Django, to
realize a simple web service that allows clients to send GET requests and get
echoing response.
Anatomy of Django Framework
Web Server (e.g. NGINX)
● Security Check● Authentication● Preprocessing
Route requests to associated function
Backend Logics (functions)
Clone Django Tutorial ProjectType cmd `git clone https://github.com/johnnylord/django-rest-tutorial.git`.
Follow the instructions in the README file to run the project.
How to check Django project (1) -- settings.py
Installed or defines apps
Rest API Framework Related Library
tutorial app containing backend logics
How to check Django project (2) -- urls.py
mysite/urls.py
tutorial/urls.py
Target URL: http://localhost:8000/rest/tutorial
Start parsing
Reque
st will
be ro
ute to
here
How to check Django project (3) -- views.py
tutorial/views.pyHandling GET request from client
Runtime Snapshots (Server Logging)
GET Request from client
Runtime Snapshots (Client Request)
Server Response Header
Server Response Body
Q & A● How does Django route the client request to the assoicate view function ?
You need to define routing rules with regular experssions in the
urls.py file and specify associated view functions to the defined
rules.
● How to send file to the Django backend through REST API ?
Ask Google !!!
Introduction of gRPCgRPC is a language agnostic, high-performance Remote Procedure Call (RPC)
framework. The main benefits of gRPC are: Modern, high-performance,
lightweight RPC framework. Contract-first API development, using Protocol
Buffers by default, allowing for language agnostic implementations
What is Remote Procedure Call ?Remote Procedure Call (RPC) is a mechanism/concept that allow a process
running locally to be able to call functions provided by a remote process
running on another machine.
What is Serialization ?Serialization is a process that convert object into a stream of bytes that can be
used to send between two running processes through network communication.
What is Protobuf ?Protocol buffers (Protobuf) are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data.
● It is a schema-based serialization method
● It has its own compiler to compile schema code to language-sepcific code
Install the Protobuf CompilerVisit the installation webpage https://grpc.io/docs/protoc-installation/
Let’s build a fibonacci service with gRPC and ProtobufType cmd `git clone https://github.com/johnnylord/gRPC-with-protobuf.git` and follow the README instruction.
Protobuf & gRPC Schema File
service/fib.proto
gRPC Service interface
Request Data structure
Response Data structure
Compile the schema fileAfter the compilation (`$ make`), the generated python wrapper of fibonacci grpc modules.
build/service/fib_pb2_grpc.py build/service/fib_pb2.py
gRPC Servicer ImplementationThe generated gRPC Servicer is just an
abstract class, you need to define the
functionality by yourself.Interface needs to be implemented.
server.py
gRPC Stub(client) to use remote service Typically, we use `stub` to indicate the access point of the remote service.
client.py
Remote Procedure Call
Instantiate request
Establish connection
Q & A● What’s the difference between gRPC and JsonRPC ?
gRPC uses protobuf and JsonRPC uses json. The underlying
serialization method is different.
● What’s the advantage of schema-based serialization method ?
Schema-based serialization can compile the object into smaller
bytestreams. It doesn’t need to include field information in the
payload.
What is MQTT ?MQTT (Message Queuing Telemetry Transport) is a publish/subscribe messaging protocol that works on top of the TCP/IP protocol. The first version of the protocol was developed by Andy Stanford-Clark of IBM and Arlen Nipper of Cirrus Link in 1999.
AWS IoT Core MQTTMQTT is widely used in IoT Application. AWS even have dedicated services based on MQTT messaging protocol.
IoT Application System
Eclispse Mosquitto MQTT BrokerEclispse Mosquitto is an open source project that implements the MQTT protocol and can be used as the message broker.
You can consider this as message broker.
Possible IoT System Design with MQTT Broker (1)
HEAT
TEMP
MOIST
Data
Possible IoT System Design with MQTT Broker (2)
HEAT
HEAT
HEAT
HEAT
Let’s build a logging system with Eclipse MosquittoType cmd `git clone https://github.com/johnnylord/eclipse-mosquitto.git` and
follow the README instruction.
MQTT CPU Publisher
publisher.py
Connect to MQTT Broker
Get system CPU usage
Publish CPU usage to MQTT broker
MQTT Subscriber
subscriber.py
Connect to MQTT Broker
Subscribe topics of interest
Callback function when recive message from MQTT
broker
Homework (5%): Mixture of Communication PatternYou have to implement a system with a simple fibonacci caculator capability and a logging feature as following:
Server RequirementThe server is a backend web server implemented with the python Django
Framework. It must exports two REST Interfaces:
1. [POST] - /rest/fibonacci/
Client can launch POST request to ask for the result of fibonacci at N
order. The body content of POST is like { “order”: 10 }
2. [GET] - /rest/logs/
Client can launc GET request and get a list of history fibonacci requests
sent before. The result will be something like { “history”: [ 10, 5, …, 1 ] }
Fibonacci RequirementYou have to implement a fibonacci caculator service with gRPC interface. The
caculator will accept an integer number from the server, and return the
caculated result to the server. Following is the formula of fibonacci function:
Logging Requirement:The logging module will save the sent fibonacci order from the client request
through the subscribed topic on the message broker.
Whenever the server asks for the history information of fibonacci requests,
the logging module will respond the server with all the history of fibonacci
requests containing the order values.
Handover rules (Deadline: 11/17)Please open a new repo on github, and finish the homework requirements. Fill your repo’s url into https://docs.google.com/spreadsheets/d/1HHp0L4ObcaZAn29K8CozpH3JXVHJLZG7j3EPqhUTBSg/edit?usp=sharing
1. You need to provide README to tell TA how to run your code2. A recorded video demonstrates the workflow of how the system (1)
process the fibonacci request, and (2) how the logging module logs all the history request at runtime.
(YOU NEED TO OPEN SEVREAL TERMINALS, AND SHOW THE SYSTEM WORKS IN RUNTIME)