Download - Azure Service Bus : under the hood
![Page 1: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/1.jpg)
IoT day 2015
Azure Service Bus : under the hood
AMQP protocol fundamentalsPaolo Patierno
Microsoft MVP for Windows Embedded
@ppatierno
![Page 2: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/2.jpg)
IoT day 2015
Sponsor
![Page 3: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/3.jpg)
Who am I ? Contacts
•Senior Software Engineer (Leonardo Ricerche S.r.l.)
•Microsoft MVP for Windows Embedded
”... constantly moving between the devices and the cloud ...”
•«DotNetCampania» member• https://paolopatierno.wordpress.com
•«TinyCLR.it» member• http://www.tinyclr.it
•«Embedded101» board of director member• http://www.embedded101.com/Blogs/PaoloPatierno.aspx
•Linkedin• http://it.linkedin.com/in/paolopatierno
•Contacts• [twitter] @ppatierno
• [email] [email protected]
• [skype] paolopat80
![Page 4: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/4.jpg)
Agenda
•IoT communication patterns
•Azure Service Bus intro …• Queues
• Topics/Subscriptions
• Event Hubs
•… under the cover : AMQP protocol• Introduction
• Architecture
• Communication
• How it fits the patterns
•Demo
![Page 5: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/5.jpg)
IoT Communication Patterns
Telemetry
Information flows
from device to other
systems for
conveying status
changes in the device
Inquiries
Requests from
devices looking to
gather required
information or asking
to initiate activities
Commands
Commands from
other systems to a
device or a group of
devices to perform
specific activities
Notifications
Information flows from
other systems to a
device or a group for
conveying status
changes in the world
1:N 1:N
![Page 6: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/6.jpg)
Azure Service Bus … Queues
•Competing Consumers pattern• all consumers read from same stream (queue)
•Message consumed by a single consumer
![Page 7: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/7.jpg)
Azure Service Bus … Topics
•Publish/Subscribe pattern• each consumer reads from its subscription (a copy of message on related
topic)
•Message consumed by more subscribers (use filters)
Subs
![Page 8: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/8.jpg)
Azure Service Bus … Event Hubs
•Partitioned Consumers pattern
![Page 9: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/9.jpg)
Proprietary messaging protocols
•Difficult to port applications• Requires re-coding all applications
•Difficult to integrate• Application level bridges to move messages
and translate message formats
•Restricted platform support• Limited to whatever vendor provides
![Page 10: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/10.jpg)
Advanced Message Queuing Protocol 1.0
•Open, standard messaging protocol• OASIS (since 2012/10) and ISO/IEC (since 2014/05)
• Enables cross-platform apps to be built using brokers, libraries and frameworks from different vendors
•Features• Efficient : binary connection-oriented protocol
• Reliable : “fire and forget” to reliable, “exactly once” delivery
• Portable data representation : cross-platform, full-fidelity exchange
• Flexible : client-client, client-broker and broker-broker topologies
• Broker model independent : no requirements on broker internals
![Page 11: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/11.jpg)
Microsoft said …
“Microsoft congratulates the AMQP community on approval of AMQP version 1.0 as an OASIS Standard. As an open and interoperable messaging protocol that can scale from mobile clients to the cloud, AMQP has benefitted from the participation of technical experts from around the world, and the achievement of this important milestone will lead to continued growth in the AMQP ecosystem. We look forward to working with the community to promote AMQP-based interoperability and innovation.”
-- Scott Guthrie
![Page 12: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/12.jpg)
AMQP 1.0 client libraries
Language Library
C# Service Bus .NET Client Library, AMQP.Net Lite
JavaApache Qpid Java Message Service (JMS) client
IIT SwiftMQ Java client
C Apache Qpid Proton-C
PHP Apache Qpid Proton-PHP
Python Apache Qpid Proton-Python
Ruby Apache Qpid Proton-Ruby
Perl Apache Qpid Proton-Perl
NodeJS Node AMQP 1.0
![Page 13: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/13.jpg)
AMQP : containers and nodes
•architecture• wire protocol
• data types system with compact binary encoding
•container• client contains producer and/or consumer
• broker contains entities (ex. queues, topics/subscriptions, event hubs)
•node• producer sends messages
• consumer receives messages
• entities store and/or forward messages
![Page 14: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/14.jpg)
AMQP : transport
client
producer
(consumer)
broker
entity
container containerconnection
session
link
node node
multiplexing frames on sessions and links
transport indipendent
![Page 15: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/15.jpg)
open (max frame size, channel max, timeout)
AMQP : communication Open
•Open TCP connection
•Open ”connection”
•Begin ”session”
•Attach ”link”
Container Container
header AMQP / SASL handshake
open (max frame size, channel max, …)
begin (in/out window, handle max, …)
begin (in/out window, handle max, …)
attach (name, source, target, handle, …)
attach (name, source, target, handle, …)
open TCP
connection
open
«connection»
begin «session»
attach «link»
header AMQP / SASL handshake
![Page 16: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/16.jpg)
AMQP : communication Send
•Send messages• to queue
• to topic
• to event hub
Producer Container
attach (name, source, target, handle, …)
attach (name, source, target, handle, …) attach «link»
flow (in/out window, link credit, …) flow control
transfer (delivery id/tag, settled, …)
disposition (first, last, settled, state, …)
sending
message
transfer (delivery id/tag, settled, …)
disposition (first, last, settled, state, …)
sending
messagestransfer (delivery id/tag, settled, …)
transfer (delivery id/tag, settled, …)
![Page 17: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/17.jpg)
AMQP : communication Receive
•Receive messages• from queue
• from subscription
• from event hub
Consumer Container
attach (name, source, target, handle, …)
attach (name, source, target, handle, …) attach «link»
flow (in/out window, link credit, …) flow control
transfer (delivery id/tag, settled, …)
disposition (first, last, settled, state, …)
receiving
message
transfer (delivery id/tag, settled, …)
disposition (first, last, settled, state, …)
receiving
messagestransfer (delivery id/tag, settled, …)
transfer (delivery id/tag, settled, …)
![Page 18: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/18.jpg)
end
AMQP : communication Close
•Detach ”link”
•End ”session”
•Close ”connection”
•Close TCP connection
Container Container
detach (handle, close, …)
end
close
close
detach «link»
end «session»
close
«connection»
detach (handle, close, …)
![Page 19: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/19.jpg)
AMQP : Telemetry
•Data transfer
•Messages settled
Device Server
transfer (<resource>)
transfer (<resource>)
transfer (<resource>)
settlement (based on QoS)
settlement (based on QoS)
flow
in_queue
![Page 20: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/20.jpg)
AMQP : Inquiry
•Correlation between inquiry
request and response
Device Server
transfer (<info>, replyTo, messageId)
transfer (<info>, correlationId = messageId)
settlement (based on QoS)
settlement (based on QoS)
in_queue out_queue
![Page 21: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/21.jpg)
AMQP : Command
•Correlation between command
request and response
Device
transfer (<cmd>, replyTo, messageId)
transfer (<result>, correlationId = messageId)
settlement (based on QoS)
settlement (based on QoS)
Serverin_queue out_queue
![Page 22: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/22.jpg)
AMQP : Notification
•Notification received
•Messages settled DeviceServer
transfer (<notifiy>)
settlement (based on QoS)
transfer (<notify>)
settlement (based on QoS)
flow
in_queue
![Page 23: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/23.jpg)
AMQP : Claim Based Security (CBS) token
•Build a token based on publisher name, expire time, URI, key, …
•Send the token to the $cbs endpoint and specify a ReplyTo
•Receive status code of the request from the above ReplyTo
Device Server
transfer (<token>, replyTo, messageId)
transfer (<status code>, correlationId = messageId)
settlement (based on QoS)
settlement (based on QoS)
$cbs <replyTo>
![Page 24: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/24.jpg)
AMQP : messages
•message• header
• annotations
• system properties (ex. correlationId, replyTo, TTL, …)
• custom/user properties
• «opaque» body (encoded as you want, ex. binary, XML, JSON, …)
•message metadata
•most times body empty, all values as properties
•filter on properties
•properties can be changed «on fly»
![Page 25: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/25.jpg)
AMQP : main features
•messaging middleware• asynchronous : produce and consumer decoupled
• credit based to receive
+5
![Page 26: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/26.jpg)
AMQP : main features
•more messaging patterns• load balancing and competing consumers (ex. on a queue)
• publish/subscribe (ex. on topic/subscription)
• partitioned consumers (ex. on event hubs)
• messages redirection to queues based on filters
• request/response w/ «correlation id» and «replyTo»
•session and transactional message transfer
![Page 27: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/27.jpg)
AMQP : advantages
•efficient• binary connection-oriented
• ”flow control” credit based
• packet size 60 bytes
•reliable• Quality of Service (best effort, at least once, exactly once)
•security• SSL/TLS
• SASL (Simple Authentication and Security Layer)• decople authentication from application layer
![Page 28: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/28.jpg)
Netduino 3 WiFi
•STM32 Cortex-M4
•2 MB dual-bank flash
•256 KB RAM
•WiFi module (TI CC 3100)
•Go-Bus ports
•Micro SD
•.Net Micro Framework
•Visual Studio 2013
•Azure Enabled (SSL/TLS)
![Page 29: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/29.jpg)
IoT day 2015
Access to Service Bus with AMQP
![Page 30: Azure Service Bus : under the hood](https://reader031.vdocuments.site/reader031/viewer/2022013111/55aa3aca1a28ab142a8b46c0/html5/thumbnails/30.jpg)
References
•Microsoft Azure• Service Bus : http://azure.microsoft.com/en-us/services/service-bus/
• Event Hubs : http://azure.microsoft.com/en-us/services/event-hubs/
•AMQP • Official web site : http://www.amqp.org
• AMQP .Net Lite : http://amqpnetlite.codeplex.com/
• Azure SB Lite : http://azuresblite.codeplex.com/
• Azure SB Lite Examples : https://github.com/ppatierno/azuresblite-examples
• Apache Qpid: https://qpid.apache.org/