dds: the iot data sharing standard

DDS The IoT Data Sharing Standard

Chief Technology Officer PrismTech

OMG DDS Co-‐Chair OMG Architectural Board

[email protected]

Angelo Corsaro, PhD

mailto:[email protected]

Cop

yrig

ht P

rism

Tech

, 201

4

Introduced in 2004, DDS is an Object Management Group (OMG) Standard for efficient, secure and interoperable, platform- and programming-language independent data sharing

DDS standardises: - Programming API - Interoperable wire-protocol - Extensible Type System - Data Modeling - Security Framework - Remote Procedure Call

The DDS Standard

*

*

(*) Under Finalisation

https://twitter.com/acorsaro










http://www.linkedin.com/in/corsaro










http://slideshare.net/angelo.corsaro









How is DDS Different/Better?

Cop

yrig

ht P

rism

Tech

, 201

4

Provides a Distributed Data Space abstraction where applications can autonomously and asynchronously read and write data

Its built-in dynamic discovery isolates applications from network topology and connectivity details

Higher Level Abstraction

DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS






























Cop

yrig

ht P

rism

Tech

, 201

4

DDS supports the definition of Common Information Models. These data models define the system’s Lingua Franca

DDS types are extensible and evolvable, thus allowing incremental updates and upgrades

Data Centricity






























Cop

yrig

ht P

rism

Tech

, 201

4

A Topic defines a domain-wide information’s class

A Topic is defined by means of a (name, type, qos) tuple, where

• name: identifies the topic within the domain

• type: is the programming language type associated with the topic. Types are extensible and evolvable

• qos: is a collection of policies that express the non-functional properties of this topic, e.g. reliability, persistence, etc.

Topic

TopicTypeName

QoS

...

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS






























Cop

yrig

ht P

rism

Tech

, 201

4

As explained in the previous slide a topic defines a class/type of information

Topics can be defined as Singleton or can have multiple Instances

Topic Instances are identified by means of the topic key

A Topic Key is identified by a tuple of attributes -- like in databases

Remarks: - A Singleton topic has a single domain-wide instance - A “regular” Topic can have as many instances as the

number of different key values, e.g., if the key is an 8-bit character then the topic can have 256 different instances

Topic and Instances






























Cop

yrig

ht P

rism

Tech

, 201

4DDS “knows” about application data types and uses this information provide type-safety and content-based routing

Content Awarenessstruct TemperatureSensor { @key long sid; float temp; float hum; }

sid temp hum101 25.3 0.6507 33.2 0.7913 27,5 0.55

1307 26.2 0.67

“temp > 25 OR hum >= 0.6”

sid temp hum507 33.2 0.7

1307 26.2 0.67

Type

TempSensor






























Cop

yrig

ht P

rism

Tech

, 201

4

DDS provides a rich set of QoS-Policies to control local as well as end-to-end properties of data sharing

Some QoS-Policies are matched based on a Request vs. Offered (RxO) Model

QoS Policies

HISTORY

LIFESPAN

DURABILITY

DEADLINE

LATENCY BUDGET

TRANSPORT PRIO

TIME-BASED FILTER

RESOURCE LIMITS

USER DATA

TOPIC DATA

GROUP DATA

OWENERSHIP

OWN. STRENGTH

LIVELINESS

ENTITY FACTORY

DW LIFECYCLE

DR LIFECYCLE

PRESENTATION

RELIABILITY

PARTITION

DEST. ORDER

RxO QoS Local QoS






























Cop

yrig

ht P

rism

Tech

, 201

4

For data to flow from a DataWriter (DW) to one or many DataReader (DR) a few conditions have to apply:

The DR and DW domain participants have to be in the same domain

The partition expression of the DR’s Subscriber and the DW’s Publisher should match (in terms of regular expression match)

The QoS Policies offered by the DW should exceed or match those requested by the DR

QoS ModelDomain

Participant

DURABILITY

OWENERSHIP

DEADLINE

LATENCY BUDGET

LIVELINESS

RELIABILITY

DEST. ORDER

Publisher

DataWriter

PARTITION

DataReader

Subscriber

DomainParticipant

offered QoS

Topicwrites reads

Domain Idjoins joins

produces-in consumes-from

RxO QoS Policies

requested QoS






























Cop

yrig

ht P

rism

Tech

, 201

4

Data Delivery QoS Policies provide control over:

who delivers data

where data is delivered, and

how data is delivered

Data Delivery

Reliability

Presentation

Destination OrderPartition

Ownership OwnershipStrength

Data Delivery






























Cop

yrig

ht P

rism

Tech

, 201

4Data Availability QoS Policies provide control over data availability with respect to:

Temporal Decoupling (late Joiners)

Temporal Validity

Data Availability

History

DurabilityLifespan Data Availability






























Cop

yrig

ht P

rism

Tech

, 201

4Several policies provide control over temporal properties, specifically:

Outbound Throughput

Inbound Throughput

Latency

Temporal Properties

Throughput

TimeBasedFilter

[Inbound]

[Outbound]Latency

Deadline

TransportPriority

LatencyBudget






























Cop

yrig

ht P

rism

Tech

, 201

4

DDS supports both the Cloud and the Fog Computing Paradigm

DDS natively supports:

- Device-to-Device Communication

- Device-to-Cloud Communication

Cloud and Fog/Edge Computing

Fog Computing

Cloud Computing

Fog Computing

Fog Computing

Device-to-Cloud Communication

Device-to-Device Communication

Fog-to-Cloud Communication

Cloud-to-Cloud Communication

Device-to-Device Communication






























Cop

yrig

ht P

rism

Tech

, 201

4

Support for fine grained access control

Support for Symmetric and Asymmetric Authentication

Standard Authentication, Access Control, Crypto, and Logging plug-in API

Security

Arthur Dent

Arthur Dent

Ford Prerfect

Zaphod Beeblebrox

Marvin

Trillian

A(r,w), B(r)

A(r,w), B(r,w), X(r)

*(r,w)

*(r)

A(r,w), B(r,w), C(r,w)

Ford Prerfect

Zaphod Beeblebrox

Trillian

Marvin

A

B

A,BX

*

*

A,B,C

Identity Access RightsSessions are authenticated and communication is encrypted

Only the Topic included as part of the access rights are visible and accessible






























Cop

yrig

ht P

rism

Tech

, 201

4DDS is independent from the

- Programming language,

- Operating System

- HW architecture

Platform Independent






























DDS is Simple!

Cop

yrig

ht P

rism

Tech

, 201

4

Chatting in Scala

import dds._import dds.prelude._import dds.config.DefaultEntities._

object Chatter { def main(args: Array[String]): Unit = { val topic = Topic[Post]("Post") val dw = DataWriter[Post](topic) dw.write(new Post(“kydos”,”Using VORTEX.. It's pretty cool!”)); }}






























Cop

yrig

ht P

rism

Tech

, 201

4

Chatting in Scala

import dds._import dds.prelude._import dds.config.DefaultEntities._

object ChatLog { def main(args: Array[String]): Unit = { val topic = Topic[Post]("Post") val dr = DataReader[Post](topic) dr listen { case DataAvailable(_) => dr.read.foreach(println) } }}






























Cop

yrig

ht P

rism

Tech

, 201

4

Chatting in C++

#include <dds.hpp>

int main(int, char**) {

DomainParticipant dp(0); Topic<Post> topic(“Post”); Publisher pub(dp); DataWriter<Post> dw(dp, topic);

dw.write(Post(“kydos”,”Using VORTEX.. It's pretty cool!”)); dw << Post(“kydos”,”Using operator << to post!”); return 0; }






























Cop

yrig

ht P

rism

Tech

, 201

4

Chatting in C++#include <dds.hpp>

int main(int, char**) {

DomainParticipant dp(0); Topic<Post> topic(“Post”); Subscriber sub(dp); DataReader<Post> dr(dp, topic);

LambdaDataReaderListener<DataReader<Post>> lst; lst.data_available = [](DataReader<Post>& dr) { auto samples = data.read(); std::for_each(samples.begin(), samples.end(), [](Sample<Post>& sample) { std::cout << sample.data() << std::endl; } }

dr.listener(lst); return 0; }






























DDS vs. Other Standards

Cop

yrig

ht P

rism

Tech

, 201

4

Originally defined by the AMQP consortium as a messaging standard addressing the Financial and Enterprise market

AMQP is an OASIS standard that defines an efficient, binary, peer-to-peer protocol for for transporting messages between two processes over a network. Above this, the messaging layer defines an abstract message format, with concrete standard encoding

https://www.oasis-open.org/committees/amqp/

Advanced Message Queueing Protocol (AMQP)






























https://www.oasis-open.org/committees/amqp/

Cop

yrig

ht P

rism

Tech

, 201

4

CoAP is an IETF RFC defining a transfer protocol for constrained nodes and constrained (e.g., low-power, lossy) networks, e.g. 8-bit micro-controllers with small amounts of ROM and RAM, connected by Low-Power Wireless Personal Area Networks (6LoWPANs).

The protocol is designed for machine-to-machine (M2M) applications such as smart energy and building automation.

Constrained Application Protocol (CoAP)

CoAP provides a request/response interaction model between application endpoints, supports built-in discovery of services and resources, and includes key concepts of the Web such as URIs and Internet media types.

CoAP is designed to easily interface with HTTP for integration with the Web while meeting specialised requirements such as multicast support, very low overhead, and simplicity for constrained environments.






























Cop

yrig

ht P

rism

Tech

, 201

4

MQTT was defined originally by IBM in the mid 90’s as a lightweight protocol for telemetry

MQTT supports a basic publish/subscribe abstraction with three different level of QoS

MQTT has recently gained much attention as a potential candidate for data sharing in the IoT

Message Queueing Telemetry Transport (MQTT)






























Cop

yrig

ht P

rism

Tech

, 201

4

Qualitative Comparison

Transport Paradigm Scope Discovery Content Awareness

Data Centricity Security Data

PrioritisationFault

Tolerance

AMQP TCP/IP

Point-to-Point

Message Exchange

D2DD2CC2C

No None Encoding TLS None Impl. Specific

CoAP UDP/IPRequest/

Reply (REST)

D2D Yes None Encoding DTLS None Decentralised

DDSUDP/IP

(unicast + mcast)TCP/IP

Publish/SubscribeRequest/

Reply

D2DD2CC2C

Yes

Content-Based

Routing, Queries

Encoding, Declaration

TLS, DTLS, DDS

Security

Tranport Priorities Decentralised

MQTT TCP/IP Publish/Subscribe D2C No None Undefined TLS None Broker is the

SPoF

[Ref: A Comparative Study of Data-Sharing Standards for the Internet of Things, Cutter Journal, Dec 2014 ]






























Cop

yrig

ht P

rism

Tech

, 201

4

Efficient and scalable Data Sharing is a key requirement of practically any IoT system

The degree of performance and fault-tolerance required by the data sharing platform varies across Consumer and Industrial Internet on Things applications

Fog Computing support is key for IIoT

CIoT/IIoT Data Sharing Requirements

High Individual Data Rates

High Aggregated Data Volumes

Low Latency

Temporal Determinism

Device-2-Device (D2D) Comms

Device-2-Cloud (D2C) Comms

Cloud-2-Cloud (C2C) Comms

Bandwidth Efficiency

Fault-Tolerance

Transport-Level Security

Data-Level Security

0,00 0,25 0,50 0,75 1,00

CIoT IIoT


Relative Importance






























Cop

yrig

ht P

rism

Tech

, 201

4

Scoring the various IoT candidates agains the data-sharing requirements shows that DDS and AMQP are those that best address IoT needs

IoT Standard Fitness


AMQP

CoAP

DDS

MQTT

0,0% 25,0% 50,0% 75,0% 100,0%

CIoT Fitness IIoT Fitness






























Tech MythologyMQTT is very wire efficient. Probably the most wire-efficient IoT data sharing standard

Cop

yrig

ht P

rism

Tech

, 201

4

MQTT encodes the topic name on every data message, as a result its wire-efficiency linearly degrades on the topic name length — which is usually greater several tens of bytes

DDS has a predictable protocol overhead. Furthermore, when running DDS in streaming mode over TCP/IP the protocol efficiency can be further improved

DDS & MQTT Wire EfficiencyProtocol Overhead

Topi

c N

ame

Leng

th

0

75

150

225

300

Size (bytes)8 16 32 64 128 256

MQTTDDSDDS Streaming































Cop

yrig

ht P

rism

Tech

, 201

4

Protocols Overhead Analysis

Message Size (bytes) IPv4 Headers(bytes)

Total Size(bytes)

MQTT(QoS = 0)

(2 to 5) + 2 + length(TopicName) + length(Payload) IP: 20-40 TCP: 20-40

min = 20 + 20 + 4 + length(TopicName) + length(Payload)max = 40 + 40 + 7 + length(TopicName) + length(Payload)

DDS 44 + length (Payload) IP: 20-40 UDP: 8

min = 20 + 8 + 44 + length(Payload)max = 40 + 8 + 44 + length(Payload)

DDS Streaming

24 + length (Payload) IP: 20-40 TCP: 20-40

min = 20 + 20 + 24 + length(Payload)max = 40 + 40 + 24 + length(Payload)






























Cop

yrig

ht P

rism

Tech

, 201

4

DDS Enables the IoT

Cop

yrig

ht P

rism

Tech

, 201

4

dds: the iot data sharing standard

Technology

topic key

singleton topic

data models

regular topic

dds differentbetter

application data types

endtoend properties

phdcopyright prismtech