getting started with vortex

Data Sharing in Industrial IoT

AngeloCorsaro,PhDChiefTechnologyOfficer

angelo.corsaro@prismtech.com

Vortex is a standard technology for efficient, ubiquitous, interoperable, secure and platform independent data sharing across network connected devices

in151 Characters

Vortex's Coordination Model

Applications can autonomously and asynchronously read and

write data enjoying spatial and temporal decoupling

DDS Global Data Space

Data Writer

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Global Data Space

Built-in dynamic discovery isolates applications from

network topology and connectivity details

Data Writer

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Dynamic Discovery

The relevant portion of the data space is projected on

the application address space. Each typed projection is commonly called a Cache

No single point of failure or bottleneck

Decentralised Data Space

Data Writer

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

TopicAQoS

Connectivity is dynamically adapted to chose the most

effective way of sharing data

Adaptive ConnectivityData

Writer

Data Writer

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

TopicAQoS

ThecommunicationbetweentheDataWriterandmatchingDataReaderscanbepeer-to-peerexploitingUDP/IP(UnicastandMulticast)orTCP/IP

ThecommunicationbetweentheDataWriterandmatchingDataReaderscanbe“brokered”butstillexploitingUDP/IP(UnicastandMulticast)orTCP/IP

A domain-wide information’s class A Topic defined by means

of a <name, type, qos>

TopicDDS Global Data Space

Data Writer

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

TopicTypeName

Topic Types

Topic Types: Language Independent

Definitions

Topic types can be expressed using different syntaxes,

including IDL and ProtoBuf

Topic Type struct CarDynamics { string cid; long x; long y; float dx; long dy; } #pragma keylist CarDynamics cid

Topic Type message CarDynamics { option (.omg.dds.type) = {name: "CarDynamics"}; required string cid = 0 [(.omg.dds.member).key = true]; required long x = 1; required long y = 2; required float dx = 3; required long dy = 4; }

ProtoBuf

Topic Types: Language Specific

Definitions

Topic Type class CarDynamics: constructor: (@cid, @x, @y, @dx, @dy) ->

CoffeeScript

Topic Type public struct CaDynamics { public string cid { get; set; } public int x { get; set; } public int y { get; set; } public int dx { get; set; } public int dy { get; set; } public CaDynamics (string cid, int x, int y, int dx, int dy) { this.cid = cid; this.x = x; this.y = y; this.dx = dx; this.dy = dy; } }

Topic Type @KeyList ( topicType = "CarDynamics", keys = {"cid"})public class CarDynamics { public String cid; public int x; public int dx; public int y; public int dy; public CarDynamics(String s, int a, int b, int c,int d) { this.cid = s; this.x = a; this.dx = b; this.y = c; this.dy = d; } @Override public String toString() { … }}

Topics, Instances and Samples

Each unique key value identifies a stream of data

Vortex demultiplexes “streams” and provides lifecycle information

A DataWriter can write samples belonging to multiple instances

Topic Instances

CarDynamics

Instances

struct CarDynamics { @key string cid; long x; long y; float dx; long dy; }

Topic Type

Sample

QoS policies allow to express temporal and availability

constraints for data

Data Writer

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

QoS - Enabled

A collection of policies that control non-

functional properties such as reliability,

persistence, temporal constraints and priority

HISTORY

LIFESPAN

DURABILITY

DEADLINE

LATENCY BUDGET

TRANSPORT PRIO

TIME-BASED FILTER

RESOURCE LIMITS

USER DATA

TOPIC DATA

GROUP DATA

OWENERSHIP

OWN. STRENGTH

LIVELINESS

ENTITY FACTORY

DW LIFECYCLE

DR LIFECYCLE

PRESENTATION

RELIABILITY

PARTITION

DEST. ORDER

RxO QoS Local QoS

QoS Policies controlling end-to-end properties

follow a Request vs. Offered

QoS Domain

Participant

DURABILITY

OWENERSHIP

DEADLINE

LATENCY BUDGET

LIVELINESS

RELIABILITY

DEST. ORDER

Publisher

DataWriter

PARTITION

DataReader

Subscriber

DomainParticipant

offered QoS

Topicwrites reads

Domain Idjoins joins

produces-in consumes-from

RxO QoS Policies

requested QoS

Interacting with the Data Cache

Each Data Reader is associated with a Cache

The Cache stores the last n∊𝜨∞ samples for each

relevant instance

Data Cache

Data Writer

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

TopicAQoS

Each Data Reader is associated with a Cache

The Cache stores the last n∊𝜨∞ samples for each

relevant instance

Data Cache

DataReader Cache

DataReader

Samples

Instances

The action of reading samples for a Reader Cache

is non-destructive.

Samples are not removed from the cache

Reading Data

DataReader Cache

DataReader

DataReader Cache

DataReader

...read

The action of taking samples for a Reader Cache

is destructive.

Samples are removed from the cache

Taking Data

DataReader Cache

DataReader

DataReader Cache

DataReader

...take

Samples can be selected using composable content

and status predicates

Sample Selectors

DataReader Cache

DataReader

Filters allow to control what gets into a DataReader

Filters are expressed as SQL where clauses or as

Java/C/JavaScript predicates

Content-Filtering

DataReader Cache

DataReader

Filter

Application

Network

Content Filters can be used to project on the

local cache only the Topic data

satisfying a given predicate

Content Filters structCarDynamics{

@keystringcid;longx;longy;floatdx;longdy;}

cid x y dx dyGR 33N GO 167 240 45 0LO 00V IN 65 26 65 0AN 637 OS 32 853 0 50AB 123 CD 325 235 80 0

“dx>50ORdy>50”

CarDynamics

cid x y dx dyLO 00V IN 65 26 65 0AB 123 CD 325 235 80 0

Reader Cache

Queries allow to control what gets out of a

DataReader Cache

Queries are expressed as SQL where clauses or as

Java/C/JavaScript predicates

Content-Based Selection

DataReader Cache

DataReader

DataReader Cache

DataReader

Application

Network

Reader Cache

Queries can be used to select out of the local cache

the data matching a given predicate

QueriesstructCarDynamics{@keystringcid;longx;longy;floatdx;longdy;}

“dx>50ORdy>50”

CarDynamics

cid x y dx dyLO 00V IN 65 26 65 0AB 123 CD 325 235 80 0

State based selection allows to control what gets out of a DataReader Cache

State base selectors predicate on samples meta-

information

State-Based Selection

DataReader Cache

DataReader

State Selector

DataReader Cache

DataReader

Application

Network

Sample State (READ, NOT_READ) allows to distinguish between new

samples and samples that have already been read

DataReader Cache

DataReader

State Selector

DataReader Cache

DataReader

Application

Network

The View State (NEW, NOT_NEW) allows to distinguish a new instance

from an existing one

DataReader Cache

DataReader

State Selector

DataReader Cache

DataReader

Application

Network

The Instance State (ALIVE, NOT_ALIVE_DISPOSED,

NOT_ALIVE_NO_WRITERS) allows to track the life-cycle transitions of the instance to which a sample belongs

DataReader Cache

DataReader

State Selector

DataReader Cache

DataReader

Application

Network

Selector Example

// == ISO C++ DDS API ==

auto data = dr.select() .content(query) .state(data_state) .instance(handle) .read();

your first vortex app

Anatomy of a DDS Application

Writing Data in C++#include <dds.hpp>

int main(int, char**) {

DomainParticipant dp(0); Topic<Meter> topic(“SmartMeter”); auto pqos = dp.get_default_pubqos() << Partition (“FR:IdF:Essonne:Gif:Essonne:Mnts:123”); Publisher pub(dp, pqos); DataWriter<Meter> dw(pub, topic);

while (!done) { auto value = readMeter() dw.write(value); std::this_thread::sleep_for(SAMPLING_PERIOD); }

return 0; }

enumUtilityKind{ ELECTRICITY, GAS, WATER};structMeter{ stringsn; UtilityKindutility; floatreading; floaterror;};#pragmakeylistMetersn

Reading Data in C++#include <dds.hpp>

int main(int, char**) {

DomainParticipant dp(0); Topic<Meter> topic(”SmartMeter”); Subscriber sub(dp); DataReader<Meter> dr(dp, topic);

LambdaDataReaderListener<DataReader<Meter>> lst; lst.data_available = [](DataReader<Meter>& dr) { auto samples = data.read(); std::for_each(samples.begin(), samples.end(), [](Sample<Meter>& sample) { std::cout << sample.data() << std::endl; } } dr.listener(lst); // Print incoming data up to when the user does a Ctrl-C std::this_thread::join(); return 0; }

Writing Data in Scalaimport dds._import dds.prelude._import dds.config.DefaultEntities._

object SmartMeter { def main(args: Array[String]): Unit = { val topic = Topic[Meter](“SmartMeter”) val dw = DataWriter[Meter](topic) while (!done) { val meter = readMeter() dw.write(meter) Thread.sleep(SAMPLING_PERIOD) } }}

Reading Data in Scala

import dds._import dds.prelude._import dds.config.DefaultEntities._

object SmartMeterLog { def main(args: Array[String]): Unit = { val topic = Topic[Meter](“SmartMeter”) val dr = DataReader[Meter](topic) dr listen { case DataAvailable(_) => dr.read.foreach(println) } }}

Writing Data in Python

import dds import timeif __name__ == '__main__': topic = dds.Topic("SmartMeter", "Meter") dw = dds.Writer(topic) while True: m = readMeter() dw.write(m) time.sleep(0.1)

Reading Data in Pythonimport ddsimport sys def readData(dr): samples = dds.range(dr.read()) for s in samples: sys.stdout.write(str(s.getData())) if __name__ == '__main__': t = dds.Topic("SmartMeter", "Meter") dr = dds.Reader(t) dr.onDataAvailable = readData

Vortex Technology Stack

Device implementations optimised for OT, IT and

consumer platforms

Native support for Cloud and Fog Computing Architectures

Device-2-DeviceDevice-2-Cloud

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

Available across IT, Consumer and OT platforms Device-2-DeviceDevice-2-Cloud

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

Polyglot and Interoperable across Programming

Languages Device-2-DeviceDevice-2-Cloud

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

Fully Independent of the Cloud Infrastructure

Private Clouds

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

Native Integration with the hottest real-time analytics

platforms and CEP Device-2-DeviceDevice-2-Cloud

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

Integration with mainstream Dashboard Technologies

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

High Performance 30 μs peer-to-peer latency

4+ Mmsgs/sec p2p throughput

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

>10 μs fog/cloud routing latency

High Performance Device-2-DeviceDevice-2-Cloud

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

SecureData-Level security with Pluggable Authentication Access Control and Crypto

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

Default Plug-insX.509 Public Key

Infrastructure (PKI) based authentication

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

Default Plug-insAccess Control List file

signed by a shared Certificate Authority for

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

Default Plug-ins

Crypto based on TLS Cipher Suite

Fog-2-Cloud

Device-2-Fog

Cloud-2-Cloud

Fog-2-Fog

structure

SecureData-Security as opposed to simply Transport-Level

security

Arthur Dent

Ford Prefect

Zaphod Beeblebrox

Marvin

Trillian

left/A(r,w), left/B(r)

left/A(r,w), left/B(r,w), left/X(r)

left/*(r,w)

left/*(r), right/(w)

left/A(r,w), left/B(r,w), right/C(r,w)

Ford Prefect

Zaphod Beeblebrox

Trillian

Marvin

Identity Access RightsSessions are authenticated and communication is encrypted

Only the Topic included as part of the access rights are visible and accessible

SecureFine-grained access control

over Partition/Topic/Instance regular expressions

Arthur Dent

Ford Prefect

Zaphod Beeblebrox

Marvin

Trillian

left/*(r,w)

Ford Prefect

Zaphod Beeblebrox

Trillian

Marvin

SecureSupport for Symmetric and Asymmetric Authentication

Arthur Dent

Ford Prefect

Zaphod Beeblebrox

Marvin

Trillian

left/*(r,w)

Ford Prefect

Zaphod Beeblebrox

Trillian

Marvin

Boundary SecurityBoundary security support is

enabled by Vortex-Fog

Device-to-Cloud Communication

Peer-to-Peer (Broker-less)

Device-to-Device Communication

Fog Computing Fog ComputingFog Computing

Boundary SecuritySeparates security concerns at

different scales and controls what information is exposed

Peer-to-Peer (Broker-less)

Device-to-Device Communication

Fog Computing Fog ComputingFog Computing

Typical Deployment

Vortex device, such as Lite, Café, and Web are used to share data between different kinds of applications within a car

Café and Web are typically used Android / HTML5 based infotainment

Lite in typically used in ECU, sensors and onboard analytics

Vortex Fog is used to transparently (for in car apps) decouple and control the data sharing within and across the car

Vortex Fog also helps defining security boundaries and policies

Vortex Fog efficiently and securely deals with car to car communication

Vortex Fog allows to decouple the transport used for in-car communication and that used for car-to-car communication

Vortex Fog efficiently and securely deals with cloud connectivity adapting traffic flows and protocols

Cloud Analytics

Fog AnalyticsFog Analytics

Fog Analytics

Vortex Cloud efficiently and securely makes data available to any device at an Internet Scale

Cloud Analytics

Fog AnalyticsFog Analytics

Fog Analytics

Use Cases

Connected Boulevard — Nice, FranceVortex enables is the data sharing platform

for the City Operating System

Tens of thousands of devices connected by Vortex

Autonomous Vehicles

coordination of fast moving autonomous vehicles

intermittent connectivity

dynamic pairing of devices

Smart Factory0.5 TB of data

produced per day

Smart-Grid

20ms deadline for phase alignment data

ESA Extremely Large & SMART Telescope (ELT)

1750 computing nodes

100.000 mirrors whose position is

adjusted 100 times per second!

ELT will allow astronomers to probe the earliest stages

of the formation of planetary systems and to detect water and organic

molecules in proto-planetary discs around

stars in the making

Launch System

80K+ data points with aggregate updates rate of ~400K msgs/sec

Connected Aircrafts

10 TB of data every 30m of flight

Oil Rig 30000 data pointsonly 1% of available data

used today

Smart Lightbulbs96Kbytes Memory

Connected Medical Devices

12 msec period for pleath data

Peer-to-Peer connectivity

Vortex enables Smart, Secure and Efficient Data Sharing in IoT

In Summary

getting started with vortex

Technology

wincc flexible getting started básico - grup dap · wincc...

getting started guide - informatica · getting started...

primary intra-school/level 1 resource - challenge card ·...

primary intra-school/level 1 resource - challenge...

250 pro instruction manual - b&h photo · getting started,...

editor - 5a18fcdc5c8aa2617926...

getting started guide - cisco...getting started guide -...

read and save these instructions evaporative vortex...

getting into the vortex meditation user guide

shmc getting started - samway - products getting...

getting started v6 · datastream getting started guide...

getting started getting started getting started to success...

getting started v6 - university of wisconsin–madison ·...

international edition - unmanned tech · getting started,...

getting started chapter 1. getting started getting started

vortex - nintendo snes - manual - gamesdatabase€¦ ·...

getting started v6 - data and information services center...

getting started with excel getting started with excel

250 pro getting started guide - horizon hobby...250 pro...

skaffold - storage.googleapis.com · [getting-started...