build it fast: 5 steps from concept to working distributed system

RTI QuickStart Training

Build it Fast: 5 Steps from Concept to Working Distributed System

Rajive Joshi, Ph.D.Principal Solution Architect

WebinarReal-Time Innovations Inc.September 25, 2013

Once Upon a Late Night …

Agenda

• Why is building distributed systems hard?• The 5 Critical Steps – Best Practice• The Prototyper – Separating Structure and Behavior• Defining structure using XML • Coding behavior using Lua – A Small Fast Dynamic

Scripting Language • Real-World Example• Summary

Why is Building Distributed Systems Hard?

• Logical Design Considerations– Data flows– Data delivery: availability, timing, ordering, reliability, filtering, fault

tolerance, etc.– Component behaviors

• Physical Design Considerations– Platform Differences: CPU, OS, Programming Languages– Discovery and Network configurations– Low Level Device I/O

• Performance & Scalability Considerations– # of data flows– # of components/endpoints– Latency vs. Throughput

The 5 Critical Steps

Articulate Concept1. Draw a diagram of the components and the interconnecting data-flows

Define Structure2. Define the data types for the interconnecting data flows (in IDL or

XML)3. Define the system structure as a collection of data-oriented

component interfaces (in XML)

Configure Behavior4. Code the component behavior (in the Lua scripting language)5. Adjust QoS policies to achieve the desired data-flow behavior

Best Practice

The RTI Connext Platform Continues to Grow…

C/C++/Java/C#/Ada • Code Generation• Edit/Compile/Link/Run

Lua Scripting (in RTI Prototyper Runtime)• Edit/Run(live update)

New!

The RTI Prototyper with Lua

Lua Component

N inputs M outputs

DDS

Prototyper (Container)

The RTI Prototyper with Lua

Lua Engine

DDS

Lua ComponentBehavior

Prototyper (Container)

The RTI Prototyper with Lua

Lua Engine

Lua ComponentBehavior

DDS

Settings(Structure/

Wiring)

Bind the Component

Interface(to data-space)

Prototyper (Container)

The RTI Prototyper with Lua

Lua Engine

Lua ComponentBehavior

DDS

Settings(Structure/

Wiring)

Bind the Component

Interface(to data-space)

Prototyperdetermines

when theLua Component

runs

Prototyper (Container)

The RTI Prototyper with Lua

Lua Engine

Lua ComponentBehavior

DDS

Settings(Structure/

Wiring)

Bind the Component

Interface(to data-space)

Prototyperdetermines

when theLua Component

runs

Lua Component state preserved

across runs(code can change!)

Prototyper (Container)

The RTI Prototyper with Lua

Lua Engine

DDS

Settings(Structure/

Wiring)

RTI Community Portal Download

Lua ComponentBehavior

For details, see:Getting StartedGuide

Bind the Component

Interface(to data-space)

Dynamically Scriptable (in Lua)Distributed Components

(using DDS)

Data Distribution Service (DDS)

The Outcome

Rapid Application

Development

Transformation

xs, ys

ws

xs, ys

xc

shapes/ShapePubSub.lua

Scale

TransformationScriptable

(in Lua)

shapes/ShapePubSub.lua

Transformation - Try it Out Yourself

Data Distribution Service (DDS)

Prototyper:shapes/ShapePubSub.lua

Subscriber(Shapes Demo)

Publisher(Shapes Demo)

Transformation - Try it Out YourselfDynamic

Live Code Update

before

after

shapes/ShapePubSub.lua

Correlation

xs, ys

ws

xc, yc

wc

xs, ys

xc

shapes/Correlator.lua

-- Interface: parameters, inputs, outputslocal reader1 = CONTAINER.READER[1]local reader2 = CONTAINER.READER[2]local writer = CONTAINER.WRITER[#CONTAINER.WRITER]

-- Globals (preserved across invocations)if not shapesize then shapesize={} end -- shapesize of the output stream

-- Cache the 'shapesize' for a color from the 2nd input stream ---reader2:take()for i, shape in ipairs(reader2.sample) do

if (not reader2.info[i].valid_data) then break end

local color = shape['color']shapesize[color] = shape['x']

end

-- Merge the 'shapesize' for a color with x and y from the 1st input stream ---reader1:take()for i, shape in ipairs(reader1.sample) do

if (not reader1.info[i].valid_data) then break end

local color = shape['color’]

writer.instance['color'] = colorwriter.instance['x'] = shape['x']writer.instance['y'] = shape['y']

writer.instance['shapesize'] = shapesize[color] or shape['shapesize']

writer:write()end

shapes/Correlator.lua

How many lines of C/C++/Java code would it take?

Correlation - Try it Out Yourself

Data Distribution Service (DDS)

Prototyper:shapes/Correlator.lua

Subscriber(Shapes Demo)

Publishers(Shapes Demo)

Choreography

xs, ys

ws

xc, yc

wc

xs, ys

xc

Pub-Sub mediation Request-Reply

Pub-Sub

Request-Reply

How many RED objects?

shapes/Choreography.luaChoreography.xml

Splitting

xs, ys

ws

xs, ys

ws

xs, ys

ws

shapes/SplitterDelayNAverage.lua

Delay by N samples

Average over N samples

Aggregation

xs, ys

ws

xc, yc

wc

x, yw

xt, ytWt

shapes/Aggregation.lua

Data Generation/Simulation

Shapes/Flower.lua

xs, ys

xc

Device I/O

xc, yc

wc

shapes/mouse/MouseInputAdapter.luashapes/mouse/mouse.c

Data Capture

xs, ys

ws

xc, yc

wc

xt, ytWt

shapes/ShapeSubscriber.lua

Examples Included in the Download

Real-Time Processing Category ExampleSimulation/Data Generation shapes/Flower.lua

shapes/Figure8.luashapes/ShapePublisher.lua

Data Capture shapes/ShapeSubscriber.lua

Transformation shapes/ShapePubSub.lua

Aggregation shapes/Aggregation.lua

Correlation shapes/Correlator.lua

Splitting shapes/SplitterDelayNAverage.lua

Choreography (pattern mediation) shapes/Choreography.luaChoreography.xml

Device I/O shapes/FileInputAdapter.luashapes/mouse/MouseInputAdapter.lua

Why should I care?

• Fast Development and Deployment– No automatic code generation, compile, or re-start– Be able to try our a variety of ideas quickly and interactively

• Extreme Usability– Intuitive: don’t reinvent, leverage the language– Minimalistic: eliminate accidental complexity– Orthogonal: avoid redundancy, stackable concepts

• Sophisticated Use Cases– Non-trivial, e.g.: correlation, splitting, aggregation, transformation,

choreography, I/O, data collection, data generation, etc.• Separation of Concerns

– Structure vs. Behavior– Developer vs. Integrator Do you care

about time to market?

Agenda

• Why is building distributed systems hard?• The 5 Critical Steps – Best Practice• The Prototyper – Separating Structure and Behavior• Defining structure using XML • Coding behavior using Lua – A Small Fast Dynamic

Scripting Language • Real-World Example• Summary

Prototyper (Container)

The RTI Prototyper with Lua

Lua Engine

DDS

Settings(Structure/

Wiring)

USER_QOS_PROFILES.xml

XML BasedApplication

Configuration

RTI Community Portal Download

Lua ComponentBehavior

For details, see:Getting StartedGuide

Structure

<!-- Domain Library --> <domain_library name="MyDomainLibrary" > <domain name="MyDomain" domain_id="25"> <register_type name="type" kind="dynamicData" type_ref="MyType"/> <topic name="MyTopic" register_type_ref="type"/> </domain> </domain_library>

<!-- Participant library --> <participant_library name="MyParticipantLibrary"> <domain_participant name="MyParticipant" domain_ref="MyDomainLibrary::MyDomain"> <publisher name="MyPublisher"> <data_writer name="MyWriter" topic_ref="MyTopic"/> </publisher> <subscriber name="MySubscriber"> <data_reader name="MyReader" topic_ref="MyTopic"/> </subscriber> </domain_participant> </participant_library>

<types>

Defining structure using XML Settings: XML Based Application Configuration

Prototyper (Container)

The RTI Prototyper with Lua

Lua Engine

DDS

Settings(Structure/

Wiring)

USER_QOS_PROFILES.xml

XML BasedApplication

Configuration

RTI Community Portal Download

Lua ComponentBehavior

For details, see:Getting StartedGuideBehavior

Why Lua?

• Fast– One of the fastest popular scripting

languages (from literature*) • Very Small (~250KB)

– Can be built for a variety of OSes or no-OS

• Easy to Learn• Solid foundation (1993)

– Minimal– Clean

• Embeddable & Extensible– Naturally in C

• Growing Community– Popular in Gaming– Adopted by Wireshark, Eclipse M2M,

Wikipedia, CoronaSDK, etc. – Rich Libraries/Ecosystem

• Open-Source! Free!!

Where can I learn Lua?

www.lua.org

Don’t worry. It’s easy!

Parse XML configuration files

Create DomainParticipant specified by the configuration name

Print valid configuration names

Prompt user for configuration name

Wait For Data to arrive OR ‘period’ to elapse

(whichever happens first)

Execute the Lua Code Component

Lua ‘intentExit’? or Completed ‘runDuration’?

Configuration name Specified?

NO

YES

NO

YES

Prototyper with Lua

Runtime Container Workflow

RTI Prototyper with LuaRuntime Container

• When can the Lua Component run?– On any one or more of the following events• on Start• on Data arrival• on Period (timer)• on Stop

– User Configurable, e.g.• Data (Event) Driven : lua.onPeriod = false• Timer (Polling) Driven : lua.onData = false

– Default: data + timer driven

Lua ComponentBehavior

Lua Component Programming Model

Interface• Incoming data is consumed using a

READER table• Outgoing data is produced using a WRITER

table• Container status and component’s intents

are communicated using a CONTEXT table

N inputs M outputs

CONTAINER.READER[1]

CONTAINER.WRITER[1]

CONTAINER.READER[N] CONTAINER.

WRITER[M]

-- Lua Component Code --

CONTAINER.CONTEXT

Lua Component Code• Decides when to read/take incoming data• Decides when to write outgoing data• Maintains global state across invocations• Dynamically Reconfigurable, i.e. code can

be changed while the container is running

Writing Data

local foo = 'HelloPublisher::HelloWriter’-- or --local foo = 1

local foo_writer = CONTAINER.WRITER[foo]

foo_writer.instance['x'] = 100 foo_writer.instance['y'] = 100 foo_writer.instance['shapesize'] = 30 foo_writer.instance['color'] = "BLUE"

foo_writer:write()

Reading Datalocal foo = 'HelloPublisher::HelloReader’-- or --local foo = 1

local foo_reader = CONTAINER.READER[foo]

foo_reader:take()

for i, shape in ipairs(foo_reader.sample) do print("\t color:", shape['color']) – key print("\t x:", shape['x']) print("\t y:", shape['y']) print("\t shapesize:”, shape['shapesize'])end

Agenda

• Why is building distributed systems hard?• The 5 Critical Steps – Best Practice• The Prototyper – Separating Structure and Behavior• Defining structure using XML • Coding behavior using Lua – A Small Fast Dynamic

Scripting Language • Real-World Example• Summary

Real-World Example: Batch Process Control (ANSI/ISA-88)

Real-World Example: Batch Process Control (ANSI/ISA-88)

Station(s)

Real-World Example: Batch Process Control (ANSI/ISA-88)

Station(s)

Recipe(s)

Real-World Example: Batch Process Control (ANSI/ISA-88)

Station(s)

Recipe(s)

ProductionLot

ProductionLot

ProductionLot

Real-World Example: Chocolate Manufacturing

Recipes

ProductionLot

Stations

ProductionLot

Step 1: Draw a diagram of the components and the interconnecting data-flows

StationController

ProductionLot

Recipe

RecipeConfigurator

To Other StationControllers

From Other Station Controllers

ProductionLot

Step 1: Draw a diagram of the components and the interconnecting data-flows

StationController

ProductionLot

Recipe

RecipeConfigurator

ProductionLot

TaskGenerator

ProductionLot

To Other StationControllers

From Other Station Controllers

Step 2: Define the data types for the interconnecting data flows (in IDL or XML)

Recipe

typedef long StationControlId;

struct RecipeType {// Uniquely identifies the recipestring<64> recipeName; //@key

// Defines the sequence of station // controllers that must be // traversed to make the product

sequence<StationControlId> steps;};

<typedef name="StationControlId" type="long" /> <struct name="RecipeType”>

<member name="recipeName" stringMaxLength="64" type="string" key="true" />

<member name="steps" sequenceMaxLength="-1" type="nonBasic" nonBasicTypeName="StationControlId" /></struct>

XML

IDL

Step 2: Define the data types for the interconnecting data flows (in IDL or XML)

enum LotStatus { WAITING_FOR_SC,PROCESSING_AT_SC,COMPLETED

};

struct ProductionLotType {long lotId; //@key

// Identfies the product string<64> productName;

// Identifies the recipe used string<64> recipeName;

LotStatus status; StationControlId assignedSC;

};

ProductionLotIDL

Step 2: Define the data types for the interconnecting data flows (in IDL or XML)

<enum name="LotStatus" bitBound="32"><enumerator name="WAITING_FOR_SC" /><enumerator name="PROCESSING_AT_SC" /><enumerator name="COMPLETED" />

</enum> <struct name="ProductionLotType">

<member name="lotId" type="long" key="true" /><member name="productName" stringMaxLength="64" type="string" /><member name="recipeName" stringMaxLength="64" type="string" /><member name="status" type="nonBasic"

nonBasicTypeName="LotStatus" /><member name="assignedSC" type="nonBasic”

nonBasicTypeName="StationControlId" /></struct>

ProductionLotXML

Step 3: Define the system structure as a collection of data-oriented component interfaces (in XML)

<domain_library name="FactoryDomainLib"><domain name="ChocolateFactory" domain_id="90"><register_type name="ProductionLotType" kind="dynamicData"

type_ref="FactoryTypes::ProductionLotType"/><register_type name="RecipeType" kind="dynamicData"

type_ref="FactoryTypes::RecipeType"/><topic register_type_ref="ProductionLotType" name="ProductionLot"/><topic register_type_ref="RecipeType" name="Recipe"/>

</domain></domain_library>

ProductionLotRecipe

Step 3: Define the system structure as a collection of data-oriented component interfaces (in XML)

<domain_participant name="RecipeConfigurator" domain_ref="FactoryDomainLib::ChocolateFactory"> <participant_qos base_name="ChocolateManufacture_Library::ChocolateManufacture_Profile"> <participant_name> <name>RecipeConfigurator</name> <role_name>RecipeConfigurator</role_name> </participant_name> <property> <value> <element> <name>lua.file</name> <value>RecipeConfigurator.lua</value> </element> <element> <name>lua.onStart</name> <value>true</value> </element> </value> </property> </participant_qos> <publisher name="RecipePublisher"> <data_writer topic_ref="Recipe" name="RecipeWriter"> <datawriter_qos base_name="ChocolateManufacture_Library::ChocolateManufacture_Profile"/> </data_writer> </publisher> </domain_participant>

Recipe

RecipeConfigurator

Step 3: Define the system structure as a collection of data-oriented component interfaces (in XML)

<participant_library name="FactoryParticipantLib"> <domain_participant name="TaskGenerator" domain_ref="FactoryDomainLib::ChocolateFactory"> <participant_qos base_name="ChocolateManufacture_Library::ChocolateManufacture_Profile"> <participant_name> <name>TaskGenerator</name> <role_name>TaskGenerator</role_name> </participant_name> <property> <value> <element> <name>lua.file</name> <value>TaskGenerator.lua</value> </element> <element> <name>lua.onStart</name> <value>true</value> </element> </value> </property> </participant_qos> <publisher name="TaskPublisher"> <data_writer topic_ref="ProductionLot" name="TaskWriter"> <datawriter_qos base_name="ChocolateManufacture_Library::ChocolateManufacture_Profile"/> </data_writer> </publisher> </domain_participant>

TaskGenerator

ProductionLot

Step 3: Define the system structure as a collection of data-oriented component interfaces (in XML)

<domain_participant name="StationController" domain_ref="FactoryDomainLib::ChocolateFactory"> <participant_qos base_name="ChocolateManufacture_Library::ChocolateManufacture_Profile"> <participant_name> <name>StationController#$(STATION_CONTROLLER_ID)</name> <role_name>StationController</role_name> </participant_name> <property> <value> <element> <name>lua.file</name> <value>StationController.lua</value> </element> <element> <name>lua.onStart</name> <value>true</value> </element> </value> </property> </participant_qos> </domain_participant>

StationController

ProductionLot

ProductionLot

Recipe

Step 3: Define the system structure as a collection of data-oriented component interfaces (in XML)

<publisher name="SCPublisher"> <data_writer topic_ref="ProductionLot" name="LotWriter"> <datawriter_qos base_name="ChocolateManufacture_Library::ChocolateManufacture_Profile"/> </data_writer> </publisher> <subscriber name="SCSubscriber"> <data_reader topic_ref="Recipe" name="RecipeReader"> <datareader_qos base_name="ChocolateManufacture_Library::ChocolateManufacture_Profile"/> </data_reader> <data_reader topic_ref="ProductionLot" name="LotReader"> <datareader_qos base_name="ChocolateManufacture_Library::ChocolateManufacture_Profile"/> <filter name="MyLots" kind="builtin.sql"> <expression>assignedSC = $(STATION_CONTROLLER_ID) AND status = 0</expression> </filter> </data_reader> </subscriber> </domain_participant>

StationController

ProductionLot

ProductionLot

Recipe

Step 4: Code the component behavior in the Lua scripting language

RecipeConfigurator

if ( CONTAINER.CONTEXT.onStartEvent ) then print("Starting RecipeConfigurator") ConfigWriter = PROTOTYPER.WRITER["RecipePublisher::RecipeWriter”] outputRecipe = ConfigWriter.instance

outputRecipe.recipeName = "DarkChocolateRecipe" local stations = { 1, 2, 3, 4, 6, 7, 8, 9, 11, 12, 13 } for i, station in ipairs( stations ) do step = "steps[".. i .."]" outputRecipe[step] = station end ConfigWriter:write() outputRecipe.recipeName = "WhiteChocolateRecipe" local stations = { 1, 2, 4, 5, 6, 7, 8, 9, 11, 12, 13 } for i, station in ipairs( stations ) do step = "steps[".. i .."]" outputRecipe[step] = station end ConfigWriter:write() outputRecipe.recipeName = "MilkChocolateRecipe" local stations = { 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13 } for i, station in ipairs( stations ) do step = "steps[".. i .."]" outputRecipe[step] = station end ConfigWriter:write()end

Step 4: Code the component behavior in the Lua scripting language

if ( CONTAINER.CONTEXT.onStartEvent ) then print("Starting TaskGenerator”) TaskWriter = PROTOTYPER.WRITER["TaskPublisher::TaskWriter"] count = 0end

local taskLot = TaskWriter.instance

-- We use the count to simulate the continuous generation of taskscount = count+1taskLot.lotId = counttaskLot.status = 0taskLot.assignedSC = 1

if (count <= 1) then taskLot.productName = "DarkChocolate" taskLot.recipeName = "DarkChocolateRecipe" taskLot.assignedSC = 1 elseif (count <= 2) then taskLot.productName = "WhiteChocolate" taskLot.recipeName = "WhiteChocolateRecipe" elseif (count <= 3) then taskLot.productName = "MilkChocolate" taskLot.recipeName = "MilkChocolateRecipe"endTaskWriter:write() if ( count > 3 ) then

CONTAINER.CONTEXT.intentExit = true end

TaskGenerator

Step 4: Code the component behavior in the Lua scripting language

-- State initialization to perform the first time the script runsif ( CONTAINER.CONTEXT.onStartEvent ) then

-- Sentinel values returned by recipeGetNextSCNumber() NEXT_STATION_COMPLETED=-1

-- Enumerated values that may appear in the Lot.status LOT_STATUS_WAITING_FOR_SC=0 LOT_STATUS_PROCESSING_AT_SC=1 LOT_STATUS_COMPLETED=2 -- Possible value for the SC's stationState SC_STATE_READY = 'READY'

SC_STATE_PROCESSING = 'PROCESSING' -- The number for this station conroller is passed as an

-- environment variablemySCNumber = tonumber(os.getenv("STATION_CONTROLLER_ID"))print("Starting SC#" .. mySCNumber)

stationState = SC_STATE_READYdelayCount = 0

-- Queues all the lots that are waiting to be processed by the SCtaskQUEUE = {}

-- Indexed by the recipe name. Stores the next SC# for that recipe recipeTable = {}

End

-- Helper functions-- :

StationController

Step 5: Adjust QoS policies to achieve the desired data-flow behavior

RecipeQoS

<datawriter_qos><reliability>

<kind>RELIABLE_RELIABILITY_QOS</kind><max_blocking_time>

<sec>60</sec></max_blocking_time>

</reliability>

<history><kind>KEEP_ALL_HISTORY_QOS</kind>

</history>

<durability><kind>TRANSIENT_LOCAL_DURABILITY_QOS</kind>

</durability></datawriter_qos>

<datareader_qos><reliability>

<kind>RELIABLE_RELIABILITY_QOS</kind></reliability>

<history>

<kind>KEEP_ALL_HISTORY_QOS</kind></history>

<durability><kind>TRANSIENT_LOCAL_DURABILITY_QOS</kind>

</durability></datareader_qos>

ProductionLotQoS

Working Distributed System

Recipes

ProductionLot

Stations

ProductionLot

Station Controllers

Task Generator

Recipe Configurator

Agenda

• Why is building distributed systems hard?• The 5 Critical Steps – Best Practice• The Prototyper – Separating Structure and Behavior• Defining structure using XML • Coding behavior using Lua – A Small Fast Dynamic

Scripting Language • Real-World Example• Summary

Prototyper (Container)

The RTI Prototyper with Lua

Lua Engine

DDS

Settings(Structure/

Wiring)

USER_QOS_PROFILES.xml

XML BasedApplication

Configuration

RTI Community Portal Download

Lua ComponentBehavior

For details, see:Getting StartedGuide

RTI Prototyper with Lua Enables… • Fast Development & Deployment

– No automatic code generation, compile, or re-start– Fewer lines of code– Change behavior (code) on the fly

• Extreme Usability– Engage domain experts (don’t need to be a middleware expert)– Natural and intuitive programming model

• Sophisticated Use Cases– Mediation of communication patterns: pub-sub, request-reply– Non-trivial, eg: correlation, splitting, aggregation, transformation,

choreography, I/O, data collection, data generation, etc.• Separation of Concerns

– Easy to Maintain and Evolve for Large and Small Teams– Developer focused on processing, not infrastructure configuration– System integrator can independently manage configuration & QoS

The 5 Critical Steps

Articulate Concept1. Draw a diagram of the components and the interconnecting data-flows

Define Structure2. Define the data types for the interconnecting data flows (in IDL or

XML)3. Define the system structure as a collection of data-oriented

component interfaces (in XML)

Configure Behavior4. Code the component behavior in the Lua scripting language5. Adjust QoS policies to achieve the desired data-flow behavior

LATER: Optimize selected components in C/C++/Java/C#, but only if necessary!

Key Benefits

• Get stuff done fast(er)! • Quickly try out new ideas, and show a working proof of

concept.• Get more done with the same staff.• Ease into the learning curve of DDS by getting something

up and running first, and then learn more as you need to. • Explore tradeoff between data-model choices.• Experiment with QoS policies for a given data model.• Script test scenarios for existing DDS system!• Test and validate an existing system. Build your own test

harness.

Ready to Ride?• Download it (experimental version):

– community.rti.com Downloads RTI Prototyper with Lua Pick your flavor (Mac, Linux, Windows, Raspberry pi)

• Install it:– Windows: Unzip – Others: Run the installer:chmod +x rti_prototyper_with_lua-5-1.0.0-x64Darwin10gcc4.2.1.run

./rti_prototyper_with_lua-5-1.0.0-x64Darwin10gcc4.2.1.runChoose an existing directory

• Try it:– Apply the 5 Critical Steps to build your working distributed system

…Late Nights No More!

THANK YOU

http://blogs.rti.com/tag/lua/http://community.rti.com/search/site/lua