PTIDES Project Overview

Slobodan Matic, Jia ZouEdward Lee, John Eidson

ActionWebs, 09/29/10

Courtesy of Kuka Robotics Corp.

Orchestrating networked computational resources with physical systems

Courtesy of Doug Schmidt

Power generation and distribution

Courtesy of General Electric

Military systems:

E-Corner, Siemens

Transportation(Air traffic control at SFO)

AvionicsTelecommunications

Factory automation

Instrumentation(Soleil Synchrotron)

Daimler-Chrysler

Automotive

Building Systems

Deterministic Implementations of Event-driven Software

Current Practice: Printing Press

• High-speed, high precision• Speed: 1 inch/ms• Precision: 0.01 inch

-> Time accuracy < 10us• Application aspects

• local (control)• distributed (coordination)• global (modes)

• Open standards (Ethernet)• Synchronous, Time-Triggered• IEEE 1588 time-sync protocolBosch-Rexroth

For distributed cyber-physical systems,

Timing needs to be a part of the network semantics, not a side effect of the implementation.

Technologies needed:– Time synchronization– Bounds on latency– Time-aware fault isolation and recovery

A Programming Model for Distributed Cyber-Physical Systems

The question we address:Given a common notion of time shared to some known precision across a network, and given bounded network latencies, can we design better distributed embedded software?

Our answer (PTIDES): Use discrete-event (DE) models for specification of systems, bind model time to real time only exactly where this is needed.

Outline• Introduction• PTIDES Basics and Workflow• PTIDES Runtime Framework• Single CPU: Tunneling Ball Device• Multiple CPU: Power Distribution Subsystem • Summary

PTIDES Basics• Programming Temporally Integrated Distributed Embedded Systems

– Based on Discrete-Event model of computation• Event processing in time-stamp order• Deterministic under simple causality conditions

– Fixed-point semantics– Can be extended to super-dense form

– Timing constraints for interaction with environment

Model vs. Physical Time

t ≥ τ

t ≤ τ

i4

doτ1

• At sensors and actuators• Relate model time (τ) to physical time (t)

model time

physical time

τ10

t1

τ4

t40

Single Processor PTIDES Example

t ≥ τ , t ≤ τ + do

t ≤ τ

i4

doτ1

τ2

model time

physical time

τ2

t2

e2 at i2

0

0

• Bounded delay between any two hardware components • Bounded sensor latency (d0)

Single Processor PTIDES Example

t ≥ τ , t ≤ τ + do

t ≤ τ

i4

doτ1

τ2

model time

physical time

τ2

t2 τ2+d0

e2 safe to process if t > τ2 + do

0

0

Single Processor PTIDES Example

t ≥ τ , t ≤ τ + do

t ≤ τ

i4

doτ1

τ2

model time

physical time t2 τ1+d0

τ10

0

e2 safe to process if t > τ2 + do

d 1

τ1

τ2

Distributed PTIDES Example

d01

Sensor

Actuator

do2

Network Interface

o3

τ

d 2

τ3

τ4

• Local event processing decisions:• Bounded communication latency (d0)• Distributed platforms time-synchronized with bounded error (e)

Merge

τ is safe to process if: t > τ + do2 + e - d 2

Distributed PTIDES Example

d01

Sensor

Actuator

do2

Network Interface

o3

τ

d 2

τ3

τ4

• Local event processing decisions:• Bounded communication latency (d0)• Distributed platforms time-synchronized with bounded error (e)

Merge

τ1

d 1

τ is safe to process if: t > τ + do2 + e - d 2

What Did We Gain?

e1 = (v1, τ1)Merge

δsafe to process analysis for e

First Point: Ensures deterministic data outputs

e2 = (v2, τ2)

safe to process analysis for e

e1, e2, …

t ≤ τ + do

t ≤ τ

i4

doτ1

τ2

Second Point: Ensures deterministic timing delay from Sensor to Actuator

PTIDES Program Design Workflow

HW Platform

PTIDES run-time

PTIDES Runtime Framework– Software components (actors) are “glued together” by a code

generator into an executable– Lightweight real-time kernel (5KB)– Actors are state machines, each with an event queue and context

• Asynchronous delivery: producer posts to event queue• Framework handles thread-safe event exchange and queuing

– Run-to-completion : No blocking code• No busy-waiting, semaphore, mailbox• Preemptive

-> Single-stack implementation– Encapsulate each resource inside a dedicated active object

• rest of app shares the dedicated active object via events – Fast and deterministic memory allocation

• Event pools: fixed-size memory blocks

PTIDES Scheduler• Two layer execution engine

– Event coordination (safe-to-process)– Event scheduling (prioritize safe events)

• Earliest Deadline First foundation– Deadline based on path from input port to actuator

• Schedulability analysis requires WCET of software components + event models

• PTIDES implementation delivers DE semantics even w/o WCET– Even if deadlines missed, events are determinate – PTIDES repeatable even if WCET wrong

PTIDESWorkflow

HW PlatformSoftware Component

Library

Ptides Model Code Generator

PTIDES run-time

Code

Plant Model

Network Model

HW in the Loop

Simulator

Causality Analysis Program Analysis

Schedulability Analysis

Analysis

Mixed Simulator

Schedulability Analysis

• Three cases:– Zero event processing time assumption (feasibility test)

• if P fails, P will not satisfy constraints on any hardware

– No resource sharing assumption (an event is processed as soon it is safe)

• if P fails, P may still satisfy constraints on other hardware

– Resource sharing (a safe event is processed according to a scheduling algorithm)

• if P fails, P does not satisfy this implementation (and algorithm)

Feasibility Test• Feasibility assumptions

– Zero event processing time– Bounds on event communication latency (d0)

• If test fails, no HW satisfies constraints– detect flows

Feasibility Test• For each input port i on a sensor-actuator path

Single CPU - Proof of Concept• Tunneling Ball Device

– sense ball– track disk– adjust trajectory

Single CPU Proof of Concept• Event sequences

– periodic– quasi-periodic– sporadic

Tunneling Ball Device – 10 rps

Tunneling Ball Device – 1 rps

Distributed Testbed Objectives• Safe event processing

– Offset attribute of a port• Check: real time vs model time + offset

– Bounded latency between hardware components• Distribution: bounded-delay networking protocol

• Scheduling– Deadlines defined by sensor – actuator model delays

• Single CPU: use Earliest Deadline First

– Distribution: local from end-to-end deadlines• Expressiveness of interaction

– Complexity of real sensor, actuator and network interfaces• Timing support

– External to microprocessor • National Semiconductor DP83640: IEEE 1588 enabled PHY chip

Synchrophasor-based measurement and control

Power swing andUnstability detection

ExperimentDiagram

Grid emulator built with National Instruments PXI

‘PMU’s built with Renesas demo boards with DP83640

Ethernet bridge or 1588 boundary/transparent clock

‘SVP’ built with Renesas demo board with DP83640

Basic PTIDESTiming Testing

Vary phasor data independentlyFreq. and phases w.r.t. global time

Sample voltage and currentSignal processingSend phasor dataLocal control: on/off breaker

Wireshark monitoring of network events

Detect discrepanciesIf unstable region send on/off command

Single platform PTIDES approximation model

Multi-platform PTIDES approximation model

Summary

• The networking problem requires timing to be a correctness property rather than a quality of service

• PTIDES model of computation offers a possible programming model for distributed CPS– Correct deterministic DE implementations– Better match for reactive state-machine systems than traditional multi-

threaded or super loop programming models– Time stamps provides runtime information for detecting and reacting to variety

of timing faults

• Goal: scalability, robustness, fault-tolerance