A Distributed Programming Infrastructure for Integrating Smart SensorsUmakishore Ramachandran, Kenneth Mackenzie, Steve DeWeerth, Irfan Essa, Thad Starner

College of Computing, Georgia Institute of Technology

Problem: Octopus Apps!

SAMPLE APPLICATIONS• distributed collaboration• aware spaces• smart environments• monitoring, control• surveillance• emergency response

BASIC IDEA• emerging app class• tentacles: sensors, actuators• arms: data fusion, routing• head: cpu-intensive processing

FEATURES• distributed, pervasive infrastructure• of widely varying device capabilities• with control-loop flavor processing• on streams of varying bandwidths• requiring rapid response• at human perceptual speeds

Approach: Smart Plumbing

Stampede: SeamlessProgramming

Results:

Sensor Stack:

MediumAccessControl

DataServiceLayer

Data FusionLayer

Application

HelperServiceLayer

InfoExchange

Service

Hardware

Stack Diagram

MediumAccessError ControlRadio Control

RoutingFiltering

ScatterGather

In-networkFusion

Deployment, Monitoring

LocationService

Timing

InfoCollection

AccessControl,Attribute

Translation,Persistent database

Radio, Sensors, Memory, CPU

Functionality

GOALS• in-stack fusion• logical naming• application-awareness

OBSERVATIONS• concurrent apps• energy, net bandwidthconstraints

Media Broker:

DFuse:

TV Watcher:

• seamless programming• across diverse hardware• of compelling applications• reveals middleware requirements

REQUIREMENTS• support for physically distributed heterogeneous devices• easy access to compute-servers (clusters, grids)• diverse computation, communication and power capabilities• support for dynamic join/leave, registration, discovery• sophisticated stream management (fusion, type-baseddiscovery, publication, discovery, filter framework)

Broker federation

StampedeRegistry

audiovideo

Re-publish

transform

share

derive

• federated data distribution• publish/subscribe model• internal data broker threads• type-lattice based transcoding

stream registration andtransformation engine

“An Architecture for Event Web”Modahl, Bagrak, Wolenetz, Jain, RamachandranIEEE FTDCS ’04, Suzhou, China

“DFuse: A Framework for Distributed Data Fusion”Kumar, Wolenetz, Agarwalla, Shin, Hutto, Paul, RamachandranACM SenSys ’03, Los Angeles, California 2003

“Media Broker: An Architecture for Pervasive Computing”Modahl, Bagrak, Wolenetz, Hutto, RamachandranIEEE PerCom ’04, Orlando, Florida

http://www.cc.gatech.edu/~rama/ubiq-presence

Funded by NSF ITR/SY grant CCR-0121638

Event Web:

StreamServer

D-Stampede Cluster

Workstations

Media Capture Clients

Display Clients

www www www www www

Key

www

Generic Workstation

Video Capture System

Video Display Client

Web Results Clientwww

distributed media analysis and correlation

• architecture and application• automates stream capture, feature extraction, correlation• identifies most related streams

optimized fusion function placement in wired and wireless networks

Fusion Channel (a ‘Virtual Sensor’)

Producers

(sensors or other fusion channels)

Consumers

(actuators or other fusion channels)

. . . . . .

f()

DisplayFilter

Collage

Sources

S1

S2

S3

Task Graph

Testbed: IPAQ Farm

ROLE ASSIGNMENT• Naïve tree building• Optimization• Maintenance

Fusion Module

Placement Module

Resource Monitor,Routing Layer

Operating System

Hardware

Cost Function

(Minimize Transmission Cost)

0

10

20

30

40

50

60

70

80

90

100

Run Time(normalized)

Remaining Capacity(%)

Number of RoleTransfers (absolute)

MT2

MPV

MTP

simplified capture and rich access to structured media stores, organized around spatiotemporal events

Feature Extraction

i_conn

o_conn

domain

thread

Dynamic thread-channel graph

channel

BASIC IDEA• space-time memory• time-sequenced data streams•communication abstractions

• channels, queues, registers• distributed garbage collection• computation as thread-channel graph

1 2 3 54 6 7 8

KT=2

Garbage collection

ChangeDetection

Model 1Location

DigitizerVideoFrame

Histogram

MotionMask

TargetDetection

TargetDetection

HistogramModel

Model 2Location

Application: SmartKiosk People Tracker

Application

ResourceManagement

Sensor Access andManagement

Experiential EventWeb Browser

Feature Extraction and Event Generators

Query Server Event

BaseProgramming Abstractions

Media Streaming Engine

Sensor Network

Media Warehouse

Applications

Architecture

Domain Events(CS 6250 Lecture)

Elemental Events(Identity/Location)

Data Events(Face, Moving Lip Detectors)

Group MeetingTime: 10:00am-11:00am

Location: CCB201

Participants: Kishore Ramachandran, Ramesh Jain, Matthew Wolenetz