Software Development Infrastructure for Sensor Networks

• Operating systems (TinyOS)

– Resource (device) management– Basic primitives– Protocols (MAC, routing) {covered in many previous lectures}

• Programming languages and runtime environments– Low level (C, nesC)– Very high level abstractions (Regiment)

• Services needed– Synchronization of clocks (RBS)– Propagation of new code to all nodes (Trickle)– Localization {not today}

Fine-Grained NetworkTime Synchronization

Using Reference Broadcasts

Jeremy Elson, Lew Girod, and Deborah Estrin

University of California, Los Angeles

OSDI 2002 - Boston, MA

Used with permission of author

The bigger picture• Isn’t this a solved problem by now???

– NTP, many other clock agreement algorithms, MACs with sync built in (802.11), time broadcasts (GPS, WWVB), high-stability oscillators (Rubidium, Cesium)

• If this isn’t the Internet:– Important assumptions no longer hold

• (fewer resources -- such as energy, good connectivity, infrastructure, size, and cost -- are available …)

– Sensor apps have stronger requirements• (…but we have to do better than the Internet anyway)

BUT...

A palette of sync methods

Time Sync Parameter Space:(max error, lifetime, scope, etc.)

Application Requirement

Available SyncMethods

Better

Better

Goal: make the set rich enough to minimize waste

A palette of sync methods

Time Sync Parameter Space:(max error, lifetime, scope, etc.)

Application Requirement

Better

Better

Goal: make the set rich enough to minimize waste

Ideally, methodsshould be tunable

Time Definitions

• Clock stability – how well it maintains a constant frequency– Short term – temperature

– Long term – aging of oscillator

• Clock accuracy – how well its frequency and time compare with standard

• Offset – time difference between 2 clocks

• Skew – frequency difference between 2 clocks

Traditional sync

Sender Receiver

At the tone: t=1

NIC

Physical Media

NIC

Send time

Access Time

Propagation Time

Receive Time

Problem: Many sources of unknown, nondeterministiclatency between timestamp and its reception

Reference Broadcasts

Sender Receiver

NIC

Physical Media

NIC

Propagation Time

Receive Time

Sync 2 receivers with each other, NOT sender with receiver

Receiver

NICI saw itat t=4 I saw it

at t=5

NICSender

Receiver 1

Receiver 2

Critical Path

NICSender

Receiver

Critical PathTime

RBS reduces error byremoving much of it from the critical path

Traditional critical path:From the time the sender

reads its clock, to when the receiver reads its clock

RBS: Only sensitive to the differences in receive time

and propagation delay

Receiver Determinism

1st testbed: Berkeley motes with narrowband (19.2K) radios

Appears Guassian

Well-Behaved = Good

• Well behaved distributions are useful– Error can be reduced statistically, by sending

multiple pulses over time and averaging

– Also, easier to model/simulate

Ignoring clock skew

• Server broadcasts m reference beacons

• Each of n receivers records local time of each m refs

• Exchange:

Offset[i,j] = 1/m Tj,k – Ti,k)k=1

m

Problem: Clock Skew

• It takes time to send multiple pulses

• By the time we do, clocks will have drifted

• So: don’t average; fit a line instead

Time

Frequency & phase of my clock wrt yours -recovered from slopeand intercept

Comparison to NTP• Second implementation:

– Compaq IPAQs (small Linux machines)

– 11mbit 802.11 PCMCIA cards

• Ran NTP, RBS-Userspace, RBS-Kernel– NTP synced to GPS clock every 16 secs

– NTP with phase correction, too; it did worse (!)

• In each case, asked 2 IPAQs to raise a GPIO line high at the same time; differences measured with logic analyzer

How NTP works

• Multiple synchronization peers provide redundancy and diversity• Clock filters select best from a window of eight clock offset samples• Intersection and clustering algorithms pick best subset of servers

believed to be accurate and fault-free• Combining algorithm computes weighted average of offsets for best

accuracy• Phase/frequency-lock feedback loop disciplines local clock time and

frequency to maximize accuracy and stability

NTP Messages

Peer 1

Peer 2

Filter 1

Peer 3

Filter 2

Filter 3

Intersectionand

ClusteringAlgorithms

CombiningAlgorithm

Loop Filter

VFOTimestamps

P/F-Lock Loop

© Mills

Clock Resolution

Clock Resolution

RBS degraded slightly (6us to 8us); NTP degraded severely (51us to 1542us)

“Here 0 sec after blue pulse!”

“Here 1 sec after blue pulse!”

“Here 3 sec afterred pulse!”

“Here 1 sec afterred pulse!”

Multi-Hop RBS• Some nodes broadcast RF synchronization pulses

• Receivers in a neighborhood are synced by using the pulse as a time reference. (The pulse senders are not synced.)

• Nodes that hear both can relate the time bases to each other

“Red pulse 2 secafter blue pulse!”

1

3

2

A4

8

C

5

7

6B

10

D11

9

Time RoutingConsider a physical topology consisting of

broadcasters (A, B, C..) andreceivers (1, 2, 3…)

(In reality, a node can play both roles)

1

3

2

A4

8

C

5

7

6B

10

D11

9

1

3

2

4

8

5

7

6

10 11

9

Time RoutingThe physical topology can be easily

converted to a logical topology; links represent possible clock conversions

Use shortest path search to find a “time route”;Edges can be weighted by error estimates

Multi-Hop RBS

0

1

2

3

4

5

6

7

1 Hop 2 Hop 3 Hop 4 Hop

Err

or

(use

c)

Std Dev

Error

1.85 +/- 1.28

2.73 +/- 1.912.73 +/- 2.42

3.68 +/- 2.57

Error (and std dev) over multiple hops, in usec

Summary• RBS can improve accuracy by removing

sender from the critical path

• Multi-hop algorithm can extend RBS property across broadcast domains, and to external standards such as UTC

• Implemented on 4 different CPU/radio platforms; no MAC tinkering required

• Facilitates post-facto sync (save energy by only syncing after an event of interest) and peer to peer sync (no global timescale)

Applications

• Acoustic Ranging

• Collaborative Signal Detection

• etc...

• Future work:– Use higher precision clocks (e.g. Pentium TSC)

– Better outlier rejection, weighting

Software Development Infrastructure for Sensor Networks

• Operating systems (TinyOS)

– Resource (device) management– Basic primitives– Protocols (MAC, routing) {covered in many previous lectures}

• Programming languages and runtime environments– Low level (C, nesC)– Very high level abstractions (Regiment)

• Services needed– Synchronization of clocks (RBS)– Propagation of new code to all nodes (Trickle)– Localization {not today}