performance comparison of dtn bundle protocol implementations

Performance Comparison of DTN Bundle ProtocolImplementations

Wolf-Bastian Pottner, Johannes Morgenroth, Sebastian Schildt, Lars Wolf

September 23, 2011: CHANTS Workshop, Las Vegas, NV

Motivation

Several Bundle Protocol (BP) implementations for Linux exist

DTN2: DTNRG’s reference implementation

IBR-DTN: Implementation for embedded OpenWRT devices

ION: JPL’s implementation for spacecrafts running on RTOS

Postellation: Viagenie’s closed-source (?) BP implementation

Spindle: Closed-source BP implementation by Raytheon BBNTechnologies

Performance?

No quantitative evaluation and comparison has been performed(yet)

Pottner, Morgenroth, Schildt, Wolf | Bundle Protocol Performance | 2

Motivation (cont’d)

Why is performance evaluation of BP implementations important?

Contacts are rare and short, communication has to be efficient

Broken interoperability can point out ambiguities in the specs

Goals of this paper

Concentrate on open-source TCPCL implementations for Linux

Quantitative “test bench” performance evaluation

Quantitative opportunistic performance evaluation

Systematic evaluation of interoperability


Metrics

Application Layer Throughput

For different storage backends

Between different implementations

In an opportunistic scenario

Interoperability

Can implementations discover each other?

Throughput between different implementations

Communication Efficiency

In an opportunistic scenario


Experimental Setup

Hardware

Athlon II X4 2.8 GHz CPUs with 4 GiB RAM

1 GBit Ethernet (good for 940 MBit/s TCP goodput with iperf)

Different Ethernet ports for measurement and control traffic

Samsung Spinpoint F3 500GB (HD502HJ) hard drive

Software

Ubuntu Linux 11.04

DTN2 v2.7, IBR-DTN v0.6.3, ION v2.4.0

Connection blocked via iptables (IBR-DTN) or daemon’s controlinterface (DTN2, ION)


Storage Backends: BP Throughput

0

200

400

600

800

1000

1 10 100 1000 10000 100000 1e+06 1e+07

Bu

nd

le T

hro

ug

hp

ut

[MB

it/s

]

Bundle Payload Size [bytes]

DTN2 DiskDTN2 Mem

0

200

400

600

800

1000

1 10 100 1000 10000 100000 1e+06 1e+07

Bu

nd

le T

hro

ug

hp

ut

[MB

it/s

]


DTN2 DiskDTN2 Mem

IBR-DTN DiskIBR-DTN Mem

0

200

400

600

800

1000

1 10 100 1000 10000 100000 1e+06 1e+07

Bu

nd

le T

hro

ug

hp

ut

[MB

it/s

]


DTN2 DiskDTN2 Mem


ION Disk (T)ION Mem (T)


Interoperability: Discovery, Bundle Exchange

Discovery Capabilities

DTN2 uses a “proprietary” discovery beacon format

IBR-DTN supports IPND v0 & v1 and DTN2 discovery format

ION is focussed on scheduled contacts only

Interoperability Evaluation Results

DTN2 and IBR-DTN discover each other just fine

Bundles can be exchanged in both directions

ION exchanges bundles with DTN2 and IBR-DTN


Interoperability: BP Throughput

Throughput (memory storage, 1 MByte payload size)HHHH

HHTX

RXDTN2 IBR-DTN ION

DTN2 687 MBit/s 635 MBit/s 93 MBit/s

IBR-DTN 881 MBit/s 679 MBit/s 90 MBit/s

ION 872 MBit/s 926 MBit/s 449 MBit/s

Observations

DTN2 may have a bottleneck in the TX component

IBR-DTN and ION show unclear bottlenecksshows unclearbottlenecks

Fastest result is between ION and IBR-DTN


Opportunistic Scenario: Setup

Scenario

A M B

Memory-based storage backends

A creates bundles for B at constant rate, M commutes from A to B

TCP throughput limited to 12.7 MBit/s (based on measurements)

Cycle: 72 s contact time, 30 s travel, 72 s contact time, 30 s travel

(almost) 9 complete cycles in 30 minutes measurement time


Opportunistic Scenario: Transferred Data

0

200

400

600

800

1000

250000 500000 1e+06 2e+06

Tra

nsf

erre

d D

ata

[MB

]


DTN2IBR-DTN

theoretical Maximum


Opportunistic Scenario: Communication Efficiency

0

20

40

60

80

100

250000 500000 1e+06 2e+06

Co

mm

un

icat

ion

Eff

icie

ncy

[%

]


DTN2IBR-DTN

theoretical Maximum


Conclusions

Motivation

Performance of BP implementations is important!

Comparison of three BP implementations: DTN2, IBR-DTN, ION

Results

All three implementations are interoperable

Performance varies widely and unpredictably

Fastest result between ION and IBR-DTN

Working with the implementations is not easy

More results

More results in our technical report (WiP, look for updates)

Mail me: [email protected]


BACKUP SLIDES

BACKUP SLIDES


Throughput (linear y)

0

200

400

600

800

1000

1 10 100 1000 10000 100000 1e+06 1e+07

Bundle

Thro

ughput

[MB

it/s

]


DTN2 DiskDTN2 Mem




Throughput (logarithmic y)

0.001

0.01

0.1

1

10

100

1000

1 10 100 1000 10000 100000 1e+06 1e+07

Bundle

Thro

ughput

[MB

it/s

]


DTN2 DiskDTN2 Mem




DTN2 Slowdown

0

50

100

150

200

250

1 2 3 4 5 6 7 8 9 10

Thro

ughput

[MB

it/s

]

Measurement Run

50e+03 Bytes Payload 100e+03 Bytes Payload

500e+03 Bytes Payload1000e+03 Bytes Payload


Interoperability: BP Throughput

Throughput (memory storage, 100 KByte payload size)HH

HHHHTX

RXDTN2 IBR-DTN ION




Throughput (memory storage, 1 MByte payload size)HHH

HHHTX

RXDTN2 IBR-DTN ION





performance comparison of dtn bundle protocol implementations

Documents