logistical networking as an advanced engineering testbed micah beck, assoc. prof. & director...

Logistical Networking as an Advanced Engineering Testbed

Micah Beck, Assoc. Prof. & DirectorLogistical Computing & Internetworking (LoCI) Lab

mbeck@cs.utk.edu

Internet2 Spring Member Meeting April 10, 2003

Panel Participants

• Gabriella Paolini, GARR (by phone)Topic: IPv6

• Geoff Hayward, Yotta YottaTopic: Wide Area Storage Networking

• Jim Ferguson, NCSATopic: Web 100 / TCP Tuning

Funding• Dept. of Energy

SciDAC

• National Science Foundation ANIR

• UT Center for Info Technology Research

Logistical Networking Research at UTK

University of Tennessee

• Micah Beck• James S. Plank• Jack Dongarra

University of California, Santa Barbara

• Rich Wolski

What is Logistical Networking

• A scalable mechanism for deploying shared storage resources throughout the network

• A general store-and-forward overlay networking infrastructure

• A way to break transfers into segments and employ heterogeneous network technologies on the pieces

Why “Logistical Networking”

• Analogy to logistics in distribution of industrial and military personnel & materiel

• Fast highways alone are not enough Goods are also stored in warehouses for

transfer or local distribution

• Fast networks alone are not enough Data must be stored in buffers/files for

transfer or local distribution

The Network Storage Stack

Applications

Logistical File System

Logistical Tools

L-Bone

IBP

Local Access

Physical

exNode

• Our adaption of the network stack architecture for storage• Like the IP Stack• Each level encapsulates details from the lower levels, while still exposing details to higher levels

IBP: The Internet Backplane Protocol

• Storage provisioned on community “depots”• Very primitive service (similar to block service, but

more sharable)• Goal is to be a common platform (exposed)• Also part of end-to-end design

• Best effort service – no heroic measures• Availability, reliability, security, performance

• Allocations are time-limited!• Leases are respected, can be renewed• Permanent storage is to strong to share!

Models of Sharing: Logistical Networking

Moderately valuable resources

• Storage, server cycles

Sharing enabled by relative plenty

Internet-like policies• Loose access control

• No per-use accounting

Primary design goal: scalability

• Application autonomy

• Resource transparency

Burdens of scalability• The End-to-End Principles

• Weak operation semantics

• Vulnerability to Denial of Service

Data Movers

• Module implementing standard point-to-multipoint transfer between IBP allocations

• Uniform API allows independence from the underlying data transfer protocol

• Not every DM can apply to every transfer• Caller responsible for determining validity

• Current options: Multi-TCP, Multi-UDP (reliable), UDP Multicast (unreliable)

The Network Storage Stack

The L-bone:Resource Discovery& Proximity queries

IBP: Allocating and managing networkstorage (like a network malloc)

The exNode:A data structurefor aggregation

LoRS: The Logistical Runtime System:Aggregation tools and methodologies

The Logistical Backbone (L-Bone)

• LDAP-based storage resource discovery.

• Query by capacity, network proximity, geographical proximity, stability, etc.

• Periodic monitoring of depots.

• 10 Terabytes of shared storage. (with plans to scale to a petabyte...)

L-Bone: January 2003

Current Storage Capacity: 10 TB

The Network Storage Stack

The L-bone:Resource Discovery& Proximity queries

IBP: Allocating and managing networkstorage (like a network malloc)

The exNode:A data structurefor aggregation

LoRS: The Logistical Runtime System:Aggregation tools and methodologies

The exNode

• The Network “File Descriptor

• XML-based data structure/serialization

• Map byte-extents to IBP buffers (or other allocations).

• Allows for replication, flexible decomposition of data.

• Also allows for error-correction/checksums

• Arbitrary metadata.

ExNode vs inode

exNode

inode

IBP Allocations

the network

local system

disk blocks

kernel

capabilities

block addresses

user

The Network Storage Stack

The L-bone:Resource Discovery& Proximity queries

IBP: Allocating and managing networkstorage (like a network malloc)

The exNode:A data structurefor aggregation

LoRS: The Logistical Runtime System:Aggregation tools and methodologies

Logistical Runtime System

Basic Primitives:• Upload, Download, Augment, Refresh

End-to-end Services• Checksums, Encryption, Compression

Download Movie

Multithreaded Transfers

Routed/Multipath

Point-to-Multipoint

Heterogeneous Multicast

Caching/Staging

Further Advanced Capabilities

• IBP over IPv6• Dual stack depot

• Specialized DataMovers• UDP (SABUL, Tsunami)• Fiber Channel over IP

• Non-standard TCP Stacks• Web 100• Future: FAST?

Panel Participants

• Gabriella Paolini, GARR (by phone)Topic: IPv6

• Geoff Hayward, Yotta YottaTopic: Wide Area Storage Networking

• Jim Ferguson, NCSATopic: Web 100 / TCP Tuning

Conclusions

• IBP is a global testbed for advanced network engineering

• Transfer rates routinely exceed 100Mbps

• New Data Movers under development can reach current applications

• Dedicated depots can support global testbed for kernel modificaitons