Internet2 QoS: Baby Steps and Big Lessons

Ben Teitelbaum <ben@internet2.edu>July 26th, 2001

IBM Internet2 Day

IBM Internet2 Day (July 26th, 2001) 2

The Goal

“The Holy Grail of computer networking is to design a network that has the flexibility and low cost of the Internet, yet offers the end-to-end quality of service of the telephone network.”

- S. Keshav (An Engineering Approach to Computer Networking, Addison-Wesley, 1997.)

IBM Internet2 Day (July 26th, 2001) 3

The Environment

“Wavelength division multiplexing (WDM) lambdas are proliferating quicker than Moore’s Law, with total bandwidth per optical fiber increasing faster still-perhaps four times a year...[and] the rate of change is speeding up...The Law of Wasted Bandwidth tells us that the companies that exploit bandwidth recklessly, will win.”

- George Gilder (Gilder Technology Report, January 2001)

IBM Internet2 Day (July 26th, 2001) 4

architecture

A Brief History of Internet2 QoS

Began chanting: “enable advanced applications,…”

Assessed requirements “What do you want?”“What can you give us?”

Recommended DiffServ and Jacobson-style Premium Service

Charted QBone initiative Developed architecture and proceeded to implement it

Frustration and disillusionment Refocusing on lighter-weight services (e.g. QBSS) and reëvaluation of application needs

frustration

deployment

new approaches

reëvaluation

IBM Internet2 Day (July 26th, 2001) 5

QBone Architecture

A Service: QBone Premium Service Built on Expedited Forwarding (EF) (RFC 2598) Assurance: near-zero loss & low, bounded jitter for marked traffic conforming to a specified peak rate

a.k.a. virtual leased line, virtual wire

Reservation Setup ProtocolNow: long-lived, manual setupProposed: SIBBS protocol between QBone domains; RSVP end-to-end between hosts

QBone Measurement Architecture Uniform collection of QoS metrics Uniform dissemination interface

IBM Internet2 Day (July 26th, 2001) 6

QBone Architecture (30 kilofoot view)

Architecture focuses on interdomain interfaces...

Edge-to-edge servicesSignalingMeasurement

GigaPoPA

CampusA

CampusC

CampusD

Backbone

CampusB

GigaPoPB

...and how edge-to-edge services concatenate to form an e2e service

Each domain needs to think in terms of provisioning edge-to-edge “virtual trunks” (policed on ingress / shaped on egress)

IBM Internet2 Day (July 26th, 2001) 7

QBone Architecture v2.0

Architecture under review by design team Collaboration with TF-NGN Premium-IP group Clarifying service assurance Fixing known bugs in measurement architecture Clarifying implications for operators and SLAs

Big issues: How to balance serviceideal with need for incremental deployment?

Delay assurance? Routing-based edge classification Exactly what policers/shapers are needed?

GigaPoPB

Backbone ?

IBM Internet2 Day (July 26th, 2001) 8

QPS Status (Good News)

The Good News: With enough elbow grease, e2e services can be built DiffServ building blocks increasingly available in modern routers

Many host OSs capable of QoS signaling (RSVP) Numerous DiffServ testbed trials Partial deployment in Internet2 backbones Has enabled some networking research

On the Impact of Policing and Rate Guarantees in Diff-Serv Networks: A Video Streaming Application Perspective, R. Guerin et al. (to be presented at SIGCOMM 2001)

Aggregation and Scalable QoS: A Performance Study, H. Fu, E. Knightly (IWQoS 2001)

IBM Internet2 Day (July 26th, 2001) 9

IBM Internet2 Day (July 26th, 2001) 10

QPS Status (Bad News)

The Bad News: Low demand for QPS (app/net chicken/egg dynamic) Router support for DiffServ is spotty; E.g...

no PQDiffServ comes with a performance cost limitations on token bucket depths inflexible classification rules

Elbow grease in short supply in most campus IT shops Building a QPS cloud requires “upgrading the world”

IBM Internet2 Day (July 26th, 2001) 11

QBone Scavenger Service

Basic ideaVoluntary marking hints to network that degraded service is OK (like Un*x nice for the network)

Scavenger traffic may be degraded at congestion points Thin best-effort network that can expand to full capacity in absence of congestion

Goals A tool to preserve/extend uncongested BE experience for interactive applications

IBM Internet2 Day (July 26th, 2001) 12

Motivations1/3

All traffic is not equal Mix of critical/non-critical traffic Since you may be competing with yourself for downstream resources, it’s in your interest to identify non-critical traffic to protect your critical traffic

Most routers support multiple queues Let’s get some value and experience out of them!

Internet2 utilization very lowPro: interactive apps work fine; Con: what a waste What new applications could be built if we weren’t shy about filling the pipes?

IBM Internet2 Day (July 26th, 2001) 13

Motivations2/3

Congestion points Yes Virginia, they do exist! If Internet2 E2E-PI succeeds, they will continue to exist

Fine-grained Netiquette Self-policing users exist (bulk transfers run at night)

HEP communityAdmin functionality (e.g. backups)

Let’s let these folks run their flows all day long

Pricing Additional control over upstream commodity usage Potential negotiation point with commodity providors

IBM Internet2 Day (July 26th, 2001) 14

Motivations3/3

Policy Users/institutions could mark non-mission traffic

IBM Internet2 Day (July 26th, 2001) 15

QBSS Service Definition

Voluntary marking for QBSS Packets indicated by DSCP=001000 Codepoint has global significance Extra-semantics for class selector PHB

Domain requirements Must propagate QBSS DSCP Router requirements

Forward independently giving it a lower probability of timely forwarding OR forward in same manner as BE

Offer a very small minimum departure rate (unlike BH PDB)QBSS gets all un-used bandwidth

IBM Internet2 Day (July 26th, 2001) 16

Implications

Very nice deployment story On day one...

Near ubiquitous deploymentShould just work with multicastTechnology transfer begins

Incremental deployment at congestion points to restore uncongested best-effort service

Policy? Pricing?

Performance?

IBM Internet2 Day (July 26th, 2001) 17

Where is Growth Coming From?

Current growth From three university dorm nets

Working with... GRAPE / TRANSPAC CERN SLAC Folding@Home / Cosm library

Applications now QBSS-enabled wu-ftp Apache bbftp

IBM Internet2 Day (July 26th, 2001) 18

Summary

QBone testbed architecture Still makes sense Some tweaks underway

Working on deploying two services QBSS (fast deployment path) QPS (slow deployment path)

How to participate Join the wg-qos@internet2.edu mailing list Join a QoS WG activity

Architecture DTScavenger DT

Application QoS Needs DTSignaling DT

IBM Internet2 Day (July 26th, 2001) 19

For more information...

Internet2 Home: http://www.internet2.edu/

Internet2 QoS Working Group Home: http://www.internet2.edu/wg/qos/ (links to all WG design teams from here)

QBone Home: http://qbone.internet2.edu/

Abilene Premium Service Home: http://www.internet2.edu/abilene/qos/

IBM Internet2 Day (July 26th, 2001) 20