Modern Queueing Theory,and Internet Address Allocation

Clean Slate WorkshopMarch 21, 2007

High PerformanceSwitching and Routi ngTe le co m Cen t er Wo r ks h op: Se pt 4, 1 997 . Balaji PrabhakarBalaji Prabhakar

21st Century Queueing Theory,and Internet Address Allocation

Clean Slate WorkshopMarch 21, 2007

High PerformanceSwitching and Routi ngTe le co m Cen t er Wo r ks h op: Se pt 4, 1 997 . Balaji PrabhakarBalaji Prabhakar

3

Overview

• I will talk about two projects

– Scalable queuing theory for Internet flows• Current theoretical models are limited: (i) They are packet-centric, (ii) do not model the “feedback loop” of

TCP dynamically, and (iii) use strong assumptions which allow a detailed analysis; but this has also made them fragile (assumption-dependent).

– IPv6 address allocation• Addresses currently allocated without taking into account a user’s

potential growth rate. This leads to address fragmentation and complicates the address lookup problem.

– I will describe new approaches to both problems.

4

Overview of flow models project

• Themes– Flows in a packet-switched Internet

• View a flow as a minimum data unit• Algorithms for enabling the cheap recognition of flows• Processing flows, not packets, at high speed

– Flow-level models• Current theoretical models have two drawbacks: (i) They are packet-centric (ii) they do not model the “feedback loop” of

TCP dynamically• We study models which address the above drawbacks• There are some interesting and serious theoretical challenges

– Scaleable Network (Queueing) Theory• Classical queueing theory makes strong assumptions which allow

detailed analysis• However, it is too fragile (assumption-dependent) and scales poorly• We propose a new approach

5

Overview of Project

• People– PIs: Amin Saberi and Balaji Prabhakar– Students: Mohsen Bayati, Abdul Kabbani, Yi Lu– Visitors/collaborators: Ashvin Lakshmikantha (UIUC), Maury

Bramson (Minnesotta), Devavrat Shah (MIT), John Tsitsiklis (MIT)– Industry collaborators (flow detection, processing)

• Flavio Bonomi and Rong Pan, Cisco Systems

6

What is a flow-level model?

• First, what is a packet-level model?– Minimum data unit: packet; i.e. decisions made per packet– Classical (20th century) queueing theory is cast in this set up

• M/M/1 queue, Jackson Networks, Kelly Networks, etc– Used extensively from the late 80s onwards– To model packet radio networks and switches

• E.g. Tassiulas and Ephremides; McKeown, Anantharam and Walrand– Very successful for determining maximum throughput algorithms,

get delay/backlog bounds– More recently used by Gupta and Kumar for determining how

throughput scales with network size in ad hoc wireless networks

7

Switch scheduling

• Crossbar constraints– each input can connect to at most one output– each output can connect to at most one input

The switch

1

2

1 2 3

3

The queueing model

8

Scheduling algorithms

Not stable

Stable (Tassiulas-Ephremides 92,

McKeown et. al. 96, Dai-Prabhakar 00)

Not stable (McKeown-Ananthram-Walrand 96)

1934 21

18

7

1

PracticalMaximal Matchings

Max Wt Matching

19

18

Max Size Matching

19

1

7

9

Limitations of packet-level models

• Open loop: Fails to incorporate end-to-end behavior– Scheduling and drop decisions on current packets affect future

arrivals

• Sees all packets as equal– Is unable to model short/long flows, short/long RTTs, etc

• Fails to capture the flow-level bandwidth sharing that results from TCP/AQM interactions

10

Flow-level models• This is more realistic: flows, of differing sizes, arrive at random times and

are transferred through the network by the congestion management algorithms and transport protocols– Flow completion (transfer) time is the main quantity of interest: what is its

mean? variance? how does it depend of flow sizes? on network topology, on round trip time, etc?

11

A flow-level model of a 2 link network

L1 L2

• This type of model introduced in the Internet context by– Bonald and Massoulie

• Active research area…– Bramson, Gromoll,

Kelly, Shah, Williams, etc

12

Bandwidth sharing: Link capacity = 1

• Max-min fair sharing– Route 1 flows: 1/5 – Route 2 flow: 2/5– Route 3 flows: 1/5

• Proportional fair sharing– Route 1 flows: 1/7 – Route 2 flow: 4/7– Route 3 flows: 2/7

13

More formally…

• The queueing model– Flows arrive according to Poisson processes on routes– Flow sizes are arbitrarily distributed (heavy-tailed)

• Questions– Throughput: How many flows can be processed per unit time? – Delays: What is the flow delay due to a particular scheme?– What effect does network topology have?– What happens if we give more bandwidth to short flows?– What is the effect of scaling network size and/or link capacities?

• The general picture– Routes on left, links on right– Adjacency graph shows captures

network topology– Bandwidth sharing due to

combination of TCP/AQM

14

Need new theory

• Classical QT: As in the books of Kleinrock and Kelly– Consider network of queues, and assume

• Poisson arrivals, exponential services, random routing– These assumptions give networks with good properties

• Reversibility• Decomposibility (independence or “product-form”)

– Hence obtain• Throughputs, delays, backlogs, etc

• Almost all classical results are some variation of the above• Our situation: none of the above holds!

– Main reason: a job (flow) requires service from multiple servers (links) simultaneously; this leads to loss of independence

– This type of network studied briefly in earlier literature as “Whittle Networks”

15

The struggle for independence• Our approach: don’t consider small-sized networks, let network sizes go to

infinity– Then, amazingly, you get all the good properties, esp independence– The key feature needed: correlation decay– In these arguments, an “infinite-sized network” is one which has about

100 or so nodes; i.e., infinity is not too far away

• Have applied this approach to the so-called “supermarket model” used to study load balancing

Allocator

16

Further work

• The main idea is: consider the thermodynamic limit of large networks

• The problem simplifies and becomes meaningful because correlations are local and die in the limit

• Now, what type of a large network should we consider?1. Internet: Power law graphs2. Ad hoc networks: Geometric random graphs3. Other random graphs, e.g. Bernoulli random graph

• In this context we can answer the questions we previously asked about how bandwidth sharing algorithms, topology, flow size, RTT, etc, affect performance

• There is a huge simulation component to ensure fidelity of models, understand accuracy with scaling, etc

IPv6 Address Allocation

18

• Address allocation sets the foundation of the network hierarchy– Therefore, it is important to have the right address allocation scheme

• The goal of this work is to answer the question: – How should addresses be allocated for future networks with a clean

slate design?

• Collaborators: Mei Wang, Ashish Goel– This is essentially the work of Mei Wang– She has been working with (at) Cisco and with CNNIC to apply her

growth-based allocation scheme, neat software being developed, etc– See her poster this afternoon for more details

Address Allocation

19

IANA

RIR

ISP/LIR

EU(ISP) EU

Local Internet Registries(ISP’s)

End Users

Internet Assigned Number Authority

Regional Internet Registries (ARIN, RIPE, APNIC, AfriNIC, LACNIC)

How are Addresses Allocated?

20

Definition – Address CollisionDefinition – Address Collision

A BA

A

21

Definitions – Address Fragmentation

A BA B A B A B

Not Fragmented Fragmented

22

Distribution of Customers Requests

0

10

20

30

40

50

1 3 5 7 9 11 13 15 17 19 21

Customer ID

Number of Requests

APNIC data

Asia Pacific (APNIC) IPv4 Allocation Data Statistics

• Number of customers is 21• Address space is 225

• Number of requests is 197

23

Address space

1 43 2a)

b)1 43 25

Current Practice

• Addresses allocated as prefixes; two methods – Sequential: Allocate in any (first) open range of suitable size– Bisection: same as above, but more systematic; a candidate for

IPv6 allocation

24

1 i n

Li Si

Ai, Ri

An+1, Rn+1

n+1

[ ]{ }niRStRLt niii ,...,1 ,),( ),,(minmax 1 =+

GAP:Growth-based Address Partitioning

Take growth-rates into account, maximize time to collision.

“A Growth-based Address Allocation Scheme for IPv6”, Mei Wang, Networking, 2005.

25

• Theoretical proofs: – Sequential is optimum given static sizes– GAP is online optimum

• Simulations• Experiments with real data

– Use IPv4 data, what if we had allocated addresses using GAP?– Growth-rates derived empirically, no additional input was needed. – Larger enhancements can be accomplished if customers provide

growth-rate estimations.

Performance Results

26

Comparison: Asia Pacific (APNIC) Allocation Data

•

27

Comparison: China (CNNIC) Allocation Data

28

Software for Address Allocation

• Being jointly developed with Cisco and CNNIC• Provides visualization of address allocation• Simulate real allocations taking dynamic requests • A platform for analysis of different algorithms

29

• Increasing interest from parties in various layers: – Regional registries, country registries, ISPs, large corporations

• Economic incentive model for accurate estimation of growth-rates

• Non-prefix based address allocation (clean slate)– Prefix assignment equivalent to assigning subtrees– The simplest non-prefix assignment would assign a pair of

subtrees, or equivalently, an edge on the hypercube

• Provider Independent (PI) address allocation

Current Status & Future Work