group round robin: improving the fairness and complexity of packet scheduling

October 27, 2005 ANCS 2005

Group Round Robin: Improving the Fairness and

Complexity of Packet Scheduling

Bogdan Caprita, Jason Nieh, and Wong Chun Chan

Columbia University

October 27, 2005 ANCS 2005

GRR

Hierarchical framework for proportional sharing of network links with good fairness and overhead

October 27, 2005 ANCS 2005

GRR

Hierarchical framework for proportional sharing of network links with good fairness and overhead

• Proportional sharing

• Related work• GRR scheduling• Results

October 27, 2005 ANCS 2005

Scheduling

Time-multiplexing a shared resource among a set of competing entities

October 27, 2005 ANCS 2005

Scheduling

Time-multiplexing a shared link among a set of competing flows

October 27, 2005 ANCS 2005

Scheduling

Time-multiplexing a shared link among a set of competing clients

October 27, 2005 ANCS 2005

Scheduling

Time-multiplexing a shared link among a set of competing clients

Goals:metric of interest (throughput, delay, fairness …)speed, simplicity

October 27, 2005 ANCS 2005

Proportional Scheduling

Allocate to each client resource share in proportion to its weight

Goals:

fairness: is the allocation really ~ weight?

speed, simplicity

October 27, 2005 ANCS 2005

Proportional Scheduling

Allocate to each client resource time (WC) in proportion to its weight (φC)

WC ≈ (φC/ΦT)WT

-

-

-

-

October 27, 2005 ANCS 2005

Proportional Scheduling

Allocate to each client resource time (WC) in proportion to its weight (φC)

WC ≈ (φC/ΦT)WT

-

-

-

-

Client work

October 27, 2005 ANCS 2005

Proportional Scheduling

Allocate to each client resource time (WC) in proportion to its weight (φC)

WC ≈ (φC/ΦT)WT

-

-

-

-

Client weight

October 27, 2005 ANCS 2005

Proportional Scheduling

Allocate to each client resource time (WC) in proportion to its weight (φC)

WC ≈ (φC/ΦT)WT

-

-

-

-

Total weight

October 27, 2005 ANCS 2005

Proportional Scheduling

Allocate to each client resource time (WC) in proportion to its weight (φC)

WC ≈ (φC/ΦT)WT

-

-

-

-

Total work

October 27, 2005 ANCS 2005

Proportional Scheduling

Allocate to each client resource time (WC) in proportion to its weight (φC)

WC ≈ (φC/ΦT)WT

Goals:fairness: is the allocation really ~ weight?

Ideal: GPS (Parekh, Gallager)speed, simplicity

October 27, 2005 ANCS 2005

Proportional Scheduling

Issues:

skewed weights –

unfair

many clients –

slow, complicated

October 27, 2005 ANCS 2005

Proportional Scheduling

Formalize:

fairness service error EC=|WC – (φC/ΦT)WT|

Issues:

skewed weights –

unfair

many clients –

slow, complicated

October 27, 2005 ANCS 2005

Proportional Scheduling

Issues:

skewed weights –

unfair

many clients –

slow, complicated

Formalize:

fairness service error EC=|WC – (φC/ΦT)WT|

complexity

running time / decision

October 27, 2005 ANCS 2005

Related Work

Round-Robin

Virtual Time

October 27, 2005 ANCS 2005

Round Robin

• Weighted Round Robin

• Deficit Round Robin

• Virtual Time Round Robin

• Smoothed Round Robin

• Stratified Round Robin

Fast ( usually O(1) ), but unfair ( Ω(N) )

October 27, 2005 ANCS 2005

Virtual Time

• Weighted Fair Queuing• Start-Time Fair Queuing• Self-Clocked Fair Queuing • Worst-case Fair Weighted Fair Queuing• Earliest Eligible Virtual Deadline First• Hierarchical Stride

Slow (usually Ω(log N) ), but fair ( O(N), O(1) )

October 27, 2005 ANCS 2005

GRR

Hierarchical framework for proportional sharing of network links with good fairness and overhead

• Proportional sharing

• Related work• GRR scheduling• Results

October 27, 2005 ANCS 2005

GRR Scheduling

• Grouping Strategy• Scheduling groups using intergroup

scheduler• Scheduling clients within groups using

intragroup scheduler

October 27, 2005 ANCS 2005

GRR Scheduling

Our approach: Exponential grouping by weight

G = { C : 2σ ≤ φC < 2σ+1 } σG = “group order”

1 2,3 4..7 8..15 16..31 32..63 …

σ=0 σ=1 σ=2 σ=3 σ=4 σ=5

number of groups: g = constant

October 27, 2005 ANCS 2005

GRR Scheduling

Our approach: Exponential grouping by weight

G = { C : 2σ ≤ φC < 2σ+1 } σG = “group order”

why? Decrease:

number of schedulable entities (few groups)

weight skew(clients within groups have similar weights)

October 27, 2005 ANCS 2005

Example:

C1 C2 C3 C4 C5

φ1=12 φ2=3 φ3=3 φ4=2 φ5=4

Grouping Strategy

October 27, 2005 ANCS 2005

G1 G2 G3

C1 C4 C5

Example:

C1 C2 C3 C4 C5

φ1=12 φ2=3 φ3=3 φ4=2 φ5=4

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Grouping Strategy

October 27, 2005 ANCS 2005

GRR Scheduling

Data structures

Group G: unordered list of clients

group weight: Φ = ∑CG φC

Group list:

Φ1 , Φ2 , Φ3 ,… Φg g ≤ 32

October 27, 2005 ANCS 2005

GRR Scheduling

Two-level scheduler

1. Intergroup scheduling

2. Intragroup scheduling

October 27, 2005 ANCS 2005

GRR Scheduling

Two-level scheduler

1. Intergroup scheduling

idea: use an existing scheduler to schedule groups

October 27, 2005 ANCS 2005

GRR SchedulingExamples (1 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

Φ 12 8 4

G1 G2 G3

October 27, 2005 ANCS 2005

GRR SchedulingExamples (1 - Intergroup):

Φ 12 8 4

G1 G2 G3

October 27, 2005 ANCS 2005

GRR SchedulingExamples (1 - Intergroup):

G1 G2 G3

12 8 4

October 27, 2005 ANCS 2005

GRR SchedulingExamples (1 - Intergroup):

G1 G2 G3

12 8 4

October 27, 2005 ANCS 2005

GRR SchedulingExamples (1 - Intergroup):

G1 G2 G3

12 8 4

Scheduling algorithm

October 27, 2005 ANCS 2005

GRR SchedulingExamples (1 - Intergroup):

G1 G2 G3

12 8 4

Scheduling algorithmWFQ HS

SCFQ WF2Q

SFQ SRR

October 27, 2005 ANCS 2005

GRR SchedulingExamples (1 - Intergroup):

G1 G2 G3

12 8 4

Scheduling algorithm

G1 G2 G3G1 G1 G2 G1 G2 G3G1 G1 G2

Schedule:

October 27, 2005 ANCS 2005

GRR Scheduling

Two-level scheduler

2. Intragroup scheduling

idea: round robin with deficit tracking

“current client” index i < |G| :

if ( Di < 1 ) {

i = i + 1; Di = Di + φC/2σ;

}

Di = Di – 1;

October 27, 2005 ANCS 2005

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Intragroup Scheduling

October 27, 2005 ANCS 2005

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 00 00 0

Intragroup Scheduling

October 27, 2005 ANCS 2005

C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 00 00 0

October 27, 2005 ANCS 2005

C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 00.5 00 0

C1

October 27, 2005 ANCS 2005

C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0.50.5 00 0

C1 C2

October 27, 2005 ANCS 2005

C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0.5 1 00 0

C1 C2 C1

October 27, 2005 ANCS 2005

C3C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C2

Deficit: 0.5 0 00 0

C1 C2 C1 C1

October 27, 2005 ANCS 2005

C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0.5 0 00.5 0

C1 C2 C1 C1 C3

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0.5 0 00.5 0

C1 C2 C1 C1 C3 C5

October 27, 2005 ANCS 2005

C4C4C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0.5 0.5 00.5 0

C1 C2 C1 C1 C3 C5 C1

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0.5 0 0

C1 C2 C1 C1 C3 C5 C1 C4

0.5 0.5

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 1 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1

0.5 0.5

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0.5 0.5

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

1 0.5

C2

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

1 0.5

C2 C5

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0.5 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

1 0.5

C2 C5

C1

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0.5 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0 0.5

C2 C5

C1 C2

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 1 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0 0.5

C2 C5

C1 C2 C1

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0 0.5

C2 C5

C1 C2 C1 C1

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0 1

C2 C5

C1 C2 C1 C1 C3

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0 1

C2 C5

C1 C2 C1 C1 C3 C5

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0.5 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0 1

C2 C5

C1 C2 C1 C1 C3 C5 C1

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0.5 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0 0

C2 C5

C1 C2 C1 C1 C3 C5 C1 C3

October 27, 2005 ANCS 2005

C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 1 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0 0

C2 C5

C1 C2 C1 C1 C3 C5 C1 C3 C1

October 27, 2005 ANCS 2005

C4C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0 0

C2 C5

C1 C2 C1 C1 C3 C5 C1 C3 C1 C1

October 27, 2005 ANCS 2005

C4C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0 0

C2 C5

C1 C2 C1 C1 C3 C5 C1 C3 C1 C1 C4

October 27, 2005 ANCS 2005

C4C1 C5C3C2C1

Intragroup Scheduling

G1 G2 G3

C1 C4 C5

Examples (2 - Intergroup):

G1={C1} Φ1=12 ; G2={C2, C3, C4} Φ1=8 ; G3={C5} Φ3=4

φ1=12 ; φ2=3, φ3=3, φ4=2 ; φ5=4

C3C2

Deficit: 0 0 0

C1 C2 C1 C1 C3 C5 C1 C4 C1 C1

0 0

C2 C5

C1 C2 C1 C1 C3 C5 C1 C3 C1 C1 C4 C5

October 27, 2005 ANCS 2005

GRR

Hierarchical framework for proportional sharing of network links with good fairness and overhead

• Proportional sharing

• Related work• GRR scheduling• Results

October 27, 2005 ANCS 2005

GRR Properties

Theoretical

Worst-case bounds

Experimental

Simulations

October 27, 2005 ANCS 2005

GRR Theoretical Properties

Error: EC < EG + O(1)

Time complexity: TCG + O(1)

October 27, 2005 ANCS 2005

Intergroup Schedulers

Scheduler Error – Θ() Time – Θ()

WFQ N Log(N)

SCFQ N Log(N)

SFQ N Log(N)

HS Log(N) Log(N)

WF2Q 1 Log(N)

SRR N*Log(φC) 1

October 27, 2005 ANCS 2005

Simulation Studies

0.1

1

10

100

1000

32 64 128 256 512 1024 2048 4096 8192

Number of Clients

Neg

ativ

e S

ervi

ce E

rro

rGRR WFQ SRR WF2Q

October 27, 2005 ANCS 2005

Simulation Studies

0.1

1

10

100

1000

32 64 128 256 512 1024 2048 4096 8192

Number of Clients

Po

sit

ive

Se

rvic

e E

rro

rGRR WFQ SRR WF2Q

October 27, 2005 ANCS 2005

GRR C & FW

Hierarchical framework for proportional sharing of network links with good fairness and overhead

• Scheduler

Integration• Analysis• Implementation

October 27, 2005 ANCS 2005

Thank You!

http://ncl.cs.columbia.edu/