on selfish routing in internet-like evironments

System & Network Reading Group

On Selfish Routing In Internet-Like Evironments

Lili Qiu (Microsoft Research)Yang Richard Yang (Yale University)

Yin Zhang (AT&T Research)Scott Shenker (ICSI)


Motivation

• Practical front– A recent trend: end hosts choose routes

• Source routing (e.g., Nimrod)• Overlay routing (e.g., Detour or RON)

– Characteristics of routing by end hosts• Improve over today’s IP routing (e.g., delay, loss rate)• Selfish by nature (i.e., optimize user-centric

performance without considering system-wide criteria)

• Theory front– Roughgarden et al. showed selfish routing can

result in serious performance degradation due to lack of cooperation


Example: Selfish Routing May Yield Sub-Optimal Performance

• Selfish routing– All traffic go through the lower link– Total latency = 1

• Optimal routing (i.e., minimize total latency)– Traffic split equally between the two links– Total latency = ¾

• The performance degradation can be unbounded for non-linear latency functions

src dest

L(x)=1

L(x)=x


Open Issues• How does selfish routing perform in Internet-like

environments?– Realistic network topologies– Realistic traffic demands– Realistic network delay functions

• How does selfish overlay routing perform?• How does selfish traffic co-exist with the

remaining traffic that uses traditional routing protocols?

• How does users’ selfish routing interact with underlying network control process (e.g., traffic engineering)


Outline

• Overview• Related work• Network model• Approach to compute traffic equilibria• Performance results

– Physical routing– Overlay routing– Multiple overlays– Interaction with traffic engineering

• Summary and future work


Overview

• Approach– Use a game-theoretic approach to

answer the above open issues– Focus on intra-domain scenarios

• Recent advances in topology mapping and traffic estimation

• Compare with theoretical results

– Focus on equilibrium behavior• Compare the performance of Nash

Equilibria with the global optima based on realistic topologies, traffic demands, latency functions


Some of Key Results• Formulate and evaluate selfish overlay routing• Unlike the theoretical worst cases, selfish routing

in Internet-like environments yields close to optimal latency– The above result is true for both source

routing and overlay routing– Selfish routing can achieve good performance

without hurting the traffic that is using default routing


Some of Key Results (Cont.)

• Mismatch between selfish routing and traffic engineering– Different objectives

• Selfish routing: minimize e2e delay• Traffic engineering: aim to balance load

– Selfish routing reduces latency at the cost of increased congestion

– The adaptive nature of selfish routing makes traffic demands less predictable and reduces the effectiveness of traffic engineering


Related Work

• Measurement results– Detour and RON projects showed that

default routing path is often sub-optimal in terms of latency, loss rate, and TCP throughput

– Possible causes of routing inefficiency• Routing hierarchy• Routing policy• Different routing objectives used by ISPs• Stability problem in routing protocols, such

as BGP• …


Related Work (Cont.)

• Theory results– Koutsoupias and Papadimitriou compared the

worst-case Nash equilibrium with a global optimal in a two-node network

– Price of anarchy (i.e., worst-case ratio between the total latency of a Nash equilibrium and that of the global optimal) can be unbounded [Roughgarden00]

– The performance degradation due to selfish routing can be compensated for by doubling the bandwidth on all links [Roughgarden01]


Network Model• Physical network

– Directed graph G=(V,E)– Latency of each edge is a function of its load

(e.g., M/M/1)

• Demands– demand(i,j): the amount of traffic from a source i

to a destination j

• Overlays– A set of overlay nodes and a set of demands– Consider mesh-like overlay topologies

• Users– Each user decides how its traffic should be routed

to optimize performance (e.g., min latency)


Network Model (Cont.)

• Route controller– Uses network-level routing

• OSPF/IS-IS: shortest-path with equal-weight splitting, with the following weight settings

– Hop-count– Random-weight– Optimized-compliant weight: minimize network

cost when assuming all traffic is compliant (i.e., following the routes determined by the network

• MPLS: general multi-commodity flow routing


Different Routing Schemes

• Physical routing– Source routing (i.e., selfish routing studied in

previous theoretical work)– Optimal routing

• Overlay routing– Overlay source routing (i.e., selfish routing

with routing constraints)– Overlay optimal routing

• Compliant routing (i.e., normal Internet routing)


Approach to Computing the Traffic Equilibria

• General approach– Simulation-based: too expensive to derive traffic equilibria– We use a game-theoretic approach to compute the traffic

equilibria directly

• Computing the equilibria of physical routing– linear-approximation algorithm, a variant of Frank-Wolfe

algorithm

• Computing the equilibria of overlay routing– Symmetric: Modified linear approximation algorithm – Asymmetric: Jacob’s relaxation algorithm

• Computing the equilibria of multiple overlays– Use the relaxation algorithm to guarantee the

convergence


Evaluation Methodology• Network topology

– A large tier-1 ISP topology– Rocketfuel topologies– Random power-law topologies

• Traffic demands– Real traffic demands from the ISP– Synthetic traffic demands

• Link latency functions– M/M/1, M/D/1, P/M/1, P/D/1, BPR

• Performance metrics– Average latency– Maximum link utilization– Network costs: piece-wise linear, increasing, convex

function [FRT02]


Outline

• Overview• Related work• Network model• Approach to compute traffic equilibrium• Performance Evaluation

– Source routing– Overlay routing– Multiple overlays– Interaction with traffic engineering

• Summary and future work


Selfish Source Routing

• Questions– Are Internet-like environments among the

worst-case?– What is the system-wide cost for selfish

source routing?

• Dimensions – Performance metrics: latency & network load– Effects of network topologies– Effects of network load– Effects of latency functions


Selfish Source Routing: Latency

• Effects of network topologies

0

5000

10000

15000

20000

25000

Abo

vene

t

AT

T

EB

ON

E

Exo

dus

Leve

l3

Spr

int

Tel

stra

Tis

cali

Ver

io

Pow

erD

2

Pow

erD

5

Pow

erD

10

Load scale factor=1

Av

era

ge

late

nc

y (

us

)

source optimal compliant

Selfish routing yields close to optimal latency, much better than compliant


Selfish Source Routing: Network Load

• Effects of network topologies

1

10

100

1000

10000

Abo

vene

t

AT

T

EB

ON

E

Exo

dus

Leve

l3

Spr

int

Tel

stra

Tis

cali

Ver

io

Pow

erD

2

Pow

erD

5

Pow

erD

10

Load scale factor=1N

etw

ork

co

st


020406080

100120140160

Abo

vene

t

AT

T

EB

ON

E

Exo

dus

Leve

l3

Spr

int

Tel

stra

Tis

cali

Ver

io

Pow

erD

2

Pow

erD

5

Pow

erD

10

Load scale factor=1

Ma

xim

um

lin

k u

tiliz

ati

on

(%

)


Selfish routing tends to overload links.


Summary: Selfish Source Routing

• The performance is qualitatively the same as we vary latency functions and network load

• Unlike the theoretical worst cases, selfish source routing yields close to optimal latency

• Selfish routing tends to overload links on the shortest paths


Outline

• Overview• Related work• Network model• Approach to compute traffic equilibrium• Performance results


• Conclusion and future work


Selfish Overlay Routing

• Questions– Does selfish overlay routing perform

well?– How does the coverage of overlay

network affect the performance?

• Dimensions– Effects of network topologies– Effects of amount of overlay coverage– Effects of how overlay nodes are

selected (e.g., random or edge nodes)


Difference between Source Routing and Overlay Routing

• Even if the overlay includes all network nodes, routing on an overlay is still different – Network-level routing can prevent any overlay

traffic from using a link by setting the corresponding entry in routing matrix to 0 (in OSPF this is achieved by assigning a large weight)

– Certain physical routes cannot be implemented by any overlay routing

• Routing flexibility is further reduced when only a fraction of nodes belong to an overlay


Selfish Overlay Routing (Full Overlay Coverage)



• Direct Link Shortest [DLS]– For any physically adjacent nodes A and B, all the

traffic from A to B is routed through the direct link AB without involving any other links. (e.g., hop-count-based OSPF)

• For an overlay that covers all network nodes and satisfies DLS– routing on the overlay = routing on the underlay

• Hop-count-based OSPF and optimized OSPF weights satisfy DLS they perform similarly as source routing

• Random OSPF weights violate DLS some links are pruned, and performance degrades


Selfish Overlay Routing (Partial Overlay Coverage)

• Overlay is formed from all edge nodes in ISPTopo

ISPTopo

7500

8000

8500

9000

9500

0 0.5 1 1.5 2 2.5

Load scale factor

Ave

rag

e la

ten

cy (

us)

all partial

ISPTopo

0

50

100

150

200

0 0.5 1 1.5 2 2.5

Load scale factorM

ax li

nk

uti

lizat

ion

(%

)

all partial


Summary: Selfish Overlay Routing

• For full overlay coverage– Overlay has full routing control when the

underlay satisfies DLS– The only way in which OSPF affects

overlay routing is by violating DLS, which reduces available network resources

– Overlay selfish routing reduces latency at the expense of higher network cost

• The effects of partial coverage are small in backbone topologies


Outline





Interactions among Competing Overlays

• Question– Can multiple overlays share network

resources fairly and effectively?• Dimensions

– Effects of network topologies– Effects of network-level routing

schemes– Effects of network load and traffic

distribution among overlays– Effects of the number of competing

overlays


Interactions among Competing Overlays (Cont.)

• Effects of network-level routing load scale factor = 1

05000

100001500020000250003000035000

com

/com

opt/

com

opt/

opt

src/

com

src/

opt

src/

src

com

/com

opt/

com

opt/

opt

src/

com

src/

opt

src/

src

com

/com

opt/

com

opt/

opt

src/

com

src/

opt

src/

src

opt-comp hop-count random

Ave

rag

e la

ten

cy (

us)

foreground background


Summary: Interactions among Competing Overlays

• With reasonable OSPF weights (e.g., hop-count)– Different routing schemes co-exist

without hurting each other

• With bad OSPF weights– Selfish overlay improves both for

themselves and for compliant traffic


Outline


– Source routing– Overlay routing– Multiple overlays– Interactions with traffic engineering



Selfish Routing vs. Traffic Engineering

• So far we assume network is dumb (i.e., static underlay routing)

• In practice, the network is smart due to traffic engineering (i.e., underlay routing adapts to varying traffic)

• Question– Will the system reach a state with both low

latency and low network cost, as selfish routing and traffic engineering each tries to optimize their objective by adapting to the other process?


Specification of Vertical Interactions

• Interactive process between two players– Traffic engineering

• Given traffic matrix Tt, where Tt(s,d) denotes traffic from source s to destination d in time slot t

• Compute routing matrix Rt for the underlay

– Selfish routing• Given routing matrix Rt for the underlay

• Produce new traffic matrix Tt


One Round during Vertical Interaction

T(t) = Traffic matrix when routing matrix is R(t-1)

R(t) = OptimizedRoutingMatrix(T(t))Traffic engineering installs R(t) to networkSelfish routing redistributes traffic to form

T(t+1)

Note that different from traditional games


Vertical Interaction with OSPF Optimizations

0.0E+00

5.0E+03

1.0E+04

1.5E+04

2.0E+04

2.5E+04

0 10 20 30 40 50

Round

Ave

rag

e la

ten

cy (

us)

overlay src: TE OSPF overlay src: hop-count compliant

020406080

100120140160180200

0 10 20 30 40 50

Round

Max

imu

m l

ink

uti

liza

tio

n (

%)


OSPF route optimization interacts poorly with selfish routing


Vertical Interaction with MPLS Optimization

0.0E+00

2.0E+03

4.0E+03

6.0E+03

8.0E+03

1.0E+04

1.2E+04

1.4E+04

0 10 20 30 40 50

Round

Av

era

ge

late

nc

y (

us

)

overlay src: TE MPLS overlay src: hop-count compliant

0

20

40

60

80

100

120

0 10 20 30 40 50

Round

Max

lin

k u

tiliz

atio

n (

%)


MPLS optimization interacts with selfish routing more effectively


Summary: Selfish Routing vs. Traffic Engineering

• OSPF route optimization interacts poorly with selfish routing

• MPLS interacts with selfish routing more effectively

• Despite the encouraging results from MPLS, several challenges exist– How to estimate traffic matrices accurately in

presence of adaptive selfish traffic?– Large optimization problems


Conclusion

• When the network-level routing is static, selfish routing achieves close to optimal latency

• The good latency achieved in selfish-routing comes at the cost of increased congestion

• When selfish routing and traffic engineering each tries to minimize its own cost by adapting to the other process, the resulted performance could be much worse


Future Work

• Study impacts of multi-AS nature of the Internet

• Study dynamics of selfish routing (i.e., how traffic equilibria are reached?)

• Improve the interactions between selfish routing and traffic engineering

• Study other selfish routing objectives (e.g., loss and throughput)



• Effects of network topologiesload scale factor = 1

0

5000

10000

15000

20000

25000co

m/c

omop

t/co

mop

t/op

tsr

c/co

msr

c/op

tsr

c/sr

cco

m/c

omop

t/co

mop

t/op

tsr

c/co

msr

c/op

tsr

c/sr

cco

m/c

omop

t/co

mop

t/op

tsr

c/co

msr

c/op

tsr

c/sr

cco

m/c

omop

t/co

mop

t/op

tsr

c/co

msr

c/op

tsr

c/sr

cco

m/c

omop

t/co

mop

t/op

tsr

c/co

msr

c/op

tsr

c/sr

c

Abovenet ATT Level3 Sprint PowerD10

Ave

rag

e la

ten

cy (

us)




• Effects of network load and traffic distribution among overlays

optimal compliant OSPF weights (LSF=1)

02000400060008000

1000012000

com

/com

opt/

com

opt/

opt

src/

com

src/

opt

src/

src

com

/com

opt/

com

opt/

opt

src/

com

src/

opt

src/

src

com

/com

opt/

com

opt/

opt

src/

com

src/

opt

src/

src

com

/com

opt/

com

opt/

opt

src/

com

src/

opt

src/

src

com

/com

opt/

com

opt/

opt

src/

com

src/

opt

src/

src

ff = 10% ff = 30% ff = 50% ff = 70% ff = 90%

Av

era

ge

late

nc

y (

us

)




• Effects of the number of overlays


Summary: Selfish Routing vs. Traffic Engineering

• OSPF route optimization interacts poorly with selfish routing

• MPLS interacts with selfish routing more effectively


Selfish Source Routing: Latency

• Effects of network load

ISPTopo

0

2000

4000

6000

8000

10000

0 0.5 1 1.5 2

Load scale factor

Ave

rag

e la

ten

cy

(us)



Selfish Source Routing: Latency (Cont.)

0100020003000400050006000700080009000

MM

1

MD

1

BP

R

PD

1

PM

1

MM

1

MD

1

BP

R

PD

1

PM

1

MM

1

Load scale factor=0.6 Load scale factor=1

Ave

rag

e la

ten

cy (

us)


•Effects of latency functions


Selfish Source Routing: network load

• Effects of network loadsISPTopo

0

20

40

60

80

100

120

140

160

0 0.5 1 1.5 2 2.5

Load scale factor

Max

lin

k u

tili

zati

on

(%

)


ISPTopo

0

1000

2000

3000

4000

5000

6000

7000

8000

0 0.5 1 1.5 2

Load scale factorN

etw

ork

co

st



Selfish Source Routing: network load (Cont.)

• Effects of latency functions

050

100150200250300350400450500

MM

1

MD

1

BP

R

PD

1

PM

1

MM

1

MD

1

BP

R

PD

1

PM

1


Max

imu

m l

ink

uti

liza

tio

n (

%)


1

10

100

1000

10000

100000

MM

1

MD

1

BP

R

PD

1

PM

1

MM

1

MD

1

BP

R

PD

1

PM

1


Net

wo

rk c

ost




load scale factor = 1

1.E+00

1.E+01

1.E+02

1.E+03

1.E+04

1.E+05

1.E+06

sour

ce

over

lay:

optW

t

over

lay:

hopt

Cnt

over

lay:

rand

Wt

sour

ceov

erla

y:op

tWt

over

lay:

hopt

Cnt

over

lay:

rand

Wt

sour

ce

over

lay:

optW

t

over

lay:

hopt

Cnt

over

lay:

rand

Wt

sour

ce

over

lay:

optW

t

over

lay:

hopt

Cnt

over

lay:

rand

Wt

sour

ce

over

lay:

optW

t

over

lay:

hopt

Cnt

over

lay:

rand

Wt

Abovenet ATT Level3 Sprint Pow erD10

Av

era

ge

la

ten

cy

(u

s)




(Cont.)load scale factor = 1

1

10

100

1000

sour

ceov

erla

y:op

tWt

over

lay:

hopt

Cnt

over

lay:

rand

Wt

sour

ceov

erla

y:op

tWt

over

lay:

hopt

Cnt

over

lay:

rand

Wt

sour

ceov

erla

y:op

tWt

over

lay:

hopt

Cnt

over

lay:

rand

Wt

sour

ceov

erla

y:op

tWt

over

lay:

hopt

Cnt

over

lay:

rand

Wt

sour

ceov

erla

y:op

tWt

over

lay:

hopt

Cnt

over

lay:

rand

Wt

Abovenet ATT Level3 Sprint PowerD10

Ma

xim

um

lin

k u

tiliz

ati

on

(%

)



Selfish Overlay Routing (Partial Overlay Coverage)

(Cont.)• Random selection of overlay nodes

0

50

100

150

200

250LS

F=

0.5

LSF

=1

LSF

=3

LSF

=0.

5

LSF

=1

LSF

=3

LSF

=0.

5

LSF

=1

LSF

=3

LSF

=0.

5

LSF

=1

LSF

=3

LSF

=0.

5

LSF

=1

LSF

=3

cover=20% cover=40% cover=60% cover=80% cover=100%

Max

imu

m l

ink

uti

liza

tio

n (

%)

all partial

on selfish routing in internet-like evironments

Documents

optimal routing

routessource routing

nimrodoverlay routing

roncharacteristics of

loadselfish routing

users selfish routing

overlay routingselfish

default routing path