Applications of Game Theory Part II(b)

John C.S. LuiComputer Science & Eng. Dept

The Chinese University of Hong Kong

On the interaction between Overlay Routing and Underlay

Routing

Y. Liu, H. Zhang, W. Gong, D. Towsley

INFOCOM 2005

First course

Motivation: Interactions Between Application Level Network and Physical

Network physical network control

– routing, congestion control,…

Control Control

Control

Result?– interactions?– controllers mismatch?

add an overlay

and another……

Outline

Problem Formulation Simulation Study Game-theoretic Study Conclusions

Routing on physical network level Inter-domain: BGP, etc. Intra-domain: OSPF, MPLS, etc.

– determine routes for all source-destination traffic demand pairs

– minimize network-wide delay, cost, etc.

Routing in Underlay Network

traffic demand pair: A->Btraffic demand pair: A->Ctraffic demand pair: C->B

A

C

E

B

D

An overlay network choose routes at application level to minimize its own delay or cost

Routing in Overlay Network

A

C

E

B

D

A

C

B

Overlay demand: A->B

logical routes: A->C->B and A->B

demand pair: A->C

Overlay gains

advantagebetter path: delay, loss, throughput, etc

is selfishpotential performance degradation to other non-overlay traffic

demand pair: C->Bdemand pair: A->B

Considering overlay and underlay

together ?

How do they interact with each other? How does selfish behavior of overlay routing

– affect overall network performance?– affect non-overlay traffic performance?– affect its own performance?

Interactions Between Overlay Routing and Underlay

Routing

Overlay Routing OptimizerTo minimize overlay cost

Underlay Routing OptimizerTo minimize overall network cost

flow allocation on logical links: “X”traffic demand for underlay

flow allocation on physical routes: “Y”

non-overlaytraffic demand

overlaytraffic demand

Iterative Dynamic Process equilibrium: existence? uniqueness? dynamic process: convergence? oscillations? performance of overlay and underlay traffic?

Approach by authors

Focusing interaction in a single AS Considering two routing models for overlay

and one routing model for underlay Simulating the interaction dynamic

process Studying this process in a Game-theoretic

framework

Routing Models Overlay routing model

– Selfish source routing Individual user controls infinitesimal amount of

traffic, to minimize its own delay

– Optimal overlay routing A central entity minimizes the total delay of all

overlay traffic demands

Underlay routing model Optimal underlay routing

A central entity minimizes the total delay of all network traffic, e.g. Traffic Engineering MPLS

Node without overlay

LinkNode with overlay

47

9

10

8

6

2

13

5

1

12

14

11

3

Simulation Study: Optimal Overlay and Optimal

Underlay14 node tier-1 POP network (Medina et.al. 2002)bimodal normal model of traffic demand3 overlay nodes

Simulation Study ( case 1: 8% overlay traffic) Optimal Overlay and Optimal Underlay

after underlay takes turnafter overlay takes turn

Iterative process Underlay takes turn at step 1, 3, 5, …

Overlay takes turn at step 2, 4, 6, … ,...5,4,3,2,1

%100)1Delay(

)1Delay(-)Delay(

=k

k

iteration

average delay of overlay traffic

perc

enta

ge

%

overlay performance improvement

iteration

average delay of all traffic

perc

enta

ge

%

underlay performance degradation

iteration

perc

enta

ge

%

overlay performance degradation

average delay of overlay traffic

after underlay takes turnafter overlay takes turn

Iterative process Underlay takes turn at step 1, 3, 5, … Overlay takes turn at step 2, 4, 6, … ,...5,4,3,2,1

%100)1Delay(

)1Delay(-)Delay(

=k

k

iteration

perc

enta

ge

%

underlay performance degradation

average delay of all traffic

Simulation Study (case 2: 10% overlay traffic) Optimal Overlay and Optimal Underlay

Game-theoretic Study Two-player non-zero sum game

overlay Underlay

X: strategy of “overlay” traffic allocation on logical linksY: strategy of “underlay” traffic allocation on physical links

: Cost of “overlay”

: Cost of “underlay”

: Constraints of “overlay”

: Constraints of “underlay”

Game-theoretic Study

• Best-reply dynamics

• Nash Equilibrium

Optimal Underlay Routing v.s. Optimal Overlay Routing

Overlay – One central entity calculates routes for all

overlay demands, given current underlay routing

– Assumption: it knows underlay topology and background traffic

A B

C

X(k)

Denote overlay’s routing decision with a single variable X(k): overlay’s flow on path ACB after round k

1-X(k)

There exists unique Nash equilibrium x*, x* globally stable: x(k) x*, from any initial x(1)

Best-reply Dynamics

iteration k iteration k

Overlay Routing Evolution Overlay Delay Evolution

x(k)

x*x(k)<x(k+1)<x*

dela

y

Underlay’s turn

Overlay’s turn

When x(1)=0, overlay performance improves

Round k Round k

x(k)

x*

Underlay’s turn

Overlay’s turn

BAD INTE

RACTION!

x(k)>x(k+1)>x*

x(k)<x(k+1)<x*

Overlay Delay Evolution

dela

y

Overlay Routing Evolution

Best-reply Dynamics

There exists unique Nash equilibrium x*, x* globally stable: x(k) x*, from any initial x(1)

When x(1)=0.5, overlay performance degrades

Conclusions & Open Issues

Selfish overlay routing can degrade performance of network as a whole

Interactions between blind optimizations at two levels may lead to lose-lose situation

Future work:– larger topology: analysis/experimentation– overlay routing and inter-domain routing– interactions between multiple overlays (****)– implications on design overlay routing– regulation between overlay and underlay (****)

On the Interaction of Multiple Overlay Routings

Performance 2005

Joe W.J. Jiang, D.M. Chiu, John C.S. Lui

Second course

Questions• These overlays tend to fully utilize available

resource.• So, is there any anarchy?• How do overlay networks co-exist with each

other?• What is the implication of interactions?• How to regulate selfish overlay networks via

mechanism design?• Can ISPs take advantage of this?

Outline

• Motivation• Mathematical Modeling• Overlay Routing Game• Implications of Interaction• Pricing• Conclusion

Motivation

• Overlays provide a feasibility for users to control their own routing.

• Routing, possible multi-path, becomes an optimization problem.

• Interaction occurs (due to same underlay)• Interaction between one overlay and

underlay traffic engineering, Zhang et al, Infocom’05.

• Interaction between co-existing overlays ?

Adaptive routing controls on multiple layers (overlays, underlay TE --traffic engineering) over one common physical network

Simultaneous feedback controls over one system

Stability ?

Performance ?

Performance Characteristics

• Objective: minimize end-to-end delay (e.g., RON)• Delay of a physical link e:

• Performance Characteristics (Underlay)

de(le)

le – aggregate traffic traversing link e

Average delay ( f : flow)

( )( )∑∈

⋅tsf

fdf,

Performance Characteristics

• Objective: minimize end-to-end delay• Delay of a physical link e:

• Performance Characteristics (Underlay)

de(le)

le – aggregate traffic traversing link e

Average delay (multipath routing)

( )( )( )

∑ ∑∈ ∈

⋅tsf fPr

rr fdf,

Performance Characteristics

• Objective: minimize end-to-end delay• Delay of a physical link e:

• Performance Characteristics (Underlay)

de(le)

le – aggregate traffic traversing link e

Average delay (multipath routing)

( )∑ ⋅e

eee ldl

System Objectives

• Network Operators– Min average delay in the whole underlay network

• Overlay Users– Min average delay experienced by the overlay

( )∑ ⋅e

eee ldl min

( )∑ ⋅e

eeoverlaye ldl min

How do Overlays Interact?

• Overlapping physical links.• Performance dependent on each

other.• Selfish routing optimization.• Overlays are transparent to each

other.• Lack of information exchange

between overlays.

Contribution

• What is the form of interaction?• Is there routing instability (oscillation),

or there is an equilibrium ?• Is the routing equilibrium efficient?• What is the price of anarchy?• Fairness issues• Mechanism design: can we lead the

selfish behaviors to an efficient equilibrium?

Mathematical Modeling

• Overlay routing: An optimization problem Decision variable: routing policy

( )Tfssss yyyy ),()2,()1,()( ,,, L=

( )),(),(2

),(1

),( ,,, fsr

fsfsfs yyyy L=

s: overlay f: flow r: path

Mathematical Modeling

• Overlay routing: An optimization problem Objective: average weighted delay (matrix form)

⎥⎦

⎤⎢⎣

⎡⎟⎠

⎞⎜⎝

⎛= ∑

i

iiTsTss yADAydelay )()()()()(

( )

( )∑ ∑

∈

⋅=f fRr

rfs

rs delayydelay ,)(

Routing Matrix Delay Function (vector form)

Overlay Routing Optimization

[ ]

0 ,

, s.t.

Minimize

,,,,; OVERLAY

)(

),(

)()()()()(

)()()(

≥≤

=∈∀

⎥⎦

⎤⎢⎣

⎡⎟⎠

⎞⎜⎝

⎛=

∑

∑

∈

−

s

fRr

fsrs

i

iiTsTss

sss

yCAy

xyFf

yADAydelay

yxCHAy

f

Convex programming

Demand constraint

(fixed transmission demand)

Capacity Constraint

Non-negative Flow Constraint

Algorithmic Solution

• Unique optimizer – Convex programming– feasible region: convex– delay function: continuous, non-decreasing,

strictly convex

• Solution– Apply any convex programming techniques.– Marginal cost network flow (probabilistic routing

ICNP’04).– This is solved in an independent, and

distributed fashion by each overlay.

But will independent optimization leads to system instability (route flop)?

Overlay Routing Game

• Nash Routing Game– Player -- N

all overlays

– Strategy -- s

feasible routing policy: feasible region of OVERLAY(s)

– Preference relation -- ≥s

low delay: player’s utility function is -delay(s)

Strategic Game: Goverlay<N, (s), (≥s)>

Illustration of Interaction

Routing Underlay

Overlay 1

Overlay 2

Overlay n

…

Routing decision on logical paths in overlay 1

Routing decision on logical paths in overlay 2

Routing decision on logical paths in overlay n

Aggregate overlay traffic

… Underlay (non-overlay) traffic

Aggregate traffic on physical links

Overlay probing

Delay of logical paths in overlay 1

Delay of logical paths in overlay 2

Delay of logical paths in overlay n

∑

Existence of Nash Equilibrium

• Definition – Nash equilibrium point (NE)

A feasible strategy profile y=(y(1),…, y(s),…, y(n))T

is a Nash equilibrium in the overlay routing game if for every overlay s∈N,

delay(s)(y(1),…y(s),…y(n)) ≤ delay(s)(y(1),…y’(s),…y(n))

for any other feasible strategy profile y’(s) .

Existence of Nash Equilibrium

• Theorem

In the overlay routing game, there exists a Nash equilibrium if the delay function

delay(s)(y(s) ; y(-s))

is continuous, non-decreasing and convex.

Good News: NO ROUTE FLOP !!!

Fluid Simulation

Six overlays

One flow per overlay

Congested network

Asynchronous routing update

Overlay performance

Transient period

Quick convergence

Overlay routing decisions

The Price of Anarchy

0 ∞1 2 3 4 5 6 7 8 ... ...

Global Performance (average delay for all flows)

• GOR: Global Optimal Routing

• NOR: Nash equilibrium for Overlay Routing Game

• NSR: Nash equilibrium for Selfish Routing

GOR

NOR

NSR

Efficiency Loss ?

Selfish Routing

• (User) selfish routing: a single packet’s selfishness• Every single packet chooses to route via a shortest

(delay) path.• A flow is at Nash equilibrium if no packet can

improve its delay by changing its route.

)~

()(

~

if ],,0[ ,

21

1 2

1

21

fdfd

otherwisef

PPiff

PPiff

ffPP

PP

P

P

P

PP

≤

⎪⎩

⎪⎨

⎧=+=−

=∈≠ δδ

δ

Selfish Routing

• Also a Nash equilibrium of a mixed strategic game– Player: flow { f }

– Strategy: p Pf

– Preference: low delay

• System Optimization Problem

( )( )

0 , , , s.t.

min

,,,; SELFISH

0

≥=≤=

∑∫yAyLCAyxHy

dttd

xCHAy

e

l

e

e

Performance Comparison

Overlay One

Overlay Two

Average Delay

Centralized Global Optimal Routing

2.50 2.38 2.44

NE of Overlay Optimal Routing

2.46 2.53 2.50

NE of Selfish Routing 2.63 2.75 2.69

Inspiration

• Is the equilibrium point efficient (at least Pareto optimal) ?

• Fairness issues of resource competition between overlays.

Example Network

3

21

5

src1 src2

sink1 sink2

4

6

Overlay 2

Overlay 1

3

21

5

src1 src2

sink1 sink2

4

6

1 unit

1 unit

y11-y1 y2 1-y2

Sub-Optimality

3

21

5

src1 src2

sink1 sink2

4

6

Overlay 2

Overlay 1 physical link delay function

de(le)

1-5 1+l

3-4 l

2-6 2.5+l

Routing

(y1, y2)

Average Delay(overlay1, overlay2

)

NE (0.5, 1.0) (1.5, 1.5)

Pareto Curve

(0.4, 0.9) (1.4, 1.4)

y1 y2

Non Pareto-

optimal !

Fairness Paradox

3

21

5

src1 src2

sink1 sink2

4

6

Overlay 2

Overlay 1physical link delay function

de(le)

1-5 a+l

3-4 bl

2-6 c+ly1 y2

a, b, c, are non-negative parameters

Everything is symmetric except two private links – a & c

Fairness Paradox

3

21

5

src1 src2

sink1 sink2

4

6

Overlay 2

Overlay 1physical link delay function

de(le)

1-5 a+l

3-4 bl

2-6 c+ly1 y2

⎪⎪⎪

⎩

⎪⎪⎪

⎨

⎧

⎟⎠

⎞⎜⎝

⎛+⎟⎠

⎞⎜⎝

⎛=

=

−⎟⎠

⎞⎜⎝

⎛+⎟⎠

⎞⎜⎝

⎛=

−

−

1

1

2

3

2

3

1

,12

3

22

3

αα

αα

α

α

c

b

a

a < c

Overlay 1 has a better “private” link !

Fairness Paradox

3

21

5

src1 src2

sink1 sink2

4

6

Overlay 2

Overlay 1

y1 y2

⎪⎪

⎩

⎪⎪

⎨

⎧

⎟⎠

⎞⎜⎝

⎛=

−⎟⎠

⎞⎜⎝

⎛+⎟⎠

⎞⎜⎝

⎛=−

2

3

4

1

2

3

42

3

2

1

1

delay

delay

a < c delay1 < delay2

∞=→<∞→2

1 delay

delayca

Unbounded

Unfairness

War of Resource Competition

1 unit 1 unit

y1

1-y1

y2

1-y2

poil(y1+y2)

pusa(1-y1) pchn(1-y2)pusa< pchn

USA

China

Min Costusa(y1 ; y2) =

y1poil(y1+y2)+(1-y1)pusa(1-y1)

War of Resource Competition

1 unit 1 unit

y1

1-y1

y2

1-y2

poil(y1+y2)

pusa(1-y1) pchn(1-y2)pusa< pchn

USA

China

Min Costchn(y2 ; y1) =

y2poil(y1+y2)+(1-y2)pchn(1-y2)

War of Resource Competition

1 unit 1 unit

poil(y1+y2)

pusa(1-y1) pchn(1-y2)

pusa< pchn

Costusa > Costchn

USA

China

Pricing (opportunity for ISP)

Inefficient Nash equilibrium

Desired equilibrium

Mechanism Design

Performance degradation (sub-optimal)

Fairness paradox

Global optimality

Improve fairness

payment new

Nash equilibrium

Pricing I – Improve Delay

• Objective: to achieve global optimality

• NE of overlay routing game

• Global optimal

( )∑ −+⋅e

se

see

se lldl )()()( min

le(s) : traffic of

overlay s

le(-s) : traffic other

than overlay s

( )∑ ⋅e

eee ldl min

)()( se

see lll −+=

Pricing I – Improve Delay

• Objective: to achieve global optimality

• New NE of overlay routing game

• Global optimal

( )[ ]

∑

∑⋅+

+⋅ −

e

se

se

e

se

see

se

pl

lldl

)()(

)()()(

min

( )∑ ⋅e

eee ldl min

)()( se

see lll −+=

)()( min ss paymentdelay +

Heterogeneous pricing

Pricing I – Improve Delay

• New NE of overlay routing game

• Global optimal

( )[ ]∑ ++⋅ −

e

se

se

see

se plldl )()()()( min ( )∑ ⋅

eeee ldl min

KKT condition:

( )( )( )( ) ( )

r

reee

se

seee

re

se

se

se

se

u

ldlpld

plldl

=

⋅++=

++⋅

∑

∑

∈

∈

−

')()(

')()()()(

KKT condition:

( )( )

( ) ( )

r

reeeeee

reeee

u

ldlld

ldl

=

⋅+=

⋅

∑

∑

∈

∈

'

'

pe(s)=le

(-s) de’(le)

Pricing II – improve fairness

• Cause of unfairness:– Over-utilize good common resources– Unfair resource (bandwidth) allocation

• Pricing Scheme

ISP

maximize profit

Improve performance &

Reduce cost

Overlay

price

p

routing decision

Incentive Resource Allocation

• For overlays:

( )( ) ∑∑ ⋅+⋅⋅=

+⋅=−

e

see

eee

ses

sss

sss

lpldl

paymentdelayyy)()(

)()()()()(

;Cost min

α

α

s : sensitivity factor

new Nash equilibrium {le}

Revenue Distribution

• For ISPs (links):

( ) ( )eeeee lclplP −⋅= max

( ) ( )( )ee

ee

eeee

e ld

ld

llcp

dl

dP '' 1

0 +==→=

: profit of link e

: revenue

: operating cost --

ee lp ⋅( )ee lP

( )ee lc ( )( )eee ldl ⋅log

Effectiveness of Pricing

Conclusion• Study the interaction between multiple co-

existing overlays.• Non-cooperative Nash routing game.• Prove the existence of NEP.• Show the anomalies and implications of the

NEP.• Present two distributed pricing schemes to

address the anomalies.

Third Course

Interaction of ISPs: Distributed Resource Allocation and Revenue Maximization

Sam C.M. Lee, Joe W.J. Jiang, D.M Chiu, John C.S. Lui

View of ISPsTier-1 ISP

Tier-2 ISP

Local ISP

Peering link

Tier-2 ISP

Local ISP

Peering link

ISP

Peer

ISP link

ISP

Peer

Peer

PeerPeer

Peer

Peer kPeer j

Tier-2 ISP(ISP)

Peer i 1. performance of the link2. charge of the link

Issues to consider:

Optimization problem of peers

Optimization problem of peers

Happiness obtained from sending traffic to peers

Delay cost in ISP link

Payment to ISP

Delay costs in peering links

Payments to peers

Constraints of peers

1.

2.

3.

4.

Solution to the peers

• Objective function is strictly concave in every transmission rate

• The optimal transmission rates and maximum utility are unique and can be found by the Lagrangian method.

Problems for an ISP

• Maximization of revenue– How to determine the optimal value of

unit price

• Resource distribution– How to determine the capacity for the

peers

Information exchange framework

ISP peer

Bandwidthallocation Bid

Compute resourcedistribution

Computeoptimalrates

Next period

ISP 1: Resource distribution

peer1

Bid = 50MBps

? ? ?

ISP

peer2 peer3

Bid = 100MBps Bid = 150MBps

Bandwidth = 600MBps

Proportional share algorithm

peer1 peer2 peer3

Bid = 50MBps Bid = 100MBps Bid = 150MBps

ISP Bandwidth = 600MBps

100MBps 200MBps 300MBps

Equal share algorithm

peer1 peer2 peer3

Bid = 50MBps Bid = 100MBps Bid = 150MBps

ISP Bandwidth = 600MBps

150MBps 200MBps 250MBps

Simulations

• When the happiness coefficients of peers are low

PSA ESA

Simulation

• When the happiness coefficients of peers are high

PSA ESA

ISP 2: Maximization of Revenue

Unit price

Demand by peer i

Determine the optimal price

Total revenue from the peers

Solution: Maximization of revenue

•Estimate the aggregate traffic ( ) from all

peers in term of the price (P)

Conclusions

• Utility maximization of a peer• Resource distribution of ISP• Revenue maximization of ISP

Fourth Course

On the Access Pricing Issues of Wireless Mesh Networks

ICDCS 2006

Ray K. Lam Dah-Ming Chiu John C.S. Lui

WMN Paints a Bright Future

• Wireless mesh network (WMN)– Wireless nodes– Multi-hop routing– Form a wireless “mesh”

• More access to the Internet– More people, rich or poor– More ubiquitous,

anywhere, anytime– More opportunities to

everyone

Internet Internet

The Critical Thing—Cooperation

Multi-hop routingRelay packets for each otherMy concerns: bandwidth, CPU time, security…

Community network with symmetric trafficHelp each other => mutual benefit

Access network with asymmetric trafficGeographically good VS poorWhy help the poor?Incentive system needed—pricing

When AP Meets a Client

• Simple analysis by Musacchio and Walrand [1]

• A game with 2 players– Access point (AP)

provides Internet access

– Client buys the service

– One deal per time slot

AP Clientp1

accept

p2

accept

p3

reject

slot 1

slot 2

slot 3

serviceduration

AP Client

p

A Beautiful Equilibrium• AP and client each maximizes

her gain– AP: guess the “right” price– Client: compare the price p with

service utility U• Web browsing utility function

• A beautiful equilibrium– AP has the same optimal price in every time slot– Client connects if her per-slot service utility is

greater than slot price (U > p)

• Encourages flat-rate pricing

To a Multi-hop Scenario

• Adding a relaying node, or “reseller” (RS)

• RS tries to mark up AP’s price to the “right” level

• AP takes note of RS’s action when setting her price

• Equilibrium is still flat-rate pricing

• Multi-hop => multiple RSs

AP Clientc1

acceptslot 1

slot 2

slot 3

serviceduration

p1

accept

c2

accept

p2

accept

c3

reject

p3

reject

RS

AP ClientRS

c

p

Drawbacks of the Simple Model

• Assuming unlimited network capacity– 2-player game represents

whole system– Treat every incoming client the

same– Unlimited admission =>

unlimited capacity• Assuming a tree-like network

– 2-hop / multi-hop linear network extension

– Does not consider multiple paths

– Pricing competition may occur

A tree-like network

A graph-like network

AP ClientRS

What If Capacity Limited?

• Cannot admit unlimited clients– Client demands bandwidth guarantee– AP admission control– AP’s system capacity: m

• 2-player game not enough– AP deals with each client differently– Client arrival model: Poisson process

• Like an M/M/m/m/M queuing system

Flat-rate Pricing Fails…

• Failure scenario– AP is full; m clients admitted– An admitted client a is paying $5/slot– A new client b arrives– AP asks b for $6/slot– If b accepts

• AP raises price for a to $6/slot, OR• Simply kicks a out

• Flat-rate pricing is not optimal!

Everybody Loves Flat Rate

• Unrealistic for variable rate• More practical—fixed-rate, non-

interrupted service– AP charges a client a fixed rate p over

time– AP cannot disconnect a client unilaterally

• AP can still charge different clients at different “fixed” rates– How to set the optimal rate on different

occasions?

Best Strategy in New Service Model

• AP sets price based on remaining capacity– Raises price when becoming full– State price: at state k, AP charges next “to-be-

admitted” client at fixed rate pk

– Policy of AP characterized by price vector

• Client’s best strategy– Connect AP if service utility per unit time > price

per unit time (U > p)

System Dynamics• State transition

– Adding a factor P(U > pk) to regular arrival rate in M/M/m/m/M model

• Reward structure– Simplification: immediate expected

profit when a client connects

0 1 2 mm-1…

M P(U > p0) M-1) P(U > p1) M-m+1) P(U > pm-1)

2 m

State transition diagram

System Dynamics• State transition

– Adding a factor P(U > pk) to regular arrival rate in M/M/m/m/M model

• Reward structure– Simplification: immediate expected

profit when a client connects

Reward Structure

0 1 2 mm-1…

p0/

0 0 0

p1/ pm-1/

Finding Optimal Price Vector

• Classical optimization– Solution for queuing system gives limiting state

probability for each state k, k

– Gain of AP is a function of price vector

– Complicated to optimize with classical techniques

• Policy-iteration method in Markovian decision theory– Reduces computational complexity by iterative

algorithm– Guarantees convergence to the best policy

Numerical Results• Capacity m=5,

population M=10, departure rate =1

• Vary arrival rate from 0.2 to 10

• Utility U uniformly distributed on [0,10]

• U normally distributed with mean 5, s.d. 1.67

• Price increases number of clients in AP and with

Limited Capacity in Multi-hop Case

• Simplification– Traffic merges at AP– AP is the bottleneck– Only AP controls

admission

• AP’s policy specified by a price matrix– At each state, different prices for requests

from different distances

– pki: price at state k for a client i-hop away

AP ClientRS

Internet

bandwidthbottleneck

System Dynamics• Removing finite population

– Complicates state information– Different arrival rates for clients at different distances

0 1 …

1P(U > m1(p0,1))

m

2P(U > m2(p0,2))

nP(U > mn(p0,n))

…

m-1 m

1P(U > m1(pm-1,1))

2P(U > m2(pm-1,2))

nP(U > mn(pm-1,n))

…

Client 1-hop away arrives

Client 2-hop away arrives

Client n-hop away arrives

State transition diagram

System Dynamics

0 1 …

0 0

p0,n/

…

m-1 m

…

Client 1-hop away arrives

Client 2-hop away arrives

Client n-hop away arrives

p0,2/

p0,1/

pm-1,n/

pm-1,2/

pm-1,1/

Reward Structure

• Removing finite population– Complicates state information– Different arrival rates for clients at different distances

Conclusion• Contributions

– Show that fixed-rate pricing fails with limited capacity

– Generalize unlimited capacity model into limited capacity model

– Devise optimal pricing for fixed-rate, non-interrupted service with Markovian decision theory

• References[1] J. Musacchio and J. Walrand. WiFi access point

pricing as a dynamic game. IEEE/ACM Trans. Networking. to appear in.