IWQoS’2006, New Haven, CT, June 19 – 21, 2006

Improving Performance of Internet Services Through

Reward-Driven Request Prioritization

Alexander Totok and Vijay Karamcheti

Computer Science Department

New York University

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

Web Server Overload Conditions

Consequences increased request response times some requests are dropped successful session throughput suffers dramatically client dissatisfaction → reduced revenues

Current solutions work with static client identity session-based admission control (SBAC) service level agreements (SLA) service membership per-client history-based approach

– looks at a client’s previous visits to the web site

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

Maximizing Profit Brought By Internet Services

Service profit (reward) maximization shopping web site: number of items sold

Idea: assign higher execution priority to the requests, whose sessions are likely to bring more reward how to predict a session’s reward?

Our solution: Reward-Driven Request Prioritization (RDRP) predicts a session's activities by comparing requests seen in it

with aggregated client behavior uses Bayesian inference analysis to dynamically compute

request priority in real time contrasts with the per-client history-based approach

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

Service Usage Profiles (Patterns)

Session structure: first-order Markov chain corresponds to a typical service usage profile (pattern)

Shopping scenario for TPC-W application (web store selling books) “Mostly Buyers” profile – more buying activity “Mostly Browsers” profile – more browsing activity

Home Search[empty cart]

Item[empty cart]

Add To CartSearchItem

Cart Register

Buy RequestBuy Confirm

Exit[browser]

Exit[buyer]

0.75

0.10.050.2

1

0.05

0.60.25

0.6

0.2

0.4

0.6

0.40.2

0.30.3

0.4

0.4

1 1

1

0.2

Mostly Buyers

Home Search[empty cart]

Item[empty cart]

Add To CartSearchItem

Cart Register

Buy RequestBuy Confirm

Exit[browser]

Exit[buyer]

0.6

0.330.050.35

1

0.05

0.020.6

0.4

0.30.4

0.5

0.50.2

0.050.35

0.4

0.6

1 1

1

0.3

Mostly Browsers

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

What Information Does RDRP Use?

User load structure: {Profilei}; {pi} – percentage of sessions belonging to Profilek

on-line request profiling and clustering analysis [Menasce’99]

Request reward rewardi: per request type specified by the service provider web shopping scenario: reward(addToCart)=1

Relative request execution cost for prediction of future server resource consumption costi: per request type – average request processing time fine-grained profiling of request execution

Only request reward is specified by the service provider

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

cost_expected

Step 2

{costi}

{rewardi}

Profile K:pi,j

User load structure

p1 Profile 3Profile 2

Profile 1

p2p3

...

Step 1

How Does The Algorithm Work?

priority =reward_attained + reward_expected

cost_incurred +

request +its

session’srequest history

Bayesiananalysis

Step4:

Pr that the session

belongs to Profile KStep 3

reward_expectedcost_expected

Uses Markov properties of service

usage profiles

reward_expected and

cost_expected

if the session belongs to Profile K

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

Prototype Implementation in J2EE

Request priority used to allocate threads and DB connections

JBossJetty Web Server JBoss JCA

service

requestrequest

profiling info RDRP

Socket Listener Database Connection Manager

priority queues

Thread pool

Sched. . .

priority queues

DB conn. pool

Sched. . .

getThread(priority, timeout)

getDBConn(priority, timeout)

request profiling info

request

execution

Request Profiling Service

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

Evaluation

Shopping scenario for TPC-W user load with two usage patterns: “Mostly Buyers”/“Mostly Browsers” new sessions: bursty arrival process (B-model [Wang’02])

Techniques compared default – FIFO prioritization session-based admission control (SBAC) per-client history-based approach

– success depends on how well prediction of a session’s behavior works

– model different correlation between sessions’ rewards and assigned priorities:

c = 0 (coin flip)

c = 0.25

c = 0.5 (good oracle)

c = 0.75 (very good oracle)

c = 1 (perfect oracle)

Reward-Driven Request Prioritization (RDRP)

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

0%

25%

50%

75%

100%

125%

150%

175%

200%

225%

135% 170% 200% 250%

Load (% of server capacity)

Rel

ativ

e re

war

d

FIFO SBAC coin flip (c=0)oracle (c=0.25) good oracle (c=0.50) very good oracle (c=0.75)perfect oracle (c=1) RDRP

Overload (135%): Reward

similar

The bigger the better

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

0 1 2 3 4Number of items bought in the session (reward)

Re

qu

es

t re

sp

on

se

tim

e (

ms

)

SBAC coin flip (c=0) oracle (c=0.25)good oracle (c=0.50) very good oracle (c=0.75) perfect oracle (c=1)RDRP

Overload (135%): Response Times

The smaller the better

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

Underload (80%): Response Times

0

500

1000

1500

2000

2500

3000

3500

4000

0 1 2 3 4

Number of items bought in the session (reward)

Req

ues

t re

spo

nse

tim

e (m

s)

FIFO coin flip (c=0) oracle (c=0.25)good oracle (c=0.50) very good oracle (c=0.75) perfect oracle (c=1)RDRP

The smaller the better

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

Discussion

Main distinguishing features of RDRP tries to predict future session’s reward is oriented towards session completion works in an abstract application-generic manner

May also take into account differences in user think times helps to distinguish between different service usage profiles in

the Bayesian inference analysis

What if clients do not show stable behavioral patterns?

IWQoS’2006, New Haven, CT, June 19 – 21, 2006

file

Thank You!