Simulation Evaluationof Hybrid SRPT Policies

Mingwei Gong and Carey WilliamsonDepartment of Computer ScienceUniversity of CalgaryApril 19, 2004

WORMS 2004 2

IntroductionWeb: large-scale, client-server systemWWW: World Wide Wait!User-perceived Web response time involves:

Transmission time, propagation delay in network

Queueing delays at busy routers in the Internet

Delays caused by TCP protocol effects (e.g., handshake, slow start, packet loss, retransmits)

Queueing delays at the Web server itself, which may be servicing 100’s or 1000’s of concurrent requests

Our focus in this work: Web request scheduling

WORMS 2004 3

Example Scheduling Policies

FCFS: First Come First Servetypical policy for single shared resource (“unfair”)e.g., drive-thru restaurant; playoff tickets

PS: Processor Sharingtime-sharing a resource amongst J jobseach job gets 1/J of the resources (equal, “fair”)e.g., CPU; VM; multi-tasking; Apache Web server

SRPT: Shortest Remaining Processing Timepre-emptive version of Shortest Job First (SJF)give full resources to job that will complete quickeste.g., ??? (express lanes in grocery store)(almost)

WORMS 2004 4

Research Methodology

Trace-driven simulationInput workload is empirical/synthetic trace

Web server simulatorEmpirical trace (1 million requests, World Cup 1998)Synthetic traces (WebTraff)

Probe-based sampling methodologyBased on PASTA: Poisson Arrivals See Time AveragesAny scheduling policy, any arrival process, any service time distribution.

WORMS 2004 5

Simulation Assumptions

User requests are for static Web contentServer knows response size in advance

Network bandwidth is the bottleneckAll clients are in the same LAN environment

Ignores variations in network bandwidth and propagation delay

Fluid flow approximation: service time = response size

Ignores packetization issues

Ignores TCP protocol effects

Ignores network effects

(These are consistent with SRPT literature)

WORMS 2004 6

Performance Metrics

Slowdown:The slowdown of a job is its observed response time divided by the ideal response time if it were the only job in the systemLower is better

We consider mean slowdown as well as the variance of slowdown (complete distribution)

WORMS 2004 7

Empirical Web Server Workload

1998 WorldCup: Internet Traffic Archive: http://ita.ee.lbl.gov/

Item Value

Trace Duration 861 sec

Total Requests 1,000,000

Unique Documents 5,549

Total Transferred Bytes 3.3 GB

Smallest Transfer Size (bytes) 4

Largest Transfer Size (bytes) 2,891,887

Median Transfer Size (bytes) 889

Mean Transfer Size (bytes) 3,498

Standard Deviation (bytes) 18,815

WORMS 2004 8

Probe-based Sampling Algorithm

The algorithm is based on PASTA (Poisson Arrivals See Time Average) principle.

S

S

S

Slowdown (1 sample)

Repeat

N

times

WORMS 2004 9

Probe-based Sampling Algorithm

For scheduling policy S =(PS, SRPT, FCFS, LRPT, …) do

For load level U = (0.50, 0.80, 0.95) do

For probe job size J = (1B, 1KB, 10KB, 1MB...) do

For trial I = (1,2,3… N) do

Insert probe job at randomly chosen point;

Simulate Web server scheduling policy;

Compute and record slowdown value observed;

end of I;

Plot marginal distribution of slowdown results;

end of J;

end of U;

end of S;

WORMS 2004 10Job Size

Slo

wdo

wn

PS

SRPT

0 8

Slowdown Profile Plot“crossover region” (mystery hump)

“asymptoticconvergence”

x y

1

8

11-p

WORMS 2004 11

Notation Details

Number of jobs in the system: JNumber of threads for a single server: KNumber of servers in the system: M

Probe jobs: 1KB, 10KB, 100KB, 1MB...Number of probes: 3000All simulation results are for 95% load

WORMS 2004 12

Single Server Scenario (M = 1)

PS: Processor SharingSRPT: Shortest Remaining Processing TimeFSP: Fair Sojourn Protocol

FSP computes the times at which jobs would complete under PS and then orders the jobs in terms of earliest PS completion times. FSP then devotes full service to the uncompleted job with the earliest PS completion time.“FSP response time dominates PS” (i.e., is never worse)

E. Friedman and S. Henderson: “Fairness and Efficiency in Web Server Protocols”, Proc. ACM SIGMETRICS 2003.

WORMS 2004 13

Mean Slowdown (M = 1)

WORMS 2004 14

Variance of Slowdown (M = 1)

WORMS 2004 15

A Hybrid SRPT/PS Policyfor a Single Server (M = 1)

Threshold-based policy, with threshold T T-SRPT

Determining whether the system is "busy" or not depends on number of jobs (J) in the system.

If J <= TThen use PSElse use SRPT

Special cases: T = 0 is SRPT, T = is PS

8

WORMS 2004 16

Mean Slowdown for T-SRPT

WORMS 2004 17

Variance of Slowdown for T-SRPT

WORMS 2004 18

A Generalized SRPT Policy for aMulti-threaded Single Server

K-SRPTMulti-threaded version of SRPT that allows up to K jobs (the K smallest RPT ones) to be in service concurrently (like PS), though with the same fixed aggregate service rate. Additional jobs (if any) in the system wait in the queue. Also preemptive, like SRPT.

Let s = min (J, K)If J <= KThen J jobs each receive 1/sElse K jobs each receive 1/s (while J-K wait)

Special cases: K = 1 is SRPT, K = is PS

8

WORMS 2004 19

Mean Slowdown for K-SRPT

WORMS 2004 20

Variance of Slowdown for K-SRPT

WORMS 2004 21

Multi-Server Scenario

M-PS:Let s = max(M, J)Each job receives a service rate of 1/s

M-SRPT:Let s = MIf J <= MThen J jobs each receive 1/s (M-J idle servers)Else M jobs each receive 1/s (while J-M wait)

M-FSP:Let s = MIf J <= MThen J jobs (under PS) each receive 1/sElse M jobs (under PS) each receive 1/s

WORMS 2004 22

Mean Slowdown for M-SRPT

WORMS 2004 23

Variance of Slowdown for M-SRPT

WORMS 2004 24

Mean Slowdown for M-PS

WORMS 2004 25

Variance of Slowdown for M-PS

WORMS 2004 26

Mean Slowdown for M-FSP

WORMS 2004 27

Variance of Slowdown for M-FSP

WORMS 2004 28

Summary

Slowdown profile plots for several policiesFor the largest jobs, FSP better than SRPT and PSFor small jobs, FSP is sometimes worse than SRPT

Multi-threaded server resultsT-SRPT and K-SRPT provide a smooth transition between SRPT and PS, implying smoother tradeoff in fairness between small jobs and large jobs

Multi-server resultsWith more servers, mean slowdown worsens, but variance of slowdown often improvesFSP does not response time dominate PS for M > 1

WORMS 2004 29

Thank You!

Questions?

For more informationEmail: {gongm,carey}@cpsc.ucalgary.ca