a case study of web server benchmarking using parallel wan emulation carey williamson rob simmonds...

A Case Study ofA Case Study ofWeb Server Web Server

Benchmarking UsingBenchmarking UsingParallel WAN Emulation Parallel WAN Emulation

Carey WilliamsonRob Simmonds Martin Arlitt

University of Calgary

Web Benchmarking with IP-TNE 2

Network Emulation A hybrid performance evaluation

methodology that combines aspects of implementation with simulation modeling

A network emulator is a network simulator with an interface that allows client applications to interact with it in real-time

“A simulator that talks back” (IP packets) Why? Provides a reliable, repeatable test

environment for distributed applications Internet games, video conferencing, Web, ...

SimulationSimulation

Real WorldReal World


IP-TNE The Internet Protocol Traffic and Internet Protocol Traffic and

Network EmulatorNetwork Emulator (IP-TNE) is a network emulator based on IP-TN (packet-level IP network simulator)

Enables interaction between IP based clients via an IP-TN simulated network (in real time!) Distributed applications can interact

with IP-TNE without modification


IP-TNE Overview


IP-TNE Overview (cont’d) CCTKit with real-time extensions

provides an environment for fast network emulation (PDES)

IP-TNE provides routing methods suitable for shared environments and dedicated test environments

Now has HTTP and TCP client models that can be used for Web server benchmarking


So What? Flexible routing model support High-performance packet reading and

writing via raw sockets (1 Gbps) Can model an arbitrary IP internetwork Detailed IP protocol models

IPv4, ICMP, ping, traceroute, pchar, MTU, ... Supports parallel execution on

shared memory multiprocessors “Blazingly fast!” - CLW, 2002


Example: Web Benchmarking

Web Server

Client 1

Client 2

Client 3

Client C

...


WAN Emulation (1 of 3)

Web Server

Client 1

Client 2

Client 3

Client C

...

“Centralized” Approach



Web Server

Client 1

Client 2

Client 3

Client C

...

“Shim” Approach(NISTnet, DummyNet, WASP)



Web Server

Client 1

Client 2

Client 3

Client C

...

Our IP-TNE Approach


Objectives of Case Study Evaluate new approach to WAN

emulation, and demonstrate feasibility

Confirm prior results by Nahum et al. on effects of WAN conditions on Web server performance

How fast can Apache Web server go? How fast can IP-TNE go?


Experimental Setup

IP-TNE on Compaq ES-40 (4 CPU) Apache (1.3.23) on another ES-40 Gigabit Ethernet (1 Gbps) in

between OS is Compaq Tru64 (v5.1A)

ANML for defining network model e.g., simple regular WAN topology


Web Workload Model Static content only Closed-loop workload generator Fixed-size Web objects

Small (1 KB) Large (64 KB)

Variable-size Web objects Median 3 KB Mean 9 KB Pareto heavy tail (alpha = 1.2) Zipf-like document popularity profile


Performance Metrics

Two primary metrics HTTP transaction rate (trans/sec) Network throughput (Mbps)

Several secondary metrics Response time Connection failure rate Packet loss rate ...


Results (Fig. 4a) For 1 KB transfers with HTTP/1.0:

Single client: 170 transactions/sec Transaction rate scales up with number of

clients up to about H = 32 Transaction rate flattens, then drops sharply

as num clients is increased more (closed loop)

Peak rate achieved: 3800 trans/sec Peak throughput approximately 40 Mbps Transaction rate is (strongly) inversely

related to the client round trip time (RTT)


Results (Fig. 4b) For 64 KB transfers with HTTP/1.0:


clients up to about H = 32 Transaction rate flattens, then drops slightly

as num clients is increased more Peak rate achieved: 220 trans/sec Peak throughput approximately 115 Mbps Transaction rate is (weakly) inversely

related to the client round trip time (RTT)


Results (Fig. 4c) For variable-size transfers with HTTP/1.0:


clients up to about H = 32 Transaction rate flattens, then drops

as num clients is increased more Peak rate achieved: 1300 trans/sec Peak throughput approximately 90 Mbps Transaction rate is inversely related to the

client round trip time (RTT) Behaviour is in between 1 KB and 64 KB results


Results (Fig. 5a)

Concurrent connections with HTTP/1.0: Single client: 600 transactions/sec Qualitatively similar results to before,

except that fewer clients are needed to drive the server to full load

Conceptually concurrent connections are no different than adding more clients


Results (Fig. 5b) Persistent connections with

HTTP/1.1: Single client: 300 transactions/sec Qualitatively similar results to before,

except that transaction rate is about 70% higher than for HTTP/1.0 (since multiple HTTP req’s per TCP conn)

Peak transaction rate 6500 trans/sec Much less dependency on RTT effects


Results (Fig. 5c)

Pipelined persistent connections with HTTP/1.1: Single client: 800 transactions/sec Qualitatively similar results to before,

except that transaction rate is about 100% higher than for HTTP/1.0

Peak transaction rate 7600 trans/sec Much less dependency on RTT effects


Results (Fig. 6a)

Effect of WAN RTT delays: Increasing the per-link propagation

delay increases the client RTT delay, which in turn reduces the transaction rate and throughput (as expected)

As RTT increases, more and more clients are needed in order to drive the Web server to full load

Similar to [Nahum et al. 2001]


Results (Fig. 6b) Effect of bandwidth asymmetry:

For asymmetric access technologies such as ADSL (Asymmetric Digital Subscriber Line), the upstream link from the client to the server can sometimes be the bottleneck for TCP, even though it is primarily carrying ACKs only

Depends on normalized bandwidth ratio Greater asymmetry, worse performance


Results (Fig. 6c)

Effect of WAN packet losses: Decreasing the router queue size at

the bottleneck link increases the packet loss ratio (as expected)

As the level of packet loss increases, the HTTP transaction rate and the network throughput decrease

Similar results to [Nahum et al. 2001]


Summary and Conclusions The IP-TNE is a useful tool for Web

server benchmarking Demonstrates feasibility of WAN

emulation using a single computer Confirms prior results by Nahum et

al. studying the effects of WAN conditions on Web server performance

Demonstrates performance advantages of HTTP/1.1


Future Work? Increase traffic to your Web site!!! Guarantee 20% increase in traffic! 1000’s of new clients per month!!!!!

Send $$$ to [email protected]


Future Work with IP-TNE Validation of IP-TNE (and IP-TN) Benchmarking IP-TNE vs IP-TNE Benchmarking Web caching appliances Evaluating SRPT scheduling in WAN setting Connection/packet-level scheduling algorithms Evaluating CATNIP approach to TCP/IP Evaluating portable (wireless) Web servers Workload sensitivities (Zipf, Pareto, corr,

mods) Experiments with dynamic content (CGI, etc) Asymmetric networks, Ensemble-TCP Parallel TCP connections: friend or foe? Evaluating effect of TCP SACK in WAN

a case study of web server benchmarking using parallel wan emulation carey williamson rob simmonds...

Documents

iptne approach slide

iptne overview slide

web server benchmarking

network emulator iptne

ip packets

experimental setup iptne

web server performance

apache web server