11

A Self-Adapting Web A Self-Adapting Web Server Architecture Server Architecture

Towards Higher Performance and Better Towards Higher Performance and Better UtilizationUtilization

2

ObjectivesObjectives

► IntroductionIntroduction► I/O in Original Web Server ArchitectureI/O in Original Web Server Architecture►Related WorkRelated Work► The Need for AdaptabilityThe Need for Adaptability► The Self-Adapting ArchitectureThe Self-Adapting Architecture►ConclusionConclusion

3

IntroductionIntroduction

►Outline limitations in original Outline limitations in original architecturesarchitectures Limited scalabilityLimited scalability Inefficient utilizationInefficient utilization ComplexityComplexity

► Survey existing Web server architecturesSurvey existing Web server architectures► Introduce a self-adapting architectureIntroduce a self-adapting architecture► Present an evaluation for the proposed Present an evaluation for the proposed

modelmodel

4

I/O in Original Web ServersI/O in Original Web Servers

► Synchronous I/OSynchronous I/O Blocking to the calling process/threadBlocking to the calling process/thread Connection-OrientedConnection-Oriented For multi-process/multi-threaded serversFor multi-process/multi-threaded servers

►MP: Switching to an active process is naturalMP: Switching to an active process is natural►MT: Support for kernel threads is neededMT: Support for kernel threads is needed

(+) Simple design(+) Simple design (-) Limited scalability(-) Limited scalability (-) Inefficient server utilization(-) Inefficient server utilization

5

I/O in Original Web ServersI/O in Original Web Servers

Synchronous I/O causes idle-time for the calling process.

6

I/O in Original Web ServersI/O in Original Web Servers

►Asynchronous I/OAsynchronous I/O Allows servers to interleave computations Allows servers to interleave computations

with slow disk I/Owith slow disk I/O Event-OrientedEvent-Oriented Works well for single-process serversWorks well for single-process servers (+) Highly efficient, scales well(+) Highly efficient, scales well (-) Complicated design(-) Complicated design (-) Notification overhead(-) Notification overhead

7

I/O in Original Web ServerI/O in Original Web Server

Asynchronous I/O allows the calling process to interleave processing with I/O.

8

Related Work1. Modify other aspects

Multi-accept (2001, 2004) Increase cache locality (2001)

2. Propose hybrid models AMPED (1999) SEDA (2001) HYBRID (2005) SYMPED (2007)

3. Replace available libraries Capriccio (2001) Lazy Asynchronous I/O (2004)

9

The Need for AdaptabilityThe Need for Adaptability

► In non-adaptive modelsIn non-adaptive models Synch. I/O under overload conditionsSynch. I/O under overload conditions Synch. I/O over a busy disk Synch. I/O over a busy disk Asynch. I/O under low load conditionsAsynch. I/O under low load conditions

►Adapt to presence of queued requestsAdapt to presence of queued requests Allows more practical scenarioAllows more practical scenario Pushes the limits of scalabilityPushes the limits of scalability Allows better utilization when this becomes Allows better utilization when this becomes

an issuean issue

10

The Self-Adapting Web ServerThe Self-Adapting Web Server

►Multi-threaded, uses thread poolsMulti-threaded, uses thread pools►Algorithm:Algorithm:

Accept(request);Accept(request); Add work to queue;Add work to queue; Worker thread takes work from queue;Worker thread takes work from queue; If there are more queued workIf there are more queued work

► Perform Asynchronous I/O;Perform Asynchronous I/O; ElseElse

► Perform Synchronous I/O;Perform Synchronous I/O; Worker thread is done;Worker thread is done; Polling thread continuously watches for I/O requests;Polling thread continuously watches for I/O requests;

11

The Self-Adapting Web ServerThe Self-Adapting Web Server

Layout of the self-adapting Web server architectureLayout of the self-adapting Web server architecture

12

The Proposed Model

Either worker or helper threads can perform I/O, whichever is less expensive

13

The Self-Adapting Web ServerThe Self-Adapting Web Server

14

The Proposed Model►Issues with previous models :

Still room for improvement Need for adaptability Efficiency using available system calls Expensive scenarios

• Proposed solution A worker thread blocks for I/O under easy conditions

Helper threads to be used otherwise

15

Conclusion►Adaptability :

Opens up a new door for ideas Provides flexible operation Better utilizes resources Provides a more logical scenario Can hopefully improve performance

►Limitations in available system calls can be relaxed