A Flexible and Efficient API for a Customizable Proxy Cache

Vivek S. Pai, Alan L. Cox, Vijay S. Pai, and Willy Zwaenepoel

iMimic Networking, Inc.http://www.imimic.com

2

Motivation

More features moving into proxy caches– The ubiquitous layer 7 device

– Filtering, reporting, CDN support, transformation

– Lots of this being done one-off, ad hoc

– Can’t know everything at deployment

Some approaches for generalization– ICAP/OPES, proprietary mechanisms

– But design considerations shifting

Goal: new approach for modern environments

3

Contributions

Designed event-friendly proxy API

Implemented on iMimic DataReactor cache

Imposes negligible performance overhead

Demo modules– High performance

– Low interference

4

Outline

Background

API Design

API Functions

Implementation and Performance

Conclusions

5

Proxy Cache Concepts

clients

proxy cache

LANorigin

servers

WAN

6

Why Program a Proxy?

It’s at the right point in network– Sees all client-side and server-side HTTP traffic

– Can react to both LAN and WAN conditions

Already examines layer 7

Groundwork in place for value-adds– Content filtering, access control, etc.

7

Enabling Technologies

Moore’s Law– CPU speeds outstripping all other components

– Lots of cycles to burn…

Proxy software– Increasing efficiency in managing connections,

disk storage, etc.

Commodity OS/hardware improvements– No longer need specialized systems to run

efficient proxy caches

8

Commodity System Improvements

1997: Appliances 4x faster than software running on a 2-processor UltraSparc

[Source: Danzig, “NetCache Architecture and Deployment”]

9

Commodity System Improvements

1997: Appliances 4x faster than software running on a 2-processor UltraSparc

[Source: Danzig, “NetCache Architecture and Deployment”]

1st NLANR cacheoff (April ’99): gap only 2.5 x– 600 req/sec (Peregrine) vs. 1500 (InfoLibria)

10

Commodity System Improvements

1997: Appliances 4x faster than software running on a 2-processor UltraSparc

[Source: Danzig, “NetCache Architecture and Deployment”]

1st NLANR cacheoff (April ’99): gap only 2.5 x

2nd cacheoff (Jan ’00): gap only 1.7x– 1450 req/sec (iMimic) vs. 2400 (Compaq)

11

Commodity System Improvements

1997: Appliances 4x faster than software running on a 2-processor UltraSparc

[Source: Danzig, “NetCache Architecture and Deployment”]

1st NLANR cacheoff (April ’99): gap only 2.5 x

2nd cacheoff (Jan ’00): gap only 1.7x

3rd cacheoff (Oct ’00): gap only 15%– 2083 req/sec (Microsoft) vs. 2400 (Compaq)

12

Commodity System Improvements

1997: Appliances 4x faster than software running on a 2-processor UltraSparc

[Source: Danzig, “NetCache Architecture and Deployment”]

1st NLANR cacheoff (April ’99): gap only 2.5 x

2nd cacheoff (Jan ’00): gap only 1.7x

3rd cacheoff (Oct ’00): gap only 10%

4th cacheoff (Dec ’01): commodity system best– Performance record: 2700 req/sec (Cintel/iMimic)

13

How free is the CPU?

Stratacache Dart-10, with Nokia phone

120 req/sec (7 Mbps) with 300 MHz CPU– CPU mostly idle; performance disk-limited

14

Outline

Background

API Design

API Functions

Implementation and Performance

Conclusions

15

Previous Customization Approaches

Write your own proxy or modify Squid– Huge code, changes likely to conflict with updates

ICAP: TCP-based offload– Proxy redirects requests/responses to a separate

server for modification

Filter-style processes– Plugins where proxy designers anticipated a need

(e.g., content filtering)

Kernel modules– Difficult programming model, but needed for

kernel-integrated proxies

16

Reasons for a New Approach

Scalability needed to > 10,000 flows– Filter processes may not scale

Limitations of ICAP-style offloading– Offloading small requests adds latency

– Need for separate ICAP server with own CPU

Programmers want flexibility– Program in C using standard OS and libraries

– Avoid problems from later code conflicts

17

Design of the Proxy API

Event-aware– Modules notified as requests/responses arrive

– Maps well to implementation of modern proxies

HTTP-Complete– Capture all key interactions in HTTP request-

response protocol for full flexibility

Support various programming models– Events, threads, processes

– Communication via function call or socket

18

HTTP Data Flows

Client Proxy Cache

Server

Storage System

Requests

Responses

CacheMisses

NewContent

CachedContent

CacheHits

19

HTTP Data Flows and the API

Client Proxy Cache

Server

Storage System

modify

modify

modify

modify

modify

20

HTTP Request-Response Structure

Requested URLRequest header line 1Request header line 2...Request header line N<blank terminating line>

Optional request “body"used in POST requestsfor forms, etc.

Header block – special first line

followed bymore detail aboutrequest/response

Body data

Response Status Code Response header line 1Response header line 2...Response header line N<blank terminating line>

Actual response “body,"containing HTML file,image binary data, etc.

21

typedef struct DR_FuncPtrs { DR_InitFunc *dfp_init; // on module load DR_ReconfigureFunc *dfp_reconfig; // on config change DR_FiniFunc *dfp_fini; // on module unload

DR_ReqHeaderFunc *dfp_reqHeader; // when req hdr done DR_ReqBodyFunc *dfp_reqBody; // on each piece of req body DR_ReqOutFunc *dfp_reqOut; // before req to remote srv

DR_DNSResolvFunc *dfp_dnsResolv; // when DNS resolution needed

DR_RespHeaderFunc *dfp_respHeader; // when resp hdr done DR_RespBodyFunc *dfp_respBody; // on each piece of resp body DR_RespReturnFunc *dfp_respReturn; // when resp returned to clt

DR_TransferLogFunc *dfp_logging; // log entry after req done DR_OpaqueFreeFunc *dfp_opaqueFree; // when each resp completes DR_TimerFunc *dfp_timer; // periodic maintenance int dfp_timerFreq; // timer period (sec) } DR_FuncPtrs;

Design of API Notifications

22

Outline

Background

API Design

API Functions

Implementation and Performance

Conclusions

23

API Functions

Content Adaptation

Content Management

Customized Administration

Utility Functions

24

Content Adaptation

Functions to allow modules to inspect and modify requests and replies through cache

Client Proxy Cache

Server

Storage System

modify

modify

modify

modify

modify

25

Content Adaptation (cont’d)

Example uses– Integration into a CDN based on URL rewriting

– Transcoding for mobile devices

Special features of cache integration– Store modified content

– Return multiple versions using HTTP Vary header

26

Content Management

Fine-grained control over cacheability– Content-freshness modification/eviction

– Content preloading

– Content querying

Example uses– News CDN needs new home page on major event

– Premium services

27

Customized Administration

Notifications on logging

Example uses– Aggregation at network operation centers

– Detection of high error rates indicates bad links

28

Utility Functions

Interfaces to underlying OS event-notification– Module may register or clear interest on FD events

– API will automatically call back module

– Independent of underlying OS mechanisms (e.g., poll, select, /dev/poll, kevent)

Configuration options processing

29

Outline

Background

API Design

API Functions

Implementation and Performance

Conclusions

30

Implementation in DataReactor

Commercial proxy server– Portable (x86, Alpha, Sparc), and

(FreeBSD, Linux, Solaris)

– Fast (exposes overheads)

– Independently measured at Proxy Cache-Offs (alone or via OEMs)

Support requires < 1000 lines of code

Implementation < 6 person-months

31

Sample Modules

Ad Remover– Matches ad patterns in Hostname, URI

Dynamic Compressor– Uses zlib to compress, store, & serve object

Image Transcoder– Color stripping via NetPBM & ijpeg helpers

Text Injector– Finds <head> tag, asks helper what to insert

Content Manager– Local telnet, then query, fetch, inject, evict objects

ICAP client– Implements ICAP 1.0 draft to use external server

32

Web Surfing Now

33

Web Surfing Without Ads

34

Sample Module Implementation

Module Name

Total Lines

Code Lines

Semi-colons

# API call sites

Ad Remover 175 115 51 4

Compressor 387 280 126 11

Transcoder + helper

391

+166

309

+118

148

+54

10

Text Injector

+ helper

473

+56

367

+32

170

+8

12

Manager 675 556 289 56

ICAP Client 1024 719 321 15

35

Measurement

Polygraph and PolyMix-3, Measurement Factory– De facto standard for proxy testing

Scales with load– Number of clients– Number of servers– Data set size– Working set size

Very long test time– Fill phase (~14 hours)– Test phase (~10 hours)

36

PolyGraph Test Phases

Fill Phase

1st L

oad

Pha

se

2nd L

oad

Pha

se 0 5 10 15 20 25 30

Time (hours)

37

PolyGraph Hit Rates

Cacheable

Offered

Actual

38

Our Test Environment

Proxy - 1.4GHz Athlon, 2GB memory 5 SCSI disks, GigE, FreeBSD

Harness– 10 Polygraph client/server machines– Target load: 1450 reqs/sec– 16000 simultaneous connections

Pmix-3: Modified Polymix-3– Single fill phase for all tests– Load phase time cut in half– Slight increase in hit rate

39

API Performance

Throughput

req/sec

Response Time, ms

Miss Time, ms

Hit Time, ms

Hit Ratio, %

Baseline 1452.87 1248.99 2742.53 19.82 57.81

API Enabled 1452.75 1248.95 2743.18 19.86 57.81

Empty Callback 1452.89 1251.25 2744.33 20.87 57.76

Add Headers 1452.62 1251.98 2745.07 20.85 57.74

Body + Headers 1452.84 1250.14 2746.98 22.10 57.85

40

Module Performance

Throughput

req/sec

Response Time, ms

Miss Time, ms

Hit Time, ms

Hit Ratio, %

Baseline 1452.87 1248.99 2742.53 19.82 57.81

Ad Remover 1452.72 1248.87 2743.55 20.42 57.81

Images 25 Trans/s 1452.65 1256.60 2753.47 23.21 57.74

Images Max Trans 1452.73 1277.76 2778.09 43.30 57.80

Max Trans Nice 19 1452.68 1250.69 2744.60 20.15 57.78

Compress 75 obj/s 1452.73 1252.24 2745.63 23.44 57.81

Compress 95 obj/s 1452.88 1258.34 2752.63 28.69 57.78

41

Outline

Background

API Design

API Functions

Implementation and Performance

Conclusions

42

Summary

CPUs getting more idle

Commodity OS suitable choices

High-concurrency servers needed

Customizable, efficient event-friendly API

Implemented with low overhead

Sample results, deployments promising

43

Ongoing Work

CoDeeN – a CDN system on PlanetLab– Uses a customized version of DataReactor

– Being built at Princeton

– Prototype: 1 week reading + 1 week reading

– Currently: ~42 nodes (one per site)

Lessons– API easy enough for busy grad students

– Logging infrastructure would be nice

– Want to mask non-HTTP failures

Questions?

vivek@imimic.com

iMimic Networking, Inc.http://www.imimic.com/

45

Cacheoff-3 Hit Times

46

Cacheoff-3 Miss Times

47

Cacheoff-3 Improvements

48

Cacheoff-3 Price/Performance

49

CacheOff-3 Results

50

CacheOff-3 Results

51

Cacheoff-4 Hit Times

52

Cacheoff-4 Miss Times

53

CacheOff-4 Results