nps: a non-interfering web prefetching system ravi kokku, praveen yalagandula, arun venkataramani,...
TRANSCRIPT
![Page 1: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/1.jpg)
NPS: A Non-interfering Web Prefetching System
Ravi Kokku, Praveen Yalagandula,
Arun Venkataramani, Mike Dahlin
Laboratory for Advanced Systems Research
Department of Computer Sciences
University of Texas at Austin
![Page 2: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/2.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 2
Summary of the Talk
Prefetching should be done aggressively, but safely
Safe: Non-interference with demand requests
Contributions:A self-tuning architecture for web prefetching
• Aggressive when abundant spare resources• Safe when scarce resources
NPS: A prototype prefetching system• Immediately deployable
![Page 3: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/3.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 3
Outline
Prefetch aggressively as well as safely
Motivation Challenges/principles NPS system design Conclusion
![Page 4: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/4.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 4
What is Web Prefetching?
Speculatively fetch data that will be accessed in the future
Typical prefetch mechanism [PM96, MC98, CZ01]
Demand Requests
Responses + Hint Lists
Prefetch Requests
Client Server
Prefetch Responses
![Page 5: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/5.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 5
Why Web Prefetching?
Benefits [GA93, GS95, PM96, KLM97, CB98, D99, FCL99, KD99, VYKSD01, …]
Reduces response times seen by users Improves service availability
Encouraging trendsNumerous web applications getting deployed
• News, banking, shopping, e-mail…
Technology is improving rapidly capacities and prices of disks and networks
Prefetch Aggressively
![Page 6: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/6.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 6
Why doesn’t everyone prefetch?
Extra resources on servers, network and clients Interference with demand requests
Two types of interference• Self-Interference– Applications hurt themselves• Cross-Interference– Applications hurt others
Interference at various components• Servers – Demand requests queued behind prefetch• Networks – Demand packets queued or dropped• Clients – Caches polluted by displacing more useful data
![Page 7: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/7.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 7
Example: Server Interference
Common load vs. response curveConstant-rate prefetching reduces server capacity
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
100 200 300 400 500 600 700 800
Demand Connection Rate (conns/sec)
DemandPfrate=1
Pfrate=5
Avg
. Dem
and
Res
pons
e T
ime
(s)
Prefetch Aggressively, BUT SAFELY
![Page 8: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/8.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 8
Outline
Prefetch aggressively as well as safely
Motivation Challenges/principles
Self-tuningDecoupling prediction from resource managementEnd-to-end resource management
NPS system design Conclusion
![Page 9: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/9.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 9
Goal 1: Self-tuning System
Proposed solutions use “magic numbers”Prefetch thresholds [D99, PM96, VYKSD01, …]Rate limiting [MC98, CB98]
Limitations of manual tuningDifficult to determine “good” thresholds
• Good thresholds depend on spare resources
“Good” threshold varies over timeSharp performance penalty when mistuned
Principle 1: Self-tuningPrefetch according to spare resources Benefit: Simplifies application design
![Page 10: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/10.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 10
Goal 2: Separation of Concerns
Prefetching has two components Prediction – What all objects are beneficial to prefetch? Resource management –How many can we actually prefetch?
Traditional techniques do not differentiate Prefetch if prob(access) > 25% Prefetch only top 10 important URLs Wrong Way! We lose the flexibility to adapt
Principle 2: Decouple prediction from resource management Prediction: Application identifies all useful objects
• In the decreasing order of importance
Resource management: Uses Principle 1• Aggressive – when abundant resources
• Safe – when no resources
![Page 11: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/11.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 11
Goal 3: Deployability
Ideal resource management vs. deployability Servers
• Ideal: OS scheduling of CPU, Memory, Disk…• Problem: Complexity – N-Tier systems, Databases, …
Networks• Ideal: Use differentiated services/ router prioritization• Problems: Every router should support it
Clients• Ideal: OS scheduling, transparent informed prefetching• Problem: Millions of deployed browsers
Principle 3: End-to-end resource management Server – External monitoring and control Network – TCP-Nice Client – Javascript tricks
![Page 12: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/12.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 12
Outline
Prefetch Aggressively as well as safely
Motivation Principles for a prefetching system
Self-tuning Decoupling prediction from resource management End-to-end resource management
NPS prototype design Prefetching mechanism External monitoring TCP-Nice Evaluation
Conclusion
![Page 13: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/13.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 13
Prefetch Mechanism
Client
PrefetchServer
DemandServer
Munger
Demand Requests
Hint Lists
Prefetch Requests
Fileset
HintServer
Server m/c
1. Munger adds Javascript to html pages
2. Client fetches html page3. Javascript on html page fetches hint list
4. Javascript on html page prefetches objects
![Page 14: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/14.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 14
End-to-end Monitoring and Control
Principle: Low response times server not loaded Periodic probing for response times Estimation of spare resources (budget) at server – AIMD Distribution of budget
• Control the number. of clients allowed to prefetch
MonitorHintServer
DemandServer
ClientGET http://repObj.html
200 OK…
while(1) { getHint( ); prefetchHint( ); }
if (budgetLeft) send(hints);else send(“return later”);
![Page 15: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/15.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 15
Monitor Evaluation (1)
End-to-end monitoring makes prefetching safe
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 100 200 300 400 500 600 700 800
Avg
De
ma
nd
Re
spo
nse
Tim
e(se
c)
Demand Connection Rate (conns/sec)
Monitor
Manual tuning,Pfrate=1
Manual tuning, Pfrate=5
No-Prefetching
![Page 16: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/16.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 16
Monitor Evaluation (2)
Manual tuning is too damaging at high load
0
20
40
60
80
0 100 200 300 400 500 600 700 800
Ba
ndw
idth
(M
bp
s)
Demand Connection Rate (conns/sec)
pfrate=1Prefetch:
Demand: pfrate=1
No-Prefetching
![Page 17: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/17.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 17
Monitor Evaluation (2)
Manual tuning too timid or too damagingEnd-to-end monitoring is both aggressive and safe
0
20
40
60
80
0 100 200 300 400 500 600 700 800
Ba
ndw
idth
(M
bp
s)
Demand Connection Rate (conns/sec)
pfrate=1Prefetch:
Demand: pfrate=1
Demand:Monitor
No-Prefetching
Prefetch:Monitor
![Page 18: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/18.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 18
Network Resource Management
Demand and prefetch on separate connections Why is this required?
HTTP/1.1 persistent connections In-order delivery of TCPSo prefetch affects demand
How to ensure separation?Prefetching on a separate server port
How to use the prefetched objects?Javascript tricks – In the paper
![Page 19: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/19.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 19
Network Resource Management
Prefetch connections use TCP Nice TCP Nice
A mechanism for background transfersEnd-to-end TCP congestion controlMonitors RTTs and backs-off when congestionPrevious study [OSDI 2002]
• Provably bounds self- and cross-interference• Utilizes significant spare network capacity
Server-side deployable
![Page 20: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/20.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 20
Measure avg. response times for demand reqs.Compare with No-Prefetching and Hand-tuned
Experimental setup
End-to-end Evaluation
PrefSvrApache:8085
DemandSvrApache: 80
Fileset
HintSvrPPM predict
Clienthttperf
NetworkCable modem,
Abilene
TraceIBM server
![Page 21: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/21.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 21
Prefetching with Abundant Resources
Both Hand-tuned and NPS give benefitsNote: Hand-tuned is tuned to the best
24.3
37.3
11.8
19.114.5
18
0
10
20
30
40
50
Cable modem Abilene
Av
g. D
em
. Re
sp. T
ime
(m
s)
No prefetching Hand-tuned NPS
![Page 22: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/22.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 22
Tuning the No-Avoidance Case
Hand-tuning takes effort NPS is self-tuning
24.3 23.3
16.4
11.815.7
18.3 19.1
14.5
0
10
20
30
Cable modem
Av
g. D
em
. Re
sp. T
ime
(m
s)
No prefetching T=0.5 T=0.2 T=0.1
T=0.05 T=0.02 T=0.01 NPS
![Page 23: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/23.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 23
Prefetching with Scarce Resources
Hand-tuned damages by 2-8x NPS causes little damage to demand
41.6 31.6
94.6
221.1
31.8 41.1
0
50
100
150
200
250
Loaded Server Loaded Network
Av
g. D
em
. Re
sp. T
ime
(m
s)
No prefetching Hand-tuned NPS
![Page 24: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/24.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 24
Conclusions
Prefetch aggressively, but safely Contributions
A prefetching architecture• Self-tuning• Decouples prediction from resource management• Deployable – few modifications to existing infrastructure
Benefits• Substantial improvements with abundant resources• No damage with scarce resources
NPS prototype http://www.cs.utexas.edu/~rkoku/RESEARCH/NPS/
![Page 25: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/25.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 25
Thanks
![Page 26: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/26.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 26
Prefetching with Abundant Resources
Both Hand-tuned and NPS give benefitsNote: Hand-tuned is tuned to the best
24.337.3
134.9
11.8 19.1
73.3
14.5 18
75.5
020406080
100120140160
Cable modem Abilene Trans-atlantic
Av
g. D
em
. Re
sp. T
ime
(m
s)
No prefetching Hand-tuned NPS
![Page 27: NPS: A Non-interfering Web Prefetching System Ravi Kokku, Praveen Yalagandula, Arun Venkataramani, Mike Dahlin Laboratory for Advanced Systems Research](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649e5f5503460f94b599cb/html5/thumbnails/27.jpg)
April 19, 2023 Department of Computer Sciences, UT Austin 27
Client Resource Management
Resources – CPU, memory and disk caches Heuristics to control cache pollution
Limit the space prefetch objects takeShort expiration time for prefetched objects
Mechanism to avoid CPU interferenceStart prefetching after all demand done
• Handles self-interference – more common case
What about cross-interference?• Client modifications might be necessary