learning relaxed belady for cdn caching
TRANSCRIPT
![Page 1: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/1.jpg)
LearningRelaxedBeladyforCDNCaching
COS316:PrinciplesofComputerSystemDesignLecture10
AmitLevy&WyattLloyd
![Page 2: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/2.jpg)
EdgeCachewithDifferentAlgos
• Clairvoyant (Bélády) shows we can do much better!
��
��
��
��
��
��
� �� ��
���������
����������
�������������������
�������
Infinite Cache
![Page 3: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/3.jpg)
3
CuttingEdgeResearchFromPrinceton!LearningRelaxedBelady forContentDistributionNetworkCaching.
Zhenyu Song,DanielS.Berger,KaiLi,andWyattLloyd.
In17thUSENIXSymposiumonNetworkedSystemsDesignandImplementation(NSDI20),February2020.
![Page 4: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/4.jpg)
Edgecache
CDNCachingGoal:MinimizeWANTrafficRequests
MissHit
User
Requests
WideAreaNetwork(WAN)trafficisexpensive
Keymetrichitratio
4
![Page 5: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/5.jpg)
CachingRemainsChallengingHeuristic-basedalgorithms(1965–):LRU,LFU,GDSF,ARC,...● Workwellforsomeworkloads,butworkpoorlyforother
ML-basedadaptationofheuristics(2017–):UCB,LeCAR,...● Alsoworkwellforsomeworkloads,butpoorlyforothers
TheBelady algorithm(1966)● Offlineoptimal:requiresfutureknowledge● Largegapinmissratiobetweenstate-of-the-artandBelady:● 20–40%onproductiontraces
5
![Page 6: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/6.jpg)
IntroducingLearningRelaxedBelady(LRB)
Newapproach:mimicBelady usingmachinelearning
6
● Machine-Learning-for-Systems(ML-for-Systems)○ Enablingtechnologies
○ Whendoesitmakesense?
![Page 7: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/7.jpg)
GeneralOverviewofourApproach
R R R R R R·········
Now
Cache
R
Pastinformation
MLarchitecture
Trainingdata
Evictioncandidates
7
![Page 8: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/8.jpg)
Challenge1:PastInformation
R R R R R R·········
Now
Cache
RWhatpastinformationtouse?
Pastinformation
MLarchitecture
Trainingdata
Evictioncandidates
8
Moredataimprovestrainingbutincreasesmemory overhead
![Page 9: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/9.jpg)
Challenge2:GenerateOnlineTrainingData
R R R R R R·········
Now
Cache
RWhatpastinformationtouse?
Generateonlinetrainingdata?
Pastinformation
MLarchitecture
Trainingdata
Evictioncandidates
9
![Page 10: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/10.jpg)
Challenge3:MLArchitecture
R R R R R R·········
Now
Cache
RWhatpastinformationtouse?
Generateonlinetrainingdata?
WhatMLarchitecturetoselect?
Pastinformation
MLarchitecture
Trainingdata
Evictioncandidates
10
Largedesignspace:features,model,predictiontarget,lossfunction
![Page 11: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/11.jpg)
Challenge4:EvictionCandidates
R R R R R R·········
Now
Cache
R
Pastinformation
MLarchitecture
Trainingdata
Evictioncandidates
Howtoselectevictcandidates?
Whatpastinformationtouse?
Generateonlinetrainingdata?
WhatMLarchitecturetoselect?
11
![Page 12: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/12.jpg)
Solution:RelaxedBelady Algorithm
Howtoselectevictcandidates?
Whatpastinformationtouse?
Generateonlinetrainingdata?
WhatMLarchitecturetoselect?RelaxedBeladyalgorithm
16
![Page 13: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/13.jpg)
Challenge:HardtoMimicBelady Algorithm
MimickingexactBeladyisimpractical● Needpredictionsforallobjects→prohibitivecomputationalcost● Needexactpredictionofnextaccess→furtherpredictionareharder
Belady:evictobjectwithnextaccessfarthestinthefuture
17
Cache(now)
A······
B
C D
Timetonextrequest
D B A C······ ······
Evict
![Page 14: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/14.jpg)
IntroducingtheRelaxedBeladyAlgorithm
Observation:manyobjectsaregoodcandidatesforeviction
RelaxedBelady evictsa random objectbeyondboundary
18
Cache(now)
A······
B
C D
Timetonextrequest
D B A C······ ······
EvictBeladyboundary
● Donotneedpredictionsforallobjects→reasonablecomputation● Noneedtodifferentiatebeyondboundary→simplifiestheprediction
![Page 15: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/15.jpg)
Challenge1:PastInformation
R R R R R R·········
MLarchitecture
Trainingdata
Evictioncandidates
Moredataimprovestrainingbutincreasesmemory overhead
PastinformationWhatpastinformationtouse?Now
Cache
R
22
![Page 16: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/16.jpg)
TrackObjectswithinaSlidingMemoryWindow
Perobjectfeatures
R R R R R R·········
Now
R
Slidingmemorywindow mimicsBeladyboundary
Onlytrackobjectswithinmemorywindow
23
WindowsizeisLRB’smainhyperparameter
![Page 17: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/17.jpg)
Challenge2:TrainingData
R R R R R R·········
Now
Cache
RWhatpastinformationtouse?
Generateonlinetrainingdata?
Pastinformation
MLarchitecture
Trainingdata
Evictioncandidates
24
![Page 18: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/18.jpg)
SampleTrainingData&LabelonAccessorBoundary
Perobjectfeatures
R R R R R R·········
Now
R
Slidingmemorywindow
Sample
Unlabeledtrainingdata
Past memorywindowAccess
Labeledtrainingdata
25
![Page 19: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/19.jpg)
Challenge3:MLArchitecture
Largepotentialdesignspace
R R R R R R·········
Now
Cache
RWhatpastinformationtouse?
Generateonlinetrainingdata?
WhatMLarchitecturetoselect?
Pastinformation
MLarchitecture
Trainingdata
Evictioncandidates
26
![Page 20: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/20.jpg)
Solution3:Feature&ModelSelection
Gradientboostingdecisiontrees
Lightweight &highgooddecisionratio
Training~300ms,prediction~30us
Features
Objectsize
Objecttype
Inter-requestdistances(recency)
Exponentialdecaycounters(long-termfrequencies)
Usegooddecisionratiotoevaluatenewdesigns
27
![Page 21: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/21.jpg)
Challenge4:EvictionCandidates
R R R R R R·········
Now
Cache
R
Pastinformation
MLarchitecture
Trainingdata
Evictioncandidates
Howtoselectevictcandidates?
Whatpastinformationtouse?
Generateonlinetrainingdata?
WhatMLarchitecturetoselect?
28
![Page 22: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/22.jpg)
Solution4:RandomSamplingforEviction
CanmimicrelaxedBeladyifwecanfind1objectbeyondtheboundary
k=64candidates;moredoesnotimprovegooddecisionratio
R R R R R R·········
Now
Cache
R
Pastinformation
Randomkcandidates
29
![Page 23: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/23.jpg)
Label
Labeleddataset
Sample
Unlabeleddataset
LearningRelaxedBelady
30
Now
Cache
RR R R R R R······
Memorywindow
RR
Train
Model EvictionCandidates
···
Sample
Predict
Evict
![Page 24: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/24.jpg)
● Simulatorimplementation○ LRB+14otheralgorithms
● Prototypeimplementation○ C++ontopofproductionsystem(ApacheTrafficServer)○ Manyoptimizations
Implementation
31
![Page 25: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/25.jpg)
● Q1:LearningRelaxedBelady(LRB)trafficreductionvsstate-of-the-art
● Q2:overheadofLRBvsCDNproductionsystem
● Traces:6productiontracesfrom3CDNs
● Hyperparameter(memorywindow/model/...)tunedon20%oftrace
EvaluationSetup
32
![Page 26: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/26.jpg)
LRBReducesWANTraffic20%trafficreductionoverB-LRU10%reductionoverthebestSOA
Wikipediatrace
Industrystandard
33
![Page 27: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/27.jpg)
CDN-B1 CDN-B3CDN-B2
LRBConsistentlyImprovesontheStateoftheArt
Wikipedia CDN-A1 CDN-A2
34
![Page 28: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/28.jpg)
LRBOverheadIsModest
Throughput:11.7Gbpsvs11.7Gbps(unmodified)
Memoryoverhead=1‒3%cachesize
PeakCPU:16%vs9%(unmodified)
35
Edgecache
Requests
User
![Page 29: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/29.jpg)
Conclusion
● LRBreducesWANtrafficwithmodestoverhead
● ML-for-systemsgenerallypromisingtoreplaceheuristics
● Keyinsight:relaxedBelady
→ Simplifiesmachinelearning&reducessystemoverhead
36
![Page 30: Learning Relaxed Belady for CDN Caching](https://reader031.vdocuments.site/reader031/viewer/2022012409/616a405c11a7b741a3507728/html5/thumbnails/30.jpg)