enabling a “risc” approach for software-defined monitoring using universal streaming vyas sekar...
TRANSCRIPT
![Page 1: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/1.jpg)
Enabling a “RISC” Approach for Software-Defined Monitoring
using Universal Streaming
Vyas Sekar
Zaoxing Liu, Greg Vorsanger,
Vladimir Braverman
![Page 2: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/2.jpg)
Network Management:Many Monitoring Requirements
SDN Controller (OpenDayLight etc.)
Traffic Engineering
Analyze new user apps
Anomaly DetectionNetwork Forensics
Worm Detection
Accounting
Botnet analysis …….
“Heavy-hitters”“Flow size distribution”
“SuperSpreaders”
“Entropy”, “Traffic Changes”
1
![Page 3: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/3.jpg)
Traditional: Packet Sampling
3
1613111
Flow reports1
Not good for fine-grained analysisExtensive literature on limitations for many tasks!
11316111131611
12
Sample packets at random, aggregate into flows
FlowId CounterFlow = Packets with same patternSource and Destination Address and Ports
Estimate: FSD, Entropy, Heavyhitters, Changes, SuperSpreaders ….
![Page 4: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/4.jpg)
4
Application-Specific Sketches
Packet Processing
Counter Data
Structures
Application-LevelMetric
Heavy Hitter Entropy Superspreader
Complexity: Need per-metric implementationRecent Example: OpenSketch [NSDI’13]Trend: Many more applications appear!
….Monitoring(on router)
Bloom-filter,Count-min Sketch,reversible sketch, etc.
Packet Processing
Counter Data
Structures
Application-LevelMetric
Packet Processing
Counter Data
Structures
Application-LevelMetric
….
Traffic
Computation(off router)
![Page 5: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/5.jpg)
5
Packet Processing
Counter Data
Structures
Application-LevelMetric
Support many applications
Holy Grail of Flow Monitoring?
Results with high accuracy
Traffic
![Page 6: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/6.jpg)
6
Our Solution: Universal Monitoring
Recent theory advances: Universal Streaming
Packet Processing
UniversalSketch
Traffic
App 1
Application-specific Computation
App n…...
UnivMon Control Plane
UnivMon Data Plane
One sketch does it ALL
![Page 7: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/7.jpg)
Theory of Universal Streaming
1. Vladimir Braverman, Rafail Ostrovsky: Zero-one frequency laws. STOC 20102. Generalizing the Layering Method of Indyk and Woodruff: Recursive Sketches for Frequency-Based Vectors on Streams. APPROX-RANDOM 2013
1331511 2 4 6 5 …... (A stream of length m with n unique items)
‘Universal’ Sketch
Estimated G-sum
frequency vector is <f1,f2 … fn>
G-sum =
As long as does not grow asymptotically faster than2,Universal Sketch can do it!
6
![Page 8: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/8.jpg)
8
Universal Sketch Data Structure
1331511 2 4 6 5
11511
25
2
L2 Heavy Hitter Algorithms
(1,4), (3,2),(5,2)
Heavy Hitters
(1,4), (5,2),(2,1)
…...
(2,1)
(5,2), (2,1)
0
1
log(n)
…...
Generate k=log(n) pairwise ind. zero-one hash functions:
H1 …. Hk
2 5
5
Similar to counting bloom filter
H1(1)=1, H1(5)=1, H1(2)=1
H2(5)=1, H2(2)=1
H3(2)=1
LevelsHeavy Hitter Alg
Heavy Hitter Alg
Heavy Hitter Alg
Heavy Hitter Alg
Count Sketch Alg+4 +2 -2
-2 -4 +2
+2 +4 -2
+4 -2 -1
+4 -2 +1
+1 -2 -4
+1
-1
+1
…...
Count-Sketch, Pick-and-drop etc.
In Parallel
![Page 9: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/9.jpg)
9
Estimating G-sum
(1,4), (3,2),(5,2)
Counters from Universal Sketch
(1,4), (5,2), (2,1)
…...
(2,1)
(5,2),(2,1)
Levels0
1
log(n)
…...
Apply arbitrary g()
(1,g(4)), (3,g(2)),(5,g(2))
(1,g(4)), (5,g(2)), (2,g(1))
(5,g(2)),(2,g(1))
(2,g(1)) Y3=g(1)Sum of the g()s
Y2=g(1)+g(2)
Y1=g(1)+g(2)+g(4)
Y0=2g(1)+2g(2)+g(4)
Estimated G-sum
Recursive Steps:Yi-1 = 2Yi + new counters – repeated counters
![Page 10: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/10.jpg)
10
Putting it together: UnivMon
Universal Sketch Offline Recursive Computation
![Page 11: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/11.jpg)
11
Comparison with custom sketches via OpenSketch
Preliminary Evaluation
N/A
![Page 12: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/12.jpg)
12
• Distributed universal streaming
• Multidimensional data
• Dynamically change monitoring scope
• Feasibility of hardware implementations?
Future Directions
![Page 13: Enabling a “RISC” Approach for Software-Defined Monitoring using Universal Streaming Vyas Sekar Zaoxing Liu, Greg Vorsanger, Vladimir Braverman](https://reader035.vdocuments.site/reader035/viewer/2022081520/5697bf9e1a28abf838c9410c/html5/thumbnails/13.jpg)
13
Conclusions• Network management needs many traffic metrics• Today’s solutions offer undesirable extremes• Generic but low fidelity (e.g., sampling)• High fidelity but high complexity (e.g., specific-sketches)
• Holy grail: Universal Monitoring• Decouple monitoring control and data plane like SDN!
• This work: Can be viable via Universal Sketches• Several open questions• e.g. dynamic, multidimensional, distributed, hardware viability