virtual network diagnosis as a servicewisdom.cs.wisc.edu/workshops/spring-14/talks/wu.pdfvirtual...
TRANSCRIPT
![Page 1: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/1.jpg)
Virtual Network Diagnosis as a
Service
Wenfei Wu (UW-Madison)
Guohui Wang (Facebook)
Aditya Akella (UW-Madison)
Anees Shaikh (Google)
![Page 2: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/2.jpg)
Virtual Networks in the Cloud
Application Traffic
GW GW Subnet 1
VM
VM
VS
MB
R R
Subnet 2
VM
VS
VM
• Data center infrastructure • Virtual networks • Virtual-to-Physical Mapping • Network services • Sharing, Isolation
Tenant 2 VM
VM
VS
VM
![Page 3: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/3.jpg)
Virtual Network Problems
• Multiple layers may have various problems oConnectivity/Performance issues in applications
• Isolation and abstraction prevent tenants from diagnosing their virtual networks
Application Traffic
VM MB VM SW SW
![Page 4: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/4.jpg)
Existing Solutions
• Infrastructure-layer tools may expose the physical network to tenants • sFlow, NetFlow
• OFRewind, NDB, etc.
• Tools in VMs are difficult to deploy in some virtual components (e.g., middleboxes) • Tcpdump, Xtrace, SNAP
![Page 5: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/5.jpg)
VND Proposal
• The cloud provider should offer a virtual network diagnostic service (VND) to the tenants
1. Diagnostic request
2. Trace collection
3. Trace parse
4. Data query
![Page 6: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/6.jpg)
VND Challenges
• Preserve isolation and abstractions
• Low overhead
• Scalability
![Page 7: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/7.jpg)
Contents
• Motivation
• VND Design & Implementation
• Evaluation
• Conclusions
![Page 8: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/8.jpg)
VND Architecture
Cloud Controller
Collection Policy
ts id … ts id …
Raw Data
Trace Collector
Raw Data
Trace Collector
Flow Trace
Collect Config Collect Config
& Topology
Policy Manager
Tenant
Query Execution
Analysis Manager
Table Server Table Server
Control Server
Query Executor
Trace Table
Query Executor
Trace Table
Trace Parser Trace Parser
Parse Config
Parse Config
![Page 9: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/9.jpg)
Data Collection (1)
• The tenant submits a Data Collection Configuration
Load Balancer
Front End
Server 1
Server 2
Server 3
Cap: Input srcIP=10.0.0.6 dstIP=10.0.0.8 tcp dstPort=80 dualDirection
Cap: output …
Virtual Appliance Link : l1 Capture Point node1 Flow Pattern field = value, ... Capture Point node2 ... Virtual Appliance Node: n1 Capture Point input, [output] ...
![Page 10: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/10.jpg)
Data Collection (2)
• Policy Manager generates a Data Collection Policy
srcIP=10.0.0.6, …, etc.
Collector (c1)
tr_id1
Trace ID tr_id1 Physical Cap Point vs1 Collector c1 at h1 Pattern field = value, ... Path vs1, ..., h1, c1
Hypervisor
vSwitch (vs1)
Load Balancer
NIC
![Page 11: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/11.jpg)
Data Parse
• The tenant submits a Data Parse Configuration
Table ID tab_id1 Filter exp Fields field_list
exp = exp and exp | exp or exp | not exp | (exp) | prim prim = field in value_set
field_list = field (as name) (, field (as name))*
Trace ID all Filter: ip.proto = tcp or ip.proto = udp Fields: timestamp as ts, ip.src as src_ip, ip.dst as dst_ip, ip.proto as proto, tcp.src as src_port, tcp.dst as dst_port, udp.src as src_port, udp.dst as dst_port
ts src_IP dst_IP proto src_port dst_port
… … … … … …
![Page 12: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/12.jpg)
Data Analysis
ts id …
Trace_ID Vnet_App Cap_Loc Pattern Trace_ID …
Analysis Manager
Collection View Parse View
ts id … ts id …
Trace Table
Query Executor
Trace Table
Query Executor
Trace Table
Query Executor
SQL Interface
Diagnostic Schema
• Trace Tables form a distributed database
Operations Diagnostic Applications Tenants
![Page 13: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/13.jpg)
Data Analysis Examples
• RTT • Correlate data packet and ack packet
• Compute the average time difference
• Throughput, loss, delay, statistics, etc.
ts src_IP dst_IP seq ack length
t1 IP1 IP2 1000 1 1400
t2 IP2 IP1 0 2400 0
t3 IP1 IP2 2400 1 1400
t4 IP2 IP1 0 3800 0
ts src_IP dst_IP seq ack length
![Page 14: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/14.jpg)
Contents
• Motivation
• VND Design & Implementation
• Evaluation
• Conclusions
![Page 15: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/15.jpg)
Trace Collection (1) • Network Overhead
• Each minute, we capture one additional VM pair’s traffic
• The duplicated traffic increases as we capture more traffic
• The original application traffic is not impacted
…
Server 1
8 VMs
…
Server 2
8 VMs
![Page 16: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/16.jpg)
Trace Collection (2)
• Memory Overhead • VMs perform memory copy and data transfer
simultaneously
• We measure memory and network throughput
• Each 1Gbps network traffic duplication causes 59 MB/s memory overhead
![Page 17: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/17.jpg)
Data Query Overhead • We use RTT monitoring as an example
o200 Mbps traffic
oCalculate average RTT periodically
• Network overhead is negligible
• Execution time scales linearly with the data size
• VND can process 2-3 Gbps traffic in real time
RTT
![Page 18: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/18.jpg)
Scalability
• Control Server is a simple web server can be scaled up easily
• Data collection distributed locally to avoid being the bottleneck
• Query execution is also distributed • Real-time analysis
• 2-3Gbps (RTT example, MySQL implementation)
• Offline analysis • Higher volume
![Page 19: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/19.jpg)
Conclusions
• The cloud provider should offer a virtual network diagnostic service to the tenants
• We design VND • Architecture, interfaces and operations
• Address several important challenges
• Our experiments and simulation demonstrate the feasibility of VND
![Page 20: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/20.jpg)
END
Q&A
Please contact with WISDOM
http://wisdom.cs.wisc.edu
![Page 21: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/21.jpg)
Functional Validation
• Middlebox Bottleneck Detection
• VND use throughput and RTT time to find abnormity
RE IDS
![Page 22: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/22.jpg)
Functional Validation(2)
• Middlebox scaling
RE IDS
![Page 23: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/23.jpg)
VND Architecture
Cloud Controller
Trace Parser
Trace Table
Table Server Policy Manager
Analysis Manager
Collect Config
Parse Config Collect Config
& Topology
Collection Policy
Flow Trace
Parse Config
Query Execution
Tenant
ts id …
Raw Data
Query Executor
Trace Collector
Trace Parser
Trace Table
Table Server
ts id …
Raw Data
Query Executor
Trace Collector
Control Server
![Page 24: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/24.jpg)
Optimizations
• Local Table Server placement • Place the collector locally with the capture points
• Avoid the trace collection traffic traversing the network
• Move data only when queries need it
• Avoid interference with existing rules using the multi-table feature on OVS
![Page 25: Virtual Network Diagnosis as a Servicewisdom.cs.wisc.edu/workshops/spring-14/talks/Wu.pdfVirtual Networks in the Cloud Application Traffic GW GWR Subnet 1 VM VM VS MB R Subnet 2 VM](https://reader034.vdocuments.site/reader034/viewer/2022042316/5f048e407e708231d40e8d5d/html5/thumbnails/25.jpg)
Scalability (2)
• Data Query simulation • A data center with 10,000 servers, each has a 10Gbps
NIC
• Virtual network size [2, 20]
• Query executors can process 3 Gbps traffic in real time
• Total link utilization [0.1, 0.9]
• Results • 30% of total link capacity can be queried in real time