internet measurement initiatives in the wisconsin advanced internet lab
DESCRIPTION
Internet Measurement Initiatives in the Wisconsin Advanced Internet Lab. Paul Barford Computer Science Department University of Wisconsin – Madison Spring, 2003. Talk Objectives. Motivate and describe Wisconsin Advanced Internet Lab (WAIL) Internal lab environment External lab environment - PowerPoint PPT PresentationTRANSCRIPT
Internet Measurement Initiatives in the Wisconsin Advanced Internet Lab
Paul Barford
Computer Science Department
University of Wisconsin – Madison
Spring, 2003
Talk Objectives
• Motivate and describe Wisconsin Advanced Internet Lab (WAIL)– Internal lab environment– External lab environment
• Provide some detail on three current projects– Anomaly detection and characterization– Distributed intrusion monitoring– Understanding packet loss
Motivation for New Tools• Any area of scientific research is limited by the tools
available for experimental study– “If your only tool is a hammer then everything looks
like a nail”• 2001 NRC report: “network research community is in
danger of ossification due to strictures of experimental systems”– Challenge: “Capturing a day in the life of the Internet”
• New experimental tools can open up areas of research that have not previously been accessible
An Internet Instance Lab • A hands-on test environment designed to recreate
paths and conditions identical to those in the Internet from end-to-end-through-core– Requires large amount of routing and end host equipment
• Network and host equipment able to recreate (not emulate) a wide range of services, configurations and traffic conditions– Complete instrumentation of end-to-end paths
– Deployment of disruptive prototypes
Key Challenges
• Design• Configurations and management• Traffic generation• Propagation delay• Validation
The Wisconsin Advanced Internet Lab
• Our realization of an IIL• Developed over past 18 months by UW/Cisco team• Supported by $3.5M equipment grant from Cisco and
UW matching funds– Used to purchase over 75 pieces of networking equipment
• Phase 1 nearing completion => Abilene recreation• Other partners: EMC, Spirent, Intel, Fujitsu, Sun• Research initiatives in many areas…
External Environment• Essential complement to internal environment• Existing infrastructure
– DOMINO systems (1 class A + 2 class B’s + Dshield)
– Surveyor + WAWM systems (~70 nodes)• New database and front end by summer ‘03
• Partnerships and other available systems– Condor/Grid Infrastructures
• Passive flow measurements– FlowScan data from UW, Internet2, others…
Project 1: Detecting Anomalies in IP Flows
• Motivation: Anomaly detection remains difficult• Objective: Improve understanding of traffic anomalies• Approach: Multiresolution analysis of data set that
includes IP flow, SNMP and an anomaly catalog• Method: Integrated Measurement Analysis Platform for
Internet Traffic (IMAPIT)• Results: Identify anomaly characteristics using wavelets
and develop new method for exposing short-lived events
Our Data Sets• Consider anomalies in IP flow and SNMP data
– Collected at UW border router (Juniper M10)– Archive of ~6 months worth of data (packets, bytes, flows)– Includes catalog of anomalies (after-the-fact analysis)
• Group observed anomalies into four categories– Network anomalies (41)
• Steep drop offs in service followed by quick return to normal behavior– Flash crowd anomalies (4)
• Steep increase in service followed by slow return to normal behavior– Attack anomalies (46)
• Steep increase in flows in one direction followed by quick return to normal behavior– Measurement anomalies (18)
• Short-lived anomalies which are not network anomalies or attacks
Multiresolution Analysis• Wavelets provide a means for describing time series
data that considers both frequency and time– Powerful means for characterizing data with sharp spikes
and discontinuities
– Using wavelets can be quite tricky
• We use tools developed at UW which together make up IMAPIT– FlowScan software
– The IDR Framenet software
Ambient IP Flow Traffic
Flow Traffic During DoS Attacks
Deviation Score for Three Anomalies
Project 2: Coordinated Intrusion Detection
• Motivation: Intrusion detection is a moving target• Objective: Coordinate intrusion monitoring between
multiple sites around the Internet• Approach: Share data from firewalls, NIDS and tarpits
(on unused IP space)• Method: Distributed Overlay for Monitoring Internet
Outbreaks (DOMINO)• Results: Blacklists can be rapidly generated, false
positives can be substantially lowered, new outbreaks can be easily identified
DOMINO: A new approach to DNIDS
• Partnership with dshield.org– 1600 firewall and NIDS logs
• Tarpits– Active monitor of unused IP space– 1 class A (this week), 2 class B’s
• A protocol for node participation, data sharing and alert clustering– Chord-based overlay network– Extension of Intrusion Detection Message Exchange
Format– Various clustering methods
Marginal Utility of Adding Nodes
SQL-Sapphire Analysis
Project 3: Understanding Packet Loss
• Motivation: Many of the most basic aspects of packet loss are not understood– Where, when, how long, how often?
• Focus: Developing a comprehensive understanding of packet loss in the Internet
• Approach: Combine understanding of protocols and queue behavior to create a probe train which can accurately measure delay and loss.
• Implications: End-to-end tools for pin-pointing loss, better transport protocols, better network management for congestion
Active versus Passive Loss Measures
• Hypothesis: Active measures of loss are correlated with passive measures of loss
• Assessment in Abilene– SNMP loss measures on all backbone routers– Active probes via Ping/Zing in Surveyor nodes at
10Hz, 20Hz and 100Hz– Tests in full mesh over one month period
Result: Active <> Passive
Summary
• Both internal lab building initiatives and external measurement initiatives in WAIL
• Internal facilities are intended to be open
• We are seeking partnerships in external measurement projects.– DOMINO in particular