Multi-level Application-based Traffic Characterization in a Large-scale Wireless Network

Maria Papadopouli1,2

Joint Research with Thomas Karagianis3 and Manolis Ploumidis 1,2

1 Department of Computer Science, University of Crete2 Institute of Computer Science, Foundation for Research and Technology-Hellas

3 Microsoft Research

*This work was partially supported by General Secretariat for Research and Technology and by European Commission with a Marie Curie IRG grant

COST-TMA: meeting @ Samos, September 22nd, 23rd 2008

2

Research interests

Traffic modeling Impact of parameters (number of flows, flow inter-arrivals, flow sizes)

on accuracy Topology & mobility modeling Traffic forecasting (moving averages, Singular Spectrum Analysis, etc) Client profiling Mobile p2p computing

Data diffusion using realistic mobility models Efficient selection of appropriate network interface/channel based on

network conditions/application requirements Efficient distributed monitoring Understanding the impact of network conditions on user experience

3

Roadmap

Objectives Testbed, data acquisition & preprocessing Data analysis

Aggregate traffic AP traffic Client traffic

Conclusions Research in progress …

4

Objectives

Classify flows into application types Identify dominant & popular application types Compare UNC network with other wired & wireless networks Characterize AP & client traffic

5

Infrastructure

6

Testbed, data acquisition & preprocessing

Testbed 488 APs, 382 monitored 6,593 distinct MAC addresses – 9,125 distinct IPs

Data acquisition Packet header traces from egress router Client SNMP data

Data preprocessing Correlation of packet headers with client SNMP Classification of flows using BLINC

7

Classification with BLINC: heuristics

Host behavior (e.g., client-server, collaborative)o Host popularity: number of distinct destination IPso Clusters of hosts using a collaborative applicationo Number of source ports

Transport layer protocol: TCP vs. UDP Cardinality of sets (ports vs. IPs) Per flow average packet size

o Constant in several applications (e.g., malware) “Farms” of services: neighboring IPs Non-payload flows (e.g., attacks)

8

Graphlet library

9

Dominant application types

Application type Flows(%) Bytes(%) Packets(%)

Network Management

9.95 0.42 1.54

Chat 2.05 0.48 1.47

Web 35.06 57.59 46.88

P2P 30.04 24.85 34.46

Online Games 1.11 0.01 0.07

FTP 0.91 1.57 1.72

Mail 0.07 0.33 0.21

AddScan 6.4 0.12 0.58

PortScan 0.39 0.32 0.28

Streaming 0.1 0.17 0.19

Unknown 13.2 14.09 12.64

10

Popular application types

Clients with at least one flow per application typeApplication type Clients(%)

Network Management 17

Chat 73

Web 99

P2P 43

Online Games 4

Ftp 7

Mail 1.5

AddScan 73

PortScan 1.4

Streaming 0.5

Unknown 84

11

Compare with other testbeds

Traffic share for most dominant application types Wired & wireless testbeds

UNC wired network Dartmouth wireless infrastructure Residential campus

% Res. Campus UNC Wired UNC Wireless Dartmouth

Web 37.5 48.68 57.59 28.6

P2P 31.9 34.85 24.85 19.3

may have missed all Web traffic that was not accessed through one of the well-known ports for Web

12

Home application type of APs

Traffic of this application type > than x% of total AP traffic Web most prevalent home application type

x Web(%) P2P(%) Ftp(%) Mail(%) Unkn

50 85.9 6.17 0.28 0 4.2

75 55.8 0.28 0 0 0.84

90 25.2 0.28 0 0 0

13

Client traffic characterization

Client home application:Application type of which this clients transfer >X% of their traffic Clients have strong application preferences

~ 50% of clients have home application type (for X=90) Web: most prevalent home application type

Clients with no home application are dominated by Web Only a minority of clients have P2P as dominant application

14

Wireless traffic load Wide range of workloads & log normality is prevalent

Light traffic load but with long tails Dichotomy among APs:

APs dominated by uploaders APs dominated by downloaders

Majority of APs send & receive packets of small size Significant number of APs with asymmetric packet sizes:

APs with large sent & small receive packets APs with small sent & large receive packets

15

Application-based characterization Most popular applications

Web browsing & p2p accounting ~81% of total traffic These applications dominate most users and APs Web dominates both AP & client traffic share

Network management & scanning activity ~17% of total flows Application-mix varies within APs of same building Wireless clients with strong application-type interests File transfer flows (e.g., ftp, p2p) are heavier in wired

network than in wireless one Flow sizes per application type

Different between wired & wireless network

16

In progress …

Focus on applications with real-time constraints Impact of “extreme” network conditions on performance

& user satisfaction Statistical analysis for client profiles

Comparable analysis with other wireless networks

17

UNC/FORTH Web Archive

Online repository of Wireless measurement traces

Packet header, SNMP, SYSLOG, signal quality

Models Tools

http://netserver.ics.forth.gr/datatraces Login/ password access after free registration

Maria Papadopouli mgp@ics.forth.gr

18

Total network traffic across APs

19

Application traffic share across APs

20

Traffic asymmetry (2/2)

21

BLINC

BLINd Classification Flows in application types

Focus on end hosts rather than on flow 3-level host behavior analysis

Social Functional Application

Application signature based classification Accurate flows classification

22

Heuristics (2/2)

1. Community heuristic Farms of services in neighboring IPs

2. Recursive detection Interaction between servers

Mail with Razor servers

23

Application level

Transport layer interaction between hosts Based on TCP 4-tuple Empirically derived signatures – graphlets

Nodes: Src,Dst IP & Src,Dst Port Edges: Flows through this TCP-tuple Protocol type

Host behavior against graphlet library

24

Bldg level application usage patterns

% of APs with home application type / bldg type Weak correlation between building category & # of

APs with home application Distinct APs different configurations

Uneven traffic distribution across APs of same bldg APs dominated by Web, P2P, or unknown traffic

25

Conclusions

Three-level characterization of large scale infrastructure Support admission control & AP selection mechanisms Indicate user trends Assist application specific traffic modeling

Web dominates both AP & client traffic share P2P systems bear a significant impact Clients have strong application preferences

26

Heuristics used in classification

1. Transport layer protocol: TCP vs. UDP

2. Cardinality of sets Ports vs. IPs Constant in several applications (e.g.,

malware)

3. Community heuristic Farms of services in neighboring IPs

4. Non-payload flows (e.g., attacks)

27

Attack graphlets

Address-Scan attack Address-Scan attack for specific IP set Port-scan attack

28

P2P Graphlets

29

Traffic asymmetry (1/2)

Asymmetry index = total downloaded / total uploaded traffic

Certain APs dominated by uploaders

Asymmetry index / application type

Asymmetry index for P2P traffic < 1 for 40% of APs

30

Flow sizes per application type

31

Wireless user application preferences

Similar between wireless & wired users Flow sizes / application type

Different between wired & wireless network Possible reasons

Application dependent User-driven