ixp data analysis - eecs.tu-berlin.de · internet exchange points (ixps) layer-2 switch as4 content...

IXP Data Analysis

Anja Feldmann TU Berlin/T-Labs

Walter Willinger Niksun Inc

Nikos Chatzis George Smaradakis

Jan Böttger Nadi Sarrar

Thomas Krenc TU Berlin/T-Labs

Steve Uhlig Queen Mary

University of London

Bernhard Ager ETH Zürich

Internet: Mental model (before 2010)

http://conferences.sigcomm.org/sigcomm/2010/slides/S3Labovitz.pdf

Internet mental model – ala 2011

http://conferences.sigcomm.org/sigcomm/2010/slides/S3Labovitz.pdf

Google, Akamai,

RapidShare, …

Question – What about IXPs

Who are these IXPs?

What do they provide us with?

Google, Akamai,

RapidShare, … IXP

A bit of history about IXPs

Simple definition (to begin with):

“A physical point where Internet traffic can be exchanged”

North America

Decommissioning of National Science Foundation Network (NSFNET) around 1994/95

Establishment and operation of 4 Network Access Points (NAPs)

IXPs in US are the “descendents” of the NAPs From 4 in 1996 to more than 80 modern IXPs in US

Typically run as for-profit commercial entities worldwide in 2012

Fulfill NAPs„ orginal role:

• Provide physical infrastructure

• Carrier-neutral meeting place for ISPs to communicate with each other, independent of third parties

Internet eXchange Points (IXPs)

Layer-2 switch

AS4

Content

Provider 2

AS5

AS1

AS2 Content

Provider 1

AS3

IXPs Offer connectivity to ASes

A bit of history about IXPs (2)

NAP like facilities in Europe

Started in early 1990s

Motivation: Linking their networks for exchanging their local traffic locally vs. astronomical transatlantic bandwidth costs.

European IXPs:

European IXP marketplace has strived

Business model: Mainly not-for profit

Vibrant and innovative European IXP scene

> 150 IXPs from huge, to medium, to small

Infrastructure of an IXP (DE-CIX)

http://www.de-cix.net/about/topology/

Robust infrastructure with redundency

The role of IXPs today

Local traffic stays local

1. Reduced transit costs to upstream providers. Avoids paying the astronomical transatlantic bw costs

2. Improved network performance and QoS due to reduced delay (e.g., decreased round-trip times) and better routes (e.g., reduced number of AS hops for typical end-to-end paths)

3. Boost to local Internet economy due to “good” Internet

4. More international bw for expensive international traffic

Offer services for the “common good”

DE-CIX: A local power-house …

DE-CIX was founded in 1995 (one location in Frankfurt)

Started with 3 ISPs, is run by a non-profit organization

Today (April 2013), DE-CIX operates a distributed infrastructure, with 5 different locations within Frankfurt

Providing service at some 18 DCs/co-location facilities within the Frankfurt metro-area

Providing interconnection services at access speed from1 Gbps to multiple 100 Gbps ports

~500 members, ~15PB/day

IXP Traffic Growth, 2007-2012 (DE-CIX)

~10-20 PB/day early 2013

Outline

Introduction to IXPs

A large European IXP

IXP peering fabric

IXP member diversity

IXP traffic

IXPs as vantage points

Summary

Data – From collaboration with IXP

Collaboration with a major European IXP

Expanding… Started cooperations with two other IXPs

Data: Summary of traffic flows:

sFlow records

Recall: Petabytes per hour… • Sampled data: 1 out of 16 K packets

• Summaries: 128 bytes IP/TCP/UDP headers

Collected from 2011 onward

Data – From collaboration with IXP

Fact 1 – IXP members/participants

Apr 25 May 1

Aug 22 Aug 28

Oct 10 Oct 16

Nov 28 Dec 4

Member ASes 358 375 383 396

Tier-1 13 13 13 13

Tier-2 281 292 297 306

Leaf 64 70 73 77

Countries of member ASes 43 44 45 47

Continents of member ASes 3 3 3 3

Daily avg. volume (PB) 9.0 9.3 10.3 10.7

Traditional classification for 2012

Fact 2 – IXP members/participants

Member ASes often offer multiple services

By Business type

Fact 3 – IXP traffic

Traffic Volume: Same as Tier-1 ISPs

IXP is interchange for Tier-2 ISPs

Outline

Introduction to IXPs

A large European IXP

IXP traffic

IXPs as vantage points

IXP peering fabric

Summary

Fact 3 – traffic – top-10 tier-2 members

Pronounced time of day effects

Some ASes fully utilize their capacity

Application classification: Challenges

Classification methods

Port-based

Payload-based

Flow feature

Host-behavior

Limitation of sFlow data

Sampled sFlow records 1 out of 16K

Only first 128 bytes of each frame

For 83% (74%) of TCP (UDP) flows we see 1 packet

Neither method sufficient => use combination

Application classification: Approach

Use combined approach

Take advantage of UDP signatures

Application classification: Results

Application mix

Do different subsets of AS-links one “sees” different application mix

What is “representative”?

Identify end-point roles

There are 3 groups of /24s: client-only, server-only, and mixed

Outline

Introduction to IXPs

A large European IXP

IXP traffic matrix

IXPs as vantage points

IXP peering fabric

Summary

Are IXPs good vantage points? Yes!

What do we see? What do we miss?

N. Chatzis, G. Smaragdakis, J. Böttger, T. Krenc, A. Feldmann, and W. Willinger “On the benefits of

using a large IXP as an Internet vantage point” ACM IMC13

What about network heterogenity?

Focus on commercial traffic (Use server-related traffic)

Identify server IPs

Cluster them according to the organization responsible for the traffic

N. Chatzis, G. Smaragdakis, J. Böttger, T. Krenc, A. Feldmann, and W. Willinger “On the benefits of

using a large IXP as an Internet vantage point” ACM IMC13

A toy example

AS1

AS4

AS2

AS3

S

S

S

S

S

S

S

S

S

S

1. AS1 and AS5 belong to the same organization 2. AS2 hosts (servers of) many organizations 3. Owner of does not have an AS 4. AS3 is an IXP or part of ISP bundling

S

AS5 C

C C

C C C

S

AS6

AS7 C

C C

S

S C

Who are the main players?

AS1

AS4

AS2

AS3

S

S

S

S

S

S

S

S

S

S AS5 C

C C

C C C

S

AS6

AS7 C

C C

S

S C

1. Are they AS1-AS7 or the owners of ? Recall that these servers may be

“responsible” for ~2/3 of the traffic

… so their owner can shape the traffic!

S S S

S

Today‟s typical assumptions…

AS1

AS4

AS2

AS3

S

S S

S

S

S

S

S

S

S AS5 C

C C

C C C

S

AS6

AS7 C

C C

S

S C

1. So we cannot explain how

The owner of shapes the traffic

• Because we do not associate all its servers with it and we associate wrong servers with it

S

AS link heterogeneity

Given a member AS

How much traffic from, e.g., Akamai servers comes/goes from/to the Akamai member AS vs.

How much traffic from, e.g., Akamai servers comes/goes from/to non-Akamai member ASes

S

S

Akamai AS (member)

non-Akamai AS

(member) Akamai servers

member AS

Using IXPs as vantage points

The end of the traditional AS-level view of Internet

An AS-level view is outdated/not very informative

An end-to-end view provides more insight!

Outline

Introduction to IXPs

A large European IXP

IXP traffic

IXPs as vantage points

IXP peering fabric

Summary

IXPs – Publicly available information Sources: euro-ix, PCH, PeeringDB, IXP‟s sites

Generally known: # IXPs ~ 350 worldwide

http://www.pch.net

IXPs – Publicly available information

0

100

200

300

400

500

600

ASNs at IXP

Unique ASNs

https://www.euro-ix.net

Generally known: # IXPs ~ 350 worldwide

Somewhat known: # ASes per IXP up to 500

IXPs – Publicly available information

0

1000

2000

3000

4000

5000

6000

7000

Europe NorthAmerica

Asia/Pacific LatinAmerica

Africa

IXP Member ASes by region

https://www.euro-ix.net/tools/asn_search

Generally known: # IXPs ~ 350 worldwide

Somewhat known: # ASes per IXP up to 500

Less known: # ASes ~ 11,000 worldwide

IXPs – Publicly available information Generally known: # IXPs ~ 350 worldwide

Somewhat known: # ASes per IXP up to 500

Less known: # ASes ~ 11,000 worldwide

Even less known: IXPs =~ Tier-1 ISP traffic

AMS-IX

IXPs – Publicly available information Generally known: # IXPs ~ 350 worldwide

Somewhat known: # ASes per IXP up to 500

Less known: # ASes ~ 11,000 worldwide

Even less known: IXPs =~ Tier-1 ISP traffic

Unknown: # of peerings at IXPs

Peering links – 2012 estimates?

Methodology Number of peering links in the entire Internet

[Dhamdhere et al.] 2010 Lower bound estimate based on BGP data)

> 20,000

[Augustin et al., Chen et al.] 2009/2010 Targeted/opportunistic traceroute from network edge

> 40,000

Dasu 2011. Targeted data plane measurements

> 60,000

IXP peering link between pair of ASes if

IP traffic exchanged

• BGP traffic only (e.g., in case of backup links)

• IP otherwise

Potential links

Member ASes in Nov/Dec‟11: 396

396x395 / 2 = 78,210 P-P links possible

Observed links

> 50,000 peering links

Peering rate > 60%!

•

•

June‟12: 421

> 55,000 peering links!

Peering rate > 60%! > 60%!

Fact 4 – IXP peerings

Fact 4 – IXP peerings Internet-wide

Single IXP > 50,000 peering links

Derivation of new lower bound

10 large IXPs in Europe: ~160,000 peering links

Remaining 340 or so IXPs: ~ 40,000 peering links

Completely ignoring all other peerings

(Conservative) lower bound on #of peering links

> 200,000 peering links in today‟s Internet (as compared to currently assumed ~ 40,000 – 60,000)

Requires a revamping of the mental picture our community has about the AS-level Internet.

Fact 4 – IXP peerings Internet-wide

Methodology Number of peering links in the entire Internet

[Dhamdhere et al.] 2010 Lower bound estimate based on BGP data)

> 20,000

[Augustin et al., Chen et al.] 2009/2010 Targeted/opportunistic traceroute from network edge

> 40,000

Dasu 2011. Targeted data plane measurements

> 60,000

2012 (This talk) data from IXPs > 200,000

Summary

Large IXPs are ideal Internet vantage points

Internet monitoring is a „big data“ problem

Large IXP study reveals diverse eco-system wrt members, business types, connectivity, traffic, etc.

Large IXP supports rich peering fabric

Single IXP doubles the estimated number of peering links

Needs revamping of mental picture of AS-level Internet

Implications for studies of AS-level Internet

ASes – can no longer be treated as „homogeneous“

AS links – simple classification (peering, cust-prov) should fade

IXP peerings – when peering links are used as cust-prov links…

AS traffic – what traffic is carried by whom?