Interoperable Measurement Frameworks:

Joint Monitoring of GEANT & Abilene

Eric L. Boyd, Internet2Nicolas Simar, DANTE

2

Overview

Internet2 E2E piPEs Geant2 Joint Research Activity 1 Joint Monitoring of GEANT & Abilene Measurement Domain and Framework

Interoperability and Collaboration

3

Internet2 E2E piPEs Overview

What is piPEs? Goals E2E piPEs Measurement

Infrastructure Abilene Measurement Domain

4

Internet2 E2E piPEs

Project: End-to-End Performance Initiative Performance Environment System (E2E piPEs)

Approach: Collaborative project combining the best work of many organizations, including DANTE/GEANT, EGEE, GGF NMWG, NLANR/DAST, UCL, Georgia Tech, etc.

5

Internet2 E2E piPEs Goals Enable end-users & network operators to:

• determine E2E performance capabilities• locate E2E problems• contact the right person to get an E2E problem

resolved. Enable remote initiation of partial path

performance tests Make partial path performance data publicly

available Interoperable with other performance

measurement frameworks

6

Measurement Infrastructure Components

Database ofPerformance

Results

Server Server

Regularly Scheduled Tests

Router Router

On-Demand Tests

Laptop computer

TestResults Test

Results

TestResults

TestRequest

ResultRequest

End-to-End Path

7

Sample piPEs Deployment

RegionalNode

RegionalNode

BackboneNode

End Node End Node

BackboneNode

BackboneNode

Network Backbone

NetworkBackboneTest Data

End Node

RegionalTest Data

ApplicationDomain Test

Data

Regularly Scheduled TestsOn-Demand TestsResult Collection

8

piPEs Deployment

1) Abilene Backbone Deployment (Complete)

Hawaii

2) Hawaii Campus Deployment (Complete)

OSU

NC State

Europe

UCSD

3) In Progress Campus and European Deployment (Q1 2004)

9

Measurement Software Components

PerformanceMeasurement

Controller (PMC)

MeasurementDomain Interface

(MDI)

Performance Measurement Point (PMP)

Web Service

NetworkMonitoring

Database

Detect

Authorize

Schedule

Test

Store

Interface

BWCTL OWAMP TraceRoute NDT

DiscoveryModule

“Detective”Applet

Internet2 Detective

AnalysisModule

Performance MeasurementDomain (PMD)

10

Abilene Measurement Domain Part of the Abilene Observatory:

http://abilene.internet2.edu/observatory Regularly scheduled OWAMP (1-way latency) and

BWCTL (Iperf wrapper) Tests Web pages displaying:

• Latest results http://abilene.internet2.edu/ami/bwctl_status.cgi/TCP/now “Weathermap” http://abilene.internet2.edu/ami/bwctl_status_map.cgi/TCP/now

• Worst 10 Performing Links http://abilene.internet2.edu/ami/bwctl_worst_case.cgi/TCP/now

Data available via web service:http://abilene.internet2.edu/ami/webservices.html

11

Data Collection / Correlation Collection Today:

• Iperf (Throughput)• OWAMP (1-Way Latency,

Loss)• SNMP Data• Anonymized Netflow Data• Per Sender, Per Receiver,

Per Node Pair• IPv4 and IPv6

Collection in the Future• NTP (Data)• Traceroute• BGP Data• First Mile Analysis

Correlation Today:• “Worst 10” Throughputs• “Worst 10” Latencies

Correlation in the Future:• 99th Percentile Throughput

over Time• Throughput/Loss for all E2E

paths using a specific link• Commonalities among first

mile analyzers• Sum of Partial Paths vs.

Whole Path

12

Data Analysis Analysis Today:

• Throughput over Time• Latency over Time• Loss over Time• Worrisome Tests? (Any

bad apples in “Worst Ten”?)

• “Not the Network” (If “Worst Ten” is good enough)

Analysis in the Future:• Latency vs. Loss• How good is the

network?• Do common first mile

problems exist?• Does a link have

problems that only manifest in the long-haul?

• Is the network delivering the performance required by a funded project?

13

Data Discovery / Interoperability

Discovery in the Future:• Where are the

measurement nodes corresponding to a specific node?

• Where are the test results for a specific partial path?

Interoperability in the Future:• Can I have a test

within or to another measurement framework?

• Can I have a measurement result from within or to another measurement framework?

14

Overview

Internet2 E2E piPEs Geant2 Joint Research Activity 1 Joint Monitoring of GEANT & Abilene Measurement Domain and Framework

Interoperability and Collaboration

15

GN2 JRA1 - Performance Monitoring and Management

3 year project, starting in September 2004 (15 European NRENs and DANTE involved).

Development of a Performance Monitoring infrastructure operating across multiple domains

User can access data from different domains in a uniform way and start on-demand tests.

16

Goals Make network management and performance

information from different domains available to various authorised user communities• GÉANT, NRENs, Regional networks NOCs• PERT - Performance Enhancement Response

Team• high data volume transfer users (as GRID)• end-user who would like to see or understand the

E2E behaviour of R&D networks.

17

Goals

Multi-domain focus, interoperable with other frameworks.

Tailor data representation for a subset of users.

Modify existing tools and integrate them within the infrastructure.

18

GN2 Framework Layers

Domain A Domain B Domain C

MeasurementPoints

User Interface

DomainControllers

Domain Controller -domain A

Domain Controller -domain B

Domain Controller -domain C

Available BandwidthMeasurement Points

User Interface 1 User Interface 2

One-way delayMeasurement Points

type 2

One-way delayMeasurement Points

type 1

19

Activity will focus in five areas:• One-way delay, IPDV and traceroute• Available Bandwidth (IP for sure, TCP/UDP

less sure)• Flow Based Traffic Measurement• Passive Monitoring• Network Equipment information

Quite IP-centric.

Measurement Points and Metric

20

Domain Controller

Ensure that different monitoring agents deployed in the various domains can inter-operate and be queried in the same way by User Interface instances independently of their localisation.

High level functionality: provide the user interface, AA, Resource discovery (pathfinder), negotiate test, interface with other domain/framework

21

User Interface

A User Interface retrieves data from different domains and tailors the data representation to a specific group of user.

Targets NOC, PERT and generic end-user.• Topology based view• SLA verification

22

Starting Point

Starts from GN1 Performance Monitoring framework (data retrieval, storage and export using a well define interface) and take into account other experiences.

Uses NRENs experience in tool development. Need to take into account the variety of tools

and metric existing across the NRENs

23

Overview

Internet2 E2E piPEs Geant2 Joint Research Activity 1 Joint Monitoring of GEANT & Abilene Measurement Domain and Framework

Interoperability and Collaboration

24

American/European Collaboration Goals

Awareness of ongoing Measurement Framework Efforts / Sharing of Ideas (Good / Not Sufficient)

Interoperable Measurement Frameworks (Minimum)• Common means of data extraction• Partial path analysis possible along transatlantic paths

Open Source Shared Development (Possibility, In Whole or In Part)

End-to-end partial path analysis for transatlantic research communities• VLBI: Onsala, Sweden Haystack, Mass.• HENP: CERN, Switzerland Caltech, Calif.

25

American/European Demonstration Goals

Demonstrate ability to do partial path analysis between “Caltech” (Los Angeles Abilene router) and CERN.

Demonstrate ability to do partial path analysis involving nodes in the GEANT network.

Compare and contrast measurement of a “lightpath” versus a normal IP path.

Demonstrate interoperability of piPEs and analysis tools such as Advisor and MonALISA

26

Demonstration Details Path 1: Default route between LA and CERN is

across Abilene to Chicago, then across Datatag circuit to CERN

Path 2: Announced addresses so that route between LA and CERN traverses GEANT via London node

Path 3: “Lightpath” (discussed earlier by Rick Summerhill)

Each measurement “node” consists of a BWCTL box and an OWAMP box “next to” the router.

27

All Roads Lead to Geneva

Path 1 — DataTag — Default RoutePath 2 — Eurolink — "Cooked” Alternate RoutePath 3 — Lightpath — "Cooked” Alternate RouteCircles Correspond to OWAMP / BWCTL Measurement Node Pair

28

Results

BWCTL: http://abilene.internet2.edu/ami/bwctl_status_eu.cgi/BW/now

OWAMP: http://abilene.internet2.edu/ami/owamp_status_eu.cgi/now

MONALISA: http://vinci.cacr.caltech.edu:8080

NLANR Advisor

29

Insights (1) Even with shared source and a single team of

developer-installers, inter-administrative domain coordination is difficult.• Struggled with basics of multiple paths.

IP addresses, host configuration, software (support source addresses, etc.)

• Struggled with cross-domain administrative coordination issues.

AA (accounts), routes, port filters, MTUs, etc.• Struggled with performance tuning measurement

nodes. host tuning, asymmetric routing, MTUs

30

Insights (2)

Connectivity takes a large amount of coordination and effort; performance takes even more of the same.

Current measurement approaches have limited visibility into “lightpaths.”• Having hosts participate in the

measurement is one possible solution.

31

Insights (3)

Consider interaction with security; lack of end-to-end transparency is problematic.• Security filters are set up based on

expected traffic patterns• Measurement nodes create new traffic• Lightpaths bypass expected ingress points

32

Next Steps for Internet2 E2E, GN2 JRA1, and EGEE

To be decided … Transatlantic Performance Monitoring

Workshop, CERN, May 17, 2004• Sharing of ideas• Determining areas of overlap• Determining degree of collaboration

Construct trial OWAMP and BWCTL measurement domain at GEANT nodes

33

Overview

Internet2 E2E piPEs Geant2 Joint Research Activity 1 Joint Monitoring of GEANT & Abilene Measurement Domain and Framework

Interoperability and Collaboration

34

Measurement Infrastructure Federation

Why a Federation?• Multiple measurement frameworks in existence

and under development (piPEs, NLANR Advisor, NLANR AMP, etc.).

• No static “best practice” measurement framework is likely to emerge, given academics being academics.

• Future measurement frameworks can build on shoulders of current efforts, not feet.

Performance Measurement Architecture Workshop (NSF Grant # ANI-0314723)

35

Measurement Infrastructure Federation Interfaces

Measurement FrameworkOther

MeasurementFramework

Discovery Access /Authentication

Data/TestRequest

ResultResponse

AnalysisTools

NOC AlarmPrograms

Resource Allocation Broker

NetworkMeasurement Tools

Inter-Framework

Tests

36

Measurement Infrastructure Federation Requirements

Agreement on Characteristic Names Access and Authentication Discovery (Measurement Frameworks, Domains,

Nodes, Databases) Test/Data Request Schema Result Report Schema Inter-Framework Tests Resource Allocation Broker for Tools Concatenation of Homogeneous Characteristics

Results Gathered by Heterogeneous Tools

37

GGF Network Measurement Working Group

Hierarchy of Network Performance Characteristics

Request Schema Requirements and Sample Implementation

Report Schema Requirements and Sample Implementation

38

Establishing a Performance Measurement Mesh

Issues include:• Scheduling in the presence of scarce

resources• Making the tool bidirectional• Adding security• Ensuring correct source/target pairs• Data collection / mining / analysis / displayExample:• BWCTL for Iperf plus prototype PMD

39

Open Research Issues

Access and Authentication Discovery of Measurement Nodes

(“Super-Traceroute”) Discovery of Measurement Databases Inter-framework Testing Compilation of results on partial paths Normalization of identical characteristics

gathered by heterogenous tools

40

Conclusions We can do partial path analysis, although

making sense of the results is still a big issue. We can speak the same measurement

language, although it’s still evolving. We are working together in growing numbers,

but we need critical mass (become de facto standard).

We need to be able to find each other. We need to be able to verify each other’s

identity.

41

Feedback

Are we on the right track? (As conceptualized, would our individual and joint goals meet the needs of the DataTag community?)

What’s missing? What is of particular importance?