An introduction to the group and its projects

Tony McGregortonym@wand.net.nz

WANDProjects

• CRCNet• Active Measurement• IP Measurement protocol• Passive Measurement• Simulation• Integrated measurement and simulation• Emulation Network• Physical layer switch• IPv6

• topology, mobile stacks, fast handover

• NZNOG ‘04

CRCNetIntroduction

• Project started almost 2 years ago

• Rural communities were frustrated by low speed unreliable Internet access

• Develop a new platform suitable to deploy future generation (>>10Mbps) wireless networks in rural and remote areas

• based around a mesh architecture

• Funded by Foundation for Research Science and Technology

CRCNetArchitecture

CRCNetStage 1 – Build Trial Network

Range of equipment• 2.4Ghz (802.11b and g)

• Orinoco radio cards and APs• Advantech and Soekris Biscuit PC• Linksys wireless Ethernet bridges

• 5.8 GHz• Proxim Quick bridge20• Trango

Current Topology

CRCNetPirongia Site

CRCNetHSK Site

CRCNetMFR Site

CRCNetWeb Casting

• Between Hamilton Zoo and the Fieldays site

• 6 wireless links

CRCNetStage Two – Platform Design

• Routing protocols for mesh networks

• Link Layer Design

• Design of a new node

• NLANR’s active measurement project• Approx 140 monitors, mostly in the USA. • International deployments

• a single AMP monitor in about a dozen other countries• some national AMPs (Australia, Taiwan, Russia soon)

• Measure• RTT• loss• topology• throughput (on demand)

• NSF funded

AMPIntroduction

AMPUSA Sites

AMPArchitecture

(amp)

(volt)

ActiveMonitor

Othertarget

ActiveMonitor

ActiveMonitorAnalysis

machine

Analysismachine

Webbrowser

Test Results

Test Results

test traffic

Cichlid

AMPDemo

AMPDemo

AMPDemo

AMPDemo

AMPDemo

AMPDemo

AMPDemo

AMPDemo

• Design• dedicated machines• 1ms accuracy• No GPS/CDMA• 1 sample per minute

• Benefits• easy and cheap => wide deployment• full mesh• manageable

• Limits• no one-way delays (bidirectional traceroute, IPMP OWD)• very short events missed

AMPCost vs Function

AMPManagement

ALL

AMP

amp-palomar

HPWREN

mySQL databse

amp-kiwi

system manager

monitors

Volt

AMP

• Beginnings of a New Zealand AMP mesh• Waikato• Auckland• APE• Ihug (offer)

• Can fund more monitors and maintenance• need hosts (here?)• hosts provide space, power and network

AMPNew Zealand

• Current active measurement protocols have weaknesses

• multiple packets (overhead, phantom routes)• measurement of components (reverse path, CPU)

• IPMP combines path and delay measurement in a single packet exchange with low router overhead

IPMPIntroduction

IPMPArchitecture

IPMP

header header header header

Target Host

IPMP

IPMP

IPMP

MeasurementHost

IPMP Enabledrouter

Peering point

High perfomance ISP

Non-IPMP Router

Packet that leavesmeasurement host(one path record)

Path record addedat first IPMP

enabled router

no change

Packet as it leavesthe kernel on the

target host

Progression of packet through the network

IPMPProtocol (IPv4)

• Router can use any timestamp it has available

• Resolving to real-time is not done in the packet forwarding critical path

• Uses a separate packet exchange (information request/reply)

• supplies real-time reference points • other router information

IPMPTimestamps

IPMPInformation Reply

0 8 6 4 10 0 1 2 3

Version Checksum

00000000 Type

00000000

00000000 Precision

Length Performance Data Pointer

Forwarding IP Address

Accuracy

IPMP Processing Overhead

(optional) Path Records

(optional) performance data

• POM made better• combined path and latency, no phantom routes etc• lower overhead• kernel based timestamps• explicit clock information• forward and reverse traceroute• DoS resistant• associates router interfaces

• One way delay from NTP• Bandwidth Estimation• Deployment (AMP, CRCnet)

IPMPUses

• To support simulation work the group developed passive header capture hardware.

• Known as Dag cards• Speeds from Ethernet to OC48 (2.5Gbps WAN)• Spun off a startup

• Endace (www.endace.com)• now OC192• better support

Passive MeasurementOverview

• Capture IP headers or full packet• Add accurate timestamp

• GPS or CDMA for external time

• Originally header trace focused• real-time flow based• security applications

• Optical splitter, electrical card relay or electrical tap

Passive MeasurementDag Overview

Passive MeasurementDag 3 block diagram

PassiveDag 4.2

PassiveWITS Traffic Archive

• Long traces from Auckland University and NZIX

•traces up to 45 days (3.2 billion packets)

•IP headers

•GPS timestamps

• Some analysis online

• Can fetch traces from NLANR

• Summary CD

• ATM-TN based• University of Calgary/Waikato partnership• parallel• BSDLite network stack (sort of)• high bandwidth delay, mixed real-time/TCP

• NS-2 with FreeBSD stack• new work• network cradle

• 802.11b link layer

SimulationIntroduction

SimulationExample –TCP splitting

Web Clients

US ProxyNZ Internet

NZ Proxy

US Internet

US Servers

international channel

SimulationThe simulation process

SimulatePre-process Post-process

HTTPLog Logfile

generator

HostList

Hostfilegenerator

tracePacket

Graphs

livehosts

DigestedLogfile

Host Information

query onhost

buffer andMSS info

Internet

SimulationParameters

SimulatorHTTP PageLatencies

Line and Buffer use

SummariseandPlot

• Bandwidth 34.369Mbps (E3)• Delay 60ms• TCP buffer size

• proxy 32767 bytes• servers as measured

• MSS as measured• US delay as measured• NZ delay not simulated

SimulationExample –TCP spliting, Network parameters

SimulationTCP Splitting – a single connection

SimulationIntroduction

SimulationIntroduction

• Simulation is only accessible to very large network operators and users

• AIM: Make simulation available to medium sized enterprises

• Integrate measurement and simulation • FRST funded

MessimIntroduction

MessimIntroduction

Monitored Network

topology discovery

measurement

simulator

validation

and analysis

ModelWorkload Network

Model

workload query resultsalerts and

• Topology discovery• automated discovery of link layer devices

• Traffic Models• further development of specific models (e.g. peer to

peer)• generic

• Extraction of simulation parameters from traces• Extended range of network stack models• Continuous validation• Hardware flows analysis

MessimProjects

Network stack

FreeBSD 5 kernel

Mozilla / Bash / KDE / etc.

Ker

nel s

pace

Use

r sp

ace

MessimNetwork Stack Cradle

MessimNetwork Stack Cradle

Network stackUse

r sp

ace

Cradle (~200 functions)

Network

Simulator

• 2d Empirical distribution

MessimGeneric models

MessimGeneric models

MessimGeneric models

• Use WEKA machine learning algorithms to• cluster• classify

• For each cluster• simplify the rule set into terms for a network

manager• produce an empirical distribution for each

• Allow simulations with different proportions of traffic

MessimGeneric models

• There is a need for a structured environment in which to build networks in the laboratory

• validation of simulations• testing on network equipment

• The emulation network is two racks of PCs that can be configured as

• routers• end hosts• delay

• Plus configuration and measurement support

Emulation NetworkIntroduction

Emulation NetworkOverview

SWIxia

PCPCPCPCPCPCPCPCPCPCRR

PC

Patch

Dag

Configure

Panel

H

H

H

RR

R

RRdelay

Configure

(DAG)Monitor

• Usage• Is a public facility• Has been used to debug AT switch Used network trace capture and replay then Ixia

script• Ihug traffic shaper• Bandwidth estimator

• Development• Physical layer switch

Emulation NetworkUsage and development

• 64 Port FastEthernet Crossbar switch• Fast / Flexible Reconfiguration• Link Monitoring• Latency Control

• Bandwidth limiting• Self Documenting Network Topology• Centralised Control

Crossbar SwitchIntroduction

Crossbar SwitchBlock Diagram – Overview

Mainboard12.8Gb/s

Uplink

DaughterBoard

DaughterBoard

DaughterBoard

DaughterBoard3.2Gb/s

Mainboard• Crossbar• Latency

• Bandwidth Limiting

Daughterboards• Ethernet Interface

• Time Division MUX

Crossbar SwitchBlock Diagram –Mother board

CPU

FPGA12.8Gb/s

UplinkDaughterBoard

DaughterBoard

DaughterBoard

DaughterBoard

DDR SDRAM(8GB max).

SDRAM

FLASH

Crossbar SwitchBlock Diagram – Daughter board

PHY

FPGA

PHY

3.2Gb/s

Eth

ern

et

Port

sEth

ern

et

Port

s

Uplink to Motherboard

Daughterboard Layout

Crossbar SwitchDaughter board Layout

• Skitter for IPv6• Hope to capture the growth of the IPv6 internet

SkamperOverview

• Small devices• One of the motivators for IPv6 is to provide addresses and

other support for small devices• a.k.a. cell phones• implementing a stack for embedded devices• little ram• moderate CPU speeds• prototype hardware development

• Fast handover between cells• normally may exceed 2s• reduce to around 150ms, l2 triggers, L3 preparation for

handover and timing improvements in protocols

IPv6 StacksOverview

• The New Zealand Network Operators Group has an annual conference

• The next one will be hosted by WAND• Jan 29-30 2004, at Waikato• Discounted registration (free?) for students• Hope to have a number of partial travel grants

for students• Could hold a parallel Academic Networking

Conference• need feedback

NZNOGConference