ph.d. defense

Experimentation Toolsfor Networking

ResearchMathieu Lacage

INRIA, Planète

Nov 15th 2010

Lacage (INRIA) Experimentation Tools & Network Research Nov 2010 1 / 55

Outline

Motivation

Simulated Packets

Direct Code Execution

Conclusion

Perspectives

Protocol evaluation

Analyticalanalysis

Protocol evaluation

Analyticalanalysis

Simulator

Protocol evaluation

Analyticalanalysis

Simulator Testbed

Protocol evaluation

Analyticalanalysis

Simulator Testbed

Smallscalefield

Protocol evaluation

Analyticalanalysis

Simulator Testbed

Smallscalefield

Largescalefield

Protocol evaluation

Analyticalanalysis

In thewild

Simulator Testbed

Smallscalefield

Largescalefield

Protocol evaluation

Analyticalanalysis

In thewild

Simulator Testbed

Smallscalefield

Largescalefield

Experimentation contextExperimentation

Realism

I One protocol isolated

-> Multiple protocolinteraction

I Steady state -> Transient behavior

I One protocol isolated -> Multiple protocolinteraction

I Steady state

-> Transient behavior

Cheaper hardwareHardware

ConsequencesMore testbeds

More Field tests

ConsequencesMore testbeds

More Field testsLess

SimulationsAlone

ConsequencesMore Testbed+

Simulation

Downsides

A lot more workI Must master many experimentation

environmentsI Must implement protocols twice

Downsides

A lot more work

I Must master many experimentationenvironments

I Must implement protocols twice

Downsides

environments

I Must implement protocols twice

Downsides

environmentsI Must implement protocols twice

Objectives

I Run real protocol implementation in simulationI Use simulation as realtime emulator

ObjectivesSmooth transition simulations/testbed/field tests

FieldSimulation

I Run real protocol implementation in simulation

I Use simulation as realtime emulator

RealCode

SimulationSimulation

RealCode

Simulation FieldSimulation

Problems

Run real protocol implementation in simulation

I Virtualization of execution environment

I Direct Code Execution

I Setup, deployment, of emulation platforms

I NEPI

I Transparency with other simulation models

I Simulated Packets

Problems

Run real protocol implementation in simulationI Virtualization of execution environment

Real Code

Simulation

I NEPI

I Simulated Packets

Problems

Use simulation as realtime emulator

I NEPI

I Simulated Packets

Problems

Use simulation as realtime emulatorI Setup, deployment, of emulation platforms

I NEPI

FieldField

DeploymentControl

I Simulated Packets

Problems

I NEPI

In both cases

I Simulated Packets

Problems

I NEPI

In both casesI Transparency with other simulation models

I Simulated Packets

Problems

I Direct Code ExecutionUse simulation as realtime emulator

I NEPI

I Simulated Packets

Problems

I Setup, deployment, of emulation platformsI NEPI

I Simulated Packets

Problems

I Setup, deployment, of emulation platformsI NEPI

I Simulated Packets

ns-3 Contributions

I ns-3: 300 KLOCI Wrote about 80 KLOC

ns-3 Contributions

I ns-3: 300 KLOC

I Wrote about 80 KLOC

UDP/Ipv6

AODV OLSR

Uan Bridge

Spectrum

Csma PointToPoint

UDP/IPv4/ARP

Mobility

ns-3 Contributions

I ns-3: 300 KLOCI Wrote about 80 KLOC

UDP/Ipv6

AODV OLSR

Uan Bridge

Spectrum

Csma PointToPoint

UDP/IPv4/ARP

Mobility

Scientific Contributions

Simulated packetsI More CPU efficient than other simulatorsI Automatic conversion simulation/network format

Direct Code ExecutionI 10x more CPU efficient than other DCE

frameworksI Large applicability scope

NEPII Unified experiment descriptionI Automated deployment

Scientific ContributionsSimulated packets

I More CPU efficient than other simulatorsI Automatic conversion simulation/network format

I More CPU efficient than other simulators

I Automatic conversion simulation/network formatDirect Code Execution

I 10x more CPU efficient than other DCEframeworks

I Large applicability scopeNEPI

I Unified experiment descriptionI Automated deployment

I 10x more CPU efficient than other DCEframeworks

frameworks

NEPII Unified experiment description

I Automated deployment

Outline

Motivation

Simulated Packets

Conclusion

Perspectives

Requirements

I Transparent conversion to/from real bytesI CPU and memory efficiencyI Fragmentation/ReassemblyI Simulation-only dataI Pretty printingI ExtensibilityI Robust Application Programming Interface (API)

Related Work

Two approaches

I Packet is list of headers: GTNetS, OMNeT++,SSFNet

TCPIP PayloadMAC

I Packet is buffer of bytes: Yans, GloMoSim

TCPIP PayloadMAC

Related Work

Two approachesI Packet is list of headers: GTNetS, OMNeT++,

SSFNet

TCPIP PayloadMAC

Related Work

Two approachesI Packet is list of headers: GTNetS, OMNeT++,

SSFNet

TCPIP PayloadMAC

Pros and Cons

List BufferFragmentation, Reassembly XConversion real bytes XSimulation-only data XPretty printing XCPU, memory efficiency X

Our solutionPacket as buffer of bytes

I Fragmentation, ReassemblyI Automatic conversion to/from real bytes

TagsI Simulation-only data

MetadataI Pretty printing

Copy On Write (COW)I CPU, memory efficiency

Performance evaluation

Micro-benchmark scenariosI ReceptionI ForwardingI TransmissionI Re-transmission

Relative speedup

ns-3 OMNet++ GTNetS YansReception 1.00 1.98 2.25 3.07Transmission 1.00 2.13 1.78 2.16Forwarding 1.00 2.83 1.92 2.44Retransmission 1.00 1.92 1.70 4.76

Contributions

More CPU efficient than other simulators

Transparent support for real network bytes

Contributions

Outline

Motivation

Simulated Packets

Conclusion

Perspectives

The manual approachGlobal variables

static int g_var;g_var ++;

static int [100]g_var_array;g_var_array[current_id ()] ++;

Redirect system calls

clock (); dce_clock ();

Re-implement all system calls

clock_t dce_clock (void) {return Simulator::Now ().GetMicroSeconds ();

The problem

Manual modifications: does not scaleI Painful to do onceI Impossible to do for software updates

Related work

Network Simulation CradleI Automated source modifications for C codeI Hard to extend to C++

WeavesI Automated textual assembly modificationsI Invalid assumptions about compiler-generated

codeCOOJA

I Automated memory virtualizationI Slow

Related workNetwork Simulation Cradle

I Automated source modifications for C codeI Hard to extend to C++

codeCOOJA

COOJAI Automated memory virtualizationI Slow

codeCOOJA

ns-3 DCE

Executable and Linkable Format (ELF) loaderI FastI Automated memory virtualizationI Automated system call redirection

Userspace system callsKernelspace system calls

ns-3 DCE

Userspace system calls

Kernelspace system calls

ns-3 DCE

Userspace system callsKernelspace system calls

Loader Performance

Scenario:

UDP/IP

PointToPointLink

UDP/IP

PointToPointLink

kernelspace DCE kernelspace DCE

UDP/IP

kernelspace DCE

udp-perf

userspace DCE

udp-perf

userspace DCE

Loader Performance

Packets perwall clocksecondvsNumber ofnodes

100000

0 10 20 30 40 50 60 70

Coojans-3

Loader Performance

Memory(bytes)vsNumber ofnodes

0 10 20 30 40 50 60 70

Coojans-3

System Performance

Packets perwall clocksecondvsNumber ofnodes

100000

0 2 4 6 8 10 12 14 16 18 20

dce-nonedce-user

dce-user+kernel

Tested programs

Binary Missing functions/sbin/ip 24/bin/ping 2/bin/traceroute 5/usr/sbin/zebra 42/usr/sbin/ospfd 43/usr/sbin/mip6d 38

Untested programs

Binary Missing functionsccnd 11/usr/sbin/httpd 18

Contributions

10x more efficient than existing alternatives

Larger scope than existing alternativesI Userspace: ping, traceroute, quagga, etc.I Kernelspace: Linux IP, TCP, etc.

Potential usecases:I Debugging platform: single debugger controls

all protocol instancesI Development platformI Test platform

Contributions

10x more efficient than existing alternativesLarger scope than existing alternatives

I Userspace: ping, traceroute, quagga, etc.I Kernelspace: Linux IP, TCP, etc.

Contributions

all protocol instances

I Development platformI Test platform

Contributions

all protocol instancesI Development platform

I Test platform

Contributions

Outline

Motivation

Simulated Packets

Conclusion

Perspectives

Objective Scenario

Simulated links/networksI ns-3 simulation models XI Realtime scheduler XI Tap device X

Light weight Virtual MachinesI Linux Network Namespaces X

Objective Scenario

VLCServer

RouterWifiAP

WifiSTA

Router

VLCClient

Objective Scenario

VLCServer

RouterWifiAP

WifiSTA

Router

VLCClient

Objective Scenario

VLCServer

RouterWifiAP

WifiSTA

Router

VLCClient

Easy Deployment

ProblemI Tap/VM creation and setupI Coherent IP address assignment across

simulation and VMsI Coherent IP forwarding tables across

simulation and VMsSolution

I Automate everythingBUT

I Need global view of experiment topology

Easy DeploymentProblem

I Tap/VM creation and setup

I Coherent IP address assignment acrosssimulation and VMs

I Coherent IP forwarding tables acrosssimulation and VMs

SolutionI Automate everything

BUTI Need global view of experiment topology

I Tap/VM creation and setupI Coherent IP address assignment across

simulation and VMs

I Coherent IP forwarding tables acrosssimulation and VMs

simulation and VMs

I Automate everything

I Automate everythingBUT

I Need global view of experiment topology

Related Work

I Emulab Tcl: ad hoc, hard to generalizeI OMF Ruby: unclear how to extend it to model

complex topologiesI OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness

Related Work

I Emulab Tcl: ad hoc, hard to generalize

I OMF Ruby: unclear how to extend it to modelcomplex topologies

I OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness

Related Work

complex topologies

I OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness

Related Work

complex topologiesI OMNeT++ NED: hard to ensure correctness

I SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness

Related Work

complex topologiesI OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctness

I Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness

Related Work

complex topologiesI OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctness

I Geni Omnispec: hard to ensure correctness

Related Work

complex topologiesI OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness

NEPI Object Model

Functional unit / BoxAttributesTrace sourcesConnectorsConnection checkingHierarchical

NEPI Object Model

Functional unit / Box

AttributesTrace sourcesConnectorsConnection checkingHierarchical

Example:I IP stackI TCP stackI Ethernet card

NEPI Object Model

Functional unit / BoxAttributes

Trace sourcesConnectorsConnection checkingHierarchical

Example:I IP checksumI IP forwarding

NEPI Object Model

Functional unit / BoxAttributesTrace sources

ConnectorsConnection checkingHierarchical

Example:I Out packetsI In packets

NEPI Object Model

Functional unit / BoxAttributesTrace sourcesConnectors

Connection checkingHierarchical

Ethernet

NEPI Object Model

Functional unit / BoxAttributesTrace sourcesConnectorsConnection checking

Hierarchical

Ethernet

node node

NEPI Object Model

Functional unit / BoxAttributesTrace sourcesConnectorsConnection checkingHierarchical

Ethernet

Switch

Ethernet

node node

cablecable

port0 port1

Objective Scenario

VLCServer

RouterWifiAP

WifiSTA

Router

VLCClient

NEPI Representation

Mobility

WifiNet

Device

ARPICMPIP

TapNodeInterface

FDNetDevice

Mobility

FDNetDevice

YansWifiPhyYansWifiPhy

YansWifiChannel

Nist Nist

WifiNet

Device

Manager

ARPIP ICMP

Manager

StaMac

TapNodeInterface

ns-3 NetNsNetNs

VlcVlc

Global IP topology

net2net1net0 Wifi Ap Wifi Sta FdFdTap Tap

n1n0 n2 n3

Demo screenshot

Contributions

An integrated experimentation environmentI Unified experiment descriptionI Entire workflow supportI Automated deploymentI Strong coherency checking

Outline

Motivation

Simulated Packets

Conclusion

Perspectives

Papers Published

I “Yet another network simulator”, Proceedings ofthe 2006 workshop on ns-2

I “NEPI: using independent simulators,emulators, and testbeds for easyexperimentation”, SIGOPS Operating

Systems Review

Papers Published

Systems Review

Papers Published

Systems Review

Papers in progress

I “The ns-3 network simulator: experiencelearned”, Software Practice & Experience

I “Direct Code Execution”, Networked SystemsDesign and Implementation

Papers in progress

I “The ns-3 network simulator: experiencelearned”, Software Practice & Experience

I “Direct Code Execution”, Networked SystemsDesign and Implementation

Activities

ChairI NSTools 2007

TPCI WNS3 2009, 2010, 2011I Simutools 2008,2009

Simulated Packets

10x more CPU efficient than other DCE frameworks

A robust implementationI Userspace and kernelspace protocolsI C,C++ protocolsI ABI compatibility for userspace protocols

10x more CPU efficient than other DCE frameworks

A robust implementationI Userspace and kernelspace protocolsI C,C++ protocolsI ABI compatibility for userspace protocols

An integrated experimentation environmentI Unified experiment descriptionI Entire workflow supportI Automated deploymentI Strong coherency checking

Impact

Instrumental in creating anI ActiveI GrowingI Open source

Simulation communityI Contributors > 80I Hundreds of usersI > 30 papers by users

2 sigcomm’091 sigcomm’10

ImpactInstrumental in creating an

I ActiveI GrowingI Open source

Simulation community

I Contributors > 80I Hundreds of usersI > 30 papers by users

Simulation communityI Contributors > 80

I Hundreds of usersI > 30 papers by users

Simulation communityI Contributors > 80I Hundreds of users

I > 30 papers by users2 sigcomm’091 sigcomm’10

Simulation communityI Contributors > 80I Hundreds of usersI > 30 papers by users

Outline

Motivation

Simulated Packets

Conclusion

Perspectives

Tracing with DCE

I Need to recompile for tracing changesI Ideal workflow:

I Trace function foo in bar.ccI Trace line 122 in bar.ccI Trace variable foo at line 144 in bar.cc

I BUT, not Java: no introspection

Tracing with DCE

I Need to recompile for tracing changes

I Ideal workflow:I Trace function foo in bar.ccI Trace line 122 in bar.ccI Trace variable foo at line 144 in bar.cc

Tracing with DCE

I Trace function foo in bar.cc

I Trace line 122 in bar.ccI Trace variable foo at line 144 in bar.cc

Tracing with DCE

I Trace function foo in bar.ccI Trace line 122 in bar.cc

I Trace variable foo at line 144 in bar.cc

Tracing with DCE

Dynamic instrumentation

I Locate code in memoryI Parse dynamic loader data structuresXI Parse debugging information

I Insert assembly hooksI Simple case (not thread-safe) XI General case harder

I Locate code in memory

I Parse dynamic loader data structuresXI Parse debugging information

I Locate code in memoryI Parse dynamic loader data structuresX

I Parse debugging informationI Insert assembly hooks

I Simple case (not thread-safe) XI General case harder

I Insert assembly hooks

I Simple case (not thread-safe) XI General case harder

I Insert assembly hooksI Simple case (not thread-safe) X

I General case harder

Multi-threaded Scheduler

Conservative algorithm implemented:I Fully transparent for usersI Measurable speedupI Thread-safety is easyI Efficient thread-safety is hardI Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?

Transparency is key !

Conservative algorithm implemented:

I Fully transparent for usersI Measurable speedupI Thread-safety is easyI Efficient thread-safety is hardI Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?

Conservative algorithm implemented:I Fully transparent for users

I Measurable speedupI Thread-safety is easyI Efficient thread-safety is hardI Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?

Conservative algorithm implemented:I Fully transparent for usersI Measurable speedup

I Thread-safety is easyI Efficient thread-safety is hardI Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?

Conservative algorithm implemented:I Fully transparent for usersI Measurable speedupI Thread-safety is easy

I Efficient thread-safety is hardI Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?

Conservative algorithm implemented:I Fully transparent for usersI Measurable speedupI Thread-safety is easyI Efficient thread-safety is hard

I Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?

Conservative algorithm implemented:I Fully transparent for usersI Measurable speedupI Thread-safety is easyI Efficient thread-safety is hardI Can we build more efficient thread-safety ?

I Can we use optimistic algorithm transparently ?Transparency is key !

Thank you !

Questions ?

ph.d. defense

users2 sigcomm091

existing alternativeslarger

reassemblyi

redirect system

vmssolutioni

generalizei

sigops operatingsystems

experimentation

Business

ph.d. thesis defense june...

aslay ph.d. defense

ph.d. thesis defense june 2005 - mit media...

ph.d. defense -...

wuletawu abera ph.d. defense

ph.d. defense: models and algorithms for pagerank...

reflective database access control lars olson ph.d. thesis...

ph.d. thesis oral defense - stanford nlp...

ph.d. thesis oral defense - stanford nlp...

ph.d. defense presentation -...

freni - ph.d. defense slides

ariu - ph.d. defense slides

ph.d. defense: october 10, 2013

jennifer beth sass, ph.d. natural resources defense ... ·...

4/30/10 ph.d defense

hierarchical component models a true story (ph.d. thesis...

ph.d. thesis oral defense - stanford nlp group · 2004 —...

ph.d. thesis defense june...

rachid fahmi ph.d. defense april 30, 2008

zahid akhtar - ph.d. defense slides