Mesh NetworkingMesh Networkinghttp://research.microsoft.com/meshhttp://research.microsoft.com/mesh

Victor BahlVictor BahlSenior ResearcherSenior ResearcherSystems and Networking GroupSystems and Networking GroupMicrosoft ResearchMicrosoft Research

Team IntroductionTeam Introduction

Executive SponsorExecutive SponsorCraig Mundie, CTO & Senior Vice President MicrosoftCraig Mundie, CTO & Senior Vice President Microsoft

Microsoft ResearchMicrosoft ResearchVictor Bahl (Project Lead), Richard Draves, Jitu Padhye, Victor Bahl (Project Lead), Richard Draves, Jitu Padhye, Lili Qiu, Alec Wolman, Brian ZillLili Qiu, Alec Wolman, Brian Zill

The Venice IncubationThe Venice IncubationJeff Erwin (Project Lead), Pierre De Vries, Ian Ferrell, Jeff Erwin (Project Lead), Pierre De Vries, Ian Ferrell, Jason Ginchereau, Steve Kelly, Alexander Popoff, KarenJason Ginchereau, Steve Kelly, Alexander Popoff, Karen

Community NetworkCommunity Network

Community Network Community Network ApplicationsApplications

Internet use increased social contact, public participation and Internet use increased social contact, public participation and size of social network. (social capitalsize of social network. (social capital - access to people, - access to people, information and resources)information and resources)

Keith N. Hampton, MIT (author of “Netville Keith N. Hampton, MIT (author of “Netville Neighborhood Study”)Neighborhood Study”)

URL:URL: http://www.asanet.org/media/neville.htmlhttp://www.asanet.org/media/neville.html

Shared Broadband Internet AccessShared Broadband Internet AccessNeighborhood watchdog (e.g. video Neighborhood watchdog (e.g. video surveillance)surveillance)Neighborhood TiVONeighborhood TiVOMedical & emergency responseMedical & emergency responseDistributed backupDistributed backupNeighborhood eBay, portalsNeighborhood eBay, portals

Bits produced locally, gets used locallyBits produced locally, gets used locallySocial interactionSocial interaction

Mesh Formation: Problem Mesh Formation: Problem FormulationFormulationQuestionQuestion

How many homes in How many homes in the neighborhood have the neighborhood have to sign up before a to sign up before a viable mesh forms?viable mesh forms?

Answer depends onAnswer depends onDefinition of “viable”Definition of “viable”Wireless rangeWireless rangeNeighborhood topologyNeighborhood topologyProbability of Probability of participation by a given participation by a given housholdhoushold

Example ScenarioExample ScenarioViable mesh:Viable mesh: group of at least group of at least

25 houses that form a 25 houses that form a connected graphconnected graph

TopologyTopology: A North Seattle : A North Seattle Neighborhood. 8214 Neighborhood. 8214 houses, 4Km x 4Kmhouses, 4Km x 4Km

Wireless rangeWireless range: 50, 100, 200 : 50, 100, 200 and 1000 metersand 1000 meters

Houses decide to Houses decide to join at join at randomrandom, independent of , independent of each other. We consider each other. We consider 0.1% to 10% participation 0.1% to 10% participation rates.rates.

Mesh FormationMesh Formation

Increasing range is key for good mesh connectivity

5-10% subscription rate 5-10% subscription rate needed for suburban needed for suburban topologies with topologies with documented wireless documented wireless rangesranges

Once a mesh forms, it is Once a mesh forms, it is usually well-connected usually well-connected

i.e. number of outliers are i.e. number of outliers are few (most nodes have > 2 few (most nodes have > 2 neighbors)neighbors)

Need to investigate other Need to investigate other joining modelsjoining models

Business model Business model considerations will be considerations will be importantimportant

SuburbiaSuburbiaUpper-middle class Upper-middle class neighbourhoodneighbourhood

Houses about 40-120’ Houses about 40-120’ apartapart

21 houses covering 7.8 21 houses covering 7.8 acres or ~1/3 acre lotsacres or ~1/3 acre lots

Microwave ovens, Microwave ovens, cordless phones, cordless phones, televisions etc. cause televisions etc. cause interferenceinterference

Angled sheetrock and Angled sheetrock and concrete walls, hills concrete walls, hills and trees absorb signal and trees absorb signal and create multi-path and create multi-path reflectionsreflections

Not a pleasant place to Not a pleasant place to roll out wirelessroll out wireless

One reason why One reason why cellular uses 80’-100’ cellular uses 80’-100’ masts for their cell masts for their cell towerstowers

20

40

60

80

100

120

140

160

0

5 GHz:5 GHz:Bandwidth is good, Bandwidth is good, provided you can get a provided you can get a mesh to formmesh to form

Published 802.11a Published 802.11a ranges led us to believe ranges led us to believe we could achieve the we could achieve the yellow circleyellow circle

Measured range from Measured range from the apartment trial is the apartment trial is the red circlethe red circle

Range is not sufficient Range is not sufficient to bootstrap mesh until to bootstrap mesh until installed % is quite high installed % is quite high (in this diagram ~50%)(in this diagram ~50%)

20

40

60

80

100

120

140

160

0

802.11a in a Multihop 802.11a in a Multihop NetworkNetworkImpact of path length on TCP Throughput

0

2000

4000

6000

8000

10000

12000

0 1 2 3 4 5 6 7

Path Length (Hops)

TC

P T

hro

ug

hp

ut

(Kb

ps)

Round Trip Delay versus Round Trip Delay versus Node Density Node Density

Average RTTavg_rtt = 0.1*curr_sample + 0.9*avg_rtt

One sample every 0.5 seconds

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0 20 40 60 80 100 120 140 160 180

Time

Ave

rag

e R

TT

A new 100Kbps CBR connection starts every 10 seconds, between a new pair of nodes. All nodes hear each other.

Collision between ISM Collision between ISM devicesdevices

Panasonic 2.4GHz Spread Spectrum Phone 5m and 1 Wall from receiver

Phone

Colliding standardsColliding standards

Performance worsens when there are large number of short-range radios in the vicinity

Courtesy: Mobilian Corp.

To make it realTo make it realIdentify and solve key problems Identify and solve key problems

build and deploy a mesh build and deploy a mesh prototypeprototype

Conclusion Conclusion Meshes are viable Meshes are viable

existing technologies are existing technologies are inadequateinadequate

Problem SpaceProblem SpaceRange and Capacity Range and Capacity [Talk by Jim K; Poster by John D. & Ranveer [Talk by Jim K; Poster by John D. & Ranveer

C.]C.]Electronically steerable directional antenna or MIMO for range enhancementElectronically steerable directional antenna or MIMO for range enhancementMultiple frequency meshesMultiple frequency meshesMulti-radio hardware for capacity enhancement via greater spectrum Multi-radio hardware for capacity enhancement via greater spectrum utilizationutilizationNew data channel MAC for higher throughputNew data channel MAC for higher throughputTools for predicting & analyzing network viability & performanceTools for predicting & analyzing network viability & performance

Multihop Routing Multihop Routing [Talk by Rich D.; Poster & Demo by Jitu P. & [Talk by Rich D.; Poster & Demo by Jitu P. & Brian Z.]Brian Z.]

L2.5 on-demand source routing. Routes selected based on link quality L2.5 on-demand source routing. Routes selected based on link quality Route selection with multiple radiosRoute selection with multiple radios

Security and FairnessSecurity and FairnessGuard against malicious users (and freeloaders)Guard against malicious users (and freeloaders)EAP-TLS between MeshBoxes, PEAPv2 or EAP-TLS between clients and EAP-TLS between MeshBoxes, PEAPv2 or EAP-TLS between clients and MeshBoxesMeshBoxesPriority based admission control, Secure traceroute Priority based admission control, Secure traceroute

Self Management & Self Healing Self Management & Self Healing [Talk by Lili Q.; Poster by AP][Talk by Lili Q.; Poster by AP]Desirable: avoid network operator - minimal human intervention Desirable: avoid network operator - minimal human intervention Watchdog mechanismWatchdog mechanismData cleaning and liar detectionData cleaning and liar detectionOnline simulation based fault isolation and diagnosisOnline simulation based fault isolation and diagnosis

Smart Spectrum UtilizationSmart Spectrum Utilization Spectrum EtiquittesSpectrum EtiquittesAgile Radios, cognitive radiosAgile Radios, cognitive radiosCognitive software & applicationsCognitive software & applications

Analytical TechniquesAnalytical TechniquesInformation theoretic tools that predict expected capacity with practical Information theoretic tools that predict expected capacity with practical constraints, based on experimental dataconstraints, based on experimental data

Digital Rights Management (DRM)Digital Rights Management (DRM)Broadband access will become popular with expanded digital content.Broadband access will become popular with expanded digital content.Increase the value proposition for end-users/subscribersIncrease the value proposition for end-users/subscribers

Ease of use (Plug and play, HCI)Ease of use (Plug and play, HCI)Make the user experience pleasant Make the user experience pleasant QoS protocols over wireless meshes to improve content deliveryQoS protocols over wireless meshes to improve content delivery

Problem Space (Cont)Problem Space (Cont)

Proof of Concept via rapid prototyping and testbed deployments

End Device

Scenario: Neighborhood Wireless MeshesInternet

Gas Station

Bus Stop

Mesh Router 2

End Device (Guest to Router 1)

Mesh Router 1

Mesh End Device

EXIT

Any StreetMesh Zone

Mesh Router 3

(Internet TAP)

Mesh Router 5

Mesh Router 7

90

101

206

Mesh Router

End DeviceEnd DeviceConnectsConnects to a Mesh to a Mesh RouterRouterStandards CompliantStandards Compliant Network InterfaceNetwork Interface

Mesh Router / MeshBoxMesh Router / MeshBoxRoutes trafficRoutes traffic within the within the mesh and to the mesh and to the neighborhood Internet neighborhood Internet GatewayGatewayServes as Serves as access pointaccess point for End Devicesfor End Devices

Neighborhood Internet Neighborhood Internet GatewayGateway

GatewayGateway between the between the mesh nodes and the mesh nodes and the InternetInternet

ITAP

Data Channel RadioMiniport Driver

Control Channel Radio

Miniport driver

Mesh Routing Functionality

Mesh Management Module

TCP / IP

Mesh Connectivity Layer(MCL)

Multi-hop Routing/Bridging Radio Selection Metric

Topology Control

Link Monitor Module

Mesh Box Configuration

SECURITY

Diagnostics Kernel Module

Diagnostics Client and Server DLLs

Research ResultsResearch ResultsSpectrum EtiquetteSpectrum Etiquette

P. Bahl, A. Hassan, P. Vries, P. Bahl, A. Hassan, P. Vries, Spectrum Etiquettes for Short Spectrum Etiquettes for Short Range Wireless Devices Operating in the Unlicensed Range Wireless Devices Operating in the Unlicensed Band - A ProposalBand - A Proposal,,,,White paper, Spectrum White paper, Spectrum Policy: Property or Commons, Stanford Law School

Multi Radio MeshesMulti Radio MeshesA. Adya, P. Bahl, J. Padhye, A. Wolman, and L. Zhou. A. Adya, P. Bahl, J. Padhye, A. Wolman, and L. Zhou. A Multi-Radio Unification Protocol for IEEE 802.11 Wireless Networks. BroadNets 2004 (also Technical Report, MSR-TR-. BroadNets 2004 (also Technical Report, MSR-TR-2003-41, June 2003)2003-41, June 2003)

Determining Mesh CapacityDetermining Mesh Capacity

K. Jain, J. Padhye, V. Padmanabhan, and L. Qiu. K. Jain, J. Padhye, V. Padmanabhan, and L. Qiu. Impact of Interference on Multi-hop Wireless Network Performance. ACM Mobicom, San Diego, CA, September 2003 ACM Mobicom, San Diego, CA, September 2003

Mesh Self ManagementMesh Self ManagementL. Qiu, P. Bahl, A. Rao, and L. Zhou. L. Qiu, P. Bahl, A. Rao, and L. Zhou. Fault Detection, Isolation, and Diagnosis in Multi-hop Wireless Networks. Technical Report, MSR-TR-2004-11, December 2003Technical Report, MSR-TR-2004-11, December 2003

Research Results (cont.)Research Results (cont.)

Single Radio Mesh PerformanceSingle Radio Mesh PerformanceR. Draves, J. Padhye, and B. Zill. R. Draves, J. Padhye, and B. Zill. Comparison of Comparison of Routing Metrics for Static Multi-Hop Wireless Routing Metrics for Static Multi-Hop Wireless NetworksNetworks.. ACM SIGCOMM 2004 (also Technical Report, ACM SIGCOMM 2004 (also Technical Report, MSR-TR-2004-18, March 2004)MSR-TR-2004-18, March 2004)

Single Radio Mesh PerformanceSingle Radio Mesh PerformanceR. Draves, J. Padhye, and B. Zill. R. Draves, J. Padhye, and B. Zill. Routing in Multi-radio, Routing in Multi-radio, Multi-hop Wireless Mesh Networks,Multi-hop Wireless Mesh Networks, To appear in ACM To appear in ACM MobiCom 2004MobiCom 2004

Multi Radio Mesh Routing & Multi Radio Mesh Routing & PerformancePerformance

L. Qiu, P. Bahl, A. Rao, and L. Zhou. L. Qiu, P. Bahl, A. Rao, and L. Zhou. Fault Detection, Isolation, and Diagnosis in Multi-hop Wireless Networks. Technical Report, MSR-TR-2004-11, Technical Report, MSR-TR-2004-11, December 2003December 2003

Capacity EnhancementCapacity EnhancementProblemProblem

Improve throughput via better utilization of the spectrumImprove throughput via better utilization of the spectrum

Design ConstraintsDesign ConstraintsRequire only a single radio per nodeRequire only a single radio per nodeUse unmodified IEEE 802.11 protocolUse unmodified IEEE 802.11 protocolDo not depend on existence of control channelDo not depend on existence of control channel

Assumption Assumption Node is equipped with an omni-direction antenna Node is equipped with an omni-direction antenna

- - MIMO technology is OKMIMO technology is OK Multiple orthogonal channels are availableMultiple orthogonal channels are availableChannel switching time is 80 usecs. Channel switching time is 80 usecs.

- - current speeds 150 microsecondscurrent speeds 150 microseconds

Capacity EnhancementCapacity Enhancement

1

2

5

4 6

3

Only one of 3 pairs is active @ any given time

In current IEEE 802.11 meshes

10 msecs 10 msecs 10 msecs

1 2

3 4

1 4

5 2

Ch 1

Ch 2

…5 6 3 6Ch 3

5 4

1 6

3 2

With MSR’s SSCH enabled meshes

PerformancePerformance

0

0.5

1

1.5

2

2.5

3

3.5

4

10 20 30 40 50

Number of FlowsT

hro

ug

hp

ut

(Mb

ps)

SSCH M802.11

Significant capacity improvement when traffic load is on multiple separate flows

0

10

20

30

40

50

60

70

80

10 20 30 40 50

Number of Flows

Th

rou

gh

pu

t (M

bp

s)

SSCH IEEE 802.11

100 nodes, IEEE 802.11a, 13 channels, every flow is multihop

Avg. per node Throughput Total System Throughput

Mesh Diagnosis Mesh Diagnosis Visualization ModuleVisualization Module

TestbedsTestbedsDetails

23 to 30 nodes23 to 30 nodesInexpensive desktops (HP d530 Inexpensive desktops (HP d530 SF)SF)Two radios in each nodeTwo radios in each node

NetGear WAG or WAB, Proxim NetGear WAG or WAB, Proxim OriNOCOOriNOCOCards can operate in a, b or g Cards can operate in a, b or g mode.mode.

PurposeVerification of the mesh software Verification of the mesh software stackstack

Routing protocol behaviorRouting protocol behaviorFault diagnosis and mesh Fault diagnosis and mesh management algorithmsmanagement algorithmsSecurity and privacy architecture Security and privacy architecture Range and robustness @ 5 GHz Range and robustness @ 5 GHz with different 802.11a hardwarewith different 802.11a hardware

Stress TestingVarious methods of loading testbed:Various methods of loading testbed:

Harpoon traffic generator Harpoon traffic generator (University of Wisconsin) (University of Wisconsin)

Peer Metric traffic generator Peer Metric traffic generator Ad-hoc use by researchers Ad-hoc use by researchers

205

201

204

203

210

226

220

227

221

225

224

206

211

207

208

209

219

215

216

218

217

214

223

Appro

x. 6

1 m

Approx. 32 m

Redmond Apartment TrialsRedmond Apartment Trials

Microsoft Campus

32 Bellaire Apts

31

Deployed by The Venice Team

UNIT FF UNIT GG

Road

UNIT HH20 Feet

ControlAptGG302

Road

Parking Lot

FF203

UNIT CC

UNIT BB

Carport

BB103

BB302

BB201

ControlAptGG302

GG202

HH301

B

B

B

B

B

B

B

B

A

A

bb

a

a

= MeshBox

A B

B

A

FF102

B

A

A

A

A

Redmond Apartment TrialRedmond Apartment Trial

Control Apt GG302 Mesh Box

Mesh Hall (Kitchen)

Apt FF201

Cambridge UK TrialCambridge UK Trial

MSR-Cambridge - 1st Floor, Mesh box Locations

UK3-GtwyUK8

UKMCE20

UK2

UK6UK1UK-MCE20 is the

Kiosk with posters.

= Mesh Box location = Mesh Box location

10 node meshWorking with ehome to create a media sharing demo in collaboration with ZCast DVB trial

Deployed by The Venice Team

Near Term GoalsNear Term Goals

Multi-radio mesh routersMulti-radio mesh routers

Directional Antenna enabled meshesDirectional Antenna enabled meshes

Multi spectral meshesMulti spectral meshes

Mesh Connectivity LayerMesh Connectivity Layer

Self Managing MeshesSelf Managing Meshes

Large Scale TestbedsLarge Scale Testbeds

We hope to take this research to the Point of IrrefutabilityWe hope to take this research to the Point of Irrefutability

Thanks!Thanks!

http://research.microsoft.com/meshhttp://research.microsoft.com/mesh

SSCH Algorithm & SSCH Algorithm & PerformancePerformanceEvery node has a channel hopping scheduleEvery node has a channel hopping schedule

4 pairs of (channel, seed) = (4 pairs of (channel, seed) = (xxii, a, aii))

Hop to a new channel every 35 packets, 1536 bytes/packetHop to a new channel every 35 packets, 1536 bytes/packetxxii ( (xxii + a + aii) mod 13) mod 13 (802.11a has 13 channels)(802.11a has 13 channels)

Parity slot Parity slot xxparityparity = a = a1 1 prevents logical partitioning prevents logical partitioning When unsynchronized, still overlap ~1/13When unsynchronized, still overlap ~1/13thth of time of time can can exchange updated schedulesexchange updated schedules

Synchronize = Change your schedule to match another node’sSynchronize = Change your schedule to match another node’s

Common case in sending is know recipient node’s scheduleCommon case in sending is know recipient node’s schedulepropagation of schedule information is more frequent than propagation of schedule information is more frequent than start of new flowsstart of new flowsif wrong, pay latency penaltyif wrong, pay latency penalty

Mesh Security Mesh Security ArchitectureArchitecturePhilosophyPhilosophy

Mesh is open to all, however, Mesh is open to all, however, people who contribute people who contribute resources to the mesh get resources to the mesh get priority.priority.

Design GoalsDesign Goals

Guard against faulty or Guard against faulty or hacked Mesh Boxeshacked Mesh Boxes

Defend against disruption Defend against disruption (e.g., denial-of-service (e.g., denial-of-service attacks) by malicious End attacks) by malicious End DevicesDevices

Protect mesh traffic privacyProtect mesh traffic privacy

Protect access to network Protect access to network resourcesresources

AssumptionsAssumptions

Difficult to hack into a Mesh Difficult to hack into a Mesh Box (similar to cable Box (similar to cable modems)modems)

End Device

Server

End Device

Laptop

Internet

Certificate

Certificate

Certificate

Internet Gateway

Compromised Mesh Box

Malicious End Device

NeighborhoodFile Server

Snooper

Regulations handle malicious RF Interference attacks

Basic FrameworkBasic FrameworkCertificationCertification

Mesh Routers have Mesh Routers have built-built-in public-key certificatesin public-key certificates for authenticating to each for authenticating to each otherother

Mesh Router owners use Mesh Router owners use them to them to issue certificatesissue certificates to End Devicesto End Devices

Mesh Box accepts Mesh Box accepts certificates issued by any certificates issued by any Mesh Box within rangeMesh Box within range

Access to resources is Access to resources is controlled based on policy controlled based on policy and End Device certificateand End Device certificate

Encryption & AnonymityEncryption & Anonymity

Traffic between Mesh Boxes Traffic between Mesh Boxes is encryptedis encrypted to foil to foil eavesdropperseavesdroppers

Misbehaving Mesh Boxes Misbehaving Mesh Boxes have their certificates have their certificates ““blackballedblackballed””

Traffic from uncertified End Traffic from uncertified End Devices Devices is preventedis prevented from from disrupting certified trafficdisrupting certified traffic