WiFi Test BedExperimentation Plan

Version 3.1

1 September 2004

Prepared under Subcontract SC03-034-191 with L-3 ComCept, Contract Data Requirements List (CDRL) item A002, WiFi Deployment and Checkout Plan

Prepared by:

Timothy X Brown, University of Colorado at Boulder303-492-1630timxb@colorado.edu

Kenneth Davey, L-3 ComCept 972-772-7501ken.davey@L-3ComCept.com

1 INTRODUCTION1.1 PurposeThis document provides experimentation plan details associated with a WiFi-based (802.11b) Wireless Local Area Network (WLAN) test bed made up of terrestrial and airborne nodes as well as broadband connectivity back to a Network Operations Center (NOC). Following an overview of the test bed in Section 2, modes and configurations are outlined in Section 3, measures of performance and effectiveness are listed in Section 4, demonstration and test experiments are detailed in Section 5, and methods & procedures are provided in Section 6.

This WiFi Test Bed Experimentation Plan is the final result of Phase 2 activities associated with experimentation planning, deployment, and initial testing. It represents the full test plan for Phase 3 of the project. This document will be updated to reflect changes and additions as they arise.

1.2 BackgroundCommunication networks between and through aerial vehicles are a mainstay of current battlefield communications. Present systems use specialized high-cost radios in designated military radio bands. Current aerial vehicles are also high-cost manned or unmanned systems.

L-3 ComCept Inc. has contracted with the Air Force Materiel Command (AFMC), Aeronautical Systems Center (ASC), Special Projects (ASC/RAB) to establish and manage a Wireless Communications Test Bed project for the purpose of assessing a WLAN made up of terrestrial and airborne nodes operating with WiFi-based (802.11b) communications. The University of Colorado has been subcontracted to design, install and operate the test bed made up of Commercial Off-The-Shelf (COTS) equipment, and to integrate and operate Unmanned Aerial Vehicles (UAVs) which will interact with it. The network shall support rapidly deployed mobile troops that may be isolated from each other, allow for ad hoc connectivity, and require broadband connection to a Network Operations Center. Experiments are to be performed to measure and report on the performance and effectiveness of the test bed communications capabilities. The Wireless Communications Test Bed project is being executed in phases. The objectives and dates associated with each phase are outlined in Figure 1.

1.3 ObjectiveThe objective of the wireless communications test-bed effort is to deploy and test a COTS-based communications network made up of terrestrial and aerial nodes that employ state-of-the-art mobile wireless and Internet Protocol (IP) technology. The solution shall support rapid deployment of mobile troops that may be isolated from each other, and require broadband connectivity to a Network Operations Center. Experiments are to demonstrate the potential for rapid deployment of an IP-centric, wireless broadband network that will support both airborne and terrestrial military operations anywhere, anytime.

Figure 1: Project Phases

1.4 ApproachA platform supporting the IEEE 802.11b (“WiFi”) industry standard for Wireless Local Area Networks has been chosen as the basis for the test bed due to its support of broadband mobile wireless communications, dynamic ad hoc mesh network operation, and being commercially available at low cost. A common 802.11 platform will be utilized for all ground-based and UAV-based nodes. Special software (routing protocols) developed by the University of Colorado to efficiently manage ad hoc mobile mesh network functionality will be applied to the ad hoc network nodes.

The terrestrial and airborne communication devices will form an IP-centric network on an ad hoc basis. Broadband links will be established to a remote NOC location. Remote monitoring capabilities will allow for remote users to access performance data obtained, and to monitor the test site and activity on a real time basis. Packet data traffic in low, medium, and high-load regimes will be utilized for measuring performance and service support abilities. Typical multimedia applications (messaging, web page download, video, and VoIP) will be evaluated.

A location has been chosen that allows for uninterrupted testing of multiple deployment scenarios. Baseline performance will be established on a ground-to-ground connected configuration. Mobile node impacts will be tested. UAV effectiveness for connecting isolated troops will be evaluated, along with UAV abilities to extend the range of communication.

2 TEST BED OVERVIEWAn overview of the test site and test bed design is provided in the sections below. More detailed test bed design information is included in sections to follow.

Aug. ’03 Phase 1: Test Bed Design & Test Plan Generation

Test Bed DesignDeployment & Test

PlanningApprovals to Operate Initial design and

planning documentation

Aug, ’03 to Feb, ‘04

Procure & Test Equipment

Software/Protocol Development

Integration & Deployment

Rehearsal Experimentation

Oct, ’03 to Sep, ‘04

Phase 3: Test Completion, Reporting

Phase 2: Build, Integration, Deployment, Eng. TestOct. ’03

Feb. ’04

Sep. ’04

Nov. ‘04May. ’04

Test Bed FinalizationFull Experimentation &

Test ExecutionFinal Reports

May, ’04 to Nov, ‘04

Figure 2: View of the Table Mountain National Radio Quiet Zone

2.1 Test Site The Table Mountain National Radio Quiet Zone (NRQZ) is owned by the Department of Commerce and operated by the Institute for Telecommunication Sciences (ITS) approximately 10 miles north of Boulder, Colorado. The site is 2.5 miles by 1.5 miles on a raised mesa with no permanent radio transmitters in the vicinity. An aerial photo of the site is shown in Figure 2. A map of the site is shown in Figure 3, with primary and alternate fixed sites, UAV operations area, and mobile unit paths identified.

In Figure 3, the grid lines are 1000ft (300m) spacing. FS1 and FS2 are powered fixed site locations, and are connected via fiber optic cable. Broadband connectivity to the Internet and the NOC is through FS1 and FS2, and over the fiber optic connection. These nodes are laptop computers with 100mW PCMCIA WiFi cards and placed inside of buildings. Their range is limited to nearby the building. Hence the placement of MNR1 very near to FS1. The green, large dashed, line highlights a public road circuit around the base of the mesa.

Table Mountain has several facilities that make it ideal for the wireless test bed needs. First it is a large 2.5sq mi zone where radio communications is controlled. The top is flat and unobstructed. The facility itself is a mountain obstacle suitable for obstructing users on opposite flanks of the mountain as in Scenario 1 in Figure 12. It is circled by public roads so that communication to or from the mountain can be easily set up from any direction. The site has buildings that can house equipment and provide AC power. Fiber optic cable runs exist between buildings. Finally, it has several areas suitable for UAV flight operations.

Figure 3: Map of Table Mountain

2.2 Network OverviewAn overview of the wireless communications network to be established by the test bed is provided in Figure 4. As shown, a WLAN comprised of fixed, mobile ground, handheld, and aerial units is connected to a remote NOC location through a Local Command Center (LCC). Local area connectivity is made available with units supporting 802.11b wireless transmission and mesh network routing. Remote monitoring and display capabilities are possible through an

MNR1

MNR3

MNR2

MNR5MNR4

FS2

FS1

MNR2alternate site

MNR5alternate site

MNR4alternate site

MNR3alternate site

FS – Fixed Site (power, network backhaul)MNR – Mesh Network Radio

1000ft (300m)

Mobile Node (MNR2 and MNR5) Paths

UAV Airstrip

UAV Flight Path

Public Road Circuit

Internet connection. LCC connectivity to the NOC and Internet through an Iridium satellite link will be tested in Phase 3 of the program.

Figure 4: Network Overview

802.11b nodes will be connected together using standard IEEE 802.11b WiFi cards. The cards being used are Orinoco Gold cards. The cards can be operated in “infrastructure” mode which allows them to communicate only through 802.11b access points. For the mesh networking that will be part of the test bed operation however, the cards will be operated in “ad hoc” mode, which allows any of the Mesh Network Radios to talk directly with each other. For stability, all the cards will be operated at 2Mbps on the test bed. All network elements will communicate using IP version 4 (IPv4). More details of the 802.11 standard can be found at: ANSI/IEEE 802.11 Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications 1999 Edition, IEEE, March 18, 1999.

The 802.11b standard allows one node to talk to another. Multi-hop routing capability is provided by the Dynamic Source Routing (DSR) protocol.1 DSR defines route discovery, forwarding, and maintenance protocols that enable the mesh network operation. Colorado University personnel have implemented this protocol so that it can be modified for testing, optimization, and monitoring. The DSR protocol is being implemented with the Click Modular Router language2, which is very flexible and provides a high degree of control on the network operation.

1 The Dynamic Source Routing Protocol for Mobile Ad Hoc Networks (DSR), David B. Johnson, David A. Maltz, Yih-Chun Hu, INTERNET-DRAFT: draft-ietf-manet-dsr-09.txt, 15 April 20032 http://www.pdos.lcs.mit.edu/click/

Internet

Network Operations Center(NOC)

Remote Monitor

Iridium Connect(Phase 3 potential)

--- 10 mile Distance ---

400-500 ft2.4GHz802.11b

16-29GHzKa/L-Band

Fiber OpticConnection

Table Mountain Test Range• 802.11b 2.4GHz Communications• Broadband Wireless• Ad Hoc IP Mesh Network• Fixed and Mobile Ground Units• Multiple Unmanned Aerial Vehicles (UAVs)

Local Command Center(LCC)

Alternative RF Link

Internet

PSTN

InternetEthernet

C-404

UCB UAV

InternetInternet

Network Operations Center(NOC)

Remote Monitor

Iridium Connect(Phase 3 potential)

--- 10 mile Distance ---

400-500 ft2.4GHz802.11b

16-29GHzKa/L-Band

Fiber OpticConnection

Table Mountain Test Range• 802.11b 2.4GHz Communications• Broadband Wireless• Ad Hoc IP Mesh Network• Fixed and Mobile Ground Units• Multiple Unmanned Aerial Vehicles (UAVs)

Table Mountain Test Range• 802.11b 2.4GHz Communications• Broadband Wireless• Ad Hoc IP Mesh Network• Fixed and Mobile Ground Units• Multiple Unmanned Aerial Vehicles (UAVs)

Local Command Center(LCC)

Alternative RF Link

InternetInternet

PSTNPSTN

InternetInternetEthernet

C-404

UCB UAV

72MHz R/C

Figure 5: Normal data traffic (red solid) is monitored by each node. Periodically each node sends a report on the data (blue dotted) to the monitor server. This data can be viewed remotely over the Internet (yellow dashed) via a web-based GUI.

2.3 Network ArchitectureThe network architecture to be used for experimentation and monitoring is shown in Figure 5. Information on each node and the interfaces involved are provided in paragraphs to follow.

2.3.1 Ad Hoc Radio Network

The ad hoc radio network is made up of ground nodes, handheld personnel communicators, fixed site, and unmanned aerial communication points. A common radio and 802.11b WLAN interface platform is used between the ground vehicle, fixed site, and aerial nodes. The software can also be run on other platforms such as laptop computers. Additional information on each node is provided in the following paragraphs.

2.3.2 Ground Node

The ground node is mounted on stationary fixtures or on a vehicle node. It is equipped with an 802.11b mesh network radio, a GPS receiver, a power supply, and end-user equipment used for test and application demonstration purposes. Application demonstration equipment to be made available is laptop computers, VoIP phones, and video monitoring equipment. The end-user application equipment is connected to the mesh network radio via an Ethernet switch. Figure 6 depicts the equipment configuration to be connected to the ground vehicles. The mesh network radio equipment used for the ground nodes, including the power supply and GPS receiver, is supplied by Fidelity Comtech, and identified by model number FCI-2601.

FixedSite

1Handheld

Nodes

Ground VehicleNodes

FixedSite

2

Ad Hoc Radio Network

Monitor Server

RemoteMonitor

Table Mountain

NOCUniv. of Colorado

Aerial Vehicle Nodes

Fiber Optic Ring

Normal Data

Monitor Backhaul

Web Access

Internet

Figure 6: Ground Vehicle Node Equipment.

2.3.3 Handheld Node

The handheld personnel communicator is a commercial Personal Digital Assistant (PDA) with 802.11b wireless communication abilities, and special software (routing protocols) applied to efficiently manage ad hoc mobile mesh network functionality. Sharp Zaurus SL-5600 Linux PDAs and Compaq IPaq 3600 PDAs with Linux installed are utilized. In addition, standard laptop computers running the Linux operating system are also capable of running the routing protocols as the handheld nodes.

2.3.4 Aerial Vehicle Node

The UAV is an adaptation of the Telemaster design using carbon fiber constructed at CU and shown in Figure 7. The payload bay is 19.5x6.5x6.5 inches. These dimensions are the maximum space and available space is reduced by airframe ribs and tapering towards the tail. Max takeoff weight is 14.5kg of which the payload is 4.5kg; more than sufficient for the radio hardware (less than 1kg). The engine is a 5HP single piston that provides a cruise speed of 100kmph. Two planes have been constructed and flown. A third is being constructed. Current control is via standard line-of-site RC.

Power Supply

GPS Receiver

802.11bMesh Network

Radio

User Devices

12VDC Ethernet

Laptop Computer

Video Camera

VoIP Phone

EthernetSwitch

Power Supply

GPS Receiver

802.11bMesh Network

Radio

User Devices

12VDC Ethernet

Laptop Computer

Video Camera

VoIP Phone

EthernetSwitch

Figure 7: UAV on airstrip (left), taking off (right).

The UAV is equipped with 802.11b mesh network radio equipment that is common to that used in the ground vehicles and fixed sites. Figure 8 depicts the equipment configuration within the UAVs. The mesh network radio equipment used for the UAVs, including the power supply and GPS receiver, is supplied by Fidelity Comtech, and identified by model number FCI-2701. The left shows the core radio, the center shows it mounted in the environmental enclosure, (Fidelity Comtech FCI-2601) and the right shows the UAV version (Fidelity Comtech FCI-2701) mounted in the UAV. The UAVs will fly at 400-500 feet AGL throughout the testing to meet AMA flight rules3.

Figure 8: Mesh Network Radio Equipment (left), in environmental enclosure (center), in UAV (right).

3 The official rules state “I will not fly my model higher than approximately 400 feet within 3 miles of an airport without notifying the airport operator.” Further, the pilot should maintain “unenhanced visual contact with the aircraft throughout the entire flight operation.” (http://www.modelaircraft.org/templates/ama/PDF-files/memanual04.pdf). This does not preclude flights higher than 400ft, but, keeping the plane in sight and the nearby Vance Brand airport in Longmont (3.5miles) suggest keeping close to this limit.

241cm

16cm

21cm

2.3.5 Fixed Sites

The fixed sites within the test bed will form a part of the active 802.11 communications network, and will be used for backhaul purposes. Backhaul traffic will be carried over the fiber ring located at the Table Mountain test site, and then transported to the monitor server over the Internet. The Monitor Server is connected to the Internet over a standard Ethernet connection. The fixed site node is equipped with an 802.11b equipped laptop computer, a GPS receiver, and a power supply.

2.3.6 Network Operating Center (NOC)

For the test bed, the NOC contains a Monitor Server. The Monitor Server monitors and collects data from the UAV and ground nodes through Fixed Site 1 and Fixed Site 2. It also provides an interface for remote monitoring, with real-time display and play-back modes. The Remote Monitor connection will be over the Internet. Connection to the Table Mountain test site will also be over the Internet.

2.3.7 Iridium

As an alternative to standard connectivity to the Internet, the testbed will also connect via the Iridium satellite network as indicated in Figure 4. Standard testbed traffic will be sent via Iridium to the Internet. Enough information is embedded in monitoring packets to evaluate the Iridium connection.

2.4 MonitoringThe test range monitoring will consist of additional software loaded upon each ad hoc node. This will collect performance statistics with time and location stamps measured from the GPS. This data is periodically sent to the Monitor Server through Fixed Sites 1 and 2. The total experimental data traffic is expected to be small and should not significantly impact the network throughput. Though small, for experimental control, we would like to minimize the monitoring traffic use of the ad hoc network.

Each network node collects the following data.

GPS position and time at start of monitor interval Time when monitor packet is sent Time stamped Experimental Messages (e.g. “WiFi interface down” or experiment

results) Per packet information for each packet sent or received including:

o Packet type (e.g. data sent or DSR control packet received)o Packet routeo Time stampo Packet Sizeo Sequence Number (inserted by packet source)

The data is then sent over time to the Monitor Server at via Fixed Sites 1 and 2. The interval between reports is at most 10 seconds.

Remote monitoring capabilities have been built into the test bed so that remote observers are able to monitor test bed performance, display results, and playback test scenarios. The remote monitoring and display capabilities are designed via a Java graphical user interface (GUI)

viewable from any web browser. See Figure 9 for a monitor display example. The following capabilities have been incorporated.

Situation Map Network Status Messages Performance Graphs for nodes, links, and routes

Figure 9: GUI screenshot showing situational map (left) and throughput graphs (right).

2.4 SecurityFor the test bed, both physical and communication security is considered. The Table Mountain facility is a fenced facility that includes storage and work buildings that can be locked. Equipment such as pole mounted antennas and other outdoor equipment can be left set up over several days. Portable radios, laptops, and UAV equipment will be stored in on-site buildings or carried to and from the site.

To limit access to the wireless communication network, Packet filtering algorithms will be used. Only packets with the specific DSR header used in our software is processed. This will prevent casual users from gaining access to the network.

The monitoring server will require a password in order to have access to the remote monitoring facilities.

3 MODES & CONFIGURATIONSIn order for the wireless network solution to be adequately tested for performance and effectiveness against deployment types, the test bed will be configured in multiple ways. Two broad scenarios, shown in Figure 10, will be used for testing the unique 802.11-based network solution.

Figure 10: Test Scenarios

Multiple configuration types will be involved with each scenario, and are detailed in the following sections. Experimentation techniques for each configuration type and characterization approaches for ease of deployment and operation are detailed in Section 5.

3.1 Scenario 1: Improving ConnectivityIn Scenario 1, radios on the ground are mounted in vehicles, carried by personal, or placed at fixed sites. The radios implement a wireless ad hoc (aka mesh) network whereby if a traffic source and destination are not in direct communication range, intermediate nodes will automatically relay the traffic from the source to the destination.

This generally provides good connectivity between ground nodes. When nodes become separated by distance or geography, then the network is disconnected. In these situations, the UAV serves as a communication relay between disconnected nodes on the ground. Ground nodes that are isolated from other ground users can reach each other through the UAV.

This scenario will demonstrate that ad hoc networks working with COTS WLAN radios can provide connectivity to widespread units. It will further demonstrate that low-cost UAVs can extend this connectivity over wider ranges and geography than is possible solely among ground units. It will demonstrate typical performance measures such as network throughput, latency, and availability that would be possible with these networks.

3.2 Scenario 2: Increased UAV RangeIn Scenario 2, we focus on an ad hoc network of UAVs. A UAV is on a long-distance mission. Communication range is limited because of power, weight, and volume constraints on the low-

Scenario 1:

where ad hoc networking with the UAV increases ground node connectivity

Scenario 2:

where ad hoc networking between UAVs increases mission range

NOC NOC

cost, light-weight vehicle. Communication range is extended by using intermediate UAVs to relay back to the control center.

The scenario will demonstrate that ad hoc connectivity between UAVs can greatly extend the low-cost, light-weight UAV mission profile. As in the first scenario, the second scenario will demonstrate typical performance measures such as network throughput, latency, and availability that would be possible with these networks.

Both of these scenarios can be extended by adding a long distance link between the ad hoc network node and the NOC. A satellite link enables the network to be deployed equally well anywhere in the world.

Based on these scenarios, Section 5 presents a set of experiments to demonstrate the performance and effectiveness of ad hoc UAV and ground networks. The next section describes the specific measures of performance and effectiveness we will use.

4 MEASURES OF PERFORMANCE & EFFECTIVENESSEach of the performance and effectiveness measures to be tested or characterized during the experiments are listed below. Measurement methods and test procedures are provided in Section 6.

4.1 Measures of PerformanceFor a detailed description of how each of the following measures of performance are to be tested, see the corresponding Methods and Procedures in Section 6.

Measures of PerformanceData Throughput

Latency (communication delay)Jitter (delay variation)

Packet Loss, RadioPacket Loss, Congestion

Communication AvailabilityRemote Connectivity

Range

Table 1: Measures of Performance

4.2 Measures of EffectivenessFor a detailed description of how each of the following measures of effectiveness is to be characterized, see the corresponding Methods and Procedures in Section 6.

Measures of EffectivenessNetwork Self-formingNode-failure Recovery

Mobility ImpactHardware Reliability

Ease of Deployment/TransportabilityEase of Operation

Data, Voice, Video, Web Page Communication

Table 2: Measures of Effectiveness

5 DEMONSTRATION & TEST EXPERIMENTSThe experiments will be comprised of a node deployment combined with a specific set of tests. The node deployment is how the nodes are physically mounted and their patterns of mobility. These will be described in the rest of this section. There are six tests that we can run during an experiment described below. Not all will be run with each experiment.

1. Throughput: Purpose – to test the throughput that can be achieved when no other traffic is present. This test uses the netperf4 utility to measure throughput between node pairs over 5 seconds. A script is given a set of source destination pairs and the throughput is measured between each pair one at a time. The default is to measure between every source and destination pair in both directions.

2. Connectivity: Purpose – to measure the ability for node pairs to send packets to each other when the network is lightly loaded. Each node sends pings once per second to a random destination node. Every 20 seconds a new destination node is randomly chosen. Every destination is chosen four times during the experiment by each node. Ping success and round trip delay statistics are collected for each pair.

3. Congestion: Purpose – to measure delays and throughputs when there are competing data streams in the network. Each node picks a random destination and either performs the netperf throughput for 10 seconds or pings once per second. The schedule of pings vs. throughput is chosen so that 2 competing throughputs are always in the network at the same time. Every destination is chosen for at least two throughput and two delay intervals by each node. Delay and throughput statistics are collected for each pair.

4. Subjective: Purpose – to assess the performance of typical network applications as perceived by a user. A user on the test bed attempts the following tasks. Download a web page of size 10 kB, 100 kB, and 300kB. The web page consists of an image of the specified size so that the download progress can be observed by the user. The user attempts to make a voice connection across the test bed. The user records the usability of these applications relative to T1 and dialup connections.

5. Node Failure: Purpose – to measure the ability of the network to route around node failures. The connectivity is run with all but one node. This excluded node alternates between shutting down its interface for 60 seconds and bringing it up for 60 seconds. Performance is measured

4 http://www.netperf.org

during connectivity periods when the nodes is down, when the node is up, and during the transitions from down to up and up to down.

6. Range: Purpose – to measure the throughput as a function of separation between two nodes. This test uses the netperf utility to measure the throughput from a source to a destination every 5 seconds. In each interval the GPS coordinates of the two nodes is recorded so that throughput can be correlated with range.

The tests themselves provide performance information. Further results for each test will be derived from post analysis on the monitoring data. Logs kept by experimenters will provide additional data. In general, all experiments will implicitly provide information on all measures and more details on the measurement process are provided in Section 6. Specific measures and their expected outcome are highlighted for each experiment.

The experiments are organized into categories based on the reference scenarios as shown in Figure 11. The tests run during each experiment are also shown in the figure. The following sections describe each of the experiments.

Figure 11: Relationship between experiments and the tests run in each experiment.

5.1 Baseline Network MeasurementsThe goal of these experiments is to understand the capabilities of the underlying ad hoc network with fixed and with mobile nodes and the role that the UAV plays in this performance. The result is four experiments as shown in. All of these experiments run the throughput, connectivity, congestion, and subjective tests.

5.1.1 Fixed Ground

Six nodes numbered 1, 2, …, 6 are mounted 2m above the surface of the ground and arranged so that they form a five hop network (1 to 2 to … to 6). Node spacing and local terrain will be used to enforce a five hop network without short cuts, for example, node 3 can only communicate reliably with nodes 2 and 4. The specific placement is shown in Figure 3 where nodes 1 to 6 are

Baseline NetworksNo UAVWith UAVFixedthroughputconnectivitycongestionsubjectivethroughputconnectivitycongestionsubjectiveMobilethroughputconnectivitycongestionsubjectivethroughputconnectivitycongestionsubjective

Scenario 2: Increased UAV RangeGround-GroundrangeUAV-

GroundrangeUAV-UAVrangethree UAVthroughput

connectivitycongestion

Scenario 1: Improving ConnectivityNo UAVWith UAVDisconnected Groupsnode

failurenode failureMobile Node at Edgerangerange

Satellite ConnectionTypicalTestbedconnectivitysubjective

labeled FS1, MNR1, …, MNR5. Nodes are powered up with no scenario or location specific configuration in their communication software and then the four tests are run.

Expected Results:

The throughput experiment will establish maximum throughputs as a function of the number of hops in the network. Throughput is bounded by the 2Mbps ad hoc network channel rate. Actual throughput even between direct neighbors will be less because of 802.11, DSR, IP, and TCP overhead. The throughput will decrease with more hops. The connectivity experiment will establish the typical round trip delay and communication availability between nodes. By design the nodes will be placed so as to form a connected network so availability should be at or very near 100%. The delay will increase roughly linearly with the number of hops in the route. The congestion experiment will establish how competition for network resources impacts throughput. With two competing flows, we would expect throughput to be less than half because the resources are split between the two and there is contention overhead. The last experiment should demonstrate a user experience similar to the experience with a few 100kbps DSL line.

5.1.2 Mobile Ground

In this test configuration, two of the nodes MNR2 and MNR5 are placed on top of vehicles and driven during the tests at speeds around 40kph. MNR5 cycles North and South and MNR2 cycles East and West as indicated in Figure 3. Otherwise the experiment is identical to the Fixed Ground experiment.

Expected Results:

More dynamics will be observed in the network as nodes move into and out of range. Jitter will be greater as the network pauses to find new routes when old routes no longer are valid. The hop count will change for a node over time. Throughput and availability will decrease while delay increases as more communication time is devoted to control packets and route error recovery. Some end-user applications may have perceptible degradations. This will show how well the network performs when the topology is dynamic.

5.1.3 Fixed Ground with UAV

In this test configuration, a UAV will fly over the test bed during the experiment. Otherwise the experiment is identical to the Fixed Ground experiment.

Expected Results:

The UAV will be used occasionally when routes have too many hops or other routes are not available. The UAV scenarios will have improved availability and reduced latency for longer routes. Throughput may decrease as the UAV blankets the test bed with its signal and interferes with other nodes communication.

5.1.4 Mobile Ground with UAV

In this test configuration, a UAV will fly over the test bed during the experiment. Otherwise the experiment is identical to the Mobile Ground experiment.

Expected Results:

The longer range and more stable UAV to ground links will be used more often to maintaining connectivity. The availability of the UAV node will improve throughput, delay, and connectivity measures compared to the Mobile Ground experiment.

5.2 Scenario 1: Improving ConnectivityThese experiments are designed to show the role of the UAV in improving connectivity among ground nodes.

5.2.1 Mobile Node at Edge

Nodes will be setup around the top of Table Mountain at the alternate sites shown in Figure 3. A mobile vehicle-mounted node will drive around the network on the roads surrounding Table Mountain. The connectivity test will be activated while the node circuits the mountain. The placement of the nodes will help provide more coverage as the mobile node circuits the mountain. The mountain has rough and irregular sides as seen in Figure 2. Five nodes, no matter how well placed, will not provide coverage all the way around its base.

Expected Results:

The mobile node will route traffic dynamically through nodes on Table Mountain as they come into and out of range. Because of the flat top and irregular steep sides of Table Mountain some portions of the road will not have connectivity to any node and availability for the mobile node will be lower than typical nodes on the mountain. This will show the limits of ad hoc networks to connect to nodes moving at the fringe of the network’s collective coverage.

5.2.2 Mobile Node at Edge with UAV

The deployment will be the same as the Mobile Node at Edge except that a UAV mounted node will fly above Table Mountain.

Expected Results:

The connectivity will improve as the UAV provides a better vantage for connecting to the mobile node. It will not be able to reach the node at all points around the mesa and routes using the ground nodes may be used.

5.2.3 Disconnected Groups

Nodes will be set up at Table Mountain at the 6 primary sites in Figure 3. The node failure test is performed with MNR3 the node that shuts down its interface. When MNR3 is down, nodes MNR1 and MNR2 will have poor connectivity to nodes MNR4 and MNR5.

Expected Results:

When MNR3 is down, the connectivity between the two separated groups will be zero or near zero. Because the routing software tries for up to 30 seconds to deliver a packet, connectivity may not be zero, but, delays will be high. When MNR3 is up, connectivity and delays will be similar to the results for the Baseline Fixed Ground experiment.

5.2.4 Disconnected Groups with UAV

The deployment will be the same as the Disconnected Groups except that a UAV mounted node will fly above Table Mountain.

Expected Results:

The results will be independent of whether MNR3 is up or down and similar to the Baseline Fixed Ground with UAV.

5.3 Scenario 2: Increased UAV RangeThese experiments are designed to specifically test the ability of the ad hoc routing to improve communications to and among UAVs.

5.3.1 Ground-Ground Range

A single fixed MNR radio will be set up at one end of the mesa. The range test will be initiated between a mobile-vehicle-mounted node and the fixed MNR radio. The vehicle will drive slowly around the test range collecting throughput data at different sender and receiver separations.

Expected Results:

Throughput will be near its maximum when the range is small and then fall off as distance increases. The throughput will become more variable as the range increases.

5.3.2 UAV-Ground Range

One or two MNR will be placed outside of FS2 to provide backhaul connectivity to the UAV operations area. The range test will be initiated between a ground-vehicle mounted node and a UAV-mounted node. A UAV mounted node will fly around the UAV operations field. A ground-vehicle mounted node will drive around the top of the mesa and on the roads in the area of the mesa. During range measurements, the non-mobile nodes will have their interface shut down to force all routes directly between the UAV and mobile node.

Expected Results:

The results will be similar to the Ground-Ground Range results with two differences. The throughput will be more variable due to plane dynamics. And, the range over which throughput can be measured will be increased.

5.3.3 UAV-UAV Range

A second plane will be added to the UAV-Ground Range setup. The range test will be initiated between the two UAVs. The two UAVs will fly at different separations. For longer ranges, planes may fly from airfields off of the test range.

Expected Results:

The results will be similar to the UAV-Ground Range setup with greater range over which through put can be measured.

5.3.4 Three UAVs

One or two MNR will be placed outside of FS2 to provide backhaul capability to the UAV operation area. Three UAVs will fly simultaneously. Based on the UAV-UAV results, one may fly at a separate airfield so that a UAV at a further separation will be included. The throughput, connectivity, and congestion tests will be measured between all nodes (UAV and ground).

Expected Results:

The throughput, connectivity, and congestion results will be similar to the Baseline Fixed Ground Experiments, since the planes provide good connectivity. The performance will be

somewhat less because the plane dynamics will cause occasional route errors. The congestion results will be worse since the better connectivity to planes will mean that they will cause more interference between each other during high loads.

5.4 Satellite ConnectivityThe testbed will be setup with ground and/or UAV nodes. The connectivity test will be executed to generate monitoring data from the testbed to the monitor server. Timestamps added by the gateway node will allow elapsed time through Iridium to the monitor server to be measured. The subjective test will be executed with some traffic directed off the testbed over the Iridium network.

Expected Results:

The Iridium link has long delay and limited bandwidth relative to standard Internet connections. Some applications will overwhelm these limited resources. The basic monitoring will work, but voice and large file downloads will have limited utility.

6 METHODS & PROCEDURESThe following sections detail methods and procedures to be used for characterizing the measures of performance and effectiveness called out in Section 4 of this document. Section 6.1 describes the measurements made during experiments. Section 6.2 describes how each of the performance and effectiveness measures are then derived along with expected results. The methods and procedures are common to varying experimental modes and configurations except where noted.

6.1 Experimental MeasuresThe measures of performance and effectiveness are derived from data collected from the test bed. This section describes the data collected at each node and how this is used to compute results. Figure 12 shows the relationship between these measurements. The measures can be divided into direct user observations and node packet statistics.

For the direct user observations, a log is kept by the experimenters. These document hardware anomalies, steps taken to complete experiments, and effort required. For subjective tests of performance when running different applications, users record observations on performance at the test bed relative to performance through the T1 connection at the test bed.

The node packet statistics are collected at each experiment node. A node records statistics on each packet. Every packet received or sent by a node has a packet trace recorded that includes the packet type, the route it is following, the size of the packet, a source sequence number that uniquely identifies the packet from the source, and a time stamp when the packet is received or sent. Note that a relay node generates two packet traces: one when it is received and one when it is sent. The route can be identified since the ad hoc routing protocol uses source routing that explicitly lists the route in data packet headers. A GPS time and location reading for a node will be recorded at least once every 10 seconds.

Figure 12: Measurement Relationships

per packettyperoutesizesequence numtimestamp

per testthroughputconnectivitycongestionsubjectivenode failurerange

per experimentprocedure logsexperimenter observation

per monitor intervalGPS positiontimestampmonitor sequence num

Packet Traces

Monitor Packets

Test Scripts

UserInput

Route Stats

Server Connectivity

Node Separation

Thruput Stats

Connectiv-ity Stats

Delay Stats

Deployment and Transportability

Loss Stats

Subjective Evaluation

Ease of OperationHardware Reliability

Radio Loss Rate

Self Forming

Remote Connectivity

Failure Recovery

Congestion Loss Rate

Mobility Impact

Throughput LatencyRange JitterAvailability

Perf

orm

ance

& E

ffect

iven

ess M

easu

res

Der

ived

Dat

aEx

perim

enta

l Dat

a

The monitor server collects the per-node per packet data and derives additional information. The GPS data from different nodes is compared to compute node separations. The sequence number allows a packet to be tracked from source to destination as it passes through each hop. The precise point where a packet is lost can be identified. Delays can also be measured along each hop and end-to-end. The packet size and timing data enables data throughput rates to be measured.

6.2 Derived Measures of PerformanceThe following subsections describe how measures of performance are derived from experimental data collected. When possible multiple methods are used to derive the measures of performance.

6.2.1 Data Throughput

Data Throughput is the maximum rate that data can be sent on a connection. In practice, you would use a link at 10-20% less than the data throughput. Since the link at maximum rate is unstable to variations.

Method/Procedure:

The throughput test measures the throughput between node pairs. In addition, the data transfer rate can be inferred from the per packet data.

Expected Results:

The throughput will depend on several factors. Mixes of low rate and high rate 802.11b users can lead to anomalous behavior that we wish to avoid here. Therefore all users will be fixed at 2Mbps nominal transmission rate. Ideal 802.11b links have significant overhead. The data throughput of an ideal link is approximately 1.7 Mbps. Smaller packets (which have relatively more overhead), multiple link hops, TCP overhead, poor link quality, etc. can all reduce the data throughput. We expect throughputs on the order of a few 100 kbps on multi-hop links.

6.2.2 Latency

Latency is the time it takes a packet to be sent from an application at a source and received by an application at the destination.

Method/Procedure:

The connectivity test sends packets between source and destination pairs using the ping utility. The ping utility directly measures round trip times between the source and the destination and reports the minimum, maximum, and average latency. The per packet timing data, synchronized via the GPS provides an alternate mechanism to measure delay.

Expected Results:

The 802.11b MAC layer introduces a minimum of about 1msec delay per hop. Additional processing by the sending and receiving nodes can add 10’s of msec. This delay will be cumulative per hop.

6.2.3 Jitter

Jitter is the latency standard deviation.

Method/Procedure:

Jitter is computed with the latency.

Expected Results:

Jitter will increase as a link quality decreases since the 802.11b interface retransmits lost packets. Further, as links become unusable, packets must wait for new routes to be found. The latency will vary and jitter increase as the number of retransmissions varies between packets.

6.2.4 Packet Loss, Congestion

A radio must send packets that it sources plus packets arriving from other radio nodes. When the total source and arrival rate exceeds the radio sending rate, packets will queue in a buffer. If a new packet arrives when the buffer is full it is discarded.

Method/Procedure:

Congestion losses can not be observed directly. Losses on links with and without extra traffic in the congestion test will measure the congestion losses.

Expected Results:

Congestion losses will only occur when the arrival rate exceeds the data throughput.

6.2.5 Packet Loss, Radio

Some packets will be lost because of errors during transmission. These errors will be due to collisions between different transmitters, noise (802.11b uses an unlicensed band), or weak signals due to extreme range.

Method/Procedure:

Every packet will be tracked and losses recorded relative to total packets sent on each link. Ping packets sent during the connectivity test will also measure the number of packets successfully sent from a source and acknowledged by the destination.

Expected Results:

Occasional losses will be the norm (less than 1%) due to random bit transmission errors. The 802.11b interface should avoid collisions between nodes. No noise sources are expected at the Table Mesa National Radio Quiet Zone. The error rate will increase with range.

6.2.6 Communication Availability

Communication Availability measures the fraction of time that a source and destination can reliably send packets between them. We don’t consider congestion losses since by sending too much traffic a destination might appear only partially available.

Method/Procedure:

The connectivity test sends packets and acknowledgements between source destination pairs at one second intervals. The fraction of packets successfully acknowledged measures the availability.

Expected Results:

Availability will be generally high. When links are strong, few packets will be lost due to link errors and availability will be more than 99%. When only weaker links are available, the availability will go down.

6.2.7 Remote Connectivity

Remote Connectivity is simply availability to destinations outside of the network such as the remote server at CU or an Internet website. This will measure how well the ad hoc network can be used to keep users online.

Method/Procedure:

The monitor packets themselves provide a tool for measuring availability. A reliable protocol is used to deliver packets back to the monitor server. This protocol will store packets for up to one hour waiting for a connection back to the remote server. Since packets will eventually reach the server, it is only a question of how long an application is willing to wait. A time delay cumulative distribution can be computed that plots the fraction of packets that reach the server as a function of maximum acceptable delay.

Expected Results:

In mobility scenarios or when connectivity is unreliable, the fraction of packets that exceed a given delay will increase.

6.2.8 Hardware Reliability

Hardware Reliability measures how often the different test bed hardware equipment functions correctly.

Method/Procedure:

Whenever a piece of hardware fails, an entry will be made in a log. A summary of these problems will be created. In addition, UAV flight operations will be video taped for post-experiment analysis.

Expected Results:

The hardware is expected to be reliable, but, the log and video tape will help detect specific failure modes that might be corrected.

6.2.9 Range

Range is the distance that two nodes can reliably communicate. As range increases, radio packet loss rates increase and throughputs will decrease.

Method/Procedure:

Range versus throughput will be measured with the range test. Range versus loss rate data will be measured with the per packet data.

Expected Results:

The range where throughput decreases and loss rates increase will increase as we change from Ground-Ground to Ground-UAV to UAV-UAV nodes.

6.2.10 Network Self-Forming

Network Self-Forming is a measure of whether nodes can turn on with no prior knowledge of the environment and be able to establish communication with each other.

Method/Procedure:

Whenever a route between two nodes is unavailable, when range and location information indicate that a route ought to be available, then the network self forming will have failed. A log of such incidents will be recorded.

Expected Results:

The ad hoc routing protocols are reliable and should always form a network.

6.2.11 Node Failure Recovery

Node Failure Recovery measures the robustness of the network to failures. When a node fails (as will be induced in some experiments), packets will be delayed until they can be routed on a new route. If a new route is not possible, then packets will be lost. When a node fails, the network should find a new route around the failed node when available. The time for this recovery should be minimal. The fraction of failures that do find a new route and the average time when it does find a new route will be measured.

Method/Procedure:

If after a node failure, routes that are actively using the failed node will be determined. Recovery from the failure can be determined by observing packet losses between the source destination pair. The time to recover can be determined by observing the worst case packet latency around the time of the recovery.

Expected Results:

The ad hoc routing protocol is designed for network variability. Node failure recovery time is expected to be on the order of a second.

6.2.12 Mobility Impact

Mobility will cause more network dynamics. These, in turn, will cause more node and link failures.

Method/Procedure:

Experiments will be repeated with and without mobile nodes and the other measures will be compared.

Expected Results:

Mobility will decrease availability and range; and increase latency, jitter, and packet losses.

6.2.13 Data, Voice, Video, Web Page Communication

The other metrics are objective measures. They may not answer whether typical network applications will work well as judged by end users.

Method/Procedure:

The subjective test will specifically record the user experience under typical applications as compared to reliable broadband connections.

Expected Results:

The performance will be similar, but, occasional communication gaps will occur.

6.2.14 Deployment & Transportability

Deployment and Transportability measures the difficulty of moving the test bed, setting up the network, and launching the UAVs.

Method/Procedure:

The total volume of test bed equipment when packaged for travel will be measured. The time to unpack, set up, and start for each equipment type will be measured.

Expected Results:

Most of the equipment is designed to be mobile and should be compact. The UAV can be partially disassembled (remove wings and tail assembly) for compact storage and transport. The communication equipment is designed to be simply turned on with minimal configuration. With a suitable airfield, a UAV can be assembled, prepared, and launched in about an hour.

6.2.15 Ease of Operation

Ease of Operation measures what personnel are needed for operation of the test bed. In addition, it provides an input on the degree of complexity associated with operating the system.

Method/Procedure:

A log of the number and type of personnel at each experiment will be kept.

Expected Results:

We are planning on 6-8 personnel for initial experiments. Some experiments require drivers for the mobile nodes and are not necessary for the communication functionality. With the attributes of 802.11-based systems, the degree of complexity should be small.

6.3 Procedure ChecklistTable 3 below provides a listing of all procedure scripts to be run against the experiments detailed in Section 5.

Table 3: Phase 3 Procedure vs. Experiment Checklist

APPENDIX A

RELATED DOCUMENTS

The documents listed below have been generated in support of the Wireless Communications Test Bed project.

1. Wireless Communications Test Bed: Design and Deployment/Test Plan, Phase 3, Version 0.2, Dated July 20th, 2004. This document provides a high-level overview of the Test Bed design and deployment/test plan.

2. Wireless Communications Test Bed: Design & Interface Specification, Version 1.2, Dated March 8th, 2004. This document represents detailed design specifications for the test bed on a network and sub-nodal basis.