i

Master Thesis

Electrical Engineering

January 2012

School of Computing Blekinge Institute of Technology 371 79 Karlskrona Sweden

Measuring of One-Way Delay in Wireless Mesh Network

Maysam Mehraban, Mohammad Eghbali Ghahyazi

ii

This thesis is submitted to the School of Computing at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Software Engineering. The thesis is equivalent to 20 weeks of full time studies.

Contact Information: Authors: Maysam Mehraban E-mail: maysam.mehraban@gmail.com Mohammad Eghbali GhahyaziE-mail: mammad_egh@yahoo.com

University advisor at Blekinge Institute of Technology: Dr. Patrik Arlos School of Computing (COM)

School of Computing Blekinge Institute of Technology 371 79 Karlskrona Sweden

Internet : www.bth.se/com Phone : +46 455 38 50 00 Fax : +46 455 38 50 57

Examiner: Prof. Adrian PopescuSchool of Computing (COM)

III

ABSTRACT Wireless mesh networks are multi-hop networks which consist of radio nodes in

mesh topology. These networks can use different wireless technologies such as IEEE 802.11, IEEE 802.16, and cellular technology. In recent years, wireless Internet broadband access has become an important aspect for current and future business. Cellular networks often deliver this service with low data rates, high coverage, and expensive cost. Also other types of wireless networks suffer from lack of mobility and the line of sight in the metropolitan areas. Lately, wireless mesh networks have appeared as an alternative to mitigate these problems and provide wireless Internet broadband access with low cost, high data rates, and satisfactory QoS. Consequently the amount of studies on network performance of these networks has been rapidly increased. One-way delay, as one of the key network performance parameters, has become more important Since SLA contracts consider it to guarantee QoS levels. Moreover, accurate one-way delay measurements are difficult to achieve due to challenges in proper synchronization of clocks and existence of asymmetric links. Recently, new methods and infrastructures have been proposed to mitigate this issue.

In this thesis, we have investigated the impact of packet rate, payload size, and number of hop counts on measurement accuracy of a tool named TOM. To do so, we used real one-way delay results obtained from DPMI as reference for comparison. To evaluate TOM measurement accuracy, TOM and DPMI were adapted to each other via source code modifications in order to deliver same measurable delay. Also, an experimental setup was also designed for this purpose. Additionally we computed theoretical lower band for one-way delays to evaluate reliability of DPMI and TOM. In the other part of this thesis, we analyzed the effect of TCP and UDP traffic flow on one-way delay performance with respect to data rate, packet size, and number of hop count, while network traces were captured with DPMI. From the results, we concluded that TOM does not have acceptable accuracy in low packet rates, and its accuracy increases linearly as the packet rate increase. Moreover, varying payload size does not have noticeable effect on TOM accuracy compared to packet rate. Absolute error of TOM measurements for single hop scenario is accumulated as number of passing hop increases. From TCP and UDP experiments, we observed that TCP has higher one-way delay compared to UDP due to existing acknowledgements and retransmissions. Interestingly, TCP and UDP are more sensitive to packet size in multi hop tests rather than single hop tests. Also, for a fixed packet size UDP performs better in high data rates.

Keywords: wireless mesh network, one-way delay, QoS, TOM, TCP, and UDP

IV

ACKNOWLEDGEMENTS

This thesis has been carried out as collaboration between Blekinge Institute of Technology and Karlstad University.

This thesis would not have been possible without the support of many people. We would like to thank our supervisor Dr. Patrik Arlos for his supervision, patience, and tactful guidance. We owe our deepest gratitude to Prof. Andreas Kassler and Peter Dely for their constant guidance and support. We would like to thank Marcel Cavalcanti de Castro for his help and support with KAUmesh network. We sincerely thank to Matthias Hirth for spending his valuable time and technical discussion. Finally, we would like to thank the rest of our thesis commitment to our families and friends for their kind cooperation and encouragement during completion of this thesis work.

V

CONTENTS

ABSTRACT ....................................................................................................................................... III

ACKNOWLEDGEMENTS ............................................................................................................... IV

CONTENTS ......................................................................................................................................... V

LIST OF FIGURES .......................................................................................................................... VII

LIST OF TABLES ........................................................................................................................... VIII

LIST OF ABBREVIATIONS ............................................................................................................ IX

1 INTRODUCTION ....................................................................................................................... 1

1.1 INTRODUCTION ...................................................................................................................... 1 1.2 AIMS ...................................................................................................................................... 2 1.3 RESEARCH QUESTIONS .......................................................................................................... 2 1.4 RESEARCH METHODOLOGY ................................................................................................... 2 1.5 CHAPTER ORGANIZATION ...................................................................................................... 3

2 BACKGROUND .......................................................................................................................... 4

2.1 RELATED WORKS .................................................................................................................. 4 2.2 NETWORK DELAYS ................................................................................................................. 4 2.3 WIRELESS MESH NETWORK ................................................................................................... 5 2.4 DCF ....................................................................................................................................... 6

2.4.1 The two-way handshake mechanism ............................................................................... 6 2.4.2 The four-way handshake mechanism ............................................................................. 7

2.5 APHD .................................................................................................................................... 8 2.6 NETFILTER ............................................................................................................................. 9 2.7 TOM .................................................................................................................................... 10 2.8 DPMI ................................................................................................................................... 12

2.8.1 Measurement Point ........................................................................................................ 12 2.8.2 Measurement Area ......................................................................................................... 12 2.8.3 Consumer ....................................................................................................................... 13

3 EXPERIMENTAL SETUP ....................................................................................................... 14

3.1 KAUMESH ........................................................................................................................... 14 3.2 NODE CONFIGURATION ........................................................................................................ 15 3.3 TOM MODIFICATION ........................................................................................................... 15 3.4 TRAFFIC GENERATION ......................................................................................................... 16 3.5 MEASUREMENT POINT ......................................................................................................... 16 3.6 THEORETICAL LOWER BOUND .............................................................................................. 17 3.7 EXPERIMENT SETTINGS ........................................................................................................ 18 3.8 DELAY ANALYSIS ................................................................................................................ 19

4 RESULTS ................................................................................................................................... 22

4.1 TOM .................................................................................................................................... 22 4.1.1 Theoretical one-way delay baseline ............................................................................... 22 4.1.2 Evaluation of TOM ........................................................................................................ 23 4.1.3 Discussion ...................................................................................................................... 27

4.2 TCP AND UDP MEASUREMENTS .......................................................................................... 30

5 CONCLUSION & FUTURE WORK ....................................................................................... 35

6 REFERENCE ............................................................................................................................. 36

VI

APPENDIX A ............................................................................................................................... 38

APPENDIX B ............................................................................................................................... 41

APPENDIX C ............................................................................................................................... 48

VII

LIST OF FIGURES Figure 2.1 The Network Delays Figure 2.2 Basic access mechanism Figure 2.3 PPDU Frame Format Figure 2.4 Four-way handshake mechanism Figure 2.5 APHD – Inter and Intra node delay Figure 2.6 IPv4 Netfilter hooks Figure 2.7 TOM algorithm Figure 2.8 TOM – Inter and Intra node delay Figure 3.1 KAUmesh Network Figure 3.2 Experimental setup Figure 3.3 Modified TOM algorithm Figure 3.4 Delay Components Figure 3.5 General frame structure from network trace Figure 3.6 Frame structure equipped with TOM options from network trace Figure 3.7 Analysis procedure of an experiment run Figure 4.1 TOM average one-way delay in single hop scenario Figure 4.2 DPMI average one-way delay in single hop scenario Figure 4.3 TOM and DPMI accuracy comparison for payload size = 500 bytes in single hop scenario Figure 4.4 TOM and DPMI one-way delay trace for packet rate = 1024 pps and payload size =500 bytes in single hop scenario Figure 4.5 Absolute Error of TOM average one-way delay in single hop scenario Figure 4.6 TOM Minimum one-way delay in single hop scenario Figure 4.7 DPMI Minimum one-way delay in single hop scenario Figure 4.8 DPMI Minimum one-way delay in single hop scenario for payload size 500 bytes and packet rate 1024 pps Figure 4.9 one-way delay for packet rate 128 pps and packet size 250 bytes by original TOM and DPMI Figure 4.10 one-way delay for packet rate 128 pps and packet size 250 bytes by enhanced TOM and DPMI Figure 4.11 Minimum one-way delay for UDP in Single hop scenario Figure 4.12 Minimum one-way delay for TCP in Single hop scenario Figure 4.13 Minimum one-way delay for UDP and TCP at Rate 128 Kbps in Single hop scenario Figure 4.14 Minimum one-way delay for UDP and TCP at Rate 2048 Kbps in Single hop scenario Figure 4.15 Minimum one-way delay for UDP at Rate 1024 Kbps in different hop counts. Figure 4.16 Minimum one-way delay for TCP at Rate 1024 Kbps in different hop counts Figure 4.17 Minimum one-way delay for UDP and TCP, size 250 bytes in Single hop scenario Figure 4.18 Minimum one-way delay for UDP and TCP, size 1000 bytes in Single hop scenario Figure 4.19 One-way delay trace of UDP for data rate 64 and 2048 Kbps with packet size 250 bytes in Single hop scenario

VIII

LIST OF TABLES

Table 2.1 Rate-dependent parameters Table 3.1 Experiment repetition of = 128 pps and = 100 Bytes Table 4.1 Theoretical delay and minimum OWD obtained by DPMI and TOM Table B.1 TOM and DPMI one-way delay statistics = 8 pps Table B.2 TOM and DPMI one-way delay statistics = 64 pps Table B.3 TOM and DPMI one-way delay statistics = 128 pps Table B.4 TOM and DPMI one-way delay statistics = 256 pps Table B.5 TOM and DPMI one-way delay statistics = 512 pps Table B.6 TOM and DPMI one-way delay statistics = 1024 pps Table C.1 UDP and TCP one-way delay statistics = 64 Kbps Table C.2 UDP and TCP one-way delay statistics = 128 Kbps Table C.3 UDP and TCP one-way delay statistics = 256 Kbps Table C.4 UDP and TCP one-way delay statistics = 512 Kbps Table C.5 UDP and TCP one-way delay statistics = 1024 Kbps Table C.6 UDP and TCP one-way delay statistics = 2048 Kbps

IX

LIST OF ABBREVIATIONS QoS Quality of Service VOIP Voice over IP QoE Quality of Experience ISP Internet Service Provider RTT Round Trip Time VoD Video on Demand IPPM IP Performance Metrics DPMI Distributed Passive Measurement Infrastructure PP Packet pair APHD Adoptive Per Hop Differentiation TOM Tool for One-way delay Measurement MAP Mesh Access Point MPP Mesh Portal ECDA Enhanced Distributed Channel Access DCF Distributed Coordination Function MAC Media Access Control ACs Access Categories WLAN Wireless Local Area Network NAT Network Address Translation HAL Hardware Abstraction Layer MP Measurement Point MAr Measurement Area MArC Measurement Area Controller MArN Measurement Area Network CI Capture Interface TSD Time Synchronization Device DCF Distributed Coordination Function CSMA/CA Carrier Sense Multiple Access with Collision Avoidance DIFS Distributed Inter Frame Space SIFS Short Inter Frame Space CTS Clear-To-Send RTS Request-To-Send NAV Network Allocation Vector

1

1 INTRODUCTION

1.1 Introduction In recent years, Wireless Mesh Networks (WMNs) have gained more interest

among studies focusing on mobile ad hoc networks. WMNs are multi-hop networks that often use the standard IEEE 802.11, IEEE 802.16. WMNs are consisted of mesh routers and mesh clients. Since WMNs are self-organized and self-configured network, mesh connectivity is automatically established and maintained by a set of nodes [1]. Ease of maintenance, incremental deployment, good area coverage, and Non Line Of Sight have made WMNs an appropriate solution in providing alternative Internet connectivity.

People around the world use real time applications such as Voice over IP (VOIP) and video on demand (VoD) at their homes, offices, on the streets, trains or any other place where Internet service is available to use. Undoubtedly, they expect good Quality of Experience (QoE) according to the Service Level Agreement (SLA) in their contracts with Internet Service Providers (ISPs). These reasons made the companies to consider the performance of the network in their researches.

Network performance has become more and more significant with the growth of the Internet. Equally important WMNs like other networks would benefit from the supporting of Quality of Service (QoS), a number of studies have been done on its key performance parameters. Furthermore, each network is different in nature and design, the measuring infrastructures and methodologies play an essential role in gathering the accurate information of the network performance parameters [2]. According to Vito et al [3], the network performance metrics can be divided in four large groups: 1) Availability. 2) Loss and Error. 3) Delay. 4) Bandwidth. Each of these parameters depends on the property of the network and requires an accurate measuring to provide QoS.

In contrast to wired networks, the throughput of WMNs is typically low; therefore, it is difficult to provide multi-media services, which have strict QoS requirements [4]. In addition, packet transmission delay can affect the QoS for IP based real time services without causing any loss in the network [5]. Moreover, delay is used as one of the main metrics of SLA which makes it more important parameter. Also it can help routing protocols to choose the best route to send and receive the information in the network.

One-Way Delay (OWD) and Round Trip Time (RTT) are two of these delay metrics. IP Performance Metrics (IPPM) defines RTT [6] as the time between the emission of the first bit of the packet from host network interface and the reception of the last bit of the packet in the same network interface, in the way that receiver resend the packet immediately after receiving the packet. OWD was defined as the time interval between sending of the first bit of the packet to the network and receiving of the last bit of the packet on the other side.

TCP and UDP are two main transport protocols in the Internet world. The majority of fundamental Internet applications use UDP as transport protocol namely Domain Name System (DNS), Simple Network Management Protocol (SNMP), the Routing Information Protocol (RIP), and Dynamic Host Configuration Protocol (DHCP). Furthermore, real time and time sensitive applications, which handle loss with insignificant quality degradation rather than delay, use UDP to transfer data packets. On the other hand, applications that use TCP such as point of sale,

2

accounting, and database software put priority on packet losses rather than packet delay, since TCP detects and deals with packet loss rather than delay.

Wireless networks generally experience temporarily losses due to fading, shadowing, hand off, and other unwanted radio effects. Meanwhile TCP has been optimized for wired networks, it interprets packet losses incorrectly to congestion. Besides, radio interference between hops affects the overall performance of UDP and TCP in multi-hop 802.11[23]. Additionally, in case of sharing common bottleneck link, UDP outperform TCP due to the fact that TCP back off and regulate its transmitting rate to the rate of UDP. The required time to reach appropriate rate is depended on end-to-end delay.

This thesis presents the impact of packet rate, payload size, and different hop counts on TOM measurement accuracy with respect to DPMI as packet’s real one-way delay value. To investigate TOM accuracy under different circumstances, an experimental testbed is created in KAUmesh testbed, and required software modifications were applied. Similarly, we conduct a set of experiments to evaluate the effect of TCP and UDP packets on end-to end delay of wireless mesh backhaul with varying data rates, packet sizes, and number of hop counts.

1.2 Aims In this thesis we seek two main aims:

Our first aim is to analyze the accuracy of TOM under different packet rates, payload sizes, and hop counts. To do this, we need to measure one-way delay with TOM and also a reference (DPMI). Also, we define accuracy as the difference between one-way delay results obtained from TOM and DPMI

The second aim is to investigate the impacts of UDP and TCP with different data rates, payload sizes, and hop counts on one-way delay in WMN. For this purpose, we need to capture network traces with DPMI and analyze them

1.3 Research Questions

1. Is it possible to compare one-way delay estimation from TOM with the similar estimation from DPMI? If not, what modifications/adjustments are required to allow us to compare TOM and DPMI results?

2. How is TOM measurement accuracy affected by different traffic patterns (combination of packet rate, payload size, and hop count)? And why do they have effect on it?

3. What is the impact of transport layer protocols (TCP and UDP) on one-way delay obtained by DPMI with changing traffic patterns (combination data rate, packet size, and hop count) in wireless mesh networks?

1.4 Research Methodology In this thesis, we evaluated TOM measurement accuracy with DPMI being

installed to obtain real one-way delay values. For this purpose, we studied related works, literatures, and source codes in order to identify state of the art and research questions. To solve the research questions, we designed an experimental testbed on the KAUmesh network to provide the same condition for both TOM and DPMI. We chose the experimental method in this thesis since we could exploit real tool, equipment, and

3

WMN available in KAUmesh testbed. Next, TOM was adapted to the experimental testbed and its source code was modified according to the requirements. We validated the experiment results against DPMI as a truthful framework and also with MGEN results in [14]. Then, the experimental setup was implemented and verified. As experiments were carried out in live environment, we repeated each experiment at nighttime to minimize the effects of unwanted interference. Furthermore, we were cautious about queuing and congestion in wireless mesh hop which could have affected our results. Afterwards, we conducted experiments with different packet rates, payload sizes, and hop counts to evaluate the accuracy of TOM results compared to one-way delay obtained from DPMI. Additionally we computed theoretical lower band for one-way delays to evaluate reliability of DPMI and TOM. The experimental setup and configuration are fully discussed in Chapter 3. At last, we performed experiments on TCP and UDP to analyze their impacts on one-way delay in WMN, while data rates, packet sizes, and hop counts were varied.

1.5 Chapter Organization The rest of this document is structured as follows. The related works, necessary

background, and key concepts used in this thesis work are detailed in Chapter 2. The experimental setup, problem solving approach, and experiment configuration are presented in Chapter 3. The results from experiment runs and analysis are discussed in Chapter 4. Finally, the conclusion and future works are presented in Chapter 5.

4

2 BACKGROUND

2.1 Related Works The main problem in measuring OWD is handling the clock synchronization; however,

there are some important factors that may affect accuracy of one-way delay measurement like operating system and threads [3]. To solve clock synchronization problem, some measurement infrastructures and methods have been proposed and developed in different networks by researchers in the past few years. Packet pair (PP) was proposed in [10], as an active measurement method for OWD. The idea of PP is that the transmission delay is the interval time between the two points in which packet A has just received completely and packet B transmission has just started, assuming Packet A and B were sent directly after each other. [11] approaches OWD measurement with use of NTP protocol and virtual clocks as an active measurement platform. They used a developed measurement tool called Netmetric which also considers clock updating during NTP synchronization. In [12], authors used a distributed passive measurement infrastructure to show the impact of packet size on OWD in 3G networks. They focused on accurate timestamps to detect asymmetric network delays. On the other hand, some other studies [13][14]proposed new methods for measuring one way delay in which clock synchronization has no role in accuracy of the measurement. Adaptive Per Hop Differentiation (APHD)[10] scheme is based on EDCA technique, which is proposed in 80211e draft. According to [13], delay requirement is embedded into the header of each packet in MAC layer and updated in each node. Moreover, authors claimed that simulation results indicate APHD can provide end to end delay assurance, but no realistic results on the accuracy of the measurement were provided. In [14], authors get two ideas from APHD and implemented a new Tool for One-way delay Measurement (TOM) which uses IP header option to carry delay information of each packet with itself. According to [14], TOM OWD results were compared with MGEN as a reference in single hop and multi hop WMN testbed (KAUmesh)[15]. They concluded that TOM accuracy depends on network load (combination of packet rate and payload size). In [16], authors considered UDP traffic with different payload sizes and data rates to evaluate the OWD in 3G networks. The results show a pattern at one of the data rates due to UMTS signaling. In study [23], authors performed their experiment through simulation and indicate that throughput of TCP decreases significantly due to the incorrect recognition between link failure and congestion. In [22], authors investigate TCP performance while they generate UDP traffic with high data rate as background traffic. Their experiment is in the ns-2 simulator on the artificial topology. On the other hand, the same study is done in [24], while their experiments are carried out by the real world measurement with a moderate UDP data rate. Both studies [22] and [24] showed that TCP performance decreases with the presence of UDP flows in wireless ad hoc networks.

In this thesis work, we analyze the delay accuracy of TOM and use delays obtained from DPMI as reference for comparison. Then we investigate the effect of the network load on accuracy estimation of OWD with TOM method. On the other hand, none of previous studies has compared TCP and UDP OWD in WMNs with high accuracy measurement equipment. Thus, we analyze the effect of TCP and UDP packets on OWD in WMN. All of these experiments are done in KAUmesh testbed.

2.2 Network delays Network delays can be categorized into three groups: equipment delay, transmission delay,

and propagation delay [7], as shown in Figure 2.1

5

Equipment delay is introduced in processing, switching and queuing of the packet in the equipment until emission of the last bit of packet and depends on the network load or congestion.

Transmission delay is the time needed to transmit all the bits of the frame carrying a packet. Data rate, media, and distance affect this delay [8].

Propagation delay is the interval between time (T2) when the first bit (or the last bit) of a packet was sent out by the sender and (T3) when this bit reach receiver on the other side. [9].

Sender Network receiver T0 T1 T2 T3

equipment

delay Propagation delay

Transmission delay

Figure 2.1: The Network Delays

2.3 Wireless Mesh Network WMN can consist of Mesh Stations, Mesh Access Points, Mesh Portals and Stations [17]. A

Mesh Station is a node which forwards traffic wirelessly and supports mesh services. A node can be a Mesh Access Point (MAP) if it provides access services to legacy client stations (STA). A mesh node connected to a second network like Internet is called Mesh Portal (MPP).WMNs have a flexible structure and can be used in different scenarios such as community networks, Hot-spot extension, Disaster recovery, public security and building automation. WMNs can provide Internet access for tourists in the cities as community networks. Hot-spot operators can increase coverage and capacity in airports and train stations by the use of WMNs [18]. They can be a quick replacement for damaged voice or data networks and can be deployed immediately after natural disaster. There are two types of wireless mesh networks [19]:

Infrastructure/Backbone WMN which consists of MSTAs and MAPs as a backbone for legacy clients. The connectivity between clients and MAPs is provided via other standards such as IEEE 802.11. The clients have no role in implementing mesh services. MPPs are used as gateways to wired networks and any other wireless networks like LTE or IEE802.16.

Client WMNs MAPs are not involved in forming a mesh network. Instead, this role is carried out by MSTAs which are usually mobile. Handling node mobility and energy constraints for MSTAs are two main challenges in these networks.

Additionally, there are some other wireless networks like Mobile ad-hoc networks and Wireless Sensor Networks (WSNs) which have some commonality with WMNs but they are considered to be a separate class of networks.

6

2.4 DCF Distributed Coordination Function (DCF) is the main mechanism to access the medium in

the IEEE 802.11 [18]. DCF uses a random access scheme to maximize the throughput while preventing packet collisions. A packet collision is defined as any case where a node is receiving more than one packet at a time, resulting in neither packet being correctly received. DCF employs carrier sense multiple access with collision avoidance (CSMA/CA), and provides two access schemes: 1) the default, two-way handshake scheme (basic access mechanism). 2) an optional four-way handshake scheme.

2.4.1 The two-way handshake mechanism In this mechanism, suppose that a node has a data to transmit. First it listens to the channel

to determine that the channel is not used by any other node. Then, it will transmit the packet if the channel is found idle for an interval of time that exceeds the distributed interframe space (DIFS). Else, the node continues monitoring the channel as long as it is found idle for DIFS interval. Meanwhile it also specifies random backoff interval which causes node to defer its access to the channel for an extra time period. This feature minimizes collision during conflicts between multiple nodes. The backoff timer is used by DCF to calculate random backoff interval. The time needed by any node to detect the transmission of a packet by any other node is introduced as slot size of the backoff timer which depends on the physical layer. Each successful transmission of packet is verified with an ACK from the destination node. The ACK is immediately transmitted by the destination node after interval time called short interframe space (SIFS). Figure 2.2 illustrates the basic access mechanism.

Figure 2.2: Basic access mechanism

From the Figure 2.2, transmission time per each packet PLCP Protocol Data Unit

(PPDU) can be computed as:

= DIFS+ +SIFS+ (2.1) Where, DIFS = 34µs SIFS = 16µs

7

Figure 2.3: PPDU Frame Format

As it is illustrated in Figure 2.3, PPDU contains PLCP preamble, PLCP header, and Physical

layer Service Data Unit (PSDU). Hence, can be expressed as below:

= + + (2.2) Where, = 4 µs = 16 µs = 4 µs Similarly, can be calculated as

= + + (2.3) Number of data bits per OFDM symbol for different modulation schemes and data

rates are shown in Table 2.1 [18].

Data rate (Mbps)

Modulation

6 BPSK 24

9 BPSK 36

12 QPSK 48

18 QPSK 72

24 16-QAM 96

36 16-QAM 144

48 64-QAM 192

54 64-QAM 216 Table 2.1: Rate-dependent parameters

2.4.2 The four-way handshake mechanism In this mechanism, DCF implements the same rules that were applied in the basic access

mechanism before transmission the data. But it will also send the request-to-send (RTS) frame to the destination node before the transmission of the actual data frame. The destination node will respond with clear-to-send (CTS) frame after CIFS if it is available to receive data and is not

8

receiving data from another node. A successful exchange of these two frames (RTS and CTS) guaranties an idle channel for the transmission of the actual frame data. The node will retransmit RTS frame after a predetermined time interval if a CTS frame is not received from destination node. Since each node consumes considerable amount energy to sense the idle channel, Network allocation vector (NAV) is used to save the energy at all nodes in the network. NAV contains information of the period of time in which channel has busy status. As RTS and CTS frames carry the information of the length of the packet to be transmitted, so other nodes can update A NAV by reading this information from RTS and CTS frames. Figure 2.3 illustrates the four-way handshake mechanism.

Figure 2.4: Four-way handshake mechanism

2.5 APHD Adaptive Per Hop Differentiation (APHD) is designed to guarantee end-to-end delay for

time sensitive applications [10]. This algorithm is based on Enhanced Distributed Channel Access (ECDA), which was offered in IEEE 802.11e for channel access [18]. ECDA is an improved version of IEEE 802.11 Distributed Coordination Function (DCF) for Media Access Control (MAC), and it can provide decentralized priority based QoS in single hop Wireless Local Area Network (WLAN) for up to four Access Categories (ACs). ECDA interprets ACs into different delay requirements on each wireless node. In contrast to ECDA, APHD offers end-to-end delay assurance for multi-hop wireless networks. To estimate one-way delay by APHD, each IP header is extended by four new fields namely end-to-end delay requirement, end-to-end

hop count, delay so far, and hops traversed so far. The traffic generator initializes the value of end-to-end delay requirement and end-to-end hop count fields with QoS requirements with aspects of delay and traversed hops of each packet. The other remaining APHD fields are initialized to zero by source and updated by each node via packet journey in wireless mesh network backbone. Hops traversed so far value is incremented by one at each forwarding hop. Delay so far will be updated at each node as accumulation of intra node delay and inter node

delay. As shown in Figure 2.5, Intra node delay at node B calculated by:

(2.4)

Where is the time when MAC layer of node B receives the packet p, and is time when the packet p gains channel access for transmission. Just before packet transmission

9

intra node delay between node B and C is estimated according to the packet length and transmission rate :

⁄ (2.5)

Finally the delay so far field is updated at the moment of transmission by node B to:

(2.6)

Figure 2.5 APHD – Inter and Intra node delay

These four fields are used to adjust packet priority in order to satisfy QoS delay

requirements by each hop.

2.6 Netfilter Netfilter is the software used for firewalling, Network Address Translation (NAT), and

packet mangling integrated in Linux 2.4.x and 2.6.x [20]. It provides a series of well-defined points called hooks inside of Linux kernel networking stack so that kernel modules can register callback function to them. Whenever a packet traverse a hook, Netfilter calls for registered function on that hook. Consequently, kernel module can perform tasks like manipulating, probing, or dropping on the packets.

Figure 2.6 shows IPv4 hooks located in different points of Linux kernel networking stack that were used in this project. All packets received from link layer traverse NF_IP_PREROUTING before being processed. Then, routing algorithm makes a decision whether this packet should be sent to user space or another hop. In the case of sending packet to local user space, the packet passes the NF_IP_LOCAL_IN hook. Otherwise, the packet is forwarded to another hop, and passes through NF_IP_FORWARD. Moreover, locally created outgoing packets pass the NF_IP_LOCAL_OUT hook. Finally, both locally created and forwarded packets traverse NF_IP_POSTROUTING just before they are handed to device driver.

B CA

p dsf

1.4 ms

p dsf

2.4 ms

𝑜𝑢 ,𝐴 𝑖𝑛 ,𝐵 𝑜𝑢 ,𝐵 𝑖𝑛 ,𝐶

𝑩𝑪 𝑩 𝑨𝑩

10

Figure 2.6: IPv4 Netfilter hooks

2.7 TOM TOM is an APHD like one-way delay passive measurement tool [14]. TOM is capable of

estimating one-way delay through wireless backbone in multi-hop wireless mesh networks from ingress node till egress node. Delay information including type, length, end-to-end hop

remaining requirement, and end-to-end delay remaining requirement are inserted into IP option of each packet. Thus, a packet with added TOM option can travel through entire network without facing problem on nodes which TOM is not installed on them. The delay experienced so far can be retrieved from IP option field on each node.

TOM consists of three major parts: measurement module, shared module, and modified

MadWifi (modified wireless card driver). TOM modules can be loaded into running machine without rebooting or recompiling. Shared module is the main part of TOM; it includes all functions for recording and storing timestamps, calculating and estimating delays, and updating delay information. Measurement module registers itself to desired Netfilter hooks and calls suitable shared module functions. Timestamp is recorded at both NF_IP_PREROUTING and NF_IP_LOCAL_OUT based on where the packet is generated. MadWifi driver is modified to collect and pass timestamps to shared module for calculating intra node delay and to estimate inter node delay. Finally, delay information is updated just before the packet is put into queue for transmission. Figure 2.7 gives an overview of TOM algorithm.

11

Figure 2.7: TOM algorithm

Ingress hop sets delay requirement for each packet, and each forwarding hop subtracts inter

and intra node delay from it. TOM computes one-way delay similar to APHD with slight differences, because it is impossible to implement APHD algorithm since the Hardware Abstraction Layer (HAL) of MadWifi is only in binary form. Subsequently, the exact time when the node B accesses the channel to transmit the packet cannot be obtained. Figure 2.8 depicts that TOM algorithm uses the time instead of , which is the time just before queuing and random access

(2.7)

B CA

p edr

92.5 ms

p edr

87.6 ms

𝑜𝑢 ,𝐵 𝑖𝑛 ,𝐵 𝑜𝑢 ,𝐴 𝑖𝑛 ,𝐶

𝑚 ,𝐵

′𝑩𝑪 ′𝑩 ′𝑨𝑩

Figure 2.8: TOM – Inter and Intra node delay

represents the time for queuing and random access. Hence, intra node delay

of packet at node B calculated by

(2.8)

12

Where is the time that packet p reaches wireless interface of the node B. Estimated

Inter node delay is measured as the time between and the time when the acknowledgment of successfully transmitted packet is received. The time of acknowledgment can be obtained from MadWifi driver. So

, estimated Inter node delay between node B and C, can be formulated as

(2.9) Where is the transmission time for packet p. Furthermore, we undertake that delay of

current packet is like the delay experienced by the previous ones. So, we calculate estimated inter node delay as

(2.10)

represents estimated internode delay for the previous packet. and

are respectively internode delay for the current and previous packets. End to end

delay requirement of current packet is updated by the node just before it is handed to HAL

(2.11)

2.8 DPMI This distributed monitoring system was proposed to provide ease of use, modularity, lower

cost, and security [12]. This infrastructure separates the task of measurement and analysis of measurement from each other. Hence, measurement equipment can use its full potential on measuring rather than analyzing it. It is also compatible with different capturing systems like PCAP-based devices, Endace DAG cards [21], and dedicated ASICs devices. It consists of three main components, Measurement Point (MP), Measurement Area (MAr), and Consumer.

2.8.1 Measurement Point This component is responsible for packet capturing, packet filtering, and distributing

captured data. MP can be either a software running on a host or a physical device in order to achieve high-speed high-performance measurement. It is controlled by Measurement Area Controller (MArC) and sends the measurement data to consumers available in the Measurement Area Network (MArN). It captures traffic from one or more physical medium via Capture Interfaces (CI) connected to receivers. Packets can be timestamped either by CI or receiver. Each receiver filters the packets against rules specified by the MArC. Filter rules indicate a consumer, amount of data to be captured from packets, and packet properties. Next, the receiver appends a header to each frame that contains a CI identifier, a MP identifier, a timestamp of captured time with picoseconds accuracy, actual packet length, and captured length. Frames are transferred to sender, which has the task redirecting them to appropriate consumer.

2.8.2 Measurement Area MAr comprises a MArN, Time Synchronization Device (TSD), MArC, and minimum one

consumer and one MP. MArC is the main part of MAr and provides a GUI for users to configure and control their measurements. It also has the task of managing MPs and supplying filter rules for them. The TSD is responsible for time synchronization between MAr subsystems. MArN capacity should be sufficient enough to handle data burst traffic and the consumer peak data requests.

13

2.8.3 Consumer MP sends the captured network traces to Consumer. The Consumer has the task of storing

network traces in designated format for analysis.

14

3 EXPERIMENTAL SETUP

3.1 KAUmesh KAUmesh is a multi-hop multi-channel wireless mesh testbed at Karlstad University shown

in Figure 3.1, which consists of 20 permanently installed Cambria GW2358-4 [25] mesh routers. Each node has three Atheros based IEEE 802.11 a/b/g WLAN cards and runs Linux 2.6.22 and MadWifi driver [15]. This testbed covers a large area and supports 802.11e service differentiation on the wireless mesh backhaul. Furthermore, each node has an Ethernet interface, which is used for monitoring purposes, downloading node configuration from management server, and sending monitoring data to management server.

Figure 3.1: KAUmesh Network

In our experiments we used node 21 and 22 as ingress & egress mesh routers for single-hop

scenario, and node 23 and 10 were added to wireless mesh backbone of experimental topology for multi-hop scenarios. This is illustrated in Figure 3.2. Management server is selected as traffic generator and another machine was chosen as receiver.

Node21

Node23

Node22

Sender“Management Server”

Receiver

A B

Node10

Physical Ethernet

Measurement Point

DAG 0 DAG1

Figure 3.2: Experimental setup

TOM was installed on top of each selected mesh router for TOM evaluation experiments, but the original version of TOM is only able to measure one-way delay of traffic generated in a

15

node to another addressed node. On the other hand, DPMI can just tap into a physical medium and collect network traces from it. Having these points as a background, the best possible solution to evaluate accuracy of TOM results against DPMI results is to connect sender to ingress hop and receiver to egress hop via physical Ethernet. In this way, measurement point can tap into physical medium at point A and B, as depicted in Figure 3.2.

In order to direct traffic to go through selected nodes, static routes were added to routing table of management server, selected mesh routers, and receiver for each scenario.

We set the wireless PHY data rate of node to the fixed value of 6 Mbit/s in order to avoid problems like CPU bottlenecks and undesirable impact of rate on results. Also, wireless cards employ basic scheme of CSMA/CA as channel access mechanism. Wireless cards are in Ad-hoc mode and operate on 5 GHz frequency band to avoid interference with campus WLAN.

3.2 Node Configuration

For TOM accuracy evaluation, TOM should be installed on each wireless mesh node. Besides, TOM should be able to timestamp the packets as soon as it enters the node from physical Ethernet and then timestamp it again just at the moment the packet reaches the queue for transmission. At this point of time, a function of mh_shared calculates intra node delay by subtracting two available timestamps. Then, intra node delay and delay experienced during transmission by previous packets are subtracted from end to end delay requirement field available in IP option of the packet. Next, MadWifi provides timestamp of acknowledgment for the successfully transmitted packet and calls a function of mh_shared module. Now, mh_shared module is able to compute inter node delay and update the delay experienced by previous packets value according to Equation 2.7. At the next hop, the packet is received by its wireless interface, and a timestamp is recorded at the moment the packet reaches MAC layer of the node. Eventually, the packet leaves the physical Ethernet of node B, and a final timestamp is generated here. So, the intra node delay can be deducted from end to end delay requirement field of the packet.

3.3 TOM Modification Concerning stated prerequisites for each node in Section 3.2, the following features were

added to TOM. Mh_measurement module registers callback function to NF_IP_PRE_ROUTING and NF_IP_POST_ROUTING Netfilter hooks. Therefore, two required timestamps can be recorded one at the moment of receiving a packet by physical Ethernet of node from callback function on NF_IP_PRE_ROUTING hook, and the other one can be recorded when packet leaves a node from physical Ethernet by callback function registered on NF_IP_POST_ROUTING hook. Moreover, this modified TOM is also compatible with multi-hop scenario. Figure 3.3 illustrates modified TOM algorithm.

In unmodified TOM, end-to-end delay is computed from dsf field available in IP option of each packet at NF_IP_LOCAL_IN hook of egress node. Then, results are printed into log messages of that node without any identification. These print functions cause bottleneck in nodes, and we noticed that the delay of network suddenly increases during high traffic rate. Besides, TOM one-way delay results should be compared against one-way delay calculated by DPMI for each identical packet. To overcome the mentioned problems, we removed print functions and updated dsf field with end-to-end delay measured by TOM at NF_IP_LOCAL_OUT hook just before packet passes physical Ethernet of the node. In DPMI, we configured DAG cards to capture first 512 bytes of each packet. Therefore, we were able to read dsf field of each packet with modification of Consumer code, and we have the calculated delay by TOM and DPMI for each identical packet in a same place.

16

Figure 3.3: Modified TOM algorithm

3.4 Traffic Generation The sender and receiver are two Linux machines, one in client mode “sender” and the other

one in server mode “receiver”, which run an existing open source C program. The receiver

listens to the specified port and waits for the client to start sending traffic. Sender takes a set of arguments as input including experiment id, run id, key id, server IP address, destination port, number of packets, payload length, and packet inter departure time in microseconds. Receiver takes experiment id, run id, key id, and port as input arguments. Each packet can be uniquely identified with combination of experiment id, run id, key id, and sequence number. The sequence number of first generated packet is zero and after that it is incremented by one for each packet transmission. Different generators were used for UDP and TCP traffic generation, but they share same features.

3.5 Measurement Point MP has two Endace DAG 3.5 E cards. The first card was used to tap into the flow between

sender and node 21, and the second one was inserted between node 22 and receiver. To avoid clock drift between the two cards, we connected the time synchronization port of the cards together with a standard RJ-45 crossover cable and selected one of the cards as master for the other one. So, both clocks drift together, and its effect will be minimized on delay measurement accuracy. Each DAG card has timestamp precision of 60 ns [13]. Thus, OWD measurement precision of DPMI is 120ns.

.

17

3.6 Theoretical lower bound In this section, we calculated theoretical lower bound for delays in order to obtain baseline

for measurement results. Therefore, we can compare theoretical one-way delay with results from DPMI and TOM result to evaluate their estimations. It must be pointed out that wireless nodes use IEEE 802.11a standard and basic scheme of DCF for media access control in our experimental setup. Basic access mechanism utilizes the Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) as well as positive MAC acknowledgment.

Figure 3.4: Delay Components

As it shown in Figure 3.4, delay for single hop case is the time between A and B, and it is

composed of , , and . represents the delay due to transmission time and propagation delay in physical medium from the dag card to the physical Ethernet of node. It is also affected by queuing and processing of packets inside node 21. Delay depends on transmission and propagation time between two nodes. Additionally, queuing and processing inside the node 22 has effect on . Finally, is equal to the propagation delay between node 22 and DAG1.So the theoretical lower bound can be calculated as follow:

(3.1)

Transmission time for Ethernet communication over full duplex link can be calculated

as = ⁄ (3.2)

DAG 0 Node 21

Node 22 DAG 1

A

B

18

Where f is the frame size in bits and r is the physical bit rate. Propagation delay over specific type of medium can be calculated as:

= ⁄ (3.3) Where represents the distance and is the propagation speed over the medium. Let be the

cable length. As it is mentioned earlier, is equal to sum of transmission and propagation delay between DAG0 and node 21. In addition, queuing and processing of packets inside the node 21 has impact on delay . Let s = 0.59 c (c is the speed of light) which is propagation delay of twisted pair. Therefore, can be formulated as:

= ⁄ +

⁄ + (3.4)

Likewise, can be expressed as

+ + (3.5) and respectively represent delay due to propagation and transmission between

two nodes (Section 2.4.1). is the queuing and processing time inside node 22. We supposed that wireless nodes have line-of-sight (LOS) to each other. Hence, propagation parameters such as multipath do not have noticeable impact on propagation delay. On the other hand, multipath impact on propagation delay considered to be negligible in indoor environments like KAUmesh testbed [26]. Thus, we used speed of light c to calculate propagation delay of as:

⁄ + DIFS+ +SIFS+ (3.6)

Where, DIFS = 34µs SIFS = 16µs

= + + = + +

Lastly, delay , which is the propagation delay in physical medium between node 22 and

DAG1, can be formulated as fallow: = ⁄ (3.7)

3.7 Experiment Settings We conducted experiments on KAUmesh tested to do a comparative study in accuracy

estimation of TOM against DPMI regarding OWD measurement. Additionally, we performed another set of experiments on TCP and UDP to investigate impacts of transport layer network on OWD in WMNs. Each experiment run lasted for 80 seconds, and was repeated for single hop, two hops, and three hops scenarios. We carried each experiment three times during night to minimize unwanted signal interference effects. We configured wireless card to operate in IEEE 802.11a mode with fixed PHY data rate 6 Mbps. Also, we disabled RTS/CTS mechanism in order to obtain lower OWD results in the experimental testbed. Channel 36 was used in single hop scenarios, channel 64 and 149 in two hops scenarios, and channel 64, 149, and 165 in three hops scenarios.

In TOM and DPMI experiments, we configured UDP traffic generator with desired packet rates and payload sizes. Packet rate was selected as 8, 64, 128, 256, 512, and 1024 pps. Then each packet rate was tried with payload size of 100, 250, 500, 750, 1000, and 1400 bytes. Number of packets, inter departure times, and data rates were calculated as function of mentioned requirements. With this combination of packet rate, packet size, and fixed PHY data

19

rate, mesh network will be occupied with traffic from low load to overload. This allows us to examine TOM measurement accuracy under different traffic intensity. Furthermore, we presented results of TOM and DPMI from a single run since comparison between DPMI and TOM is desired, and whatever happens in network should affect both of them.

In TCP and UDP experiments, experiment configurations were selected based on desired data rates and payload sizes. We calculated number of packets and inter departure time as input arguments for traffic generator. The settings of the TCP and UDP experiment are demonstrated in Appendix A. Also, we performed some pre-trail experiments where we executed a number of runs per each experiment, but we didn’t see any significant change in results. Table 3.1 depicts results from 16 runs for data rate 128 pps and payload size 100 bytes. It can be seen that each experiment results have small variation from each other. Therefore, the redundancy seen in the results make the excessive repetition of experiments unnecessary which made us to limit the experiments run to three times.

3.8 Delay Analysis The modifications in Section 3.3 with the mentioned experimental setup described in

Section 3.1 enable both TOM and DPMI to compute almost same measureable delay. Although delays calculated by TOM are always expected to be less than the equivalent ones measured by DPMI because of two main reasons: Firstly TOM timestamps each packet at the moment the packet reaches physical interface of the first node while DPMI timestamps the packet when it traverses the physical medium. The second reason is that TOM timestamps the packet before it is handed to driver at the last node in wireless mesh backhaul whereas DPMI timestamps each packet by tapping into physical medium.

The network traces gathered by MP contain upper 512 bytes of each packet and the capture header attached to it. Figure 3.5 illustrates a general frame structure from network trace collected by MP. Existing consumer software is able to extract information from network traces related to each packet including CI identifier, MP identifier, and timestamp of captured time, IP header information, experiment id, run id, key id, and sequence number. Additionally, we modified consumer to have the capability of reading TOM results embedded in IP option of each packet. Figure 3.6 depicts structure of a frame equipped with TOM options from network trace. Individual packets can be identified with combination of experiment id, run id, key id, and sequence number. Additionally, packet id was used to identify TCP packets. Once packets are identified, we have the estimated delay by TOM from dsf field in IP option, and DPMI results can be calculated for packet p from the timestamps as

(3.8)

Figure 3.5: General frame structure from network trace

Figure 3.6: Frame structure equipped with TOM options from network trace

20

Where is the timestamp of the packet p after it leaves the network interface card

of the Sender. MP records the timestamp just before packet p reaches the receiver. CI identifier indicates whether the packet has just left the sender or reached the destination. Figure 3.7 demonstrates analysis procedure of an experiment run.

Figure 3.7: Analysis procedure of an experiment run

Since OWD accuracy estimation of DPMI is 120 ns at its highest (Section 3.4), DPMI

results were chosen as reference OWD for each packet to be compared against TOM results. Moreover, absolute error γ of TOM measurements were computed as basis of accuracy of

evaluation

(3.9)

represents one-way delay measured by DPMI for the packet , and is one-way value obtained from TOM.

21

Experiment

run Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

1 2.537 5.311 2.35 0.307

2 2.598 5.624 2.351 0.354

3 2.557 5.312 2.358 0.314

4 2.583 5.471 2.35 0.344

5 2.546 5.527 2.337 0.332

6 2.581 5.609 2.361 0.326

7 2.556 5.48 2.355 0.335

8 2.557 5.163 2.352 0.291

9 2.579 5.652 2.351 0.334

10 2.541 5.27 2.335 0.28

11 2.559 5.553 2.352 0.313

12 2.562 5.536 2.348 0.353

13 2.574 5.329 2.362 0.292

14 2.564 5.591 2.355 0.361

15 2.556 5.433 2.35 0.329

16 2.554 5.582 2.347 0.309

Table 3.1: Experiment repetition of = 128 Kbps and = 100 Bytes

22

4 RESULTS In this chapter, we present analysis of the results obtained from experiments conducted on

KAUmesh testbed. Section 4.1 evaluates results from TOM and DPMI against theoretical baseline equivalents. Then, it discusses estimation accuracy issues related to TOM from different prospective. Lastly, it deals with problem identification of TOM and presents preliminary version of enhanced TOM. Section 4.2 gives analysis of TCP and UDP behavior regarding one-way delay in WMNs.

4.1 TOM

As it was discussed before in Section 3.6, we conducted a set of experiments on KAUmesh testbed in order to evaluate the estimation accuracy of TOM results against real one-way delay obtained from DPMI. Experiments configuration can be found in Section 3.6, and the experiments were performed for one, two, and three hops scenarios.

As mentioned in Section 2.5, TOM uses average of the previous packets’ inter node delays

for the current packet. Thus, we take the average one-way delay obtained from each experiment as basis of comparison between TOM and DPMI results.

4.1.1 Theoretical one-way delay baseline This section provides computation of theoretical lower bound of delay for different payload

sizes and compares the results to equivalent ones obtained by DPMI and TOM. The required equations to calculate theoretical lower bound for different experiment configurations are available in Section 3.6.Wireless mesh nodes were configured to operate in 802.11a mode with physical data rate of 6 Mbps. Physical mediums between each DAG card and physical interface of the node are 45 meters long. Furthermore, physical links are full duplex and have 100 Mbps speed. The distance between wireless node 21 and 22 is 9 meters. We assumed that the network is in its optimal condition where queues are empty and packets are processed instantly. Therefore, we selected the packet rate of 8 pps to minimize the effect of packet queuing and processing. The theoretical minimum one-way delays are shown in Table 4.1 along with the measured DPMI and TOM results.

Payload [Bytes]

Theoretical [ms]

DPMI [ms]

TOM [ms]

100 0.345818 0.579834 74.12391

250 0.557818 0.78553 78.51385

500 0.911152 1.143396 78.61514 750 1.264485 1.492262 79.01311

1000 1.617818 1.862884 70.75549

1400 2.183152 2.444148 35.09196

Table 4.1: Theoretical delay and minimum OWD obtained by DPMI and TOM

It can be clearly seen that results obtained by DPMI looks reasonable compared to equivalent ones from theoretical method. Also, DPMI one-way delay results follow the same pattern as theoretical one, and the gap between DPMI and theoretical results seems to be a constant value. This is due to the fact that queues and processes in nodes are not zero as we had assumed initially. One-way delays estimated by TOM differ widely from the theoretical values.

23

Hence, we choose DPMI results as the reference one-way delay values for TOM accuracy evaluation.

Figure 4.1: TOM average one-way delay in single hop scenario

Figure 4.2: DPMI average one-way delay in single hop scenario

4.1.2 Evaluation of TOM Figure 4.1 and 4.2 respectively show delay statistics obtained by TOM and DPMI for a

series of experiment runs with different packet rates and payload sizes in 3-dimensional charts.

10

2451

2

25

6

12

8648

0

0.05

0.1

0.15

0.2

0.25

100500

1000

Rp [pps]

ave

rage

OW

D [

s]

Pl [Bytes]

10

24

51

2

25

6

12

8648

0

0.00005

0.0001

0.00015

0.0002

0.00025

100500

1000

Rp [pps]

C.I

. 9

9%

[s]

Pl [Bytes]

10

2451

2

25

6

12

8648

0

0.0001

0.0002

0.0003

0.0004

0.0005

0.0006

0.0007

0.0008

100500

1000

Rp [pps]

C.I

. 9

9%

[s]

Pl [Bytes]

10

2451

2

25

6

12

8648

0

0.05

0.1

0.15

0.2

0.25

100

5001000

Rp [pps]

ave

rage

OW

D [

s]

Pl [Bytes]

(a) (b)

(a) (b)

24

Figure 4.1.a shows average one-way delay calculated by TOM, and Figure 4.1.b represents 99% confidence interval for average one-way delay. Likewise, Figure 4.2.a depicts average one-way delay obtained from DPMI, and Figure 4.2.b demonstrates 99% confidence interval associated with average one-way delays. The X axis, Y axis, and Z axis correspondingly represent packet rate R, average one-way delay OWD, and payload size .

By comparing Figure 4.1 and 4.2, we can observe that in single hop environment TOM cannot provide acceptable accuracy for low packet rates, and it becomes more accurate with the increase in packet rates. Packet rate 8 pps shows the worst accuracy among rates for all payload sizes. By increasing packet rate from 8 to 1024 pps, we can observe that accuracy of TOM improves for each higher rate compared to lower ones for same payload size. The packet rate 1024 pps shows the best accuracy among all packet rates.

Figure 4.3 shows average one-way delay for payload size 500 bytes and different packet

rates for both TOM and DPMI. From the Figure 4.3, we can see the impact of packet rate on the accuracy of TOM. At packet rate 8 pps, one-way delay average margin error of TOM compared to DPMI is too high. It also visualizes that one-way delay average measured by TOM becomes close to its real value by increasing the packet rate.

Figure 4.3: TOM and DPMI accuracy comparison for payload size = 500 bytes in single hop

scenario

Figure 4.4 indicates TOM and DPMI one-way delay traces for packet rate 1024 pps and payload size 500 bytes in which TOM shows good accuracy. There is a shift between TOM and DPMI since TOM uses average weighted of delay experienced by previous packets, as inter node delay (Section 2.5).

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

8 64 128 256 512 1024

OW

D [

s]

𝑃𝑙 [𝐵𝑦𝑡𝑒𝑠]

DPMI

TOM

25

Figure 4.4: TOM and DPMI one-way delay trace for packet rate = 1024 pps and payload size =500 bytes in single hop scenario

TOM calculation error in comparison with DPMI for single hop scenario is illustrated in Figure 4.5 in 3-dimentional plots. Packet rate R is presented at X axis, one-way delay absolute error of TOM at Y axis, and payload size at Z axis. From the mentioned figure, we can conclude that TOM accuracy linearly increases with increasing packet rates for all payload sizes. It is also obvious that different payload sizes do not have a significant impact on measurement accuracy of TOM in multi hop wireless mesh network. Also, this pattern was observed for two and three hop counts. Therefore, TOM has same behavior for different hop scenarios.

Figure 4.5: Absolute Error of TOM average one-way delay in single hop scenario

7.7 7.72 7.74 7.76 7.78 7.8 7.82

x 104

1

1.5

2

2.5

3

3.5

4

4.5

5x 10

-3

Packet Number

Tim

e

DPMI

TOM

00.020.040.06

0.08

0.1

0.12

0.14

10

0 25

0 50

0 75

0

10

00

14

00

Rp [pps]

OW

D γ

Pl [Bytes]

26

Figure 4.6 and 4.7 depict the minimum one-way delay calculated by TOM and DPMI for different packet rates and payload sizes in single hop scenario. At first glance we can see that minimum delays calculated by TOM do not follow the same trend as the real values obtained by DPMI do. One explanation for this behavior can be the idea of inter node delay estimation (Equation 2.7). Hence, each delay calculated by TOM doesn’t represent the delay experienced

by that identical packet.

Figure 4.6: TOM Minimum one-way delay in single hop scenario

Figure 4.7: DPMI Minimum one-way delay in single hop scenario

Appendix B provides the average, minimum, maximum, and standard deviation of one-way delays calculated by both TOM and DPMI for different packet rates, payload sizes, and hop counts. In multi hop network, table available in Appendix B show that minimum delay calculated by TOM does not follow same pattern as DPMI. Moreover, we can observe that

0

0.02

0.04

0.06

0.08

10

0 25

0 50

0 75

0

10

00

14

00

Rp [pps]

Min

imu

m O

WD

[s]

Pl [Bytes]

0

0.0005

0.001

0.0015

0.002

0.0025

0.003

10

0 25

0 50

0 75

0

10

00

14

00

Rp [pps[

Min

imu

m O

WD

[s]

Pl [Bytes]

27

maximum one-way delays by TOM have some oddities for different packet rates and payload sizes which need further investigation. By looking at individual traces of TOM and comparing them to DPMI ones, we can observe that DPMI does not report any strange behavior in network while these high peaks are recorded by TOM. Figure 4.8 depicts this strange TOM behavior for payload size 500 bytes and packet rate 1024 pps in single hop scenario. .

Figure 4.8: TOM and DPMI one-way delay trace for packet rate = 1024 pps and payload

size =500 bytes in single hop scenario

If we take table average delay of packet rate 1024 pps and payload size 1400 bytes for

different hop counts from Appendix B, it is visible that TOM achieves acceptable accuracy for single hop scenario, and it maintains the accuracy for other hop counts. Also, this scheme can be observed for other payload sizes in this packet rate. On the other hand, Table B.3 shows average one-way delay of packet rate 128 pps and payload size 1400 bytes where TOM does not have acceptable accuracy for all cases and absolute error value is multiplied with number of traversed hops, because each packet experiences more queuing delay and MAC layer retransmission. This pattern is followed for all payload sizes in this packet rate.

From Appendix B, maximum one-way delay obtained from TOM is always much higher than the equivalent ones by DPMI, and TOM does not show any accuracy. Besides, results of minimum one-way delay form TOM are closer to reference values, but its accuracy is not acceptable.

4.1.3 Discussion

Measurements in previous section show that TOM accuracy improves with increasing the packet rate independent of payload size. To diagnose the source of error, we started with debugging the code with the packet rate 8 Kbps which has the most error margin among all experiments. We noticed that TOM algorithm works well except in inter node delay estimation which introduces strange delays to mh_shared module. TOM uses two timestamps obtained from MadWifi driver to estimate and update inter node delay. The first one is generated at

0 50 100 150 200 250 300 35010

-3

10-2

10-1

100

101

Packet Number

Logarith

mic

scale

Tim

e

DPMI

TOM

28

450 500 550 600 650 700 7501

2

3

4

5

6

7

8

9x 10

-3

Packet Number

Tim

e

DPMI

TOM

ath_tx_start() function of MadWifi just before a packet is put to queue for transmission. The other one is produced at ath_tx_processq() in MadWifi which is called by the scheduler from ath_intr(). ath_intr() is notified whenever hardware receives an interrupt for the acknowledgment of a successfully transmitted packet. As soon as a node receives an acknowledgement, it calculates and updates inter node delay to be used for next packet.

At first, we were suspicious whether driver gets an acknowledgment per packet or not. Therefore, we changed the corresponding value to get an acknowledgement per packet, but the estimation didn’t get any better. Then we found out that sometimes we get an extra timestamp at

ath_tx_start(). The extra timestamp is the source of error for calculation of internode delay, and it results in calculation of inter node delay as the time interval between the moment a packets enters the transmission queue and timestamp of acknowledgment of another packet. So, inter node delay is calculated from disordered acknowledgments. It also explains the reason behind poor accuracy for low packet rates (higher inter packet arrival time).

We have identified the pattern of appearing extra packet timestamp and enhanced TOM accordingly. Figure 4.9 and 4.10 correspondingly depict one-way delay trace of packet rate 128 pps and packet size 250 bytes for original and enhanced version of TOM. By comparing the figures, we can observe that enhanced TOM shows better accuracy in low packet rates. Also, it clear that enhanced TOM shows lower value in comparison with DPMI results due to the fact TOM timestamp packets before it’s handed to network card driver at NF_IP_POST_ROUTING

hook. This modification can improve TOM accuracy, but there are some minor issues with it. We leave this for future work and do not pursue this problem any further.

Figure 4.9: one-way delay for packet rate 128 pps and packet size 250 bytes by original TOM and DPMI

29

Figure 4.10: one-way delay for packet rate 128 pps and packet size 250 bytes by enhanced

TOM and DPMI

450 500 550 600 650 700 7500

0.5

1

1.5

2

2.5

3

3.5

4

4.5x 10

-3

Packet Number

Tim

e

DPMI

TOM

30

4.2 TCP and UDP measurements

This section presents results from the experiments conducted for investigating the impact of TCP and UDP on one-way delay in KAUmesh testbed. In these experiments data rates, packet sizes, and hop counts were varied according to Appendix A.

For each experiment run, we calculated minimum, maximum, average and standard deviation. Since the testbed suffers from problems like queuing and processing delays, we focused on minimum delay for analyzing one-way delay.

The graphical representations of the minimum one-way delays for UDP and TCP flow in single hop scenario are presented in Figure 4.11 and 4.12 as a 3-dimential plot. The data rate is presented at X axis, payload size at Y axis and minimum one-way delay at Z axis.

Figure 4.11: Minimum one-way delay for UDP in Single hop scenario

Figure 4.12: Minimum one-way delay for TCP in Single hop scenario

10

0 25

0 50

0 75

0

10

00

14

00

0

0.0005

0.001

0.0015

0.002

0.0025

0.003

64

12

8

25

6

51

2

10

24

20

48

30

72

40

96

𝑃𝑙 [𝐵𝑦𝑡𝑒𝑠]

Min

OW

D [

s]

Rd [Kbps]

88 23

8 48

8 73

8 98

8

13

88

0

0.0005

0.001

0.0015

0.002

0.0025

0.003

64

12

8

25

6

51

2

10

24

20

48

30

72

40

96

𝑃𝑙 [𝐵𝑦𝑡𝑒𝑠]

Min

OW

D [

s]

Rd [Kbps]

31

As shown in Figure 4.11 and 4.12, it is obvious that one-way delay rises linearly with increasing the packet size. Minimum one-way delay shows identical pattern in both TCP and UDP. This could be due to the fact that TCP and UDP are transport layer protocol, and we do our measurement in lower layer. Therefore, we see similar treatment for both at network layer. In multi hop scenarios, minimum one-way delay results indicate exactly the same pattern for both UDP and TCP flows.

Figures 4.13 and 4.14 respectively show minimum packet delay of rate 128 Kbps and

2048 Kbps in single hop scenario. Payload size is presented at x-axis as well as two minimum delay values of TCP and UDP for each single payload size. Y axis indicates the corresponding minimum one-way delay.

Figure 4.13: Minimum one-way delay for UDP and TCP at Rate 128 Kbps in Single hop

scenario

Figure 4.14: Minimum one-way delay for UDP and TCP at Rate 2048 Kbps in Single hop

scenario

As expected, we can see from Figure 4.13 and 4.14 that an increase in payload size leads to a linear increase in one-way delay for both TCP and UDP. These figures also indicate an

0

0.0005

0.001

0.0015

0.002

0.0025

0.003

100 250 500 750 1000 1400

Min

OW

D [

s]

𝑃𝑙 [𝐵𝑦𝑡𝑒𝑠]

UDP

TCP

0

0.0005

0.001

0.0015

0.002

0.0025

0.003

100 250 500 750 1000 1400

Min

OW

D [

s]

𝑃𝑙 [𝐵𝑦𝑡𝑒𝑠]

UDP

TCP

32

insignificant difference in one-way delay value for TCP and UDP where one-way delay of TCP is slightly greater than UDP tests. This is because IEEE 802.11 employs exponential backoff period and CSMA/CA as channel access mechanism. Consequently, TCP acknowledgments and retransmission make channel busier than UDP scenario. Same pattern has been seen for other data rates in multi hop scenarios.

Figure 4.15 and 4.16 illustrates variation of one-way delay value in different hop count

scenarios while UDP and TCP traffic were generated at data rate of 1024 Kbps. As expected, same linear increase in one-way delay is apparent for all three scenarios as payload size increases. These figures also demonstrate direct relation between number of hops and one-way delay value. It is clear that one-way delay results are approximately multiplied with number of traversed hops. This is due to the fact that packet experience more queuing, processing, propagation, and transmission time while it traverse through number of hops. Interestingly, one-way delay is much more sensitive to packet size in multi hop, compared to single hop scenarios.

Figure 4.15: Minimum one-way delay for UDP at Rate 1024 Kbps in different hop counts.

Figure 4.16: Minimum one-way delay for TCP at Rate 1024 Kbps in different hop counts

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

100 250 500 750 1000 1400

Min

OW

D [

s]

𝑃𝑙 [𝐵𝑦𝑡𝑒𝑠]

Single Hop

Multi Hop2

Multi Hop 3

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

100 250 500 750 1000 1400

Min

OW

D [

s]

𝑃𝑙 [𝐵𝑦𝑡𝑒𝑠]

Single Hop

Multi Hop2

Multi Hop 3

33

Figure 4.17 and size 4.18 respectively show one-way delay of packet size 250 and 1000 bytes for both TCP and UDP protocols in single hop tests. According to Figure 4.17, and considering overall TCP and UDP for payload size 250, with increasing the data rate one-way delay degrades for UDP protocol, though we cannot see any significant variation in one-way delay for TCP protocol. Furthermore, UDP has better performance than TCP with respect to different data rates. This low performance can be explained by the existence of acknowledgment packet while wireless mesh backbone uses CSMA/CA over half duplex links. Therefore, TCP delay increases compared to UDP.

Figure 4.17: Minimum one-way delay for UDP and TCP, size 250 bytes in Single hop

scenario

Figure 4.18: Minimum one-way delay for UDP and TCP, size 1000 bytes in Single hop

scenario In Figure 4.18, one-way delay does not follow the same pattern which was shown on

Figure 4.17, but it indicates the better performance in high rates similar to Figure 4.17.To investigate the enhanced performance in high data rates rather than low data rates, we analyzed network traces of data rate 64 and 2048 Kbps with the same packet size of 250 bytes as shown in

0

0.0001

0.0002

0.0003

0.0004

0.0005

0.0006

0.0007

0.0008

0.0009

64 128 256 512 1024 2048 3072 4096

Min

OW

D [

s]

Rd [Kbps]

UDP

TCP

0

0.0002

0.0004

0.0006

0.0008

0.001

0.0012

0.0014

0.0016

0.0018

0.002

64 128 256 512 1024 2048 3072 4096

Min

OW

D [

s]

Rd [Kbps]

UDP

TCP

34

Figure 4.19.We observed that at 64 Kbps data rate, most of the packets have higher one-way delay compared to 2048 Kbps data rate. This could be due to a queue that is released, and minimum OWD take place right after it. In low data rate spike looks sharper while it differs in high data rate because of lower inter departure time between packets. This pattern is repeated for the whole packets in each experiment.

Figure 4.19: One-way delay trace of UDP for data rate 64 and 2048 Kbps with packet size

250 bytes in Single hop scenario Appendix C demonstrates average, maximum, minimum and standard deviation at

different data rates, packet sizes, and hop counts. It can be clearly seen that there are some oddities at maximum one-way delay column in multi hop scenarios. By analyzing each individual experiment trace, we observed high peaks delay for number of packets. We repeated the experiments to minimize any unwanted interference that caused this high peaks, but the same peaks were existed in experiment traces which need more investigation. According to average column of mentioned tables, average one-way delay results follow the same pattern that minimum one-way delay does, with respect to different packet sizes and data rates.

If we take the tables available in Appendix C into consideration, it can be clearly seen that payload size 100 bytes for UDP and 88 bytes for TCP have the lowest minimum OWD with different data rates in different hop counts scenarios. For UDP flow in single hop scenario, the lowest minimum OWD was resulted from combination of payload size 100 bytes and data rate 1024 Kbps, while the lowest minimum OWD was resulted from combination of payload size 88 and data rate 256 Kbps for TCP flow in single hop scenario. In two hops scenario, the lowest minimum OWD was observed at payload size 100 bytes and data rate 2048 Kbps for UDP experiments, payload size 88 bytes and data rate 64 Kbps for TCP experiment runs. But the lowest minimum OWD in three hops scenario was observed at data rate 128 Kbps for both UDP and TCP. Furthermore, there is no noticeable difference between 95% and 99% confidence intervals. But there is a significant difference between confidence interval and standard deviation values due to the existence of a few sharp spikes in network trace of each experiment runs.

1600 1610 1620 1630 1640 1650 1660 1670 1680 1690 17000.5

1

1.5

2

2.5

3

3.5

4x 10

-3

Packet Number

Tim

e

Rate=64 Kbps

Rate=2048 Kbps

35

5 CONCLUSION & FUTURE WORK

In this thesis work, firstly we evaluated measurement accuracy of TOM in comparison with DPMI results as reference OWD values under different packet rates, payload sizes, and hop counts. For this purpose, we designed an experimental setup in KAUmesh testbed where both TOM and DPMI could deliver same measurable delay. Also, we modified source codes based on the requirements. We observed that TOM does not show acceptable accuracy in low packet rates. Then, we investigated the main source of miscalculation in TOM, and presented preliminary enhanced version of TOM. According to our outcomes TOM accuracy can be improved and provide delay statistics for routing, packet scheduling, and gateway selection. Finally, we investigated the impacts of TCP and UDP on OWD in wireless mesh networks with changing the data rate, packet size, and hop count. We recognized that TCP has scarcely higher one-way delay compared to UDP, and UDP delay performance increases with data rate. Moreover, both TCP and UDP are more sensitive to packet size in multi hop rather than single hop wireless mesh network.

To sum up and answer the first research question “Is it possible to compare one-way delay

estimation from TOM with the similar estimation from DPMI? If not, what

modifications/adjustments are required to allow us to compare TOM and DPMI results? ”, we designed an experimental setup (Chapter 3) to deliver the same measureable delay for both TOM and DPMI. To measure the same amount of delay as DPMI is capable of to do, some new features were added to TOM by kernel programming. Also we calculated minimum one-way delay theoretical values to TOM and DPMI results. Hence, mh_shared and mh_measurement modules of TOM were modified accordingly to fulfill these requirements.

To answer the second research question “How is TOM measurement accuracy affected by

different traffic patterns (combination of packet rate, payload size, and hop count)? And why do

they have effect on it?”, we indicated that TOM does not have acceptable accuracy in low packet rates and sizes, and it becomes more accurate with increasing them. TOM shows reasonable accuracy in high packet rates. Different payload sizes do not have a noticeable impact on TOM measurement accuracy, although TOM accuracy is more sensitive to packet rate rather than packet size. On the other hand, TOM measurement error margin rises with each passing hop. All of these faulty measurements occur because according to IEEE 802.11 some packets such as multicast packets do not require MAC level acknowledgments. Therefore, MadWifi passes timestamp of a packet arrival to transmission queue with acknowledgement of another packet. Additionally, based on our findings TOM was modified in order to achieve higher accuracy in low packet rates. The results from preliminary improved version TOM shows acceptable accuracy in both low and high packet rates, although it has a few extremely high peaks.

To answer the third research question “What is the impact of transport layer protocols

(TCP and UDP) on one-way delay obtained by DPMI with changing traffic patterns

(combination data rate, packet size, and hop count) in wireless mesh networks?”, we observed that UDP has slightly better performance compared to TCP in sense of one-way delay. Also, we found out that UDP has a better one-way delay at high data rate compared to low rates for a fixed packet size. These are due to the fact that IEEE 802.11 utilizes exponential backoff period and CSMA/CA as channel access scheme. Consequently, TCP acknowledgments and retransmission make channel busier than UDP scenario. Both TCP and UDP are more sensitive to packet size in multi hop rather than in single hop.

Our findings for TOM can be used to achieve higher accuracy measurement in wireless

mesh networks. Preliminary TOM modification reveals that TOM can be fixed with current MadWifi or ath5k as an alternative driver which has open source HAL and supports more chipsets. Furthermore, TCP and UDP investigation can be extended with taking loss as a metric.

36

6 REFERENCE

[1] R. Bruno, M. Conti, E. Gregori, "Mesh networks: commodity multihop ad hoc networks," Communications Magazine, IEEE , vol.43, no.3, pp. 123- 131, March 2005. [2] J. Corral, G. Texier, and L. Toutain, “End-to-end active measurement architecture in IP networks (SATURNE),” in Proc. PAM Workshop , pp. 241–247, 2003. [3] L. De Vito, S. Rapuano, and L. Tomaciello, “One-Way Delay Measurement: State of the Art,” Instrumentation and Measurement, IEEE Transactions on, vol. 57, no. 12, pp. 2742-2750, 2008. [4] D. Niculescu et al., “Performance of VoIP in a 802.11 wireless mesh network,” IN PROC.

IEEE INFOCOM, pp. 1-11, 2006. [5] U. Hofmann, T. Pfeiffenberger, and B. Hechenleitner, "One-way-delay measurements with CM toolset," Performance, Computing, and Communications Conference, 2000. IPCCC '00.

Conference Proceeding of the IEEE International , vol., no., pp.41-47, Feb 2000. [6] G. Almes, S. Kalidindi, and M. Zekauskas, A Round-Trip Delay Metric for IPPM, RFC 2681, Sep. 1999.

[7] A. Hernandez and E. Magana, “One-way Delay Measurement and Characterization,” in

Networking and Services, International conference on, , vol. 0, p. 114-120, 2007. [8] N.Simar, “A first step in the World of IP QoS, ” DANTE in Print 43, Sept. 2001. [9] W. U. Zaman, S. A. Ahmad, A. Abbas, and A. Qadeer, “Modeling of processor delay and

overall reduction in network latencies for real time,” in Proc. I SCON Emerging Technol. ,

vol. 1, pp. 76–79, Aug. 2002. [10] J. Li, Z. Li, and P. Mohapatra, “Adaptive per hop differentiation for end-to-end delay assurance in multihop wireless networks,” Ad Hoc Networks, vol. 7, no. 6, pp. 1169-1182, Aug. 2009. [11] K. Stangherlin et al., “One-way delay measurement in wired and wireless mobile full-mesh networks,” in Wireless Communications and Networking Conference (WCNC), 2011

IEEE, pp. 1044-1049, 2011. [12] P. Arlos, M. Fiedler, and A. A. Nilsson, “A Distributed Passive Measurement Infrastructure,” in Passive and Active Network Measurement, vol. 3431, C. Dovrolis, Ed. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 215-227, 2005. [13] P. Arlos, On the Quality of Computer Network Measurements. Ph.D. Thesis, Blekinge Institute of Technology, 2005. [14] B. Staehle, D. Staehle, R. Pries, M. Hirth, P. Dely, and A. J. Kassler, “Measuring One-Way Delay in Wireless Mesh Networks - An Experimental Investigation,” in The 4

th ACM

37

International Workshop on Performance Monitoring, Measurement and Evaluation of

Heterogeneous Wireless and Wired Networks (PM2HW2N), (Tenerife, Canary Island, Spain), October 2009. [15] P. Dely, and A. Kassler. “KAUMesh - A Multi-Radio Multi-Channel Mesh Testbed,” In ADHOC ’09, Uppsala, Sweden, May 2009. [16] R. Kommalapati, Performance of 3G Data Services Over Mobile Network in Sweden. Master's thesis, Blekinge Institute of Technology, 2010. [17] J. Camp and E. Knightly, “The IEEE 802.11s Extended Service Set Mesh Networking

Standard,” Communications Magazine, IEEE, vol. 46, no. 8, pp. 120-126, 2008. [18] “IEEE Standard for Information Technology-Telecommunications and Information Exchange Between Systems-Local and Metropolitan Area Networks-Specific Requirements - Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications.” 2007. [19] I. F. Akyildiz and X. Wang, “A survey on wireless mesh networks,” Communications

Magazine, IEEE, vol. 43, no. 9, p. S23-S30, 2005. [20] The Netfilter project homepage. [Online]. Available: http://www.netfilter.org/ [Accessed: August 15, 2011]. [21] Endace Meacurement Systems. [Online]. Available: http://www.endace.com/ [Accessed: August 15, 2011]. [22] V. Gupta, S.V. Krishnamurthy, and M. Faloutsos, “Improving the Performance of TCP in

the Presence of Interacting UDP Flows in Ad Hoc Networks,” IFIP Networking, May 2004. [23] G. Holland, N. Vaidya, “Link failure and congestion: analysis of TCP performance over mobile ad hoc networks,” in ACM Annual International Conference on Mobile Computing

and Networking (MOBICOM), pp. 219–230, 1999. [24] C. Rohner, E. Nordstr¨om, P. Gunningberg, and C. Tschudin, “Interactions between TCP, UDP and routing protocols in wireless multi-hop ad hoc netwrorks,” in Proc. IEEE ICPS

Workshop on Multi-hop Ad hoc Networks: from theory to reality (REALMAN’05), Santorini, Greece, Jul. 2005. [25] Gateworks, Cambria GW2358-4. [Online]. Available: http://www.gateworks.com/products/cambria/datasheets/gw2358-4ds.pdf. [Accessed: October 15, 2011]. [26] J.B. Andersen, T.S. Rappaport, S. Yoshida, "Propagation measurements and models for wireless communications channels," Communications Magazine, IEEE , vol.33, no.1, pp.42-49, Jan 1995

38

Appendix A

39

Rate[Kbps] payload size[bytes] Waiting time[ms] Number of packets

64

100 12207 6553 250 30517 2621 500 61035 1310 750 91552 873

1000 122070 655 1400 170898 468

128

100 6103 13107 250 15258 5242 500 30517 2621 750 45776 1747

1000 61035 1310 1400 85449 936

256

100 3051 26214 250 7629 10485 500 15258 5242 750 22888 3495

1000 30517 2621 1400 42724 1872

512

100 1525 52428 250 3814 20971 500 7629 10485 750 11444 6990

1000 15258 5242 1400 21362 3744

1024

100 762 104857 250 1907 41943 500 3814 20971 750 5722 13981

1000 7629 10485 1400 10681 7489

2048

100 381 209715 250 953 83886 500 1907 41943 750 2861 27962

1000 3814 20971 1400 5340 14979

3072

100 254 314572 250 635 125829 500 1271 62914 750 1907 41943

1000 2543 31457 1400 3560 22469

4096

100 190 419430 250 476 167772 500 953 83886 750 1430 55924

1000 1907 41943

40

1400 2670 29959 Rate[Kbps] payload size[bytes] Waiting time[ms] Number of packets

64

88 7447 10742 238 2754 29052 488 1343 59570 738 888 90087 988 663 120605

1388 472 169433

128

88 14895 5371 238 5507 14526 488 2686 29785 738 1776 45043 988 1327 60302

1388 944 84716

256

88 29789 2685 238 11014 7263 488 5372 14892 738 3552 22521 988 2653 30151

1388 1889 42358

512

88 59578 1342 238 22029 3631 488 10744 7446 738 7104 11260 988 5307 15075

1388 3777 21179

1024

88 119156 671 238 44058 1815 488 21487 3723 738 14208 5630 988 10613 7537

1388 7555 10589

2048

88 238313 335 238 88116 907 488 42974 1861 738 28417 2815 988 21226 3768

1388 15109 5294

3072

88 357469 223 238 132173 605 488 64462 1241 738 42625 1876 988 31839 2512

1388 22664 3529

4096

88 476625 167 238 176231 453 488 85949 930 738 56833 1407 988 42452 1884

1388 30218 2647

41

Appendix B

42

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 DPMI 0.74715 3.766895 0.579834 0.348106 0.027467 0.036098 593

TOM 123.1651 130.5699 74.12391 5.262651 0.41525 0.545732 593

250 DPMI 0.937094 3.74502 0.78553 0.307501 0.024263 0.031888 593

TOM 122.7602 130.5884 78.51385 6.358534 0.501721 0.659373 593

500 DPMI 1.310249 4.400432 1.143396 0.338068 0.026675 0.035057 593

TOM 122.7573 130.5783 78.61514 6.380489 0.503454 0.66165 593

750 DPMI 1.67842 4.768074 1.492262 0.356037 0.028093 0.036921 593

TOM 123.1746 126.8951 79.01311 5.272529 0.41603 0.546756 593

1000 DPMI 2.017767 4.980326 1.862884 0.317494 0.025052 0.032924 593

TOM 123.0006 127.8344 70.75549 5.773414 0.455552 0.598697 593

1400 DPMI 2.626131 5.630553 2.444148 0.337752 0.02665 0.035025 593

TOM 122.7064 126.8818 35.09196 7.167071 0.565519 0.743218 593

Tw

o H

ops

100 DPMI 1.311904 4.252076 1.159251 0.417639 0.032954 0.043309 593

TOM 347.7644 686.428 203.2423 136.9372 10.80505 14.20024 593

250 DPMI 1.72222 4.606545 1.582861 0.403627 0.031848 0.041856 593

TOM 349.1151 737.0968 244.7488 136.7391 10.78942 14.17971 593

500 DPMI 2.440074 5.385101 2.285779 0.436051 0.034407 0.045218 593

TOM 347.4486 686.4643 202.506 137.4957 10.84912 14.25816 593

750 DPMI 3.159762 6.200195 2.990544 0.423879 0.033446 0.043956 593

TOM 347.7207 687.7571 201.3965 137.5563 10.85391 14.26445 593

1000 DPMI 3.856835 6.806433 3.698051 0.413425 0.032621 0.042872 593

TOM 347.3514 681.7848 206.8637 136.2375 10.74985 14.12769 593

1400 DPMI 4.979725 8.393407 4.83042 0.389171 0.030708 0.040357 593

TOM 349.6845 1173.663 206.1931 141.1033 11.13378 14.63227 593

Thr

ee H

ops

100 DPMI 1.987409 4.8545 1.728118 0.506287 0.039949 0.052501 593

TOM 572.2813 1353.011 337.7825 272.7708 21.52303 28.28606 593

250 DPMI 2.65491 5.53143 2.364755 0.541344 0.042715 0.056137 593

TOM 576.2125 1253.261 330.8397 274.2684 21.6412 28.44135 593

500 DPMI 3.710608 6.630898 3.419638 0.471212 0.037181 0.048864 593

TOM 575.8061 1255.262 335.0901 272.9896 21.54029 28.30875 593

750 DPMI 4.791416 7.684112 4.481316 0.486808 0.038412 0.050481 593

TOM 571.7902 1258.317 334.4703 271.5781 21.42892 28.16237 593

1000 DPMI 5.901799 8.80611 5.539 0.558231 0.044047 0.057888 593

TOM 574.6812 1263.535 329.1718 275.6557 21.75066 28.58521 593

1400 DPMI 7.608978 10.54305 7.237852 0.546021 0.043084 0.056622 593

TOM 575.4351 1254.947 323.4134 272.8296 21.52766 28.29215 593

Table B.1: TOM and DPMI one-way delay statistics = 8 pps.

43

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 DPMI 0.673888 3.70425 0.519216 0.297441 0.008166 0.010732 5073

TOM 15.66998 21.68133 8.675766 1.718696 0.047183 0.06201 5073

250 DPMI 0.931987 4.052997 0.75233 0.311259 0.008545 0.01123 5073

TOM 15.65211 21.77257 7.391112 1.87818 0.051562 0.067764 5073

500 DPMI 1.30963 4.410148 1.106322 0.31789 0.008727 0.011469 5073

TOM 15.6767 21.70996 6.961891 1.816809 0.049877 0.065549 5073

750 DPMI 1.630048 4.644752 1.463652 0.312784 0.008587 0.011285 5073

TOM 15.68568 19.15009 7.771724 1.822542 0.050034 0.065756 5073

1000 DPMI 1.977881 5.025149 1.818061 0.31588 0.008672 0.011397 5073

TOM 15.69252 19.43364 7.483978 1.716405 0.047121 0.061927 5073

1400 DPMI 2.591501 5.534709 2.389074 0.306181 0.008406 0.011047 5073

TOM 15.72106 19.46174 8.907531 1.778728 0.048832 0.064176 5073

Tw

o H

ops

100 DPMI 1.357999 7.271886 1.127422 0.684047 0.018779 0.02468 5073

TOM 46.51411 520.4435 25.72823 68.25997 1.873945 2.462781 5073

250 DPMI 1.750175 4.735589 1.556039 0.4228 0.011607 0.015254 5073

TOM 46.30859 522.3014 22.39639 68.0127 1.867157 2.45386 5073

500 DPMI 2.461332 5.464137 2.255082 0.420887 0.011555 0.015185 5073

TOM 46.39418 522.0063 23.42698 68.17015 1.871479 2.459541 5073

750 DPMI 3.157254 6.124496 2.958536 0.426438 0.011707 0.015386 5073

TOM 46.52585 520.5344 23.02813 67.9554 1.865584 2.451793 5073

1000 DPMI 3.866532 6.907284 3.666163 0.423667 0.011631 0.015286 5073

TOM 46.56575 524.7254 23.78162 68.61046 1.883567 2.475427 5073

1400 DPMI 5.01453 8.026838 4.806161 0.428159 0.011754 0.015448 5073

TOM 46.37521 525.2847 21.88607 67.55124 1.854488 2.437211 5073

Thr

ee H

ops

100 DPMI 2.075648 5.002558 1.766682 0.513389 0.014094 0.018523 5073

TOM 77.00577 1026.201 31.88702 135.5065 3.720068 4.888998 5073

250 DPMI 2.7272 5.811632 2.435982 0.529671 0.014541 0.01911 5073

TOM 77.28009 1025.417 33.05632 136.3149 3.742262 4.918166 5073

500 DPMI 3.790405 6.738901 3.507555 0.51537 0.014148 0.018594 5073

TOM 77.00169 1030.199 32.59044 135.2376 3.712687 4.879297 5073

750 DPMI 4.862822 10.79446 4.617453 0.534845 0.014683 0.019297 5073

TOM 77.10187 1038.566 34.79451 135.4168 3.717607 4.885764 5073

1000 DPMI 5.932003 15.94853 5.663991 0.557526 0.015306 0.020115 5073

TOM 76.81491 1031.997 35.08585 132.7801 3.645219 4.79063 5073

1400 DPMI 7.657094 15.67388 7.386089 0.602204 0.016534 0.021729 5072

TOM 77.33966 1021.297 36.25305 135.3733 3.716777 4.884673 5072

Table B.2: TOM and DPMI one-way delay statistics = 64 pps

44

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 DPMI 0.765301 4.095316 0.524402 0.352123 0.006847 0.008998 10137

TOM 7.885202 10.09716 2.942788 1.6056 0.031219 0.041029 10137

250 DPMI 1.022094 4.472196 0.745535 0.380838 0.007406 0.009733 10135

TOM 7.946924 9.722748 3.044892 1.57049 0.030539 0.040135 10135

500 DPMI 1.453229 5.197883 1.08707 0.475307 0.009242 0.012146 10136

TOM 8.027054 9.836008 4.369682 1.509902 0.02936 0.038585 10136

750 DPMI 1.90802 5.898356 1.454234 0.579464 0.01127 0.014812 10131

TOM 8.10107 10.86579 3.953996 1.45463 0.028292 0.037182 10131

1000 DPMI 2.346284 7.805288 1.807809 0.702612 0.013662 0.017955 10136

TOM 8.194751 10.26067 4.129002 1.39981 0.027219 0.035772 10136

1400 DPMI 3.045744 7.571399 2.37912 0.890291 0.017311 0.022751 10136

TOM 8.324245 10.31226 3.818833 1.315158 0.025573 0.033608 10136

Tw

o H

ops

100 DPMI 1.388283 4.774928 0.992239 0.474478 0.009225 0.012123 10139

TOM 24.87124 16448.27 7.9608 302.1274 5.873909 7.719624 10139

250 DPMI 1.86148 5.331218 1.539707 0.498549 0.009695 0.012741 10135

TOM 23.51433 507.9928 8.203779 48.84142 0.949753 1.248188 10135

500 DPMI 2.688448 6.427646 2.234101 0.591249 0.011497 0.015109 10136

TOM 23.64733 510.6246 9.362052 49.06178 0.953992 1.253757 10136

750 DPMI 3.457057 11.59835 2.955139 0.701708 0.013646 0.017934 10134

TOM 23.76671 509.9277 9.657619 49.07239 0.954292 1.254152 10134

1000 DPMI 4.200676 9.846807 3.657103 0.797787 0.01551 0.020384 10139

TOM 23.90027 509.9785 10.33425 48.9844 0.952346 1.251595 10139

1400 DPMI 5.510104 17.56102 4.793585 1.085104 0.021121 0.027758 10115

TOM 23.85924 512.9544 10.325 48.66403 0.947237 1.24488 10115

Thr

ee H

ops

100 DPMI 2.10624 5.706191 1.83028 0.455083 0.008852 0.011633 10130

TOM 38.99554 1014.042 11.99689 98.26634 1.911324 2.511905 10130

250 DPMI 2.808694 7.481694 2.471864 0.572003 0.01113 0.014627 10123

TOM 39.23313 1014.301 13.28749 98.56852 1.917863 2.520499 10123

500 DPMI 3.996427 9.90504 3.531456 0.816032 0.015879 0.020869 10121

TOM 39.44362 1013.748 13.51568 98.35134 1.913826 2.515193 10121

750 DPMI 5.165514 11.58798 4.608154 0.962086 0.018729 0.024614 10113

TOM 39.34858 1013.23 14.61012 97.19175 1.892007 2.486519 10113

1000 DPMI 6.261345 12.22396 5.6656 0.90925 0.017699 0.023261 10114

TOM 39.1115 1017.72 14.88831 94.85916 1.846508 2.426723 10114

1400 DPMI 8.107911 15.31571 7.386804 1.115921 0.021751 0.028586 10087

TOM 36.27588 1009.317 16.65858 63.85083 1.244565 1.635636 10087

Table B.3: TOM and DPMI one-way delay statistics = 128 pps

45

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 DPMI 0.944826 4.959226 0.508368 0.417881 0.005756 0.007564 20226

TOM 2.788879 5.912354 0.910746 0.585441 0.008063 0.010597 20226

250 DPMI 1.350061 5.624294 0.733793 0.534923 0.007365 0.00968 20238

TOM 3.04675 5.464018 1.472682 0.517108 0.00712 0.009357 20238

500 DPMI 2.054044 7.535696 1.084566 0.857414 0.011809 0.015519 20228

TOM 3.473528 10.14312 2.346999 0.452341 0.00623 0.008187 20228

750 DPMI 2.622275 7.946193 1.448393 1.004178 0.013829 0.018174 20232

TOM 3.959418 6.829094 2.684605 0.318234 0.004382 0.00576 20232

1000 DPMI 3.229841 10.88232 1.80614 1.246224 0.017169 0.022564 20216

TOM 4.38995 11.7782 3.134263 0.327669 0.004514 0.005933 20216

1400 DPMI 4.246648 12.76457 2.376199 1.631812 0.022471 0.029532 20233

TOM 5.092027 12.3608 3.44116 0.409233 0.005635 0.007406 20233

Tw

o H

ops

100 DPMI 1.729228 6.670356 1.10811 0.662476 0.009127 0.011995 20214

TOM 10.98265 502.6751 3.363963 36.74495 0.506246 0.665321 20214

250 DPMI 2.526223 9.501875 1.532436 0.979107 0.013512 0.017757 20148

TOM 11.06466 505.107 3.970284 34.65747 0.478267 0.628549 20148

500 DPMI 3.772076 12.93165 2.247274 1.475093 0.020373 0.026775 20114

TOM 10.37591 506.4335 5.033498 24.47892 0.33809 0.444325 20114

750 DPMI 4.990465 21.1525 2.954185 2.012411 0.027819 0.036561 20078

TOM 11.05272 503.7376 5.880114 25.48828 0.352346 0.463061 20078

1000 DPMI 6.276334 16.68376 3.628552 2.526352 0.034964 0.04595 20032

TOM 11.90889 504.507 6.649252 27.27673 0.377501 0.496121 20032

1400 DPMI 443.8314 484.6882 26.37994 45.52319 0.696742 0.915674 16375

TOM 449.3231 970.8885 22.48301 57.4172 0.878782 1.154916 16375

Thr

ee H

ops

100 DPMI 2.508977 6.474018 1.835346 0.713483 0.009829 0.012917 20218

TOM 18.9774 1003.481 4.429939 71.80975 0.989247 1.300091 20218

250 DPMI 3.446614 9.981275 2.456963 1.00717 0.013887 0.018251 20182

TOM 18.86518 1003.042 5.171183 68.20706 0.940453 1.235965 20182

500 DPMI 4.928383 12.85589 3.525436 1.459054 0.020176 0.026516 20065

TOM 18.77659 1004.117 6.310374 61.09363 0.844821 1.110283 20065

750 DPMI 6.432806 16.75224 4.598081 1.946032 0.026914 0.035371 20060

TOM 18.22242 1000.859 8.157193 51.1078 0.706822 0.928922 20060

1000 DPMI 7.981825 20.32572 5.669534 2.414596 0.033461 0.043975 19980

TOM 18.53144 998.6792 9.033954 47.63247 0.660074 0.867485 19980

1400 DPMI 391.0208 427.4353 19.65469 54.67925 0.792392 1.041379 18268

TOM 401.0724 1388.715 25.1763 72.00351 1.043449 1.371324 18268

Table B.4: TOM and DPMI one-way delay statistics = 256 pps

46

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 DPMI 1.390271 6.522536 0.516236 0.634524 0.006206 0.008157 40128

TOM 1.418691 5.495267 0.84639 0.42336 0.004141 0.005442 40128

250 DPMI 2.067824 13.00889 0.717044 1.020034 0.009977 0.013112 40132

TOM 2.513693 8.341748 1.385772 0.593296 0.005803 0.007626 40132

500 DPMI 3.082028 14.05674 1.084983 1.382339 0.013521 0.017769 40130

TOM 4.011721 8.549951 2.626564 0.560273 0.00548 0.007202 40130

750 DPMI 4.138676 11.94179 1.443207 1.86018 0.018194 0.023911 40131

TOM 5.601372 9.085574 3.391533 0.610106 0.005967 0.007842 40131

1000 DPMI 5.22813 21.66474 1.800716 2.365607 0.023135 0.030404 40141

TOM 7.071586 17.82507 4.866302 0.708365 0.006928 0.009104 40141

1400 DPMI 205.5649 223.7743 2.469231 17.99553 0.186037 0.244494 35920

TOM 206.9497 221.3245 16.36037 18.89089 0.195293 0.256659 35920

Tw

o H

ops

100 DPMI 2.522159 8.474529 1.107574 1.081087 0.010586 0.013913 40039

TOM 5.940513 501.0804 2.10872 20.68382 0.202538 0.26618 40039

250 DPMI 4.265265 20.52516 1.509011 2.232627 0.02189 0.028769 39936

TOM 8.663382 513.4263 3.706465 20.57898 0.201771 0.265172 39936

500 DPMI 26.89867 266.3993 2.238512 45.52092 0.447869 0.5886 39660

TOM 31.51516 659.2652 4.762015 49.52108 0.487226 0.640324 39660

750 DPMI 264.0835 382.8831 3.238541 47.29229 0.554679 0.728972 27901

TOM 268.8671 1401.2 18.48045 55.93746 0.656076 0.86223 27901

1000 DPMI 341.0808 467.0732 3.721178 54.88107 0.7304 0.959909 21664

TOM 346.9997 1146.914 15.64045 66.04984 0.879043 1.155258 21664

1400 DPMI 477.2864 650.0847 9.315193 78.85217 1.231972 1.619085 15713

TOM 483.8465 1128.914 30.55629 94.79846 1.481114 1.946513 15713

Thr

ee H

ops

100 DPMI 3.348117 9.189367 1.822293 1.153974 0.011292 0.014841 40092

TOM 9.973343 999.6237 3.092002 41.37415 0.404873 0.532093 40092

250 DPMI 4.803022 13.47345 2.456427 1.731977 0.016955 0.022283 40061

TOM 11.55362 1000.774 4.727462 37.8684 0.37071 0.487195 40061

500 DPMI 7.633396 21.35772 3.528595 3.005591 0.029439 0.038689 40018

TOM 13.93461 754.9095 7.322569 24.56782 0.240634 0.316247 40018

750 DPMI 238.3487 257.7037 6.480098 21.40681 0.236553 0.310883 31435

TOM 244.8354 980.0194 15.61992 36.72589 0.405834 0.533356 31435

1000 DPMI 303.3444 328.1961 8.42841 15.6295 0.193086 0.253758 25146

TOM 312.0154 1055.031 35.01957 39.63973 0.489708 0.643585 25146

1400 DPMI 414.0324 444.195 12.08371 30.13549 0.434004 0.570379 18497

TOM 424.4135 1164.537 32.83702 60.52326 0.871642 1.145532 18497

Table B.5: TOM and DPMI one-way delay statistics = 512 pps

47

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 DPMI 2.41411 10.75476 0.505746 1.284057 0.008944 0.011754 79158

TOM 2.83166 8.389173 1.053484 0.73263 0.005103 0.006706 79158

250 DPMI 3.435203 13.48186 0.72819 1.761369 0.012269 0.016124 79151

TOM 4.827152 12.64646 1.708481 0.728135 0.005072 0.006666 79151

500 DPMI 5.897167 20.42633 1.091361 2.921233 0.020351 0.026745 79130

TOM 7.730799 18.5973 4.012862 1.309529 0.009123 0.011989 79130

750 DPMI 120.7402 131.9962 1.592356 5.586563 0.044567 0.058571 60337

TOM 121.8554 129.4998 16.59484 6.323195 0.050444 0.066294 60337

1000 DPMI 154.764 170.6467 1.81849 6.744235 0.060426 0.079413 47830

TOM 155.0603 166.7203 8.619487 7.975153 0.071454 0.093907 47830

1400 DPMI 209.9866 223.0013 2.68421 11.33796 0.117257 0.154102 35892

TOM 210.8468 223.6778 16.79762 15.68889 0.162254 0.213238 35892

Tw

o H

ops

100 DPMI 4.617702 16.11149 1.095533 2.402811 0.01676 0.022027 78930

TOM 7.697175 506.8746 2.736198 14.59783 0.101824 0.133819 78930

250 DPMI 111.5125 172.2575 7.459343 15.19072 0.117101 0.153897 64620

TOM 113.3139 653.5248 10.78945 24.05553 0.185438 0.243707 64620

500 DPMI 180.0229 265.304 6.816149 18.4771 0.178789 0.234969 41004

TOM 182.9078 735.1627 11.91235 31.21899 0.302083 0.397005 41004

750 DPMI 270.8081 383.1368 7.988572 36.77562 0.433793 0.570101 27585

TOM 275.8462 929.929 17.45832 47.34392 0.558453 0.733932 27585

1000 DPMI 358.6873 472.2817 6.687641 47.87955 0.654784 0.860532 20516

TOM 364.254 2048.43 18.85224 64.46106 0.881547 1.158549 20516

1400 DPMI 426.0615 607.0697 9.795725 38.13665 0.562067 0.738681 17661

TOM 430.7817 1021.257 29.81154 59.15744 0.871876 1.14584 17661

Thr

ee H

ops

100 DPMI 5.173516 17.67534 1.831055 2.203813 0.015368 0.020197 78970

TOM 8.911392 997.8226 3.212388 20.8182 0.145176 0.190793 78970

250 DPMI 106.4306 126.1287 3.05342 5.935775 0.044418 0.058376 68576

TOM 110.0455 722.5267 16.55606 26.39396 0.197511 0.259573 68576

500 DPMI 173.7465 190.905 4.606664 9.529376 0.090255 0.118615 42800

TOM 178.4399 740.5386 12.77034 30.20382 0.286066 0.375955 42800

750 DPMI 245.7829 281.8297 4.657626 14.97471 0.168781 0.221815 30215

TOM 252.4572 1435.459 15.52639 38.31495 0.43185 0.567547 30215

1000 DPMI 309.8858 332.4345 7.12961 17.75732 0.223303 0.293469 24268

TOM 318.4214 1346.705 20.74329 47.14825 0.592901 0.779204 24268

1400 DPMI 411.0348 437.7846 13.02248 24.69583 0.355971 0.467826 18465

TOM 421.1631 1397.966 33.84077 59.78868 0.861808 1.132607 18465

Table B.6: TOM and DPMI one-way delay statistics = 1024 pps

48

Appendix C

49

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 UDP 0.705506 3.699601 0.594974 0.309552 0.007501 0.009857 6543 88 TCP 0.73689 3.78561 0.636578 0.303892 0.004892 0.006429 14826

250 UDP 0.953508 3.955722 0.852704 0.319282 0.012247 0.016095 2611

238 TCP 0.97869 3.890455 0.879169 0.325089 0.012141 0.015956 2754

500 UDP 1.300048 4.25458 1.197755 0.307676 0.016725 0.021981 1300 488 TCP 1.336442 4.334867 1.252532 0.246373 0.013177 0.017317 1343 750 UDP 1.706499 4.633844 1.595795 0.310655 0.020714 0.027223 864 738 TCP 1.714992 4.62544 1.627267 0.269827 0.017737 0.023311 889

1000 UDP 2.041404 5.083919 1.936674 0.310252 0.023943 0.031467 645 988 TCP 2.097695 5.01436 1.980782 0.262942 0.02 0.026284 664

1400 UDP 2.714217 5.645514 2.56002 0.367196 0.033268 0.043721 468

1388 TCP 2.708396 5.515516 2.571106 0.265112 0.023892 0.031399 473

Tw

o H

ops

100 UDP 1.272288 4.291892 1.037002 0.434819 0.010536 0.013846 6543 88 TCP 1.316101 4.333913 1.098931 0.43327 0.009872 0.012974 7400

250 UDP 1.668154 4.711807 1.462936 0.425579 0.016324 0.021453 2611 238 TCP 1.743042 4.74155 1.522243 0.423147 0.015832 0.020807 2744

500 UDP 2.363252 5.405605 2.164304 0.407693 0.022162 0.029126 1300

488 TCP 2.457787 5.410254 2.212584 0.439364 0.023586 0.030997 1333 750 UDP 3.079849 6.105065 2.864838 0.410323 0.027376 0.035978 863

738 TCP 3.166935 6.05017 2.916277 0.414932 0.02743 0.036049 879

1000 UDP 3.84002 6.833375 3.571451 0.464048 0.035812 0.047065 645 988 TCP 3.892513 6.839931 3.632605 0.459805 0.03524 0.046313 654

1400 UDP 4.982478 8.041024 4.694939 0.478584 0.04383 0.057603 458 1388 TCP 5.526541 202.2602 4.825294 9.063406 0.815925 1.072308 474

Thr

ee H

ops

100 UDP 1.918115 25.78223 1.708269 0.590661 0.014301 0.018795 6553 88 TCP 2.064188 346.2441 1.764536 4.363141 0.099552 0.130833 7379

250 UDP 2.574611 5.514503 2.344728 0.503853 0.019289 0.025351 2621

238 TCP 2.688931 67.36362 2.396941 1.356534 0.05083 0.066802 2736 500 UDP 3.63044 6.48737 3.402114 0.504712 0.027331 0.035919 1310

488 TCP 3.75142 6.652892 3.480554 0.563687 0.030147 0.03962 1343 750 UDP 4.715654 7.585049 4.479766 0.512055 0.033967 0.04464 873 738 TCP 4.837816 7.728398 4.560351 0.553786 0.036403 0.047842 889

1000 UDP 5.795709 8.822978 5.561471 0.539713 0.041332 0.05432 655 988 TCP 5.916634 8.776248 5.631507 0.59814 0.045495 0.059791 664

1400 UDP 7.486876 10.3398 7.25776 0.506168 0.045858 0.060268 468

1388 TCP 7.635408 11.832 7.356644 0.695743 0.0627 0.082402 473

Table C.1: UDP and TCP one-way delay statistics = 64 Kbps

50

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 UDP 0.704848511 3.75169525 0.5823375 0.30496859 0.005236 0.006881 13034 88 TCP 0.728863 3.755689 0.64671 0.300733 0.004857 0.006383 14727

250 UDP 0.950151 3.94392 0.85038 0.313533 0.008488 0.011155 5242

238 TCP 0.964855 3.984928 0.873029 0.312208 0.008268 0.010866 5478

500 UDP 1.295001 4.344881 1.191557 0.308734 0.011842 0.015563 2611 488 TCP 1.33571594 4.38565 1.234114 0.317287 0.008493 0.011162 5361

750 UDP 1.69104 4.783988 1.599014 0.279294 0.013097 0.017212 1747 738 TCP 1.701305 4.676819 1.616776 0.30885 0.01436 0.018872 1777

1000 UDP 2.081446 5.001545 1.954973 0.333718 0.018071 0.02375 1310 988 TCP 2.095559 5.098939 1.993358 0.353167 0.019002 0.024972 1327

1400 UDP 2.6619 5.56612 2.555847 0.264837 0.016966 0.022298 936 1388 TCP 2.683849 5.716622 2.588928 0.308216 0.019651 0.025826 945

Tw

o H

ops

100 UDP 1.141153 4.192889 1.014113 0.339521 0.005828 0.007659 13037 88 TCP 1.421834 331.0218 1.211822 3.900851 0.063036 0.082843 14711

250 UDP 1.897989 321.1181 1.451135 8.103283 0.155112 0.203852 10484 238 TCP 1.891706 325.7397 1.651108 4.540677 0.120319 0.158126 5471

500 UDP 2.308258 5.274475 2.153278 0.39355 0.015067 0.019801 2621

488 TCP 2.799432 348.8691 2.231717 8.827894 0.334411 0.439491 2677

750 UDP 3.089227 6.042778 2.864003 0.409247 0.019246 0.025293 1737 738 TCP 3.177958 6.10739 2.943933 0.437468 0.020397 0.026807 1767

1000 UDP 4.712349 324.2875 3.732085 13.3401 0.72239 0.949381 1310 988 TCP 3.879552 6.836474 3.643334 0.42291 0.02284 0.030017 1317

1400 UDP 4.902583 7.801294 4.690707 0.492674 0.031562 0.04148 936 1388 TCP 5.901603 329.975 4.782498 13.74154 0.876128 1.151427 945

Thr

ee H

ops

100 UDP 1.880169 4.856586 1.546383 0.514284 0.008834 0.01161 13018 88 TCP 1.955631 4.935801 1.645446 0.484816 0.007827 0.010286 14739

250 UDP 2.580334 5.535185 2.346337 0.509606 0.013795 0.01813 5242

238 TCP 2.786096 352.2561 2.422571 5.688503 0.152401 0.200289 5352 500 UDP 3.644024 6.613791 3.410876 0.521259 0.019956 0.026226 2621

488 TCP 3.724335 6.646514 3.498614 0.512166 0.019369 0.025455 2686 750 UDP 4.7187 7.593393 4.484475 0.520613 0.024413 0.032084 1747 738 TCP 5.002843 189.2093 4.558623 5.326015 0.247771 0.325627 1775

1000 UDP 5.784817 8.701503 5.554438 0.515593 0.02792 0.036693 1310

988 TCP 5.885984 8.807361 5.635321 0.554933 0.029857 0.039239 1327 1400 UDP 7.478903 10.33783 7.254601 0.506176 0.032427 0.042617 936

1388 TCP 7.611083 12.07685 7.332146 0.692194 0.044133 0.058 945

Table C.2: UDP and TCP one-way delay statistics = 128 Kbps

51

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 UDP 0.691212 3.768683 0.579953 0.308151 0.003748 0.004925 25973 88 TCP 0.726991 3.778935 0.632465 0.307029 0.003528 0.004636 29097

250 UDP 0.93738 3.971577 0.812709 0.307587 0.005906 0.007762 10419

238 TCP 0.953625 3.970683 0.866711 0.291619 0.005473 0.007192 10907

500 UDP 1.320152 4.318655 1.213312 0.312363 0.008464 0.011124 5232

488 TCP 1.33571594 4.38565 1.234114 0.317287 0.008493 0.011162 5361 750 UDP 1.699511 4.683674 1.580179 0.314973 0.010442 0.013724 3495 738 TCP 1.948852 5.654276 1.611292 0.542382 0.019751 0.025957 2897

1000 UDP 2.06583 5.054355 1.957417 0.296776 0.011383 0.01496 2611 988 TCP 2.201665 5.310178 1.973391 0.422783 0.017373 0.022832 2275

1400 UDP 2.611497 5.305588 2.512097 0.288311 0.013095 0.01721 1862

1388 TCP 2.651172 5.686879 2.580047 0.249881 0.011387 0.014965 1850

Tw

o H

ops

100 UDP 1.136838 4.236579 1.003623 0.378594 0.004604 0.00605 25979

88 TCP 1.370958 335.7624 1.203477 2.234589 0.026984 0.035463 26343

250 UDP 1.608353 4.638612 1.398206 0.420003 0.008059 0.010592 10433

238 TCP 1.886007 343.3729 1.647711 4.298114 0.080563 0.105878 10934

500 UDP 2.384737 5.412996 2.16341 0.431695 0.011697 0.015373 5232

488 TCP 2.449386 5.419791 2.219975 0.412888 0.011624 0.015276 4847 750 UDP 3.54589 339.8549 2.864242 11.18197 0.370717 0.487205 3495

738 TCP 3.940687 345.3639 2.948701 8.725205 0.317504 0.417272 2901 1000 UDP 3.937064 11.62672 3.735244 0.558594 0.021385 0.028105 2621

988 TCP 4.177451 8.051038 3.656268 0.672255 0.027636 0.03632 2273 1400 UDP 4.960202 8.038581 4.690468 0.432502 0.019645 0.025818 1862

1388 TCP 5.002503 7.916272 4.778028 0.379691 0.017555 0.023071 1797

Thr

ee H

ops

100 UDP 1.978454 6.617308 1.675487 0.612647 0.00745 0.009791 25979 88 TCP 2.25741 5.323887 1.784027 0.543753 0.007901 0.010384 18193

250 UDP 2.570113 5.525351 2.325893 0.510073 0.00979 0.012867 10427 238 TCP 2.65584 7.670164 2.426267 0.521457 0.009827 0.012915 10817 500 UDP 3.646791 6.591916 3.415406 0.512712 0.013879 0.018241 5242

488 TCP 3.727739 6.686688 3.493965 0.507214 0.013599 0.017872 5344

750 UDP 4.727602 7.812798 4.479408 0.52418 0.017378 0.022839 3495

738 TCP 5.596902 10.74457 4.57716 1.479478 0.054547 0.071687 2826 1000 UDP 5.787105 8.660436 5.557418 0.505566 0.019355 0.025437 2621 988 TCP 6.316022 10.7792 5.632043 0.98161 0.041272 0.054241 2173

1400 UDP 7.493475 10.42408 7.259488 0.515298 0.023343 0.030678 1872 1388 TCP 7.651006 12.65216 7.336378 0.756375 0.035642 0.046842 1730

Table C.3: UDP and TCP one-way delay statistics = 256 Kbps

52

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 UDP 0.685878 3.752053 0.577271 0.316916 0.002763 0.003632 50530 88 TCP 0.783543 3.884137 0.63926 0.326511 0.003211 0.00422 39723

250 UDP 0.917839 3.946424 0.808418 0.302494 0.004113 0.005405 20779

238 TCP 0.949166 3.970325 0.863672 0.285907 0.003811 0.005008 21626

500 UDP 1.304143 4.31782 1.178205 0.296287 0.005686 0.007473 10429

488 TCP 1.324906 4.365921 1.240432 0.300587 0.005699 0.00749 10685 750 UDP 1.689532 4.713476 1.570344 0.303571 0.007117 0.009353 6990 738 TCP 2.033076 5.765259 1.619995 0.573477 0.015331 0.020149 5375

1000 UDP 2.063368 5.077541 1.963615 0.31414 0.008504 0.011176 5242 988 TCP 2.292433 5.738259 1.996398 0.44536 0.013723 0.018035 4046

1400 UDP 2.65995 5.693078 2.54941 0.323369 0.010358 0.013613 3744

1388 TCP 2.705026 5.574644 2.589047 0.314867 0.011288 0.014835 2989

Tw

o H

ops

100 UDP 1.504131 548.7475 0.986398 13.29135 0.115081 0.151242 51242

88 TCP 1.7444 5.39738 1.217067 0.507304 0.006502 0.0085454 23383

250 UDP 1.591848 4.60279 1.421511 0.428811 0.005831 0.007664 20772

238 TCP 1.878163 345.5822 1.526237 3.34257 0.04503 0.05917905 21167

500 UDP 2.31335 5.344033 2.117693 0.425397 0.008155 0.010717 10454

488 TCP 2.504743 5.537987 2.236128 0.437985 0.008323 0.0109387 10637 750 UDP 3.187482 275.5122 2.861619 5.569765 0.130571 0.171599 6990

738 TCP 3.882661 8.226574 2.96849 1.049906 0.028125 0.036963 5353 1000 UDP 4.055899 310.5577 3.578842 8.606316 0.232979 0.306186 5242

988 TCP 4.461258 319.497 3.655315 5.438086 0.167193 0.219728 4064 1400 UDP 4.975048 8.023441 4.692614 0.460913 0.014784 0.019429 3734

1388 TCP 5.171156 8.889079 4.782081 0.480449 0.017273 0.022701 2972

Thr

ee H

ops

100 UDP 1.921626 4.87876 1.669943 0.542821 0.004694 0.006169 51377 88 TCP 3.090449 7.73412 1.763523 0.610919 0.009758 0.012825 15056

250 UDP 2.559017 6.174386 2.330303 0.516967 0.007041 0.009253 20710 238 TCP 3.18317 7.842898 2.415836 0.790206 0.012757 0.016766 14739 500 UDP 3.636264 6.600917 3.396332 0.513646 0.009859 0.012957 10427

488 TCP 3.728501 6.652296 3.496826 0.519406 0.009871 0.012972 10637

750 UDP 4.728529 7.714093 4.494608 0.519509 0.012179 0.016006 6990

738 TCP 5.761471 10.87803 4.561126 1.524319 0.041684 0.054782 5137 1000 UDP 5.793701 8.727014 5.548596 0.509272 0.013786 0.018118 5242 988 TCP 6.571297 280.3586 5.638361 4.83587 0.148879 0.195661 4053

1400 UDP 7.498161 10.46407 7.251561 0.517984 0.016592 0.021805 3744 1388 TCP 7.743213 13.22752 7.346213 0.828935 0.029959 0.039372 2941

Table C.4: UDP and TCP one-way delay statistics = 512 Kbps

53

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 UDP 0.699501 3.936231 0.553608 0.366137 0.002273 0.002988 99633 88 TCP 1.040913 5.245149 0.648022 0.346478 0.003665 0.004817 34324

250 UDP 0.915262 3.947079 0.804484 0.322348 0.003572 0.004695 31280

238 TCP 1.056186 4.324556 0.87297 0.377469 0.004054 0.005328 33304

500 UDP 1.290497 4.327536 1.166105 0.313492 0.004263 0.005603 20773

488 TCP 1.302271 4.346907 1.22726 0.253171 0.003418 0.004492 21075 750 UDP 1.588791 4.668236 1.384318 0.315417 0.005242 0.0068892 13908 738 TCP 2.043746 5.767763 1.608431 0.57895 0.011042 0.014512 10560

1000 UDP 2.049818 5.074859 1.919925 0.301302 0.005782 0.007599 10430 988 TCP 2.29598 5.71686 1.99008 0.446302 0.009829 0.012918 7920

1400 UDP 2.654251 5.700827 2.518833 0.311774 0.007061 0.00928 7489

1388 TCP 2.695262 5.76669 2.581656 0.296895 0.00772 0.010145 5682

Tw

o H

ops

100 UDP 1.2743 223.1244 0.965595 4.712031 0.033402 0.043898 76446

88 TCP 3.562976 332.1303 1.223624 4.421168 0.083896 0.110259 10668

250 UDP 1.808684 337.4113 1.405478 8.208332 0.079278 0.104189 41181

238 TCP 3.551874 273.1526 1.555562 3.413087 0.068151 0.089565 9635

500 UDP 2.294122 5.332947 2.111912 0.430096 0.005847 0.007684 20787

488 TCP 2.959262 6.910861 2.24793 0.825476 0.013197 0.0173437 15030 750 UDP 3.379687 337.0176 2.824783 9.186068 0.152629 0.200588 13915

738 TCP 3.89254 240.1296 2.959669 2.753736 0.058085 0.076337 8634 1000 UDP 3.743136 6.76018 3.561258 0.435551 0.008358 0.010984 10432

988 TCP 4.338898 8.208573 3.661513 0.732561 0.017453 0.022937 6768 1400 UDP 4.864025 7.845283 4.691184 0.415976 0.009421 0.012381 7489

1388 TCP 5.282286 338.6258 4.77165 6.202721 0.161721 0.212538 5651

Thr

ee H

ops

100 UDP 2.487688 22.20488 1.66893 1.55787 0.010194 0.013397 89718 88 TCP 6.493218 10.71024 1.796186 1.648082 0.036728 0.048269 7735

250 UDP 2.585036 5.541086 2.301037 0.559447 0.005424 0.007128 40871 238 TCP 6.663319 10.74809 2.425671 1.657345 0.0389 0.051123 6973 500 UDP 3.632224 6.593824 3.393114 0.52282 0.007113 0.009349 20751

488 TCP 6.466868 12.60918 3.493309 1.49479 0.031934 0.041968 8417

750 UDP 4.70883 8.851409 4.468501 0.528616 0.008786 0.011546 13907

738 TCP 6.389429 12.41708 4.568994 1.647395 0.034128 0.044852 8951 1000 UDP 5.787422 8.742392 5.526721 0.51847 0.009952 0.013079 10427 988 TCP 6.498644 10.69033 5.637765 1.047819 0.025469 0.033472 6502

1400 UDP 7.496567 10.4208 7.257641 0.510394 0.01156 0.015192 7489 1388 TCP 7.761945 343.197 7.353962 5.274329 0.149819 0.196895 4761

Table C.5: UDP and TCP one-way delay statistics = 1024 Kbps

54

Number of hops

[bytes]

Mean [ms]

Max [ms]

Min [ms]

Std. [ms]

C.I. 95% [ms]

C.I.99% [ms]

No. of Samples

Sing

le H

op

100 UDP 0.91091 4.283726 0.576377 0.641423 0.002857 0.003754 193666 88 TCP 2.106363 5.726934 0.656307 0.606429 0.009186 0.012073 16741

250 UDP 0.93881 4.039288 0.791311 0.389877 0.00268 0.003523 81269

238 TCP 2.279109 5.751968 0.852406 0.593476 0.009768 0.012837 14181

500 UDP 1.292842 4.374921 1.164556 0.33624 0.003248 0.004269 41156

488 TCP 2.228296 5.719841 1.229405 0.378512 0.006291 0.008268 13906 750 UDP 1.661113 4.747927 1.542449 0.307503 0.003622 0.00476 27691 738 TCP 2.102863 5.8347583 1.605392 0.598344 0.008984 0.011807 17040

1000 UDP 2.034072 5.065143 1.920998 0.304743 0.004145 0.005448 20762 988 TCP 2.296043 5.835354 1.985908 0.448408 0.007804 0.010256 12683

1400 UDP 2.633003 5.666733 2.496123 0.312901 0.005024 0.006602 14902

1388 TCP 2.684459 5.841434 2.572954 0.271139 0.005047 0.006632 11089

Tw

o H

ops

100 UDP 1.712793 19.786 0.987292 1.063086 0.004876 0.006409 182580

88 TCP 5.101126 325.2511 1.213968 3.645409 0.067452 0.0886475 11220

250 UDP 1.658687 4.93741 1.40369 0.557183 0.003874 0.005092 79447

238 TCP 5.200462 14.21791 1.534581 1.391235 0.027102 0.0356174 10123

500 UDP 2.30721 5.322516 2.10619 0.470056 0.00454 0.005966 41185

488 TCP 5.32952 20.4175 2.83232 1.52442 0.02754 0.036268 8212 750 UDP 3.300496 327.4541 2.832234 6.059613 0.071376 0.093805 27687

738 TCP 5.931218 23.32538 2.949417 1.704857 0.038638 0.050779 7479 1000 UDP 4.020464 324.3002 3.536582 8.48162 0.115375 0.151629 20760

988 TCP 5.343997 21.11322 3.656626 1.052614 0.02304 0.03028 8018 1400 UDP 5.240507 349.098 4.669607 9.878363 0.158603 0.208439 14902

1388 TCP 5.415179 157.9645 4.808068 1.863756 0.041242 0.054201 7845

Thr

ee H

ops

100 UDP 59.63816 69.92382 4.192531 3.268815 0.021425 0.028158 89416 88 TCP 7.773832 17.71444 1.780987 2.111213 0.048348 0.06354 7325

250 UDP 100.6917 108.373 2.382338 7.131061 0.051743 0.068002 72962 238 TCP 8.058261 297.4535 2.423406 5.188715 0.120328 0.158138 7143 500 UDP 3.947073 7.996619 3.388643 0.819885 0.007957 0.010458 40781

488 TCP 8.205454 19.46837 3.483296 1.681952 0.040764 0.053572 6540

750 UDP 5.119016 8.91763 4.460275 0.75459 0.008919 0.011722 27495

738 TCP 8.199027 16.85631 4.578054 1.511581 0.035733 0.046962 6874 1000 UDP 6.338265 10.22303 5.546034 0.877871 0.011973 0.015736 20650 988 TCP 8.078534 19.93066 5.613566 1.310422 0.033549 0.04409 5861

1400 UDP 8.252577 12.19285 7.228673 1.091588 0.01755 0.023065 14861 1388 TCP 8.043739 196.4188 7.359505 2.572258 0.065652 0.086281 5897

Table C.6: UDP and TCP one-way delay statistics = 2048 Kbps