analyzing throughput and stability in cellular networks...the comparison across mnos shows a large...

Analyzing Throughput and Stability in CellularNetworks

Ermias Walelgne 1 Jukka Manner 1 Vaibhav Bajpai 2 Jorg Ott 2

April 25, 2018 — NOMS’18, Taipei1(ermias.walelgne | jukka.manner)@aalto.fi,Aalto University, Finland

2(bajpaiv | ott)@in.tum.de, Technical University of Munich, Germany

April 25, 2018 — NOMS’18, Taipei

Introduction

Introduction

Introduction — Motivation

Mobile-broadband subscriptions have grown more than 20% annually in thelast five years (ITU 2017) [1]Smartphones (in 2017) have 70% share of the total market of all device types(IDC) [2].Quality & performance of the cellular network depends:

radio technology,limitations of device hardwarewireless link characteristics (e.g interference, fading, etc.)mobility, location and time of the dayInfrastructure of Mobile Network Operators (MNO)

About 30% of cellular measurements from netradar [3] experience suddendrops to zero bitrate for(≥ 200 ms).

Analyzing Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 1 / 20

Introduction


Mobile-broadband subscriptions have grown more than 20% annually in thelast five years (ITU 2017) [1]

Smartphones (in 2017) have 70% share of the total market of all device types(IDC) [2].Quality & performance of the cellular network depends:




Introduction


Mobile-broadband subscriptions have grown more than 20% annually in thelast five years (ITU 2017) [1]Smartphones (in 2017) have 70% share of the total market of all device types(IDC) [2].

Quality & performance of the cellular network depends:radio technology,limitations of device hardwarewireless link characteristics (e.g interference, fading, etc.)mobility, location and time of the dayInfrastructure of Mobile Network Operators (MNO)



Introduction


Mobile-broadband subscriptions have grown more than 20% annually in thelast five years (ITU 2017) [1]Smartphones (in 2017) have 70% share of the total market of all device types(IDC) [2].Quality & performance of the cellular network depends:




Introduction

Introduction — Research Question

What are the factors affecting the throughput and stability of cellular networks?


Introduction

Introduction — Contribution

1 Time of a day and the location affect the throughput:Throughput drops during peak hours in cities as congestion increases.Radio technology switches from legacy (UMTS) to advanced (HSDPA) duringthe course of a day.

2 Predict the probability sudden drops in bit rate in a cellular network with90% accuracy

relying on easily accessible information (e.g device model, location, networktechnology).

3 Predict the average TCP download speed based on the first 5 seconds ofmedian bit rate value of throughput.


Methodology

Methodology

Methodology — Measurement platform

0

1

2

3

4

5

0 2.5 7.5 105Time (Sec.)

TCP

down

lik sp

eed (

Mbps

)

It measures & collects information including throughput (TCP), signalstrength, radio technology type, RTT( UDP ) etc. towards Amazon Cloudinstances & Aalto University servers.The measurement server tests the download speed by sending a random dataover TCP for 10 seconds.During the measurement session, both the client and the server record thenumber of bytes transferred every 50 ms.


Methodology

Methodology — Data Set and Measurement Trials

Based on a longitudinal dataset collected using the netradar measurementplatform [3].Focused on stationary nodes during a ten second measurement session — tominimize the variability that might arise from mobility.A year-long measurement data (∼750K ) from 3 Finnish MNO.

Network Total # Of LTE HSPA HSPA+ HSDPA UMTS OthersOperator Measurements

Elisa 373K 45.75% 8.49 % 14.70% 1.12% 25.43% 5%DNA 235K 63.30% 7.52% 8.17% 10.69% 0.9% 9.42%

TeliaSonera 140K 34.74% 1.04% 14.27% 1.29% 33.60% 15.06%

Table: Measurement Distribution per radio Technology for each MNO


Data Analysis

Data Analysis

Data Analysis — Device Model

EDGE GPRS HSDPA HSPA HSPA+ LTE UMTS

20112012

20132014

20152011

20122013

20142015

20112012

20132014

20152011

20122013

20142015

20112012

20132014

20152011

20122013

20142015

20112012

20132014

2015

10−3

10−2

10−1

100

101

102

Release year of the device

Dow

nloa

d sp

eed

(Mbp

s)

Figure 1: TCP download speed of different device models per network technology.

The release year of the device models does not correlate to TCP downloadspeed.It is not always the newest device model whose TCP download performanceis best.


Data Analysis

Data Analysis — Mobile Network Operator

0.00

0.25

0.50

0.75

1.00

0 10 20 30 40 50 60 70 80TCP Download Average (MBps)

CD

F

DNAElisaTeliaSonera

0.00

0.25

0.50

0.75

1.00

0 10 20 30 40 50TCP Upload Average (MBps)

CD

F

DNAElisaTeliaSonera

Figure 2: Mean TCP throughput for LTE networks of 3 MNO downlink (left) and uplink(right) speed.

Clear variation between MNOs on mean uploading speed for LTE.MNO’s are significant for network performance variation.


Data Analysis

Data Analysis — Subscribers Location

Elisa (Network Operator With LTE)

10 Mbps

15 Mbps

20 Mbps

25 Mbps

30 Mbps

35 Mbps

40 Mbps

DNA (Network Operator With LTE)

10 Mbps

20 Mbps

30 Mbps

40 Mbps

50 Mbps

60 Mbps

70 Mbps

80 Mbps

90 Mbps

100 Mbps

110 Mbps

Sonera (Network Operator With LTE)

10 Mbps

20 Mbps

30 Mbps

40 Mbps

50 Mbps

60 Mbps

70 Mbps

80 Mbps

90 Mbps

Figure 3: Mean TCP throughput distribution by area in Finland for LTE networks ofthree MNOs: Elisa (left), DNA (middle), TeliaSonera (right).

The comparison across MNOs shows a large variation in throughput perlocations.Users in a metropolitan area & (subscribed to DNA or Elisa) get betterthroughput than urban areas.

Better infrastructure provisioning — base station density — sufficient corenetwork capacity.


Data Analysis

Data Analysis — Radio Technology Changes and Time of DayHSPA to HSPA+

UMTS to HSDPA

0 5 10 15 20

0%

2%

4%

6%

0%2%4%6%8%

Time of a day

Hand

over

freq

uenc

y

Figure 4: Frequency of radio technology switches over time of day, for the TeliaSoneranetwork (similar to other MNOs).

The occurrence of switches (from legacy to more advanced technology e.gUMTS to HSDPA) increases during peak hours.


Data Analysis

Data Analysis — Network Stability

0

1

2

3

4

5

0 2.5 7.5 105Time (Sec.)

TCP d

ownli

k spe

ed (M

bps)

Classified the data into two groups:

dropped: measurement sessions that experience a sudden dropout >200 ms.non-dropped: sessions without this phenomenon.


Data Analysis

Data Analysis — Received Data per Recording Interval

After sudden dropout Before sudden dropout

'O 200 +

Q) 175

Q) 150 o.-

tll tll 125 •

'O .0 100 <11:S:

75 o- D rl 50 •

25

0

0.0 0.51.0l.52.02.53.03.54.04.5 5.0 0.0 0.51.0l.52.02.53.03.54.04.5 5.0

Sudden dropout duration (sec)

oa

s

HSPA

LTE

UMTS

Do

wn

lo

ad

sp

ee

d

(M

bp

s)

Figure 5: TCP maximum download rate observed before and after a sudden dropout ofa certain duration (≥ 200 ms) per radio technology.

A sudden dropout duration that stayed for at least 200 ms does have animpact on download bit rate.The impact is visible especially after the dropout period is over (left side ofthe figure).


Data Analysis


mode

modedropout

dropout

Bytes downloaded at every 50 ms sample time span

Bytes downloaded at every 50 ms sample time span

Measurements with NO sudden dropout

Measurements with sudden dropout

Mean & median of dropped diverge with a relatively higher standarddeviation than the non-dropped measurements.Network inconsistency and jitter is present in the dropped measurementsthan in non-dropped ones.


Data Analysis


10 1 101 1021000.0

0.2

0.4

0.6

0.8

1.0

CDF

Average of the first 3 bit-rates samples before and after dropout

Average of the first 3 bit-rates before dropout Average of the first 3 bit-rates after dropout

Download Speed (Mbps)

Figure 6: Average of the first 3 bit rates samples before and after a sudden dropout.

The effect of sudden dropout is reflected even after the sudden dropout (zerobit rate) is over.


Data Analysis

Data Analysis — Sudden Dropout by Network TechnologyEDGE

GPRSHSDPA

HSPAHSPA+

LTEUMTS

0 5 10 15 20

0%2%4%6%

0%2%4%6%

0%2%4%6%

0%2%4%6%8%

0%2%4%6%

0%2%4%6%

0%2%4%6%

Time of a day

Sudd

en dr

opou

t freq

uenc

y

The sudden dropouts is distributed in all radio technology that has been used.Some technologies such as UMTS and HSPA show frequent sudden dropoutsduring daytime.


Data Analysis

Data Analysis — Switching Radio Technology

0.00

0.25

0.50

0.75

1.00

0 10 20 30 40 50Download speed (Mbps)

CD

F

Unknown to LTEUnknown to UMTSHSPAHSPA to HSPA+LTEHSPA+UMTS to HSPA+UMTSHSDPA

Figure 7: Impact of radio technology switches for download speed.

When sudden dropout happens a switch in radio technology causes asignificant variation in TCP download speed.

E.g. a change from UMTS to HSPA+ has better download speed than fromHSPA to HSPA+.


Data Analysis

Data Analysis — Feature Importance and Selections

Downlink inter-arrival(variance)

Average throughput(TCP)

Device Model

Area province

Radio type at the beginning

Latency(UDP)

Radio type at the end

Uplink speed

Signal strengths

Mobile Network Operator

Time of a day(hours)

Battery level

Handover

Cell ID Changes

Day of week

Frequency of handover

Platform

Byte inter-arrival(mean)

0 250 500 750

Mean Decrease Accuracy

Downlink (TCP)

Downlink inter-arrival(variance)

Area province

Uplink speed

Device Model

Latency(UDP)

Radio type at the beginning

Day of week

Battery level

Time of a day(hours)

Signal strengths

Radio type at the end

Handover

Mobile Network Operator

Cell ID Changes

Frequency of handover

Byte inter-arrival(mean)

Platform

0 500 1000 1500 2000

Mean Decrease Gini

(a) with TCP throughput

RadioHandoverStartEndStartingNetTechNames

EndingNetTechNameradioEvolution

RadioHandoverdeviceVendor

CellIDChangedbattery_level

weekdayDModel

hourssignalStrengths

carrierlatency

0 25 50 75Mean Decrease Accuracy

radioEvolutionStartingNetTechNames

EndingNetTechNameCellIDChangedRadioHandover

carrierdeviceVendor

RadioHandoverStartEndDModel

weekdayhours

signalStrengthsbattery_level

latency

0 2000 4000 6000Mean Decrease Gini

(b) without TCP throughput

Latency, carrier network, signal strength radio network technology, time ofthe day and device model found to be important predictive variables forclassification.


Data Analysis

Data Analysis — Classification model

0.0 0.2

ROC

0.4 0.6 0.8 1.0

False positive rate

True

pos

itive

rate

Models:Conditional Inference TreeBaggingRandomForest

o.o

0.2

0.4

0.6

0.8

1.0

Figure 9: True positive and false positive rates of the three classification models;random forest shows the best Receiver Operating Characteristic (ROC) curve.

Random forest based classification produces better prediction with accuracyof 90% & error rate of 10.2% on the testing dataset.


Data Analysis

Data Analysis — Predicting the Average Throughput

Prediction of througput is useful (eg. to improve video performance incellular networks [4]).How to predict the overall mean throughput only using the first five secondsof TCP bit rate measurement?Train a model using Randome Forest algorithm from caret [5] package.The model predict average throughput: using only the first 5 sec. downloadrates (which would be easily available right after startup) withRoot-Mean-Square Error (RMSE) ( 0.003 Mbps) & 98% R-squared.


Conclusion

Conclusion

Conclusion

1 Throughput observed by a cellular network user depends on various factors.E.g location and time of the day where metropolitan areas during peak hoursshowed more drops in throughput.

2 Network stability:TCP being sensitive to losses and jitter ∼30% sudden drops to zero bitrate -could create performance degradations.classify stability based on sudden dropouts only relying on easily accessibleinformation.

3 Predicting the average throughput:useful to anticipate future performance & to adjust application demands.trained a model that predicts average throughput based on throughput of TCPslow start phase.

4 Future work extending — our predictive approach :Using Crowdsourcing Data for Adaptive Video Streaming in Cellular Network.

[email protected]


[email protected]

Conclusion

Conclusion

1 Throughput observed by a cellular network user depends on various factors.E.g location and time of the day where metropolitan areas during peak hoursshowed more drops in throughput.

2 Network stability:TCP being sensitive to losses and jitter ∼30% sudden drops to zero bitrate -could create performance degradations.classify stability based on sudden dropouts only relying on easily accessibleinformation.

3 Predicting the average throughput:useful to anticipate future performance & to adjust application demands.trained a model that predicts average throughput based on throughput of TCPslow start phase.

4 Future work extending — our predictive approach :Using Crowdsourcing Data for Adaptive Video Streaming in Cellular Network.

[email protected] Throughput and Stability in Cellular Networks April 25, 2018 — NOMS’18, Taipei 19 / 20

[email protected]

Conclusion

References

ITU, “ITU releases 2017 global information and communication technologyfacts and figures.”

IDC, “IDC idc 50th anniversary transformation everywhere,” 2017.

“Netradar measurement platform.” http://www.netradar.org/.

X. K. Zou, J. Erman, V. Gopalakrishnan, E. Halepovic, R. Jana, X. Jin,J. Rexford, and R. K. Sinha, “Can Accurate Predictions Improve VideoStreaming in Cellular Networks?,” HotMobile, pp. 57–62, 2015.

M. Kuhn, “Building Predictive Models in R Using the caret Package,”Journal of Statistical Software, vol. 28, no. 5, pp. 1–26, 2008.


http://www.netradar.org/

analyzing throughput and stability in cellular networks...the comparison across mnos shows a large...

Documents