master thesis change detection in telecommunication data ...633628/fulltext01.pdf · series...

Master Thesis

Change Detection in Telecommunication Data using TimeSeries Analysis and Statistical Hypothesis Testing

Tilda Eriksson

LiTH-MAT-EX–2013/04–SE

Change Detection in Telecommunication Data using TimeSeries Analysis and Statistical Hypothesis Testing

Applied Mathematics, Linkoping University, Institute of Technology

Tilda Eriksson

LiTH-MAT-EX–2013/04–SE

Master Thesis: 30 hp

Level: A

Supervisors: Lars-Olof Bjorketun,Ericsson GSM RANTorkel Erhardsson,Mathematical Statistics, Linkoping University

Examiner: Torkel Erhardsson,Mathematical Statistics, Linkoping University

Linkoping: June 2013

Abstract

In the base station system of the GSM mobile network there are a large numberof counters tracking the behaviour of the system. When the software of thesystem is updated, we wish to find out which of the counters that have changedtheir behaviour.

This thesis work has shown that the counter data can be modelled as astochastic time series with a daily profile and a noise term. The change detectioncan be done by estimating the daily profile and the variance of the noise termand perform statistical hypothesis tests of whether the mean value and/or thedaily profile of the counter data before and after the software update can beconsidered equal.

When the chosen counter data has been analysed, it seems to be reasonablein most cases to assume that the noise terms are approximately independentand normally distributed, which justifies the hypothesis tests. When the changedetection is tested on data where the software is unchanged and on data withknown software updates, the results are as expected in most cases. Thus themethod seems to be applicable under the conditions studied.

Eriksson, 2013. v

vi

Keywords: counter data, software update, change detection, stochastic, timeseries, daily profile, noise, mean value, statistical hypothesis tests

URL for electronic version:

http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-94530

Acknowledgements

I would like to thank Helene and Lars-Olof at Ericsson, for making me feelwelcome and for giving me the opportunity to grow, both personally and in myfield of study.

I would also like to thank Torkel at Linkoping University, for helping mewhen my knowledge in statistics was insufficient.

Thanks to my parents, for always letting me know that I could study what-ever I wanted. And to my fellow students at Yi, who made my initial years atthe university very pleasant.

Special thanks to Richard and Maud, for being true friends through difficulttimes. And to my husband. Thank you for your unconditional love and foralways believing in me.

Finally, to myself, for never giving up.

Linkoping, June 2013Tilda Eriksson

Eriksson, 2013. vii

Nomenclature

Most of the reoccurring abbreviations and symbols are described here.

Symbols

X Random Variablex Observation of a Random Variableµ Population Meanσ2 Population Varianceσ Population Standard Deviationx Sample Means2 Sample Variances Sample Standard DeviationH0 Null HypothesisH1 Alternative HypothesisT Test Statistictobs Observation of the Test StatisticC Critical Region, Rejection Regionα Level of Significanceλα/2 α-QuantileP P-valuedf Degrees of Freedomn Number of ObservationsXt Stochastic Time Seriesst Seasonal Component, Daily Profilep Periodmt Trend ComponentYt Noise Termh Lag

Eriksson, 2013. ix

x

Abbreviations

acf Auto Correlation Functioncdf Cumulative Distribution Functioniid Independent and Identically Distributedpdf Probability Density Function

BRP Basic Recording PeriodBSC Base Station ControllerBSS Base Station SystemBTS Base Transceiver StationCN Core NetworkDL DownlinkEGPRS Enhanced General Packet Radio ServiceGSM Global System for Mobile communicationsKPI Key Performance IndicatorMS Mobile StationOSS Operation and Support SystemPDCH Packet Data CHannelPI Performance IndicatorRAN Radio Access NetworkROP Result Output PeriodRPI Resource Performance IndicatorSTS Statistics and Traffic Measurement SubsystemTBF Temporary Block FlowTS Time Slots

Contents

1 Introduction 1

1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.3 Methods used To Perform the Thesis Work . . . . . . . . . . . . 2

1.4 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.5 Topics Covered . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Counter Data 5

2.1 Overview of the Ericsson GSM System . . . . . . . . . . . . . . . 5

2.2 The STS Counters . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2.1 Collection and Behaviour of the Counter Values . . . . . . 7

2.3 Processing of Counter Data . . . . . . . . . . . . . . . . . . . . . 7

2.3.1 Key Performance Indicators and ChangeDetection . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3 Statistical Hypothesis Testing 9

3.1 Random Variables And Their Probability Distributions . . . . . 9

3.1.1 Independent and Identically Distributed Random Variables 11

3.1.2 The Normal Distribution and the Central LimitTheorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.2 Statistical Description of Data . . . . . . . . . . . . . . . . . . . 13

3.3 How to Perform a Hypothesis Test . . . . . . . . . . . . . . . . . 14

3.3.1 Using P-values . . . . . . . . . . . . . . . . . . . . . . . . 15

3.4 Comparing the Means of Two Samples . . . . . . . . . . . . . . . 16

3.4.1 Why The Test Statistic Is t-Distributed . . . . . . . . . . 18

3.5 Comparing the Variance of Two Samples . . . . . . . . . . . . . . 19

3.6 Testing Normality . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.6.1 Histogram . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.6.2 Lilliefors Test . . . . . . . . . . . . . . . . . . . . . . . . . 22

4 Time Series Analysis 25

4.1 Time Series and Hypothesis Tests . . . . . . . . . . . . . . . . . . 25

4.2 Estimation of Trend, Seasonality and theRemaining Noise Term . . . . . . . . . . . . . . . . . . . . . . . . 26

4.3 Testing the Independence Assumption of the Noise . . . . . . . . 28

Eriksson, 2013. xi

xii Contents

5 Analysis of Counter Data 335.1 Selection of Data to Work With . . . . . . . . . . . . . . . . . . . 335.2 The Daily Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.3 Outliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.4 Missing Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.5 Testing Independence and Normality of the Noise . . . . . . . . 39

6 The Change Detection Method 416.1 The Change Detection Method and Its

Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416.1.1 Change Detection of Mean Value . . . . . . . . . . . . . . 426.1.2 Change Detection of Daily Profile . . . . . . . . . . . . . 42

6.2 Normality Assumption . . . . . . . . . . . . . . . . . . . . . . . . 436.3 Level of Significance . . . . . . . . . . . . . . . . . . . . . . . . . 43

7 Applying the Change Detection Method To Counter Data 457.1 Selection of Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

7.1.1 Data Volume . . . . . . . . . . . . . . . . . . . . . . . . . 457.1.2 The Throughput KPI and Underlying PI:s and

Counters . . . . . . . . . . . . . . . . . . . . . . . . . . . 467.2 Interpreting the Figures . . . . . . . . . . . . . . . . . . . . . . . 477.3 Counter Data Without Known Changes . . . . . . . . . . . . . . 47

7.3.1 Level of Significance . . . . . . . . . . . . . . . . . . . . . 477.4 Counter Data With Known Changes . . . . . . . . . . . . . . . . 53

8 Discussion 638.1 Why Time Series Analysis and Statistical Hypothesis Testing . . 638.2 Problems With Hypothesis Tests . . . . . . . . . . . . . . . . . . 648.3 Analysis of the Result . . . . . . . . . . . . . . . . . . . . . . . . 64

8.3.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . 648.3.2 Change Detection . . . . . . . . . . . . . . . . . . . . . . 648.3.3 Weekdays Versus Week . . . . . . . . . . . . . . . . . . . 64

8.4 How the Method Could Be Used . . . . . . . . . . . . . . . . . . 658.5 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 658.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

List of Figures

2.1 The GSM system model . . . . . . . . . . . . . . . . . . . . . . . 52.2 The Counters Divided Into Object Types In the STS . . . . . . . 62.3 The KPI-Pyramid . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.1 An example of a pdf. . . . . . . . . . . . . . . . . . . . . . . . . . 103.2 An example of a cdf. . . . . . . . . . . . . . . . . . . . . . . . . . 103.3 The pdf of normal distributions with standard deviation σ. . . . 123.4 The α-quantile. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.5 Two-sided quantile for a standard normal distribution. . . . . . . 133.6 The null distribution and rejection of H0. . . . . . . . . . . . . . 153.7 One-sided P-value. . . . . . . . . . . . . . . . . . . . . . . . . . . 163.8 Two-sided P-value. . . . . . . . . . . . . . . . . . . . . . . . . . . 163.9 The t-distribution. . . . . . . . . . . . . . . . . . . . . . . . . . . 173.10 The F-distribution. . . . . . . . . . . . . . . . . . . . . . . . . . . 193.11 The pdf of an F-distributed test statistic and its two-sided rejec-

tion region. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.12 Histogram of random samples from a normal distribution. . . . . 213.13 Histogram of a large number of random samples from a normal

distribution with a fitted normal pdf. . . . . . . . . . . . . . . . . 213.14 Definition of the test statistic for the Kolmogorov-Smirnov and

Lilliefors test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4.1 A time series and its estimated seasonal component and trend. . 264.2 The estimated noise term of figure 4.1. . . . . . . . . . . . . . . . 284.3 Normally distributed iid noise. . . . . . . . . . . . . . . . . . . . 294.4 The sample acf of a time series with a trend and seasonal behaviour. 304.5 The sample acf of an iid-sequence. . . . . . . . . . . . . . . . . . 30

5.1 An example of counter data. . . . . . . . . . . . . . . . . . . . . . 335.2 An example of counter data with missing values. . . . . . . . . . 345.3 An example of counter data with an outlier. . . . . . . . . . . . . 345.4 An example of estimated trend and daily profile for counter data. 365.5 An example of estimated trend and daily profile for counter data. 375.6 An example of the sample acf for counter data with a daily profile. 385.7 Counter data with a removed outlier. . . . . . . . . . . . . . . . . 385.8 An example of the sample acf for the noise. . . . . . . . . . . . . 395.9 An example of the sample acf for the noise. . . . . . . . . . . . . 405.10 Testing the normality assumption of the noise. . . . . . . . . . . 40

Eriksson, 2013. xiii

xiv List of Figures

7.1 Relations between the KPI, PI:s and counters used. . . . . . . . . 467.2 No significant change in User Data Volume. . . . . . . . . . . . . 487.3 No significant change on KPI-level. . . . . . . . . . . . . . . . . . 497.4 No significant change on PI level. . . . . . . . . . . . . . . . . . . 507.5 No significant change on counter level. . . . . . . . . . . . . . . . 517.6 Change detected. . . . . . . . . . . . . . . . . . . . . . . . . . . . 527.7 Change still detected when the possible outliers are removed. . . 537.8 Change detection of Throughput. . . . . . . . . . . . . . . . . . . 547.9 Change detection of Simultaneous TBFs per PDCH. . . . . . . . 547.10 Change detection of DLTBFPEPDCH. . . . . . . . . . . . . . . . 557.11 Change detection of DLEPDCH. . . . . . . . . . . . . . . . . . . 557.12 Change detection of Radio Link Bitrate. . . . . . . . . . . . . . . 567.13 Change detection of MC19DLACK. . . . . . . . . . . . . . . . . . 577.14 Change detection of MC19DLSCHED. . . . . . . . . . . . . . . . 577.15 Change detection of Multislot Utilization. . . . . . . . . . . . . . 587.16 Change detection of Average Maximum Number of TS reservable

by MSs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 587.17 Change detection of MAXEGTSDL. . . . . . . . . . . . . . . . . 597.18 Change detection of TRAFF2ETBFSCAN. . . . . . . . . . . . . 597.19 Change Detection of User Data Volume. . . . . . . . . . . . . . . 607.20 Change Detection of Daily Profile. . . . . . . . . . . . . . . . . . 607.21 Change Detection of Daily Profile. . . . . . . . . . . . . . . . . . 617.22 Change Detection of Daily Profile. . . . . . . . . . . . . . . . . . 61

Chapter 1

Introduction

This first chapter will present some background to the problem formulation, fomu-late the questions to be answered in the forthcoming chapters and describe whattopics are covered.

1.1 Background

The mobile telecommunication system is a very large infrastructure with morethan 4,5 Billion GSM only subscribers around the world. The GSM system hasbeen developed over 20 years and is continuously enhanced. This has resultedin a complex system. One way of monitoring that the GSM system behavesas expected is to use statistics and counters. In the Ericsson BSS system (theradio part of the GSM system) there are over 2000 different types of counters,often with instances in every cell, that track the behaviour of the system. Dueto new software releases the behaviour of the system changes. Often thesechanges are visible in the counter data. Today the changes in the counter datais detected by hand. This is a very time consuming process and many changesare undetected due to the large number of counters. The purpose of this masterthesis is to investigate if the changes in the counters due to new software releasescan be detected using mathematical methods and computer calculations. If thisis achieved, it can reduce the number of counters that has to be analysed byhand.

An earlier attempt was made to solve the problem, but the change detectionfailed because of assumptions that the counter data behaved in a deterministicway. Due to the stochastic behaviour of the system, probability theory andstatistics seems to be a better approach.

1.2 Problem Statement

The thesis problem statement is: Can any mathematical method/methods todetect changes in counter data after new software releases be found? And if amethod is found, how can it be implemented and tested on counter data?

To answer the question a couple of other questions arises. How do thecounters behave? What type of changes are we looking for? Is the detectionto be made on unprocessed data or data that has been processed in some way?

Eriksson, 2013. 1

2 Chapter 1. Introduction

Which data is to be used to perform the analysis? Should knowledge of thesystem be used or should the data be treated as unknown? How do we decideif the method is working or not?

1.3 Methods used To Perform the Thesis Work

Methods used to perform the thesis work was to gain knowledge about thecounter data to work with and to gain some knowledge about the system, toput the counters into a broader context. Literature on different mathematicalmethods, mainly in statistics and also some literature in machine learning andsignal processing was also studied. And a set of data to work with was chosen.

When a possible method was found, the assumptions that has to be made onthe data was tested. Then the method was implemented in Matlab and testedon parts of the counter data. Finally, an analysis of the method was performed.

1.4 Scope

The purpose of the method searched for in this thesis work is only to detectchanges, not to determine whether the changes should or should not occur. Themethod is also limited to the use of a reference week, a fixed time for softwareupdate and a test week after the update. Consideration is therefore not takento long term trends and seasonal behaviour of the data over longer time periods.The period of the seasonal component of the data is considered as 24 hours forall data and every day of the week is treated as equal. When the data is plotted,the x-axis shows the number of hours elapsed.

1.5 Topics Covered

There are eight chapters (including this introduction). Main topics dealt withare:

Chapter 2: We get to know the counters, the system that they are part of, thebehaviour of the counters and how the counter data is used and processed.

Chapter 3: This chapter is devoted to mathematical concepts used in thisthesis work regarding statistics and probability theory, mainly about sta-tistical hypothesis testing and the underlying assumptions.

Chapter 4: This chapter presents some theory of time series and how to studythe properties of the noise term.

Chapter 5: In this chapter the results from the analysis of the counter data ispresented.

Chapter 6: Describes the method for change detection in counter data.

Chapter 7: Here the results from applying the change detection method tocounter data is presented.

1.5. Topics Covered 3

Chapter 8: Here we find a discussion about the method and the results, exam-ples of how the method could be used and thoughts about future work onthe subject. Here we also find conclusions and motivations for the choiceof method.

4 Chapter 1. Introduction

Chapter 2

Counter Data

There are about 3000 different types of counters in the Ericsson Base Station Systemof the GSM-network. They track the behaviour of the system. In this chapter wewill get an overview of the counters and how the counter data is processed andused.

2.1 Overview of the Ericsson GSM System

The Ericsson GSM system is divided into three parts. The Core Network (CN),the Operation and Support System (OSS), and the Base Station System (BSS).The CN includes control servers that sets up calls, handle subscription informa-tion, location of the user, authentication, ciphering, etc. The OSS is a supportnode and the BSS is the radio part of the system that contains Base TransceiverStations (BTS), which is the radio equipment with antenna, transmitters andreceivers, and a control unit, the Base Station Controller (BSC). See Figure 2.1.

Figure 2.1: The GSM system model

Eriksson, 2013. 5

6 Chapter 2. Counter Data

In different parts of the BSC there are counters tracking the behaviour of thesystem and the traffic that it handles. The counters considered in this reportare called PEG-counters and can only be incremented. They count system- oruser events, resource usage and traffic volumes.

The geographical area over which the mobile users are distributed is dividedinto cells and one BSC handles many cells. The counters often have instancesin every cell, so one counter can have many instances since the number of cellsis large. The counters track the behaviour of the system on system level, BSC-level and cell level and there is a subsystem in the BSC that collects and handlesthe counter values. This subsystem is called Static and Traffic MeasurementSubsystem (STS).

2.2 The STS Counters

The Static and Traffic Measurement Subsystem collects counter values from thecounters in the BSC. The STS collects the counter values with a fixed timeinterval of 5 or 15 minutes, the Basic Recording Period (BRP), saving a valuethat is the difference between the current value and the last value. The countervalues in the STS is thus a measure of what has happened during the latest timeinterval, the BRP-interval. Since the counters in the BSC are only incremented,the values in the STS will be positive numbers or zero.

Figure 2.2: The Counters Divided Into Object Types In the STS

In the STS, the counters are divided into object types. There are up to 30counters in every object type, they are mostly linked to each other in systembehaviour. Figure 2.2 shows how the counters are divided into object types inthe STS.

2.3. Processing of Counter Data 7

2.2.1 Collection and Behaviour of the Counter Values

When the counter values are collected from the STS it is done at fixed timeintervals called ROP (Result Output Period). Since the counter values arecollected every 5 or 15 minutes in the STS, the values collected between ROP-intervals are accumulated. Thus the value collected from the STS will still bea measure of what has happened during the latest time interval, but now thetime interval is ROP instead of BRP. The ROP-interval is often an hour, andthus the STS counters consists of observations of the number of events duringan hour, recorded every hour.

2.3 Processing of Counter Data

The counter data is used to observe the system behaviour. It can be used toanalyse trends in the traffic behaviour, fault indication and as a measure of enduser performance. Since the data is collected from a large number of cells, it canbe relevant to aggregate the collected data to BSC level, to get an overview ofthe system behaviour in all cells. This is done by a sum over all the cells. Thedata can also be aggregated in time. For example the data on BSC level houris a summation of all the collected values from every cell during the previoushour.

2.3.1 Key Performance Indicators and ChangeDetection

The Key Performance Indicators (KPI) describes the end-users perception ofthe performance of the BSS. The KPI:s are calculated from counter data. Over100 counters are used.

Figure 2.3: The KPI-Pyramid

There are different levels of the KPI:s. On the highest level are the KPI:valuesthat describe the end-user perception. The next level shows the supporting Per-

8 Chapter 2. Counter Data

formance Indicators (PI), which describe system performance. After that comesthe Resource Performance Indicators (RPI), which are not dealt with in this re-port but could be treated the same way. See figure 2.3. The KPI:s and PI:s arecalculated on cell level and can be aggregated on BSC- or system level.

When the system performance is evaluated, the KPI- and supporting PI-values are used. To see if the system behaviour has changed after a softwareupdate, the KPI-values are analysed and when a KPI-value is found differentthe analysis continues further down the pyramid of figure 2.3 to see where thesystem change takes place. Since focus are on KPI- and PI-values, countersthat are not involved in the KPI-evaluations are often missed in the changedetection.

Chapter 3

Statistical HypothesisTesting

In this chapter the concept of random variables will be introduced, together withtheir distributions and how to perform statistical hypothesis tests.

3.1 Random Variables And Their ProbabilityDistributions

A deterministic function shows the same value over and over again if you havethe same input, but a random variable can change value every time you observeit, even if the input is unchanged. The value varies in an uncontrollable way,but the probability of its behaviour can often be determined.

The probability of a certain outcome x of a random variable X is denotedP (X = x) and is specified by the probability distribution of that random vari-able. The probability distribution is a kind of weight function, with highervalues for the outcomes that occur more often and much smaller values for theoutcomes that occur very seldom. If the variable is continuous, the probabilitydistribution is called probability density function, pdf, and is denoted fX(x).When the pdf is integrated over all possible values of X, its integral equalsone. An area that equals one is thus distributed over all the possible outcomesand the most probable outcome has the largest piece of the area. Figure 3.1is showing an example of a pdf. Any function f(x) ≥ 0,∀x ∈ < that satisfies∫< f(x)dx = 1 can be used as a probability density function. [2]

For a continuous random variabe X, the probability that X take values inA is defined as:

P (X ∈ A) =

∫A

f(x)dx

If A = (−∞, x], P (X ∈ A) is called the cumulative distribution function, cdf,denoted FX(x):

FX(x) = P (X 6 x) =

∫ x

−∞f(t)dt

The cumulative distribution function thus accumulates the probability from −∞up to a value x. [7] Figure 3.2 is showing an example of a cdf.

Eriksson, 2013. 9

10 Chapter 3. Statistical Hypothesis Testing

Figure 3.1: An example of a pdf.

Figure 3.2: An example of a cdf.

To describe the behaviour of a random variable, a measure of the centraltendency of its values and the dispersion around that value can be useful. Theexpected value, also called the mean value or average value, is a measure of thatcentral tendency. It is the average of many independent observations of that

3.1. Random Variables And Their Probability Distributions 11

particular random variable.[2] The expected value is defined as follows:

E(X) =

∫<xfX(x)dx = µ

The measure of dispersion around the central tendency is called the variance.It is defined as the average squared deviations from the mean: [7]

V (X) = E[(X − µ)2] =

∫<

(x− µ)2fX(x)dx = σ2

To get the dispersion in the same unit as X itself the square root of the varianceis often used, called the standard deviation, σ.

3.1.1 Independent and Identically Distributed RandomVariables

Random variables X1, ..., Xn are called iid random variables if they are inde-pendent and identically distributed. The random variables are identically dis-tributed if they have the same probability distribution and they are independentif:

fX1,...,Xn(x1, ..., xn) = fX1(x1)· · · fXn(xn)

This means that the knowledge of the value of one random variable does notaffect the knowledge of the other.

3.1.2 The Normal Distribution and the Central LimitTheorem

A commonly known distribution is the Normal Distribution, also called Gaus-sian Distribution. Many phenomena around us is approximately normally dis-tributed and it is used throughout much of the statistical theory. The math-ematical properties of the normal distribution also make it very useful. Theprobability density function of the normal distribution is calculated as follows:

fX(x) =1

σ√

2πe−

(x−µ)2

2σ2

And the cumulative distribution function:

FX(x) =1

σ√

2π

∫ x

−∞e−

(t−µ)2

2σ2 dt

Figure 3.3 shows the probability density function of normal random variableswith different standard deviation.

A random variable that is normally distributed with expected value µ andstandard deviation σ is denoted X ∈ N(µ, σ). When µ = 0 and σ = 1 you getthe standardized normal distribution with probability density function:

ϕ(x) =1√2πe−

x2

2

and cumulative distribution function:

Φ(x) =1√2π

∫ x

−∞e−

t2

2 dt


Figure 3.3: The pdf of normal distributions with standard deviation σ.

Figure 3.4: The α-quantile.

The x-value where the area of the probability density function above x equalsα is called the α-quantile. See figure 3.4. For a standardized normal distributionthe α-quantile is denoted λα.

Φ(−x) is the area of ϕ(x) from −∞ to −x. Since ϕ(x) is an even function

3.2. Statistical Description of Data 13

ϕ(−x) = ϕ(x) and the area of ϕ(x) from x to ∞ is equal to Φ(−x). Since thewhole area is one Φ(−x) = 1− Φ(x). [2] The total area below −x and above xis thus 2 ∗ Φ(−x). If that area is α, then x = λα/2. See figure 3.5. This fact isused in two-sided hypothesis tests.

Figure 3.5: Two-sided quantile for a standard normal distribution.

The normal distribution has many applications. One reason for that is thatthe sum of many iid random variables are approximately normally distributedwith mean value nµ and standard deviation σ

√n, where n is the number of

random variables. This is called the central limit theorem. The sample mean:

x =1

n

n∑i=1

xi

of a sequence of n iid random variables is thus approximately normally dis-tributed with mean µ and standard deviation σ/

√n, another fact used in the

hypothesis testing.Linear combinations of independent and normally distributed random vari-

ables are also normally distributed. If X1 ∈ N(µ1, σ1) and X2 ∈ N(µ2, σ2),then X1 −X2 ∈ N(µ1 − µ2,

√σ2

1 + σ22). [2]

3.2 Statistical Description of Data

A set of data, or sample, is treated as observations of a larger population. Whenthe data is described, the purpose is to describe the behaviour of the populationas accurately as possible. Even though the population does not have to be real,the name arises from the fact that in statistics, the object of interest is often agroup of individuals, a population.


We wish the sample to reflect the characteristics of the whole population.Thus it has to be drawn at random. A random sample of observations of therandom variable X is a set of iid random variables X1, ..., Xn each with thedistribution of X. [7]

The probability model for the population of the data is usually not known.It has to be estimated using the sample. The estimator is a function of thesample observations and such a function is called a statistic. It is a randomvariable since it is a function of random variables:

T = t(X1, ..., Xn)

Since it is a random variable, a distribution function describes its behaviour. Itis called the sampling distribution and it is derived from the distribution of thepopulation. For example the population mean, µ, is estimated using the samplemean:

X =1

n

n∑i=1

Xi

which is a statistic with a sample distribution and corresponding expected valueand variance. The expected value of the sample mean is the mean of the wholepopulation.

The sample variance, S2 is calculated:

S2 =1

n− 1

n∑i=1

(Xi − X)2

S is the sample standard deviation and the expected value of the sample varianceis the population variance, σ2.

3.3 How to Perform a Hypothesis Test

When performing a hypothesis test we want to test an assumption about arandom sample, x = (x1, ..., xn). The assumption could be that the sampleobservations come from a normal distribution, that the distribution mean hasa certain value etc. The assumption that you want to test is called the nullhypothesis, H0. The null hypothesis is tested against an alternative hypothesisH1 that is considered true if H0 is rejected. If H1 is a single interval, for exampleif the assumption is µ = 0 and H1 : µ > 0, the test is called one-sided and ifinstead H1 : µ 6= 0 (µ > 0 or µ < 0) the test is called two sided.

First you have to find a so called test statistic:

T = t(X), X = (X1, ..., Xn)

The test statistic is a function of the random variables that the sample observa-tions are observations of. The test statistic tells us how those random variablesbehave if the null hypothesis is true. It is based on knowledge of the distribu-tion under the null hypothesis, the null distribution, and different assumptionsrequires different test statistics with different null distributions. There are somewell specified test statistics to use for different kinds of tests.

H0 is rejected if the observation of the test statistic, tobs = t(x), belongs to aso called critical region, C. The critical region is chosen such that P (T ∈ C) = α

3.3. How to Perform a Hypothesis Test 15

Figure 3.6: The null distribution and rejection of H0.

if H0 is true. If the test is one-sided the critical region is the values above theα-quantile of the null distribution, see section 3.1.2. If the test is two sided thecritical area is below −tα/2 and above tα/2. See figure 3.6. The null distributionthen has its most probable values between −tα/2 and tα/2 and if tobs takesvalues there the assumption of H0 cannot be rejected. α is called the level ofsignificance of the hypothesis test and the test is called significant at level α ifthe hypothesis is rejected and the assumption is considered false. Usual valuesof α are 0.05, 0.01 and 0.001. [2]

Suppose that the value of tobs ∈ C. Then H0 is rejected, thus we assumethat the assumption we want to test does not hold. But it could also be thecase that the hypothesis is true and that tobs takes a value that is less probablebut still probable under the null hypothesis. Therefore α is the probability thatH0 is rejected if H0 is true. This is called type I error. Another kind of erroris the so called type II error, which is the probability of not rejecting H0 whenH0 is not true.

If H0 is not rejected this does not necessarily mean that H0 is true.

3.3.1 Using P-values

If the hypothesis test is one-sided, the P-value is the area of the null distributionabove the observed value of the test statistic, tobs, and is defined as follows:

P = P (T ≥ tobs)

This is shown in figure 3.7. If the P-value is smaller than the level of significance,α, the null hypothesis is rejected. This means that we do not believe in H0 ifthe result is unlikely when H0 is true. [2]

When the hypothesis test is two-sided a two-sided P-value is calculated:

P = P (|T | > tobs)


Figure 3.7: One-sided P-value.

Figure 3.8: Two-sided P-value.

The null hypothesis is rejected if P < α, and as seen in figure 3.8 the rejectionof H0 is more accurate for smaller P-values.

3.4 Comparing the Means of Two Samples

When comparing the means of two samples from normal distributions, a t-distributed test statistic is used. The density function of the t-distributionis shown in figure 3.9 with different degrees of freedom, df . The degrees offreedom is the number of samples minus the number of estimated parameters.[9] If df =∞, the t-distribution becomes the standard normal distribution.

The test is called a two-sample t-test. There are two different cases, one

3.4. Comparing the Means of Two Samples 17

Figure 3.9: The t-distribution.

where the variance of the two samples are considered equal and one wherethe variances are not equal. In both cases a t-test is used, but with differentappearance of the test statistic.

If the sample standard deviation of the two samples are considered equal,the test statistic is defined as follows:

T =X1 − X2

Sp

√1n1

+ 1n2

where Sp is called the pooled standard deviation, and S2p is an estimate of the

common variance: [8]

S2p =

(n1 − 1)S21 + (n2 − 1)S2

2

n1 + n2 − 2

The term df = n1 + n2 − 2 is the number of degrees of freedom and S21 and S2

2

are the respective sample variances.If the variances are not considered equal, σ2

1 6= σ22 , the test statistic is defined

as:

T =X1 − X2√S21

n1+

S22

n2

where S1 and S2 are the sample standard deviations. This test statistic is onlyapproximately t-distributed. The number of degrees of freedom is calculated as:[9]

df =(S21

n1+

S22

n2)2

(S21/n1)2

n1−1 +(S2

2/n2)2

n2−1


We want to test the null hypothesis, H0 : µ1 = µ2 (or µ1 − µ2 = 0) againstthe alternative hypothesis H1 : µ1 6= µ2. The test is thus two-sided. If thenull hypothesis is true, the test statistic is t-distributed with the given degree offreedom according to equal variance or not. The observation of the test statisticis:

tobs =x1 − x2

sp

√1n1

+ 1n2

if the variances are considered equal and:

tobs =x1 − x2√s21n1

+s22n2

if the variances are unequal. If |tobs| > tα/2,df the null hypothesis is rejected.tα/2,df means that the area above tα/2 of a t-distributed probability densityfuction with df degrees of freedom equals α/2. α is the pre-defined level ofsignificance. See figure 3.6. You can also calculate the P-value:

P = P (|T | > tobs)

and reject H0 if P > α.The test is based on an assumption that the samples are random and come

from normal distributions. Even if the distributions are not normal this kind oftest can still be used, since the test is based on the sample mean and the samplemean is approximately normally distributed if the number of observations islarge enough, according to the central limit theorem.

3.4.1 Why The Test Statistic Is t-Distributed

When we compare the mean of two different samples we want to say somethingabout the mean of the entire population, not just the observations drawn fromit. But since only samples of the two populations are available to us, we have toestimate the mean of the population using the sample mean. This gives rise touncertainty in the estimated mean and the sample mean has its own probabilitydistribution, expected value and standard deviation.

If the standard deviation is not known, it has to be estimated as well. Thisgives rise to more uncertainty. If the standard deviation is known: [2]

X − µσ/√n∈ N(0, 1)

Since s, the estimate of σ, has to be used instead of σ, X−µs/√n

has to be used

instead of X−µσ/√n

, where s is the estimate of the sample standard deviation:

s =

√√√√ 1

n− 1

n∑i=1

(xi − x)2

The stochastic variable to be studied is thus:

3.5. Comparing the Variance of Two Samples 19

T =X − µS/√n, T ∈ t(n− 1)

T is t-distributed with n− 1 degrees of freedom.If you exchange X with X1 − X2 and use the fact that:

X1 − X2 ∈ N(µ1 − µ2,

√σ2

1

n1+σ2

2

n2)

you get the test statistics for the two sample t-test, see section 3.1.2.

3.5 Comparing the Variance of Two Samples

Figure 3.10: The F-distribution.

The ratio of the sample variances of two independent samples from normaldistributions is F-distributed with n1 − 1 numerator and n2 − 1 denominatordegrees of freedom, where n1 and n2 are the number of observations of thesamples. [8] The F-distribution is shown in figure 3.10. This fact can be usedwhen testing if two samples from normal distributions have equal variance. Thetest statistic for the test is

F =S2

1

S22

The null hypothesis H0 : σ21 = σ2

2 is rejected if fobs > Fα/2,n1−1,n2−1 or iffobs < F1−α/2,n1−1,n2−1 for a two sided test. See figure 3.11.

When calculating a two-sided P-value for an F-distributed test statistic, caremust be taken as to whether the value of fobs is in the upper or lower tail of the


Figure 3.11: The pdf of an F-distributed test statistic and its two-sided rejectionregion.

F-distribution. If fobs is below the median of the null F-distribution it is in thelower tail and if it is greater than the median it is in the upper tail. The twosided P-value is:

P = 2 ∗ (1− P (Fn1−1,n2−1 > fobs))

if it is in the lower tail and:

P = 2 ∗ P (Fn1−1,n2−1 > fobs)

if it is in the upper tail.[5]The F-test is more sensitive to the normality assumption than the t-test,

since the central limit theorem can not be used.

3.6 Testing Normality

The assumptions that has to be made to use a statistical hypothesis test ofthe kind described in section 3.4 and 3.5 is that the samples are independentand identically distributed, hopefully with a normal distribution. To test thenormality assumption two different methods are considered in this report, oneof them visual.

3.6.1 Histogram

A histogram is a graphical representation of the distribution of a set of observa-tions and it can be used for density estimation. The values of the observationsare divided into bins and the height of each bin represents the number of obser-vations in that bin. See figure 3.12. One rule of thumb for the number of binsis the square root of the number of observations. [10]

3.6. Testing Normality 21

Figure 3.12: Histogram of random samples from a normal distribution.

Figure 3.13: Histogram of a large number of random samples from a normaldistribution with a fitted normal pdf.

Since the histogram is an estimate of the probability distribution for a set ofdata, a probability density function can be fitted to the data if the populationdistribution is known. If we want to test if the observations come from a specific


distribution we can compare the shape of the histogram to the fitted pdf. Seefigure 3.13. If we compare the shape of figure 3.12 with the shape of figure 3.13,with the same probability distribution but a much larger number of observations,we see that the shape of the histogram approaches the shape of the populationpdf when the number of observations gets larger.

3.6.2 Lilliefors Test

The Kolmogorov-Smirnov test is a statistical test to determine if a set of obser-vations comes from a normal distribution. The test statistic for the Kolmogorov-Smirnov test, D, is the largest absolute difference between the sample cdf, Fn(x),and the population cdf, F (x). [7] See figure 3.14.

D = supx|F (x)− Fn(x)|

This test statistic is distribution free and a table must be used for the criticalvalues.

Figure 3.14: Definition of the test statistic for the Kolmogorov-Smirnov andLilliefors test.

When the mean and variance are not specified and have to be estimated fromthe sample, the Kolmogorov-Smirnov test has to be modified. This is done byusing the Lilliefors approach of the Kolmogorov-Smirnov test, where the tablesfor the test statistic is changed to fit a normal distribution with estimated meanand variance. [6] The test statistc is then:

D = supx|F ∗(x)− Fn(x)|

where Fn(x) is sample cdf and F ∗(x) is the normal population cdf with estimatedmean, µ = x, and variance, σ2 = s2. If D exceeds the critical value in the

3.6. Testing Normality 23

Lilliefors table, the null hypothesis that the observations come from a normaldistribution is rejected.

Chapter 4

Time Series Analysis

In this chapter we will learn how to estimate trend and seasonal components in atime series and how that can be used for testing assumptions on the remaining noiseterm.

4.1 Time Series and Hypothesis Tests

A time series is a set of observations {xt}nt=1, each one being recorded at aspecific time t. [3] If the time series consists of observations of random variables,i.e. if xt is an observation of Xt, the time series is a realization of a stochasticprocess. Like in the case of random variables, every time series is one of manypossible realizations of the same underlying stochastic process. Thus two timeseries that are observations of the same process are not necessarily equal.

When the set of times is a discrete set, the time series is a discrete-timeseries. This is the case when observations are made on fixed time intervals.Another property of a time series is stationarity. A time series is said to bestationary if it has the same behaviour regardless of when you observe it. Moreprecisely, Xt, t = 0, 1, ... and the time shifted series Xt+h, t = 0, 1, ... has to havethe have the same statistical behaviour.

In many applications the behaviour of a given time series is not specified andhas to be analysed. You have to look for dependence among the observations,periodic behaviour, long-term trends, etc. A suitable model for the time serieshas to be found and often the signal has to be separated from the noise term.A probability model for the remaining noise term has to be found and this canbe used as a diagnostic tool for the accuracy of the model of the signal.

When statistical hypothesis tests are to be performed on a time series, wewish its behaviour to be like that of iid random variables. If a sequence ofrandom variables is iid and the mean of the series is zero the stochastic processis called an iid-process and is denoted IID(0, σ2). If there is a seasonal be-haviour in the time series there is a dependence among the observations. Thisdependence has to be eliminated for the hypothesis test to be used.

Eriksson, 2013. 25

26 Chapter 4. Time Series Analysis

4.2 Estimation of Trend, Seasonality and theRemaining Noise Term

In a time series we can often find an underlying trend and a seasonal variation,a repeated pattern over a shorter time interval. The seasonal variation can bemodelled as a periodic component, st, with a fixed period p, such that st = st+p.A model can then be created for the time series by dividing the series into atrend component mt, the seasonal component st and a noise term Yt. The noiseterm is a random variable that describes the deviations from the model. Thiskind of model is called the classical decomposition:

Xt = mt + st + Yt

Since the trend is described in the trend component, the mean value of the noiseterm is zero. The sum of the periodic component over a whole period is alsozero: [3]

p∑k=1

sk = 0

Figure 4.1: A time series and its estimated seasonal component and trend.

Figure 4.1 shows a time series with estimated trend and seasonal behaviour.The trend is a deterministic component that describes the level of the signal.The seasonal component is also a deterministic component that describes theseasonal behaviour around the trend. The remaining stochastic noise term isthe deviations of the signal from those components.

To estimate the seasonal component, a suitable period is choosen, withp = 2q if p is even. Then a trend is estimated using a moving average filter,which is a weighted average of neighbouring points that is used to flatten the

4.2. Estimation of Trend, Seasonality and the

Remaining Noise Term 27

fast variations in the data. This leaves the slow components, the trend. Aspecial kind of moving average filter is used that is suitable for estimation ofseasonal components: [3]

mt = (0.5xt−q + xt−q+1 + ...+ xt+q−1 + 0.5xt+q)/p, q < t ≤ n− q

If p instead is odd, then p = 2q + 1 and:

mt =1

2q + 1

q∑j=−q

xt−j , q + 1 ≤ t ≤ n− q

After the trend is estimated, then for every k = 1, ...p, the average of thedeviations from the trend is calculated, wk. How many points that can be usedfor calculating the average of the deviations is decided by the number of periodsin the time series when the first and last q values is eliminated in the trend-estimation. The deviations from the trend is calculated for every k = 1, ...pas:

{(xk+jp − mk+jp), q < k + jp ≤ n− q}

and wk is then calculated by taking the average of those deviations over thenumber of periods fitted in the data available. To make sure that the seasonalcomponent sum to zero, the seasonal component is estimated as:

sk = wk −1

p

p∑i=1

wi, k = 1, ..., p

since

p∑k=1

sk =

p∑k=1

(wk −1

p

p∑i=1

wi) =

p∑k=1

wk − p ∗1

p

p∑i=1

wi =

p∑k=1

wk −p∑i=1

wi = 0

and since s is periodic with period p, sk = sk−p for k > p.Now the seasonal component has been estimated. To estimate the trend

component the seasonal component is first removed from the original data. Thedeseasonalized component, dt, is defined as:

dt = xt − st, t = 1, ..., n

Then another moving average filter is used to estimate the trend since the earliertrend estimation we did was just for calculating the seasonal component. Thistrend estimation uses a moving average filter of the type:

mt =1

2q + 1

q∑j=−q

dt−j , q + 1 ≤ t ≤ n− q

It also eliminates the components of fast variations and is thus a low-pass filtersince the components of low frequencies passes the filter. This is not the only wayof estimating the trend. Instead of this trend estimation, a constant representingthe average value could be used.


To get the noise term the trend is eliminated from the deseasonalized timeseries and we get:

Yt = xt − st −mt

The estimated noise term of the time series in figure 4.1 is shown in figure 4.2.

Figure 4.2: The estimated noise term of figure 4.1.

4.3 Testing the Independence Assumption of theNoise

As discussed throughout this report, for many hypothesis tests we need thesamples to be iid. When the mean of two samples is to be compared, the datais averaged over the number of observations. If the number of observations isa whole number of periods, the seasonal behaviour is extinguished. Thus thebehaviour of the noise term can be used to test the assumptions of the hypoth-esis test. We want to check if the noise term is approximately independentIID(0, σ2) and if it is normally distributed N(0, σ2).

To see if the noise sequence is independent a first test is to plot the sequence.A typical behaviour if there are dependence among the observations is that manyobservations in a row are having the same sign. Figure 4.3 shows an independentnoise sequence that is normally distributed.

Another more distinct method to see if the estimated noise sequence is in-dependent is to use the so called sample autocorrelation function, sample acf.The autocorrelation of a sequence at lag h is the correlation between the ob-servations in the sequence. The autocorrelation function, acf, for a stationary

4.3. Testing the Independence Assumption of the Noise 29

Figure 4.3: Normally distributed iid noise.

time series is defined as:

ρX(h) =γX(h)

γX(0)= Corr(Xt+h, Xt)

where γX(h) = Cov(Xt+h, Xt) is the covariance defined as:

γX(r, s) = Cov(Xr, Xs) = E[(Xr − µX(r))(Xs − µX(s))]

To test the independence of the noise sequence the sample acf is used as anestimate of the acf, since we do not have a model for the noise sequence butonly observed data. The sample autocorrelation function is:

ρ(h) =γ(h)

γ(0), −n < h < n

where the sample autocovariance function is:

γ(h) =1

n

n−|h|∑t=1

(xt+|h| − x)(xt − x), −n < h < n

and the sample mean is calculated as:

x =1

n

n∑i=1

xi

If the data is containting a trend the sample acf will exhibit slow decaywhen h increases and if there is a periodic component it will exhibit a similar


Figure 4.4: The sample acf of a time series with a trend and seasonal behaviour.

Figure 4.5: The sample acf of an iid-sequence.

behaviour with the same periodicity. [3] Figure 4.4 shows the sample acf for asequence with trend and seasonal behaviour.

For an iid sequence the autocorrelation is ρX(0) = 1 and ρX(h) = 0, |h| > 0,so if the sample acf is close to zero for all nonzero lags, the noise sequence can

4.3. Testing the Independence Assumption of the Noise 31

be regarded as iid-noise. See figure 4.5. For a formal test, one can use thefact that for large n the sample autocorrelations of an iid sequence with finitevariance are approximately iid with distribution N(0, 1

n ). We can thereforecheck if the observed noise sequence is consistent with iid noise by examinethe sample autocorrelations of the residuals. If the sequence is iid, 95% of thesample autocorrelations should fall between the bounds ±1.96/

√n. [3]

To check if the noise sequence is normally distributed a histogram or theLilliefors Test can be used, see 3.6.1 and 3.6.2.

Chapter 5

Analysis of Counter Data

In this chapter we will get to know the behaviour of the counter data.

5.1 Selection of Data to Work With

When performing an analysis of counter data, a set of data that represents ausual behaviour of the system is preferred. Therefore data associated with acommonly used KPI-value and its underlying PI- and counter-values is used.The data is aggregated to BSC-level and one sample represents the latest hour.

Figure 5.1: An example of counter data.

Figure 5.1, figure 5.2 and figure 5.3 show some examples of counter data. Infigure 5.2 we see that some data points are missing. Missing values are handledin section 5.4.

Eriksson, 2013. 33

34 Chapter 5. Analysis of Counter Data

Figure 5.2: An example of counter data with missing values.

Figure 5.3: An example of counter data with an outlier.

5.2. The Daily Profile 35

In figure 5.3 we see a data point that lies far outside the other data points.This can be an indication that there is something wrong with that particularcollection of data and that point needs to be removed for the analysis to be asaccurate as possible. The so called outliers are handled in section 5.3.

As seen in all the figures shown here, the data exhibits a seasonal behaviour.This seasonal behaviour is the variations over day and comes from the trafficpattern with peak hour or hours, called busy hour(s) since the traffic is high.

5.2 The Daily Profile

The data exhibit a similar behaviour every 24 hours. This is the daily profileand it is estimated using the technique in section 4.2.

Figures 5.4 and 5.5 show the estimation of the trend and daily profile fortwo counters and their remaining noise term.

The daily profile can also be seen in the autocorrelation function. In figure5.6 the sample acf, ρX(h) exhibits a slow decay for larger values of h and theperiodicity p = 24 is clearly seen. This is a typical behaviour for a time serieswith a periodic component, see section 4.3.

5.3 Outliers

First the classical decomposition model of the data, see section 4.2, is estimatedignoring values that are not numbers. The variance of the data is estimated andthen the deviations from the model are calculated. The data point where thedeviation is largest is eliminated and the variance is estimated again. If the newvariance is significantly smaller than the previous variance, the removed point isregarded as an outlier. If the variance is not significantly smaller it means thatthe point with the largest deviation from the model is still close to the otherdata points and it is put back since it was not regarded as an outlier. Figure5.7 shows data with a removed outlier.

This method is a heuristic, since the one-sided hypothesis test applied is usedunder violated assumptions. There is a strong correlation between the first andsecond sample. But when tested on the data studied, it seems that the datapoints that lie far outside the other data points are removed.

After the outliers have been removed a new estimation of daily profile andtrend has to be made, since the outliers could have affected the previous result.

5.4 Missing Values

For the estimation of the daily profile, the time series has to have 24 samples aday. Therefore a sample missing has to be regarded as a missing value for theperiod to be correct. If there is a missing value at t = i, the value of sk wheresk = sk+jp = si is estimated using the average of the deviations:

{(xk+jp − mk+jp), q < k + jp ≤ n− q}but with the addition of the criteria k + jp 6= i, see section 4.2.

The missing value xi is then replaced with the estimated value for thatparticular time, mi + si. Missing days are not treated in this thesis work.


Figure 5.4: An example of estimated trend and daily profile for counter data.

5.4. Missing Values 37

Figure 5.5: An example of estimated trend and daily profile for counter data.


Figure 5.6: An example of the sample acf for counter data with a daily profile.

Figure 5.7: Counter data with a removed outlier.

5.5. Testing Independence and Normality of the Noise 39

5.5 Testing Independence and Normality of theNoise

The noise term is what is left when the daily profile and the trend are eliminatedfrom the data. In this report it is seen as the deviations around the mean value,when the data is averaged over a whole number of days. This is due to the factthat the seasonal components averaged over a whole period is zero, see section4.2, and that the mean is approximately constant, see figure 5.4 and 5.5. Thusthe variance of the noise term is used in the t-test for equal mean in the changedetection method. To see if the noise term is iid, we use the autocorrelationfunction as seen in section 4.3.

Figure 5.8: An example of the sample acf for the noise.

For most of the counters there are still dependence in the noise term, sincethe autocorrelation has more than small deviations from zero for non-zero lags.In most cases the sample acf looks like something between figure 5.8 and figure5.9. This looks reasonaly good if we compare with the original signal, an exampleshown in figure 5.6.

For the normality assumption of the noise term, a histogram and the Lil-liefors test is used, see section 3.6.1 and section 3.6.2. Most of the countersexhibit a behaviour close to that of a normally distributed population, butsince the number of points is less than 200 it is not always easy to see in ahistogram. The Lilliefors test with a 0.01 level of significance is used and thenthe hypothesis of normality cannot be rejected for most of the counters. Seefigure 5.10.

Sometimes the analysis of the whole week shows that the noise is not nor-mally distributed. When changing the data to only weekdays the analysis looksdifferent. Then the hypothesis of normality can not be rejected.


Figure 5.9: An example of the sample acf for the noise.

Figure 5.10: Testing the normality assumption of the noise.

Chapter 6

The Change DetectionMethod

This chapter presents the change detection method and the theory behind it. In thenext chapter we will see the results from applying the change detection to counterdata.

6.1 The Change Detection Method and ItsLimitations

A shown in chapter 5, the counter data can be modelled as a stochastic timeseries with a daily profile. To detect changes in the data we cannot just compareif the values are the same, due to the stochastic behaviour. We also need toknow which changes to detect. In this thesis work focus are on changes in themean value and in the daily profile, since it is two factors that characterize thedata.

The change detection method uses about two weeks of data. The first week,or set of data, is a reference week or reference data. For this week we want tofreeze as many factors as possible to be able to detect changes due to softwarechanges and not to changes in the environment. The second week, or set ofdata, is the test week or test data.

Different days of the week have different traffic patterns. Since only a limitedamount of data is used, there will be too few data-points to estimate a modelfor each day, and therefore different days has to be treated as equal. The testdata has to be from the same weekdays as the reference data to eliminate therisk that changes due to difference between weekdays will appear.

No consideration is taken to long term trends and seasonal patterns overlonger periods than 24 hours, and this can affect the result. Therefore thechange detection can only answer the question if the test week is significantlychanged from the reference week. Care must be taken when choosing the datafor the reference week since it has to behave as a typical week if we want to saysomething about whether the test week behaves as a typical week and not justas the reference week.

One thing to be done to get a more correct model, and at the same time use

Eriksson, 2013. 41

42 Chapter 6. The Change Detection Method

the amount of data provided, is to use only weekdays since there is a greater dif-ference in traffic pattern between weekdays and weekends than there is betweendifferent weekdays. (Note that differences in this pattern can occur in differentcountries.) Since less data can be used, a trade-off must be made between usingmore data or a better model.

One more limitation of the change detection method is that it can showdifferent results when performing it on different number of days. Care must betaken to what you want to achieve by using the method. If the data changesin the end of the week, a change will occur according to the change detectionmethod if you test against the whole week, but if you only choose the beginningof the week, the result will be different since the change was in the end of theweek.

Before the change detection takes place, missing values and outliers are han-dled, see section 5.4 and section 5.3. Then the daily profile is estimated, thenoise variance is calculated and statistical hypothesis tests are performed to testwhether the mean value and/or the daily profile of the data has changed.

6.1.1 Change Detection of Mean Value

The change detection focuses on the mean value of the counter data. This is achange easy to understand and to calculate. As discussed before, comparisonof the mean of the two data-sets cannot just be performed by comparing if thetwo values are equal. Due to the stochastic behaviour, the two means can differbut still be considered equal in a probabilistic sense.

When performing the change detection of the mean value, the data is aver-aged over day and thus the daily profile is eliminated. Since the data is alsoaveraged over the same number of days and the same weekdays before and afterthe software change, we judge that also variations due to differences betweenweekdays are eliminated reasonably well.

The noise term describes the deviations from the mean under the sameprobability model. Therefore the variances of the noise terms is used in thehypothesis test.

For the hypothesis test of equal mean we use a two-sided t-test, see section3.4. Since we have to know if the variance is equal or not, a two-sided hypothesistest of equal variance has to be performed on the noise variance prior to thet-test. The F-test of equal variance is described in section 3.5.

If the hypothesis of equal mean is rejected at the pre-defined level of signif-icance, α, the test result is considered significant and a change has occurred inthe test data. To tune the parameter α, data without a software change canbe used. If a change is detected, the P-value can be used as a measure of howaccurate the rejection of the hypothesis that there is no change is. If the P-valueis less than α, the hypothesis of no change is rejected and then the closer tozero the P-value is, the more can we trust the detection of a change, see P-valuesection 3.3.1.

6.1.2 Change Detection of Daily Profile

The change detection is also done on the daily profile. The daily profile looksdifferent over the week and is affected by changes in behaviour of the mobileusers, mobile subscription conditions, weekends, larger events in the area, etc. A

6.2. Normality Assumption 43

model of the daily profile is estimated using the reference week and the varianceof the noise term is calculated. Then the test data after software update isused. The deviations from the daily profile of the reference week is calculatedand de-trended since the trend change is not considered here. The remainingnoise term should be of the same size as the reference noise if the model is wellsuited also for this data. If the variance of the noise term is significant greaterthan the variance of the reference week noise the model for the daily profile isnot suited for this data and thus the daily profile has changed. Since it is veryunlikely that the model is better fitted for the test week, it is only relevant toknow if the variance is significantly greater than the variance of the referenceweek. Thus a one-sided hypothesis test is used.

6.2 Normality Assumption

If the normality assumption of the counter data is violated, the result from thechange detection of daily profile may or may not be correct since the test isbased on the fact that the data is normally distributed. Therefore a warningoccurs if the normality assumption is rejected at the 0.01-level of significance.

The change detection of the mean is not so sensitive to the normality as-sumption, see section 3.4.

6.3 Level of Significance

If the observed value of the test statistic is close to the rejection region, theresult of the change detection method can change when the level of significanceis changed. For smaller values of α, a rejection of the null hypothesis of nochange is a stronger indication that a change has occurred. Common values ofα are 0.05, 0.01 and 0.001, see section 3.3. This can also be seen in the P-valuesince a P-value around the level of significance can show which value of α thatchanges the outcome of the hypothesis test.

A better scenario would be that of two reference weeks instead of one, whereone of the reference weeks could be used for choosing an appropriate value of αfor that particular environment. You could also change the value of α dependingon the use of the method. If you would like all changes to be detected, a largervalue of α is suitable, for example 0.05, and if you do not want any false alarm,a smaller value of α is suitable, for example 0.001.

44 Chapter 6. The Change Detection Method

Chapter 7

Applying the ChangeDetection Method ToCounter Data

In this chapter we will see the results of the change detection method on counterdata with known changes and on counter data without changes.

7.1 Selection of Data

When performing change detection on counter data, it is preferable to use aset of data on which the change detection method can be analysed. Thereforea set of data with a feature change that affects a KPI called Throughput andits underlying PI:s and counters is used. A measure of user data volume isalso used, since it could affect the other results. All data comes from EGPRSDownlink (DL) transfers, which means that it is data and not speech and thatit is directed from the network to the mobile.

To know if the change detection method is working correctly, also data with-out any known changes is used.

7.1.1 Data Volume

The User Data Volume is the data that the end-user, the subscriber, pays for.The User Data Volume is not an end-user KPI, but in general if the performanceof the network improves then the users will be able to transfer a greater volumeof data within the same time. Thus, this measure is important to monitor, sincean unexpected change in data volume may indicate performance problems.

If the data volume is too low, this indicates that the traffic is very low anddeviations found in other KPIs and PIs may be due to too scarce data ratherthan ”true” system effects. The user data volumes used in this thesis work arehigh and should not affect the result in that way.

Eriksson, 2013. 45

46 Chapter 7. Applying the Change Detection Method To Counter Data

7.1.2 The Throughput KPI and Underlying PI:s andCounters

The Throughput is a KPI that measures the total percieved transfer velocityof the data volume that is going from the network out to the user on all datachannels. It is thus a measure of end-user perceived performance, which is thedefinition of a KPI, see section 2.3.1. The Throughput is measured in [kbit/s].Figure 7.1 is showing some underlying PI:s and counters associated with theThroughput.

Figure 7.1: Relations between the KPI, PI:s and counters used.

What the Simultaneous TBFs per E-PDCH shows is the number of ”users”carried on each individual Packet Data CHannel (PDCH). It is a measure ofthe traffic load in the system. This is also the case for the PI Average numberof simultaneous EGPRS DL TBFs in Cell. TBF stands for Temporary BlockFlow and it is the logical identifier for the data to be sent to one Mobile Station(MS). Simultaneous TBFs per E-PDCH is dependent on two counters calledDLTBFPEPDCH and DLEPDCH. The formula for calculation is:

Simultaneous TBFs per E-PDCH =DLTBFPEPDCH

DLEPDCH

The Radio Link Bitrate per PDCH measures the performance of the radiointerface between mobile and network per channel. If each PDCH is thoughtof as a ”radio pipe” through which data can be transferred then the measuredradio link bit rate represents the average size of each such ”radio pipe”. Thereare many different influences on the quality of a radio link: signal strength;interference; time dispersion and more. The formula for the Radio Link BitRate (DL) per PDCH is:

Radio Link Bit Rate =MC19DLACK

20 ∗MC19DLSCHED[kbit/s]

7.2. Interpreting the Figures 47

The Average Maximum Number of Time Slots (TS) reservable by MSs isa measure of how much of the data channels that the MSs can reserve. Theformula for the Average Maximum Number of TS reservable by MSs is:

Average Max Nr of TS Reservable =MAXEGTSDL

TRAFF2ETBFSCAN

The Multislot Utilization shows, in percentage, the amount of mobiles thatget as many packet data channels as they can handle. Typically the MultislotUtilization is high at low traffic hours while it decreases at high traffic hours.

7.2 Interpreting the Figures

In the figures the test data is plotted together with the reference data, with thecorresponding mean values. The header tells us which KPI, PI or counter thatis studied and the question ”Significant change of mean” is answered using thechange detection method at the level of significance given by α. The P-value isdefined in section 3.3.1.

When plotting the daily profile, the daily profile of the reference data isplotted together with the test data. The question ”Significant change of dailyprofile” is answered using the method described in section 6.1.2 at the level ofsignificance given by α.

7.3 Counter Data Without Known Changes

The data is taken from the same BSC, the same software release and two weeksin a row. The KPI:s and counters are the ones described in section 7.1. Worthnoticing is that there is no change in the User Data Volume, see figure 7.2.This could be an indication that the end-user behaviour is similar and shouldtherefore not affect the result.

Detection of changes in the KPI, in 4 of the 5 PI:s and in the 6 counters donot occur. Figure 7.3, figure 7.4 and figure 7.5 are showing some examples of theresult from the change detection. The PI that is considered changed is shownin figure 7.6. The reason for the change is not known. As seen around sample150, the change could be due to outliers, but when the 3 samples creating theoutlying values are removed there is still a significant change of mean value. Thisis seen in figure 7.7. Another reason could be that there is something wrongwith the underlying counters for a day around sample 100 or that something inthe environment changed that day, that only had an impact on this PI.

When performing an hypothesis test of equal mean, an hypothesis test ofequal noise variance is first performed. On the 0.01-level of significance, thehypothesis of equal noise variance can not be rejected, except in one case. Thisis in the PI Multislot Utilization, discussed earlier and shown in figure 7.6.

7.3.1 Level of Significance

Since the data used is unchanged, the method should not find any changes exceptin special cases. Therefore this data can be used for tuning the parameter α.There is no detection of a change for α = 0.01, both when using a whole week


Figure 7.2: No significant change in User Data Volume.

7.3. Counter Data Without Known Changes 49

Figure 7.3: No significant change on KPI-level.


Figure 7.4: No significant change on PI level.

7.3. Counter Data Without Known Changes 51

Figure 7.5: No significant change on counter level.


Figure 7.6: Change detected.

7.4. Counter Data With Known Changes 53

Figure 7.7: Change still detected when the possible outliers are removed.

and when using only weekdays, except in the PI mentioned earlier. That onehas a rather low P-value, even zero for the daily profile, see figure 7.6, and therejection of the hypothesis of no change is certain. When using α = 0.05 onthe other hand, there is a significant change in some of the counters when usingonly weekdays. Therefore α = 0.01 could be a suitable level of significance inthis case, and this is going to be used when detecting changes in the same KPI,PI:s and counters of data with changed software.

7.4 Counter Data With Known Changes

The data chosen is the KPI, PI:s and counters described in section 7.1, witha feature on as reference and the feature turned off as test data. The effect ofthe feature and the settings that are used, is that it gives fewer resources to thesame number of users, in order to decrease the total resource consumption inthe system. As a consequence, there will be impacts on the end user experiencedThroughput due to an increased number of TBFs on each PDCH. The feature issaving resources and that has a price in perceived velocity of the data volume tothe users, thus the Throughput decreases when the feature is activated. Whenthe feature is turned off, the Throughput should increase. The change detectionof the Throughput is seen in figure 7.8.

Figure 7.9 is showing the change detection of the Number of SimultaneousTBFs on Each PDCH. The feature will affect this PI in such a way that itwill increase when the feature is activated and thus decrease when the feature isturned off. This is due to the fact that system sets up fewer physical channels forthe same amount of data. The change in the Number of Simultaneous TBFs onEach PDCH should also be seen in the underlying counters DLTBFPEPDCH


and DLEPDCH, where the ratio DLTBFPEPDCH/DLEPDCH should decreasewhen the feature is turned off, see section 7.1.2. DLTBFPEPDCH and DLEPDCHare shown in figure 7.10 and 7.11.

Figure 7.8: Change detection of Throughput.

Figure 7.9: Change detection of Simultaneous TBFs per PDCH.


Figure 7.10: Change detection of DLTBFPEPDCH.

Figure 7.11: Change detection of DLEPDCH.


We should probably see a different behaviour of the Radio Link Bit Rate.Fewer channels are used and that may improve the radio environment by de-creasing the interference. Hence, the velocity of the radio pipe, the Radio LinkBit Rate, may increase. We therefore expect the Radio Link Bit Rate to godown when the feature is turned off. This is seen in figure 7.12. Since theRadio Link Bit Rate is proportional to the counter MC19DLACK and inverseproportional to MC19DLSCHED, we expect a drop of level in MC19DLACK ora higher value for MC19DLSCHED, see section 7.1.2 and figure 7.13 and 7.14.

Figure 7.12: Change detection of Radio Link Bitrate.

Furthermore, as the feature strives to increase the resource utilization byputting more TBFs on each individual PDCH, there will also be an impact onthe number of channels allocated to a user. The amount of mobiles that get asmany channels as requested is lower with the chosen feature settings. This isthe PI Multislot Utilization and it is shown in figure 7.15.

The Average Maximum Number of TS reservable by MSs shown in figure 7.16should increase when the feature is activated. As the Average Maximum Numberof TS Reservable by MSs changes, MAXEGTSDL and TRAFF2ETBFSCANhas to change as well, see section 7.1.2. Figures 7.17 and 7.18 are showning this.

The change detection indicates that the User Data Volume is changed, asseen in figure 7.19. This is a small change and should probably only affect theother counters slightly.

Regarding the change detection of the daily profile, we do not know how thatwould be affected by the feature. Worth noticing is that we can see changes inthe daily profile in 11 of 13 cases when the feature is turned from on to off.Figure 7.20, figure 7.21, and figure 7.22 show some examples of the changedetection of daily profile.

At the 0.01-level of significance there is no change in the result of the change


detection of mean when only weekdays instead of a whole week are used. As forthe hypothesis test of equal noise variance at the same level of significance, seesection 3.5 , the result varies and there is no particular pattern to be seen.

Figure 7.13: Change detection of MC19DLACK.

Figure 7.14: Change detection of MC19DLSCHED.


Figure 7.15: Change detection of Multislot Utilization.

Figure 7.16: Change detection of Average Maximum Number of TS reservableby MSs.


Figure 7.17: Change detection of MAXEGTSDL.

Figure 7.18: Change detection of TRAFF2ETBFSCAN.


Figure 7.19: Change Detection of User Data Volume.

Figure 7.20: Change Detection of Daily Profile.

Chapter 8

Discussion

In this chapter the choice of method will be motivated and topics related to it willbe discussed. The results will be analysed and thoughts on how to use the methodwill be presented. Future work on the subject will also be discussed. Finally theconclusions regarding this thesis work will be presented.

8.1 Why Time Series Analysis and StatisticalHypothesis Testing

The problem formulation is if we can detect if the behaviour of a counter haschanged after a software update. We know the time when the change did ordid not occur and we have a data set before and one after. Therefore it seemsstraightforward to formulate a hypothesis about whether the mean of the data-set has significantly changed or not. Due to the stochastic behaviour of the datawe can not just compare if the mean values are the same, the error term has tobe taken into account. Thus the choice of hypothesis testing is clarified.

For a hypothesis test to work, it is helpful if the data-set is an independentand identically distributed sample and hopefully from a normal distribution.But since the data-set is given in form of a time series with seasonal variationsthe independence assumption is not valid. Since it is so important something hasto be done about it. Time series analysis is used. When the seasonal componentand the trend are estimated and eliminated something can be said about theremaining noise term. Its variance gives us the magnitude of the variations ofthe data when the seasonal behaviour is averaged out. Therefore the noise termcan be used to see if the independence assumption is justified. If the noise termcan be assumed to be iid, we know that the data averaged over the period canbe treated as iid.

The second question is if the normality assumption can be used. Since weuse the average value over 24 hours we can use the central limit theorem, section3.1.2, with n=24. This is large enough to assume that the mean value over aperiod is drawn from a normal distribution and thus the statistical hypothesistests can be performed. The normality of the noise terms is tested using theLilliefors test, section 3.6.2, and the hypotheis of normality can not be rejectedin most cases.

Eriksson, 2013. 63

64 Chapter 8. Discussion

8.2 Problems With Hypothesis Tests

One problem with hypothesis testing is that if H0 is not rejected this does notnecessarily mean that H0 is true. To adress this problem different levels ofsignificance can be used depending of what you want to see. If the value of αis higher, the risk of detecting a change where there is none is larger and if thevalue of α is lower, changes could be missed. This is up to the user to decide.

8.3 Analysis of the Result

Are the necessary assumptions of the method met sufficiently well, and does themethod work as expected?

8.3.1 Assumptions

It seems reasonable to assume that the iid-assumption could be used, since thedependencies in the noise is small for most of the counters, see section 5.5.

In most cases the normality assumption cannot be rejected at the 0.01-levelof significance. This does not necessarily mean that the hypothesis of normal-ity is true, but we can assume that the normality assumption is a reasonableapproximation in this case for the change detection to be good enough. Careshould be taken though, when using the change detection of daily profile fornon-normal data, therefore the method uses a warning.

8.3.2 Change Detection

When the change detection is tested on data where the software is unchangedand on data with known software update, the results seem to be as expectedin most cases. Thus the method seems to work reasonably well under givenconditions. When using only weekdays instead of a whole week the changedetection did not change results at the 0.01-level of significance, thus we canassume that both approaches could be used.

8.3.3 Weekdays Versus Week

The analysis of the noise term sometimes shows that the noise is not normallydistributed when we use a whole week, but that the hypothesis of normalitycan not be rejected when we use only weekdays, see section 5.5. This can be anindication that the weekends have a different behaviour then the weekdays andwe have treated them as equal.

When all days of the week are treated as equal, the model is not correctand the noise variance will be over estimated. When the noise variance is overestimated, the changes are less likely to be detected.

If we instead only use weekdays, we get less data to work with and a lessaccurate estimation of the variance. A tradeoff must be made between usingmore data or a better model, as discussed in section 6.1. This scenario is stronglydependent on the magnitude of the change of user behaviour between weekdaysand weekend.

8.4. How the Method Could Be Used 65

Instead of having to chose between a whole week and only weekdays, bothcan be used and the results could be compared. If the results differ, a closeranalysis should be made.

8.4 How the Method Could Be Used

To reduce the amount of data to analyse, the method could be used in sucha way that it only gives an alarm when a change is detected. It could startwith changes with a small P-value, to make sure that the changes that aremore certain are seen. Then the level of significance could increase and moreuncertain changes could be more closely analysed, since the method could detectfalse changes for larger values of α.

8.5 Future Work

The change detection method has so far only been implemented in Matlab. Someof the functions are hand-written and some functions from Matlab’s StatisticalToolbox are used. The data is collected from an Access Database and writtento a text-file, which is later read into Matlab. This requires some work by handthat probably could be done by a script.

A moving average filter is used for modelling the trend when the data isdeseasonalized, see section 4.2. But when using the change detection of meanvalue, the average value is used. If the average value was used for the model ofthe trend as well, this could give a more correct estimation of the noise variance.

The method could be improved, if the model allowed for different means anddaily profiles for the weekdays and weekends, respectively. This would howeverrequire some more data for the estimation of weekends.

The missing values should perhaps be treated as missing and not replacedwith the value of the model. Since the noise is zero for those points, it couldaffect the estimation of the noise variance. This is on the other hand a verysmall problem, since the missing values are very few.

The method should be modified to handle outliers in a more theoreticallycorrect way.

The method also has to be modified if used on data without the daily profile.If there is no seasonal behaviour at all, the data is only de-trended and theremaining noise term is analysed as in the first case. The change detection ofmean value is also the same.

Worth investigating is if the environmental factors can be eliminated insome way from the data, using more knowledge of what the data stands for.Then the method could detect changes due to the software update and not toenvironmental changes.

8.6 Conclusions

The counter data is behaving as a stochastic time series with a daily profileand the change detection can be done by estimating the daily profile and thevariance of the noise term and perform statistical hypothesis tests if the mean

66 Chapter 8. Discussion

and/or the daily profile of the counter data before and after the software updatecan be considered equal.

When the data is averaged over day, the daily profile is eliminated. Sincethe data is also averaged over the same number of days and the same daysof the week before and after the software change, we judge that even changesdue to differences of weekdays are eliminated reasonably well. Consideration ofseasonal components over longer time periods such as monthly behaviour cannotbe made using this method as it is presented in this report.

Care must be taken when setting up the test so that the prerequisites areproperly met. But since the same consideration must be taken when performingthe analysis by hand, this should not be a particular problem for the method.The method is only a tool to ease the analysis and to reduce the amount of datato investigate closer.

When the counter data has been analysed, it seems to be reasonable in mostcases to assume that the noise terms are sufficiently mutually independent andnormally distributed, which justifies the hypothesis tests. When the changedetection is tested on data where the software is unchanged and on data withknown software updates, the results are as expected in most cases. Thus themethod seems to work under given conditions.

The problem statement of the thesis work was if any mathematical methodto detect changes in counter behaviour could be found. Since a method hasbeen found and analysed, and we have seen that it seems to work for the testcases used, the answer to the problem statement is yes: a mathematical methodfor change detection in counter data could be found, using time series analysisand statistical hypothesis testing.

Bibliography

[1] Gunnar Blom, Bjorn Holmquist, (1998), Statistikteori med tillampningar,B, Studentlitteratur, Lund.

[2] Gunnar Blom, Jan Enger, Gunnar Englund, Jan Grandell, Lars Holst,(2005), Sannolikhetsteori och statistikteori med tillampningar, Studentlit-teratur, Lund.

[3] Peter J. Brockwell, Richard A. Davis, (2003), Introduction to Time Seriesand Forecasting, Second Edition, Springer, New York.

[4] Peter J. Brockwell, Richard A. Davis, (2006), Time Series: Theory andMethods, Second Edition, Springer, New York.

[5] James Gallagher, The F test for Comparing Two Normal Variances: Cor-rect and Incorrect Calculation of the Two-Sided p-value?, Teaching Statis-tics, Vol. 28, No. 2, (2006).

[6] Hubert W. Lilliefors, On the Kolmogorov-Smirnov Test for Normality withMean and Variance Unknown, Journal of the American Statistical Associ-ation, Vol. 62, No. 318, (1967), pp. 399-402.

[7] Bernard W. Lindgren, (1993), Statistical Theory, Fourth Edition, Chapman& Hall/CRC, USA.

[8] Douglas C. Montgomery, (2013), Design and Analysis of Experiments,Eight Edition, Wiley, Singapore.

[9] John K. Taylor, Cheryl Cihon, (2004), Statistical Techniques for Data Anal-ysis, Second Edition, Chapman & Hall/CRC, USA.

[10] Wikipedia. Histogram. Retrieved 16 May, 2013, fromhttp://en.wikipedia.org/wiki/Histogram.

Eriksson, 2013. 67

68 Bibliography

Copyright

The publishers will keep this document online on the Internet - or its possi-ble replacement - for a period of 25 years from the date of publication barringexceptional circumstances. The online availability of the document implies apermanent permission for anyone to read, to download, to print out single copiesfor your own use and to use it unchanged for any non-commercial research andeducational purpose. Subsequent transfers of copyright cannot revoke this per-mission. All other uses of the document are conditional on the consent of thecopyright owner. The publisher has taken technical and administrative mea-sures to assure authenticity, security and accessibility. According to intellectualproperty law the author has the right to be mentioned when his/her work isaccessed as described above and to be protected against infringement. For ad-ditional information about the Linkoping University Electronic Press and itsprocedures for publication and for assurance of document integrity, please referto its www home page: http://www.ep.liu.se

Upphovsratt

Detta dokument halls tillgangligt pa Internet - eller dess framtida ersattare- under 25 ar fran publiceringsdatum under forutsattning att inga extraordi-nara omstandigheter uppstar. Tillgang till dokumentet innebar tillstand forvar och en att lasa, ladda ner, skriva ut enstaka kopior for enskilt bruk ochatt anvanda det oforandrat for ickekommersiell forskning och for undervisning.Overforing av upphovsratten vid en senare tidpunkt kan inte upphava dettatillstand. All annan anvandning av dokumentet kraver upphovsmannens med-givande. For att garantera aktheten, sakerheten och tillgangligheten finns detlosningar av teknisk och administrativ art. Upphovsmannens ideella ratt in-nefattar ratt att bli namnd som upphovsman i den omfattning som god sedkraver vid anvandning av dokumentet pa ovan beskrivna satt samt skydd motatt dokumentet andras eller presenteras i sadan form eller i sadant sammanhangsom ar krankande for upphovsmannens litterara eller konstnarliga anseende elleregenart. For ytterligare information om Linkoping University Electronic Pressse forlagets hemsida http://www.ep.liu.se

c© 2013, Tilda Eriksson

Eriksson, 2013. 69

master thesis change detection in telecommunication data ...633628/fulltext01.pdf · series...

Documents