© 1998, geoff kuenning introduction to statistics concentration on applied statistics statistics...

© 1998, Geoff Kuenning

Introduction to Statistics

• Concentration on applied statistics• Statistics appropriate for measurement• Today’s lecture will cover basic

concepts– You should already be familiar with

these


Independent Events

• Occurrence of one event doesn’t affect probability of other

• Examples:– Coin flips– Inputs from separate users– “Unrelated” traffic accidents

• What about second basketball free throw after the player misses the first?


Random Variable

• Variable that takes values with a specified probability

• Variable usually denoted by capital letters, particular values by lowercase

• Examples:– Number shown on dice– Network delay

• What about disk seek time?


Cumulative Distribution Function (CDF)

• Maps a value a to probability that the outcome is less than or equal to a:

• Valid for discrete and continuous variables• Monotonically increasing• Easy to specify, calculate, measure

F a P x ax ( ) ( )


CDF Examples

• Coin flip (T = 1, H = 2):

• Exponential packet interarrival times:

0

0.5

1

0 1 2

0

0.5

1

0 1 2 3


Probability Density Function (pdf)

• Derivative of (continuous) CDF:

• Usable to find probability of a range:

f xdF x

dx( )

( )

P x x x F x F x

f x dxx

x

( ) ( ) ( )

( )

1 2 2 1

1

2


Examples of pdf

• Exponential interarrival times:

• Gaussian (normal) distribution:

01

0 1 2 3

0

1

0 1 2


Probability Mass Function (pmf)

• CDF not differentiable for discrete random variables

• pmf serves as replacement: f(xi) = pi where pi is the probability that x will take on the value xi

P x x x F x F x

pix x xi

( ) ( ) ( )1 2 2 1

1 2


Examples of pmf

• Coin flip:

• Typical CS grad class size:0

0.5

1

00.10.20.30.40.5

4 5 6 7 8 9 10 11


Expected Value (Mean)

• Mean

• Summation if discrete• Integration if continuous

E x p x xf x dxi ii

n

( ) ( )1


Variance & Standard Deviation

• Var(x) =

• Usually denoted 2• Square root is called standard deviation

E x p x

x f x dx

i ii

n

i

[( ) ] ( )

( ) ( )

2 2

1

2


Coefficient of Variation (C.O.V. or C.V.)

• Ratio of standard deviation to mean:

• Indicates how well mean represents the variable

C.V.


Covariance

• Given x, y with means x and y, their covariance is:

– typos on p.181 of book• High covariance implies y departs from

mean whenever x does

Cov( , ) [( )( )]

( ) ( ) ( )

x y E x y

E xy E x E y

xy x y

2


Covariance (cont’d)

• For independent variables,E(xy) = E(x)E(y)

so Cov(x,y) = 0• Reverse isn’t true: Cov(x,y) = 0 doesn’t

imply independence• If y = x, covariance reduces to variance


Correlation Coefficient

• Normalized covariance:

• Always lies between -1 and 1• Correlation of 1 x ~ y, -1

Correlation( , )x y xyxy

x y

2

xy

~1


Quantile

• x value at which CDF takes a value is called a-quantile or 100-percentile, denoted by x.

• If 90th-percentile score on GRE was 1500, then 90% of population got 1500 or less

P x x F x( ) ( )


Quantile Example

0

0.5

1

1.5

0 2

-quantile0.5-quantile


Median

• 50-percentile (0.5-quantile) of a random variable

• Alternative to mean• By definition, 50% of population is sub-

median, 50% super-median– Lots of bad (good) drivers– Lots of smart (stupid) people


Mode

• Most likely value, i.e., xi with highest probability pi, or x at which pdf/pmf is maximum

• Not necessarily defined (e.g., tie)• Some distributions are bi-modal (e.g.,

human height has one mode for males and one for females)


Examples of Mode

• Dice throws:

• Adult human weight:

0

0.1

0.2

2 3 4 5 6 7 8 9 10 11 12

Mode

Mode

Sub-mode


Normal (Gaussian) Distribution

• Most common distribution in data analysis

• pdf is:

• -x +• Mean is , standard deviation

f x ex

( )( )

1

2

2

22


Notationfor Gaussian Distributions

• Often denoted N(,)• Unit normal is N(0,1)• If x has N(,), has N(0,1)

• The -quantile of unit normal z ~ N(0,1) is denoted zso that

x

Px

z P x z( ) ( )


Why Is GaussianSo Popular?

• If xi ~ N(,) and all xi independent, then ixi is normal with mean ii and variance i

2i2

• Sum of large no. of independent observations from any distribution is itself normal (Central Limit Theorem) Experimental errors can be modeled

as normal distribution.


Central Limit Theorem

• Sum of 2 coin flips (H=1, T=0):

• Sum of 8 coin flips:

0

0.5

1

0 1 2

0

0.1

0.2

0.3

0 1 2 3 4 5 6 7 8


Measured Data

But, we don’t know F(x) – all we have is a bunch of observed values – a sample.

What is a sample?– Example: How tall is a human?

• Could measure every person in the world (actually even that’s a sample)

• Or could measure every person in this room

– Population has parameters

– Sample has statistics• Drawn from population

• Inherently erroneous

Measured Data


Central Tendency

• Sample mean – x (arithmetic mean)– Take sum of all observations and divide by the

number of observations• Sample median

– Sort the observations in increasing order and take the observation in the middle of the series

• Sample mode– Plot a histogram of the observations and choose

the midpoint of the bucket where the histogram peaks


Indices of Dispersion• Measures of how much a data set varies

– Range– Sample variance

– And derived from sample variance: • Square root -- standard deviation, S• Ratio of sample mean and standard deviation – COV

s / x– Percentiles

• Specification of how observations fall into buckets

sn

x xii

n2 2

1

1

1


Interquartile Range

• Yet another measure of dispersion• The difference between Q3 and Q1• Semi-interquartile range -

• Often interesting measure of what’s going on in the middle of the range

SIQRQ Q

3 1

2


Which Index of Dispersion to Use?

Bounded?

Unimodalsymmetrical?

Range

C.O.V

Percentiles or SIQR

But always remember what you’re looking for

Yes

Yes

No

No


• If a data set has a common distribution, that’s the best way to summarize it– Saying a data set is uniformly distributed is more

informative than just giving its sample mean and standard deviation

• So how do you determine if your data set fits a distribution?– Plot a histogram– Quantile-quantile plot– Statistical methods

Determining a Distribution for a Data Set


Quantile-Quantile Plots

• Most suitable for small data sets• Basically -- guess a distribution• Plot where quantiles of data should fall

in that distribution– Against where they actually fall in the

sample• If plot is close to linear, data closely

matches that distribution


ObtainingTheoretical Quantiles

• We need to determine where the quantiles should fall for a particular distribution

• Requires inverting the CDF for that distribution

qi = F(xi) xi = F-1(qi)

– Then determining quantiles for observed points

– Then plugging in quantiles to inverted CDF


Inverting a Distribution

• Many common distributions have already been inverted (how convenient…)

• For others that are hard to invert, tables and approximations are often available (nearly as convenient)


Is Our Example Data Set Normally Distributed?

• Our example data set was-17, -10, -4.8, 2, 5.4, 27, 84.3, 92, 445, 2056• Does this match the normal distribution?• The normal distribution doesn’t invert nicely

– But there is an approximation for N(0,1):

– Or invert numerically

x q qi i i 4 91 10 14 0 14. . .


Data For Example Normal Quantile-Quantile Plot

i qi yi xi1 0.05 -17 -1.646842 0.15 -10 -1.034813 0.25 -4.8 -0.672344 0.35 2 -0.383755 0.45 5.4 -0.12516 0.55 27 0.12517 0.65 84.3 0.3837538 0.75 92 0.6723459 0.85 445 1.03481210 0.95 2056 1.646839


Example NormalQuantile-Quantile Plot

-500

0

500

1000

1500

2000

2500

-1.65 -1.03 -0.67 -0.38 -0.13 0.13 0.38 0.67 1.03 1.65

Definitely not normal– Because it isn’t linear– Tail at high end is too long for normal


Estimating Populationfrom Samples

• How tall is a human?– Measure everybody in this room– Calculate sample mean – Assume population mean equals

• What is the error in our estimate?

xx


Estimating Error

• Sample mean is a random variable Mean has some distributionMultiple sample means have “mean

of means”• Knowing distribution of means can

estimate error

Confidence Intervals

• Sample mean value is only an estimate of the true population mean

• Bounds c1 and c2 such that there is a high probability, 1-, that the population mean is in the interval (c1,c2):

Prob{ c1 < < c2} =1- where is the significance level and100(1-) is the confidence level

• Overlapping confidence intervals is interpreted as “not statistically different”


Confidence Intervals

• How tall is Fred?– Suppose average human height is 170 cmFred is 170 cm tall

Yeah, right – Suppose 90% of humans are between 155

and 190 cmFred is between 155 and 190 cm

• We are 90% confident that Fred is between 155 and 190 cm


Confidence Intervalof Sample Mean

• Knowing where 90% of sample means fall, we can state a 90% confidence interval

• Key is Central Limit Theorem:– Sample means are normally distributed– Only if independent– Mean of sample means is

population mean – Standard deviation (standard error) is

n


EstimatingConfidence Intervals

• Two formulas for confidence intervals– Over 30 samples from any

distribution: z-distribution– Small sample from normally

distributed population: t-distribution• Common error: using t-distribution for

non-normal population– Central Limit Theorem often saves us


The z Distribution

• Interval on either side of mean:

• Significance level is small for large confidence levels

• Tables of z are tricky: be careful!

x zs

n

1 2


Example of z Distribution

• 35 samples: 10 16 47 48 74 30 81 42 57 67 7 13 56 44 54 17 60 32 45 28 33 60 36 59 73 46 10 40 35 65 34 25 18 48 63

• Sample mean = 42.1. Standard deviation s = 20.1. n = 35

• 90% confidence interval is

x

42 1 1 64520 1

3536 5 47 7. ( . )

.( . , . )


The t Distribution

• Formula is almost the same:

• Usable only for normally distributed populations!

• But works with small samples

x ts

nn

1 2 1 ;


Example of t Distribution

• 10 height samples: 148 166 170 191 187 114 168 180 177 204

• Sample mean = 170.5. Standard deviation s = 25.1, n = 10

• 90% confidence interval is

• 99% interval is (144.7, 196.3)

x

170 5 1833251

10156 0 185 0. ( . )

.( . , . )


Getting More Confidence

• Asking for a higher confidence level widens the confidence interval– Counterintuitive?

• How tall is Fred?– 90% sure he’s between 155 and 190 cm– We want to be 99% sure we’re right– So we need more room: 99% sure he’s

between 145 and 200 cm

For Discussion Next TuesdayProject Proposal1. Statement of hypothesis2. Workload decisions3. Metrics to be used4. Method

Reading for Next TimeElson, Girod, Estrin, “Fine-grained Network Time

Synchronization using Reference Broadcasts, OSDI 2002 – see readings.html

For Discussion Today

• Bring in one either notoriously bad or exceptionally good example of data presentation from your proceedings. The bad ones are more fun. Or if you find something just really different, please show it.

© 1998, geoff kuenning introduction to statistics concentration on applied statistics statistics...

Documents