Understanding and Interpreting Statistics in Assessments

Clare Trott and Hilary Maddocks

This Session

• Why use statistics in assessments?• “Averages”, Standard Deviation, variance,

Standard Error• Normal distribution, confidence intervals • Scales• Overlapping confidence intervals

• Why use statistics in assessments?

• What are the assumptions made?

Feedback

AVERAGE

MEAN

MEDIAN

MODE

Which is better? When?

What is • Standard Deviation? • Variance? • Standard Error?

Central Tendency

Spread

Feedback

MODE

MOSTOFTENDE MEDIAN

MED IAN

Make

Everyone

Add

Numbers

(and)Share

Means

Standard Deviation

2, 3, 6, 9, 10Mean = 6, SD = 3.16

2, 2, 6, 10, 10Mean = 6, SD = 3.58

• Measures the average amount by which all the data values deviate from the mean

• Measured in the same units as the data

Variance and Standard Deviation

Σ (𝑥−𝑥 )2

𝑛Variance =

Mean ()

Standard deviation = σ =𝜎 2=¿

Standard Error

This is the variance per person

𝑆𝐸= 𝜎2

𝑛

Normal Distribution

• What is Normal Distribution?

• Why is it useful?

Confidence Intervals

• What are Confidence Intervals?

• Why are they important?

Feedback

Number of standard deviations from the mean

Normal Distribution

Confidence Intervals

The wider the range the more confident we can be that the true score lies in this range

TRUE SCORE

Due to inherent error in measurement it is better to quote a 95% confidence interval

99% Confidence

Interval

95% Confidence

Interval

C I

Confidence Intervals

-1.645 1.645

1.96-1.96

2.575-2.575

90% Confidence Interval

95% Confidence Interval

99% Confidence Interval

True Score

• True score lies inside CI 95% of occasions

• 1 in 20 (5%) will not include the true score

95% Confidence Intervals

Scales

• What scales are used in reporting?

• How are they defined?

• Why are standardised scores preferred?

Feedback

Very low lowLow

averageaverage

High average

high Very high

100 130110 120908070

50 9875 902510 2

10 1612 14864

Scaled scores

Standardised scores

Percentiles

standardised

percentile

scaled

130 and above

98th >16 + 3SD Within top 2%

Very high

120-129 91-97 14-15 + 2SD Above 91%

high

110-119 75-90 12-13 + 1SD Above 75%

High average

90-109 25-74 8-11 Mean Above 25%

average

80-89 10-24 6-7 -1SD Above 16%

Low average

70-79 2-9 4-5 -2SD Above 10%

Below average

Below 70 Below 2 < 4 -3SD Lowest 2%

Very low

Simplified Table

Scale to Standardised

• 1 to 5 ratio• 10 scaled 100 standardised• 9 scaled 95 standardised• 11 scaled 105 standardised

• 15 scaled 125 standardised• 6 scaled 80 standardised

100 130110 120908070

Very low lowLow

averageaverage

High average

high Very high

Standardised scores against standard deviations

-1sd

-2sd

-3sd

1sd

2sd

3sd

mean

50 9875 902510 2

Very low low averageHigh

averagehigh

Very high

Low average

Percentiles against standard deviations

-1sd

-3sd

-2sd

3sd

2sd

1sd

mean

10 1612 14864

Very low low averageHigh

averagehigh

Very high

Low average

Scaled scores against standard deviations

mean

3sd

-2sd

-3sd

1sd-1sd

2sd

Differences in Class Intervals

Suppose we have the class intervals for two tests which could be linked, and we wish to find whether there is a significant difference between the two sets.

Test 195% Confidence Interval 102 ± 15.8, standard error 2.96

Test 295% Confidence Interval 118 ± 23, standard error 6.63

86.2 102 117.8 105 118 131

There appears to be no significant difference as there is a distinct overlap.

H0 : There is no significant difference in the two Confidence Intervals

(the new confidence interval contains zero)

H1 : There is a significant difference in the two Confidence Intervals

(the new CI does not contain zero

=(118 – 102)

Formula ±1.96√𝑆𝐸12+𝑆𝐸2

2Difference in scores

±1.96√2.962+6.632

= 16 ± 14

New CI 2 16 30

This does not contain zero so we reject H0 and so there is a significant difference in the two tests.

Test 195% Confidence Interval 95 ± 6, standard error 3.06

Test 295% Confidence Interval 106 ± 10, standard error 5.102

88 95 102

96 106 116

There appears to be no significant difference as there is a distinct overlap.

H0 : There is no significant difference in the two Confidence Intervals

(the new confidence interval contains zero)

H1 : There is a significant difference in the two Confidence Intervals

(the new CI does not contain zero

=(106 – 95)

±1.96√𝑆𝐸12+𝑆𝐸2

2Difference in scores

±1.96√3.062+5.1022

= 11 ± 11.6

New CI -0.6 11 22.6

This does contain zero so we accept H0 and so there is no significant difference in the two tests.