The Efficient Exam

Shlomo Yitzhaki

Hebrew University

Talk’s Structure

• Characterization of Grades in Exams

• Monotonic correlation

• Properties of Gini’s Mean Difference

• Properties of the Efficient Exam

Characterization of grades

• Grades are an Ordinal Variable• It is as if we are measuring height of

people standing behind a screen• We ask who has the X centimeter and

those that are taller than X respond positively.

• Height is the number of positive answers.• It is impossible to plot a cumulative

distribution of grades

Characterization of Grades

• If cumulative distributions of two groups intersect, then there are two alternative legitimate exams that will result in contradicting ranking of average grades.

• Hence, one can improve her country performance in international exams like PISA, by pointing out the alternative exam.

Monotonic Correlation

• It is assumed that we are examining a uni-dimensional ability

• Otherwise we have to examine whether the correlation is monotonic.

• The method to do that is based on plotting Concentration curves (A variant of Lorenz curve for two variables).

• The Method is already published (Economics Letters, 2012).

Properties of GMD

• Gini’s Mean Difference can be decomposed in a way that makes the decomposition of the variance a special case.

• This way one can find the implicit assumptions behind the variance.

• Properties described in a 540 pages book• Entitled “The Gini Methodology” by

Springer Statistics N. Y. 2013.

GMD vs. Variance

• Variance = cov(X, X)

• GMD = 4 cov(X, F(X))

• Note that F is uniform [0, 1].

• Gini covarince cov(X, F(Y)), cov(Y,F(X))

• They don’t have to have the same sign.

• Known in economics as “Index number problem.

Properties of GMD

• ANOVA Translates into ANOGI• Two correlation coefficients between two

variables two Gini Covariance, two regression coefficients, mixed GMD-OLS regression, etc..

• If the two correlations between two variables are equal, then we get an identical decomposition of the variance of a sum of random variabes

Properties of the Efficient exam

• Because of the limited number of questions, There is “Binning”

• Main proposition: To maximize between-group variability, the distribution of grades in the “efficient exam” should be Uniform.

• No proof is presented in this talk.

A sketch of the proof

• The proof is based on the proposition that the distribution of the cumultive distribution is uniform [0, 1].

• Using Lorenz curve then the question is what is the optimal size of a “bin”

• Two stages: Every “bin” should be positive. Mid-point is optimal

Transvariation

• Two possible ways to rank groups:• According to average grade• According to transvariation: The probability

of a randomly selected member of the high (low) average group to be better than the randomly selected member of the lower (higher) average group.

• Under efficient exam both criteria are equivalent.

Applications

• The arguments are relevant to any test based on ordinal variable.

• I owe this point to Emil

• This is the reason why I was invited

Thank you

• For your Patience