measuring and interpreting test results for teaching core standard expectations

Measuring and InterpretingTest Results for Teaching Core Standard Expectations

From a very recent research report

Knowledge of mathematics is crucial to educational and financial success in contemporary society and is becoming ever more so.

High school students’ mathematics achievement predicts college matriculation and graduation,

early-career earnings, and earnings growth

Elementary school students’ knowledge of fractions and division uniquely predicts their high school mathematics achievement,

even after controlling for a wide range of relevant variables

suggesting that efforts to improve mathematics education should focus on improving students’ learning in those areas.

Robert S. Siegler, Greg J. Duncan, Pamela E. Davis-Kean, Kathryn Duckworth, Amy Claessens, Mimi Engel, Maria Ines Susperreguy, and Meichu Chen, Early Predictors of High School Mathematics Achievement, Psychological Science, June 14, 2012.

Close examination of test questions, and student responses including to the distracters, can tell us a tremendous amount about these issues.

This is what I would like to discuss today.

Close examination of test questions, and student responses including to the distracters, can tell us a tremendous amount about these issues.

This is what I would like to discuss today.

Grade 3

Grade 3Percent choosing each answer

Percent Correct A % B % C % D %

53 11 30 53 5

44 38 44 8 10

Comments

It is literally impossible for this to happen if students actually do understand the area model.

So the 91% correct response to the first means something entirely different.

The First Problem What of (B)

(30%)? It seems that

these children counted the number of WHITE regions

A B C D

11% 30% 53% 5%

The Second Problem What of (A) (38%)? It seems that these

children also counted the number of WHITE regions

In both situations, it is clear that they had very little understanding that the model worked with EQUAL AREAS

A B C D

38% 44% 8% 10%

And look at the results for division of fractions.

Grade 6Percent choosing each answer

Percent Correct A % B % C % D %

37 9 24 37 31

37 8 37 11 44

The First Problem Note the number of

students who simply multiplied (B).

9% Added top and bottom

Too many checked (D), so we could not measure other errors.

These are issues with the teaching of the subject.

A B C D

9% 24% 37% 31%

The Second Problem

11%, about the same as before added top and bottom.

Too many checked (D), so we could not measure other errors, including the dominant “multiplication”

These again are issues with the teaching of the subject.

A B C D

8% 37% 11% 44%

Grade 6Percent choosing each answer

Percent Correct A % B % C % D %

45 27 22 6 45

34 7 36 34 24

The First Problem The correct answer is

(D), which is unfortunate since it seems that a common strategy here is to check (D) when the students don’t know.

so we could not measure errors, though they seem to all be in placing the decimal – a problem with understanding magnitude.

A B C D

27% 22% 6% 45%

The Second Problem

Note that more answered (B) than the correct answer (C)

(A) and (B) both show a lack of understanding of magnitude.

Too many checked (D), so we could not measure other errors.

These again are issues with the teaching of the subject.

A B C D

7% 36% 34% 24%

And look at the results for fraction addition.

Grade 5

The First Problem

The correct answer is (D) which a simple size estimate shows which is somewhat unfortunate.

(A) could have just been a simple arithmetic error, but again ¾ + 2/7 > ¾ + ¼ = 1, so it should not have happened.

But what of (B)?

A B C D

18% 37% 9% 37%

The Second Problem

Note that more answered (A) than the correct answer (C)

(A) shows the relatively surprising error of adding top and bottom separately on the frational parts of the mixed number

These again are issues with the teaching of the subject.

A B C D

42% 4% 34% 20%

We have huge reasons to make sure students learn fractions and division completely and carefully.

But, for mathematical reasons, student difficulty with fractions

and division rest on earlier problems with place value

But these difficulties actually start with PLACE VALUE as it is taught in the earliest grades.

Grade 1Percent choosing each answer

Percent Correc

t A % B % C % D %

40 40 16 44 0

As one can see, the percent correct is not always a good measure of what is going on. But the distracters tell us quite a bit.

Grade 2

Percent Correct A % B % C % D %

45 49 45 4 2

The distracters here give very important, troubling but consistent information.

This is an inevitable consequence of lessons like the following

Typical U.SLesson on Place Value

Note Focus onManipulativesLinear modelFor 10’s, area for100’s, volume for1000’s. This isIllogical and confusingWhat represents 10,000?

This is codified in the U.S. curriculum to the extent that if a third grade text does not have this lesson it will typically be rejected as being mathematically insufficient.

By contrast here is how this topic is handled in the high achieving countries

First Grade RussianText: Place Value

First Grade: RussianText: Models forPlace Value.

Especially note use ofDecimeters for putting(2 place) Place valueon number line

Second Grade: RussianText. Note consistencyOf models for higherPlaces and tight focus

E

Even 1000’s areConsistent

Consistent models makeComparison easier.Note attention toComparisons

“Bundles” In Core Standards

This should warn us that there is much, much more going on in Core Standards than one might think. I believe care is necessary in choosing our Core Standard “experts.”