graeme cosslett nzcer 2010 unpacking pat tests from the nzcer marking website
TRANSCRIPT
Graeme Cosslett
NZCER
2010
Unpacking PAT Tests from the NZCER Marking
Website
Building a greater understanding of the online reporting
Reporting at the:
• Individual Student level
• Class level
• Schoolwide level
Understanding:
Scale scores, Stanines, Box plots, Measurement error, Interpreting reports, Generating the reports
How are they doing?
Class of 2010
Test scores
Alex 22 Juliet 31
Allen 25 Kelly 18
Amy 40 Michael 36
Andrew 7 Nichole 30
Bean 6 Patrick 35
Becca 19 Paul 10
Bill 11 Peter 31
Catherine 7 Rose 18
Cathy 23 Sara 24
Christina 9 Simon 1
Daniel 18 Spencer 26
Dianna 41 Steve 14
Gary 24 Tessa 15
Janine 14 Tom 28
Jessie 14 Tony 16
Joy 6 Tracy 25
Grouped test scores
0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44
Dianna 41 Daniel 18
Amy 40 Kelly 18
Michael 36 Rose 18
Patrick 35 Tony 16
Juliet 31 Tessa 15
Peter 31 Janine 14
Nichole 30 Jessie 14
Tom 28 Steve 14
Spencer 26 Bill 11
Allen 25 Paul 10
Tracy 25 Christina 9
Gary 24 Andrew 7
Sara 24 Catherine 7
Cathy 23 Bean 6
Alex 22 Joy 6
Becca 19 Simon 1
Average 20.1
St.Dev 10.5
Median 18.5
Ordered test scores
How well are these students doing?
It depends!
Questions left unanswered
• What kind of knowledge and skill do students have at different achievement levels?
• How much better is a student doing who gets 35 compared to 25?
• How do scores on this test compare with scores on a harder or easier test?
• What score should I expect the students to get in a year’s time?
We can start with a measurement scale.
How do we answer these kinds of questions?
Measurement scales
• Allow you to convert a test score to a location on an equal interval scale.
• You can then see how one score compares with another, no matter which PAT test was used.
• You compare a student’s location on the scale with national norms for different year levels.
Measurement scales
• You can locate the difficulties of each test question on the same scale.
• We can then compare a student’s location on the scale with the location of different questions.
Measuring progress
More comprehension skills
Less comprehension skills
By Year 6, 50% of students have
reached this level
Questions involve longer passages containing abstract
ideas and more complex vocabulary
Create your own reading comprehension and mathematics scales that cater for achievement from Year 4 to 10. Describe the characteristics of tasks at different points on each scale.
Tasks that can be done using counting strategies.
How difficult?
• Put the tasks in order from least difficult to most difficult.
• Locate each task somewhere on your scale
• Which tasks are the most difficult? Why?• How would a typical Year 7 do on each of
the tasks?• On each of your scales locate where you
think the average Year 7 student would be
20 patm
34 patm
66 patm
75 patm
Year 7 Student
Year 10 Student
Comparing students and items
Students will typically answer correctly:
• 50% of the test items that are at the same location on the scale as they are
• more than 50% of the test items that are below their own location on the scale
• less than 50% of the test items that are above their own location on the scale.
Increasing skill as a reader
As students improve in comprehension they demonstrate increasing skill in:
– using abstract information– using separated information– using multiple pieces of information– using implied information– rejecting competing information– using vocabulary– using grammatical structures.
Vocabulary demands
Word List Dog & Pukeko (%) Square Eyes (%)
1 83.33 69.2
2 11.1 10.0
3 2.8 2.5
4 1.4 6.7
5 - 2.5
6 - -
7 - 1.7
8 - 0.8
other 1.4 6.66
Percentage of words in word list one to 10
The items in PAT Reading Comprehension according to text
Unpacking grammatical structures
Year 6
Year 7
Average progression in reading vocabulary for New Zealand students
Year 5
Year 4
Year 8
Year 9
Year 10
Year 6
Year 7
Average progression in mathematics for New Zealand students
Year 5
Year 4
Year 8
Year 9
Year 10
PAT:Mathematics Scale
Year 3
The PAT:Mathematics Scale
Stanines
Stanines
Stanines
Where on the scale?
Two students score 50% on PAT Mathematics tests.
Student one has sat Test 5 and student two has sat Test 6.
Test 6 is harder than Test 5.
Who will be located higher on the scale?
What Tom got right
1 202 223 234 245 267 288 299 32
10 3313 3414 3516 3617 3919 40
Tomsixsix
2855.9 ± 3.6
sevensix
Toms’ Maths Report
• Which national year group is Tom being compared with?
• How good is stanine 7?• How many Geometry and Measurement
questions did Tom get wrong?• Which was the easiest Geometry and
measurement question?• Was Tom expected to get question 30
correct?
SamYear 6March, 20082044.1 3.55
Year 6
Question types
• Retrieval
Comprehension without needing to infer
• Local inference
Comprehension of implied information from parts of the text
• Global inference
Comprehension of implied information from across the text as a whole
Local inference
• What does the poet compare herself with in the last verse? (Inferring the meaning of a simile from 1 line.)
• Why did Glen shrug? (Inferring a character’s intention from implied information across 6 sentences.)
• What was Adam imagining when he was playing with the water cannon? (Inferring a character’s thoughts from information in 4 paragraphs.)
Global inference
• What is the main purpose of this text?
• Which sentence would the author most likely agree with?
• What impression of the brain does this text create?
• What will probably happen next in the story?
PAT:Vocabulary items
90 patv
70 patv
50 patv
28patv
16 patv
Year 7 Student
Year 10 Student