the transition from mpg to map
DESCRIPTION
The Transition from MPG to MAP Deborah Adkins, Research Specialist, Leslie Yudman, ELA Content Specialist, NWEA Fusion 2012, the NWEA summer conference in Portland, Oregon This session addresses transitioning students from MPG to MAP. It examines the point at which most partners transition students and the nature of the effect on scores. Findings from a counterbalanced research study will be presented. The study was conducted with second graders who took MPG as first graders. These students were administered items from the 2-5 reading pool both with and without audio assistance. The session will conclude with proposed next steps. Learning Outcome: - Learn when to transition children from MPG to MAPTRANSCRIPT
Transitioning Students from MPG to MAP
© 2011 Northwest Evaluation Association. All Rights Reserved. 2
• This session addresses transitioning students from MPG to MAP.
• It examines the point at which most partners transition students and the nature of the effect on scores.
• Findings from a research study will be presented.
• Next steps.
What will be covered?
© 2011 Northwest Evaluation Association. All Rights Reserved. 3
Most of our partners transition their students from MPG to MAP
during the fall of 2nd grade for both reading and mathematics.
When do most partners transition students?
© 2011 Northwest Evaluation Association. All Rights Reserved. 4
What is the impact ?
Total# lower in fall (RIT)
%lower in fall (RIT)
Mean RIT diff
MPG to MAP SP10 to FA10 (1st-2nd) 113,234 69,662 62% 10
MPG to MPG SP10 to FA10 (1st-2nd) 46,357 22,029 48% 6
MPG to MAP SP10 to FA10 (2nd -3rd) 49,940 26,685 53% 9
MAP to MAP SP10 to FA10 (2nd -3rd) 286,956 130,514 45% 7
For reading an additional 9% of students experience a drop in scores when transitioning from MPG to MAP at
the beginning of 2nd grade vs. the beginning of 3rd grade
© 2011 Northwest Evaluation Association. All Rights Reserved. 5
What is the impact ?
Total # lower in fall (RIT)
%lower in fall (RIT)
Mean RIT diff
MPG to MAP SP10 to FA10 (1st-2nd) 125,238 76,318 61% 8
MPG to MPG SP10 to FA10 (1st-2nd) 45,988 20,985 47% 5
MPG to MAP SP10 to FA10 (2nd -3rd) 47,432 28,515 60% 7
MAP to MAP SP10 to FA10 (2nd -3rd) 282,306 130,647 46% 6
For mathematics the point of transition made little difference
© 2011 Northwest Evaluation Association. All Rights Reserved. 6
There is something about the 2 forms of the tests that is different.• Item type• Audio support for MPG• Content
What does this mean?
© 2011 Northwest Evaluation Association. All Rights Reserved. 7
• The study was conducted with second graders who took MPG as first graders.
• These students were administered items from the 2-5 reading pool both with and without audio assistance.
• Items in the field test were re-calibrated under both the non-audio & audio conditions and evaluated
• Field study tests were rescored
Field Study (Fall 2011)
© 2011 Northwest Evaluation Association. All Rights Reserved. 8
Results of Fall 2011 Study
Studentsthe sample was similar to the population
MPG (spring grade 1)to MAP 2-5 (fall grade 2) Total# lower in
fall%lower in fall
Mean RIT diff
2010 - population 113,234 69,662 62% 10
2011 - study sample 1,161 814 70% 12
© 2011 Northwest Evaluation Association. All Rights Reserved. 9
Spring Grade 1 to Fall Grade 2Transition
FA11 Grade2Q1 (142-165)N = 177
SP11 Grade1Q1 (145-166)N = 215
FA11Q1 N = 6
FA11Q2 N = 30
FA11 Grade2<142N = 2
SP11 Grade1<145N = 8
FA11 Grade2Q2 (166-175)N = 108
SP11 Grade1Q2 (167-176)N = 318
FA11Q3 N = 5
FA11Q4 N = 3
FA11Q3 N = 36
FA11Q4 N = 8
FA11<142 N = 2
FA11Q1 N = 164
FA11 Grade2Q3 (176-186)N = 94
SP11 Grade1Q3 (177-186)N = 323
FA11Q4 N = 50
FA11Q1 N = 94
FA11Q2 N = 85
FA11 Grade2Q4 (187-215)N = 173
SP11 Grade1Q4 (187-214)N = 290
FA11>215 N = 6
FA11Q2 N = 30
FA11Q3 N = 64
FA11Q1 N = 17
FA11 Grade2>215N = 0
SP11 Grade1>214N = 7
FA11Q4 N = 7
© 2011 Northwest Evaluation Association. All Rights Reserved. 10
ItemsCalibrations varied for 55-90% of the items across all goals
Results of Fall 2011 Study
+20 to 24 +15 to 19 +10 to 14 +5 to 9 +0 to 4 -1 to -4 -5 to -9 -10
Number of Items 5 8 17 11 12 4 3 1
Baseline RIT Range 160-187 158-182 152-213 162-216 155-213 183-207 177-211 209
GoalTotal Number
of Items
Easier with Audio by 5+ RIT
More Difficult with Audio by 5+ RIT
Essentially the Same - within 4 RIT
PercentageRIT Range
Dif PercentageRIT Range
Dif PercentageInformational Comprehension 20 55.0% 5 to 17 0.0% NA 45.0%
Literary Comprehension 20 75.0% 6 to 24 15.0% 5 to 10 10.0%
Word Meaning & Analysis 21 71.4% 6 to 22 4.8% 7 23.8%
© 2011 Northwest Evaluation Association. All Rights Reserved. 11
Items (cont.)
Results of Fall 2011 Study
© 2011 Northwest Evaluation Association. All Rights Reserved. 12
Calibrations by Goal - Informational Comprehension
Results of Fall 2011 Study
© 2011 Northwest Evaluation Association. All Rights Reserved. 13
Calibrations by Goal - Literary Comprehension
Results of Fall 2011 Study
1916
© 2011 Northwest Evaluation Association. All Rights Reserved. 14
Calibrations by Goal - Word Meaning & Analysis
Results of Fall 2011 Study
9
© 2011 Northwest Evaluation Association. All Rights Reserved. 15
Field tests raw scoresThe majority of students score higher on audio across both forms of the test (72.1% form A & 78.5% form B) with only 9.4% & 7.2% yielding the same score across forms respectively.
Results of Fall 2011 Study
© 2011 Northwest Evaluation Association. All Rights Reserved. 16
Field tests re-scored (scaled score)
Results of Fall 2011 Study
Spring 2011 Grade 1(MPG Reading)Fall 2011 Grade 2 (MAP Reading 2-5)
MAP 2-5 Audio FT NonAudio FT
Quartile RIT Range N RIT Range N N N
<1 <145 8 <142 4 0 0
1 145-166 215 142-165 458 30 179
2 167-176 318 166-175 253 124 301
3 177-186 323 176-186 199 383 366
4 187-214 290 187-215 241 621 315
>4 >214 7 >215 6 3 0
© 2011 Northwest Evaluation Association. All Rights Reserved. 17
Growth Target Spring to Spring
Growth Targets
Spring to Spring Growth Targets
QuartileOverall Drop in Score from SP to FA No Drop in Score SP to FA
N Met Didn't Meet % Met N Met Didn't Meet % Met N Met Didn't Meet % Met
<1 8 5 3 62.5% 1 0 1 0.0% 7 5 2 71.4%
1 208 94 114 45.2% 138 46 92 33.3% 70 48 22 68.6%
2 313 159 154 50.8% 232 97 135 41.8% 81 62 19 76.5%
3 314 160 154 51.0% 231 95 136 41.1% 83 65 18 78.3%
4 281 127 154 45.2% 182 53 129 29.1% 99 74 25 74.7%
>4 7 2 5 28.6% 7 2 5 28.6% 0 0 0 0.0%
Total 1131 547 584 48.4% 791 293 498 37.0% 340 254 86 74.7%
© 2011 Northwest Evaluation Association. All Rights Reserved. 18
What does this mean?
• It is necessary to consider the purpose of the test– MPG Reading: Assesses pre-and emergent readers
in grades K-2– MAP 2-5 Reading: Assesses students in grades 2-5
who can read
• Tests administered to students for their intended purpose provide information on the construct of interest (e.g. performance on informational comprehension, etc.). When tests are administered otherwise, the likelihood of introducing construct irrelevance increases.
© 2011 Northwest Evaluation Association. All Rights Reserved. 19
The strategy largely depends on what you plan to do/decisions that will be made based on the data in conjunction with what works best
for your students
Transition Strategies
© 2011 Northwest Evaluation Association. All Rights Reserved. 20
Reading• When the student can read connected text
sufficiently to access the items• When the student has been exposed to grade
level content• When the student scores at the upper end of
the scale for 2 consecutive administrations
Transition Strategies
© 2011 Northwest Evaluation Association. All Rights Reserved. 21
Mathematics• When the student has been exposed to grade
level content • When the student scores at the upper end of
the scale for 2 consecutive administrations
Transition Strategies
© 2011 Northwest Evaluation Association. All Rights Reserved. 22
• Concentrating on item development at the ends of the scale
• Updated selection strategy for MPG Fall 2012
• Releasing Common Core aligned MPG for Fall 2012
• Additional studies
– Fall 2012 –MPG goal scores predict MAP scores
– Fall 2013 – Drop when transitioning from MPG Common Core to MAP Common Core
– Fall 2013/Winter 2014 - MPG Common Core goal scores predict MAP Common Core scores
• Research investigating use of a transition instrument
– To identify which test is more appropriate to students
Next Steps
© 2011 Northwest Evaluation Association. All Rights Reserved. 23
Questions?
Contact information:
Transition Study : [email protected]
Product: [email protected]
Thank You