curriculum-embedded performance assessments as a measure of student learning and growth
DESCRIPTION
Curriculum-Embedded Performance Assessments as a Measure of Student Learning and Growth. Ruth Chung Wei Stanford University June 2014. Stanford Center for Assessment, Learning, & Equity. Mathematics Design Collaborative. Mathematics Design Collaborative. - PowerPoint PPT PresentationTRANSCRIPT
Curriculum-Embedded Performance Assessments as a
Measure of Student Learning and
GrowthRUTH CHUNG WEISTANFORD UNIVERSITY
JUNE 2014
Stanford Center for Assessment, Learning, & Equity
Mathematics Design
Collaborative
Mathematics Design CollaborativeFormative Assessment Lessons
RESEARCH QUESTIONS1. What is the relationship between
performance on curriculum embedded performance tasks to other on-demand measures of student learning (e.g, performance tasks, standardized large-scale assessments)?
2. What is the feasibility of applying growth models to the performance task scores to assess growth in student performance?
Stanford Center for Assessment, Learning, & Equity
DATA SOURCES – LDC STUDYLDC
TasksOn
Demand
Pre-test
On Deman
d Post-test
District Admin Data
State A(7 districts) 30 teachers(ELA, History, Science)
1 Task Spring 2013
CRESST writing
task
CRESST writing
task
State tests,SAT/ACT,
Demographics,
Attendance,District-level
data,Links to School,
Class, andTeacher
State B(1 district)30 teachers(ELA, History, Science)
1 TaskSpring 2013
CRESST writing
task
CRESST writing
task
DATA SOURCES – MDC STUDYMDC Study•1 large urban school district
(about 30 Algebra teachers within one school network)
• 8 unit performance tasks• 1 summative performance-based
assessment • District administrative data
(concurrent and prior achievement, demographics, attendance)
Stanford Center for Assessment, Learning, & Equity
Preliminary Results LDC Performance Tasks• Can they be scored
reliably?• Are they comparable?
Stanford Center for Assessment, Learning, & Equity
Generalizability Study LDC Scoring Rubrics
Number of RatersDIMENSIONS 1 2 3 4Focus 0.92 0.96 0.97 0.98Controlling Idea 0.86 0.92 0.95 0.96Reading Research 0.83 0.91 0.94 0.95Development 0.89 0.94 0.96 0.97Organization 0.83 0.90 0.93 0.95Conventions 0.69 0.82 0.87 0.90Content Understanding 0.87 0.93 0.95 0.97Total Score 0.91 0.96 0.97 0.98
Estimated Reliability of the ELA Explanatory Task as a Function of Number of Raters Used to Calculate Scores
DISENTANGLING SOURCES OF ERROR VARIANCE
Source Focus
Control-ling Idea
Reading
Research
Dev’t Org.Conve
n-tions
Content
Und.
student
37.85 (91%)
37.52 (86%)
32.28 (83%)
37.7 (88%)
32.92 (81%)
23.66 (60%)
30.48 (86%)
rater 0.39 (1%)
0 (0%)
0 (0%)
0.2 (0%)
0.99 (2%)
5.15 (13%)
0.59 (2%)
Error (s x r)
3.36 (8%)
6.29 (14%)
6.58 (17%)
4.8 (11%)
6.93 (17%)
10.68 (27%)
4.41 (12%)
Source Table for the ELA Explanatory Task
Stanford Center for Assessment, Learning, & Equity
Work in Progress Good to Go Exemplary
Task Clarity &
Coherence• –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
• –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
• –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
Content • –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
• –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
• –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
Text(s) • –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
• –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
• –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
Student Product
• –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
• –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
• –––––––––––––––––––––• ––––––––––––––• –––––––––––––––––––––––
LDC TASK JURYING RUBRIC
COMPARABILITY OF LDC TASKS?
4 5 6 7 8 9 10 1102468
1012
2
810
6 6
2 1
Distribution of Total Scores on LDC Task Jurying Rubric
Total Score
Freq
uenc
y
COMPARABILITY OF LDC TASKS?
Work In Progress
Good to Go Exemplary0
5
10
15
20
25 21
12
2
Distribution of Holistic Ratings on LDC Task Jurying Rubric
Holistic Score
Freq
uenc
y
RELATIONSHIP BETWEEN TASK QUALITY SCORE AND STUDENT PERFORMANCE – ELA TASKS Total Task
Quality Score
Holistic Task Rating
Total LDC Essay Score
Pearson Correlation .135* .367**
Sig. (2-tailed) .022 .000
N 290 290Stanford Center for Assessment, Learning, & Equity
DIFFERENCES IN MEAN LDC ESSAY SCORES BY HOLISTIC TASK QUALITY RATING – ELA TASKSHolistic
Task Quality Rating N
Mean LDC
Essay Score
Std. Dev.
Std. Error Min Max
Work in Progress 159 12.9 5.062 0.401 7 28Good to Go 131 17.31 6.135 0.536 7 28Exemplary 0Total 290 14.89 5.979 0.351 7 28Difference in Mean LDC Essay Score between groups is significant at the 0.000 level.
CORRELATION BETWEEN LDC ESSAY SCORES AND CRESST ON-DEMAND TESTS – ELA TASKS
Pre Test
Essay
Pre Test Total
Post Test
Essay
Post Test Total
Total LDC
Essay Score
Total LDC Essay Score
Pearson Correlation
.191** .170** .278** .199**
Sig. (2-tailed) .001 .004 .000 .001N 278 287 278 282
Post Test Essay
Pearson Correlation
.692** .796** .848** .199**
Sig. (2-tailed) .000 .000 .000 .001N 335 348 343 282
ANALYSES TO BE COMPLETED
• Correlational studies of other on-demand assessments with curriculum-embedded assessments
• Growth modeling on longitudinal student achievement data
• Introducing performance assessment data as part of the growth model -- either as a predicted outcome or as a predictor