ncme april 30 2013 saad chahine
DESCRIPTION
The study presented in this paper is written from a report maker’s perspective, primarily asking: How can the design of score reports support the utility of achievement data by personnel in the school system? Thus, the focus is on understanding how score reports can be designed optimally to enhance their use by the educational community. Before attempting to address this issue, we as a community need to understand how educators interpret reports. This paper is an initial step in this process of enhancing score reports by investigating the thought process of educators. It proposes a conceptual model, “Data Habit of Mind,” to study educators’ understanding, interpretation and potential applications of results from large-scale assessments.TRANSCRIPT
Investigating Educators’ Statistical Literacy and Score Report Interpretation
Saad Chahine, PhD April 30, 2013 NCME | San Francisco
13
-04
-30
Direction
• Need for research on how educators read and interpret score reports
• Current research focuses on the culture of data use in schools
• This research study asks the question:
How can the design score reports support the utility of achievement data by personnel in the school system?
Research Questions
1. How are educators proficient in statistical literacy?
2. How do educators interpret and propose to use test score reports?
ππ
33
Statistical Literacy
Score Report Interpretation
Conceptual Framework
“habit of mind” as a metaphor
Methods
• Cognitive interviews with 20 educators
• Statistical literacy tasks
• Two mock score reports
• Six stage protocol analysis
ππ
77
ππ
88
Example Statistical Literacy Task
Coopers Score Report ππ
99
Jabberwocky Score Report ππ
1010
13-04-30 Mount Saint Vincent University 11
Skill Minimum Maximum Mean Std. D.
Statistical Literacy
Describing 3.00 4.00 3.77 0.24
Organizing and Reducing 2.00 4.00 2.89 0.65
Representing 1.67 4.00 3.33 0.65
Analyzing and Interpreting 2.00 4.00 3.38 0.53
Score Report Interpretation: Cooper’s Fitness Test Score Report
Describing 4.00 4.00 4.00 0.00
Summarizing 3.00 4.00 3.75 0.44
Questioning 1.00 4.00 2.10 0.09
Proposed Application 1.00 4.00 2.45 1.10
Score Report Interpretation: Jabberwocky Test Score Report
Describing 4.00 4.00 4.00 0.00
Summarizing 3.00 4.00 3.75 0.44
Questioning 1.00 4.00 1.95 0.89
Proposed Application 1.00 4.00 2.00 0.97
ππ
1212
Discussion P1
• Data Habit of Mind represents a starting point
• Educators on average appear to have well developed statistical literacy skills
• Educators on average appear to struggle on score report interpretation skills: Questioning & Proposing an Application
ππ
1313
Discussion P2
• We can begin to identify clusters and patterns
• Potentially there are four ways in which we can cluster the patterns that we see when it comes to data habit of mind
ππ
1414
4-Way Data Habit of Mind
ππ
1515Mount
Sain
t V
ince
nt
Univ
ers
ity
Mount
Sain
t V
ince
nt
Univ
ers
ity
