uncovering the sources of student learning challenges: analyzing and interpreting data december 4,...
TRANSCRIPT
Uncovering the Sources of Student Learning Challenges:
Analyzing and Interpreting Data
December 4, 2014Roanoke, VA
Welcome and Introduction
Aimee EvanVirginia Middle School Research Alliance Lead
REL Appalachia
Regional Educational Laboratory (REL) Program
• U.S. Department of Education, Institute of Education Sciences (IES).
• RELs provide regional support for:– Applied research and evaluation.
– Technical support and information sharing to build capacity to use data for improved education outcomes.
3
REL Appalachia’s Mission
• Meet the applied research and technical support needs of Kentucky, Tennessee, Virginia, and West Virginia.
• Bring evidence-based information to policymakers and practitioners:– Provide support for a more evidence-reliant education system.
– Inform policy and practice for states, divisions, schools, and other stakeholders.
– Focus on high-priority, discrete issues and build a body of knowledge over time.
4
What Does REL Appalachia Do?
• Assess regional research needs by monitoring emerging education issues and challenges.
• Maintain and refine research alliances through ongoing dialogue between educators in each region and researchers.
• Provide analytic technical support to increase use of data and analysis to understand policies and programs, make decisions, and support effective practice.
• Conduct research and evaluation studies of rigor and method appropriate to the questions the studies attempt to answer.
• Distribute results of REL research across the region.
• Coordinate and partner with other RELs and federal, state, and local education research and technical assistance organizations.
5
Research Alliances
• What is a research alliance?– A partnership between education stakeholders and REL Appalachia.
• What is the purpose of a research alliance?– As partners, REL Appalachia and alliance members develop and carry out a
research and analytic technical assistance agenda on priority topics.
• Who are the education stakeholders in an alliance?– May include representatives from one or more schools, divisions, state
education agencies, and other organizations (e.g., colleges and universities).
6
How Do We Do the Work: REL Appalachia Staff
• Justin Baer, Director, REL Appalachia
• Lydotta Taylor, Alliance Lead, REL Appalachia
• Kellie Kim, Analytic Technical Support Lead, REL Appalachia
• Becky Smerdon, Early Warning Systems Content Lead
• Aimee Evan, VMSRA Lead
• Angela Estacion, VMSRA Project Lead
7
What Is the Virginia Middle School Research Alliance?
• Collaborative working group of practitioners and researchers.– Superintendents, assistant superintendents, directors of curriculum &
instruction, directors of assessment & testing, principals, teachers from the following divisions:
• Campbell County• Greene County• Harrisonburg
County• Louisa County• Nelson County• Norton City
• Pulaski County• Rockbridge County• Russell County• Salem City• Smyth County• Staunton City
– Executive directors of SURN (School-University Research Network) and VSUP (Virginia School-University Partnership).
– Virginia Department of Education (VDOE).– REL Appalachia researchers in rural education, early warning systems, data use.
8
Goals of the Virginia Middle School Research Alliance
• Assist middle school practitioners in using data to inform instructional decisionmaking and improve student outcomes by:
– Identifying struggling students who need additional support.
– Selecting, implementing, and monitoring interventions to support students.
• Focus on expanding the state’s early warning system (EWS) to middle schools.
9
Virginia Middle School Research Alliance Projects
• Catalog EWS data, assessments, and interventions currently being collected/used in schools.
• Document how data are currently being used in schools, and what supports and barriers are in place to help or hinder use.
• Determine the most powerful data to use to identify students in need of further assistance.
• Provide workshops on using data efficiently and effectively to:– Identify struggling students and determine how to target resources to meet
their needs.
– Monitor each student’s progress.
10
Virginia Middle School Research Alliance Future Work
• Workshops* will continue to build the capacity of educators to interpret meaning from data in order to best meet students’ needs by:– Improving classroom instructional strategies (Spring 2015).– Improving division and schoolwide strategies (Summer 2015).
– Workshop on determining the most powerful data in your own division to identify struggling students (Winter 2015).
*These activities are currently under review by IES.
11
Workshop Overview and Goals
Aimee Evan
12
Introductions
• Name
• Role
• Division
13
Goals for Participants
• Understand a process to use data to identify sources and causes of students’ learning challenges.
• Recognize what data sources can be used to accurately identify sources of students’ learning challenges and to monitor their progress toward improvement.
• Learn what building leadership can do to facilitate and improve teachers’ use of data.
• Understand your own and your school’s status in developing data-use capacity.
14
Logistics
• Binder Navigation
– Agenda
– Tabs
– Survey
• Housekeeping:– Bathrooms
– Breaks
– Lunch
15
Introduction to the Early Warning System Process
and Foundational Elements
Aimee Evan
What Is an Early Warning System (EWS)?
• Provides a systematic way to identify (“flag”) students early who are at risk of failure.
• Grounded in research.• Relies on readily available (and familiar) data:
– ABCs: Attendance, Behavior, and Course grades/assessment results.• Provides information that is actionable by educators in schools and
divisions.• Requires educators to diagnose further student needs, and to use
professional judgment to support at-risk students.• Targets resources to support at-risk students while they are still in
school, before they go too far down the road of academic failure and drop out.
• Examines patterns and identifies school climate issues.Sources: Allensworth & Easton, 2005, 2007; Balfanz, 2009; Balfanz & Herzog, 2005
17
The EWS Continuous Improvement CycleStep 1: Establish
roles and responsibilities
Step 2: Use the EWS Middle Grades Tool
Step 3: Review the EWS data
Step 4: Interpret the EWS data
Step 5: Assign and provide
interventions
Step 6: Monitor students and interventions
Step 7: Evaluate and refine the EWIMS
process
Source: Therriault et al., 2013
18
The EWS Continuous Improvement Cycle
Infrastructure and Support Are Just as Important as the Process.
19
EWS – A Systems Perspective
Step 1: Establish roles and responsibilities
Step 2: Use the EWS Middle Grades Tool
Step 3: Review the EWS data
Step 4: Interpret the EWS data
Step 5: Assign and provide interventions
Step 6: Monitor students and interventions
Step 7: Evaluate and refine the EWIMS
process
A. Data and Computer Data Systems
B. Educator Knowledge & Skills for Data Use
C. School Organization for Data Use
20
References
Allensworth, E. M., & Easton, J. Q. (2005, June). The on-track indicator as a predictor of high school graduation. Chicago: University of Chicago, Consortium on Chicago School Research. Retrieved from http://ccsr.uchicago.edu/sites/default/files/publications/p78.pdf
Allensworth, E. M., & Easton, J. Q. (2007, July). What matters for staying on-track and graduating in Chicago public high schools: A close look at course grades, failures, and attendance in the freshman year. Chicago: University of Chicago, Consortium on Chicago School Research. Retrieved from http://ccsr.uchicago.edu/sites/default/files/publications/07%20What%20Matters%20Final.pdf
Balfanz, R. (2009). Putting middle grades students on the graduation path: A policy and practice brief. Baltimore: Johns Hopkins University, Everyone Graduates Center. Retrieved from http://new.every1graduates.org/putting-middle-grades-students-on-the-graduation-path-a-policy-and-practice-brief/
Balfanz, R., & Herzog, L. (2005). Keeping middle grades students on-track to graduation: Initial analysis and implications. Presentation at the second Regional Middle Grades Symposium, Philadelphia, PA.
Therriault, S. B., O’Cummings, M., Heppen, J., Yerhot, L., Scala, J., & Perry, M. (2013). Middle grades early warning intervention monitoring system implementation guide. Washington, DC: National High School Center at American Institutes for Research. Retrieved from http://betterhighschools.org/EWS_middle.asp
21
Local Example of Foundational Elements:How Northside Middle School Uses Data
Lori Wimbush, Principal April Griffin, English Teacher Christina Hall, Math Teacher
Linda Shiflett, Special Education Teacher Laurie Spickard, Data Specialist
“Schools to Watch”: Northside Middle School
23
Year of Change
Northside Middle School
Reading Reading Reading Reading Reading Reading Reading Mathematics Mathematics Mathematics Mathematics Mathematics Mathematics Mathematics
All Students
Disadvantaged
SWD
Black
Hispanic
White
LEP All Students
Disadvantaged
SWD Black Hispanic
White LEP
2009-10 89 80 70 75 77 93 69 79 65 60 66 65 81 69
2010-11 94 90 88 94 90 94 79 90 84 78 85 83 92 71
24
New Math SOL Results
Northside Middle School Mathematics
All Students
Disadvantaged
SWD Black Hispanic
White LEP
2010-11 Results 90 84 78 85 83 92 712011-12 Results 88 80 66 68 83 81 79
2011-12 State Results 68 54 40 52 61 75 592012-13 Results 85 79 61 73 77 87 73
2012-13 State Results 71 57 41 55 64 77 592013-14 Results 85 78 61 76 79 86 86
2013-14 State Results 74 61 43 60 67 80 62
25
New Reading SOL Results
Northside Middle School Reading
All Students
Disadvantaged
SWD Black Hispanic
White LEP
2010-11 Results 94 90 88 94 90 94 792011-12 Results 93 87 75 83 93 94 72
2011-12 State Results 89 81 66 80 84 93 802012-13 Results 79 68 42 62 61 83 44
2012-13 State Results 75 59 43 59 65 82 542013-14 Results 79 68 42 62 61 83 44
2013-14 State Results 75 59 43 59 65 82 54
26
2013–14 Annual Measurable Objectives (AMOs)
Proficiency Gap Dashboard for Federal AccountabilityReading Reading Reading Mathe-
maticsMathe-matics
Mathe-matics
AMO Target
AMO Result
Met AMO Target
AMO Target
AMO Result
Met AMO Target
All Students 69 75 YES 66 85 YES
Gap Group 1: Students with Disabilities, English Language Leaners, Economically Disadvantaged Students (unduplicated)
59 60 YES 57 76 YES
Gap Group 2: Black Students 57 66 YES 56 76 YES
Gap Group 3: Hispanic Students 60 50 3YR 60 79 YES
Key:YES = Met objectives based on the current year resultTS = Too small; objective not evaluated due to too few studentsNO = Did not meet objective - = No data for groupN/A = Not applicable
3YR = Met objective based on the 3 year average resultR10 = Met objective by reducing failure rate by at least 10 percent< = A group below state definition for personally identifiable results* = Data not yet available
27
Changes as a School
• Promote atmosphere where ALL students can improve!• 90 minutes Math and English each day.• All teachers remediate during class or before/after school.
They turn in weekly SOL evaluation sheets.
28
Weekly SOL Evaluation
29
More Changes as a School
• Use Student-Based Performance By Question/By Teacher to drive instruction.
• Continue to use regular common assessments analysis to drive instruction.
30
Student Performance By Question/Teacher
31
Analysis by Question
32
Analysis by Student
33
Changes as a School, continued
• Monthly and bi-weekly data meetings by subject area:– Common planning.– Benchmark analyses.
34
Benchmark Analysis
35
Benchmark Analysis
36
Benchmark Analysis
37
Math
• Common planning and Math Team meetings:
– Analyze common assessments and share successful teaching strategies.
– Communicate across grade levels to help use common language when teaching concepts.
– Continuous cumulative reviews.
– Error analysis after assessments.
– Various math programs.
– Math Tutors help support teachers to fill in gaps in learning.
38
English
• Students reading 20 minutes per day:
– Teachers model reading.
– Conference with students about various topics.• Students choose what to read and work on needed skill:
– Allows students to make choices all year on their reading interests as well as assignments.
– Students make choices and they take ownership of their learning.
• Ultimate goal is for students to become lifelong readers.• All teachers in our building can count on students having a
book to read during any down time.
39
Special Education
• Use data to determine self-contained Math and English classes.
• Re-evaluate student learning plans.• Use data to drive IEPs.• Look at the individual student and his/her specific needs.• Focus on improvement for ALL students.
40
Social Studies
• Remediation occurs the month prior to SOLs, during lunch.• Creation of computer lab with laptops to allow for increased
computer use.• Continuation of benchmark analysis meetings.
41
