Virginia Middle School Research AllianceProgress Made and Planning the Road Ahead

Lydotta TaylorAlliance Lead, REL Appalachia

Aimee Evan,VA Alliance Lead, REL Appalachia

May 6, 2014VASS Annual Meeting, Roanoke, VA

2

Background on REL Appalachia

Regional Educational Laboratory (REL) Program

• U.S. Department of Education, Institute of Education Sciences.• RELs provide regional support for:

– Applied research and evaluation.– Technical support and information sharing to build capacity to use data for

improved education outcomes.

3

REL Appalachia’s Mission

• Meet the applied research and technical support needs of Kentucky, Tennessee, Virginia, and West Virginia.

• Bring evidence-based information to policymakers and practitioners:– Provide support for a more evidence-reliant education system.– Inform policy and practice for states, districts, schools, and other stakeholders.– Focus on high-priority, discrete issues and build a body of knowledge over

time.

4

How We Work: Research Alliances

• What is a research alliance?– A partnership between education stakeholders and REL Appalachia.

• What is the purpose of a research alliance?– As partners, REL Appalachia and alliance members develop and carry out a

research and analytic technical assistance agenda on priority topics.• Who are the education stakeholders in an alliance?

– May include representatives from one or more schools, divisions, state education agencies, and other organizations (e.g., colleges and universities).

5

What Does REL Appalachia Do?

6

• Assess regional research needs by monitoring emerging education issues and challenges.

• Maintain and refine research alliances through ongoing dialogue between educators in each region and researchers.

• Provide analytic technical support to increase use of data and analysis to understand policies and programs, make decisions, and support effective practice.

• Conduct research and evaluation studies of rigor and method appropriate to the questions the studies attempt to answer.

• Distribute results of REL research across the region.• Coordinate and partner with other RELs and federal, state, and local

education research and technical assistance organizations.

7

Virginia Middle School Research Alliance Goals

8

Collaborative working group of researchers and practitioners.

– Superintendents, assistant superintendents, directors of curriculum & instruction, directors of assessment & testing, principals, teachers from the following divisions:

– Executive directors of SURN (School-University Research Network) and VSUP (Virginia School-University Partnership).

– Virginia Department of Education’s Office of School Improvement.– Researchers in rural education, early warning systems, data use.

What Is the Virginia Middle School Research Alliance?

– Buena Vista City– Chesterfield County– Louisa County– Norton City– Page County– Pulaski County

– Russell County– Rockingham County– Smyth County– Staunton City– Tazewell County– Wythe County

Goals of the Virginia Middle School Research Alliance

• Assist middle school practitioners in using data to inform instructional decisionmaking and improve student outcomes by:

– Identifying struggling students who need additional support.

– Selecting, implementing, and monitoring interventions to support students.

• REL Appalachia’s work with the alliance will focus on expanding the state’s early warning system to middle schools.

• Early warning system implementation is taking place beginning with four interrelated projects in 2014‒15.

9

What Is an Early Warning System (EWS)?

• Provides a systematic way to identify (“flag”) students early who are at risk of failure.

• Grounded in research.• Relies on readily available (and familiar) data: ABCs – Attendance,

Behavior, and Course grades/assessment results.• Provides information that is actionable by educators and other adults in

schools and divisions.• Requires educators to diagnose further student needs, and to use

professional judgment to support at-risk students.• Targets resources to support at-risk students while they are still in school,

before they go too far down the road of academic failure and drop out.• Examines patterns and identifies school climate issues.

10

Why Middle School Matters

• A student’s decision to drop out of school is a gradual process that starts well before high school.*– The habits and patterns set in middle school follow students to high school.– Only 50% of students entering high school two or more years behind grade

level in math/literacy are promoted to grade 10.– Students repeating grade 9 have a 20% likelihood of graduating from high

school.• Students at risk of dropping out can be identified as early as grade 6 and

share characteristics of low attendance rates, course failure, and low assessment scores.**

• An EWS helps catch these students before it’s too late.

*Sources: Allensworth & Easton (2005, 2007)**Sources: Balfanz (2009); Balfanz, R., & Herzog, L. (2005) 11

12

Projects toward EWS Implementation

13

The EWS Process*

Step 1: Establish roles and

responsibilities

Step 2: Use the EWS tool

Step 3: Analyze EWS data

Step 4: Interpret EWS data

Step 5: Assign & provide

interventions

Step 6: Monitor students

Step 7: Evaluate & refine EWS

*Sources: Therriault, S. B., O’Cummings, M., Heppen, J., Yerhot, L., Scala, J., & Perry, M. (2013)

Virginia Middle School Research Alliance Projects

• Catalog data currently being collected in schools.• Document how data are currently being used in schools, and what

supports and barriers are in place to help or that hinder use.• Identify most powerful data to track.* • Provide workshops on using data efficiently and effectively to:*

– Identify and target resources to struggling students.– Monitor each student’s progress toward mastery.

*These are proposed activities and not yet approved by IES.

15

What We’ve Learned Thus Far…

• Most data are available for an early warning system; using data is difficult. – Extracting or “triangulating” multiple data points is complex.– Data are not always centrally located in one program.– Not all pertinent data are electronic; some are kept in hard copy only.

• Analyzing historical patterns for students who are struggling may not be possible because some data do not “save” from year to year.

• Sharing information about students across school levels—from elementary to middle to high school—is not common.

16

Discussion

• What is the one thing we must get right to make an EWS worth undertaking?

• What are current hurdles or barriers that would make implementing an EWS difficult? acting on data difficult?

• Where does an EWS help meet your goals in your division? Where does it fall short?

17

Work Ahead…

• Data-use interviews: How are you using data? What supports exist? What barriers hinder?

• Validation study*: What are the most powerful data to track?• Workshop for data leaders* (July 23): superintendents, assistant

superintendents, curriculum specialists, directors of assessment/accountability, building principals.– What does effective data practice look like?– How do you create an entrenched strategy around using data, from the

division to the classroom? What foundational elements have been found to support effective data use? What additional/different elements are needed to use data effectively?

• Workshops* to assist in creating or building on these foundational elements.

*These are proposed activities and not yet approved by IES.

18

What’s Next? Planning the Road Ahead

• What questions do you still have?• What information would be useful, and when?• What information/materials do you need right away?• Further topics of interest?

19

What Are the Possibilities?

• Research studies.• Technical assistance.• Workshops.• Bridge events.• Webinars.

20

References and Materials

• Allensworth, E. M., & Easton, J. Q. (2005, June). The on-track indicator as a predictor of high school graduation. Chicago: University of Chicago, Consortium on Chicago School Research. Retrieved from http://ccsr.uchicago.edu/sites/default/files/publications/p78.pdf

• Allensworth, E. M., & Easton, J. Q. (2007, July). What matters for staying on-track and graduating in Chicago public high schools: A close look at course grades, failures, and attendance in the freshman year. Chicago: University of Chicago, Consortium on Chicago School Research. Retrieved from http://ccsr.uchicago.edu/sites/default/files/publications/07%20What%20Matters%20Final.pdf

• Balfanz, R. (2009). Putting middle grades students on the graduation path: A policy and practice brief. Baltimore: Johns Hopkins University, Everyone Graduates Center. Retrieved from http://new.every1graduates.org/putting-middle-grades-students-on-the-graduation-path-a-policy-and-practice-brief/

• Balfanz, R., & Herzog, L. (2005). Keeping middle grades students on-track to graduation: Initial analysis and implications. Presentation at the second Regional Middle Grades Symposium, Philadelphia, PA.

• Therriault, S. B., O’Cummings, M., Heppen, J., Yerhot, L., Scala, J., & Perry, M. (2013). Middle grades early warning intervention monitoring system implementation guide. Washington, DC: National High School Center at American Institutes for Research. Retrieved from http://betterhighschools.org/EWS_middle.asp

Connect with Us!

www.relappalachia.org

@REL_Appalachia

Aimee Evanaevan@QRA-LLC.net

703-655-3695

21