SPDG Implementation Conversations

Show-Me ImplementationSeptember 14, 20103:00 PM – 4:00 PM, EDT

Review from last conversation

• Stages of Implementation– Exploration– Installation– Initial Implementation– Full Implementation– Innovation

• Two components that when combined, result in successful and sustainable outcomes– Program/Initiative (Innovation)

• Set of evidence-based practices• Selected on: Need, fit, Resource Availability, Evidence,

Readiness for Replication, Capacity to Implement– Supporting Infrastructure (Implementation)

• Ensuring that the interventions are implemented correctly with the “right people”, at the “right time”, in the “right amounts” (Implementation Fidelity)

How does your state select programs/initiatives to support at the state level?1. Mandated by State/Federal 2. Supported/endorsed by US Department

of Education3. “Pet” project of influential individuals4. “Tradition” (we’ve always funded that)5. Research-based and evidence of positive

student outcomes6. Other

Has your state done a review (evaluation) of the programs/initiatives it supports to determine depth of implementation and impact?

• YES• NO

If yes, how was it done?1. Internal review (State-developed

process)2. External evaluator3. Other

What statewide programs/initiatives do you support in your state?1. SW-PBS2. RTI3. PLC4. Literacy5. Early Childhood6. High Schools That Work7. Dropout Prevention8. Other

Missouri Experience• Multiple programs/initiatives

supported at the state level with both state and federal funds

• Many lacked data to show level (depth) of implementation and/or impact on student achievement

• Competing interests for $$$ and ??? about accountability prompted Commissioner to call for an implementation audit

Implementation Audit• Purpose: “assess the range of implementation of

educational initiatives and the relationship between the degree of implementation and changes in student achievement.”

• Conducted by outside evaluator (The Leadership and Learning Center)

• Department staff selected around 20 programs/initiatives supported at the state level

• State staff identified districts/buildings implementing the programs/initiatives

• Stratified, random sample

Implementation AuditImplementation was assessed based upon• On-line surveys• Interviews• Document reviews• A scoring rubric that considered– Learning context– Instructional practices– Professional development– Leadership practices

Student outcomes were measured based on Missouri Assessment Program (MAP) data in language arts, math and science

Audit Results/Recommendations

• On a four-point scale (4 being the highest degree of implementation) programs/initiatives received both an average overall score and a “range of implementation” score

• Low implementation may not be a reflection of a poor program, but rather a reflection of “initiative fatigue” at the school and district level

• Depth of implementation is most clearly related to gains in student achievement (Effective program implemented deeply will show best results)

• Insufficient resources to effectively implement & sustain the program was the most frequently cited concern

Audit Results/Recommendations

Other implementation tools• Currently use SET for SW-PBS http://

pbismissouri.org/exemplar.html • In the process of developing

implementation assessment tools similar to SET for Missouri Integrated Model (MIM), Response to Intervention (RtI) and Professional Learning Communities (PLC)—stay tuned

• Missouri Comprehensive Guidance—Internal Improvement Review document http://missouricareereducation.org/CDs/Guidance_Manual/AppendixD.pdf

Resources• Missouri SW-PBS

– http://pbismissouri.org/• Missouri Integrated Model (MIM)

– http://mimschools.org/• Professional Learning Communities (PLC)

– http://dese.mo.gov/divteachqual/sii/prolearning/• Response to Intervention (RtI)– http://dese.mo.gov/3tieredmodels/rti/

• Missouri Comprehensive Guidance– http://

missouricareereducation.org/curr/cmd/guidanceplacementG/mcgp.html

• Missouri Implementation Audit—final report– http://dese.mo.gov/Appendix6.pdf

Questions…discussion…

15

Contact: Pam Williamspam.williams@dese.mo.gov

Missouri Department of Elementary & Secondary EducationP. O. Box 480 Jefferson City, Missouri 65102-0480573-751-2965