jennifer coffey, ph.d. spdg program lead. kathe shelby, oh karen jones, de teresa farmer, al ...

Welcome to SIG Day

Jennifer Coffey, Ph.D. SPDG Program Lead

SIG DAY AGENDA

SPDG Program Area Meeting2011 OSEP Project Directors’ Conference

SIG Day Planning Committee

Kathe Shelby, OH Karen Jones, DE Teresa Farmer, AL Letha Bauter, OK Veronica MacDonald, TN Renee Scott, KY Jennifer Coffey Jeanna Mullins, MSRRC Pat Gonzalez Audrey Desjarlais

Please Welcome

The Regional Parent Technical Assistance Directors

And don’t forget…

Early Bird on Professional Development Systems - Tuesday, July 19 (7:30am-8:30am) Location: Maryland A

SIG Night  - Tuesday, July 19 -  (6:00-8:00pm)Location: Ireland’s Four Fields, 3412 Connecticut Ave. NW, Metro Station: Cleveland Park

The Year in Review & Looking Forward

Opportunities to Learn and Share Professional Development Series Revising the Program Measures and

Creating Methodology OSEP Policies SPDG Competition Building Partnerships

Opportunities to Learn and Share

Monthly Webinars – “Directors’ Calls” Evaluator Community of Practice Resource Library “Regional Meetings” Project Director’s Conference PLC’s

SPD

G R

EG

ION

AL M

EETIN

G

RES

OU

RC

ES

To view the SPDG Regional Meeting Materials go to: http://signetwork.org/content_pages/27

PLCs Adolescent Literacy Behavior & School

Climate Coaching Collaboration with

IHEs Family Engagement Grant Management Implementation

Conversations

Low Incidence Affinity Group

RTI & Multi-Tiered models of intervention

Scaling Up & Comprehensive PD Approaches

Secondary Transition

PR

OFE

SS

ION

AL LE

AR

NIN

G

CO

MM

UN

ITIE

S - W

IKIS

To view the Professional Learning Community (PLC) Wikis go to : http://signetwork.org/wikis

Evidence-based Professional Development

Models of and Evaluating Professional Development› Date: January 12, 3:00-

4 :30pm ET › Speakers: Julie Morrison, Alan Wood,

& Li Walter (SPDG evaluators)

  SPDG REGIONAL MEETINGS

› Topic: Evidence-based Professional Development

Evidence-based PD Innovation Fluency

› Date: March 24, 3:00-4:30pm ET› Speaker: Karen Blase, SISEP

Professional Development for Administrators› Date: April 19, 3:00-4:30pm ET› Speakers: Elaine Mulligan, NIUSI Leadscape› Rich Barbacane, National Association of Elementary

School Principals Using Technology for Professional

Development› Date:May 18, 2:00-3:30pm ET› Speaker: Chris Dede, Ph.D., Learning Technologies

at Harvard’s Graduate School of Education

20

11

PR

OFE

SS

ION

AL

DEV

ELO

PM

EN

T S

ER

IES Go to the Home Page to link each webinar segment:

http://signetwork.org

• Evidence-Based Intervention Practices Insert your SPDG initiative here

• Evidence-Based Implementation Practices Professional Development

Staff Competence: Selection, Training, Coaching, and Performance Assessment Drivers Adult learning methods/principles Evaluation

14

Two Types of Evidence-Based Practices

15

HOW?

CA: ERIA’s Evidence-based Practices

The Program Guide articulates a comprehensive set of practices for all stakeholders.

16

Implementation Practices Intervention Practices

Initial Training

Team-based Site-level Practice and Implementation

Implementation Rubric facilitates self-eval

Ongoing Coaching

Booster Trainings

Implementation Rubric reflection on next steps

The 5 Steps of ERIA

Data-informedDecision-making Screening and

Assessment Progress Monitoring

Tiered Interventions and Learning Supports

Enhanced Literacy Instruction

CA: Two Integrative Evaluation Tools Serve as Implementation Drivers

Program Guide articulates PD model› introduces and illustrates › contextualizes the training› gets away from “you had to be there”

Implementation Rubric operationalizes PD model› drives ongoing implementation› enables fidelity checks› is possible to evaluate

Everyone is on the same page Sustainability (beyond funding, staff turnover) Scale-up (recruit new sites/districts, beyond SPDG) Diversity of approaches enabled

17

18

HOW?

19

Training must be … › Timely › Theory grounded (adult learning)› Skill-based

Information from Training feeds back to Selection and feeds forward to Coaching

Best Practices in Training

Selection Training Coaching

(Blase, VanDyke, & Fixsen, 2010)

20

Design a Coaching Service Delivery Plan

Develop accountability structures for Coaching – Coach the Coach!

Identify on-going professional development for coaches

Best Practices in Coaching

Coaching Performance Assessment

Training

(Blase, VanDyke, & Fixsen, 2010)

21

Must be a transparent process

Use of multiple data sources

Fidelity of implementation should be assessed at the local, regional, and state levels

Tied to positive recognition

Information from this driver feeds back to Selection, Training, and Coaching and feeds forward to the Organization Drivers

Best Practices in Performance Assessment (Fidelity)

22

Assess fidelity of implementation at all levels and respond accordingly

Identify outcome measures that are …› Intermediate and longer-term› Socially valid› Technically adequate: reliable and valid› Relevant data that is feasible to gather, useful

for decision making, widely shared and reported frequently

Best Practices in Decision Support Data Systems

Why focus on professional development?

“No intervention practice, no matter what its evidence base, is likely to be learned and adopted if the methods and strategies used to teach or train students, practitioners, parents, or others are not themselves effective.”

"Let's Be Pals: An Evidence-based Approach to Professional Development." Dunst & Trivette, 2009

Using Research Findings to Inform Practical

Approaches to Evidence-Based Practices

Using Research Findings to Inform Practical

Approaches to Evidence-Based Practices

Carl J. Dunst, Ph.D. Carol M. Trivette, Ph.D.Orelena Hawks Puckett Institute

Asheville and Morganton, North Carolina

Presentation Prepared for a Webinar with the Knowledge Transfer Group, U.S. Department of Health and Human Services, Children’s Bureau Division of Research and

Innovation, September 22, 2009

“Adult learning refers to a collection of theories, methods, and approaches for describing the characteristics of and conditions under which the process of learning is optimized.”

Six Characteristics Identified in How People Learna Were Used to Code and Evaluate the Adult Learning Methods

Planning

Introduce Engage the learner in a preview of the material, knowledge or practice that is the focus of instruction or training

Illustrate Demonstrate or illustrate the use or applicability of the material, knowledge or practice for the learner

Application

Practice Engage the learner in the use of the material, knowledge or practice

Evaluate Engage the learner in a process of evaluating the consequence or outcome of the application of the material, knowledge or practice

Deep Understanding

Reflection Engage the learner in self-assessment of his or her acquisition of knowledge and skills as a basis for identifying “next steps” in the learning process

Mastery Engage the learner in a process of assessing his or her experience in the context of some conceptual or practical model or framework, or some external set of standards or criteria

a Donovan, M. et al. (Eds.) (1999). How people learn. Washington, DC: National Academy Press.

Additional Translational Synthesis Findings

The smaller the number of persons participating in a training (<20), the larger the effect sizes for the study outcomes.

The more hours of training over an extended number of sessions, the better the study outcomes.

The practices are similarly effective when used in different settings with different types of learners.

Practices

Number

Mean Effect Size (d)

95% Confide

nce Interval

StudiesEffect Sizes

Pre-class exercises 9 9 1.02 .63-1.41

Out of class activities/self-instruction

12 20 .76 .44-1.09

Classroom/workshop lectures 26 108 .68 .47-.89

Dramatic readings 18 40 .35 .13-.57

Imagery 7 18 .34 .08-.59

Dramatic readings/imagery 4 11 .15 -.33-.62

Effect Sizes for Introducing Information to Learners

Practices

Number

Mean Effect Size (d)

95% Confide

nce Interval

StudiesEffect Sizes

Using learner input for illustration 6 6 .89 .28-1.51

Role playing/simulations 20 64 .87 .58-1.17

Real life example/real life + role playing 6 10 .67 .27-1.07

Instructional video 5 49 .33 .09-.59

Effect Sizes for Illustrating/Demonstrating Learning Topic

Practices

Number

Mean Effect Size (d)

95% Confide

nce Interval

Studies

Effect Sizes

Real life application + role playing 5 20 1.10 .48-1.72

Problem solving tasks 16 29 .67 .39-.95

Real life application 17 83 .58 .35-.81

Learning games/writing exercises

9 11 .55 .11-.99

Role playing (skits, plays) 11 35 .41 .21-.62

Effect Sizes for Learner Application

Practices

Number

Mean Effect Size (d)

95% Confidence

Interval

Studies

Effect Sizes

Assess strengths/weaknesses

14 48 .96 .67-1.26

Review experience/make changes

19 35 .60 .36-.83

Effect Sizes for Learner Evaluation

Practices

Number

Mean Effect Size (d)

95% Confidence

Interval

Studies

Effect Sizes

Performance improvement 9 34 1.07 .69-

1.45

Journaling/behavior suggestion

8 17 .75 .49-1.00

Group discussion about feedback 16 29 .67 .39-.95

Effect Sizes for Learner Reflection

Practices

Number

Mean Effect Size (d)

95% Confidence

Interval

Studies

Effect Sizes

Standards-based assessment 13 44 .76 .42-

1.10

Self-assessment 16 29 .67 .39-.95

Effect Sizes for Self-Assessment of Learner Mastery

Summary of Training Findings

To be most effective need to actively involve the learners in judging the consequences of their learning experiences (evaluate, reflection, & mastery)› Need learner participation in learning new

knowledge or practice› Need learner engagement in judging his or

her experience in learning and using new material

Innovation Fluency

Definition: Innovation Fluency refers to the degree to which we know the innovation with respect to:› Evidence› Program and Practice Features› Implementation Requirements

35

Implementation Pre-Requisites

After you› Have chosen based on student needs› Looked for “best evidence” to address

the needAn Evidence-Based Practice or ProgramAn Evidence-Informed Initiative or

Framework Systems Change and Its Elements

36

Implementation Pre-Requisites

After you› Have chosen based on student needs› Looked for “best evidence” to address

the needAn Evidence-Based Practice or ProgramAn Evidence-Informed Initiative or

Framework Systems Change and Its Elements

Then it’s time to:› Clearly identify and operationalize

the elements

37

Professional Problem Solving 9 Critical Components

Parent Involvement

Problem Statement

Systematic Data Collection

Problem Analysis

Goal Development

Intervention Plan Development

Intervention Plan Implementation

Progress Monitoring

Decision Making

Critical Component Ideal Implementation

Acceptable Variation

Unacceptable Variation

Unacceptable Variation

Critical Component 1: Description

Description of implementer

behavior

Professional Practices in Problem Solving: Benchmarks and Innovation ConfigurationsIowa Area Education Agency Directors of Special Education, 1994

38

Interaction of Leadership and Implementation Support Drivers Regarding Administrators

Project Level Providing Leadership

District Level Providing Leadership

Building Level Providing Leadership

Develop systems for district and building administrators to implement practices with success

Develop systems for building administrators to implement practices with success

Develop systems for building staff to implement practices with success

Purpose: To Develop project Capacity (e.g., data systems, information resources, incentives) and Competency (e.g., selection, training, coaching) so administrators can implement practices with success

Provides guidance, visibility, funding, political support for MiBLSi

StudentsStudents

Building StaffBuilding Staff

Building Leadership Team

Building Leadership Team

LEA District Leadership Team

LEA District Leadership Team

Across State

Multiple District/Building Teams

All staff

All students

Multiple schools w/in local district

Who is supported?

How is support provided?

Provides guidance, visibility, funding, political support

Provides coaching for District Teams and technical assistance for Building Teams

Provides guidance and manages implementation

Provides effective practices to support students

Improved behavior and reading

ISD Leadership Team

ISD Leadership Team

Regional Technical

Assistance

Regional Technical

Assistance

MichiganDepartment of

Education/MiBLSi Leadership

MichiganDepartment of

Education/MiBLSi Leadership

Multiple schools w/in intermediate district

Provides guidance, visibility, funding, political support

MiBLSi Statewide Structure of Support

40

Developing CapacityThrough “Manualization”

Manuals are created to provide information and tools for implementation

Various levels

District Level

Building Level

41

Developing Capacity Through “Practice Profiles” (Implementation Guides)

• Implementation Guides have been Developed for

– Positive Behavioral Interventions and Supports at the Building Level

– Reading Supports at the Building Level

– Building Leadership Team

– District Leadership Team

• Quick Guides have been developed for

– Principals

– Coaches

42

Practice Profile: Building Leadership Team Example

43

Program Performance Measures

Thank You› To all who joined the large group discussions› To the small working group members:

Patti Noonan Jim Frasier Susan Williamson Nikki Sandve Li Walter Ed Caffarella Jon Dyson Julie Morrison

Performance Measures

Performance Measurement 1: Projects use evidence-based professional development practices to support the attainment of identified competencies.

Performance Measurement 2: Participants in SPDG professional development demonstrate improvement in implementation of SPDG-supported practices over time.

Performance Measurement 3: Projects use SPDG professional development funds to provide follow-up activities designed to sustain the use of SPDG-supported practices. (Efficiency Measure)

Performance Measurement 4: Highly qualified special education teachers that have participated in SPDG supported special education teacher retention activities remain as special education teachers two years after their initial participation in these activities.

Continuation Reporting

2007 grantees Everyone else

Applying for a New SPDG

Bidders’ Webinars Planning with potential partners Gathering needs data Increasing your knowledge about

evidence-based professional development

SPDG Program – OSEP Policies

No-cost extensions Personnel changes Continuation Reports Carrying out the activities in your

application

PTI/CPRC Poll Results Must contract or subgrant with a PTI or a CPRC

› Mandatory Recommended that

› Use good contracting practices› Communicate regularly and ensure all understand

expectations› Track progress with implementation and outcome

data – working together toward at least 1 of your SPDG’s objectives

› When applying for a new SPDG, meet with your PTI/CPRC (or both) to discuss the work you might do together

Highlights from the Survey It provides us with an avenue for receiving parent

input; disseminating information, resources and training to parents; and increasing parent involvement in their child's education.

We value the insights our Parent Center provides and use their feedback to adjust practices so that parents are more engaged in their children's education. They provide a perspective we wouldn't otherwise have. They, also, are a tremendous asset to the project through their development of parent materials and professional development. The professional development they provide for parents is the bridge between parents and the work within the project.

One of the greatest strengths is the desire, of both organizations, to provide family members with high quality, meaningful professional development to strengthen their ability to advocate, not only for their own children, but for all children. The collaboration is built on mutual respect and the shared vision of building meaningful family involvement in the schools and districts in our state.

The relationship in the past has been strained, but through our conversations with the new SPDG proposal, we have turned a corner and I am hopeful we can have a true partnership going forward.

Our PTI is well connected with families across the state, they work well with our

office and are willing to participate to help improve achievement for SWD. The PTI has recently been more involved and has brought in a national parent engagement expert. We are working to build from this Epstein work and bring those components of parent engagement into our SDPG work.

Challenges for these partnerships

Didn't fund them at the same level as other partners so PTI is not able to be at the table for all conversations. Overcoming entrenched idea by school districts that family engagement is an add on after they have perfected their process - okay to involve families in development

Regarding collaboration, there are not many challenges as our PTI is very responsive and collaborative. It's getting the work done and the systems in place that sometimes delay other activities to occur or move forward.

Challenges cont. Keeping collaborative efforts aligned with

the multiplicity of educational initiatives in the state; occasional conflict on perspectives of how grant funds and activities are to be carried out and perceptions of how the other PTI activity is aligned with grant objectives.

We need to schedule more time for face to face collaboration, and plan to do so in the future.

Matching grant needs with expertise in the PTI/CPRC

Ideas to Improve the Partnerships

SIGnetwork- perhaps build some networking/PD sessions that the SPDG and PTIs are required to attend/participate in together.

The requirement of the SPDG for the SEA to partner with the PTI is a topic that needs to be addressed. The feeling that I have gotten when it is raised is that regardless of the PTI's position, it is the SEA's responsibility to make the relationship work….

Continue to share ideas about examples of collaborative efforts.

More ideas

Continue to share ideas about examples of collaborative efforts.

Clarify role of PTI. Emphasize cooperation, communication, collaboration with both SEA and LEAs. Deemphasize advocacy/adversarial role.

Perhaps some joint webinars regarding what other SPDG projects are doing around parent and family collaboration would be helpful.

Go Mobile: Data Collection with an iPadCarlos Romero,

SPDG Program Technology Survey FindingsAudrey Desjarlais, Signetwork CoordinatorMay 2011

Response Rate

49 Respondents 76% state grantee response rate (35 of

46 funded state grantees) Included evaluators, directors, and

coordinators Representation: 25 – state

departments, 4 – IHEs, 6 - independent contractors, 5 - unknown

Top 6 Purposes to learn about technology applications:

Develop online training modules Evaluate activities and programs (includes

conduct needs assessment) Setup online habitats for

collaboration/communication Provide TA, consultation, coaching, and

mentoring Manage data (including collection, analysis,

reporting) Knowledge Management - disseminate

news/info

Technologies to learn about: Learning Management Systems Web conferencing Online meetings spaces Podcasts Collaborative Document Editing Learning Modules Multiuser Virtual Learning Environment (Second

Life) Online surveys Project Management Rapid e-learning modules Wikis

Resource/information repositories Video conferencing Media sharing sites Mobile device technologies Blogs Social Networking Vidcast Threaded discussions Vlogs Video Text-based live chat

Technologies to learn about:

Barriers to exploring, adopting and implementing emerging technologies:

Lack of knowledge (individual, staff) Lack of time (to investigate learn, and practice) Lack IT support Lack the infrastructure (at all levels: state,

district, school) Lack funding Accessibility issues: security/firewalls, internet

connection speed Rigorous approval process Things change too quickly

Strategies for how we’ll tackle those barriers for implementing technologies

Follow-up to Regional Meetings Website Updates

May 2011

EV

EN

TS

CA

LEN

DA

R New Events Calendar is Live – Details on all our PD events – PLC sessions, Implementation Conversations, Directors’ Webinars, Evaluators’ Events, Reporting Guidance and Bidders’ Webinars!

RES

OU

RC

E LIB

RA

RY Resource Library is LIVE! Includes links, materials, resources, and

information from SPDG grantees and National OSEP funded TA&D Centers.

EVA

LUATIO

N Resources focused on Evaluation is now available. Includes information on reporting, instruments, data collection and display, and more.

Concurrent Sessions

• Evaluators – Maryland C

• Directors, Coordinators and PTI Directors – Maryland AB

Speed Share: SPDG Partnerships with PTIs and CPRCS• 6 Speed Share Rounds

• Pairs will have 5 minutes to share information with one another.

SPEED

SH

AR

E - D

ISC

US

SIO

N

ITEM

S

• Introduce yourself (name, state, role).• Identify the top 1-3 activities your state has

been successful with parent engagement.• Explore how your projects make use of

contracts with parent organizations (PTIs/CPRCs)

• Describe the challenges you’ve experienced collaborating with one another.

• Explain how we can build relationships with PTIs and CPRCS. What are the effective practices?

• Describe what kinds of needs assessment you did to develop your collaborative contract and action plan.

• Share product or material links.• Share strategies for Family Engagement

Dissemination.

Communities of Practice

• Cynthia Glimpse, Technical Assistance Coordination Center (TACC)

• Amy Jenks, Director, New Hampshire SPDG

PLACEHOLDER - ADD Presenter’s SLIDES HERE

PLC Discussions – 3, 2, 1 Format1 – Coaching

2 – Collaboration with IHEs, Leadership Development, and Capacity Building and Restructuring

3 - Implementation Conversations

4 – Response to Intervention / Multi-tiered Models of Intervention

5 – Scaling Up and Comprehensive Systemic Professional Development

6 - Technology

Opportunities to Learn and Share