Program will start

3:00PM Eastern2:00PM Central

1:00PM MountainNoon Pacific

11:00AM Alaska

Conference Planning Committee

Jack Cummings

Peg Dawson

Susan Gorin

Patti Harrison

Ron Palomares

Rick Short

Participant Evaluation

www.indiana.edu/~futures/eval.html

RESEARCH TO PRACTICE: The Role of Evidence-Based Interventions in Practice

Thomas KratochwillUniversity of Wisconsin-

Madison

Purpose of Presentation Provide an Overview of the

Evidence-Based Intervention (EBI) Movement and Implications for School Psychology

Review Implications of the EBI Movement for Graduate Education Programs

Purpose (Cont.) Review Implications of the EBI

Movement for Practice Implications for the Futures

Conference

Resources Chambless & Ollendick (2001).

Empirically Supported Psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685-716

Resources (Cont.) Kratochwill & Callan-Stoiber

(in press). Evidence-based interventions in school psychology: Conceptual foundations of the Procedural and Coding Manual… School Psychology Quarterly, 17 (4),

The Evidence-Based Intervention Movement in Perspective

Evidence-Based Medicine The Scientist-Practitioner

Model The Managed Care Movement Reform in Special Education Professional Organizations

Professional Organizations Committee on Science and Practice,

Society for Clinical Psychology, Division 12 American Psychological Association (APA)

Committee for Empirically –Supported Practice, Division of Clinical Child Psychology Division 53, APA

Interdisciplinary Committee on Evidence-Based Youth Mental Health Care

Professional Organizations Task Force on Evidence-Based

Interventions in School Psychology (Sponsored by APA Division 16 and the Society of the Study of School Psychology and Endorsed by the National Association of School Psychologists)

Purpose of the Task Force To identify prevention and

intervention outcome studies relevant to the practice of school psychology

To code those studies according to Task Force criteria

To determine to what degree the interventions are evidence-based

Purpose (Cont.) To offer the field of school

psychology and related fields some guidelines for using and adopting effective programs

To provide a template for improving research in the field of school psychology and education

Task Force Domains of Interest School-and community-based

intervention programs for social and behavioral problems

Academic intervention programs

Family and parent intervention programs

Domains (Cont.) School-wide and classroom-

based programs Comprehensive and

coordinated school health care programs

Methodological Focus

Group Research Single-Participant Research Qualitative Research

Procedures Confirmatory Program

Evaluation

The Review Process Six components considered Each component involves

validity issues that need to be addressed during the review process

EXISTING RESEARCH REPORTS ARE CONSIDERED ALONG VALIDITY LINES

Statistical Conclusion Validity Internal Validity Construct Validity External Validity

ORGANIZATION OF RESEARCH DOMAIN The Problem Formulation

Challenge Validity Issues in Problem

Formulation (e.g., operational specification and operational detail)

IDENTIFICATION OF RESEARCH Threats of Validity (e.g.,

fugitive literature, file drawer problem, publication bias)

Resources needed to search

REVIEW OF STUDIES Unrepresentative Studies Methods of Gathering Studies Retrieval Bias Sample Representiveness

EVALUATION AND ANALYSIS OF RESEARCH SYNTHESIS Narrative Reviews of Research

(experimental and nonexperimental)

Quantitative Reviews of of Research (meta-analysis and generalized causal inference)

SUMMING UP: INTERPRETATION, PRESENTATION, AND DISSEMINATION

Omission of Details Omission of Evidence on

Moderators Telling the Story to the

Profession

RESEARCH TO PRACTICE: The Role of Practitioners in Evidence Based Interventions (EBIs)

What are the Big Issues?

Research to Practice (cont.) Issues surrounding the role

of practitioners in the development and use of evidence-based interventions (EBIs).

Issues related to the research-practice gap.

Next Step Agendas Review five assumptions that

guide the integration of EBIs into practice.

Suggest potential solutions to problems surrounding the adoption of EBIs in practice.

Five Assumptions Guiding the Integration of EBIs into Practice Settings The development and use of

EBIs in practice needs to be a shared responsibility (agenda) among researchers and practitioners.

Implementation of EBIs in practice may require development and use of both practice guidelines and expert consensus guidelines.

(Assumptions cont.) Offering professionals a menu

of EBIs is insufficient to promote application in practice. Manuals and guidelines must be accompanied by additional strategies for effective integration of EBIs into practice.

Assumptions (Cont.) To close the research-practice

gap we must facilitate professional development for faculty and practitioners.

(Assumptions cont.) To strengthen the connection

between research and practice we must promote and use a scientist-practitioner model in graduate training and professional work, especially when evaluating intervention outcomes.

Shared Responsibility for EBI Development Involvement of Practitioners

on the Task Force Participation in Practice

Research Networks Evaluation of EBIs in Practice

With Case Studies and Ethnographic/Qualitative Methods

Development of Guidelines Practice Guidelines Expert Consensus Guidelines

AACAP Practice Parameters for the Treatment of Children and Adolescents with Conduct Disorders

A. Treat comorbid disorders (e.g., ADHD, specific developmental disabilities, intermittent explosive disorder, affective or bipolar disorder, anxiety disorder, and substance use disorder).

(cont.)B. Family interventions included parent

guidance, training, and family therapy.1. Identify and work with parental strengths.2. Train parents to establish consistent positive

and negative consequences and well-defined expectations and rules. Work to eliminate harsh, excessively permissive, and inconsistent behavior management practices.

3. Arrange for treatment of parental psychopathology (i.e., substance abuse).

(cont.)C. Individual and group psychotherapy

with adolescent or child. Technique of intervention (supportive versus explorative; cognitive versus behavioral) depends on patient’s age, processing style, and ability to engage in treatment. Usually a combination of behavioral and explorative approaches is indicated, especially when there are internalizing and externalizing comorbidities.

(cont.)D. Psychosocial skill-building training should

supplement therapy.E. Other psychosocial interventions should be

considered as indicated.1. Peer intervention to discourage deviant peer

association and promote a socially appropriate peer network.

2. School intervention for appropriate placement, to promote an alliance between parents and school, and to promote prosocial peer group contact. Vocational training may be useful.

3. Juvenile justice system intervention, including court supervision and limit-setting, and special programs when available.

(cont.)

4. Social services referral, to help the family access benefits and service providers, (e.g., case managers).

5. Other community resources, such as Big Brothers and Big Sisters programs, Friends Outside, and Planned Parenthood, as indicated.

(cont.)6. Out-of-home placement (crisis shelters,

group homes, residential treatment)7. Job and independent-living skills training

(pp. 1335-1345).

Source: From Steiner, H. (1997). Practice parameters for the assessment and treatment of children and adolescents with conduct disorder. American Academy of child and Adolescent Psychiatry, 36(10 Suppl), 1225-1395.

Guidelines Must Be Enhanced Use Enhanced Guidelines that

focus on context Use Enhanced Guidelines that

focus on theoretical mechanism for intervention

Use Enhanced Guidelines that focus on theoretical mechanisms for change (at the individual, classroom, and system levels)

Facilitate Professional Development Focus on Graduate Programs

and Practitioners in Schools Adopt Competency-Based

Models of Education Increases Acceptability Promotes Integrity of

Interventions

Promote Evaluation of Intervention Outcomes Consistent with the Scientist-

Practitioner Model Facilitate the Intervention

Utility of Assessment

Specific Task Force Strategies and Tactics to Guide The Use of EBIs in Practice

Develop a Practice Research Network in School Psychology

Promote Effectiveness Studies

Develop and use Consumer Guidelines for practitioners when implementing interventions

(Strategies and Tactics cont.)

Use Enhanced Guidelines for Interventions

Dissemination and Education Efforts

Practice Research Network Practitioners engaged in the

evaluation of EBIs Example: Hawaii Empirical

Basis to Services Task Force[Clinical Psychology: Science and Practice, 9 (2) 2002]

Promoting Effectiveness Studies Efficacy vs. Effectiveness Effectiveness Studies Focus on

Generalization of Interventions Moving Beyond Effectiveness

to “Real Life Study Design” (Hohmann & Shear, 2002)

Practitioner Partnerships Practice Evaluation of EBIs Establishing the Data Base for

Practice

Example Dimensions to Consider in Use of Evidence-Based Intervention in School Psychology Practice

Dimension Considered in Practice1. Does your client appear similar to

those described in the EBI?2. Are you able to replicate the

intervention based on the description provided in the manual and/or procedures?

3. Are the conditions of implementation of the EBI similar to those of your setting?

(cont.)

4. Are the measures you used to assess outcome identical to those used to establish the EBI?

5. Were all the measures recommended in the EBI used to evaluate the intervention?

6. Was ongoing evaluation (repeated assessment) of student progress conducted?

(cont.)

7. Have new outcome measures been added to the intervention evaluation?

8. Did you use an intervention manual or protocol that specified the intervention?

9. Can individual characteristics of students be identified that are related to intervention outcomes?

(cont.)10. When group intervention data are

reported, is the percentage of individuals showing the effect reported?

11. When individual data are reported, have the data been presented in graphic form?

12. Have the EBI positive effects reported in research been replicated with your student(s)?

(cont.)

13. Have you replicated the EBI more than once?

14. Have others in your school setting replicated the EBI?

15. Would you rate the effects as strong as the original EBI effects?

16. Would you rate the effects as clinically meaningful?

(cont.)

17. Did you and/or staff find the EBI acceptable for use in your school?

18. Was the EBI cost-efficient for implementation in your school?

19. Are there specific contextual factors in your setting that could account for the success or failure of the EBI?

(cont.)

20. Do you plan to adopt the EBI for future implementation in your setting?

Source: Adapted from: Hayes, S.C., Barlow, D.H., & Nelson-Gray, R.O. (1999). The scientist practitioner: Research and accountability in the age of managed care.

Enhanced Guidelines for Interventions Understanding Basic Principles

of Change Understanding Indications and

Contraindications of EBIs Understanding Variability in

Intervention Implementation Problem Solving Strategies Evaluation of EBIs in Practice

Dissemination Efforts and Education Graduate Education Practice

Implications for Graduate Education Focus on Prevention/Intervention

Research Courses Focus on Prevention/Intervention

Content in the Curriculum Expand Faculty Through

Technology (no more renaissance faculty)

Implications for Practice Implications for Assessment Implications for Consultation Implications for Intervention

Implications for Practice: Assessment Emphasis on Screening to

Identify Students At-risk Emphasis on Assessment to

Plan Interventions Emphasis on Assessment to

Monitor Intervention Outcomes

Implications for Practice: Consultation Emphasis on Problem Solving Emphasis on Collaborative

Teaming Emphasis on Organizational

Development Consultation Emphasis on Professional

Development Activities

Implications for Practice: Intervention Emphasis on Prevention Emphasis on Multi-Level

Systems of Prevention Emphasis on Concomitant

Academic and Social/Emotional Intervention Focus

Emphasis on Intervention Responsiveness in Decision-Making

Summary and Conclusions This presentation provided an

overview of several guiding assumptions embraced by the Task Force on Evidence-Based Interventions in School Psychology.

Summary and Conclusions The assumptions can guide

specific strategies the school psychology community can use to move forward the agenda of integrating EBIs into graduate education and practice.

Summary and Conclusions The development of EBIs is a

shared scientific agenda among researchers and practitioners.

For Further Information, Contact:

Thomas R. Kratochwill, PhDSchool Psychology Program1025 West Johnson StreetUniversity of Wisconsin-MadisonMadison, Wisconsin 53706Phone: (608) 262-5912E-Mail: tomkat@education.wisc.edu