The Generation and Use of The Generation and Use of Evidence in Child WelfareEvidence in Child Welfare

Aron ShlonskyAron ShlonskyVisiting Scholar, Haruv InstituteVisiting Scholar, Haruv Institute

Associate ProfessorAssociate ProfessorFactor-Inwentash Chair in Child WelfareFactor-Inwentash Chair in Child Welfare

University of TorontoUniversity of TorontoFactor-Inwentash Faculty of Social WorkFactor-Inwentash Faculty of Social Work

Evidence-Informed Social Work Is a Evidence-Informed Social Work Is a Concept Awaiting ImplementationConcept Awaiting Implementation

Much discussedMuch discussed

Required in most Required in most jurisdictionsjurisdictions

Not clearly DefinedNot clearly Defined

Not yet implemented Not yet implemented in pure formin pure form

“--- the conscientious, explicit and judicious use of current best evidence

in making decisions regarding the welfare of

service-users and carers” (Sheldon, 2003, p. 1).

The Need for EBP in Social The Need for EBP in Social WorkWork

Overwhelming and Overwhelming and unreliable sources of unreliable sources of informationinformationChanging state of Changing state of knowledgeknowledgePotential to cause harmPotential to cause harmOpportunity costsOpportunity costsSocial justice / Social justice / informed choiceinformed choiceSystematic rubricSystematic rubric

QuickTime™ and a decompressor

are needed to see this picture.

What is EIP?

EBP, EBM, EIP, EST’s, EBP’s, EBP, EBM, EIP, EST’s, EBP’s, PG’s, and BP’s – huh?PG’s, and BP’s – huh?

Evidence-based Practice (EBP) is based on Evidence-based Practice (EBP) is based on Evidence-based Medicine (EBM) Evidence-based Medicine (EBM) Sackett et al (1997) at McMasterSackett et al (1997) at McMasterEvidence-informed practiceEvidence-informed practice– Describes EBP and EBMDescribes EBP and EBMEmpirically Supported TreatmentsEmpirically Supported Treatments (EST’s or (EST’s or EBP’s) EBP’s) – – Use of interventions with evidence of Use of interventions with evidence of their efficacy and/or effectivenesstheir efficacy and/or effectivenessPractice guidelines (PG’s) andPractice guidelines (PG’s) andBest Practices (BP’s) are lists of Best Practices (BP’s) are lists of (hopefully) validated interventions(hopefully) validated interventions

Clinical state and

circumstances

Client Preferences and

actions

Current Best

Evidence

Clinical Expertise

Haynes, Devereaux, and Guyatt (2002)

The EBP Model: A The EBP Model: A Philosophical and Practical Philosophical and Practical

ApproachApproach

Steps of EBP (based on Sackett et al., 2000)Steps of EBP (based on Sackett et al., 2000)

Become Motivated to Apply EBPBecome Motivated to Apply EBP1.1. Convert information need (prevention, assessment, treatment, Convert information need (prevention, assessment, treatment,

risk) into an answerable question.risk) into an answerable question.2.2. Track down the best evidence to answer the question.Track down the best evidence to answer the question.3.3. Critically appraise the evidence for its validity (closeness to the Critically appraise the evidence for its validity (closeness to the

truth) impact (size of the effect) and applicability (usefulness in truth) impact (size of the effect) and applicability (usefulness in our practice).our practice).

4.4. Integrate critical appraisal with our practice experience, client’s Integrate critical appraisal with our practice experience, client’s strengths, values circumstances.strengths, values circumstances.

5.5. Evaluate effectiveness and efficiency in exercising steps 1-4 and Evaluate effectiveness and efficiency in exercising steps 1-4 and seek ways to improve them next time.seek ways to improve them next time.

6.6. Teach others to follow the same processTeach others to follow the same process

Gibbs, 2003

Challenges to EBPChallenges to EBPPhilosophical straw menPhilosophical straw menLack of evidenceLack of evidenceAccess to evidenceAccess to evidenceComplexity of social servicesComplexity of social servicesLack of timeLack of timeTranslation across different contextsTranslation across different contextsDecision-making in an uncertain worldDecision-making in an uncertain worldCognitive dissonanceCognitive dissonanceNo manual for integrating evidence with contextNo manual for integrating evidence with context

Organizational Mandate

Organizational Resources / Constraints

Com

mun

ity

Trai

ning

/ S

uper

visi

on

Political ContextS

ocio

/ H

isto

rical

Con

text

Economic Context

Pro

fess

iona

l Con

textand

Client Preferences and Actions

ResearchEvidence

Expertise

and Client State

and Circumstances

Client Preferences

Client Preferences Research

EvidenceResearchEvidence

ExpertiseProfessional

Regehr, Stern & Shlonsky (2007)

Evidence in NOT DeterministicEvidence in NOT Deterministic

Evidence-informedEvidence-informedSystematic processSystematic processInclusiveInclusiveCriticalCriticalFlexibleFlexible

DNO

DETERMINISM

Tensions Within the North Tensions Within the North American Child Welfare SystemAmerican Child Welfare SystemChild welfare or child protection?Child welfare or child protection?Child welfare worker - friend or foe?Child welfare worker - friend or foe?System - should it be residual or System - should it be residual or preventive?preventive?How do you balance safety, culture, How do you balance safety, culture, community, family rights, and the individual community, family rights, and the individual needs of children?needs of children?How do you assess the risk the system How do you assess the risk the system poses to the child?poses to the child?

Decision Points in Child WelfareDecision Points in Child WelfareReport / ScreeningReport / Screening

investigateinvestigateInvestigationInvestigation

VerifyVerifyclosecloseOpen for ongoingOpen for ongoing

Ongoing Services (in-home)Ongoing Services (in-home)Close CaseClose CaseOut-of-home careOut-of-home care

Ongoing Services (out-of-home)Ongoing Services (out-of-home)ReunificationReunificationLTFCLTFCGuardianshipGuardianshipAdoptionAdoption

Risk / Prognostic Tools

Diagnostic Tools

Effective Treatment

Monitoring

Evaluation

Proper blend: Clinical Skills and Proper blend: Clinical Skills and EvidenceEvidence

Requires a structure that Requires a structure that teaches clinical skills and teaches clinical skills and constantly creates and constantly creates and evaluates evidence.evaluates evidence.Ongoing struggleOngoing struggleResponsibility to both find Responsibility to both find and evaluate evidenceand evaluate evidenceFind ways to integrateFind ways to integrateNo set recipeNo set recipe

Types of Evidence in CWTypes of Evidence in CW

Clinical ExperienceClinical ExperienceResearch literatureResearch literatureAdministrative DataAdministrative DataSurvey Survey EvaluationEvaluation– QuantitativeQuantitative– QualitativeQualitative

The The CycleCycle of EBP of EBP

Clinical state and

circumstances

Client Preferences and

actions

Current Best

Evidence

Clinical Expertise

Actuarial risk

assessment

Contextual assessmentContextual

assessment

Other valid assessment measures?

Effective services

Appropriate for this client?

Barriers (e.g., cultural conflict)

Client’s preferred course or at least

willing to try? Adapated from Haynes, Devereaux, and

Guyatt, 2002; as presented in Shlonsky and Wagner, 2005.

Campbell Collaboration Campbell Collaboration EvolutionEvolution

Systematic Reviews of EvidenceSystematic Reviews of EvidenceImportance of questionImportance of questionImportance of non-RCT’sImportance of non-RCT’s– EconometricEconometric– ObservationalObservational– Risk/diagnosisRisk/diagnosisIncreasing number of reviewsIncreasing number of reviewsEquity GroupEquity GroupUser AbstractsUser Abstracts– TranslationTranslation

http://www.cebc4cw.org/scientific-rating/scale

California Clearinghouse Scientific California Clearinghouse Scientific Ratings of 1Ratings of 1

• Cognitive Therapy• Coping Cat• Coping Power• Coping with Depression for Adolescents• Interpersonal Psychotherapy (Adults & Children of Depressed Adults)• Mindfulness Based Cognitive Therapy (Adults)• Motivational Interviewing (MI) • Multidimensional Treatment Foster Care - Adolescents (MTFC-A) • Multisystemic Therapy??• Nurse-Family Partnership (NFP) • Parent-Child Interaction Therapy (PCIT) • The Incredible Years • Trauma-Focused Cognitive Behavioral Therapy (TF-CBT)• Triple P-Positive Parenting Program

Slide courtesy of Richard P. Barth

Critical Thinking is KeyCritical Thinking is KeyEight Critical Thinking Guidelines

Ask Questions: Be willing to wonder. Define Your Terms: Key to Research Examine the Evidence Analyze Assumptions and Biases Avoid Emotional Reasoning Don’t Oversimplify Consider Other Interpretations Tolerate Uncertainty

http://cla.calpoly.edu/~cslem/Invit/1/Chp1c.html

QuickTime™ and a decompressor

are needed to see this picture.

A Well Cultivated Critical ThinkerRaises vital questions and problems;

Formulates them clearly andprecisely;

Gathers and assesses relevant information

Uses abstract ideas tointerpret information;

Comes to well-reasoned conclusions and solutions;

Tests them against relevant criteria and standards;

Thinks openmindedly;

Recognizes and assesses assumptions, implications, and practical consequences;

Communicates effectivelyElder, 2007

Knowledge needs to be managed much more Knowledge needs to be managed much more effectively than it has been in the pasteffectively than it has been in the past

The challenge is to rethink our view of what knowledge is & how best

to facilitate its rapid generation, sharing, and application in a manner that closes the policy-research-practice gap

DiscussionDiscussion