knowing what works and actually doing it

Knowing What Works and Knowing What Works and Actually Doing ItActually Doing It

Presentation to the Indiana Center for Evidence-Based Practices Learning Institute,

September 21, 2010Kristy Pierce – Danford, MPA, Crime and Justice Institute

Crime and Justice Institute (CJI) at Community Crime and Justice Institute (CJI) at Community Resources for JusticeResources for Justice

About UsAbout Us

• Community Resources for Justice– Direct services

• Residential, reentry and day programming for adult and youth offenders, at-risk youth, and adults with mental illness and developmental disabilities

• Crime and Justice Institute– Nonpartisan consulting

• Policy analysis, research services, and capacity building technical assistance to improve public safety systems throughout the country.

2September 21, 2010www.cjinstitute.org

http://www.cjinstitute.org/

AgendaAgenda

• Purpose• Integrated Model• Transformation Challenges• Assessing the Organization• Strategic Planning and Work Plan Development• Assuring Quality and Fidelity• Managing Change• Stakeholder Collaboration• Take Away Points• Discussion



Toward Implementing EBPToward Implementing EBP

•Clearly identify the problem and desired outcomes

•Search for the research (i.e., evidence) that may help address the problem

•Critically evaluate the evidence

•Assess the extent to which your current practices are consistent with identified evidence-based practices

•Develop an implementation strategy and put it to use

•Align business practices to support implementation

•Evaluate the impact of new practices on the desired outcomes

September 21, 2010 www.cjinstitute.org 4


Some of the most commonly cited reasons:

• We are focusing on the wrong issues

• We are giving too much attention to the low risk and too little to the high risk

• Programs have not applied research knowledge nor are these practices applied with fidelity

• The system is not in alignment

Why Are Recidivism Rates Higher Than Desired?Why Are Recidivism Rates Higher Than Desired?



1. Organizational development (ineffective technology transfer strategies)

2. System that is unable or unwilling to practice true collaboration

3. Failure to measure, reinforce, and hold everyone accountable for EBP

4. Ineffective leadership

5. Fidelity, fidelity, fidelity

The Top Five Reasons Why EBP’s FailThe Top Five Reasons Why EBP’s Fail

6September 21, 2010 www.cjinstitute.org


CJI/NIC Integrated Model for the Implementation CJI/NIC Integrated Model for the Implementation of Evidence Based Policy and Practiceof Evidence Based Policy and Practice

7

Created by CJI through a cooperative agreement

with NIC in 2002

Visit www.cjinstitute.org/projects/integratedmodel September 21, 2010 www.cjinstitute.org

http://www.cjinstitute.org/projects/integratedmodel


Transformation ChallengesTransformation Challenges

• Establishing proficiency in assessment and case planning

• Applying motivational interviewing and cognitive behavioral skills that target criminogenic needs

• Maintaining fidelity• Aligning organizational practices and budgets• Managing change• Getting on the same page• Being patient, having a plan, and possessing the

perseverance to carry it out8September 21, 2010 www.cjinstitute.org


Play ListPlay List• “I think I went to a training on that once.”• “The assessment is somewhere in the file.”• Doing MI = EBP• “I hate this QA stuff. It’s too much paperwork and way

too invasive.”• “All they care about is if I meet standards.”• “My boss says one thing and the judge says another.”• “Just make your numbers look good, no one cares

about the story behind the numbers.”• “Why isn’t it happening; it’s in the policy?”



Evidence-Based OrganizationsEvidence-Based Organizations

• In a Correctional EBO…– Everyone shares a common mission and

vision– Resources are used effectively and efficiently– Offenders are held accountable– Data drives decisions– Learning and innovations are welcome– System players communicate and collaborate



Organizational DevelopmentOrganizational Development

• Prepares an organization to:– Achieve its mission efficiently and effectively– Respond to change– Sustain itself over the long term

• Encourages a systems perspective

• Applies behavior change principles to organizations as well as individuals

• Improves organizational performance

September 21, 2010 11www.cjinstitute.org


Organizational Case ManagementOrganizational Case Management

Assessment

Intervention

Monitoring & Measurement Feedback



AssessmentAssessment

• Determines the existing status of an individual, organization, and/or practice by providing information on the potential and options for change.

• Assessment strategies include:– Surveys/Interviews– Observation– Data Review and Analysis



Organizational ResponsivenessOrganizational Responsiveness

• Assessing the Organization– Organizational Assessment

• Conducting Self-Assessment• Selecting Assessment Approaches• Administering Assessments• Analyzing Assessments• Applying Assessment Results• Reassessment



Plan for Results and IntervenePlan for Results and Intervene

• Strategic Planning and Work Plan Development– Developing Strategic Plans and Work

Plans• The Strategic Planning Process• The Work Planning Process



Thoughtful Plans and ActionsThoughtful Plans and Actions

• Intervention activities are designed to respond to the needs/issues identified in the assessment

• Put together a comprehensive plan that can be put into action and communicated

• Revisit Mission, Vision and Values • SMART goals, clear objectives, specific

timelines and accountability• Charter committees to avoid scope creep• From Strategy to Tactics• Revisit, update and revise



Monitoring & Measurement FeedbackMonitoring & Measurement Feedback

• Monitoring and measuring performance on both a short and long-term basis provide data on changes in knowledge, skills, attitudes, and behavior.

• Types of measures include:– Process measures:

• Provide feedback throughout change process.

– Outcome measures:• Individual: Measure actual change in knowledge, skills,

attitudes, and/or behavior.

• Organizational: Measure improvement and progress toward goals.



Fidelity MattersFidelity Matters

• Assuring Quality and Fidelity– What is Quality Assurance and Why

Does it Matter?

• Things to Keep in Mind When Creating a Quality Assurance Plan– Data overload, organizational change,

resource limitations, timing, keep it simple and focused

• Celebrate Successes18September 21, 2010 www.cjinstitute.org


Staying TrueStaying True• Whatever you do, do no harm• Risk: Do you match supervision and services with

risk level?• Need: Do you focus on criminogenic needs?• Treatment: Do you utilize cognitive behavioral

techniques?• Responsivity: Are you responsive to the

characteristics of individuals?• Fidelity: Are you doing evidence-based work? Are

you doing it well? Is it leading to desired outcomes?


10 Steps to Assure Fidelity10 Steps to Assure Fidelity1. Define the outcome(s) you are trying to achieve.

2. Logic Models –Describe what is intended to happen by connecting your actions to your outcomes.

3. Determine what indicators will need to be measured. Prioritize these measures based on what you need to know first and what you have the resources to collect.

4. Decide how to measure the indicators. Plan for how to collect the data, by whom and how often.

5. Pull it together. Develop a plan that describes how these measures will be brought together.NOTE: If you find there are still too many measures on which to realistically collect data, do another round of prioritization.

September 21, 2010www.cjinstitute.org

20

10 Steps to Assure Fidelity10 Steps to Assure Fidelity

6. Communicate the plan.

7. Collect the data.

8. Analyze and report the data.

9. Put the data to use.

10. Continuously improve until you are satisfied with the outcome and then move onto the next desired outcome and repeat.


21

Change is HardChange is Hard

• Managing Change–Techniques for Facilitating Change

–Leadership

–Communication

–Aligning Business Practices


Leadership At All LevelsLeadership At All Levels

• Executives create, communicate, and sustain a vision for the organization.

• Senior and mid-level managers translate mission and vision into practice.

• Frontline supervisors ensure implementation with fidelity to mission, vision, and policy.

• Frontline employees disseminate evidence-based approaches to colleagues, clients, and the community.


Effective Leaders at All LevelsEffective Leaders at All Levels

• Know how to LEAD and to MANAGE• Are committed to vision and mission• Are willing to learn and change• Possess the integrity to see what is wrong with

the system & the courage to commandeer change

• Understand organizational change and how to manage it

• Will engage in system reform over the long term


Effective Leaders at All LevelsEffective Leaders at All Levels• Use data to guide decision making

• Disseminate information effectively

• Are flexible in response to changing data and changing conditions

• Effectively build teams

• Know when and how to involve others in decisions

• Address internal and external conditions affecting organizational policy, practice, and overall health


Policies and ProceduresPolicies and Procedures

• This is where the rubber meets the road

• Review existing policies and procedures for alignment with EBP and OD principles, and include needed revisions as part of your implementation plan.


Workforce DevelopmentWorkforce Development• Ultimately, it’s the employees that determine

whether system reform and culture change are successful.

• Expectations of the workforce change with reform, and the competencies needed to do the job need be well defined.

• To avoid mixed messages, new competencies should be infused throughout workforce processes.

– Recruitment and selection --Training

– Performance appraisals -- Promotions September 21, 2010 27www.cjinstitute.org

CommunicationCommunication• You can never have too much!• Work with a “diagonal slice” of employees

to determine what needs to be communicated, and how to do it.

• Consider the various methods of communication available: newsletters, emails, videos, blogs, etc.

• Prioritize your messages—what does everyone need to know?

• A written communication plan is a helpful tool.


What it Takes to ChangeWhat it Takes to Change• Courage to make changes and take risks• Refusal to “muddle through”• Willingness to:

– Gain knowledge and deal with the complexities– Take on “Sacred Cows”– Use knowledge based on solid research

• Make decisions based on evidence and data• Make a long-term commitment to reform• Master skill sets and deliver evidence-based

interventions • Curtail use of costly, ineffective resources –

“reallocate”29September 21, 2010

www.cjinstitute.org

Don’t Go It AloneDon’t Go It Alone

• Stakeholder Collaboration– Why Collaborate?

– What is Collaboration?

– Misperceptions of Collaboration

– Tips for Collaboration

– Who Should be Included?

– How are Structure and Expectations Set?


Tangible Outcomes of Collaborative Teams

Increased knowledge &

understanding,- better decisions -greater political

support.

Resource sharing

rather thancompetition.

Effective system change.

Example: Redeployment

of monies to support a

needed project

Example: Reallocation

of jail bedspace.

Example: Full team press

conference.31

Keys to EBO SuccessKeys to EBO Success

• Give your organization a physical

• Create and implement a strategic plan

• Build the capacity of leaders at all levels

• Align all business practices

• Develop a continuum to address risk, need, and responsivity with fidelity

• Use data to drive decisions

• Communicate, communicate, communicate


Pace YourselvesPace Yourselves

• Plan, plan, plan

• Value living documents

• Ensure reforms fit your agency and its needs

• Be clear, focused and flexible

• Learn from each other

• Expect to make refinements in practice based on evidence

• Ensure quick wins

• Celebrate33September 21, 2010

www.cjinstitute.org

www.cjinstitute.orgwww.cjinstitute.orgOr contact:Elyse Clawson, Executive [email protected]

Kristy Danford, Project [email protected]

34

For more information:For more information:

How do you take an EBP and make it work? How do you make the transformation to an EBO?


knowing what works and actually doing it

Documents