Health Santé Canada Canada

The Fundamentals of Evaluating Prevention Programs

Colleen Ryan,Canada’s Drug Strategy Secretariat, Health Canada

2 Health Santé Canada Canada

Presentation Overview

• Why Evaluate?

• Why Don’t We Evaluate?

• Role of Evaluation and the MEM

• Challenges in the Field of Prevention

• Basic Steps in Evaluation and What Will Be Accomplished in This Meeting

3 Health Santé Canada Canada

Why Evaluate?

Decision-making and overall good management of initiatives

Program/project improvement

Understanding what work’s and what doesn’t

Tracking unanticipated outcomes

Overall accountability for investment

Public relations/fundraising

4 Health Santé Canada Canada

Why Don’t We Evaluate?

Time

Financial Resources

Expertise

We already “know” the program is effective

Philosophy

Long-term change vs. short-term funds

Fear

5 Health Santé Canada Canada

Evaluation and the MEM

The MEM as an important instrument of Evaluation

22 recommendations issued in the 2nd Round Evaluation

?? recommendations issued in the 3rd Round Evaluation

Capacity issues (financial and technical) identified as primary reason for lack of progress to date

6 Health Santé Canada Canada

Challenges in the Field of Prevention

Need to establish “perceived” risk for substance use/abuse at the outset

Difficult to measure longer-term impacts

Costly to track cohorts over a prolonged period of time

Challenges in establishing valid control or comparison groups

7 Health Santé Canada Canada

Basic Steps in Evaluation

Step 1: Identify goals (anticipated outcomes) – Logic Model

Step 2: Describe the project/program – Logic Model

Step 3: Identify the evaluation questions you want answered – Logic Model and Performance Indicators

Step 4: Identify data sources and data collection tools - Performance Indicators

Step 5: Collect the information (focus of next meeting)

Step 6: Organize the information (focus of next meeting)

Step 7: Analyze the data (focus of next meeting)

Step 8: Report the results – reassess goals and program content (focus of next meeting)