Final report: Preparation and writing

June 17, 2014

Housekeeping notes

• This webinar will be recorded, and will be available on the Centre’s website as an educational resource

• The slides have been sent to participants

• Log in with a phone whenever possible for optimal audio quality

• We have staff online to assist with any technical difficulties

• There will be a short evaluation survey for all webinar attendees at the completion of the webinar

Housekeeping notes

• This webinar will be followed by a question and answer period, however questions are encouraged throughout the presentation.

To submit questions electronically, use the question box located in your control panel.

To submit questions verbally, use the raised hand icon also located in the control panel.

About the Centre We bring people and knowledge together to strengthen

the quality and effectiveness of mental health services for children, youth and their families and caregivers.

Three strategic goals:

Learning Collaboration Leadership

Foster a culture of organizational learning to support agencies in using evidence to improve client outcomes.

Build and develop collaborative partnerships to sustain capacity within mental health services.

Be a true learning organization and lead by example.

Welcome and introductions

Today’s presenters

Ashlee Mulligan,

Knowledge Broker

Ontario Centre of Excellence for Child and Youth Mental Health

Chantal Lafleur,

Program Manager

Child and Family Centre

Goals of this webinar

1. To provide information on the Centre’s final report guidelines

2. To provide tips for preparing a clear and concise document

Preparing the final report

Why produce a final report?

o To guide future evaluation activities within the agency

o For stakeholders to be able to work with the findings

o For others to be able to replicate the project

o To orient new personnel to the program

o To have a communication piece that highlights the outcomes of the program

o To be accountable to funders

Source: Torres, Preskill, Piontek (2005)

Important deadlines for deliverables

Deliverables Deadlines

Draft report (optional) October 17, 2014

Final report October 31, 2014

Accounting summary of expenditures report

October 31, 2014

Evaluation questionnaires

Exit survey – October 31, 2014

Follow-up survey – March 31, 2015

Formatting requirements

o Calibri 11-pt font

o Double-spaced on 8.5 x 11 inch paper

o Minimum 1-inch margins on all sides

o Main body of report should be no more than 25 pages, not including references and appendices

Formatting requirements cont’d

1. Title page

2. Executive summary

3. Table of contents

4. Introduction and literature review

5. Methodology

6. Results, discussion & interpretation

7. Stakeholder involvement and knowledge exchange

8. Conclusion & action plan

9. Next steps in ensuring sustainability

10. References and appendices

Title page

Must include:

• A title for the project

• The name of the project lead and/or authors

• The name of the program and organization

• EPG or EDG number

• Date of submission

Project titles

“Beginning evaluation: A formal evaluation

of the child and youth crisis service”

“Intensive Community and Home Services

(ICHS): A preliminary evaluation, 2011”

“An evaluation of caring families: A holistic approach

for treating families in conflict”

X “Planning Evaluation Grant EPG # xxxx: Final Report”

Executive summary

Overview of the report that includes:

• name of the organization and project title

• the purpose of the project

• descriptions of agency and program

• a brief description of the evaluation methods

• a description of the results

• conclusions and recommendations

Executive summary template

Organization name: Program title

Project lead name, position title

Provide a 2- to 3-sentence summary of the project. It will be what appears in the list of records returned by a database search. This section should be written in the present tense.

The purpose

Should be in point form with a maximum of 3 points

Consider these starting points: To evaluate…To compare…To determine…To produce…To identify…To develop or To compare

What is the intended outcome of the evaluation? Is it to inform service planning and delivery? To determine best practices? To increase capacity for future and ongoing evaluation?

The program

Provide an overview of the agency’s work and a specific description of the program being evaluated. Who does the organization serve? Where is it located and what treatment approaches are used? What is the treatment setting? Where does the program evaluation fit in the bigger picture? (maximum 150 words)

Executive summary template

The plan

Provide a very brief description of the evaluation methods. If different from the above, include characteristics of the clients involved, sample size, intervention type, measures and treatment setting. What is being measured, how is it being measured and when is it being measured? (maximum 100 – 150 words)

The product

Provide a description of the results of the evaluation in relation to the purpose and the plan. Include information about the implications of the findings, next steps and knowledge exchange plans. (maximum 200 – 250 words)

Amount awarded: $XX,XXX.XX

Final report submitted: Month DD, YYYY

Region: MCYS region

Table of contents

• The table of contents provides the reader with a list of the various sections of the final report

• Relevant page numbers should accompany the main headings

Table of contents

Introduction and literature review

• Provide a clear description of the mental health program being evaluated, including:

o a description of the primary components or activities of the program

o target audience

o average number of clients per year

o how long it has been implemented in the organization

• Describe the extent to which the program is evidence-informed and how staff are trained to deliver the program

Introduction and literature review

• Include the logic model in this section

• Describe:

o the purpose of the evaluation

o intended evaluation goals

o specific process and outcome evaluation questions addressed

• Provide a review of related research

Methodology

Include a description of:

• participant characteristics

• sample size

• evaluation design (incl. type of intervention/treatment)

• ethical considerations

Methodology

Include:

• outcomes and indicators chart (a.k.a. evaluation matrix/framework)

• description of data collection methods (incl. instruments)

• description of data analysis plan

• description of evaluation limitations

Results

Present findings clearly and objectively for each of the evaluation questions:

o consider the use of headings

o mention the type of analysis performed, and if relevant, whether the result was statistically significant

Tables and charts can be very helpful for presenting the results of quantitative analyses, but be sure to use labels and descriptors

Discussion and interpretation

• In the discussion/interpretation, compare findings with those from the literature review and explain any differences

• Discuss any challenges encountered during the project and lessons learned

• Discuss the implications of the findings for the program, organization, stakeholders, and future evaluations

Stakeholder involvement and knowledge exchange

Describe:

• how information was shared with stakeholders during the evaluation

• knowledge exchange activities accomplished to date

• further plans regarding knowledge exchange activities

Sharing the findings

The final report and/or a summary of the report can be shared widely with various stakeholders including:

o program and agency staff

o members of the Board

o community partners

o clients and their families

o community members

o community of practice

o funders

Sharing the findings

Traditional Innovative

Presentations Conferences Symposiums Workshops

Bulletins Newsletters

Posters

Infographics Games

Graphic novels Videos

Theatrical presentations

Photovoice World Café

Conclusion and action plan

• Provide a conclusion and action items that reflect the findings

• Identify one to three main messages you would like to convey to others about your experience in developing the evaluation framework and doing the pilot evaluation

• Describe concrete next steps for the evaluation

Next steps in ensuring sustainability

• Describe how your program’s/agency’s evaluation capacity has grown over the last year and how you concretely plan on sustaining or further developing that capacity after the end of the grant

References and appendices

Cite references using APA style, in alphabetical order

Examples of various sources in APA style can be found here: http://www.michener.ca/lrc/lrcapa.php

Appendices • Additional information on the evaluation

framework

• Copies of publicly available measures

• Newly developed measures, interview guides or focus group discussion guides

• More in-depth information on the program or more additional results (tables or figures)

• Copies of materials used for knowledge exchange (e.g. presentation, newsletter article)

Questions?

Questions or comments?

• To submit questions electronically, use the question box located in your control panel

• To submit questions verbally, use the raised hand icon also located in the control panel

Contact

For more information on this webinar or topic, please contact:

Ashlee Mulligan

Knowledge Broker

Ontario Centre of Excellence for Child and Youth Mental Health

amulligan@cheo.on.ca

Evaluation

Please complete the survey at the end of this webinar.

Your feedback is very important to us, so we thank you for taking the time to share your thoughts!