Final Evaluation Report2013

This evaluation was done to ensure accountability for

its delivery. Accountability includes meeting program

objectives and evaluating the effectiveness of the

program. One main purpose of accountability

measures is to identify successes and failures of the

performance of a program.

Purpose

The findings of this evaluation can be used to: Strengthen relationships; Strengthen the project design and

delivery; Maintain a record of the project’s

progress; and, report on the project’s outcomes to other stakeholders

Evaluation Findings:

16 Ready for School sessions ran from April

2012- August 2013

9 schools (locations)

Number of parent/caregivers = 239

Number of children = 217

Number of siblings = 100

Introduction/Statistics:

Data Sources:The data for analysis in this study comes from quantitative and qualitative methods:

202 Pre/Post Evaluation reports (completed by parents/caregivers who participated in the two-week sessional of the program) 85% Response rate.

16 Parent Focus Group Surveys (average of 12 parents in each group)

10 Partner Surveys 3 Staff Focus Group Survey (19 staff members in total)

Methodology:

Pre/Post Parent Evaluation survey results:

What affect does RfSC activities have on the parents’/caregivers’ perceptions of:

a) their children’s behaviour (level of independence in completing a task) associated with school readiness, and

b) their own knowledge of existing support systems (Canadian School System).

Methodology: Quantitative Data

Pre/Post Parent Evaluation survey (Part 1):

Methodology: Quantitative Data

Separates from a parent without getting upset  

1 2 3 4 5 6 7 8 9 10

Follows simple rules and directions 

1 2 3 4 5 6 7 8 9 10

Seeks out and maintains ongoing social interaction with at least one other child

1 2 3 4 5 6 7 8 9 10

Please circle the child’s current level of independence in completing the following tasks:Evaluation Scale: 1 - Adult does for child ------------------------------- 10 - child does on their own

Importance of parent involvement/ways to get involved in your child’s school

1 2 3 4 5 6 7 8 9 10

Kindergarten Curriculum 1 2 3 4 5 6 7 8 9 10

Getting support for your child so he/she can succeed in school (eg. Special needs, settlement worker, ESL, etc)

1 2 3 4 5 6 7 8 9 10

Methodology: Quantitative Data

Please circle your current level of knowledge on the Canadian School System:Evaluation Scale: 1 – I don’t know anything about this topic ----------- 10 – Very Knowledgeable

Pre/Post Parent Evaluation survey (Part 2):

Evaluation Process: Values were assembled in a data spreadsheet. Comparison was performed to measure the difference

between the pre and post program responses. Simple statistical operations like average and

percentage were used.

For example, for the task: Separates from parent without getting upset if:Pre-evaluation = 5 Post evaluation = 10Difference = 50% improvement in this behaviour

Methodology: Quantitative Data

Qualitative Data Tools:

◦Parent/Caregiver Focus Groups (202);

◦Community Partners Questionnaire (10)

◦Staff Questionnaire/Focus Group (19)

◦Follow-up Phone calls (50)

Methodology: Qualitative Data

Parent Focus Group Questions:

◦ 6 questions in the areas of:

Location/hours

Program Staff

Parent Workshops

Children’s program

Changes in child’s behaviour over the 2 week session

Parent/Caregiver connections

Methodology: Qualitative Data

Community Partners Questionnaire:

◦ 6 questions in the area of:

Overall experience with RfSC

Communication between RfSC and your

agency/school

Feedback from parents

Challenges experienced as a partner

Improvements for next year

Dream/vision for RfSC?

Methodology: Qualitative Data

Staff Questionnaire and Focus Group:

Questions in the area of:

Suggestions for next year with respect to training

Most important thing children learned

Suggestions to increase independence,

communication and social skills

Most useful part of the program for parents

Experience with our community partners

Dream for Ready for School Connects

Methodology: Qualitative Data

Evaluation process:

◦ All surveys/questionnaires were analyzed.

◦ Emerging trends were established.

◦ Trends were interpreted to measure the

effectiveness of the RfSC program and its

activities.

Methodology: Qualitative Data

Quantitative Data: Pre/Post Parent Evaluation Surveys:

Impact on Children:

◦ An overall average of 79% of parent/caregivers reported an

increase in level of independence after participating in RfSC

activities.

◦ Highest increase was 95% which was found at Oakridge.

◦ Lowest overall average was 53% increase which was found at

Crescent Town Session 4.

◦ Subtask: Separating from Parents - parents reported seeing the

most improvement in their children.

Findings

School: Separating Follows rules and directions

Social Interaction

Oakridge 100% 92% 92%

Carlton V. 100% 87% 94%

Danforth G. 100% 77% 77%

Pauline 75% 81% 81%

Sprucecourt 1 90% 50% 50%

Sprucecourt 2 100% 84% 92%

Secord 1 93% 64% 86%

Secord 2 92% 92% 92%

Findings:Percentage of parents reporting increase in level of independence:

Continued next page->

School: Separating Follows rules and directions

Social Interaction

Rose 1 82% 64% 55%

Rose 2 70% 70% 60%

Crescent T 1 93% 80% 53%

Crescent T 2 100% 71% 71%

Crescent T 3 86% 72% 64%

Crescent T 4 58% 58% 42%

George W 1 100% 92% 54%

George W 2 100% 71% 71%

Findings: (Con’d) Percentage of parents reporting increase in level of independence:

Findings

Sepa

rate

s fro

m P

aren

t

Follo

ws ru

les

Social

inte

ract

ion

0

10

20

Overall Average % Increase in Level of Independence

Average % Increase

23%

16%

11%

Findings

Oakrid

ge D G

Spru

ceco

urt 1

Seco

rd 1

Rose

1CT

1CT3

GW1

0

2

4

6

8

10

12

Pre SurveyPost Survey

Findings:Change in level of independence: Separating from Parents

Quantitative Data: Pre/Post Parent Evaluation Surveys: Impact on Parent/caregiver:

◦ 78% of parents/caregivers reported an increase in

knowledge in Canadian School System after participating in

RfSC workshops.

Highest average increase - 96% (Pauline PS)

Lowest average increase - 55% (Rose Ave session 1)

Subtask: Importance of parent involvement – reported as most

improved – 25%

Findings:

FindingsParents/caregivers reported an increase in their knowledge of the Canadian School System at the end of the two-week RfSC session.

20.522

23.525

Overall Average % Increase in Parent/Caregiver Knowledge of Canadian Ed-

ucation System

Average % Increase

25%

24%

22%

Findings

Oakrid

ge CVDG

Paul

ine

Spru

ceco

urt 1

Spru

ceco

urt 2

Seco

rd 1

Seco

rd 2

Rose

1

Rose

2CT1 CT2 CT3 CT4

GW1GW

20

2

4

6

8

10

12

PrePost

FindingsChange in level of knowledge of Importance of Parent Involvement

Overall RfSC Program: Better outreach/promotion plan:

◦ provide the school with a detailed information package about RfSC.

◦ hold an orientation session for all stakeholders (School secretary, Vice-Principal, EY/Family & Parenting centers staff, Daycare staff, etc.) to attend.

Recommendations

Children’s Programming: Staff recommended taking the time to establish rapport

with community partners especially when it involves sharing a common space.

discuss expectations, and possibly draw up a mutually fulfilling contract that all parties can agree upon.

i.e.. Availability of indoor space and outdoor space, room setup, use of equipment/materials, etc. Make sure both parties are communicating and checking in regularly with one another to ensure expectations are being met.

Recommendations:

Parent/Caregiver Workshops Both workshop facilitators and

parents/caregivers have reported that the time allotted for the workshops is not sufficient for the amount of information that needs to be covered.

Presenters should ask participants prior to presenting what they would like to know. By customizing the presentation to the needs and interests of the participants,

allow them plenty of time to ask questions.

Recommendations

Parents prefer presentation style that is interactive, stimulating, lots of visual materials

Standardize presentation workshops to provide all presenters at all locations with an outline/template of an effective structure to delivering an interactive workshop

Recommendations

Program Evaluation improvements: Staff members can discontinue completing

the Pre/post Evaluation form for children due to the difference in perspective being negligible.

Recommendations

Kindergarten Teacher Post-Program Survey: This evaluation tool was introduced this year in

order to provide additional follow up information from the perspective of the children’s kindergarten teacher they were placed in September.

tool was not implemented uniformly across all locations and it was quite challenging to get back completed surveys from the kindergarten teacher.

strategies/methods in obtaining this critical information should be investigated further.

Recommendations

Partner Survey: Tool could undergo a transformation:

◦ Include an area that identifies the respondent’s role (i.e.. Principal, workshop facilitator, OEYC Manager, teacher, etc.)

◦ Should be reviewed from the perspective of different respondents to see which questions are applicable to which role.

Recommendations (Lastly…)

◦Data analyzed was from 44 families (four school locations: Danforth Gardens, Oakridge, Carlton Village, and Pauline)

◦78% response rate◦92% of parents reported having no concerns

with their child’s school, teacher or classmates.◦The most important things families learned:

Canadian School System School Expectations and Routines Nutrition Dealing with Separation Behaviour Management

Follow – up check-ins with Families:

◦70 % of parents accessed recommended services or resources.

◦66% of families visited their local library.◦68% of families were connected to either local

recreational programs at community centers or participated in OEYC.

◦52% of parents still see/interact with other families from the program.

Follow – up findings: Cont’d

Utilize standardized questions that are asked at all locations. Ensure the same questions are being asked at every location.

Ensure questions are being responded to correctly and completely. When asking parents the questions, read each question carefully, record the response, then reread the question to make the question was responded to correctly. i.e.. How is your child adjusting? Were you able to access any of the services or resources that were recommended to you this summer? Which ones?

Explore ways to increase family participation in accessing resources that were recommended to them in the summer program.

Recommendations for future follow-up work: