sparking life niagara - niagararegion.ca · the sparking life niagara project was based on the...

43
Public Health Ontario | Niagara Region Public Health SPARKING LIFE NIAGARA Final Program Evaluation Report 2011 - 2012

Upload: phungdat

Post on 12-Sep-2018

222 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Niagara Region Public Health

SPARKING LIFE NIAGARA Final Program Evaluation Report

2011 - 2012

Page 2: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

1 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

TABLE OF CONTENTS 1.0 Executive Summary…………………………………………………………………………………………………2-3

2.0 Introduction…………………………………………………………………………………………………………….4-7

2.1 Background and Context……………………………………………………………………………………………..4-5

2.2 Program Need…………………………………………………………………………………………………………….5-6

2.3 Program Description……………………………………………………………………………………………….....6-7

3.0 Evaluation Overview……………………………………………………………………………………………….7-9

3.1 Evaluation Sponsor and Stakeholders…….………………………………………………………………………7

3.2 Evaluation Design…………………………………………………………………………………………………………..7

3.3 Evaluation Objectives…………………………..………………………………………………………………………..8

3.4 Evaluation Questions…………………………………………………………………………………………………..8-9

4.0 Methods………………………………………………………………………………………………………………..9-13

4.1 Participants……………………………………………………………………………………………………………….9-10

4.2 Measurement Tools………………………………………………………………………………………………..11-12

4.3 Data Analysis…………………………………………………………………………………………………………..12-13

5.0 Results…………………………………………………………………………………………………………………13-29

5.1 Academic Achievement….……………………………………………………………………………………….13-19

5.2 Emotional and Mental Health Outcomes………………………………………………………………..19-20

5.3 Physical Health and Fitness Outcomes…………………………………………………………………….20-26

5.4 Program Design & Adherence…………………………………………………………………………………26-29

6.0 Limitations and Recommendations………………………………………………………………………29-32

6.1 Program Implementation……………………………………………………………………………………….29-31

6.2 Evaluation……………………………………………………………………………………………………………….31-32

7.0 References.............................................................................................................33-34

8.0 Appendices………………………………………………………………………………………………………….35-42

8.1 Appendix A – Student Pre Questionnaire………………………………………………………………..35-36

8.2 Appendix B – Student Mid-Point Questionnaire…………………………………………………………..37

8.3 Appendix C – Student Post Questionnaire……………………………………………………………….38-40

8.4 Appendix D – Parent Telephone Interview………………………………………………………………40-42

Page 3: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 2

1.0 EXECUTIVE SUMMARY Introduction

The Sparking Life Niagara (SLN) program was piloted at two high schools in the Niagara Region

during the 2011 to 2012 academic year. Students participating in the program performed

aerobic activity for a minimum of 20 minutes between 65 to 85% of their maximum heart rate

on school days.

This program was based on research conducted by Dr. Ratey, a professor at Harvard University,

who conveys the ability of physical activity to improve academic performance, mood,

concentration, and learning. Particularly for youth and school-aged children, noted benefits of

physical activity include the prevention of obesity, high blood pressure and type II diabetes,

improved musculoskeletal, cardiovascular fitness and plasma lipids, and improved mental

health including self-concept, anxiety and depression.

The evaluation focused on three main overarching outcomes: 1) academic achievement, 2)

mental/emotional well-being, and 3) physical health and fitness measurements.

Key Findings

ACADEMIC ACHIEVEMENT:

Improved math skills as measured through standardized testing (KTEA-II)

Improved reading comprehension skills as measured through standardized testing (OCA,

B CIBS-II)

Average number of credits accumulated per semester increased

The majority of students (66.7%, N=28) responded that the program helped them pay attention in class while only 9 said that it did not

Increased interest in school including motivation and enthusiasm to attend class, improved grades, homework or credit completion and greater efforts to concentrate and pay attention in class as noted by parents/guardians (48.6%, N=18)

Table 1. KTEA-II Math Composite, Mean Percentiles at Start and End of Program (N=37)

KTEA-II Assessment

Mean Percentile Rank Program Start

Mean Percentile Rank Program End

Gains

Math Computation 13.82 19.73 5.91*

Math Concepts & Applications

19.14 28.66 9.52**

* Significant differences post program (p<0.01) ** Significant differences post program (p<0.0001) MENTAL/EMOTIONAL WELL-BEING:

65.9% of students (N=27) who responded noticed an overall positive change in mood from participating in the program

* p<0.001

Page 4: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

3 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

Sample Comments from Students and Parents

“I was in depression for the longest time and I haven't been since

the program”

“My family, peers and myself have noticed that I became happier,

more energetic and much more confident”

“The biggest one would be the anxiety, she’s a little more relaxed

and more focused on school and able to get her work done”

Those who did notice a change mentioned that they felt happier, more energetic, were more confident and “in a better mood”

Improvement in mood and behaviour including anxiety, happiness, relaxation, concentration, confidence, attitudes, and anger as noted by parents (29.7%, N=11)

PHYSICAL HEALTH AND FITNESS:

Improved aerobic fitness as measured by the Beep Test

No change in physical measurements including BMI, resting heart rate, blood pressure,

and waist circumference

Increased physical activity levels in 70.7% of students (N=29)

Next Steps/Future Recommendations

Increased physical activity time in the classroom did not affect learning negatively but

may help improve academic achievement, physical fitness, and mental health

Uptake of the SLN program to other schools for the 2012 to 2013 academic year

Funding application for a rigorous evaluation of the program

Use of social media for consciousness raising

Development of a program manual/toolkit

Adoption of a school-wellness policy

Sample Comments from Students and Parents

“Yes, since starting the program I run every day after dinner, and I

have also joined a soccer team”

“I think it would help her with mood and concentration at school

as well”

“It really increased her physical activity. She had almost no other

way of being physically active outside of this program. Her mood

swings are not as severe, and I noticed her sleeping better”

Page 5: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 4

2.0 INTRODUCTION

This evaluation report summarises the results from the 2011-2012 Sparking Life Niagara pilot

program. The program aimed to improve academic achievement in high school students

through physical activity. The report first gives a brief introduction to the background and

context, including identified need for the program and actual program description, before

outlining the methodology for the evaluation. Clear objectives and evaluation questions are

identified along with the description of the appropriate outcome assessment tools. Steps of

the evaluation are depicted and the key findings are then communicated. Lastly, this report

identifies key recommendations for the program given the evaluation results and appropriate

references and appendices are included.

2.1 Background and Context

The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an

Associate Professor at Harvard University. In 2008, Dr. Ratey published a book called “Spark:

The Revolutionary New Science of Exercise and the Brain.” In this book, he explores the science

and biology behind the relationship between physical activity and the brain. Specifically, he

focuses on aerobic exercise but also relays the importance of strength training. Throughout his

book, Dr. Ratey explores the role physical activity can play on learning, stress, anxiety,

depression, attention-deficit/hyperactivity disorder (ADHD), addictions, hormonal changes and

aging.

Some of the most critical and relevant findings of this research include the ability of physical

activity to improve learning through three important mechanisms: by improving alertness,

attention and motivation (Hill et al., 2010; Winter et al., 2006); by encouraging and

strengthening the ability of neurons to bind to one another, which is critical for storing new

information (Geinisman, 2000); and lastly, by promoting neurogenesis of nerve cells in the

hippocampus (Van Praag et al., 2005; Cotman & Berchtold, 2002).

Naperville Central High School in Illinois implemented physical activity into their daily routine in

an attempt to improve academic achievement and overall mental, emotional, and physical

health of its student body. Physical education teachers at this high school adopted what is

called “Zero Hour PE,” an hour of physical activity prior to the start of classes, in an attempt to

improve literacy and reading comprehension. After just one semester, students who

participated in “Zero Hour PE” showed a 17% improvement in reading and comprehension

compared to only a 10.7% improvement among those students who did not participate in the

program (Ratey, 2008).

Page 6: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

5 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

The new approach to physical education classes started with focusing on fitness instead of

learning how to play sports. This involved teaching students how to use heart rate monitors to

ensure they were working at a high enough intensity as opposed to focusing on competition

and sport (Ratey, 2008). This novel method to physical education in Naperville has made the

district rank among the state’s top ten academically (Ratey, 2008). Results from 2001 and 2002

also showed that only 3% of freshmen were overweight or obese in this district, well below the

national averages (Ratey, 2008).

Based on the astounding results observed at Naperville Central High, the Sparking Life

movement is urging schools to find innovative ways to incorporate daily aerobic physical

activity in an attempt to improve academic achievement. Sparking Life Niagara (SLN) is a

program that adopted Naperville’s philosophy at Lakeshore Catholic High School in Port

Colborne and Eastdale Secondary School in Welland. This initiative aimed to help students be

healthier and do better in school (Niagara Region Public Health, 2012).

2.2 Program Need

The need for this program is two-fold. First and foremost, the primary objective of the program

is to help students improve their academic achievement. Students in the program were either

identified as at-risk of not graduating by their teachers or had credits they needed to recover to

graduate on time. Teachers are trained to grade and assess students’ academic abilities in

various ways. Increased concentration, mood, emotional, mental and physical health can all

contribute to improvements in academic performance (Ratey, 2008).

The Canadian Society for Exercise Physiology (CSEP) has put forth physical activity guidelines for

youth aged 12-17 years of age. These guidelines state that youth should get at least 60 minutes

of moderate to vigorous physical activity daily (CSEP, 2012). The 2007-2009 Canadian Health

Measures Survey (CHMS) shows that only 7% of Canadian children and youth are meeting the

recommended guidelines at least six days a week (Colley et al., 2011). Moreover, the Public

Health Agency of Canada and the Canadian Institute for Health Information (PHAC & CIHI, 2011)

state that the prevalence of obesity has nearly tripled for the 12 to 17 year old group in the last

few decades. The prevalence has risen from 3% in 1979 to approximately 9.4% in 2004 (PHAC

& CIHI, 2011).

According to Statistics Canada (2010), the national average of overweight and obese youth

aged 12 to 17 was 19% in 2008. The consequences of this are evident in recently published

reports such as Active Healthy Kids Canada (2012) Report Card on Physical Activity. The

Institute of Medicine also released a report on May 8, 2012 that included multiple

Page 7: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 6

recommendations for tackling obesity. Out of these recommendations, supporting programs

designed to increase physical activity is one to take note of. Moreover, Recommendation #5

states that to combat obesity, we need to have quality physical education and physical activity

opportunities in schools (National Research Council, 2012). The Sparking Life Niagara program

objectives clearly resonate with these recommendations.

2.3 Program Description

The Sparking Life Niagara program was carried out at Lakeshore Catholic High School and

Eastdale Secondary School from September 2011 to June 2012. In collaboration with the

District School Board of Niagara (DSBN), the Niagara Catholic District School Board (NCDSB),

Niagara Sport Commission (NSC), and Public Health Ontario (PHO), Niagara Region Public Health

(NRPH) implemented the program with students in grades 9 through 12. Students completed

aerobic exercise for at least 20 minutes at 65% to 85% of their maximum heart rate daily.

Throughout the day, ‘energizer’ activities (5 to 10 minute physical activity bouts) were also

implemented to get the students to increase their energy levels. Students received a physical

education credit for completion of the Sparking Life Niagara program.

Eastdale Secondary School

Eastdale Secondary School ran the Eagle Spark Initiative from September to January for the first

two periods of the school day in a literacy class. Forty minutes were put aside for the morning

workout and the activity was carried out in the school gym where treadmills, elliptical machines

and spinning bicycles were used. A polar heart rate monitor was worn and breakfast was

provided to the participants after the workout. Following the morning exercise session,

students were involved in numerous activities including literature assignments, independent or

oral reading, and performing plays. Two ‘energizers’ were built in the morning class and

students continued with their regular classes after lunch. In the second semester, the initiative

was organized into a credit recovery class without extra emphasis on literacy unlike first

semester. Physical measurements collected by the nurses were the only assessments carried

out in the second semester.

Lakeshore Catholic High School

Lakeshore Catholic High School ran the Sparking Life Niagara initiative in a “success” classroom.

The primary focus of the class was to help with credit recovery for those students that were

identified as at-risk. This included students with both poor academic performance and/or

behavioural issues. Students focused on completing online learning courses or worked on

Page 8: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

7 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

course packages to obtain credits. No formal lessons took place in the “success” classroom, but

support for the students came from the physical education teacher, business teacher and a

Child and Youth Worker. Treadmills, elliptical machines, an outdoor track, field or spinning

bicycles were used to complete 20 minutes of aerobic activity. An ‘energizer’ was carried out in

the afternoons to help elevate mood and energy levels. Heart rate monitors were utilized to

measure target heart rates during the activity.

3.0 EVALUATION OVERVIEW

3.1 Evaluation Sponsors and Stakeholders

The implementation and evaluation of Sparking Life Niagara was made possible through

numerous partnerships. Niagara Region Public Health managed and oversaw the project.

Public Health Ontario provided support with the evaluation. Niagara Sport Commission assisted

with funding applications and provided volunteers to help carry out the program. The District

School Board of Niagara along with the Niagara Catholic District School Board implemented and

evaluated the program in the classroom. Evaluation resources included people such as

teachers, public health nurses, volunteers, students, and epidemiologists. No monetary budget

was assigned to the evaluation.

Other stakeholders in this project included those who donated exercise equipment, grants and

other resources to the project such as Sport Chek’s Power of Sport 4 Kids program. Students

and their parents/guardians who were involved in the program were also important

stakeholders to keep in mind and feedback from them was obtained.

3.2 Evaluation Design

The evaluation included both a process and outcome evaluation. A time-series design was used

for the outcome evaluation. Outcome measurements were collected at the start, middle, and

end of the second semester as outlined in Figure 1 but only pre and post-test data were used.

PARTICIPANT

SELECTION

PRE-TEST DATA

COLLECTION

MID-POINT

DATA

COLLECTION

POST-TEST

DATA

COLLECTION

SPARKING LIFE

NIAGARA

INTERVENTION

Figure 1. Sparking Life Niagara Evaluation Design

Page 9: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 8

3.3 Evaluation Objectives

The objectives of this evaluation are as follows:

Assess program process

Assess program outcomes, impacts and effects

Assess program likeability and feasibility

Identify program and evaluation limitations and recommendations

Communicate key findings

3.4 Evaluation Questions

Assessing program process

Did program participants adhere to the program?

Did staff/volunteers carry out the program as intended?

What should be changed in the program delivery and implementation?

Assessing program outcomes, impacts and effects

Students

Did students notice any change in their mood or concentration after the program?

Did students improve their physical, emotional and/or mental health?

Did students increase or decrease their interest in physical activity or being active?

Did students improve their academic performance?

Did students find that Sparking Life Niagara helped them learn?

Teachers

Did teachers notice any changes in the students’ behaviour, mood, and learning?

Parents

Did parents notice any changes in the students’ behaviour, mood, and learning?

Did parents notice any change in the students’ interest in physical activity or school?

Assessing program likeability and plausibility

Should the Sparking Life Niagara initiative be continued?

Would students want to participate in the program next year? Why or Why not?

What did the students like best and worst about the program?

Page 10: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

9 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

Would parents/guardians want their children to participate in the program next year?

Why or why not?

4.0 METHODS

4.1 Participants

During the Sparking Life Niagara pilot project, 65 students were initially recruited for the

program. Twenty-six students participated and completed the program first semester, 18 male

and 8 female. The second semester, another 24 students completed the program, 13 female

and 11 male. Therefore, in total 50 students completed the program during the 2011 – 2012

academic school year. However, one student did it both first and second semester, meaning

that technically there were only 49 different students that data was collected from. The

students ranged in age from 14 to 19 years old and the mean age was 15.64. There were a total

of 21 females and 29 males in the program enrolled in grades 9 through 12. Loss to follow-up

was 23%. Please refer to Figures 2 and 3 for a breakdown of the participants.

Figure 2. Sparking Life Niagara Participants by Semester and Gender*

*Please note that one student participated in both semesters

65 students recruited

15 lost to follow up (7

semester 1; 8 semester 2)

50 completed program

program

26 completed program

in semester 1

24 completed program

in semester 2

18 males 8 females 11 males 13 females

Page 11: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 10

Figure 4 shows the grade distribution of the participants in the Sparking Life Niagara program.

There were a total of 13 students in grade 9 and 14 in grade 10. A total of 9 students were

enrolled in grade 11 and the remaining 14 were enrolled in grade 12.

58.0%

42.0%

Male

Female

Figure 3. Sparking Life Niagara Participants by Gender (N=50)*

*Please note that one student participated in both semesters.

0

2

4

6

8

10

12

14

16

Grade 9 Grade 10 Grade 11 Grade 12

Nu

mb

er o

f St

ud

ents

Figure 4. Sparking Life Niagara Participants by Grade Level (N=50)*

*Please note that one student participated in both semesters.

Page 12: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

11 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

4.2 Measurement Tools

Pre/Post Student Questionnaire

A pre and post student questionnaire was completed to gain insight into the program. The

questionnaire was designed to ask students numerous questions regarding what they liked or

did not like about the program, their learning styles and interests, and their opinions towards

physical activity. For a list of questions in the pre and post student questionnaire please refer

to Appendices A and B.

Parent/Guardian Telephone Interviews

Telephone interviews were conducted with the parents/guardians of students participating in

the Sparking Life Niagara program. Parents were asked about whether they had noticed any

changes in their child’s behaviour, mood, learning, or interest in physical activity and/or school.

Please refer to Appendix C for the complete interview script.

Informal Feedback

Informal feedback was received from Niagara Sport Commission interns and volunteers who helped out with the project. Moreover, feedback from teachers, guidance counselors, and the school board was also informally collected and has been taken into consideration.

Physical Measurements

A range of physical measurements were taken by Public Health Nurses. These included height, weight, waist circumference (WC), resting heart rate (RHR) and blood pressure (BP). Body Mass Index (BMI) was calculated using weight (in kilograms) divided by height (in metres squared) (kg/metres2). Measurements were taken at the start, midpoint and end of the program and were repeated multiple times to ensure accuracy and consistency. A calibrated scale, measuring tape and an automated blood pressure monitor were used for data collection.

Fitness Measurements

The Beep Test was used to measure aerobic fitness. The test required participants to stand

behind a line and run to a second line, 20 metres away before they heard the next beep. The

time between beeps shortened as the level increased. Those with better cardiovascular fitness

scored higher on the test. The test was terminated when a participant did not reach the

following line for two consecutive beeps. There are a total of 21 levels on the Beep Test and

Page 13: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 12

average scores for male youth are between levels 7 to 9 while for female youth betweem levels

6 to 8.

Academic Measurements

In order to minimize classroom disruption, schools selected the academic measurements for

use in the pilot evaluation. Three different tests were used to measure academic achievement.

Kaufman Test of Educational Achievement (KTEA-II)

The Kaufman Test of Educational Achievement, Second Edition (KTEA-II) was used to assess

numeracy skills. This validated tool can be used to measure current achievement for a wide age

range (4-6 to 25 years old). The Math Composite tests basic arithmetic and numerical

reasoning skills through the Computation and Concepts & Applications tests. This test provides

numerous scores including raw scores, standard scores and percentiles. Raw scores represent

the number of points earned by each student on the test while standard scores are norm-

based. Standard scores for the KTEA-II can be either age-based or grade-based. The KTEA-II

standard scores have a mean of 100 and a standard deviation of 15. The range of the standard

scores is usually from 40 to 160. Meanwhile, percentile scores represent the percentage of

individuals that the student outperformed at his/her age or grade level.

Ontario Comprehension Assessment (OCA)

To assess reading comprehension, the Ontario Comprehension Assessment (OCA) test was used. This test is usually carried out at the beginning and end of the semester to track progress. The OCA is most appropriate for those in grades 7 to 10. This test is graded out of 4 levels. Level 3 is considered to be the provincial standard. The OCA assesses literal and inferential thinking and making connections. Brigance Comprehensive Inventory of Basic Skills II (CIBS II) The Brigance test was used to measure English/language arts academic achievement. This test is appropriate for those in junior kindergarten (JK) to high school skill levels and has multiple components including the listening vocabulary composite, listening comprehension, word recognition, oral reading, reading vocabulary composite, reading comprehension, functional word recognition, and spelling grade placement tests. The Reading Comprehension section is scored based on average grade level achieved varying from kindergarten (K) to Grade 12 skill levels. 4.3 Data Analysis

Page 14: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

13 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

Qualitative Analysis

Qualitative analysis involved coding and categorizing responses obtained in the questionnaires

and feedback from parents, guardians, students, teachers, school boards, and volunteers. This

required going through the text to code and categorize relevant and related information

together. From the coding, major themes were identified and conclusions were then drawn.

Quantitative Analysis

All quantitative analyses were done using Microsoft Excel Version 2010 (Microsoft Corp.,

Redmond, WA). Dependent t-tests for matched-pairs (McNemar’s Test) were used to test for

any significant differences between scores at the beginning and end of the program. All tests

were two-tailed and assessed at the 5% significance level. This allows for 95% certainty that

the differences observed before and after the program are true differences and not purely due

to chance. Power and sample size calculations were not performed.

5.0 RESULTS 5.1 Academic Achievement

Kaufman Test of Educational Achievement (KTEA-II)

Results from the Kaufman Math Composite show improvements in students’ arithmetic and

numerical reasoning skills. In the Math Computation component, mean standard scores at the

beginning of the program were 77.38. By the end of the program, this significantly improved to

82.57 (p<0.01) as seen in Figure 5. Similarly, standard scores for Concepts & Applications also

improved from 84.3 to 90.1 at the end of the program (p<0.0001). Scores for the Computation

and Concepts & Applications test components improved on average by 5.19 points and 5.78

respectively. Moreover, mean percentiles at the start and end of the program were also

statistically significant improving by 5.91 for Computation (p<0.01) and 9.52 for Concepts &

Applications (p<0.0001) as displayed on Table 1.

Page 15: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 14

84.3

90.1

0

10

20

30

40

50

60

70

80

90

100

Stan

dar

d S

core

Mean

Figure 6. Kaufman Concepts & Application Mean Standard Scores Pre and Post Program (N=37)

Pre

Post

* Statistically significant differences in mean pre and post program (p<0.0001)

*

77.38

82.57

0

10

20

30

40

50

60

70

80

90

100

Stan

dar

d S

core

Mean

Figure 5. Kaufman Computation Mean Standard Scores Pre and Post Program (N=37)

Pre

Post

* Statistically significant differences in mean pre and post program (p <0.01)

*

Page 16: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

15 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

Table 1. KTEA-II Math Composite, Mean Percentiles at Start and End of Program (N=37)

Kaufman Assessment

Mean Percentile Rank Program Start

Mean Percentile Rank Program End

Gains

Math Computation

13.82

19.73

5.91*

Math Concepts & Applications

19.14

28.66

9.52**

* Significant differences post program (p<0.01) ** Significant differences post program (p<0.0001)

Results from the student questionnaires revealed that at the start of the program, the majority

of students (65.1%, N=28) did not enjoy math. The main reasons for not liking it was that it was

too difficult or that they were not good at it (N=18). The students who did enjoy math (N=14)

stated that the reasons they enjoyed math was because they were good at it, or because it

would be useful in the future.

Ontario Comprehension Assessment (OCA)

* Significant differences post program

* p<0.001

0

0.5

1

1.5

2

2.5

3

3.5

4

Cla

ssif

yin

g &

Org

aniz

ing

Idea

s

Co

mm

un

icat

ing

Cle

arly

Co

mp

reh

en

sio

nSt

rate

gies

De

mo

nst

rati

ng

Un

de

rsta

nd

ing

Exte

nd

sU

nd

ers

tan

din

g

Mak

ing

Infe

ren

ces

Me

taco

gnit

ion

Me

an S

core

Figure 7. OCA Mean Scores Pre and Post Program (N=36)

Pre

Post

*

*

* *

Page 17: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 16

Figure 7 displays the mean scores in the various OCA tests at the beginning and end of the

program. Significant improvements were noted in the Classifying & Organizing Ideas (p<0.001),

Comprehension Strategies (p<0.001), Making Inferences (p<0.01), and Metacognition (p<0.002)

subtests.

* Significant differences post program

Figure 8 shows the number of students at or above the provincial standard (level 3 or 4) at the

beginning and end of the program. Using McNemar’s Test, significant increases in the number

of students achieving the provincial standard post the SLN intervention were observed for the

Classifying & Organizing Ideas (p<0.0001), Communicating Clearly (p<0.05), Comprehension

Strategies (p<0.01), and Making Inferences (p<0.05) subtests.

At the start of the program, the majority of the 44 students who filled out the questionnaires reported that they did not enjoy reading (54.5%, N=24). The main consensus among those who did not enjoy reading was that it was boring (N=13) or difficult (N=6). Reasons for enjoying reading included that it was fun and easy, interesting or was an important skill in life (N=15).

Brigance Comprehensive Inventory of Basic Skills II (CIBS II)

One subset of students (N=7) were assessed using the Brigance Comprehensive Inventory of

Basic Skills II (CIBS II). Figure 9 shows the reading, oral and writing composite average grade

placements of students at the start and end of the program from September 2011 to January

2012. Using McNemar’s Test, significant positive differences were observed for the Reading

Vocabulary (p<0.008), Reading Comprehension – Long Passages (p<0.03) and the Oral Reading

0

5

10

15

20

25

30C

lass

ifyi

ng

&O

rgan

izin

g Id

eas

Co

mm

un

icat

ing

Cle

arly

Co

mp

reh

en

sio

nSt

rate

gies

De

mo

nst

rati

ng

Un

de

rsta

nd

ing

Exte

nd

sU

nd

ers

tan

din

g

Mak

ing

Infe

ren

ces

Me

taco

gnit

ion

Nu

mb

er o

f St

ud

ents

Figure 8. Number of Students At or Above the Provincial Standard (Level 3 or 4) for the OCA Pre and Post SLN (N=36)

Pre

Post

*

*

*

*

Page 18: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

17 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

(p<0.01) tests. Significant negative differences were observed for the Listening Vocabulary test

(p<0.04).

* Significant differences post program ** Please note N=6 for Listening Vocab test

School Credits

Before entering the Sparking Life Niagara program students were earning on average 3.4 credits

per semester as based on data for 30 students. During the program, these students were

earning on average 4.3 credits per semester as shown in Figure 10. A dependent t-test carried

out on the 30 available sets of data revealed that this difference in the average credit

accumulation per semester was significant with a p<0.02. It should be noted that an important

component of the program was credit recovery, which could account for the increase in the

average credit accumulation per semester as students received a credit for participating in the

program.

0

2

4

6

8

10

12

Wo

rdR

eco

gnit

ion

Re

adin

gV

oca

bu

lary

Re

adin

gC

om

pre

he

nsi

on

- Sh

ort

Re

adin

gC

om

pre

he

nsi

on

- Lo

ng

List

enin

g V

oca

b

List

enin

gC

om

pre

he

nsi

on

Ora

l Re

adin

g

Fun

ctio

nal

Wo

rd

Spe

llin

g

Pre

Post

Figure 9. Brigance Comprehensive Inventory of Basic Skills II (CIBS II) Subtests Average Grade Placements at Start and End of the Sparking Life Niagara Program (N=7)

Ave

rage

Gra

de

Pla

cem

ent

*

*

*

*

**

Page 19: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 18

Sample Comments from Students

“Yes I did find that this program helped me to pay attention in class

because it put me in a good mood and then I wasn't lazy and tired”

“When I worked out, I found that paying attention was much easier as I

was awake and alert. It helped me a lot when it came to listening”

“I did not find that this program helped me pay attention in class because I

was still having trouble focusing like I did before in other classes”

“After morning aerobic activity I felt great! I was more awake and I had a

lot more energy then I normally would if I didn't work out”

“Like I said, I feel the same after exercising so I don't know”

* Significant differences in mean before and during program (p<0.02)

Did students find that Sparking Life Niagara helped them learn?

The majority of students (59.5%, N=25) replied that participating in the program made it easier to learn. When asked if the program helped them pay attention in class, 28 students (66.7%) responded that it did while only 9 said that it did not.

After the morning aerobic activity, the majority of students felt more alert (N=25) and

energetic (N=12). Students reported that this in turn made them feel awake, ready to pay attention, and concentrate in class. After exercise bouts, students also reported that they felt more awake and alert (N=17), energetic (N=11) or tired (N=9).

3.4

4.3

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Before During

Me

an N

um

be

r o

f C

red

its

Figure 10. Average Credit Accumulation Before and During SLN (N=30)

*

Page 20: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

19 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

Sample Comments from Students

“I was in depression for the longest time and I haven't been since the

program”

“My family, peers and myself have noticed that I became happier, more

energetic and much more confident”

“I can go to school without wanting to leave all the time”

Did parents notice any changes in the students’ learning and interest in school?

A fair amount of parents/guardians (48.6%, N=18) noticed an increased interest in school in their child including motivation and enthusiasm to attend class, improved grades, homework or credit completion and greater efforts to concentrate and pay attention in class.

Much fewer parents/guardians (N=2) saw a decreased interest in school including attendance since participation in the SLN program.

5.2 Emotional and Mental Health Outcomes

Mood and Behaviour

Did students notice any changes in their mood and/or behaviour?

When asked whether they noticed an overall change in their mood from participating in the

program, 27 of the 41 respondents (65.9%) said that they did while 31.7% said that they did

not. Those who did notice a change mentioned that they felt happier, more energetic,

were more confident and “in a better mood.”

Sample Comments from Parents

“Grades went up a bit from before I noticed”

“A lot of her work is done now which was an issue before”

“No I'd say his attendance and going to school has gotten a lot worse”

“He is concentrating on his homework a lot more and he tries to pay closer

attention in class”

“He’s actually tried a bit more in school. He got all of his credits this

semester for one thing”

Page 21: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 20

Sample Comments from Students

“I think being physically active is very important if you want to have a

healthy body”

“It keeps you fit, and helps improve your physical and mental attributes,

which leads to overall happiness”

“It gives you a chance to get moving and provides an activity you can do

instead of being bored”

“Because I love sports and it is fun to play them”

Did parents notice any changes in the students’ mood and/or behaviour?

Some parents/guardians (29.7%, N=11) saw an improvement in mood and behaviour

including improvements in anxiety, happiness, relaxation, concentration, confidence,

attitudes, and anger.

5.3 Physical Health and Fitness Outcomes

Physical Activity Beliefs, Levels and Interests

Almost all students (95.5%, N=42) thought that physical activity was important. The main

reason reported was physical benefits, such as staying fit or “being healthy.” Other

important benefits mentioned included improved mental health such as increased

happiness and decreased stress.

Most students (90.7%, N=39) also mentioned that they enjoy physical activity. The most

mentioned reasons included physical health benefits, improvement in mood, as well as

enjoyment.

Sample Comments from Parents

“Behaviour has been pretty good. A little bit of improvement from before

I've noticed”

“His attitude is much better, not as angry as before”

“The biggest one would be the anxiety, she’s a little more relaxed and

more focused on school and able to get her work done”

Page 22: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

21 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

Sample Comments from Students

“At the start of the year I hardly did anything but now I participate every

day”

“Yes I play basketball again”

“Yes, since starting the program I run every day after dinner, and I have also

joined a soccer team”

“No I think my physical activity levels are the same”

While 72.7% of students (N=32) have participated in physical activity before in school, only

59.5% (N=25) have participated in any recreational or organized physical activity. When

asked about their favourite sport and activities, soccer (N=12), hockey (N=6) and volleyball

(N=5) were the top three answers.

Did students notice any changes in their interest in physical activity?

Out of the 41 students who responded, 29 or 70.7% stated that the program has made

them increase their physical activity levels. Additionally, 58.5% reported an increased

interest in participating in extra-curricular activities and 53.7% reported an increased

interest in taking physical activity courses.

Did parents notice any changes in the students’ interest in physical activity?

Almost half of parents/guardians (45.9%, N=17) reported an increased interest in physical activity in their child since participating in the Sparking Life Niagara program. Another 8 parents/guardians reported no change in physical activity levels due to their child always being physically active prior to SLN.

Sample Comments from Parents

“We have joined the gym”

“He normally rides his bike around after school but he's always done that”

“She's out for a walk right now with her friends actually”

“But you know she is doing her sit ups, running up and down and walking”

“She’s motivated to exercise that’s for sure”

“Now that she has participated in this program she’s more interested in

exercising, in the fitness aspect of it”

Page 23: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 22

Body Mass Index (BMI)

Percentage distribution by Body Mass Index was calculated for the 45 students that had

available data. This was done using height and weight measurements and age-and-sex-specific

classifications were made as defined by Cole et al., 2000. For students where age information

was unavailable, age was approximated by grade level. Figure 11 shows that there is a high

proportion of students classified as overweight/obese both pre and post intervention.

According to Statistics Canada (2010), the national average of overweight and obese youth

aged 12 to 17 was 19% in 2008. The minor differences observed before and after could be due

to the short length of the program. A change in BMI is perhaps a long-term outcome of regular

aerobic activity as opposed to short-term.

Since students in Semester 1 of the program had time off for the Winter Holidays, a comparison

between BMI between the first and second semester seemed logical. Usually people tend to

eat a lot more during holidays and perhaps this may have confounded the results. When BMI

was analyzed separately for each semester, no significant differences were observed although

the test was closer to significance for semester two (p<0.8 for semester 1 and p<0.4 for

semester 2). Similarly, when both semesters were analyzed together (N=45) the difference

between BMI pre and post was only 0.14 and this was also not significant (p<0.4).

Other possible explanations could include the dose of activity. Since the program aims to

primarily improve academic achievement, the prescribed dose of 20 minutes per day may not

51.1 53.3

48.9 46.7

0

20

40

60

80

100

120

Pre Post

Per

cen

t (%

) Figure 11. Percentage Distribution of Body Mass Index (BMI)

Classification at Start and End of Program (N = 45)

Overweight/Obese

Normal

Page 24: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

23 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

be sufficient enough to achieve significant weight loss. Moreover, students could also attain a

larger lean muscle mass and decrease their total fat mass with no overall change in BMI. This is

why the use of BMI has also been contested in the literature. BMI does not distinguish

between lean muscle mass and fat mass and thus it has been deemed less useful, especially for

certain populations such as athletes.

Waist Circumference (WC)

Waist circumference was measured by nurses for each student pre, mid, and post the Sparking

Life intervention. Figure 12 shows the average waist circumference by gender in centimetres at

the start and end of the program. Research literature has noted that waist circumference is a

better predictor than Body Mass Index (BMI) of cardiovascular disease risk. The average waist

circumference of the female participants was 83.1 cm at the beginning of the program. This

slightly increased to 83.3 cm at the end. On the other hand, males had a beginning average

waist circumference of 85.3 cm and this slightly decreased to 85.2 cm by the end. However,

none of these differences were statistically significant.

Resting Heart Rate (RHR)

Average resting heart rates in beats per minute varied slightly from the start to the end of the program in both males and females as shown in Figure 13. The mean RHR of males was slightly lower than that of females. The changes observed pre and post intervention were not significantly different in either gender.

83.1 83.3

85.3 85.2

0

10

20

30

40

50

60

70

80

90

100

Pre Post

Ave

rage

Wai

st C

ircu

mfe

ren

ce (

cm)

Figure 12. Average Waist Circumference by Gender, Pre and Post Program (N=45)

Female (n = 20)

Male (n = 25)

Page 25: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 24

Blood Pressure (BP)

Figure 14a shows the average blood pressure, both systolic and diastolic for females pre and

post program. Blood pressure remained fairly consistent at the beginning and end of the

program and no significant differences were found when dependent t-tests were carried out.

83.5 83.0

81.0

78.7

0

10

20

30

40

50

60

70

80

90

100

Pre Post

Ave

rage

Res

tin

g H

eart

Rat

e (b

pm

)

Figure 13. Average Resting Heart Rate by Gender, Pre and Post Program (N=45)

Female (n = 20)

Male (n = 25)

110.4 108.7

69.0 67.4

0

20

40

60

80

100

120

Pre Post

Ave

rage

Blo

od

Pre

ssu

re (

mm

Hg)

Figure 14a. Average Systolic and Diastolic Blood Pressure for Females Pre and Post Program (N=20)

Systolic

Diastolic

Page 26: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

25 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

Figure 14b shows the average blood pressure for males at the beginning and end of the

program. These values remained fairly consistent both pre and post intervention and no

significant differences were detected in the analysis.

Beep Test (BT)

Average Beep Test levels at the beginning of the program were 3.7 as displayed in Figure 15. At the end of the program, a noticeable improvement in aerobic fitness was observed. Average Beep Test levels at the end of the program were 5.0 and this was statistically significant with a p value of <0.0001. This improvement highlights just one benefit of routine cardiovascular exercise. The highest level achieved pre intervention was 9.5 while post intervention was 10.1.

113.9 113.7

68.1 68.8

0

20

40

60

80

100

120

Pre Post

Ave

rage

Blo

od

Pre

ssu

re (

mm

Hg)

Figure 14b. Average Systolic and Diastolic Blood Pressure for Males Pre and Post Program (N=25)

Systolic

Diastolic

3.7

5.0

0

1

2

3

4

5

6

Pre Post

Bee

p T

est

Lev

el

Figure 15. Average Beep Test Levels Pre and Post Program (N=39)

* Significant differences in mean pre and post program (p<0.0001)

*

Page 27: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 26

Sample Comments from Students

“Yes, because it helped make my life easier, it helped me feel better, more

aware of my school work, and it was a fun class”

“No, I would like to go to a gym or weight training class instead”

“I enjoyed being active and awake, it really made my day a lot better most

of the time”

“Unless it was 4th or 5th period, because I'm so tired in the morning”

5.4 Program Design & Adherence

Program Design

Likeability and Plausibility

Would students want to participate in the program next year? Why or why not?

When asked if they would want to participate in the program again most students (65.9%, N=27) said that they would. Reasons for participating again included enjoyment, improved mood, and the ability of the program to help students “tackle the rest of the day.”

The 31.7% of students that did not want to participate in the program again stated that SLN was boring, they did not like physical activity, or they wanted to use their time differently (other courses, take a spare, or prefer the gym).

Would parents/guardians want their child to participate in the program next year?

Why or why not?

A total of 30 parents/guardians (81.1%) said that they would want their child to participate in the program again. Reasons why they thought their child would do well participating again included enjoyment, positive teacher role modeling, motivation for school, physical activity and health benefits, increased concentration and alertness, improved body image and overall mental well-being.

Page 28: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

27 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

What did students like best and worst about the program? When asked what the best part of the Sparking Life Niagara program was students

responded with having more energy as the top answer (N=23). The next best things were using the equipment (N=17) and seeing friends (N=17) as seen in Figure 16.

When asked about the worst parts of the program, the most common answers included the early morning time of the workouts (N=9) and the lack of variety in equipment and exercises (N=6). To a lesser extent students disliked having assessments such as the Beep Test (N=3), or being sweaty (N=2).

When 40 students were asked to provide suggestions for improvements, the most common

answer was to change the workout out routine to incorporate different exercises, more equipment or a larger sport focus (N=11). Other suggestions included changing the time of the workout to later in the day (N=9), having a larger class (N=3) or providing more food (N=3).

Sample Comments from Parents

“I believe so, I think it would benefit her and she's enjoyed it. I think it

would help her with mood and concentration at school as well”

“I know she looks forward to that [SLN], always gets her wattle bottle

ready and a little snack for later”

“No he likes a variety of sports. He doesn’t like the cycle routine, would

want to do more sports, more variety of activities”

“Yes definitely, he’s talking about doing it again next year”

“I would have preferred it in the morning. I don’t really see the purpose of

it towards the end of the day”

“It really increased her physical activity. She had almost no other way of

being physically active outside of this program. Her mood swings are not

as severe, and I noticed her sleeping better”

Sample Comments from Students

“Changing my attitude [best part of the program]”

“Having to do exercises so early in the morning since I am not used to it”

“I can't think of one, it was great all around”

“None of my friends are in this class”

“The equipment, I think they need more of a selection than just cardio”

“Nothing, I love it”

Page 29: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 28

*Please note that students can list more than one answer.

Program Adherence

Fitness logs kept by the teachers were used to track program adherence. The tracking forms of

47 students indicated that on average, students completed at least 20 minutes of aerobic

activity for 41.8 (SD=17.52) days, or 58.5% of the expected frequency. When students did work

out and engage in the aerobic activity, 68.2% of their time was spent in their target heart rate

zone. This translated to an average number of 30.8 (SD=17.85) workouts in their target heart

rate zone out of the total 64 workouts in semester 1 and 87 workouts in semester 2.

Correlation coefficients were calculated for relationships between adherences to the exercise

and improvements in academics and aerobic fitness. The Kaufmann Math Computation and

Concepts & Application tests were used for improvements in academics. None of these

correlations were significant at the alpha 0.05 level. Three correlation coefficients were found

to be significant with the change in Beep Test scores achieved pre and post intervention. These

correlations included the difference in Beep Test scores and mean workout days in the target

heart rate zone (r= -.475; p<0.01), mean total aerobic days (r= -.552; p<0.001), and mean

percentage of aerobic days (r= -.455; p<0.01). Figure 17 shows the correlation between

differences in Beep Test scores and the mean total aerobic days. The more days the students

were active, the less the difference between their pre and post Beep Test score. This may

mean that those students who were more active, were fitter, and thus had less room for

improvement as opposed to those who were less active and likely not as fit. Moreover, the

graph shows that most of the differences in Beep Test scores were positive, meaning that most

students improved with only a few students decreasing or worsening their scores post

intervention.

0

5

10

15

20

25

Using the equipment Seeing friends Having more energy Reaching my THR

Co

un

t (n

)

Figure 16. The Best Parts of Sparking Life Niagara According to Students (N=41)*

Page 30: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

29 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

* Significant correlation r= - .552, p<0.001

6.0 LIMITATIONS AND RECOMMENDATIONS

6.1 Program Implementation

Limitations

Reading Comprehension Assessment: The OCA is designed to test reading comprehension and is most appropriate for those in grades 7 to 10. However, our participants ranged from grade 9 to 12. Therefore, this test may not be as suitable for those students who are in grade 11 or 12 (N=23) but this test was chosen based on the already available resources the schools had.

Facilities/Resources: The Sparking Life Niagara initiative used a large number of resources including exercise equipment such as treadmills and stationary bikes as well as people. Therefore, schools may have a hard time implementing this program if they do not find creative ways to implement the initiative without adequate facilities or resources.

Timing of intervention: In both semesters one and two, the exercise session took place primarily in the mornings. However, in some cases, some students were scheduled to have the class in the afternoon, during period four or five. This timing may have affected the results. Moreover, students in the afternoon sessions had fewer opportunities to report on any changes in learning as they did not have class after the workouts.

-2

-1

0

1

2

3

4

5

6

0 10 20 30 40 50 60 70 80Bee

p T

est

Dif

fere

nce

Number of Aerobic Days

Figure 17. Correlation Between Beep Test Difference and Mean Total Aerobic Days (N=34)

Page 31: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 30

Teaching: In particularly the teachers mentioned that they taught to the areas that proved to have the lowest scores in the academic testing components. This increased focused in these areas could have been the cause for the improved scores observed in a variety of the academic tests.

Recommendations

Program Manual/Toolkit: A manual/toolkit on how to incorporate the program in daily

school activities may be helpful, especially for teachers who do not have a physical

education background. Moreover, for schools that do not have extensive facilities and

resources, ways to adapt the program with limited resources should be outlined. Ideas

on how to keep students interested should also be included to improve program

adherence. The development of a manual will also ensure more consistency in the

delivery of the program among schools.

Reading Comprehension Assessment: If resources allow, a reading comprehension

assessment test that is suitable for students up to grade 12 should be adopted. This will

increase the validity and generalizability of the results but the adoption of a new test

will incur more expenses for the schools as manuals and training of teachers will need to

be paid for. Possible tests could include the Gray Oral Reading Test-Fourth Edition

(GORT-4) or the Wide-Range Achievement Test 4 (WRAT 4).

Training: Teacher training may be helpful in program delivery. A one-day extensive

training session should be held with all teachers who will be involved in program

delivery. Training will help teachers be more knowledgeable and confident in their

abilities to carry out the SLN initiative. Moreover, this ensures that all teachers receive

the same training, thus helping to standardize the intervention across schools.

School Wellness Policy: Adopting a school wellness policy may encourage teachers to

implement the program and increase awareness regarding the benefits of physical

activity and reduce stigma around the program. Other teachers may start to

incorporate ‘energizer’ activities throughout classes to increase energy levels.

Social Media Use: With the increased use of technology, students may respond better

to social media. Creating a Facebook Page for the SLN initiative or a Twitter account

may help students remain motivated and stay on track. Moreover, monthly emails

Page 32: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

31 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

could be sent to teachers and students to remind them to stay on track and continue

with the program.

Timing and Role Models: The timing of the exercise session should be in the morning.

This will prove to be more beneficial as students still have classes ahead as opposed to

exercising in period five at the end of the day. Moreover, teachers should exercise with

students to act as role models in the classroom and thus increase adherence to the

program.

6.2 Evaluation

Limitations

Self-Reporting Bias: Questionnaires, interviews and discussions conducted with parents, students, teachers, and volunteers may be biased as they relied on participants’ self-reporting. Participants may have felt embarrassed to answer truthfully and thus this may have led to social desirability bias. Moreover, people can easily under or overestimate answers that are subjective and thus caution should be taken when generalizing these results to other populations and contexts.

No Comparison Group: Without a comparison group we cannot be certain that the intervention caused the changes that were observed in our participants. With this type of evaluation design we can infer correlation or association but not causation. A more rigorous evaluation design could address many threats to internal validity such as history and maturation.

Self-Selection Bias: The Sparking Life Niagara initiative was piloted at two schools that desired to participate. These schools or the teachers who volunteered to run the program may be systematically different from other schools in the area. For example, the two schools that participated may have a really strong tradition of athletic excellence and success and thus value physical activity more than other schools may.

Small Sample Size: The sample size of the SLN initiative was fairly small (N=50). This small sample size makes it hard to detect significant differences at an appropriate power and thus reduces the generalizability and validity of the results.

Short Time Frame: The short time frame of the intervention was also a limitation. The length of the program may have not been sufficient to observe and make inferences about the possible long-term outcomes of the program such as changes in BMI.

Page 33: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 32

Student Evaluators: Two separate student evaluators conducted the telephone interviews and interpretations from first to second semester. This means that interviews could have been conducted slightly differently and thus the results may have been interpreted differently according to the evaluator.

Recommendations

Use of Control Group: The use of a control group can help improve the internal validity of the results. This means that inferences about program effects and outcomes would be easier to attribute. A control group can also address other threats to internal validity such as history and maturation.

Randomization: If funds are not available to implement the program at all schools in the area then randomization should be used in selecting schools to participate. This will decrease the chance of self-selection bias and increase the validity of the results.

Larger Sample Size: The SLN initiative needs to be evaluated on a much larger and rigorous basis. A larger sample size will help reduce the possibility of chance, bias or confounding in the results and can make it easier to detect significant differences or changes among and within groups. This will also increase the power of the study, making results much more valid and reliable.

Evaluation Staff: Every effort should be made to have the same evaluation staff conduct all parts of the telephone interviews, interpretation and other related duties of the evaluation. This will ensure that data and information obtained is more consistent and uniform, increasing the validity of the results.

Program Uptake: The evaluation shows favourable results for the participants in the SLN program. However, the program should also be applied in a variety of other contexts, as it would provide benefits for all students, not just those struggling academically. This program can also be applied to other populations as the benefits of physical activity are not strictly just for youth.

Page 34: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

33 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

7.0 REFERENCES

Active Healthy Kids Canada (2012). Is Active Play Extinct? The Active Healthy Kids Canada 2012 Report Card on Physical Activity for Children and Youth. Toronto: Active Healthy Kids Canada. Canadian Society for Exercise Physiology (CSEP). (2012). Canadian physical activity guidelines

for youth 12-17 years. Retrieved from website:

http://www.csep.ca/english/view.asp?x=804

Colley, R. C., Didier, G., Janssen, I., Craig, C. L., Clarke, J., & Tremblay, M. S. (2011). Physical

activity of Canadian children and youth: Accelerometer results from the 2007 to 2009

Canadian Health Measures Survey. Statistics Canada Catalogue: Health Reports, 22(1),

15-23.

Cotman, C.W. and Berchtold N.C. (2002). A behavioural intervention to enhance brain health

and plasticity. Trends Neurosci, 25, 295-301.

Geinisman, Y. (2000). Structural synaptic modifications associated with hippocampal LTP and

behavioural learning. Cerebral Cortex, 10, 952-962.

Hill, L., Williams, J.H.G., Aucott, L., Milne, J., Thomson, J., Greig, J., Munro, V. and Mon-Williams,

M. (2010). Exercising attention within the classroom. Developmental Medicine & Child

Neurology, 52, 929-934.

National Research Council (2012). Accelerating progress in obesity prevention: Solving the

weight of the nation. Washington, DC: The National Academies Press.

Niagara Region Public Health. (2011). Sparking Life Niagara Background. Retrieved from

http://www.niagararegion.ca/living/health_wellness/healthylifestyles/spark-

life/background.aspx

Public Health Agency of Canada & Canadian Institute for Health Information (PHAC & CIHI).

(2011). Obesity in Canada. Retrieved from

https://secure.cihi.ca/estore/productFamily.htm?locale=en&pf=PFC1636

Ratey, John. (2008). Spark: The revolutionary new science of exercise and the brain. New York,

NY: Little, Brown & Company.

Statistics Canada, Geography Division for the Health Statistics Division. (2010). Health indicator maps: Overweight or obese youth 2007/2008 by health region (catalogue no. 82-583-XIE, Vol. 2010, No.1. (CANSIM Table 105-0502).

Page 35: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 34

Van Praag, H., Shubert, T., Zhao, C. and Gage, F.H. (2005). Exercise enhances learning and hippocampal neurogenesis in aged mice. The Journal of Neuroscience, 25(38), 8680-8685.

Winter, B., Breitenstein, C., Mooren, F.C., Voelker, K., Fobker, M., Lechtermann, A., Krueger, K.,

Fromme, A., Korsukewitz, C., Floel, A. and Knecht, S. (2006). High impact running improves learning. Neurobiology of Learning and Memory, 87(4), 597-609.

Page 36: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

35 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

8.0 APPENDICES

Appendix A.

Sparking Life Niagara – Student Pre Questionnaire

2 - STUDENTS WHEN: PROGRAM ONSET TYPE: Questionnaire

Current Engagement in School

NAME:_______________________________

DATE:________________________________

Do you enjoy learning? Yes No

Why / Why not?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Describe How do you learn best?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Do you enjoy reading? Yes No

Why / Why not?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Did you enjoy math? Yes No

Why / Why not?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Are there any classes you think are important for your future? Why?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

What aspects of school do you enjoy the most?

Page 37: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 36

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

*Discuss physical activity with students - Physical activity is any form of activity that brings your heart

rate up. This can include walking, riding a bike, cycling and playing a sport

Current Engagement in Physical Activity

Do you enjoy physical activity? Yes No

Why / Why not?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Have you participated in physical activity in school before? Yes No

If so, what?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Have you participated in any recreational or organized physical activity? Yes No

If so, what?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Do you have a favourite sport or physical activity? Yes No

If so, what?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Do you participate in physical activity in your spare time? Yes No

If so, what do you do?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Do you think physical activity is important? Yes No

Why? / why not?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Page 38: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

37 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

Appendix B.

Sparking Life Niagara – Student Mid-Point Questionnaire

Student Testimonials Mid-Point

NAME:_______________________________ DATE:_________________________________

1. What is your opinion about the Sparking Life Program at LCHS

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

2. How has your daily exercise routine affected your life, i.e., personally, academically, socially, or

physically?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Page 39: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 38

Appendix C. Sparking Life Niagara – Student Post Questionnaire

5 - STUDENTS WHEN: End Of Program TYPE: Questionnaire NAME:________________________________

DATE:_________________________________

General:

Overall, how did you feel when you went to class after morning aerobic activity?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

How did you feel after exercise bouts?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Mental Factors:

Did you find the program helped you to pay attention in class? Explain.

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Do you find the program made it easier to learn? Explain.

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

How did you find this school year compared to other school years? Easier? Harder? Explain.

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

How did you find math this year?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Page 40: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

39 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

How did you find reading this year?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Social Factors:

Did you enjoy learning this year? Yes No

Please explain:

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Have you noticed a change in your overall mood? Yes No

Please explain:

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Physical Activity Factors:

Has the program made you increase your physical activity levels?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Did the program increase your interest in:

Participating in extracurricular physical activities? Yes No

Taking physical activity courses? Yes No

Page 41: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 40

Overall:

Did you enjoy the program overall? Yes No

Why / Why not?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Does exercise help you learn? Yes No

Why / Why not?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

What is the best thing about the Spark program?

Using the equipment Seeing friends

Having more energy Reaching my target heart rate

Other: ____________________________________________________

What is the worst part of the program?

_____________________________________________________________________________________

_____________________________________________________________________________________

If you were in the program next year, how would you want the program to be different?

_____________________________________________________________________________________

_____________________________________________________________________________________

Would you want to participate in the Spark program again? Yes No

Why / Why not?

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

Page 42: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

41 Sparking Life Niagara Evaluation 2012 | Public Health Ontario

Appendix D. Sparking Life Niagara – Parent Telephone Interview

Good morning/afternoon/evening

May I please speak with Mr. /Mrs. ______________?

Hi, my name is Arlinda Ruco and I’m with the Sparking Life Niagara team from Niagara Region Public

Health.

I am calling in regards to your son/daughter student’s name who is participating in our program at

Lakeshore Catholic High School to find out about his/her experience so far with Sparking Life Niagara.

Are you able to take 10-15 minutes to answer a few questions about any observations you have made

since your son/daughter started the program. Your feedback will help us evaluate this program to see

what is working well and what areas we can improve.

If not, is there a better time that I can contact you?

Thank you. Before we start, I would like to note that your involvement in this interview is completely

voluntary. You may decline to answer any question and can end the interview at any time. Finally, all the

information you provide is confidential and anonymous.

Question One. Have you noticed any differences in student’s name behaviour since starting the Sparking

Life Niagara program?

What was his/her behaviour like at home before the program?

What types of changes have you seen since the program? (probe: overall mood, level of

communication, interactions with other family members)

Question Two. Has student’s name demonstrated a greater interest in school since participating in

Sparking Life Niagara?

How did he/she feel about school before the program?

What types of changes have you seen since the program? (probe: feelings about homework,

enthusiasm for school in the morning, learning challenges)

Question Three. Has student’s name demonstrated a greater interest in being active since participating

in Sparking Life Niagara?

How often did he/she engage in physical activity before the program?

Since the program, has there been a change in his/her physical activity levels outside of school?

What types of activities has he/she done and how often? (probe: organized sport or other

informal exercise, group or individual activities, intensity level)

Page 43: SPARKING LIFE NIAGARA - niagararegion.ca · The Sparking Life Niagara project was based on the research conducted by Dr. John Ratey, an Associate Professor at Harvard University

Public Health Ontario | Sparking Life Niagara Evaluation 2012 42

Has student’s name expressed an interest in participating in physical activity throughout the

summer? Explain. (probe: signed up for organized sport team, gym membership, house-league).

Question Four. Have you noticed any other changes in student’s name that you feel are related to

his/her participation in the program? (probe: social circles, academics, related interests)

Question Five. Would you want student’s name to participate in this program again next school year?

If Yes Great. Would you mind sharing some reasons why you feel he/she would do well to participate again?

If No Okay. Would you mind sharing some of the reasons why you don’t think he/she should participate again?

Those are all my questions, thank you for your time. If you had any additional comments or concerns in

regards to Sparking Life Niagara, you can contact:

Eastdale Contact: Tracy Garratt 905.724.7458 x456 Lakeshore Contact: Chris McLean/Josh Oort 905.835.2451 Thanks again for your help, have a great day/evening.

Interviewer’s Comments:

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

_____________________________________________________________________________________

*Note any observations made during the interview which may be relevant to the parent’s responses. For

example, was it the mother/father who completed the interview? Was it a busy time? Did the parent

seem well informed (i.e. mention whether or not they had talked to their child specifically about their

experience in the program)?