1

THE JOURNEY OF HOPE CURRICULA:

BUILDING RESILIENCE AFTER A NATURAL DISASTER

Authors:

Lori Holleran Steiker PhD

Tara Powell MSW, MPH

The University of Texas

Evaluation of Save the Children’s

Journey of Hope Program in

Tuscaloosa, Alabama

2

TABLE OF CONTENTS

Executive Summary…………………………………………………………………………………... 3

Acronyms…………………………………………………………………………………………........ 6

1.0 Introduction…………………………………………………………………………………........ 7

2.0 Journey of Hope Program: Background and Overview

2.1 Background of Journey of Hope Program in Alabama……………………………………….. 9

2.2 Overview of the Journey of Hope Programs…………………………………………………... 9

3.0 Evaluation Objectives, Outcomes & Limitations

3.1 Objectives…………………………………………………………………………………........ 13

3.2 Limitations…………………………………………………………………………………….. 13

3.3 Intended Users…………………………………………………………………………………. 13

4.0 Evaluation Methodology, Scope and Results

4.1 Elementary and Adolescent Journey of Hope……………………………………………………... 14

4.1.1 Methodological Tools……………………………………………………………………....... 14

4.1.2 Setting………………………………………………………………………………………... 16

4.1.3 Data Analyses and Results…………………………………………………………............... 16

Elementary Journey of Hope

Written Questionnaires…………………………………………………………………… 16

Disciplinary Referrals……………………………………………………………………. 18

Youth Coping Inventory………………………………………………………………….. 19

Adolescent Journey of Hope

Communities that Care Survey…………………………………………………………... 21

Strengths and Difficulties Questionnaire………………………………………………… 21

Qualitative Findings Adolescent and Elementary………………………………………… 22

4.2 Teen Journey of Hope

4.2.1 Methodological Tools……………………………………………………………………........ 30

4.2.2 Setting………………………………………………………………………………………... 30

4.2.3 Analyses and Results…………………………………………………………………………. 30

5.0 Key Findings & Discussion

5.1 Elementary and Adolescent Journey of Hope…………………………………………………….. 34

5.2 Teen Journey of Hope……………………………………………………………………………... 36

5.3 Limitations………………………………………………………………………………………… 37

5.4 Conclusions………………………………………………………………………………………. 38

Appendices............................................................................................................................................. 40

Appendix 1: Teen Journey of Hope Questionnaire

Appendix 2: Strengths and Difficulties Questionnaire

Appendix 3: Written Questionnaires

Appendix 4: Qualitative Interview Guide

Appendix 5: CTC Survey

Appendix 6: Youth Coping Inventory

References............................................................................................................................................... 50

3

Executive Summary

Background

In response to a devastating tornado outbreak in Alabama in April 2011, Save the Children, in

partnership with University of Texas, conducted and evaluated the Journey of Hope (JoH)

program in Tuscaloosa, Alabama from September 2011-May 2012. The JoH program was

developed by Save the Children after Hurricane Katrina in response to the specific psychosocial

needs of children after that disaster. The premise of the program is to “help children and adults

cope, build on their natural resiliency and strengthen their network of social support with friends

and caring others.”1 Thus, the program’s goal is to increase children and adolescents’ knowledge

and skills to enhance coping strategies and protective factors that promote resilience. The JoH

program consists of four developmentally appropriate curricula that teach socio-emotional and

coping skills in an interactive learning environment facilitated by social workers and counselors.

Junior Journey of Hope

(JJOH)

Ages 5-7 Eight 30-45-minute sessions with groups of 5-7 children

focusing on topics such as safety anger, sadness, bullying, and

self-esteem incorporating cooperative play, art and literacy.

Elementary Journey of

Hope (EJOH)

Ages 8-10 Eight 1-hour sessions with groups of 6-10 youth focusing on

topics such as safety, self-esteem, anger and aggression

incorporating literacy, art, and music.

Adolescent Journey of

Hope (AJOH)

Ages 11-14 Eight 1-hr sessions with groups of 6-10 early adolescents

focusing topics such as stress, fear, safety, self-esteem and anger

incorporating games, journaling, and discussion.

Teen Journey of Hope

(TJOH)

Ages 15-18 Three 1-hr sessions with groups of 8-15 older teens and

emerging adults focusing on topics such as stress and coping

through games, discussion, and relaxation exercises.

Objectives

The 2011-12 Tuscaloosa JoH evaluation builds on previous findings from JoH evaluations

conducted in 2009 in New Orleans after Hurricane Katrina and in 2011 after a 6.3 magnitude

earthquake struck Christchurch, New Zealand. The New Orleans results were mostly qualitative

and found that after youth participated in the JoH curricula they expressed: increased social

well-being, emotional well-being and knowledge and skills. Quantitative and qualitative data

from the evaluation in New Zealand found youth who participated in the JoH exhibited reduced

emotional distress and increased emotional regulation and awareness. Building on previous

findings, the objectives of the Tuscaloosa evaluation were to:

Assess the impact of the Elementary and Adolescent JoH curricula in a post-disaster

setting

Inform continuous program improvement related to process, content, and evaluation

Methodology

The evaluators followed a mixed methods approach to the research, which utilized quantitative

and qualitative measures to assess the impact of the program on participants. The following

methods were used to evaluate each curricula:

1 Save the Children Fact Sheet

4

EJoH

Validated scales measured children’s coping a were delivered pre and post intervention

for both the experimental and control groups;

District level disciplinary referral data for children participating in the EJoH gathered in

the fall 2011 and spring 2012;

In-depth interviews and focus groups conducted post program participation measuring:

what topics resonated with participants, which topic was most important and what they

liked about the group;

A standardized survey measuring knowledge and skills gained from the program

delivered post intervention.

AJoH:

Validated scales measured children’s emotional distress, coping and school bonding were

delivered pre and post intervention;

In-depth interviews and focus groups conducted post program participation measuring:

what topics resonated with participants, which topic was most important and what they

liked about the group.

Key Findings

From a thorough analysis of all data, the results of the JoH program evaluation demonstrate

enhanced socio-emotional functioning in the following areas:

Decreased emotional distress

Reduced externalizing behaviors (i.e. school disruptions)

Enhanced interpersonal communication and peer relationships

Elementary Journey of Hope

Decreased emotional distress 1. Written standardized surveys indicated that youth who participated in the EJoH learned

how to make themselves feel better when having difficulties with emotions (mean of 4.73

on 5 point Likert scale); and

2. Thematic qualitative findings revealed that participants had increased abilities to deal

with stress and augmented coping skills gained from participation in the program.

Reduced externalizing behaviors

1. Disciplinary referrals dropped by 30% of those who participated in the Elementary

Journey of Hope program;

2. Written standardized surveys indicated youth learned about how to manage anger (4.83

on a 5.0 scale); and

3. Qualitative themes illustrated comments about understanding anger, recognizing the

related feelings, and new coping mechanisms.

5

Adolescent Journey of Hope

Decreased emotional distress 1. The Strengths and Difficulties Questionnaire (SDQ) illustrated that after adolescent boys

6th

-8th

grade and adolescent girls in 8th

grade participated in the program their emotional

distress symptoms were reduced from borderline emotional problems to a normal range.

Enhanced interpersonal communication and peer relationships

1. Students who participated in the AJoH self-reported improved peer relationships on the

Communities that Care Survey from pre to post-test (13.5 to 14.2); and

2. Qualitative data indicated youth enhanced friendships, trust, were able to talk with others

to feel better, and even understanding others (i.e., empathy).

Limitations

A major limitation was not being able to use a quasi-experimental study design to evaluate AJoH

due to an inadequate number of participants for a control group. Another limitation was not

being able to assess the impact of the program on participants beyond the end of the program

implementation period due to limited funding. Future studies would benefit from a longitudinal

study design.

Recommendations

Recommendation #1: Continue to implement the JoH program in response to disasters.

This is the second domestic evaluation of the JoH and indicates the curricula are positively

impacting their target audiences. Given the dearth of post-disaster, school-based psychosocial

programs, the JoH program provides a unique contribution to the field and fills a gap in

programming available for youth after a disaster.

Recommendation #2: Continue to evaluate JoH to ensure that it is an evidence-based program

which positively impacts participants.

While it is challenging to use a quasi-experimental study design in the midst of a disaster, the

timing of the curriculum evaluation is valuable to understanding its efficacy. Conducting

ongoing and consistent evaluations will increase the aggregate sample and thus the ability to

determine statistically significant changes resulting from the intervention. Ideally, in the next

evaluation, quantitative measures should be added for the following constructs which

thematically arose in qualitative findings including: self-efficacy, affect regulation, and social

skills.

Recommendation #3: Implement the JoH program with youth in settings other than disaster

response, for example, inner city youth who deal with stressors such as violence, crime and

poverty.

This will allow for more rigorous methodological design (being that the settings are not in a

period of crisis), and help determine the efficacy of the intervention without the inevitable

limitations of school settings post disaster.

6

Acronyms

Adolescent Journey of Hope AJoH

Communities that Care Survey CTC

Elementary Journey of Hope EJoH

Institutional Review Board IRB

International Institute of Children’s Rights and

Development

IICRD

Journey of Hope JoH

Junior Journey of Hope JJoH

Strengths and Difficulties Questionnaire SDQ

Teen Journey of Hope TJoH

Youth Coping Inventory YCI

7

1.0 Introduction

Save the Children Program in Alabama

During the week of April 25, 2011 a series of tornadoes struck the South, Midwest and

Northeastern United States. This tornado outbreak was the largest recorded in US history and

took hundreds of lives and left thousands displaced. Tuscaloosa, Alabama, was among the cities

devastated by those series of tornadoes. On April 27th

, 2011 an E-4 tornado, spanning more than

a mile wide struck Tuscaloosa, causing more than 1,000 injuries and approximately 65 deaths in

the area. This was the highest death toll from a tornado in the United States since 1955 (NASA,

2011).

Children were among those most adversely impacted by the tornadoes in Tuscaloosa. Hundreds

were injured or separated from their family, and many experienced loss of family members,

caretakers and peers. Moreover, these tornadoes not only displaced hundreds of children from

their homes, but also damaged or destroyed a number of schools.

Natural disasters such as hurricanes, earthquakes, tsunamis, floods and wildfires can impact a

child’s social and emotional well-being (Garrett et al., 2007; Kataoka, Rowan, & Hoagwood,

2009; Walsh, 2007). The psychosocial impact on children affected by disasters can be

debilitating and may appear immediately or surface weeks or months after experiencing the

event. The emotional strain on children affected by a traumatic event may be exhibited in a

variety of ways, including re-experiencing the event, intrusive thoughts, avoidance of similar

situations around the trauma, hyper-arousal, and anger (Wang et al., 2006). Externalizing

symptoms (anger and acting out at school and at home) and internalizing symptoms (anxiety and

depression) may also be mental health consequences for children who have experienced a

disaster (Jaycox, 2006). Furthermore, common post-disaster psychological disorders can include

acute stress reactions, adjustment disorders, depression, panic disorders, PTSD, and anxiety

disorders (Kar, 2009)

There are a variety of risk factors which can augment the likelihood of post-disaster

psychological symptoms in youth. Greater exposure to the disaster, witnessing others in life-

threatening situations, having family members die, demographic factors (age, gender), pre-

8

existing characteristics of the child, the post-disaster recovery environment, child’s

psychological resources, parental distress or length of displacement can put children at a higher

risk for developing post-traumatic stress symptoms (Cohen et al., 2009; Kar, 2009). Conversely,

research has shown protective factors for children such as parental and social support, promoting

a sense of control, normality, and empowerment may mitigate post-traumatic stress symptoms

(Cohen et al., 2009; Williams, Alexander, Bolsover, & Bakke, 2008).

The Journey of Hope curricula seek to mitigate the negative psychological sequelae after a

disaster by enhancing protective factors such as establishing a sense of control and normalcy

through discussion and activities, and providing psycho-educational knowledge about emotions

such as fear, anger, and anxiety that children may encounter after a traumatic event.

9

2.0 Journey of Hope Program: Background and Overview

2.1 Journey of Hope Programming in Tuscaloosa, AL

The Journey of Hope curricula were first brought to Tuscaloosa in September of 2011 because of

the unique psychosocial needs of the children who experienced the devastation of the tornado.

Hundreds of children were displaced from their homes and schools and had lost the sense of

stability they had before the tornado. As the city began to rebuild and schools were slowly

being re-opened, Save the Children partnered with Tuscaloosa city schools to respond to the

emotional needs of those children most affected by the disaster.

These developmentally appropriate set of curricula were provided to youth between the ages of 6

and 18 over the course of the 2011 and 2012 school year and to date have served 2,897 youth in

Tuscaloosa2.

Table 1: Beneficiaries of Journey of Hope Fall 2011

Adolescent Journey of Hope: 156

Elementary Journey of Hope: 467

Junior Journey of Hope: 476

Fall Total: 1,099

Spring 2012

Adolescent Journey of Hope: 295

Elementary Journey of Hope: 543

Junior Journey of Hope: 709

Journey of Hope for Teens: 231

Journey of Hope for Caregivers: 20

Spring Total: 1798

Overall Total: 2,897

2.2 Overview of Journey of Hope Programs

After a disaster, children continue to endure the everyday stresses of living in a community that

is rebuilding, often with limited access to services. Many of these children have witnessed

significant changes in their homes, neighborhoods, and schools and continue to process those

changes in their everyday lives. The Journey of Hope curricula were originally brought to

Tuscaloosa, Alabama, in response to the emotional strain on the youth who were impacted by the

tornadoes. As the schools were rebuilding and slowly re-opening, many of the youth were

exhibiting signs of distress. Numerous children were displaced or lost a loved one and all had

2 See Table one for breakdown of beneficiaries

10

experienced the devastation the tornado brought to the city. In response, Save the Children US

liaised with Tuscaloosa city schools to bring the Journey of Hope psychosocial curricula to

Tuscaloosa.

The Junior (JJoH), Elementary (EJoH), Adolescent (AJoH) and Teen Journey of Hope (TJoH)

are developmentally appropriate curricula that offer children between the ages of 6 and 18 the

opportunity to better normalize their emotions and develop positive coping strategies through

cooperative play, creative arts and literacy. These curricula, which are based in social cognitive

theory, teach children social and emotional skill building to promote self-efficacy, problem

solving and positive coping so they may have the capacity to overcome current and future

traumas (Bandura, 1998). Additionally, the Journey of Hope curricula enhance protective factors

for youth such as promoting healthy peer relationships, a positive school environment, and stable

relationships with adults.

The JoH curricula, which have been administered in Tuscaloosa city schools since September,

2011, are designed to teach children the skills to overcome the adversity of a natural disaster.

The premise of the curricula is: “to help children cope, build on their natural resiliency and

strengthen their network of social support with friends and caring others”.3

Working in coordination Save the Children U.S., consultants from the University of Texas

embarked on a comprehensive evaluation to identify the impact of the Journey of Hope curricula.

These findings build on previous findings from an evaluation conducted in 2009 by the Institute

for Child Rights and Development (IICRD) in New Orleans after Hurricane Katrina and from a

recent evaluation conducted in Christchurch, New Zealand, in 2011 in response to a 6.3

magnitude earthquake that struck the city. Following are specific learning objectives of the

Journey of Hope curricula.

Learning Objectives

1. To support children in understanding and normalizing emotions associated with trauma or

difficult circumstances;

2. To support children in developing positive coping strategies to deal with these emotions;

3 Save the Children Fact Sheet

11

3. To build on the innate strengths of children, their families, schools, and communities to further

develop positive coping mechanisms; and

4. To instill a sense of hope, empowering children to feel more in control over stressors.

Program Design

The Journey of Hope curricula use a child-centered, strengths-based approach to provide

children, adolescents and emerging adults with positive resources to understand and cope with

emotions caused by traumatic situations. The curricula are organized into 30 minute to hour-long

sessions that can be implemented within a school term or in a summer camp4. The core tenets of

the JoH are to help children, adolescents, older teens and emerging adults: 1) understand and

normalize key emotions; 2) identify triggers and stressors; and 3) develop positive coping

strategies to deal with these emotions. Table 2 provides a description of each program including

length, topics and structure.

Table 2: Description of JoH programs

Program Description

Junior Journey of Hope (ages 5-7) Eight 30-45-minute sessions with groups of 5-7 children focusing

on topics such as safety, anger, sadness, bullying, and self-esteem

incorporating cooperative play, art and literacy.

Elementary Journey of Hope (ages 8-10) Eight 1-hour sessions with groups of 6-10 youth focusing on topics

such as safety, fear, anger and aggression and self-esteem

incorporating games, books, art, and music.

Adolescent Journey of Hope (11-14) Eight 1-hr sessions with groups of 6-10 early adolescents focusing

topics such as stress, fear, safety, self-esteem and anger

incorporating games, journaling, and discussion.

Teen Journey of Hope (15-18) Three 1-hr sessions with groups of 8-15 older teens and emerging

adults focusing on topics such as stress and coping through games,

discussion, and relaxation exercises.

Each session of the Journey of Hope curricula follows a similar routine to create a safe place

where participants feel comfortable participating in activities and sharing their feelings to help

normalize emotions. Moreover, each program utilizes developmentally appropriate learning

strategies, including:

Cooperative games to enhance social skills, encourage teamwork, and build awareness of

stressors in a non-competitive manner;

Books and dialogue to enhance emotional intelligence and reinforce messages to help

normalize emotions after a trauma; and

4 See Table 2 for a description of each Journey of Hope program

12

Music, art, journaling and dance and/or movement to give children an opportunity

process their emotions through an alternative outlet of expression.

Each program is adapted to use age appropriate activities. The core content of the JJoH, EJoH

and AJoH are as follows:

The core content and structure of the TJoH is as follows:

Session Topic:

1 Introduction: Creating Safety

2 Fear: Understanding and Coping

3 Anxiety: Understanding and Coping

4 Sadness: Understanding and Coping

5 Anger and Aggression: Understanding and Coping

6 Bullying: Understanding and Coping

7 Self-Esteem and Taking Action: I Believe I can

8 Me, My Emotions and My Community

Session Topic:

1 Introduction: Types and Sources of Stress

2 Stress Reactions: Understanding and Coping

3 Coping Mechanisms: Resources and Coping

13

3.0 Evaluation Objectives, Anticipated Outcomes and Limitations

This section of the evaluation describes the objectives, outcomes, and limitations of the

evaluation process including: (1) core objectives of the evaluation; (2) limitations; and (3) the

intended end users of the evaluation.

3.1 Objectives of the Evaluation

The key objectives of the evaluation were to:

Objective 1: To assess the impact of the Elementary, Adolescent and Teen Journey of Hope

curricula in Tuscaloosa as delivered by social workers and counselors;

Objective 2: To evaluate the applicability of the JoH curricula in a post-disaster

context; and

Objective 3: To provide Save the Children with results from the evaluation, aimed at delivering

a more thorough knowledge on the curricula’s impact.

In accordance with the objectives, the anticipated outcomes of the evaluation were to:

To provide a more thorough evidence base of the Journey of Hope curricula for Save the

Children;

To deliver a solid understanding of the applicability of the JoH curricula (Elementary,

Adolescent and Teen); and

To provide information on improvements that could enhance the impact of the curricula.

3.2 Limitations of the Evaluation

The following limitations are noted:

The absence of a control group for the Adolescent and Teen Journey of Hope was a

limitation of the study due to missing consent forms;

Missing data made it was impossible for analysis of the Strengths and Difficulties (SDQ)

for youth who participated in the EJoH; and

Because of the short duration of the evaluation funding period, long-term follow-up was

not feasible. Future studies would benefit from a longitudinal study design.

3.3 Intended end users of the evaluation

Following is list of recognized stakeholders who may be end users of the evaluation:

Save the Children staff (psychologists, social workers and the management team);

Save the Children Alliance personnel and management;

Communities impacted by a disaster both in Tuscaloosa and globally; and

Agencies and schools within Tuscaloosa who have received the program.

14

4.0 Evaluation: Methodology, Scope and Results

4.1 Elementary and Adolescent Journey of Hope

The Elementary and Adolescent Journey of Hope curricula were first delivered in September,

2011. To date the curricula have been implemented with 1460 youth in Tuscaloosa city schools.

The evaluation used methodological tools which will be described below. Due to the

developmental age of the youth and sensitivity of survey instruments a variety of scales were

distributed based on the age of the participants. The sample includes the following:

The Youth Coping Inventory was completed at pre and post-test by N=61 (waitlist-

control group) and N=73 (experimental group) who were between the ages of 7-10 who

participated in the EJoH and with N=106 students 11-14 who participated in the AJoH;

The Communities that Care questionnaire was completed with (N=106) adolescents that

participated in the AJoH;

Disciplinary referrals were gathered pre and post intervention for all students who

participated in the evaluation;

The Strengths and Difficulties Questionnaire was completed by (N=106) students who

participated in the AJoH;

Save the Children’s written questionnaires were completed by N=71 youth who

participated in the EJoH; and

Individual interviews were conducted with N=30 students after participation in the EJoH

and AJoH curricula.

4.1.1 Methodological Tools

To attain a comprehensive understanding of the impact of the curricula, specific measures were

used. The evaluators used both quantitative and qualitative methodological tools. Validated

scales that measured coping skills, school bonding, and emotional symptoms of the participants

were completed both before and after participation in the JoH program. Written questionnaires

were also distributed post-program to measure knowledge and skills gained through

participation, and qualitative interviews were conducted using open-ended questions about what

the participants gained from the program.

15

The following data collection methodologies were used:

The SDQ The Strengths and Difficulties Questionnaire (SDQ) is an evidence-based

brief 25 item scale which is a behavioral screening questionnaire for

children (parent and teacher report ages 3-10) and adolescents (self-

report ages 11-16) (Goodman, 2001). The SDQ consists of five subscales

measuring emotional symptoms, conduct problems,

inattention/hyperactivity, peer relationship problems, and pro-social

behavior. The scale has high internal reliability using Cronbach’s Alpha

of .73. The internal consistency of SDQ has also been supported across

multi-cultural settings including; Dutch (.80), UK (.88), and Australia

(.77). An analysis of SDQ subscales in Australia also illustrated moderate

to strong internal reliability (Hawes & Dadds, 2004).5

Written A scale created by Save the Children staff in New Orleans and the

Questionnaires International Institute of Children’s Rights (IICRD) was used to measure

knowledge and skills the children gained from participating in the EJoH

program. This five point Likert scale, measured participants’: (1) favorite

activities; (2) knowledge about anger and aggression; (3) comfort in the

group; (4) feelings of safety in the group; and (5) knowledge about the

impact of bullying.6

Interviews Qualitative interviews were conducted using an interview guide for the

EJoH and AJoH. This guide asked questions such as: (1) what

participants liked about the group; (2) which feeling was most important;

(3) what participants didn’t like about the group; and (4) the favorite

activities of the participants.7 The guide was developed by IICRD during

the pilot evaluation in New Orleans. It was then adapted to fit the needs

of the participants in Tuscaloosa.

Disciplinary The number of disciplinary referrals was obtained from Tuscaloosa City

Referrals Schools8. The disciplinary referrals were gathered at two time points, the

term prior to participation in the Journey of Hope program and the term

following participation in the program.

5 See appendix 2 for Strengths and Difficulties Questionnaire

6 See appendix 3 & 4 for Written Questionnaire

7 See appendix 5for qualitative interview guide

8 Disciplinary referrals included: excessive distraction of other students, fighting, vandalism, offensive touching,

threat, harassment or intimidation of student or other offenses deemed inappropriate by the principal.

16

Community

that Cares

Survey

The Community that Cares (CTC) survey is a validated measure that

assesses risk and protective factors for adolescent problem behaviors

such as delinquency and substance use. The CTC has a series of sub-

scales that measure 10 protective factors and 22 risk factors. For the

purpose of the Journey of Hope evaluation we used the school bonding

sub- scale.9

School Bonding

The school bonding sub-scale measures protective factors such as liking

school, time spent on homework, and perceiving schoolwork as relevant.

This subscale is also negatively related to drug use and other outcomes

such as dropping out of school. Internal consistency reliability for the

School bonding subscale ranged from .71-.77(Arthur, Hawkins, Pollard,

Catalano & Baglioni, 2002).

Youth Coping The YCI is a self-report measure of coping style. Participants rate the

Inventory(YCI) frequency of their use of various coping strategies on 31 items,

scored on a 5-point Likert scale. Responses are categorized into three

broad coping strategies. Internal consistency for the YCI is high

(Cronbach’s alpha = .86) (McCubbin et al., 1996). The predictive validity

of the YCI has also been established (McCubbin et al., 1996).

4.1.2 Setting

The evaluation followed ethical Institutional Review Board (IRB) guidelines established by the

University of Texas at Austin. Each child who participated in the pre and post-test test

questionnaires and qualitative interviews was provided permission by their parent or guardian

through a consent form and assented to participate. The school principals and social workers

liaised with Save the Children psychologists and social workers to gather the consent forms from

the parents.

4.1.3 Data Analysis and Results

Elementary Journey of Hope

Written Questionnaires

Written questionnaires were completed post-intervention to assess each component of the

intervention including: favorite activities, knowledge about anger and aggression, comfort in the

9 See appendix 6 for CTC survey

17

group, feelings of safety in the group and knowledge about the impact of bullying.10

Descriptive

statistics were conducted to find the means of the survey items. Following are results of the

written questionnaires for the Elementary Journey of Hope intervention. The sample of written

surveys for the EJoH program included (N=71) students between the ages of 6-12. The majority

of respondents were female (N=42, 59.2%) and were a mean age of 9 years old.11

The post-test questionnaire was completed on a five point Likert scale with a score of 1

indicating “not at all” and a score of 5 equaling “a lot”. Findings yielded positive results with

students overwhelmingly (4.49 to 4.83 on the Likert scale) expressing that they enjoyed the

group, learned specific knowledge and skills and liked the activities.12

Notable findings from the

written surveys include that group participants rated: learned how to make themselves feel better

(4.73); learned about bullying; (4.82) and learned how to handle anger (4.83)-the highest on the 5

point scale.

Table 3: Demographic information

Age N Percent

6 1 1.4

7 9 12.7

8 11 15.5

9 29 40.8

10 12 16.9

11 5 7.0

12 1 1.4

Missing 2 2.8

Total 71 100.0

Gender N Percent

Female 42 59.2

Male 27 38.0

10

See appendix 3 for complete written questionnaire 11

Table 3 provides demographic information 12

Table 4 illustrates the mean scores for each of the questions from the written survey

18

Table 4 provides the mean scores for the written questionnaires:

Table 4: Means of Written Questionnaires for the EJoH

Questions Mean

Shared Feeling 4.61

Like Activities 4.80

Respected in group 4.42

Rules 4.49

Felt Safe in group 4.41

Wanted to go to group 4.62

Learned about bullying 4.82

Learned how to handle anger 4.83

Like the Books 4.57

Learn about self 4.53

Learned to make self feel better 4.73

Disciplinary Referrals

District level disciplinary referral data was gathered for all students who participated in the

Elementary Journey of Hope program and had parental consent. The sample of disciplinary

referral data included (N=126) students who participated in the EJoH. Findings indicate a

decline in disciplinary referrals between baseline and post participation in the EJoH. At time one

(December, 2011), there were 36 reported disciplinary referrals of the students who participated

in the EJoH and time two (May, 2012) there were 25 referrals. These findings are promising

being that one sign of distress among youth is externalizing behaviors. Furthermore, as will be

outlined in the qualitative section, the ability to manage anger and aggression were specific

themes that arose. Additionally, the highest mean on the written surveys was that students

learned how to manage anger (4.83) which further supports that the Elementary JoH program

may have an impact on externalizing behaviors.13

13

Table 5 provides a graph of the change in disciplinary referrals from the fall of 2011 and spring of 2012

19

Table 5: Disciplinary Referrals EJOH

Youth Coping Inventory

Quantitative data for the Elementary Journey of Hope used a quasi-experimental pre and post-

test design. The sample in the experimental group consisted of (N=73) youth between the ages of

7-12 who participated in the Elementary Journey of Hope program and (N=61) youth who were

in the waitlist-control group. The data were collected in four schools (Alberta, Central,

University Place Elementary, and Arcadia), and were evenly split between grade and gender.

Demographically the majority of participants were African American (74.9%), followed by

Caucasian (11.6%) and Latino (5.5%).14

Findings indicate a positive change in means in the experimental group for the communication

and tension management sub-scale of the YCI, whereas the wait-list control group means slightly

increased.15

While the changes were not statistically significant, there were differences in the

means between experimental and control group. Furthermore, the experimental group’s change

was nearing significance and was in the direction predicted by the hypothesis that participation in

the EJoH may have a positive impact on externalizing behaviors as measured by the

communication and tension management subscale of the YCI.16

Future studies would benefit

from adding validated teacher or parent report measures in addition to more self-report scales

14

See table 6 for complete demographic information 15

Lower scores indicated a higher ability to manage anger. Items 18, 23 and 25 were recoded 16

See table 7 for mean changes in the YCI

0

5

10

15

20

25

30

35

40

Term 1 2012 Term 2 2012

Disciplinary Referrals EJOH

Disciplinary Referrals EJOH

20

which would provide a more comprehensive picture on the impact of the EJoH on externalizing

behaviors.

Table 6: Demographics of Elementary Journey of Hope participants who completed the YCI

Grade %

3rd

38.5%

4th

34.9%

5th

26.5%

Gender

Female 50.2%

Male 49.8%

Race

African American 74.9%

Asian .7%

White 11.6%

Hispanic 15.5%

Table 7: Mean differences YCI communication and tension management

Mean Pre test Mean Post-test N Significance

Experimental group 2.44 2.20 73 .075

Control group 2.41 2.43 61 .823

Adolescent Journey of Hope

Quantitative data for the AJoH used a pre, post-test design with (N=105) students who

participated in program. A quasi-experimental design was not completed because of the

difficulty identifying students who were going to participate in the following round of

implementations, which resulted in an inability to obtain parental consent. The sample of the

AJoH included students between 6th

and 8th

grade, were mostly female (79%) and the majority

were African American (79.8%) followed by Hispanic (11.3%) and Caucasian (8.1%).17

17

See table 8 for demographic information

21

Table 8: AJoH demographic information

Grade %

6th

44.4%

7th

39.5%

8th

16.1%

Gender

Female 79.0%

Male 21.0%

Race

African American 79.8%

Asian .8%

White 8.1%

Hispanic 11.3%

CTC-school bonding

Paired sample T-tests indicated students who participated in the AJoH raised their school-

bonding scores by almost a point between pre and post-test. This finding is particularly

noteworthy because peer bonding is considered a protective factor against future negative

psychological outcomes.

Table 9: Means differences CTC school bonding pre and post-test

Mean Pre test Mean Post-test N Significance

Adolescent JOH 13.5 14.2 105 .07

Strengths and Difficulties Questionnaire

The SDQ illustrated gender specific outcomes including positive changes in total emotional

symptoms and reduced peer problems among boys, and reduced conduct problems for girls.

Table 10 indicates that peer problems among males were reduced from 3.28 to 2.60. This is

particularly notable because a score above 3.0 falls in the borderline range of peer difficulties,

while below 3.0 is in the normal category for peer problems. Similarly, the total emotional

symptoms for boys were reduced from 11.84 (borderline) to 10.92 (normal). The mean

difference of conduct problems for girls was also noteworthy because their self-reported

outcomes illustrated a drop in symptoms of .5. Moreover, the changes in all measures can be

identified as clinically significant because of the fall from borderline range to the normal range

of functioning (Ogles, Lunnen, & Bonesteel, 2001). According to Ogles, Lunnen & Bonesteel

22

(2001) clinical significance can indicate a meaningful change when a person is “empirically

indistinguishable” from peers who are “normal”, after participating in a mental health or

psychosocial program.

Table 10: Mean differences Peer Problems SDQ (Boys only) Mean Pre test Mean Post-test N Significance

SDQ Peer problems 3.28 2.60 25 .06

Total Emotional

Symptoms

11.84 10.92 25 .331

Table 11: Mean differences Conduct Problems SDQ (Girls Only)

Mean Pre test Mean Post-test N Significance

Conduct Problems 2.88 2.57 91 .06

Other noteworthy quantitative findings were the differences of 8th

graders in relation to 6th

and

7th

graders. The 8th

graders reported reduced conduct problems and reduced emotional

symptoms on the SDQ and increased problems solving skills on the YCI. Conduct symptoms on

the SDQ significantly dropped from 2.55 to 1.75 (p<.05) and problem solving skills on the YCI

significantly increased from 33.12 to 36.24 (p<.05). Emotional symptoms also fell from 12.30

(borderline emotional symptoms) to 10.90 (normal emotional symptoms). Moreover, the change

in means can again be identified as clinically significant considering the scores fell from the

borderline range of emotional symptoms to a normal level of functioning.

Table 12: Mean differences 8th

Grade only

Mean Pre test Mean Post-test N Significance

SDQ conduct 2.55 1.75 20 .01

YCI Problem Solving 33.12 36.24 20 .02

Total Emotional

Symptoms

12.30 10.90 20 .241

Qualitative Findings

The last week of January, the researchers conducted interviews and focus groups in three schools

(University Place, Eastwood, Alberta) where the Elementary and Adolescent Journey of Hope

were implemented. One interview guide was used for the EJoH and AJoH given the similarities

in the program design and structure. The evaluative inquiries revolved around questions about

23

the following: (1) what the students learned in the group, (2) what they liked about the group,

and (3) what they didn’t like or thought could be improved. The primary analysis method

utilized was the semi-structured interviews, or guided conversations with the goal of discovering

the informants’ experiences (Glesne & Peshkin, 1992). The interview schedule was semi-

structured, meaning that it had basic questions to start with, but those were followed up with

intuitive responses to the participants’ answers. This allowed for the emergence of the

participant’s agenda rather than the researcher’s sense of what is important. The interview

schedule was written by the research team based on their experience and knowledge of the

Journey of Hope curricula and the literature on children/adolescents, trauma, loss, and coping.

Most of the questions and probes were open-ended to elicit the participants’ beliefs, thoughts,

and experiences in their own words. The interviewers were particularly careful not to use labels

and descriptors that might bias the responses.

Interview guidelines were utilized to frame the narrative-style interviews. We began with a brief

explanation that we are asking them what they thought of the program. As the interviews

unfolded, the researchers’ goal was to obtain a balance between inviting the participants toward

the general areas of the research questions (e.g., experience of the Journey of Hope program,

impact of the lessons, feelings about their experiences and the effects of the program in relation

to their experience) and allowing them to feel free to discuss whatever was important to them.

The protocol for group formation emerged from the context--the participants’ administrators,

teachers, and counselors chose participants and formed the groups to be interviewed. With the

prerequisite that the student had parental/guardian consent and participated in at least six of the

eight sessions in Journey of Hope program, students were referred for interview by the teachers

and counselors. For the elementary school interviews, both interviewers sat in each interview

and focus group to allow one to focus primarily on recording data and the other to facilitate

process. This also allowed for one to escort the students to and from the classrooms while the

other prepared the space. The middle school interviews were done by one interviewer to ensure

that the student felt comfortable in the one-to-one setting and so that they did not feel a heavy

adult presence which might inhibit their responses. Each interview lasted approximately 30-45

minutes depending upon the number of students (interviews were shorter in duration for dyads

24

and individuals than for larger focus groups). The opening question addressed the student’s

perceptions of the Journey of Hope program activities and the rest of the session was spent

eliciting elaborations on the students’ responses. The role of primary facilitator was alternated,

while the other researcher observed and recorded data. Audiotapes accompanied the researchers’

written notes.

Data Analysis

The interviews were transcribed by two scholarship winning social work undergraduate students

who were both familiar with the curriculum, the project, the researchers and the present research

protocol. There were a total of 6 interviews and 8 focus groups. The analysis of the transcribed

data involved the process of coding to elicit patterns and themes in the data. The coding

involved breaking codes into subcategories reflecting the participants’ conditions, interactions,

strategies, consequences, styles while moving to increased specificity (Lofland & Lofland, 1995;

Strauss, 1987) Ultimately, themes were identified by ideas that occurred repetitiously.

N-Vivo software was utilized as well as traditional manual coding when analyzing the data. The

N-Vivo program, used to aid in the organization and analysis of the data, involves the coding of

the data in “tree structures” at increasingly integrative levels. It also allows for specific word

searches, juxtapositions, and frequency of words or phrases. The combination of computer and

traditional manual coding allowed for systematic and efficient analysis as well as time to reflect

and think about the connections and themes. Even when using the computer as a means for

analyzing qualitative data, the process is both creative and mechanical. Richards and Richards

(1994) make the distinction between “textual level operations” (e.g., moving of the data) which

are done by the computer, such as retrieving codes, and “conceptual level operations”

(development of themes) done by the person. Ultimately, the researcher builds relations between

the data and the themes.

The qualitative analysis began with 14 broad initial codes which included: games, feelings,

books, natural disasters, drawing, bullying, safety, learning, anger, friends, trust, peer groups,

learning about self, and sadness. The researchers then conducted more focused coding. The N-

Vivo code tree was used to make the broad codes more specific. Next, codes were evaluated to

see which were used more than others, less productive codes were omitted, and the most

25

resonant ones were selected. Codes were collapsed, supported or dropped. Ultimately, the

coding procedure proceeded until core categories emerged to the point of saturation (i.e., where

further analysis does not elicit new themes). The final and most noteworthy themes revolved

around the concepts of feeling regulation, dealing with anger/bullying, valuing the program,

self-efficacy, and finding new ways to cope.

The students described on numerous occasions how the group helped them to realize things that

people have choice in how they deal with their feelings and that when they aren’t working, they

can find new ways to deal with the feelings. For example, a student notes:

“Me, like I’ll have really (bad) feelings, that I could…I used to before the group, but

when the group happened I learned how to cope with some of it, how was a good way

to deal with something. What I liked is that, it’s . . . it’s not a place where you like have

to learn. You learn stuff but you also have fun while you learning and it’s also good to

help people who get out of control with their anger like me and I’m, and it helped me

learn how to control it more better and that’s how, that’s why I like it.”

And another student adds:

“Because, because like, it really helps with your feelings and everything. I mean when-

before I started this program I was really, really mad all the time, and now I’m just

like, not that mad anymore and stuff. And I really think it would help somebody with

anger issues and stuff like that.”

One student even recognized that the feelings could even be dangerous if positive coping

mechanisms were not utilized:

“Because like some people I know, they get angry a lot, so they, when they talk about

anger it could help them. Some people are going through lots of different things, like

they’re going through grief so that could help them, and also, they tell us about ways

you can like let your anger out without going to the extreme, ‘cause, like the extreme

would be, um, you trying to hurt yourself or trying to hurt others because you’re-

you’re so angry or you’re so upset, that’s how you let it out. And some people like,

she’s said, don’t go to the extremes, like when you were very upset, and stuff, you

think, “Oh I just want to leave the planet,” and you try to hurt yourself. Well, um, she

just said don’t do that because you have to think about how would your family feel,

how would your friends feel, and, uh, stuff like that.”

26

Several students describe how violence or bullying is a reaction that can be changed through new

awareness and choices:

“Wow, uh, you know, how, you know, this whole school basically is just…these kids,

some of them are, you know, don’t have that coping skill……they just react at either a

fight or altercation, but since…….we’ve been going to the program, and, uh, it’s kind

of fun because, you know, going in there, and since we—since we’re done, this year,

um, we, since then all we’ve, uh, you know, covered how to cope and today in class this

girl named H., she um, she was bothering me so, you know, I said something back of

course, and then, um, I actually stopped myself before I said what I really wanted to

say, so, I was like, ‘You know what? Cope.’”

Students not only learned how to deal with their own anger, but also how to empathize with

others who are angry and might bully as a coping mechanism.

As one student articulated:

“Uh, I learned that there are some, like, we don’t know what’s going on with them.

Like not saying that I did, but like sometimes bullies should think that, ‘cause they

don’t know what they’re going through. But I know I didn’t bully anybody in that

group. It’s just the fact that, um, well, some people they were going through stuff and

like when people bullied them or tried to make them feel bad, it didn’t help them at all.

It just made them feel worse. So I learned in the group—how other people were…I saw

some people have problems going on at home, some have problems that they are going

on in their heads, and we don’t know anything about them, so I learned that, I think

that bullies should just stop and think for a second, like what they’re going through

and stuff.”

One student even expressed increased positive sense of self from the changes experienced from

the program: “I was proud to be in the group ‘cause I learned how to control my anger.”

While many of the students commented on how much “fun” they had throughout the program,

especially noting the games (bullying game, parachute, fleece ball game, body guard, dancing,

journal, and parties among the favorites), the students readily articulated the value of the games

in augmenting their self-awareness, affect regulation abilities, and coping skills. For example,

a student stated:

“And the games helped us with a skill. You know to have fun but at the same time help

us understand. Like one day I came in and I was angry about something and then we

27

played this little toss around the ball game and then, you know, it kinda made me feel

happier.”

Another child understood the implications of the affect regulation feeling work on a deeper level,

stating the following:

“How you feel sometimes, how you feel like you can let it, to let it out sometimes you

feel better. So we talked about different ways you could like, um, in your notes—like

you could have a diary or you could just talk it out to yourself or stuff like that, or you

could call a friend and talk to them. And then on anger, well, um, it helped you learn

how to, um, like just calm yourself down every now and then. How to relax. ‘Cause

sometimes I know I get angry, but, um, like it helped me to just think about what I

learned and stuff.”

Many of these comments reflect the theme of self-efficacy – the program clearly enhances

students perception that they are able to handle themselves better than before the program and be

good to themselves and others. This enhances their sense of safety, comfort, as well as their

social abilities. As stated by one student:

“All I know is that I had fun, and then, like, I remember before I even started this

program I was always like mad and getting on people’s nerves and now I do that less

and I’m like happy now.”

It emerged from the data that the students felt safe in the groups to talk about what they needed

to and to express themselves openly. For example, a student expressed:

“I could tell I could trust the group, and—‘cause like I said at the beginning, like

nothing ever went out of the group and, you know, I was just waiting for that one day

someone’d come up to me, ‘So this really happened?’ and like something like that, but

that never happened.”

This creation of a safe “holding environment” allowed for the exploration and learning that is

evidenced by their descriptions of their new coping skills and comfort levels. The importance of

trust is expressed throughout the interviews.

As one student states:

28

“We could trust them (the facilitators) a lot and I like to know that I’m able to trust

people because if you say that I can trust you but then you say you can’t, that I can’t-

that I can’t trust you, I’m probably gonna get very furious and just tell you just to leave

me alone because you’re playing with me and I don’t like to be played with about

certain feelings and emotions.”

As described by one student:

“Okay the thing I liked was, um….you know, you can go in there every day knowing

that, she said everything in that room stays in that room, confidential between the

people in that session and Miss K and Miss D. And, um, you know, you can go in there

and you can tell them everything, you know, you can tell them about what your day

was like….what your future is, and they won’t tell anything. They’ll try to explain to

you or plot it out how-what you should do for the problem that you have, and then, you

know, they just know what to say.”

And another student, commenting on a male facilitator:

“He asked us if we understood. And like took, and he asked us “Do we have anything

we can talk about?” And we can talk about everything with him. And he, uh, always

have an answer for us

29

Table 13: Summary of Salient Qualitative Themes

Questions

N

%

What did you do in the group?

Played Games 3 10.0

Talked about feelings 12 40.0

Talked about bullying 10 33.3

Learned about who or what feels safe 4 13.3

Learned about how to deal with anger 4 13.3

What did you learn?

How to deal with sadness 5 16.6

That it is normal to be scared 4 13.3

Not to take anger out on others 4 13.3

How to deal with bullies 6 20.0

What was your favorite activity?

Parachute 3 10.0

Discussion of what others liked about group members 3 10.0

Bullying game 5 16.6

Body guard 4 13.3

Team Juggle 2 6.6

Which feeling was most important? Bullying 7 23.3

Happiness 2 6.6

Fear 2 6.6

Anger and Aggression 9 30.0

Safety 1 3.3

Self-esteem 3 10.0

Sad 1 3.3

30

4.2 Teen JoH

The Teen Journey of Hope (TJoH) was first implemented in Tuscaloosa in the spring of 2012.

The program was adapted from the Caregivers Journey of Hope as a three session model to

identify stressors, help build coping skills and build supports for the teens. The evaluation in

Tuscaloosa was the first to be conducted for the TJoH, and was completed with nine groups

(N=139) of high school students who participated in the program in the spring of 2012. The

study used a pre/post-test design to identify the knowledge and skills gained from the program.

The researchers were then able to conduct statistical analyses to evaluate what the participants

gained from the curriculum.

4.2.1 Methodological Tools

To assess the outcomes of the Teen Journey of Hope, the evaluator adapted a previously used

quantitative survey that was used for the Caregiver’s Journey of Hope. The design included a

pre and post-test design which evaluated the knowledge and skills gained through the Teen

Journey of Hope program with (N=139) teens who participated in the intervention. Participants

who were evaluated filled out a survey that asked questions about knowledge before taking part

in the program and after completion. The survey instrument was based on a Likert scale ranging

from 1 (not at all) to 5 (A lot). Questions revolved around the following areas: (1) knowledge

about stress, (2) identifying personal and community supports, and (3) understanding coping

strategies.18

4.2.2 Setting

The setting that was used in the evaluation included three city schools where the program was

delivered. To be eligible to complete the Teen Journey of Hope questionnaire, the participants

were required to complete a registration form and attend the three session program.

Demographically, the participants were 47.8% (N=66) female and 49.7% (N=71) male, were

evenly split between grades 9-12 and 93.7% (N=134) identified as African-American19

.

4.2.3 Data Analysis and Results

Descriptive statistics were conducted using the statistical software (SPSS) to identify the

demographic characteristics of the sample and whether the program met participants’

18

See appendix 1 for the Teen Journey of Hope Survey instrument 19

See table 14 for Demographic Information

31

expectations and appropriate comfort level. Paired samples T-tests were then completed to

indicate whether there were statistically different results from baseline to post-test.

Table 14: Demographic Information Table 15: Comfort and Expectations

N %

Grade

9 28 21.2

10 39 29.5

11 36 25.2

12 29 22.0

Female 66 47.8

Male 71 49.7

The outcomes of the paired t-tests yielded significant (p<.05) improvements on four of the

survey items: (1) signs of stress, (2) coping, (3) how to manage stress and (4) types of stress.

The most significant changes in means were knowledge about types of stress (2.91 pre-test, 3.86

post-test) and knowledge on coping strategies to handle stress (3.23 pre-test, 3.86 post-test).

Other positive changes in means, although not significant, showed an increased awareness of

breathing exercises to help reduce stress, knowledge about how stress affects the body and

identifying social supports as a way to improve well-being20

.

Descriptive statistics also measured the level of comfort participants felt and whether the

program met their expectations. Results indicate 73.1% (N=84) participants rated a four or five

on whether the program met their expectations, and 65.9% (N=83) scored a 4 or 5 on comfort

level while participating in the curriculum21

.

The participants were also provided two open ended questions about: (1) what they liked about

the program and (2) ways the program could be improved. A thematic finding of the participants

who completed the open ended questions reported they enjoyed the activities and the games22

.

One response about the activities was:

20

See table 16 for mean scores from the TJoH quantitative survey 21

See table 15 for complete scores on comfort and expectations 22

27 participants stated they enjoyed participating in the games and activities.

N %

Expectations N %

3 (Somewhat) 22 19.1%

4 31 27.0%

5 (A lot) 53 46.1%

Comfort

1 (not at all) 8 6.9%

2 10 8.6%

3 (Somewhat) 15 12.9%

4 35 24.5%

5 (A lot) 48 41.4%

32

“I liked the games most, and that I was able to feel comfortable and able to express

myself.”

Another theme from the open ended responses was that participants found it helpful to learn

about how to cope with stress23

. Specific responses from participants included:

“JOH is an excellent program that is great to help people find positive ways to cope

with stress.”

and

“How we can talk about everybody's stress and how to deal with the different types of

stress, how to cope and what we do to relieve our stress.”

While almost half of the participants (N=26 of 57 responses) stated they enjoyed the program as

it was, suggestions on how to improve the program included: (1) the TJoH needs more sessions

to further discuss stressors and coping, and (2) the sessions needed to be longer because of the

valuable information that was discussed.

The results from this pre-post-test design for the Teen Journey of Hope indicate the program is

reaching the goal of building knowledge and coping skills among the participants. These results

are promising given the short duration of the program (3 1-hr long sessions) and considering this

is the first time the Teen Journey of Hope has been implemented in a school setting.

23

25 participants stated they benefitted from the discussions on how to cope with stress.

33

Table 16: TJoH pre and post-test comparisons on quantitative survey

Table 17: Means of TJoH quantitative survey N Mean Pre test Mean Post-

test

Significance

Support 139 3.18 3.36 .07

Stress affects body 138 3.93 4.09 .19

Signs of stress 134 3.51 3.99 .00

Coping knowledge 137 3.23 3.86 .00

Breathing 136 3.27 3.85 .65

Manage Stress 134 3.72 3.99 .05

Type of Stress 132 2.91 3.86 .00

Social Support 139 3.68 3.81 .21

2

2.5

3

3.5

4

4.5

Signs of stress Coping Breathing Types of Stress

Pre test

Post-test

34

5.0 Key Findings and Discussion

The evaluation of the Elementary, Adolescent and Teen Journey of Hope curricula in Alabama

has a number of noteworthy findings. First, the evaluation adds to the evidence base of the

Journey of Hope’s curricula in a post-disaster setting. While the original evaluation conducted

by IICRD provided qualitative findings and the evaluation in New Zealand illustrated the

applicability of the program in an international setting, the current research enhances Save the

Children’s knowledge on the impact of the curricula in the United States. The use of multiple

data sources including validated scales, written questionnaires, individual interviews, focus

groups and district level disciplinary referrals enabled triangulation of the data. Furthermore,

using more than quantitative self-report measures strengthened the vigor of the evaluation. As

Dunning and colleagues (2004) found in a meta-analysis of self-report measures, people often

over or under-estimate their functioning and do not often accurately report the effect of emotions

such as anxiety and fear on their functioning (Dunning, Health & Suls, 2004). By gathering data

from multiple sources, the evaluators were able to identify outcomes that may not have been

captured solely from self-reported quantitative measures.

This research also suggests that the curricula are appropriate in a post-disaster setting. This is

essential because of the dearth of evidence-based psychosocial curricula after a disaster (Roberts

& Everly, 2006). Many evidence-based curricula are treatment oriented, but the Journey of

Hope curricula can be implemented with all youth who have experienced a disaster because of its

strengths-based approach to understanding common feelings and reactions after a traumatic

situation. Moreover, the Journey of Hope curricula provide a distinct contribution to addressing

the psychosocial needs of youth after a disaster. The final section of this report discusses the

implications of the evaluation for the Teen, Adolescent and Elementary Journey of Hope.

5.1 Elementary and Adolescent Journey of Hope

The evaluation of the Elementary and Adolescent Journey of Hope are valuable on multiple

levels. First, the outcomes of the research provide a more thorough evidence base for the

curricula. Additionally, the evaluation supports the applicability of the curricula in a post

disaster setting. The outcomes of the study indicate that the curricula not only provides children

35

with psycho-educational knowledge, but also helps them process emotions such as anger and

enhances peer relationships which may enrich coping skills in future adverse events.

Elementary Journey of Hope

Participating in the EJoH increased psycho-educational knowledge.

The data clearly supported that youth increased their psycho-educational knowledge. The

written questionnaires asked direct questions about what was learned in the program, and high

scores on the Likert scale (4.0-5.0) indicate that the Elementary Journey of Hope program is

meeting its objective that participants learn about various emotions that they may encounter after

a disaster. Additionally the qualitative interviews illustrated that information learned through

the program resonated within many participants, and has helped youth process knowledge about

feelings such as sadness, fear, how to deal with bullies, and how not to take their anger out on

others.

Participation in the EJoH reduces externalizing behaviors.

The reduction in externalizing behaviors was particularly noteworthy considering one symptom

of a traumatic event is acting out. When youth do not have a safe and secure outlet to process

their emotions they can become reactive to and express their emotions through anger, aggression

and bullying. Multiple data sources (written questionnaires, qualitative interviews, disciplinary

referral data, Youth Coping Inventory) supported that youth learned about how to more

effectively process their externalizing symptoms. The standardized written questionnaires

illustrated that the “learning how to handle anger” section was a topic that resonated with the

youth (4.83 out of 5.0). Additionally, the 30% reduction in disciplinary referrals indicated that

participants of the EJoH were acting out less in their classes and at school. The Youth Coping

Inventory also illustrated improvements in the experimental group of tension management and

communication skills for those who participated in the EJoH. Finally, the thematic findings from

the qualitative interviews clearly indicated that youth found ways to better handle and process

their feelings of anger and aggression.

36

Adolescent Journey of Hope

Involvement aided in building peer relationships.

Qualitative interviews clearly illustrate that students enjoyed participating in the curricula

because they were able to interact with their peers in a safe, confidential setting. Additionally,

the adolescents reported better peer relationships in the Communities that Care survey from pre

to post-test. This finding is noteworthy considering peer relationships are a protective factor

from the negative psychological impacts of a disaster. While this outcome was not significant, it

was nearing significance and should be more closely monitored in future research through

adding more peer connectedness scales as well as comparing the group who participated in the

AJoH (experimental group) to a group who has not participated in the program (control group).

By comparing the outcomes from the two groups, an “effect size” would be available to measure

the impact on those who participated in the program versus those who did not.

Participation in the curricula supported children in understanding and normalizing emotions

associated with trauma or difficult circumstances.

The significant reductions of emotional distress scores on the Strengths and Difficulties

Questionnaire among 6th

-8th

grade males and 8th grade females coupled with qualitative

interviews indicate youth learned skills to help reduce emotional distress and positively cope

with their emotions. As research on disasters such as Hurricane Katrina, the September 11th

World Trade Center attacks, the Tsunami in Indonesia and other disaster affected areas has

found, adolescents have a higher prevalence of emotional distress symptoms in the longer-term

post-disaster period if they do not receive mental health services (Agustini, Asniar, & Matsuo,

2011; McLaughlin et al., 2009; Mijanovich & Weitzman, 2010). Given this knowledge, the

evaluation provides valuable information about the AJoH’s possible impact on reducing

emotional distress. Furthermore, it lays the groundwork for a future randomized controlled study

design measuring emotional distress symptoms in youth who participate in the AJoH.

5.2 Teen JoH

This evaluation is the first to be conducted on the Teen Journey of Hope. By evaluating the

TJoH, the evaluators were able to understand the impact of this pilot curriculum. The data

presented statistically significant (p<.05) findings between pre and post-test on knowledge-based

37

questions which are key objectives of the program such as: (1) signs of stress, (2) coping (3)

breathing exercises and (4) types of stress These findings indicate the Teen Journey of Hope is

clearly reaching its audience of older teens. Moreover, programs that target older teens have

historically been a gap in the Save the Children’s psychosocial programming; therefore, this

evaluation is the first to identify the impact of an intervention for this population.

Enhanced knowledge about stressors and coping strategies

This was a crucial finding considering research has directly linked adolescents who were

exposed to trauma and left untreated at higher risk for depressive symptoms and chronic PTSD

than those who received some form of psychological support (Goenjian et al., 2005). Providing

an outlet for teens to learn about positive ways to cope with stressors may, therefore, reduce

long-term negative psychological sequela.

Increased knowledge of types of stress

This was a noteworthy outcome of the study given that many teens experience a variety of

stressors after a disaster. The Teen Journey of Hope is a program that can reduce these feelings

and help teens identify and cope with specific types of stress they may encounter.

Future research may inquire about the long-term impacts of the program and whether the teen

program helps improve overall well-being of participants after a disaster. Moreover, subsequent

research using scales that measure stress, anxiety, self-esteem and self-efficacy would enhance

the knowledge base of the TJoH and lend to a more in-depth understanding of the programmatic

impact.

5.3 Limitations

This study is a subsequent step to evaluating the efficacy of the Journey of Hope curricula.

While results support the evidence for the curricula, some limitations must be identified. First,

the waitlist-control design was only available for the students who participated in the Elementary

Journey of Hope. Future evaluations would benefit from a control group with the older students

who participated in the Adolescent and Teen Journey of Hope to compare the program effects.

Second, there were limited self-report measures for the younger youth, therefore, specific

38

outcomes such as self-efficacy, anger management, and peer relationships were not self-reported.

Teacher report measures were distributed to identify these outcomes, however, limited responses

made analysis impossible. While a strength of the evaluation was the availability of the

disciplinary referral data, other measures of externalizing behaviors would provide a more

extensive knowledge of the programmatic impact.

Another limitation was that there were only two time points measured in the evaluation. Given

the short time period of the evaluation, the researchers were unable to measure the longitudinal

effects of the curricula. A follow-up evaluation measuring three time points would help Save the

Children understand the longer term impact of the program.

5.4 Conclusions

Despite the limitations of the evaluation, the findings lend support to the evidence of the Journey

of Hope curricula. Given the abrupt nature of disasters and the breakdown of systems and

services, it is often difficult to measure the impact of a psychosocial program (Athula et al.,

2010). The current evidence-base of mental health programs for people exposed to conflict and

disasters is weak, therefore, continued research must be completed on programs such as the

Journey of Hope that address the mental health needs of youth.

Although there is a growing collection of empirical data about evidence-based programming in

post-disaster mental health, researchers have posited that continued evaluations are needed and

further evidence should be gathered through culturally appropriate research in order to

effectively respond to the affected populations. Moreover, the research needs to be appropriate

both culturally and regionally in order to adequately address the needs of those impacted by the

disaster (Athula et al., 2010). Considering the Elementary and Adolescent Journey of Hope

curricula have been evaluated in three culturally diverse areas (New Orleans, LA; Tuscaloosa,

AL; and Christchurch, NZ) and yielded promising results in each setting with minimal

adaptation, the curricula provide a unique contribution to the field. While funding has been

time-limited for each of the evaluations, they build a basis for future funding which would solely

support more controlled longitudinal research on the curricula from entities such as Substance

Abuse and Mental Health Service Administration (SAMHSA) or National Institute of Health

39

(NIH). After a natural disaster, there is a great need for evidence-based programs to address the

emotional well-being of children, adolescents, older teens and emerging adults. The Journey of

Hope curricula help fill those needs and close the gap in psychosocial programming after a

disaster.

40

Appendix 1: Teen Journey of Hope Questionnaire

Journey of Hope Questionnaire

Pre-Test

Directions: Please answer the following questions based on how you currently feel. The questions will also be asked after the session. There are no right or wrong answers but are to help Save the Children staff assess the

impact of the program. The survey is on a 5 point scale, please indicate your agreement with the statement from 1-5 (1=Not at all, 3=Somewhat, 5=Very).

Not at all Somewhat A lot

1 2 3 4 5

1 How much awareness do I have about supports in my school?

2 How much do I know about how stress affects my body?

3 How well can I identify signs of stress?

4 How well do I understand coping strategies to handle my stress?

5 What knowledge do I have about breathing and muscle relaxation to lower stress?

6 How much stress do I currently feel?

7

How often do I consider a positive future for myself?

8 How well can I identify positive ways to manage stress?

9 How well do I know the different types of stress?

10 How important do I consider social supports (friends and family) in improving my well-being?

41

Journey of Hope Questionnaire Post-Test

Directions: Please answer the following questions based on how you currently feel. There are no right or wrong answers but are to help Save the Children staff assess the impact of the program. The survey is on a 5 point scale,

please indicate your agreement with the statement from 1-5 (1=Not at all, 3=Somewhat, 5=Very).

Not at all Somewhat A lot

1 2 3 4 5

1 How much awareness do I have about supports in my school?

2 How much do I know about how stress affects my body?

3 How well can I identify signs of stress?

4 How well do I understand coping strategies to handle my stress?

5 What knowledge do I have about breathing and muscle relaxation to lower stress?

6 How much stress do I currently feel?

7

How often do I consider a positive future for myself? 8 How well can I identify positive ways to manage

stress? 9 How well do I know the different types of stress?

10

How important do I consider social supports (friends and family) in improving my well-being?

11 How comfortable did you feel participating in the activities?

12 Did this program meet your expectations?

1. What did you like best about Journey of Hope?

2. In what ways could we improve Journey of Hope?

42

Appendix 2: Strengths and Difficulties Questionnaire

For each item, please mark the box for Not True, Somewhat True or Certainly True. It would help us if you

answered all items as best you can even if you are not absolutely certain . Please give your answers on the

basis of how things have been for you over the last six months.

Not True Somewhat True Certainly True

1. I try to be nice to other people. I

care about their feelings

2. I am restless, I cannot stay still for

long

3. I get a lot of headaches, stomach-

aches or sickness

4. I usually share with others, for

example CD’s, games, food

5. I get very angry and often lose my

temper

6. I would rather be alone than with

people of my age

7. I usually do as I am told

8. I worry a lot

9. I am helpful if someone is hurt,

upset or feeling ill

10. I am constantly fidgeting or

squirming

11. I have one good friend or more

12. I fight a lot. I can make other

people do what I want

13. I am often unhappy, depressed or

tearful

43

14. Other people my age generally

like me

Not True Somewhat True Certainly True

15. I am easily distracted, I find it

difficult to concentrate

16. I am nervous in new situations. I

easily lose confidence

17. I am kind to younger children

18. I am often accused of lying or

cheating

19. Other children or young people

pick on me or bully me

20. I often offer to help others

(parents, teachers, children)

21. I think before I do things

22. I take things that are not mine

from home, school or elsewhere

23. I get along better with adults than

with people my own age

24. I have many fears, I am easily

scared

25. I finish the work I'm doing. My

attention is good

44

Appendix 3: Save the Children Scales

Elementary Journey of Hope Questionnaire

Please let us know what you thought of the program by answering the questions to the best of your ability. The first part of the form asks you to fill out the answers on a five point scale including: always, most of the time, a little, never, and I don’t know. There are no right or wrong answers we just want to see what you thought of the Journey of Hope.

Part I

Always Most of the Time

A Little Never I don’t know

1. In this group, I learned to share my feelings o o o o o 2. In this group, I liked being part of the activities

o

o

o

o

o

3. In this group, I was respected

o

o

o

o

o

4. I helped make and follow group rules

o

o

o

o

o

5. In this group, I felt I could talk to adults

o

o

o

o

o

6. In this group, I learned about what makes me feel safe

o

o

o

o

o

7. I wanted to go to the group each time

o

o

o

o

o

45

Part 2

Yes Yes, a little

Not Really

No I don’t know

1. In this group, I learned about better ways of showing my feelings

o o o o o

2. In this group, I learned how to deal with bullying

o o o o o

3. I learned it is bad to worry o o o o o

4. In this group, I learned to show my anger without hurting anybody

o o o o o

5. I did not like the games o o o o o

6. I liked the books o o o o o

7. After this group, I learned more about me o o o o o

8. I know how to make myself feel better when I’m upset

o o o o o

What did you like BEST in the group (write your answer)

______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Is there anything you did NOT like? (write your answer)

______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

PLEASE COMPLETE:

How old are you?________ Are you a: Boy Girl

Thank You!!

46

Appendix 4: Qualitative Interview Guide

Qualitative Interview Guide

Interview Guidelines (narrative-style interviews):

Brief explanation that we are asking them what they think of the program.

*Ask them to state their name, school and who ran the group they were a part of

1. What did you do in the group?

Probe-What did you talk about?

2. What did you like about the group/program?

Probe-What’s your favorite activity?

Probe-What’s your favorite topic?

3. What didn’t you like about it?

4. What is XX (facilitator’s name) like? (probe: Are they like your teachers?)

5. Do you think anyone else should participate in this group?

Probe-Do you think any of your friends or family should participate in this program?

Probe-(If so) Why do you think others should participate in the program?

6. What did you learn?

Probe: What, if anything, did you learn about yourself?

Probe: What did you learn about other group members?

7. Which topic was the most important to you?

Probe: Are there any feelings you still have trouble with, if so can you talk about it?

8. How comfortable did you feel sharing in the group?

Probe: Did you talk a lot or a little

9. Was there anything you didn’t talk about that you think would have helped you?

10. Do you still have your journal? Do you still look at it? Have you shown it to anyone?

11. What if anything can be improved about the Journey of Hope program?

47

Appendix 5: CTC School Bonding Survey

How much do you agree with the following statements?

NO! no yes YES!

1. I like school

2. Most mornings I look forward

to school.

3. I do extra work on my own.

4. When I have an assignment to

do, I keep working on it until its

finished.

5. I like my classes this year

48

Appendix 6: Youth Coping Inventory

Please complete the following questions to the best of your ability. Remember, there are no right or wrong answers.

Read each of the statements below that describes a behavior for coping with problems. Decide how often you do each of the described behaviors when you face difficulties or feel tense

Never Hardly ever Sometimes Often Most of

the Time

1. Apologize to People

2. Talk to a teacher or counselor at school about what bothers you

3. Read

4. Get more involved in activities at school

5. Try to improve yourself (get in shape, get better grades, etc.)

6. Try to reason with parents and talk things out; compromise

7. Try to think of the good things in your life

8. Say nice things to others

9. Get angry and yell at people

10. Work hard on schoolwork or other school projects

11. Pray

12. Try, on your own, to figure out how to deal with your problems or tensions

13. Try to make your own decisions

49

Never Hardly ever Sometimes Often Most of

the Time

14. Go to church

15. Swear

16. Organize your life and what you have to do

17. Go along with parents’ requests and rules

18. Blame others for what’s going wrong

19. Tell yourself the problem is not important

20. Try to help other people solve their problems

21. Get professional counseling (not from a school teacher or school counselor)

22. Try to keep up friendships or make new friends

23. Daydream about how you would like things to be

24. Play video games

25. Let off steam by complaining to your friends

26. Say mean things to people; be sarcastic

27. Do things with your family

28. Talk with a friend about how you feel

29. Try to see the good things in a difficult situation

30. Work on a hobby you have

31. Do a physical activity (jog, bike, etc.)

50

References

Agustini, N., Asniar, I., & Matsuo, H. (2011). The prevalence of long-term post-traumatic stress

symptoms among adolescents after the tsunami in Aceh. Journal of Psychiatric and Mental

Health Nursing, 18(6), 543-549. doi: 10.1111/j.1365-2850.2011.01702.x

Arthur, M., Hawkins, J., Pollard, J., Catalano, R. & Baglioni, A (2002). Measuring risk and protective

factors for substance use, delinquency, and other adolescent problem behaviors. Evaluation

Review 26(6), 575-601.

Athula, S., Aamir, J., Leonardo, D., Aasim, A., Darryl, M., Sandya, S., . . . Chesmal, S. (2010). Ethical

Issues in post-disaster clinical interventions and research: A developing world perspective. Key

findings from a drafting and consensus generation meeting of the working group on disaster

research and ethics (WGDRE) 2007. Asian Bioethics Review, 2(2), 124-142.

Cohen, Jaycox, Walker, Mannarino, Langley, & DuClos. (2009). Treating Traumatized Children after

Hurricane Katrina: Project Fleur-de Lis™. [Article]. Clinical Child & Family Psychology Review,

12(1), 55-64. doi: 10.1007/s10567-009-0039-2

Garrett, A., Grant, R., Madrid, P., Brito, A., Abramson, D., & Redlener, I. (2007). Children and

megadisasters: lessons learned in the new millennium. [doi: DOI: 10.1016/j.yapd.2007.03.011].

Advances in Pediatrics, 54(1), 189-214.

Goenjian, A., Walling, D., Steinberg, A., Karayan, I., Najarian, L., & Pynoos, R. (2005). A prospective

study of posttraumatic stress and depressive reactions among treated and untreated adolescents 5

years after a catastrophic disaster. American Journal of Psychiatry, 162(2302-2308).

Jaycox, L., Morse, L., Tanielian, T., Stein, B. (2006). How schools can help students recover from

traumatic experiences a tool kit for supporting long-term recovery. Arlington, VA: Rand

Kar, N. (2009). Psychological impact of disasters on children: review of assessment and interventions.

World Journal of Pediatrics, 5(1), 5-11. doi: 10.1007/s12519-009-0001-x

Kataoka, S., Rowan, B., & Hoagwood, K. (2009). Bridging the divide: In search of common ground in

mental health and education research and policy. Psychiatric Services, 60(11).

51

Lofland, J., & Lofland, L. (1995). Analyzing social settings: A guide to qualitative observation and

analysis. Belmont, CA: Wadsworth, Inc.

McLaughlin, K., Fairbank, J., Gruber, M., Jones, R., Lakoma, M., Pfefferbaum, B., . . . Kessler, R.

(2009). Serious emotional disturbance among youths exposed to Hurricane Katrina 2 years post

disaster. Journal of American Academy of Child and Adolescent Psychiatry, 48(11).

Mijanovich, T., & Weitzman, B. (2010). Disaster in context: The effects of 9/11 on youth distant from the

attacks. Community Mental Health Journal, 46(6), 601-611. doi: 10.1007/s10597-009-9240-5

NASA. (2011). Tornado track in Tuscaloosa, Alabama, from

http://earthobservatory.nasa.gov/IOTD/view.php?id=50434

Ogles, B., Lunnen, K., & Bonesteel, K. (2001). Clinical significance: History, application, and current

practice. Clinical Psychology Review, 21(3), 421-446. doi: 10.1016/s0272-7358(99)00058-6

Richards, T. J., & Richards, L. (1994). Using computers in qualitative research. In N. Denzin & Y.

Lincoln (Eds.), Handbook of qualitative research (pp. 445-462). London: Sage Publications.

Roberts, A., & Everly, G. (2006). A meta-analysis of 36 crisis intervention studies. Brief Treatment and

Crisis Intervention, 6(1). doi: 10.1093/brief-treatment/mhj006

Strauss, A. (1987). Qualitative analysis for social scientists. Cambridge: Cambridge University Press.

Walsh, F. (2007). Traumatic loss and major disasters: strengthening family and community resilience.

Fam Process, 46(2), 207-227.

Wang, Y., Nomura, Y., PatHorenczyk, R., Doppelt, O., Abramovitz, R., Brom, D., & Chemtob, C.

(2006). Association of direct exposure to terrorism, media exposure to terrorism, and other

trauma with emotional and behavioral problems in preschool children. Annals of the New York

Academy of Sciences, 1094(1), 363-368. doi: 10.1196/annals.1376.051

Williams, R., Alexander, D., Bolsover, D., & Bakke, F. (2008). Children, resilience and disasters: recent

evidence that should influence a model of psychosocial care. Current Opinion in Psychiatry,

21(4), 338-344 310.1097/YCO.1090b1013e328305b328306e328304.

52