evaluation of the project p.a.t.h.s. based on students...

Title Evaluation of the project P.A.T.H.S. based on students' weeklydiaries: Findings from eight datasets

Author(s) Shek, DTL; Sun, RCF

Citation TheScientificWorldJournal, 2012, v. 2012, article no. 354254

Issued Date 2012

URL http://hdl.handle.net/10722/152905

Rights This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

The Scientific World JournalVolume 2012, Article ID 354254, 14 pagesdoi:10.1100/2012/354254

The cientificWorldJOURNAL

Research Article

Evaluation of the Project P.A.T.H.S. Based on Students’ WeeklyDiaries: Findings from Eight Datasets

Daniel T. L. Shek1, 2, 3, 4, 5 and Rachel C. F. Sun6

1 Department of Applied Social Sciences, The Hong Kong Polytechnic University Kowloon, Hong Kong2 Public Policy Research Institute, The Hong Kong Polytechnic University Kowloon, Hong Kong3 Department of Social Work, East China Normal University, Shanghai, China4 Kiang Wu Nursing College of Macau, Macau5 Division of Adolescent Medicine, Department of Pediatrics, Kentucky Children’s Hospital, University of Kentucky College of Medicine,Lexington, KY 40506-9983, USA

6 Faculty of Education, The University of Hong Kong Pokfulam, Hong Kong

Correspondence should be addressed to Daniel T. L. Shek, [email protected]

Received 5 November 2011; Accepted 25 December 2011

Academic Editor: Joav Merrick

Copyright © 2012 D. T. L. Shek and R. C. F. Sun. This is an open access article distributed under the Creative CommonsAttribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work isproperly cited.

This paper aims to investigate the effectiveness of the Tier 1 Program of the Project P.A.T.H.S. (positive adolescent training throughholistic social programmes) based on eight datasets collected between 2005 and 2009. A total of 1,138 students who participatedin the program were randomly invited (from the whole grade or in some classes) to write a piece of journal in the form of aweekly diary in order to reveal their perceptions and feelings regarding the program and the perceived benefits of the program.Based on an integration of findings from different databases, results showed that the respondents generally (1) had positive viewson the program, (2) had positive views on the instructors, and (3) perceived that they had acquired competencies at the societal,school, familial, interpersonal, and personal levels after joining the program. Acknowledging the limitations of diaries, the presentqualitative findings provide support for the effectiveness of the Tier 1 Program of the Project P.A.T.H.S. in Hong Kong.

1. Introduction

The purpose of this paper is to report evaluation findingsof a positive youth development program in Hong Kong(Project P.A.T.H.S., i.e., positive adolescent training throughholistic social programmes) based on weekly diaries writtenby students participating in the program. The personal diaryis an important source of information to understand thesocial reality. For example, Anne Frank’s diary documentsthe thoughts, feelings, and experiences of a Jewish teenagegirl who had fled and hid in Amsterdam with her familywhen Nazi forces conquered The Netherlands. In view ofthe invaluable information gathered from people’s diaries,diaries for analyzing human behavior, discovering history,or evaluating a program represent an attractive researchapproach. In the context of social sciences research, diariesare commonly used in nonexperimental research to collectdata on human behavior. For example, Leigh [1] asked

participants to record their time, quantity, and type ofdrinking and sexual activity over 10 weeks in order toexamine the temporal relationships of alcohol consumptionto sexual behavior. Behavioral diaries have also been used inbehavioral modification programs [2]. There are studies tosuggest the strengths and weaknesses of using daily diariesas a research strategy [3, 4], and the debate has continued inthe literature [5, 6].

In the field of education, it is not uncommon to find thata teacher would keep reflective journals as a means to assesspersonal teaching performance, identify one’s pedagogicalstrengths and weaknesses, and evaluate the effectiveness ofthe changes implemented in teaching [7]. It is believed thata reflective journal “suggests question, identifies new areas toexplore, reveals meaningful absences, and uncovers recurringpatterns” ([8], page 16). Therefore, some studies utilizedteachers’ reflective diaries as a kind of data collection methodto investigate the essential components of effective teaching

2 The Scientific World Journal

practice or pedagogy, and to examine ways of contributing toprofessional development [9].

Similarly, in contrast to the traditional learning modelwhere students mainly take a passive role in receivingwhat their teachers’ transmit or reciting the informationfrom books, students are perceived as active learners whoconstruct their personal knowledge based on their ownexperiences in reflective learning models, where they areencouraged to be autonomous learners who reflect on theirlearning and to have more awareness on their learning pro-cesses, competencies, and limitations. Therefore, reflectiveobservation is highlighted to be an indispensable process intranslating concrete experiences to concepts and actions inthe experiential learning process [10], in which keeping adiary is considered as an opportunity for personal reflectionthat, in turn, enhances students’ autonomy and develops anownership of their learning process [11].

In order to guide students to become autonomouslearners, students studying in human services programs werecommonly asked to write diaries or reflective journals in dif-ferent ways. First, some courses simply aimed to encouragestudents to be critical and reflective learners. For example,in a module on management education, students were askedto write eight diaries with predesigned topics throughoutthe course [12]. Second, some courses were targeted towardsstrengthening the students’ ability to integrate their courselearning into actual field settings. For example, social workstudents were required to write reflective journals withpredesigned questions at several stages of their field work,on top of other written assignments [13]. Third, studentswere guided to be determinant masters of their future.For example, students in a health-discipline course wereasked to write a journal with no specific guidelines ontheir experiences of their practices in rural areas, so as toidentify the positive and negative aspects of rural practicesand develop their interest in working in the rural areas in thefuture [14]. Chaloner [15] affirmed that “using a personaljournal implies that the learner will construct a memory ofthe learning experience and how it can be applied to theworkplace by using a personal narrative. Through such anindividualized and personal approach, it is hoped that thelearner will be motivated enough to assume the autonomythat he/she is being offered” (page 22).

Besides understanding the subjective world views of theinformants, a survey of the literature in the field of evaluationshows that there is an increasing number of studies usingdiaries as a tool for evaluation. Wagner [16] used studentjournals to evaluate the effectiveness of a course. Follicket al. [17] evaluated a daily activity diary for chronic painpatients. Travers [18] examined the use of a reflective diarymethodology for exploring the lived experiences of stressand coping among 30 college students. Cohen et al. [19]examined the development and use of an online diary in theevaluation of a preventive care program that provided real-time communication between the evaluator and participants.Knowles and Tarrier [20] evaluated diary intervention onlevels of anxiety and depression in a group of intensive careunit survivors and demonstrated the therapeutic effects ofdiaries.

The main purpose of this paper is to report evaluationfindings of the Project P.A.T.H.S. using student diaries as anevaluation method. The project P.A.T.H.S. is a positive youthdevelopment program that attempts to promote holisticdevelopment among adolescents in Hong Kong, with aninitial earmarked grant of HK$400 million by The HongKong Jockey Club Charities Trust. The Project is divided intotwo tiers of programs. The Tier 1 Program is a universalcurriculum-based program for secondary 1 to 3 students inparticipating schools and the Tier 2 Program is a selectiveprogram for students with greater psychosocial needs. Incontrast to mainstream approaches that focus on tacklingyouth developmental problems, the field of positive youthdevelopment focuses on enabling the talents, strengths,interests, and future potentials in children and adolescents.Roth et al. [21] pointed out that positive youth developmentprograms are commonly regarded as “programs that provideopportunities and support to help youth gain the competen-cies and knowledge they need to meet the increasing chal-lenges they will face as they mature” (page 423). Althoughmany western programs have been developed to promotepositive development in children and adolescents, not all ofthem are successful in promoting adolescent development.For example, Catalano et al. [22] reviewed 77 programson positive youth development, and their review showedthat only 25 programs were successful. Obviously, evaluationis an important issue to be considered when assessing theeffectiveness of positive youth development programs.

To understand the program effects of the ProjectP.A.T.H.S., several complementary program evaluationstrategies, including both quantitative and qualitativemethodologies, have been adopted [23]. These includedobjective outcome evaluation, subjective outcome evalua-tion, process evaluation, interim evaluation, and qualitativeevaluation based on focus groups, in-depth interviews, andcase studies. Through the adoption of different evaluationstrategies, there is multiple evidence supporting the effec-tiveness of the Tier 1 Program of the Project P.A.T.H.S.based on different types of data collected from differentparticipants utilizing different methods. For example, bothquantitative findings [24, 25] and qualitative findings [26,27] based on different studies consistently suggest that theProject P.A.T.H.S. is helpful in promoting the psychosocialcompetencies of the program participants.

To further understand the perceived program effects ofthe Tier 1 Program, evaluation findings based on weeklydiaries written by students are reported in this paper.Actually, there are published papers on the evaluation of theProject P.A.T.H.S. based on weekly diaries [28, 29]. In thispaper, the findings based on weekly diaries in eight datasetswere aggregated in order to give a picture regarding theeffectiveness of the Project P.A.T.H.S. in Hong Kong. In short,the purpose of this paper is to give a descriptive picture onthe views and experiences of the program participants.

2. Methods

2.1. Participants and Procedures. From 2005 to 2009, a totalof 244 secondary schools participated in the P.A.T.H.S.

The Scientific World Journal 3

Table 1: Number of respondents writing weekly diaries.

S1 S2 S3

Study 12005/2006

EIP

Study 22006/2007

FIP

Study 32008/2009

FIP

Study 42006/2007

EIP

Study 52007/2008

FIP

Study 62008/2009

FIP

Study 72007/2008

EIP

Study 82008/2009

FIP

Total schools that joinedP.A.T.H.S.

52 207 197 49 196 198 48 167

(i) 10-hour program 23 95 104 27 113 110 29 104

(ii) 20-hour program 29 112 93 22 83 88 19 63

Total schools that joined thisstudy

4 6 2 4 6 2 4 6

(i) 10-hour program 1 0 0 0 0 0 0 0

(ii) 20-hour program 3 6 2 4 6 2 4 6

No. of diaries collected 95 216 58 260 164 62 132 151

Percentage of informants inthe selected schools

15% 19% 15% 36% 15% 16% 24% 16%

Note: S1: secondary 1 level; S2: secondary 2 level; S3: secondary 3 level; EIP: experimental implementation phase; FIP: full implementation phase.

Project. Among all the participating schools, 12 schoolsthat joined the secondary 1 level of the Project, 12 schoolsthat joined the secondary 2 level, and 10 schools thatjoined the secondary 3 level were randomly selected andinvited to take part in eight research studies. In each ofthe selected schools, the school contact person was askedto randomly invite some students (from the whole gradeor in some classes) to write a piece of journal in the formof a weekly diary in order to reveal their perceptions andfeelings after joining the Tier 1 Program of the project. Thestudents were informed of the purpose and confidentialityof this study, and their consent for participation was soughtprior to data collection. The title of the reflective journalwas “Participating in the Tier 1 Program of the ProjectP.A.T.H.S.: Experiences and Feelings.” The reflection wasexpected to be not less than 200 words in Chinese, and itshould be related to the students’ experiences, feelings, andcomments in connection with their participation in the Tier1 Program. The students could either complete it at homeor during class time. The total number of students’ diariesreceived was 1,138, comprising 369 written by secondary1 students, 486 written by secondary 2 students, and 283written by secondary 3 students (see Table 1). As the caseswere randomly selected and the response rate was 100%,the recruited sample could be regarded as respectable. Nosecondary 1 students were recruited in the 2007/2008 schoolyear because several cohorts of data had been collected.

While the basic research design of the studies generatingthe datasets was primarily qualitative in nature (i.e., writtencomments on the program, instructors, and benefits), itis possible to use quantitative methods (such as countingand frequency analyses) to create a descriptive profile ofthe findings [28]. In the present study, the secondarydata analysis approach was adopted by integrating andsynthesizing the data generated from the different databases.

2.2. Data Analyses. In each of the eight studies, the datawere analyzed using general qualitative analyses techniques

involving three steps [30]. First, relevant raw codes weredeveloped for words, phrases, and/or sentences that formedmeaningful units at the raw responses level. Second,the codes were further combined to reflect higher-orderattributes at the category of codes level (i.e., themes andsubthemes). Third, the categories of codes were furtheranalyzed to reveal the broader themes at the thematiclevel. Three main themes were identified from the weeklydiaries: views on the program, views on the instructors,and perceived benefits of the program. Both intra- andinterrater reliabilities on the coding regarding these viewswere calculated, and the results were satisfactory (85–100%).In the present analyses, because the researchers designedthe program of the Project P.A.T.H.S., they were consciousof their own biases and expectations on the program tobe effective. This paper reports the analyses based onthe aggregated findings of these eight studies. The majorcategories and subcategories of responses are presented in thetables, and narrative illustrating the themes can be seen in theSection 3.

3. Results

Responses regarding the students’ perception of the programand the instructor, as well as the perceived benefits of theprogram, are presented in this paper. The integrated findingsare presented in Tables 2–4. As shown in Table 2, there wasa total of 2,290 meaningful units regarding students’ viewson the Tier 1 Program implemented from 2005 to 2009. Theresponses were categorized into “overall impression,” “pro-gram content,” “learning process,” “program arrangement,”and “other comments.” About 81% of the total responses wascoded as positive. Most of the respondents had positive viewson the “overall impression,” followed by “learning process”and “program content”. There were 85.4, 77.2, and 83.1% ofsecondary 1, 2, and 3 students who reported positively onthe program, respectively, although some negative commentswere recorded. Three cases illustrating the respondents’


Ta

ble

2:V

iew

son

the

prog

ram

acro

ssst

udy

1to

stu

dy8.

Vie

ws

onth

epr

ogra

mR

espo

nse

sS1

S2S3

Tota

lSt

udy

120

05/0

6E

IPSt

udy

220

06/0

7FI

PSt

udy

320

08/0

9FI

PSt

udy

420

06/0

7E

IPSt

udy

520

07/0

8FI

PSt

udy

620

08/0

9FI

PSt

udy

720

07/0

8E

IPSt

udy

820

08/0

9FI

P

Ove

rall

impr

essi

on

Posi

tive

:(e

.g.,

mea

nin

gfu

l,ex

celle

nt,

ben

efici

al,a

nd

liked

the

prog

ram

)

2611

643

274

3750

5910

671

1

Neg

ativ

e:(e

.g.,

mea

nin

gles

s,ba

d,di

sapp

oin

ted,

and

not

inte

rest

ing)

01

518

97

28

50

Neu

tral

:(e

.g.,

mix

edbl

essi

ngs

,fai

r,n

otdi

ffer

ent

from

oth

erpr

ogra

ms,

and

sim

ilar

toth

esu

bjec

tsin

prim

ary

sch

ools

)

30

03

01

03

10

Subt

otal

2911

748

295

4658

6111

777

1

Pro

gram

con

ten

t

Posi

tive

:(e

.g.,

prac

tica

l,cr

eati

ve,r

ich

con

ten

t,an

dea

sily

un

ders

tood

)27

2535

6831

4378

6637

3

Neg

ativ

e:(e

.g.,

disl

iked

grow

thpu

zzle

s,bo

rin

gco

nte

nt,

too

man

yw

orks

hee

ts,a

nd

diss

atis

fied

wit

hth

eac

tivi

ties

)

630

108

2649

2421

174

Neu

tral

:(e

.g.,

com

men

tson

teac

hin

gm

ater

ials

)0

00

00

00

22

Subt

otal

3355

4576

5792

102

8954

9

Lear

nin

gpr

oces

s

Posi

tive

:(e

.g.,

fun

ny,i

nte

rest

ing,

hap

py,

and

acti

vepa

rtic

ipat

ion

)34

7025

107

6226

4810

247

4

Neg

ativ

e:(e

.g.,

bori

ng,

un

invo

lved

,poo

rle

arn

ing

atm

osph

ere,

and

non

sen

se)

518

296

03

333

160

Subt

otal

3988

2720

362

2981

105

634


Ta

ble

2:C

onti

nu

ed.

Vie

ws

onth

epr

ogra

mR

espo

nse

sS1

S2S3

Tota

lSt

udy

120

05/0

6E

IPSt

udy

220

06/0

7FI

PSt

udy

320

08/0

9FI

PSt

udy

420

06/0

7E

IPSt

udy

520

07/0

8FI

PSt

udy

620

08/0

9FI

PSt

udy

720

07/0

8E

IPSt

udy

820

08/0

9FI

P

Pro

gram

arra

nge

men

t

Posi

tive

:(e

.g.,

good

deliv

ery

mod

e,go

odar

ran

gem

ent,

and

rew

ard

sch

eme)

00

50

110

026

42

Neg

ativ

e:(e

.g.,

inad

equ

ate

reso

urc

es,p

oor

arra

nge

men

t,in

suffi

cien

tti

me,

and

bad

clas

sroo

mm

anag

emen

t)

00

00

54

67

22

Subt

otal

00

50

614

633

64

Oth

erco

mm

ents

Posi

tive

2355

797

95

4117

254

Neg

ativ

e0

40

60

00

111

Neu

tral

00

07

00

00

7Su

btot

al23

597

110

95

4118

272

Gra

nd

tota

lPo

siti

ve(%

)11

0(8

8.7)

266

(83.

4)11

5(8

7.1)

546

(79.

8)14

0(7

7.8)

134

(67.

7)22

6(7

7.7)

317

(87.

6)18

54N

egat

ive

(%)

11(8

.9)

53(1

6.6)

17(1

2.9)

128

(18.

7)40

(22.

2)63

(31.

8)65

(22.

3)40

(11)

417

Neu

tral

(%)

3(2

.4)

00

10(1

.5)

01

(0.5

)0

5(1

.4)

19To

tal(

%)

124

(100

)31

9(1

00)

132

(100

)68

4(1

00)

180

(100

)19

8(1

00)

291

(100

)36

2(1

00)

2290

Tota

lin

each

grad

ePo

siti

ve(%

)49

1(8

5.4)

820

(77.

2)54

3(8

3.1)

1854

Neg

ativ

e(%

)81

(14.

1)23

1(2

1.8)

105

(16.

1)41

7N

eutr

al(%

)3

(0.5

)11

(1)

5(0

.8)

19To

tal(

%)

575

(100

)10

62(1

00)

653

(100

)22

90


Ta

ble

3:V

iew

son

the

inst

ruct

orac

ross

stu

dy1

tost

udy

8.

Vie

ws

onth

ein

stru

ctor

Res

pon

ses

S1S2

S3

Tota

lSt

udy

120

05/2

006

EIP

Stu

dy2

2006

/200

7FI

P

Stu

dy3

2008

/200

9FI

P

Stu

dy4

2006

/200

7E

IP

Stu

dy5

2007

/200

8FI

P

Stu

dy6

2008

/200

9FI

P

Stu

dy7

2007

/200

8E

IP

Stu

dy8

2008

/200

9FI

P

Ove

rall

impr

essi

on

Posi

tive

:(e

.g.,

nic

e,lik

edin

stru

ctor

s,co

uld

mot

ivat

est

ude

nts

,an

dth

ankf

ult

oin

stru

ctor

s)

28

041

26

2216

97

Neg

ativ

e:(e

.g.,

disg

ust

ing,

stri

ct,

alw

ays

blam

ing,

and

inco

nsi

sten

tfa

cilit

atio

nsk

ills

betw

een

inst

ruct

ors)

31

03

00

00

7

Subt

otal

59

044

26

2216

104

Teac

hin

gpe

rfor

man

ce

Posi

tive

:(e

.g.,

exce

llen

t,w

ell

prep

ared

,pro

fess

ion

al,a

nd

shar

edow

nex

peri

ence

s)

928

1751

2817

2745

222

Neg

ativ

e:(e

.g.,

bori

ng

teac

hin

g,po

orcl

assr

oom

man

agem

ent,

poor

teac

hin

gsk

ills,

and

tau

ght

ina

hurr

ym

ann

er)

03

04

30

231

34

Subt

otal

931

1755

3117

5046

256

Teac

hin

gat

titu

de

Posi

tive

:(e

.g.,

enth

usi

asti

c,de

vote

d,w

illin

g/ab

leto

hel

pst

ude

nts

,an

dpa

tien

t)

57

440

1012

2757

162

Subt

otal

57

440

1012

2757

162

Oth

ers

Posi

tive

33

015

00

40

25N

egat

ive

00

03

00

20

5N

eutr

al0

02

20

20

06

Subt

otal

33

220

02

60

36G

ran

dto

tal

Posi

tive

(%)

19(8

6)46

(92)

21(9

1.3)

147

(92.

4)40

(93)

35(9

4.6)

80(7

6.2)

118

(99.

2)50

6N

egat

ive

(%)

3(1

4)4

(8)

010

(6.3

)3

(7)

025

(23.

8)1

(0.8

)46

Neu

tral

(%)

00

2(8

.7)

2(1

.3)

02

(5.4

)0

06

Tota

l(%

)22

(100

)50

(100

)23

(100

)15

9(1

00)

43(1

00)

37(1

00)

105

(100

)11

9(1

00)

558

Tota

lin

each

grad

ePo

siti

ve(%

)86

(90.

5)22

2(9

2.9)

198

(88.

4)50

6

Neg

ativ

e(%

)7

(7.4

)13

(5.4

)26

(11.

6)46

Neu

tral

(%)

2(2

.1)

4(1

.7)

06

Tota

l(%

)95

(100

)23

9(1

00)

224

(100

)55

8


Ta

ble

4:Pe

rcei

ved

prog

ram

effec

tive

nes

sac

ross

stu

dy1

tost

udy

8.

Eco

logi

call

evel

Subd

omai

ns

Des

crip

tion

s

S1S2

S3

Tota

lSt

udy

120

05/0

6E

IPSt

udy

220

06/0

7FI

PSt

udy

320

08/0

9FI

PSt

udy

420

06/0

7E

IPSt

udy

520

07/0

8FI

PSt

udy

620

08/0

9FI

PSt

udy

720

07/0

8E

IPSt

udy

820

08/0

9FI

P

Soci

etal

Soci

aln

orm

sFo

ster

edpr

osoc

ialn

orm

sor

invo

lvem

ent

326

27

98

07

62

Soci

alre

spon

sibi

lity

and

affai

rs

En

han

ced

know

ledg

eab

out

Ch

ina

and

Hon

gK

ong

69

01

00

00

16

En

han

ced

know

ledg

eab

out

publ

icaff

airs

10

00

00

00

1

Rec

ogn

ized

citi

zen’

sre

spon

sibi

litie

s4

100

30

00

017

Subt

otal

1445

211

98

07

96

Sch

ool

Sch

ool

belo

ngi

ngn

ess

Incr

ease

dbe

lon

gin

gnes

sto

sch

ool

00

00

00

10

1

Subt

otal

00

00

00

10

1

Fam

ilial

Fam

ilyre

lati

onsh

ips

En

han

ced

rela

tion

ship

sw

ith

pare

nts

and

oth

erfa

mily

mem

bers

522

1912

104

111

84

Subt

otal

522

1912

104

111

84

Inte

rper

son

alG

ener

alin

terp

erso

nal

com

pet

ence

En

han

ced

team

spir

it9

150

180

00

042

Mad

efr

ien

ds7

150

60

00

028

Got

alon

gw

ith

oth

ers

1965

041

00

00

125

En

han

ced

peer

rela

tion

ship

028

110

2312

1217

103

En

han

ced

rela

tion

ship

wit

hin

stru

ctor

s/so

cial

wor

kers

02

00

45

48

23

Abl

eto

diff

eren

tiat

ebe

twee

nh

ealt

hyan

du

nh

ealt

hyfr

ien

ds7

220

24

00

035

Abl

eto

han

dle

love

affai

rs0

00

40

00

04

Stre

ngt

hen

edso

cial

bon

din

g0

00

220

08

030

En

han

ced

soci

alco

mp

eten

ce0

170

210

00

038

Tota

lin

subc

ateg

ory

4216

411

114

3117

2425

428

Spec

ific

inte

rper

son

alco

mp

eten

ceIm

prov

edin

terp

erso

nal

skill

s0

014

08

70

1948

Impr

oved

com

mu

nic

atio

nsk

ills

330

821

1615

98

110

En

han

ced

pres

enta

tion

abili

ty0

00

150

00

015

Lear

ned

toac

cept

,car

e,tr

ust

,re

spec

t,an

du

nde

rsta

nd

oth

ers

338

1519

117

1318

124


Ta

ble

4:C

onti

nu

ed.

Eco

logi

call

evel

Subd

omai

ns

Des

crip

tion

s

S1S2

S3

Tota

lSt

udy

120

05/0

6E

IPSt

udy

220

06/0

7FI

PSt

udy

320

08/0

9FI

PSt

udy

420

06/0

7E

IPSt

udy

520

07/0

8FI

PSt

udy

620

08/0

9FI

PSt

udy

720

07/0

8E

IPSt

udy

820

08/0

9FI

PP

rom

oted

coop

erat

ion

wit

hot

her

s0

44

019

110

442

Incr

ease

dw

illin

gnes

sto

expr

ess

00

00

30

30

6To

tali

nsu

bcat

egor

y6

7241

5557

3035

4934

5Su

btot

al48

236

5216

988

4759

7477

3

Pers

onal

Gen

eral

gain

sP

rom

oted

stu

den

ts’

deve

lopm

ent

00

20

324

3510

83

Acq

uir

edsk

ills/

know

ledg

e14

266

3128

90

2914

3Le

arn

edso

met

hin

gu

sefu

lan

dap

plic

able

toda

ilylif

e13

180

00

00

435

Hel

pfu

lto

stu

den

ts0

013

04

80

1742

Posi

tive

chan

ges

onst

ude

nts

00

00

00

09

9To

tali

nsu

bcat

egor

y27

4421

3164

2135

6931

2B

ehav

iora

lco

mp

eten

ceIm

prov

edre

fusa

lski

lls0

110

00

00

011

Impr

oved

deci

sion

-mak

ing

skill

s0

00

00

70

815

Lear

ned

fin

anci

alm

anag

emen

tsk

ills

00

00

00

53

8

En

han

ced

tim

em

anag

emen

tsk

ills

00

00

10

20

3

En

han

ced

lead

ersh

ipsk

ills

00

00

00

10

1G

ener

albe

hav

iora

lcom

pet

ence

00

20

153

84

32To

tali

nsu

bcat

egor

y0

112

016

1016

1570

Pers

onal

Cog

nit

ive

com

pet

ence

Pro

mot

edra

tion

alan

dan

alyt

ical

thin

kin

g5

180

260

00

049

En

han

ced

crea

tivi

ty0

20

02

00

04

Impr

oved

prob

lem

-sol

vin

gsk

ills

726

1322

147

915

113

Gen

eral

cogn

itiv

eco

mp

eten

ce0

04

622

513

2070

Tota

lin

subc

ateg

ory

1246

1754

3812

2235

236


Ta

ble

4:C

onti

nu

ed.

Eco

logi

call

evel

Subd

omai

ns

Des

crip

tion

s

S1S2

S3

Tota

lSt

udy

120

05/0

6E

IPSt

udy

220

06/0

7FI

PSt

udy

320

08/0

9FI

PSt

udy

420

06/0

7E

IPSt

udy

520

07/0

8FI

PSt

udy

620

08/0

9FI

PSt

udy

720

07/0

8E

IPSt

udy

820

08/0

9FI

PM

oral

com

pet

ence

Cu

ltiv

ated

per

son

alet

hic

s/vi

rtu

es2

66

250

30

1860

En

han

ced

resp

onsi

bilit

y0

00

70

00

07

En

han

ced

empa

thy

03

06

00

00

9P

rom

oted

the

idea

ofeq

ual

ity

and

fair

nes

s3

00

00

00

03

Abl

eto

dist

ingu

ish

betw

een

righ

tan

dw

ron

g5

543

190

30

791

Abl

eto

corr

ect

bad

hab

its/

defe

cts

414

03

00

00

21

Cu

ltiv

ated

prop

erat

titu

des

tow

ards

sex

and

love

rela

tion

ship

00

00

40

410

18

Gen

eral

mor

alco

mp

eten

ce0

00

04

015

625

Tota

lin

subc

ateg

ory

1477

960

86

1941

234

Em

otio

nal

com

pet

ence

En

han

ced

emot

ion

alm

anag

emen

t/ex

pres

sion

1157

2728

99

1017

168

Tota

lin

subc

ateg

ory

1157

2728

99

1017

168

Bel

iefs

inth

efu

ture

Pro

mot

edbe

liefs

inth

efu

ture

00

00

01

512

18

Ass

ista

nce

inca

reer

plan

nin

g0

00

00

00

44

Faci

litat

edgo

alse

ttin

g6

234

295

49

989

Tota

lin

subc

ateg

ory

623

429

55

1425

111

Res

ilien

ceC

ult

ivat

edre

silie

nce

00

424

1315

1020

86O

ptim

ism

15

09

04

00

19W

ays

tofa

cead

vers

ity

012

00

00

00

12To

tali

nsu

bcat

egor

y1

174

3313

1910

2011

7


Ta

ble

4:C

onti

nu

ed.

Eco

logi

call

evel

Subd

omai

ns

Des

crip

tion

s

S1S2

S3

Tota

lSt

udy

120

05/0

6E

IPSt

udy

220

06/0

7FI

PSt

udy

320

08/0

9FI

PSt

udy

420

06/0

7E

IPSt

udy

520

07/0

8FI

PSt

udy

620

08/0

9FI

PSt

udy

720

07/0

8E

IPSt

udy

820

08/0

9FI

PPo

siti

vese

lfFa

cilit

ated

per

son

algr

owth

1142

034

00

00

87D

evel

oped

clea

ran

dpo

siti

veid

enti

ty0

00

00

10

12

En

han

ced

self

-effi

cacy

00

42

33

05

17E

nh

ance

dse

lf-c

onfi

den

ce3

270

92

314

967

En

han

ced

self

-det

erm

inat

ion

014

015

21

42

38Fa

cilit

ated

self

-un

ders

tan

din

g18

417

2818

530

1316

0D

evel

oped

pote

nti

als/

wid

ened

hor

izon

00

00

00

07

7

Tota

lin

subc

ateg

ory

3212

411

8825

1348

3737

8C

her

ish

ing

life

Lear

ned

toch

eris

hlif

e1

30

419

82

1377

Cu

ltiv

ated

posi

tive

life

view

s/le

arn

edlif

epr

inci

ples

00

30

014

039

56

Lear

ned

totr

easu

rep

eopl

ean

dth

ings

121

012

00

90

43

Self

-refl

ecti

on3

97

93

311

1863

Pro

mot

edre

flec

tion

oflif

e5

50

2812

130

871

Tota

lin

subc

ateg

ory

1038

1090

2438

2278

310

Subt

otal

113

437

105

413

202

133

196

337

1936

Oth

ers

Posi

tive

com

men

tsIn

tere

stin

lear

nin

g1

40

102

00

017

Stim

ula

tin

g2

00

00

00

02

Fru

itfu

l10

00

00

00

010

Aca

dem

icim

prov

emen

ts0

00

00

00

44

Hea

lthy

livin

gst

yle

00

012

20

00

14O

ther

s10

84

80

60

440

Tota

lin

subc

ateg

ory

2312

430

46

08

87N

egat

ive

com

men

tsP

rogr

amw

asn

oteff

ecti

ve0

01

00

20

03

Tota

lin

subc

ateg

ory

00

10

02

00

3

Oth

ers

Neu

tral

com

men

tsC

ould

not

hel

pm

uch

onst

ude

nts

’dev

elop

men

t0

00

00

03

03

Oth

ers

00

00

00

03

3To

tali

nsu

bcat

egor

y0

00

00

03

36

Subt

otal

2312

530

48

311

96G

ran

dto

tal

203

752

183

635

313

200

260

440

2986


perceptions of the program at each grade are presentedbelow.

Student A (Secondary 1, 2006/2007). “This is my first timejoining the P.A.T.H.S. Project. I learned about emotion,choice and other knowledge about development from theprogram . . . I personally think that this program has greatimpact on me and my classmates. We could learn via theworksheet activities as well as from the advice given by theclassmates. What we learned are closely related to realitywhich could be applicable to daily life. Therefore, I think theprogram is very meaningful.”

Student B (Secondary 2, 2007/2008). “After completion ofthe Tier 1 Program in the first semester, I felt I becamemore mature in interpersonal communication and personalgrowth. We studied various problems such as teenagers’learning attitudes and hedonism during class. Throughgames or group discussion, we learned how to analyze andacquire correct information, and how to face difficultiesand remain positive under adverse circumstances. I like thisprogram very much because all the program content isclosely related to common teenage problems, such as bodyshaping and substance abuse, which is closely related to ourdaily life. If we do not have good virtues and healthy mind,we will follow the wrong track easily. Therefore, the programserves as not only a light in the dark but also a teacher whoteaches and guides us to the right way.”

Student C (Secondary 3, 2008/2009). “Fortunately, I joinedthe Project P.A.T.H.S. from Secondary 1 to Secondary 3.This is one of my favorite subjects. The atmosphere duringthe class was relaxing that made the subject become moreacceptable. Moreover, I really liked the design of the program.Teachers and social workers used multiple resources forteaching in order to guide the students to think fromdifferent angles and make judgment. Problems such as schoolorientation, conflicts with parents and peer relationship thattroubled me a lot were discussed in the class. I could solvethese problems by applying the skills I learned from the class.Obviously the project works.”

There were 558 responses regarding “views on instruc-tors,” which were categorized into “overall impression,”“teaching performance,” “teaching attitude,” and “others”(see Table 3). About 90% was positive responses. Most ofthe respondents had positive views on instructors’ “teachingperformance.” Several cases illustrating the respondents’views on the instructors are shown below.

Student D (Secondary 1, 2006/2007). “I remembered whenI first joined the Project P.A.T.H.S., I felt very excited . . .Miss Wong was our instructor. Although the topics of eachP.A.T.H.S. session were different, the goal was the same—to help students to develop good interpersonal relationship.The instructor would encourage the students to share theiropinions on a topic and then asked other students to discussor analyze each student’s viewpoints. The instructor wouldalso arrange some activities such as group discussion and

role-play. These activities promoted the relationship amongclassmates in only one year’s time.”

Student E (Secondary 3, 2008/2009). “There was a period oftime when I lost self-confidence due to a failure. I kept myselfin the corner of my world and could not find a way out untilI heard some sharing from the instructor. She said, “Life islike an obstacle race. No matter what obstacles you encounterduring the race, once you pass the finishing line, you willhave victory.” I suddenly realized that self-pitying did nothelp anything. Rather, I should regain my strength to facenew challenges. The words from the instructor become mymotto, which drives me to keep moving forward.”

The perceived benefits of the program to the students areshown in Table 4. The 2,986 meaningful units were catego-rized into six main categories (i.e., societal, school, familial,interpersonal, personal levels of competence, and others).Most of the responses were related to the effectiveness of theprogram on a personal level (n = 1,936), particularly in thesubcategory of “positive self” (n = 378, e.g., “enhanced self-understanding”), followed by “general gains” (n = 312, e.g.,“acquired skills/knowledge”) and “cherishing life” (n = 310,e.g., “promoted reflection of life”). In addition, many respon-dents reported that they acquired interpersonal competence(n = 773), which could be categorized into “general inter-personal competence” (e.g., “got along with others”) and“specific interpersonal competence” (e.g., “learned to accept,care, trust, respect, and understand others”). Some narrativesbased on the respondents revealed that the program wasperceived positively by the program participants.

Student F (Secondary 1, 2006/2007). “I learned much usefulknowledge in this program. Firstly, I learned many interper-sonal skills such as how to get along with classmates, teachers,friends, senior people and family members. I also learnedto appreciate others and myself. I knew myself much morethan before. By realizing my strengths and weaknesses, I wasable to improve myself. In addition, I learned to distinguishbetween what I should do and what I should not. Mostimportant of all, the program enabled me to meet newfriends and improve my communication skills. All these skillswill be very useful for my future development.”

Student G (Secondary 2, 2007/2008). “When I looked back,I really learned a lot of things. I felt I truly grew up. Infamilial aspect, I understood that family members shouldmutually help each other and care for one another. I need tohelp doing the family chores when I have time. As parentshave been taking care of us whole-heartedly, we have torepay and be thankful to them. I learned the importance ofcommunication. Now I would spend more time to chat withmy parents and share my feelings with them. In doing so,my parents were reassured and became less worried aboutme . . . In social aspect, I knew that we should follow socialdisciplines such as keeping the voice down in public area toavoid interrupting other people . . . In global aspect, I learnedthat there were many people in the world living in poverty.We should do whatever we could do to help these people.”


Student H (Secondary 3, 2008-2009). “I could still rememberwhen I first participated in the Project P.A.T.H.S., I didnot understand why we should sacrifice two sessions oflesson for studying this program. However, I was graduallyinvolved in it. This program provided me an opportunity toknow more about my teachers and classmates . . . Althoughthe three-year program is going to end, we still have alot to learn throughout our lives. Among the units I havelearned, the most impressive topic is “Life and Death”. “Lifeand Death” is actually not far away from us. Rememberthe Wenchuan earthquake last year? Life of the students inSichuan was changed in only one second. As Hong Kongstudents, we should make deepest reflection—bliss is nottaken for granted!”

4. Discussion

The purpose of this study was to evaluate the Tier 1 Programof the Project P.A.T.H.S. using findings based on weeklydiaries collected in the experimental and full implementationphases (2005–2009) of the project. There are several char-acteristics of this study. First, a large sample of participants(n = 1,138 students) was involved. Second, eight datasetscollected across three grades at different points in time wereanalyzed. Third, this is the first known scientific evaluationstudy of a positive youth development program utilizingstudent diaries based on different cohorts in China. Fourth,this is also the first evaluation study utilizing diaries basedon such a large sample of participants in the global context.Finally, the data were analyzed in both quantitative andqualitative ways. As far as the design and data collection ofthe studies are concerned, the study has the characteristics ofnaturalistic inquiry, inductive analysis, holistic perspective,qualitative data, unique case orientation, and contextualsensitivity. Regarding data analyses, both qualitative (e.g.,coding and categorizing) and quantitative (mainly counting)analyses were used.

Based on the weekly diaries written by the students, thestudies reported in this paper showed that the participantsgenerally had positive perceptions of the Tier 1 Programof the Project P.A.T.H.S. and the instructors. The findingspresented in Tables 2 and 3 suggest that the participants hadpositive impressions of the program, and they appreciatedthe program content and learning process involved. Inaddition, Table 4 shows that the participants overwhelminglyregarded the program as beneficial to the cognitive, emo-tional, social, and productive quality of life of the programparticipants. For example, a sizable number of responseswere related to positive self-image after joining the program(Table 4). In addition, many participants remarked that theirinterpersonal competencies were enhanced after joining theprogram. In short, the findings in Table 4 basically suggestthat the program is beneficial to the development of theprogram participants.

The above findings are generally consistent with previousresearch findings in the experimental and full implemen-tation phases, and they basically concur with both thequantitative and qualitative findings based on objective

outcome, subjective outcome, qualitative, and process eval-uation findings (e.g., [24–27, 31]). In short, the presentfindings provide additional evidence for supporting theeffectiveness of the Tier 1 Program of the Project P.A.T.H.S.in Hong Kong. From the perspective of triangulation, theexisting evaluation findings suggest that the effectiveness ofthe Tier 1 Program of the Project P.A.T.H.S. is supportedby evaluation data collected from different sources and bydifferent strategies.

The present findings demonstrate the utility of usingstudent diaries to evaluate positive youth developmentprograms. By asking the program participants to reflecton their experiences, subjective perceptions and perceivedbenefits of the program can be properly understood in aless mechanical and artificial manner. Such an approachcan help to capture the dynamic nature of the perceivedqualities and effectiveness of the program. In addition to itsholistic emphasis and ease of data collection, Maxwell [32]commented that “qualitative research is a rigorous meansof investigating causality” (page 3). Slayton and Llosa [33]also remarked that “qualitative methods can confirm thatit is actually the program that is responsible for the effect”(page 2544). Obviously, the use of reflective journals enjoysthe strengths of qualitative studies in assessing the changesin quality of life among the participants after joining theprogram.

Shek et al. [34] proposed that several principles shouldbe maintained in order to enhance the quality of qualitativeresearch. As a qualitative study, the following attributes areintrinsic to this study. First, a general qualitative orientationwas adopted. Second, recruitment processes for the partic-ipants and justifications for the number of participants aredescribed. Third, details of the data collection are given.Fourth, issues of biases and ideological preoccupation areaddressed. Fifth, inter- and intrarater reliabilities informa-tion is presented. Finally, the categorized data are kept in asystematic filing system in order to ensure that the findingsare auditable.

Nevertheless, several limitations of the study should benoted. First, for students who are sensitive about the issueof “invasion of privacy” [12], the use of a weekly diary maybe regarded as an obtrusive evaluation device. Second, asthe diary method is written in nature, students who arenot good at using words to express their experiences maywrite very little. This problem may be particularly acute inschools that admit students with poor academic performanceor literacy. This is also a problem in the Chinese culturewhere Chinese people may lack the language to describe theirfeelings. Third, as there was only one occasion on whichthe participants expressed their views in a one-way manner,it was not possible to have dialogues with the informantsto further understand some of their views. In particular,theoretical sampling approaches could be attempted infuture studies.

Fourth, although the present findings are interpretedin terms of the positive program effects and experiencesof the program participants, it should be noted thatthere are several alternative explanations. These alternativeexplanations should be considered because high proportions


of responses were positive across the eight studies. Thefirst alternative explanation is that the students were afraidof being punished by the program implementers if theydid not respond in a favorable manner. However, thispossibility can be partially dismissed because the teachersdid not mark the weekly diaries and the identities of thestudents were basically anonymous. The second alternativeexplanation is that the students consciously acted in a “nice”manner to help the workers to illustrate a positive programeffect. However, this alternative explanation can be partiallydismissed because negative comments were recorded andstudents are normally encouraged to express their viewshonestly. The third alternative explanation is that the highproportion of positive responses observed, were in fact,biased responses. However, this alternative explanation canalso be dismissed because the schools and students wererandomly selected. The fourth alternative explanation is thatbecause there is no control group, natural developmentalmaturation may account for the positive evaluation findings.That is, through maturation, the participants may see thingsin a more favorable manner. Furthermore, threats to internalvalidity in the study should be realized.

Finally, although the 11 principles proposed by Sheket al. [34] were upheld in this study, peer and memberchecking (principle 8) was not carried out because of timeand manpower constraints. Despite these limitations, thisstudy provides pioneering evaluation findings across samplesand time that support the positive nature of the ProjectP.A.T.H.S. and its effectiveness in promoting holistic youthdevelopment among Chinese adolescents in Hong Kong.

Acknowledgments

The preparation for this paper and the Project P.A.T.H.S.were financially supported by The Hong Kong Jockey ClubCharities Trust. The authorship is equally shared between thefirst and second authors.

References

[1] B. C. Leigh, “Alcohol consumption and sexual activity asreported with a diary technique,” Journal of Abnormal Psychol-ogy, vol. 102, no. 3, pp. 490–493, 1993.

[2] L. Peterson, G. Tremblay, B. Ewigman, and C. Popkey, “Theparental daily diary: a sensitive measure of the process ofchange in a child maltreatment prevention program,” BehaviorModification, vol. 26, no. 5, pp. 627–647, 2002.

[3] B. C. Leigh, “Using daily reports to measure drinking anddrinking patterns,” Journal of Substance Abuse, vol. 12, no. 1-2,pp. 51–65, 2000.

[4] D. M. Morrison, B. C. Leigh, and M. R. Gillmore, “Daily datacollection: a comparison of three methods,” Journal of SexResearch, vol. 36, no. 1, pp. 76–81, 1999.

[5] M. R. Gillmore, J. Gaylord, J. Hartway et al., “Daily datacollection of sexual and other health-related behaviors,”Journal of Sex Research, vol. 38, no. 1, pp. 35–42, 2001.

[6] A. A. Stone, R. C. Kessler, and J. A. Haythornthwaite,“Measuring daily events and experiences: decisions for theresearcher,” Journal of Personality, vol. 59, no. 3, pp. 575–607,1991.

[7] D. Gorman, “Self-tuning teachers: using reflective journals inwriting classes,” Journal of Adolescent and Adult Literacy, vol.41, no. 6, pp. 434–442, 1998.

[8] D. Hobson, “Beginning with the self: using autobiographyand journal writing in research,” in Teacher Doing Research:Practical Possibilities, G. Burnaford, J. Fischer, and D. Hobson,Eds., pp. 1–17, Erlbaum, Mahwah, NJ, USA, 1996.

[9] H. Bartholomew, J. Osborne, and M. Ratcliffe, “Teachingstudents “ideas-about-science”: five dimensions of effectivepractice,” Science Education, vol. 88, no. 5, pp. 655–682, 2004.

[10] D. A. Kolb, Experiential Learning, Prentice Hall, EnglewoodCliffs, NY, USA, 1984.

[11] E. L. Deci and R. Flaste, Why We Do What We Do: Understand-ing Self-Motivation, Penguins Books, New York, NY, USA,1996.

[12] A. Rothwell and S. Ghelipter, “The developing manager:reflective learning in undergraduate management education,”Reflective Practice, vol. 4, no. 2, pp. 241–254, 2003.

[13] N. Ames and S. A. Diepstra, “Using intergenerational oralhistory service-learning projects to teach human behaviorconcepts: a qualitative analysis,” Educational Gerontology, vol.32, no. 9, pp. 721–735, 2006.

[14] L. McAllister, E. McEwen, V. Williams, and N. Frost, “Ruralattachments for students in the health professions: are theyworthwhile?” The Australian Journal of Rural Health, vol. 6, no.4, pp. 194–201, 1998.

[15] B. Chaloner, “An evaluative study on building the bridgebetween the training room and the workplace,” EducationalResearch and Evaluation, vol. 19, no. 1, pp. 21–37, 2006.

[16] Z. M. Wagner, “Using student journals for course evaluation,”Assessment and Evaluation in Higher Education, vol. 24, no. 3,pp. 261–272, 1999.

[17] M. J. Follick, D. K. Ahern, and N. Laser-Wolston, “Evaluationof a daily activity diary for chronic pain patients,” Pain, vol.19, no. 4, pp. 373–382, 1984.

[18] C. Travers, “Unveiling a reflective diary methodology forexploring the lived experiences of stress and coping,” Journalof Vocational Behavior, vol. 79, no. 1, pp. 204–216, 2010.

[19] D. J. Cohen, L. C. Leviton, N. Isaacson, A. F. Tallia, and B.F. Crabtree, “Online diaries for qualitative evaluation: gainingreal-time insights,” American Journal of Evaluation, vol. 27, pp.163–184, 2006.

[20] R. E. Knowles and N. Tarrier, “Evaluation of the effectof prospective patient diaries on emotional well-being inintensive care unit survivors: a randomized controlled trial,”Critical Care Medicine, vol. 37, no. 1, pp. 184–191, 2009.

[21] J. Roth, J. Brooks-Gunn, L. Murray, and W. Foster, “Promotinghealthy adolescents: synthesis of youth development programevaluations,” Journal of Research on Adolescence, vol. 8, no. 4,pp. 423–459, 1998.

[22] R. F. Catalano, M. L. Berglund, J. A. M. Ryan, H. S. Lonczak,and J. D. Hawkins, “Positive youth development in the UnitedStates: research findings on evaluations of positive youthdevelopment programs,” Annals of the American Academy ofPolitical and Social Science, vol. 591, pp. 98–124, 2004.

[23] D. T. L. Shek and R. C. F. Sun, “Effectiveness of the tier 1program of project P.A.T.H.S.: findings based on three yearsof program implementation,” TheScientificWorldJournal, vol.10, pp. 1509–1519, 2010.

[24] D. T. L. Shek and C. M. S. Ma, “Impact of the ProjectP.A.T.H.S. in the junior secondary school years: individualgrowth curve analyses,” TheScientificWorldJournal, vol. 11, pp.253–266, 2011.


[25] D. T. L. Shek and L. Yu, “Prevention of adolescent problembehavior: longitudinal impact of the Project P.A.T.H.S. inHong Kong,” TheScientificWorldJournal, vol. 11, pp. 546–567,2011.

[26] D. T. L. Shek and C. Chan, “Qualitative evaluation of theproject P.A.T.H.S. based on junior secondary school studentsin Hong Kong,” International Journal of Adolescent Medicineand Health, vol. 22, no. 4, pp. 511–525, 2010.

[27] D. T. L. Shek and R. C. F. Sun, “Subjective outcome evaluationbased on secondary data analyses: the Project P.A.T.H.S. inHong Kong,” TheScientificWorldJournal, vol. 10, pp. 224–237,2010.

[28] D. T. L. Shek, “Using students’ weekly diaries to evaluatepositive youth development programs: are findings based onmultiple studies consistent?” Social Indicators Research, vol. 95,no. 3, pp. 475–487, 2010.

[29] D. T. L. Shek and R. C. F. Sun, “Using student weekly diaryto evaluate positive youth development programs: the caseof Project P.A.T.H.S. in Hong Kong,” International Journal ofAdolescent Medicine and Health, vol. 1, no. 2, pp. 93–102, 2008.

[30] M. B. Miles and A. M. Huberman, Qualitative Data Analysis,Sage, Thousand Oaks, Calif, USA, 1994.

[31] D. T. L. Shek, R. C. F. Sun, and C. S. M. Ng, “Processevaluation of the implementation of the Secondary 3 Programof Project P.A.T.H.S. in the Experimental ImplementationPhase,” International Public Health Journal, vol. 1, no. 3, pp.311–324, 2009.

[32] J. A. Maxwell, “Causal explanation, qualitative research, andscientific inquiry in Journal item: education,” EducationResearch, vol. 33, pp. 3–11, 2004.

[33] J. Slayton and L. Llosa, “The use of qualitative methods inlarge-scale evaluation: improving the quality of the evaluationand the meaningfulness of the findings,” Teachers CollegeRecord, vol. 107, no. 12, pp. 2543–2565, 2005.

[34] D. T. L. Shek, V. M. Y. Tang, and X. Y. Han, “Evaluation ofevaluation studies using qualitative research methods in thesocial work literature (1990–2003): evidence that constitutes awake-up call,” Research on Social Work Practice, vol. 15, no. 3,pp. 180–194, 2005.

evaluation of the project p.a.t.h.s. based on students...

Documents