a systematic review of internet-based self-management interventions for youth with health conditions

16
A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions Jennifer Stinson, 1,3,5 RN, PHD, CPNP, Rita Wilson, 2 RN, BSCN, MEd, MN, Navreet Gill, 3 RN, BSCN, Janet Yamada, 3,4 RN, MSC, and Jessica Holt, 5 BHSC 1 Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, 2 Ryerson University, 3 Child Health Evaluative Sciences, 4 Centre for Nursing and Research Institute, and 5 Department of Anesthesia and Pain Medicine, The Hospital for Sick Children (SickKids) Objective Critically appraise research evidence on effectiveness of internet self-management interventions on health outcomes in youth with health conditions. Methods Published studies of internet interventions in youth with health conditions were evaluated. Electronic searches were conducted in EBM Reviews-Cochrane Central Register of Controlled Trials, Medline, EMBASE, CINAHL and PsychINFO. Two reviewers indepen- dently selected articles for review and assessed methodological quality. Of 29 published articles on internet interventions; only nine met the inclusion criteria and were included in analysis. Results While outcomes varied greatly between studies, symptoms improved in internet interventions compared to control conditions in seven of nine studies. There was conflicting evidence regarding disease-specific knowledge and quality of life, and evidence was limited regarding decreases in health care utilization. Conclusions There are the beginnings of an evidence base that self-management interventions delivered via the internet improve selected outcomes in certain childhood illnesses. Key words adolescent; child; chronic illness; internet; self-management. It is estimated that 15–20% of children and adolescents have significant ongoing health care needs related to a subacute or chronic (lasting more than 3 months) health condition (Newacheck et al., 1998). Chronic illness in childhood can negatively impact all aspects of quality of life (QOL) (Wallander & Varni, 1998; Yeo & Sawyer, 2005). Disease course can be unpredictable and children commonly experience a myriad of symptoms that may restrict physical and social interactions. Disease manage- ment is often complex, involving diverse and multiple therapies and requires constant monitoring. Older school-aged children and adolescents are expected to assume greater responsibility for disease management than when they were younger due to their growing inde- pendence and autonomy (Sawyer & Aroni, 2005). However, adherence to disease management activities is less than optimal (Fiese & Everhart, 2006; Yeo & Sawyer, 2005). Poor adherence and inappropriate self- management behaviors may reduce the potential benefits of treatment. Furthermore, the vast majority of adolescents do not receive comprehensive disease education linked with self-management therapy due to (a) difficulty acces- sing these services (e.g., long-wait times), (b) limited avail- ability of trained professionals (e.g., psychologists) especially in rural areas, and (c) costs (e.g., time off work) associated with these therapies (Barlow & Ellard, 2004). Enhanced disease awareness and greater self-man- agement may prevent or diminish illness exacerbation and associated adverse health outcomes. Self management can be defined as ‘‘the individual’s ability to manage the symptoms, treatment, physical and psychological consequences and life style changes inherent in living with a chronic illness’’ (Barlow, Wright, Sheasby, Turner, & Hainsworth, 2002). Self-management inter- ventions typically encompass information-based material and cognitive and/or behavioral strategies designed to increase participants’ knowledge, self-efficacy and use of self-management behaviors (Barlow et al., 2002; Sawyer & All correspondence concerning this article should be addressed to Jennifer Stinson, Child Health Evaluative Sciences, Research Institute, SickKids, 555 University Avenue, Toronto, Ontario, M5G 1X8 Canada. E-mail: [email protected]. Journal of Pediatric Psychology 34(5) pp. 495510, 2009 doi:10.1093/jpepsy/jsn115 Advance Access publication November 23, 2008 Journal of Pediatric Psychology vol. 34 no. 5 ß The Author 2008. Published by Oxford University Press on behalf of the Society of Pediatric Psychology. All rights reserved. For permissions, please e-mail: [email protected] at Ball State University on November 20, 2014 http://jpepsy.oxfordjournals.org/ Downloaded from

Upload: j

Post on 27-Mar-2017

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

A Systematic Review of Internet-based Self-Management Interventionsfor Youth with Health Conditions

Jennifer Stinson,1,3,5 RN, PHD, CPNP, Rita Wilson,2 RN, BSCN, MEd, MN, Navreet Gill,3 RN, BSCN,

Janet Yamada,3,4 RN, MSC, and Jessica Holt,5 BHSC

1Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, 2Ryerson University, 3Child Health Evaluative

Sciences, 4Centre for Nursing and Research Institute, and 5Department of Anesthesia and Pain Medicine,

The Hospital for Sick Children (SickKids)

Objective Critically appraise research evidence on effectiveness of internet self-management interventions on

health outcomes in youth with health conditions. Methods Published studies of internet interventions in

youth with health conditions were evaluated. Electronic searches were conducted in EBM Reviews-Cochrane

Central Register of Controlled Trials, Medline, EMBASE, CINAHL and PsychINFO. Two reviewers indepen-

dently selected articles for review and assessed methodological quality. Of 29 published articles on internet

interventions; only nine met the inclusion criteria and were included in analysis. Results While outcomes

varied greatly between studies, symptoms improved in internet interventions compared to control conditions

in seven of nine studies. There was conflicting evidence regarding disease-specific knowledge and quality of

life, and evidence was limited regarding decreases in health care utilization. Conclusions There are the

beginnings of an evidence base that self-management interventions delivered via the internet improve

selected outcomes in certain childhood illnesses.

Key words adolescent; child; chronic illness; internet; self-management.

It is estimated that 15–20% of children and adolescents

have significant ongoing health care needs related to a

subacute or chronic (lasting more than 3 months) health

condition (Newacheck et al., 1998). Chronic illness in

childhood can negatively impact all aspects of quality of

life (QOL) (Wallander & Varni, 1998; Yeo & Sawyer,

2005). Disease course can be unpredictable and children

commonly experience a myriad of symptoms that may

restrict physical and social interactions. Disease manage-

ment is often complex, involving diverse and multiple

therapies and requires constant monitoring. Older

school-aged children and adolescents are expected to

assume greater responsibility for disease management

than when they were younger due to their growing inde-

pendence and autonomy (Sawyer & Aroni, 2005).

However, adherence to disease management activities is

less than optimal (Fiese & Everhart, 2006; Yeo &

Sawyer, 2005). Poor adherence and inappropriate self-

management behaviors may reduce the potential benefits

of treatment. Furthermore, the vast majority of adolescents

do not receive comprehensive disease education linked

with self-management therapy due to (a) difficulty acces-

sing these services (e.g., long-wait times), (b) limited avail-

ability of trained professionals (e.g., psychologists)

especially in rural areas, and (c) costs (e.g., time off

work) associated with these therapies (Barlow & Ellard,

2004). Enhanced disease awareness and greater self-man-

agement may prevent or diminish illness exacerbation and

associated adverse health outcomes.

Self management can be defined as ‘‘the individual’s

ability to manage the symptoms, treatment, physical and

psychological consequences and life style changes inherent

in living with a chronic illness’’ (Barlow, Wright, Sheasby,

Turner, & Hainsworth, 2002). Self-management inter-

ventions typically encompass information-based material

and cognitive and/or behavioral strategies designed to

increase participants’ knowledge, self-efficacy and use of

self-management behaviors (Barlow et al., 2002; Sawyer &

All correspondence concerning this article should be addressed to Jennifer Stinson, Child Health Evaluative Sciences,Research Institute, SickKids, 555 University Avenue, Toronto, Ontario, M5G 1X8 Canada.E-mail: [email protected].

Journal of Pediatric Psychology 34(5) pp. 495–510, 2009

doi:10.1093/jpepsy/jsn115

Advance Access publication November 23, 2008

Journal of Pediatric Psychology vol. 34 no. 5 � The Author 2008. Published by Oxford University Press on behalf of the Society of Pediatric Psychology.All rights reserved. For permissions, please e-mail: [email protected]

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 2: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

Aroni, 2005). Studies in both adult (Barlow et al., 2002;

Krishna, Balas, Spencer, Griffin, & Boren, 1997; Marks &

Allegrante, 2005; Murray, Burns, See, Lai, & Nazareth,

2005), and pediatric chronic illness (Barlow & Ellard,

2004; Elgar & McGrath, 2003; Last, Stam, Onland-van

Nieuwenhuizen, & Grootenhuis, 2007) have shown that

comprehensive interactive interventions that augment

medical treatments with self-management lead to better

health outcomes and improved QOL than care that is

strictly medically focused. Self-management interventions

are meant to augment and not replace the information and

support provided by health care professionals.

e-Health technologies offer an innovative approach to

improving the health service delivery and acceptability

of self-management interventions for youth with health

conditions. Murray and colleagues (2005) conducted a

Cochrane review of Interactive Health Communication

Applications (IHCA), which are computer-based interven-

tions that encompass health information plus at least one

of the following: social support, decision-making support,

and/or behavior change support. The most common mode

of delivering IHCAs is the internet; however, they can also

take the form of CD-ROMs, telehealth, personal digital

assistants, or computer games. Internet interventions are

treatments based on effective face-to-face interventions

(e.g., cognitive–behavioral therapies) that are operationa-

lized and transformed for delivery via the internet with

the goal of symptom improvement. Usually they are

highly structured, self-guided or partly self-guided (i.e.,

use of therapist or health coach), tailored to the user’s

needs, and are interactive. Interactivity is enhanced by

graphics, animations, audio and video clips, as well as

interactive and continuous self-monitoring (diaries), infor-

mation exchange (chat rooms), follow-up and feedback

(e.g., decision-making support through emails)

(Ritterband et al., 2003b).

The literature on internet interventions is rapidly

growing with studies on the feasibility and utility, as well

as some evidence of their efficacy and effectiveness in

improving symptom management (Cuijpers, van Straten,

& Andersson, 2008; Murray et al., 2005; Wantland,

Portillo, Holzemer, Slaughter, & McGhee, 2004). Murray

and colleagues (2005) in their Cochrane review of IHCAs

found they had largely positive effects on end-users in

terms of being more knowledgeable and feeling more

socially supported and that they may improve behavioral

and clinical outcomes as well as self-efficacy compared to

non-users. However, they were not able to determine their

effect on emotional and economic outcomes (Murray et al.,

2005). While the internet has emerged as one of this age

group’s main health information sources (Gray, Klein,

Noyce, Sesselberg, & Cantrill, 2005), few self-administered

multimedia internet programs for youth with health con-

ditions have been developed and validated compared to

those that exist for adults (Murray et al., 2005).

This article is a report of a systematic review on

internet-based interventions to promote self-management

in youth with health conditions. More specifically, in

this article we will: (a) describe the internet programs,

(b) describe study designs and evaluation methods,

(c) summarize their findings in terms of efficacy and effec-

tiveness of the internet interventions, (d) determine their

methodological quality, and (e) recommend future direc-

tion in the development and testing of internet-based

interventions.

MethodsData Sources

Electronic searches were conducted by a Library

Information Specialist (CN) familiar with the field. The

EBM Reviews—Cochrane Central Register of Controlled

Trials (First Quarter 2008), Medline (1950–January

Week 5, 2008), EMBASE (1980–January Week 5, 2008),

CINAHL (1982–December 2007), and PsychINFO (1967–

January Week 5, 2008) were searched. Subject headings

and MeSH terms included ‘‘chronic illness,’’ ‘‘chronic

mental illness,’’ ‘‘computer assisted instruction,’’ ‘‘elec-

tronic communication,’’ ‘‘computer software,’’ ‘‘internet,’’

and ‘‘web-based.’’ Other keywords such as ‘‘self-care

skills,’’ ‘‘health education,’’ ‘‘health knowledge,’’ ‘‘health

behavior,’’ ‘‘disease management,’’ and ‘‘self-help techni-

ques’’ were used to search for the ideal publication type.

We also included a broad list of common pediatric chronic

illnesses as search engines rarely index papers using gen-

eric terms such as chronic condition or disability. All

search titles and abstracts were independently rated for

relevance by two reviewers (J.S., R.W.). There was 100%

agreement on the selected review articles. Reference lists

from all identified appropriate papers and review papers

were examined and then hand searched for additional

relevant studies. No attempt was made to locate unpub-

lished material or contact researchers for unpublished

studies (e.g., dissertations or conference proceeding

abstracts).

Study Selection

To be eligible, articles had to meet the following criteria:

(1) The articles had been published in a peer-reviewed

journal during the past 15 years (1993–2008).

496 Stinson, Wilson, Gill, Yamada, and Holt

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 3: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

(2) The intervention targeted school aged children

(6–12 years) and/or adolescents (13–18 years) with

subacute or chronic health conditions.

(3) The study evaluated an internet-based or enabled

self-management intervention. We excluded studies

that involved other forms of IHCAs including hand-

held computers, video or telehealth programs,

CD-ROMs, and video games as it may be difficult to

compare outcomes across these different modalities

and they vary in their degree of interactivity.

(4) The evaluation examined outcomes based on the

potential benefits of internet-based IHCAs includ-

ing: effects on patients in terms of knowledge,

social support, self-efficacy, emotions and health

behaviors; health outcomes; QOL; and resource

utilization (Murray et al., 2005).

(5) The study was a randomized controlled trial (RCT),

quasi-randomized trial, before-and-after design or

feasibility RCT study. Given the dearth of research

in this area, we broadened the typical inclusion

criteria for systematic reviews beyond RCTs to avoid

missing important or significant data that would

inform the development of this rapidly expanding

field of clinical research.

(6) The article was published in English due to addi-

tional costs related to translation.

Review Process

Two of the authors (J.S., R.W.) examined each computer

search to identify potentially eligible articles using the

inclusion criteria outlined above. They based their initial

judgment on titles and abstracts; subsequently, all articles

meeting the criteria were reviewed and eligibility was deter-

mined by consensus of all authors.

Data Extraction

A systematic approach to data extraction was used to pro-

duce a descriptive summary of participants, interventions

and study findings (see Tables I and II). The first reviewer

(R.W.) independently extracted the year of publication,

journal of publication, study participants, study focus

(e.g., type of internet intervention) and main results from

each study. Detailed data on the intervention (e.g., treat-

ment fidelity) and control groups were extracted as well as

whether a theory or conceptual framework was used to

guide the development and evaluation of the intervention.

Summaries of the main results regarding effects on self-

management outcomes were based on the text in the ori-

ginal manuscript, and focused on quantitative summaries

when available. A second reviewer (J.S.) checked the data

extraction. When data were missing from a study, no

attempt was made to contact authors for additional infor-

mation and we limited our review to the material included

in the peer-reviewed articles.

Methodology Quality

There are few validated tools available to rate the metho-

dological quality of RCTs, especially non-pharmacological

trials. We chose to use the CONSORT (Consolidated

Standards of Reporting Trials) checklist which has recently

been extended for trials of non-pharmacological treatments

(Boutron, Moher, Altman, Schulz, & Ravaud, 2008a,b).

The CONSORT statement was developed to improve the

quality of reporting of clinical trials (Moher, Schulz, &

Altman, 2001) and has more recently been used to rate

the methodological quality of randomized controlled trials

(Stinson, McGrath, & Yamada, 2003). A copy of the check-

list can be found at http://www.consort-statement.org/.

The checklist has 23-items and we arbitrarily (23items/3)

assigned a score of <8 as poor quality, 9–16 as fair, and a

score of >17 indicating good methodological quality. Two

reviewers independently assessed the methodological qual-

ity of each study using this quality assessment measure

(J.S., J.H.). All papers were blinded to the names of the

authors, institutions and journals. There was 91% agree-

ment between the two reviewers. Any disagreements in

ratings were resolved by consensus.

Data Synthesis

We initially attempted to report on effects in terms of

standardized mean difference (SMD) and confidence inter-

vals. However, given the heterogeneity of outcomes mea-

sures used across the studies, the author’s main qualitative

conclusions were reported instead.

Results

A total of 806 articles were identified from the electronic

searches. One hundred and seven articles were removed

after accounting for duplicates, leaving 699 articles for

further consideration. Of these, 673 articles were removed

as they were general or review articles or other non-internet

based IHCA interventions (5¼CD-ROMs, 7¼ gaming,

2¼ telehealth for symptom monitoring, and 1¼ chat

room for social support). From the 29 remaining inter-

net-based studies, a further 11 were excluded as they

were not targeting the population of interest, 1 internet

intervention provided only social support, and 8 were

excluded for design reasons leaving 9 studies (7 RCTs,

1 pilot RCT, and 1 quasi-experimental study) for

Systematic Review of Internet Interventions for Youth with Health Conditions 497

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 4: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

Tab

leI.

Sam

ple

,M

eth

od

san

dR

esu

lts

for

Inte

rnet

Inte

rven

tio

ns

Refe

ren

ces,

Co

un

try

Part

icip

an

ts

(nu

mb

er,

sex,

age

gro

up

,

recr

uitm

en

t)

Health

con

ditio

n

Stu

dy

desi

gn

Inte

rven

tio

n

(N)

Co

mp

ari

son

Gro

up

(N)

Meth

od

olo

gic

al

qu

alit

y

ratin

ga

Tim

ing

of

ou

tco

me

ass

ess

men

t

Ou

tco

mes

measu

red

Eff

ect

b

Dro

po

uts

(%)

(1)

Ch

anet

al.

(2007),

Ch

ina

120

you

th

45;

75

6–1

7ye

ars

Com

mu

nit

y

Per

sist

ent

asth

ma

RC

TIn

-hom

eIn

tern

et-

bas

edte

lem

edic

ine

syst

emfo

rsy

mp

tom

mon

itor

ing,

edu

cati

on

and

case

man

agem

ent

(th

ree

virt

ual

and

thre

ecl

inic

visi

ts)

(60)

Tra

dit

ion

alof

fice

-

bas

edin

-per

son

edu

-

cati

onan

dca

sem

an-

agem

ent

(6cl

inic

visi

ts)

(60)

14

Bas

elin

e,2,

6

12,

26

and

52

wee

ks

Dia

gnos

tic

Ad

her

ence

�Sy

mp

tom

dia

ry

�A

sses

smen

tp

eak

flow

tech

niq

ue

Dis

ease

con

trol

�LF

T

�PE

F

�Q

OL

�U

tiliz

atio

nof

serv

ices

�R

escu

e

ther

apy

use

�A

sth

ma

know

led

ge

�Sy

mp

tom

s

þ þ 0 0 0 0 0 0 0

21.6

(2)

Hic

ks,

von

Bae

yer,

and

McG

rath

(2006),

Can

ada

47

you

th–

par

ent

dya

ds

30;

17

9–1

6ye

ars

Com

mu

nit

y

Rec

urr

ent

pai

n/

hea

dac

he

RC

TIn

-hom

eIn

tern

et-

bas

edco

gnit

ive–

beh

a-

vior

alp

ain

trea

tmen

t

pro

gram

(25)

Stan

dar

dm

edic

alca

re

wai

tlis

t

(22)

15

Bas

elin

ean

dat

1c

and

3d

mon

ths

pos

ttre

atm

ent

Pai

n

QO

L

þe

0

28

(3)

Jan

etal

.

(2007),

USA

164

child

ren

101;

63

6–1

2ye

ars

Clin

ical

Per

sist

ent

asth

ma

RC

TB

lue

Ang

elfo

rA

sthm

a

Kid

s:an

Inte

rnet

-

bas

edm

ult

imed

ia

asth

ma

edu

cati

on

pro

gram

wit

hin

tera

c-

tive

asth

ma

tele

-mon

i-

tori

ng

and

retr

ieva

l

anal

ysis

syst

emfo

r

revi

ewof

accu

mu

late

d

dat

aon

sym

pto

ms

and

PE

FR

(88)

Tra

dit

ion

alas

thm

a

edu

cati

on(v

erbal

and

wri

tten

boo

klet

),in

di-

vid

ual

ized

asth

ma

self-

man

agem

ent

acti

on

pla

nan

das

thm

a

sym

pto

md

iary

(76)

16

Bas

elin

ean

d

12

wee

ks,

PE

F,

sym

pto

m

dia

ryan

dp

ul-

mon

ary

spir

o-

met

ric

test

sat

4,

8,

and

12

wee

ks

Dis

ease

con

trol

�A

vera

gew

eekl

y

PE

F �Sy

mp

tom

scor

es

�A

sth

ma

con

trol

Ad

her

ence

QO

L

Ast

hm

akn

owle

dge

þ þ þ þ þ þ

17

498 Stinson, Wilson, Gill, Yamada, and Holt

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 5: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

(4)

Jose

ph

et

al.

(2007),

USA

314

you

th

199;

115

5–1

9ye

ars

Sch

ool

Mild

,p

er-

sist

ent

asth

ma

sym

pto

ms

wit

h/w

ith

-

out

form

al

dia

gnos

is

RC

TSc

hoo

lIn

tern

et-b

ased

,

tailo

red

asth

ma

man

-

agem

ent

pro

gram

and

pro

acti

vefo

llow

-up

by

hea

lth

care

refe

rral

coor

din

ator

(162)

Reg

ula

ted

bro

wsi

ng

of

gen

eric

asth

ma

edu

ca-

tion

web

site

sco

n-

trol

led

by

limit

ed

secu

rity

acce

ssan

d

tim

elim

ited

(30

min

-

ute

s)an

dse

lf-i

nit

iate

d

acce

ssto

hea

lth

care

refe

rral

coor

din

ator

(152)

16

Bas

elin

e,an

d

12

mon

ths

pos

tbas

elin

e

Fu

nct

ion

alst

atu

s

�N

um

ber

ofsy

mp

-

tom

day

s/2

wee

ks

�Sy

mp

tom

nig

hts

/

2w

eeks

�D

ays

ofre

stri

cted

acti

vity

/2w

eeks

�D

ays

ofre

stri

cted

acti

vity

�D

ays

sch

ool

mis

sed

/30

day

s

Med

ical

care

use

�E

Dvi

sits

/12

mon

ths

�H

osp

ital

izat

ion

s/

12

mon

ths

QO

L

þ þ þ þ þ 0 þ 0

17

(5)

Kri

shn

aet

al.

(2003),

USA

228

child

ren

and

par

ents

80;

148

7–1

7ye

ars

Clin

ical

Ast

hm

aR

CT

Inte

ract

ive

Mul

tim

edia

Pro

gram

for

Ast

hma

Con

trol

and

Tra

ckin

g

(IM

PA

CT

):A

clin

ic-

bas

edin

tern

et-

enab

led

Inte

ract

ive

mu

ltim

edia

CD

-RO

M

self-m

anag

emen

ted

u-

cati

onp

rogr

am

(121)

Tra

dit

ion

alcl

inic

-

bas

edp

atie

nt

edu

ca-

tion

(107)

15

Bas

elin

ean

d3

and

12

mon

ths

Kn

owle

dge

QO

L

Dis

ease

outc

omes

(dec

reas

ed

ays

wit

h

asth

ma

sym

pto

ms)

Hea

lth

serv

ices

uti

li-

zati

on(E

Dvi

sits

)

þ 0 þ þ

10

(6)

Rit

terb

and

etal

.

(2003

a ),

USA

24

child

ren

and

par

ents

5;

19

6–1

2ye

ars

Com

mu

nit

y

En

cop

resi

sPilo

t

RC

T

Inte

rnet

-bas

ed

enh

ance

dto

ilet

trai

n-

ing

pro

gram

that

inco

rpor

ates

beh

av-

iora

lth

erap

yan

d

med

ical

man

agem

ent

(12)

Usu

alca

re

(12)

10

Pre

trea

tmen

t

and

3w

eeks

pos

ttre

atm

ent

Bow

elh

abit

s(#

acci

-

den

ts)

Kn

owle

dge

Bow

elsp

ecific

pro

ble

ms

þ 0 0

0

(7)

Ru

nge

,

Lec

hel

er,

Hor

ns,

Tew

s,an

d

Sch

aefe

r

(2006),

Ger

man

y

438

you

th

5;

19

8–1

6ye

ars

Clin

ical

and

com

mu

nit

y

Ast

hm

aQ

uas

i-

exp

eri-

men

tal

(non

-

stra

ti-

fied

)

Tw

oin

terv

enti

on

grou

ps:

(1)

Inte

rnet

-bas

ed

asth

ma

edu

cati

on

pro

gram

(IE

P)

in

add

itio

nto

SPM

P

(146)

(2)

SPM

P—

seri

esof

5

stan

dar

diz

eded

uca

-

tion

alse

ssio

ns

(127)

Usu

alca

re(w

ait-

list

con

trol

grou

pre

ceiv

ed

inte

rven

tion

at6

mon

ths)

(85)

12

Bas

elin

ean

d6

and

12

mon

ths

Cos

t-ben

efit

anal

ysis

Red

uct

ion

inas

thm

a-

rela

ted

emer

gen

cies

Sch

ool

abse

nte

eism

Res

cue

med

icat

ion

use

Lu

ng

fun

ctio

n

Hea

lth

care

reso

urc

e

uti

lizat

ion

QO

L

þf

þf

þf

þf

0 þf

þg

18%

inIE

P

þSP

MP

63%

in

SPM

P

Con

tin

ued

Systematic Review of Internet Interventions for Youth with Health Conditions 499

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 6: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

Tab

leI.

Co

ntin

ued

Refe

ren

ces,

Co

un

try

Part

icip

an

ts

(nu

mb

er,

sex,

age

gro

up

,

recr

uitm

en

t)

Health

con

ditio

n

Stu

dy

desi

gn

Inte

rven

tio

n

(N)

Co

mp

ari

son

Gro

up

(N)

Meth

od

olo

gic

al

qu

alit

y

ratin

ga

Tim

ing

of

ou

tco

me

ass

ess

men

t

Ou

tco

mes

measu

red

Eff

ect

b

Dro

po

uts

(%)

(8)

Wad

e,

Car

ey,

and

Wol

fe

(2006),

USA

40

you

th

and

fam

ily

17;

23

5–1

6ye

ars

Clin

ical

Mod

erat

eor

seve

retr

au-

mat

icbra

in

inju

ry

RC

TIn

-hom

eIn

tern

et-

bas

edco

gnit

ive–

beh

a-

vior

alfa

mily

trea

tmen

t

pro

gram

wit

hw

eekl

y/

biw

eekl

ysy

nch

ron

ous

vid

eoco

nfe

ren

cin

gan

d

usu

alth

erap

y

(20)

Inte

rnet

reso

urc

es

com

par

ison

(IR

C)

grou

pre

ceiv

ing

usu

al

psy

chos

ocia

lca

re

(24)

15

Bas

elin

ean

d

follo

w-u

p

(tim

elin

en

ot

spec

ifie

d)

Ch

ildbeh

avio

rp

ro-

ble

ms

Soci

alco

mp

eten

ce

Self-m

anag

emen

t/

com

plia

nce

0 0 þ

0

(9)

Will

iam

son

etal

.

(2006),

USA

40

adol

esce

nt–

par

ent

dya

ds

40

11–1

5ye

ars

NR

Obes

ity

BM

I>85th

per

cen

tile

h

BM

I>30

i

RC

TIn

-hom

eIn

tern

et-

bas

edin

tera

ctiv

elif

e-

styl

ebeh

avio

rm

odifi-

cati

onp

rogr

am

(NR

)

Pas

sive

non

-in

tera

c-

tive

Inte

rnet

hea

lth

edu

cati

onp

rogr

am

(lin

ksto

hea

lth

ylif

e-

styl

ew

ebsi

tes)þ

face

-

to-f

ace

cou

nse

llin

g

sess

ion

s(N

R)

11

Bas

elin

ean

d

mon

ths

6,

12,

18

and

24

du

rin

gth

e

inte

rven

tion

Bod

yfa

t

Bod

yw

eigh

t

þj

þk

17.5

Not

e.F

,fe

mal

ean

dM

,m

ale.

PE

F,

pea

kex

pir

ator

yfl

ow;

LF

T,

lun

gfu

nct

ion

test

;E

D,

emer

gen

cyd

epar

tmen

t;IE

P,

inte

rnet

edu

cati

onp

rogr

am;

SPM

P,

stan

dar

diz

edp

atie

nt

man

agem

ent

pro

gram

;N

R,

not

rep

orte

d.

a CO

NSO

RT

Ch

eckl

ist

use

dto

rate

met

hod

olog

ical

qu

alit

y.bþ

,E

ffec

tre

fers

toa

stat

isti

cally

sign

ific

ant

chan

gein

the

outc

omes

mea

sure

dbet

wee

ngr

oup

s.b0

,N

osi

gnif

ican

td

iffe

ren

cein

outc

omes

mea

sure

dbet

wee

ngr

oup

s.c T

ime

2(T

2).

dT

ime

3(T

3).

e Dif

fere

nce

was

stat

isti

cally

sign

ific

ant

atT

2an

dT

3.

f Dif

fere

nce

was

stat

isti

cally

sign

ific

ant

for

IEP

com

par

edto

con

trol

and

SPM

Pgr

oup

s.g IE

Pan

dSP

MP

com

par

edto

con

trol

grou

p.

hM

easu

reof

obes

ity

use

dfo

rad

oles

cen

ts.

i Mea

sure

ofob

esit

yu

sed

for

par

ents

.j D

iffe

ren

cew

ason

lyst

atis

tica

llysi

gnif

ican

tat

T2

for

adol

esce

nts

.kD

iffe

ren

cew

ason

lyst

atis

tica

llysi

gnif

ican

tat

T2

for

par

ents

.

500 Stinson, Wilson, Gill, Yamada, and Holt

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 7: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

Tab

leII

.C

hara

cteri

stic

so

fIn

tern

et-

base

dIn

terv

en

tio

ns

Less

on

s/se

ssio

ns

Art

icle

Co

nte

nt

of

inte

rven

tio

n(p

lus

inte

rac-

tive

featu

res)

Co

nta

ctU

sage

Treatm

en

tin

tegri

tyN

um

ber

Len

gth

Du

ratio

n

(1)

Ch

an

etal

.(2

007)

�W

eb-b

ased

inte

ract

ive

asth

ma

edu

-

cati

on

�D

isea

sem

anag

emen

t

�A

sth

ma

in-h

ome

mon

itor

ing

usi

ng

dig

ital

vid

eoca

mer

ato

cap

ture

pea

k

flow

met

eran

din

hal

edm

edic

atio

n

tech

niq

ue

atp

resc

hed

ule

dti

mes

;

vid

eos

sen

tto

case

man

ager

wh

o

pro

vid

edfe

edbac

kon

tech

niq

ue

�W

eb-b

ased

dai

lyas

thm

ad

iary

Reg

ula

rw

eekl

y

con

tact

wit

hca

se

man

ager

2/w

eek

for

6w

eeks

then

wee

kly

Sign

ific

antl

y

grea

ter

dia

ry

adh

eren

cein

inte

rnet

grou

p

(p<

.01;

how

ever

no

dia

ryen

trie

s

wer

ere

cord

edfo

r

60–8

0%

ofd

ays.

Ele

ctro

nic

Pea

k

flow

was

only

one-

fou

rth

of

that

requ

ired

by

pro

toco

l

�N

Ra

for

rese

arch

team

(fou

rca

sem

an-

ager

s)

�C

ust

omiz

edp

arti

ci-

pan

ted

uca

tion

and

mon

itor

ing

bas

edon

a

clin

ical

pat

hw

ay

6 3in

-per

son

visi

tsan

d

3vi

rtu

al

visi

ts

NR

a52

wee

ks

(2)

Hic

ks

etal

.(2

006)

�Sy

mp

tom

trac

kin

g,id

enti

fyin

gtr

ig-

gers

,an

dov

ervi

ewof

pai

nre

du

ctio

n

met

hod

s

�In

form

atio

non

hea

dac

hes

,go

alse

t-

tin

g,d

eep

bre

ath

ing

�Ph

ysic

alp

ain

man

agem

ent

met

h-

ods

�R

elax

atio

nst

rate

gies

(plu

sp

erso

na-

lized

rela

xati

onta

pe)

�C

ogn

itiv

est

rate

gies

(ch

angi

ng

neg

a-

tive

thin

kin

g)in

clu

din

gth

ough

t

jou

rnal

�H

ealt

hy

lifes

tyle

choi

ces

�D

evel

opin

gp

lan

tom

anag

ep

ain

,

reco

gniz

ep

rogr

ess,

and

man

agin

gth

e

pro

gram

�T

wo

chap

ters

for

par

ents

onp

ro-

mot

ing

hea

lth

ybeh

avio

r

�Pap

eror

onlin

ep

ain

dia

ry(p

arti

ci-

pan

tch

oice

)

Em

ail

con

tact

(5

tim

es)þ

tele

ph

one

con

tact

(3ti

mes

)

NR

�N

Ra

for

rese

arch

team

�St

and

ard

ized

web

-

bas

edm

ater

ial

�St

and

ard

ize

sch

edu

le

for

emai

l/p

hon

e

con

tact

7O

ne

chap

-

ter

per

wee

k

7w

eeks

Con

tin

ued

Systematic Review of Internet Interventions for Youth with Health Conditions 501

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 8: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

Tab

leII

.C

on

tin

ued

Less

on

s/se

ssio

ns

Art

icle

Co

nte

nt

of

inte

rven

tio

n(p

lus

inte

rac-

tive

featu

res)

Co

nta

ctU

sage

Treatm

en

tin

tegri

tyN

um

ber

Len

gth

Du

ratio

n

(3)

Jan

etal

.

(2007)

Bas

icin

form

atio

non

asth

ma

self-

man

agem

ent

�A

ctio

np

lan

�E

lect

ron

icsy

mp

tom

dia

ry

�R

etri

eval

anal

ysis

syst

emto

revi

ew

accu

mu

late

dd

ata

onsy

mp

tom

san

d

PE

RF

vari

abili

ty

MD

feed

bac

k

rega

rdin

gch

ange

s

inm

edic

atio

ns

via

emai

lor

tele

ph

one

bas

edon

sym

p-

tom

sas

nee

ded

Dia

ryad

her

ence

was

sign

ific

antl

y

hig

her

inin

tern

et

grou

pbu

t

dec

lined

over

tim

e

inbot

hgr

oup

s

�N

Ra

for

rese

arch

team

/cas

em

anag

er

NR

aN

Ra

12

wee

ks

(4)

Jose

ph

et

al.

(2007)

�A

sth

ma

pat

hop

hys

iolo

gy

�T

rigg

erav

oid

ance

�Sy

mp

tom

man

agem

ent

(med

ica-

tion

s,co

ntr

olle

rm

edic

atio

nad

her

-

ence

,co

rrec

tu

seof

MD

I,et

c)

�Sm

okin

gce

ssat

ion

/red

uct

ion

�M

essa

ges

voic

edov

erto

acco

mm

o-

dat

elo

wlit

erac

y

Pro

acti

veco

nta

ct

by

refe

rral

coor

di-

nat

oras

nee

ded

Inte

rnet

stu

den

ts

mor

elik

ely

to

com

ple

teal

lfo

ur

sess

ion

sco

mp

ared

toco

ntr

olgr

oup

�N

Ra

for

rese

arch

team

4E

stim

ated

tim

eto

com

ple

te

each

ses-

sion

was

30

min

ute

s

180

day

s

pos

t

bas

elin

e

(5)

Kri

shn

a

etal

.(2

003)

�B

asic

pat

hop

hys

iolo

gyof

asth

ma

�E

nvi

ron

men

tal

trig

gers

�Sy

mp

tom

man

agem

ent

wit

hqu

ick-

relie

fan

dco

ntr

olm

edic

atio

ns

�St

rate

gies

toco

ntr

olan

dm

anag

e-

men

tas

thm

a

�Se

lf-m

anag

emen

tst

rate

gies

(dec

i-

sion

-mak

ing

skill

s,sy

mp

tom

reco

g-

nit

ion

,co

mm

un

icat

ing

wit

hm

edic

al

team

)

�A

nim

ated

and

real

-life

vign

ette

s

�Q

uiz

zes

aten

dof

each

sess

ion

NR

40–1

00%

ofp

ar-

ents

com

ple

ted

thre

ese

ssio

ns;

wh

ile7-

to17-y

ear

old

sw

ho

com

-

ple

ted

visi

t3

had

mas

tere

d48–

100%

ofm

ater

ial

and

com

ple

ted

58%

orm

ore

of

the

pro

gram

�N

Ra

for

rese

arch

team

44

En

tire

pro

-

gram

can

be

com

-

ple

ted

in

80

min

-

ute

s;

vari

ed

bas

edon

du

rati

onof

clin

icvi

sit.

12

mon

ths

(6)

Rit

terb

a-

nd

etal

.

(2003

a )

�T

hre

eco

rem

odu

les

(an

atom

y/p

hy-

siol

ogy,

med

icat

ion

,an

dbeh

avio

ral

trea

tmen

t)

�27

trea

tmen

t/ed

uca

tion

alm

odu

les

(e.g

.,fe

ars

ofto

ilet

use

;so

cial

isol

a-

tion

;ad

min

iste

rin

g,ad

just

ing,

and

tap

erin

gla

xati

ves;

die

t;h

ygie

ne;

and

pre

ven

tin

gre

lap

se)

�F

ollo

w-u

pm

odu

le

�A

ud

iofo

rte

xtp

rese

nte

d

�Pri

nta

ble

sum

mar

ies

aten

dof

each

sect

ion

and

qu

izze

s

�A

nim

atio

ns

Con

tact

ed3

tim

es

toan

swer

qu

es-

tion

sre

gard

ing

use

ofsi

te

Site

acce

ssed

aver

-

age

of14

tim

es

du

rin

g3

wee

k

per

iod

Tw

o

mod

ule

sw

ere

nev

erac

cess

edby

par

tici

pan

ts

�N

Ra

for

rese

arch

team

�B

oth

grou

ps

wer

e

calle

dat

regu

lar

inte

rval

s

3p

lus

add

i-

tion

alm

odu

les

bas

edon

ind

i-

vid

ual

nee

ds

60–9

0m

in

toco

m-

ple

teco

re

mod

ule

s;

5–1

0m

in

for

trea

t-

men

tm

od-

ule

s;

follo

w-u

p

sess

ion

s

com

ple

ted

1w

eek

afte

rco

re

mod

ule

s

3w

eeks

502 Stinson, Wilson, Gill, Yamada, and Holt

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 9: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

(7)

Ru

nge

etal

.(2

006)

SPM

P

�Pro

per

use

ofd

iffe

ren

tin

hal

er

dev

ices

,cl

assi

fica

tion

ofre

lieve

ran

d

con

trol

ler

med

icat

ion

s,av

oid

ance

of

asth

ma

trig

gers

,lim

itat

ion

ofal

lerg

en

exp

osu

re,

pea

kflow

mea

suri

ng,

and

its

inte

rpre

tati

onIE

P

�A

dd

itio

nal

asth

ma

mod

ule

wit

h

qu

iz

�In

tera

ctiv

ead

ven

ture

gam

e

�SP

MP

mat

eria

l

�M

edic

alm

odu

lein

clu

din

gin

div

i-

du

alm

edic

atio

np

lan

,sc

hed

ule

d

chat

sw

ith

asth

ma

exp

ert,

and

on-l

ine

pea

kflow

pro

toco

l

�C

hat

room

Abili

tyto

com

mu

-

nic

ate

wit

hh

ealt

h

care

pro

vid

ers

via

emai

l(n

ot

sch

edu

led

)

Log

ged

onan

aver

age

of2

h/

mon

th.

Pea

k

flow

pro

toco

l,ch

at

room

,an

dad

ven

-

ture

gam

ew

ere

mos

tw

idel

yu

sed

�N

Ra

for

rese

arch

team

�N

Ra

for

thos

e

adm

inis

teri

ng

SPM

P

SPM

P—

5IE

P—

1

add

itio

nal

mod

ule

SMPM

—2

hou

rs

IEP—

NR

a

NR

a

(8)

Wad

e

etal

.(2

006)

�O

verv

iew

and

goal

iden

tifica

tion

�Im

por

tan

ceof

atti

tud

e

�Pro

ble

mso

lvin

g

�C

ogn

itiv

ean

dbeh

avio

rch

ange

s

�C

omm

un

icat

ion

�C

risi

sm

anag

emen

t

�Pla

nn

ing

for

the

futu

re

�St

ress

and

ange

rm

anag

emen

t

�W

orki

ng

wit

hth

esc

hoo

l

�Si

blin

gco

nce

rns

�Pai

nm

anag

emen

t

�M

arit

alco

mm

un

icat

ion

�V

ideo

clip

s,p

ract

ice

exer

cise

s

�C

hat

room

�vi

deo

con

fere

nci

ng

Reg

ula

rly

sch

ed-

ule

dvi

deo

con

fer-

ence

sw

ith

ther

apis

tev

ery

1–

2w

eeks

Tot

aln

um

ber

of

vid

eoco

nfe

ren

ces

ran

ged

from

0-1

2

(M¼

9.5

0,

SD¼

2.1

9).

Th

e

tota

ln

um

ber

of

sess

ion

sco

m-

ple

ted

ran

ged

from

1–2

4w

ith

fam

ilies

rece

ivin

g

onav

erag

etw

o

sup

ple

men

tal

sess

ion

s.

�E

xten

sive

trai

nin

gof

inte

rvie

wer

san

d

ther

apis

t

�In

terv

enti

onad

mi-

nis

tere

dby

sin

gle

ther

apis

t

�D

etai

led

trea

tmen

t

man

ual

�E

nd

ofse

ssio

n

chec

klis

ts

�W

eekl

ysu

per

visi

on

ofth

erap

ist

14

NR

aN

Ra

(9)

Will

iam

s-

onet

al.

(2006)

�N

utr

itio

ned

uca

tion

�C

oun

selli

ng

�Lifes

tyle

and

ph

ysic

alac

tivi

ty

�W

eigh

tan

dp

hys

ical

acti

vity

grap

hs

�T

elev

isio

nse

lf-m

onit

orin

g

�Pro

ble

mso

lvin

g

�B

ehav

iora

lco

ntr

acti

ng

�Lin

ksto

Afr

ican

-Am

eric

anh

ealt

h

web

site

s

�Lin

ksto

wom

en’s

hea

lth

web

site

s

�Q

uiz

zes,

goal

sett

ing,

and

food

mon

itor

ing

wit

hfe

edbac

kvi

aem

ail

from

cou

nse

llor

Inte

rnet

cou

nse

l-

ling

via

wee

kly

emai

lsþ

fou

rfa

ce-

to-f

ace

cou

nse

llin

g

sess

ion

s

Web

site

use

dec

reas

edov

er

tim

ew

ith

sign

ifi-

can

td

ecre

ase

in

year

2co

mp

ared

toye

ar1

�N

Ra

for

rese

arch

team

52

NR

a24

mon

ths

a NR

,n

otre

por

ted

.

Systematic Review of Internet Interventions for Youth with Health Conditions 503

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 10: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

assessment and methodological quality rating as outlined

in Fig. 1. A list of the excluded articles is available from the

first author on request.

Characteristics of Study Participants

Summary data for each study and details of internet inter-

ventions are summarized in Tables I and II, respectively.

Studies included children with asthma, recurrent pain,

encopresis, traumatic brain injury and obesity. The

number of participants ranged from 24 to 438 and

involved a total of 1415 children, adolescents and adults.

Four of the studies specifically targeted youth (Studies: 1,

3, 4, 7); while five interventions targeted youth and their

parents (Studies: 2, 5, 6, 8, 9), with one of those interven-

tions being geared to the entire family including siblings

(Study: 8). One study on weight loss specifically targeted

female adolescents (Study: 9). Two of the studies targeted

African-American participants (Studies: 4, 9), two studies

were comprised of mainly Caucasians (Studies: 5, 8), one

study was comprised of Taiwanese participants (Study: 3),

and the remaining studies did not describe the partici-

pants’ ethnicity (Studies: 1, 2, 6, 7).

Characteristics of Interventions

There was marked variability both in the content and pro-

cedural aspects of the interventions (i.e., therapist involve-

ment) as shown in Table II. The description of the

interventions in the publications themselves was extremely

varied with respect to details of both content and proce-

dure. The interventions also varied in length, duration, and

the number of sessions. The shortest intervention lasted

3 weeks and the longest was 2 years, with a mean of 36.53

weeks and a standard deviation of 35.93 weeks. Six were

solely in-home interventions (Studies: 2, 3, 6–9), while one

was a combination of clinic and home-based sessions

(Study: 1). In the remaining two studies, participants

accessed the intervention at school in one study (Study:

4) and during regular clinic visits in the other (Study: 5).

Participants in the control groups in three studies were

provided with internet restricted access to general illness

specific information in addition to standard medical care

(Studies: 4, 8, 9); while two were waitlist control groups

(Studies: 2, 7), three provided standard education and

medical care (Studies: 1, 3, 5), and one was a usual med-

ical care group (Study: 6).

All of the interventions were psycho-educational in

nature, involving combinations of information and skills

training modules targeting improved health outcomes.

Most skills training embraced self-management skills direc-

ted to a variety of issues, including symptom management

(recognizing symptoms, how to use medications, avoiding

triggers; Studies: 1–9), cognitive restructuring to reduce

pain and/or stress (Studies: 2, 5, 6, 8), and promote pro-

blem solving/decision-making and healthy lifestyle choices

(Studies: 2, 5, 6, 8, 9) as outlined in Table II. Specific

therapeutic techniques included cognitive (e.g., changing

negative thoughts, positive self-talk) and/or behavioral

therapy (e.g., modeling, reinforcement, and self-mastery)

and educational modules specific to the illness concerned.

Three of the interventions were described as behavioral

therapies (Studies: 5, 6, 9), two as Cognitive–Behavioral

Therapy (CBT) (Studies: 2, 8), while four interventions did

not describe the techniques used in detail (Studies: 1, 3, 4,

7). Parents were included in five of the nine studies as

active participants (Studies: 2, 5, 6, 8, 9). One study had

specific modules for parents (Study: 2) and in two studies

children and parents were asked to complete the sessions

together (Studies: 6, 8). Six of the nine studies included

electronic symptom monitoring (pain diary, asthma

attacks, weight monitoring) as part of the intervention

(Studies 1–3, 5, 7, 9). Two studies included chat rooms

as part of the intervention (Studies: 7, 8).

One of the benefits of internet-based interventions is

that they allow for standardization of the treatment.

However, the majority of these interventions also involved

the role of a health coach or therapist to help tailor the

information and cognitive–behavioral strategies to partici-

pant’s individual needs. Very few studies reported the mea-

sures that were taken to ensure treatment integrity

especially regarding therapist contact. Of those that did,

only one elaborated on the training that the interviewers

29 Internet-based articlesretrieved and assessed

for relevance

699 Articles assessedfor relevance

806 Articles identifiedand screened

107 Duplicates

673 Excluded based oninclusion/exclusion criteria

20 Internet-based articles excluded based on:Intervention• Primary prevention = 8 • Social support without education = 1 Population• Acute hospitalized children = 2 • Parents = 1 Design• Pilot RCTs of interventions included in review = 2 • RCTs (same subjects; Williamson et al., 2006) = 2 • Qualitative study = 1 • Usability testing = 2 • Pilot non-RCT = 1

9 Unique studies onInternet-based

self-management studies

Figure 1. Study selection.

504 Stinson, Wilson, Gill, Yamada, and Holt

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 11: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

and therapist received to ensure implementation fidelity

(Study: 8). Three studies described the use of a standar-

dized protocol that was followed by the participants in the

intervention and control groups (Studies: 1, 2, 6), while

one study incorporated the use of specific treatment integ-

rity tests such as checklists or periodic observation by a

third party (Study: 8).

Outcomes of Studies

The studies included in this review evaluated a variety of

outcome measures as shown in Table I. Improving health

outcomes in terms of symptom management or disease

control were investigated in all of the studies using a

wide variety of outcomes (e.g., symptom free days, use of

medications, days of school missed, and activity restric-

tions). Seven of the nine studies demonstrated significant

improvements in health outcomes compared to the control

groups (Studies: 2–7, 9). Four of the nine studies exam-

ined the effects of treatment on disease related knowledge

(Studies: 1, 3, 5, 6); knowledge significantly increased

compared to the control group in two studies on asthma

(Studies: 3, 5) but not in a third asthma study (Study: 1)

and the encopresis study (Study: 6). Six of the nine studies

investigated the effect of the treatment on the participants’

QOL (Studies 1–4, 5, 7); two found a significant difference

from the control group (Studies: 3, 7).

Four of the nine studies, all involving asthma partici-

pants, reported on the use of health care resources

(Studies: 1, 4, 5, 7); two studies found significant

decreases in utilization of health care resources (i.e., emer-

gency room visits, physician consultations) (Studies: 5,7),

one found significant decreases in emergency room visits,

but not hospitalizations (Study: 4), and one found no sig-

nificant decreases in the utilization of services (Study: 1)

compared to control groups. None of the studies measured

outcomes on self-efficacy, social support, or emotional

well-being. Finally, timing of outcome measurement

varied greatly; four studies measured outcomes pre- and

post-intervention (Studies: 2, 3, 6, 8), four studies mea-

sured outcomes up to 1-year (Studies: 1, 4, 5, 7) and one

study assessed durability of treatment outcomes at regular

6-month intervals for 2 years (Study: 9). Interestingly, this

study revealed that the significant reductions in weight loss

at 6-months postintervention were not maintained over the

remaining 2-year follow-up period.

Four studies investigated the cost-effectiveness of the

intervention (Studies: 2, 3, 5, 7). However, none evaluated

cost-effectiveness from the participants’ perspective (i.e.,

travel time and/or time off work). Instead, cost-effectiveness

was appraised from the perspectives of labor costs,

resource utilization, health insurance, and societal costs.

In one study, the intervention was estimated to be 5.5

times more cost-effective than the traditional office-based

program with respect to resource utilization, specifically the

therapist’s time (Study: 2). Krishna and colleagues (2003)

found a reduction in emergency room visits resulting from

the intervention that translated into a savings of approxi-

mately $907.10 per child as opposed to the group receiving

solely traditional patient education who only realized a sav-

ings of $291.40 per child. One study looking at costs of

program delivery found an estimated labor cost of $6.66

per treatment student for a referral co-ordinator as the

students were provided access to computers in the

school setting (Study: 4). The study whose main focus

was the economic implications of an internet-based edu-

cation program (IEP), found that the cost-benefit ratio

improved from 0.55 to 0.79 when adding an IEP to a tradi-

tional education program (Study: 7). Runge and colleagues

(2006) also observed incremental morbidity cost savings,

with cost-benefit ratios of 1.07 and 1.42 for patients with

moderate or severe asthma, for standard education and

IEP, respectively. Although cost savings did not exceed

program costs in the entire study population, the signifi-

cant decline in morbidity costs indicates that IEPs may

yield further cost savings in the long term (Study: 7).

All of the studies reported on the feasibility of the

interventions. Most often, feasibility was discussed in

terms of treatment efficiency, access and compliance.

Two studies reported that the intervention might be

more suitable for certain patient populations (Studies: 8,

9). For example, an internet-based cognitive–behavioral

program was more effective for older children and those

of lower socio-economic status (SES) compared to younger

children and those with higher SES in children with trau-

matic brain injury (Study: 8). In another study, Williamson

et al. (2006) argued that internet interventions may over-

come obstacles to traditional face-to-face interventions and

be considered more feasible for African-Americans. Of the

six studies requiring participants to use a home computer,

three included internet access as one of their inclusion

criteria (Studies: 1–3), and three provided participants

with computers and free internet access (Studies: 6, 8,

9). The remaining studies required them to be able to

access it either at home, school or in a clinic setting

(Studies: 4, 5, 7). Compliance varied greatly across the

studies. One of three studies including an electronic

diary component showed very poor compliance in both

the intervention and control group with no diary entry

on 60% of study days (Study: 1). Furthermore, the two

studies having a duration of at least 1 year showed a steady

Systematic Review of Internet Interventions for Youth with Health Conditions 505

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 12: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

decline in website usage throughout the study period

(Studies: 1, 9).

Five of the nine studies reported on the usability of the

intervention (Studies: 2, 3, 5, 6, 8). In one study, more

than 80% of the children rated the website and videocon-

ferencing as easy to use; however, 25% perceived that the

website was not as easy to use when compared to other

sites used for health information (Study: 8). Hicks and

colleagues (2006) found that participants identified the

treatment website as ‘‘one of the well-designed aspects of

the program’’ (p. 732). However, with respect to helpful-

ness, the therapist’s telephone calls were endorsed more

frequently than the website. One study measured the par-

ticipant’s attitude toward computers (i.e. enjoyment, satis-

faction and comfort with computers) at baseline; which

was found to positively correlate with greater website use

for parents in the behavioral group (Study: 9). Finally, few

studies reported on the usage of the internet interventions

(e.g., session time and site visits).

Methodologic Quality of Studies

Using the 23-item CONSORT statement checklist for non-

pharmacological RCTs (Boutron et al., 2008a), the studies

methodological quality ranged from a score of 10–16 out

of 23 with mean of 13.77 and standard deviation of 2.22 as

outlined in Table I. Given that all of the studies fell within

the fair quality range it was difficult to determine whether

outcomes differed by methodological quality scores.

However, it should be noted that the pilot RCT (Study:

6) and the quasi-experimental study design (Study: 7)

had lower quality scores than six of the seven RCTs. Two

of the nine studies provided precise details on: (a) inter-

ventions intended for each group, and how and when they

were administered; (b) how the interventions were stan-

dardized; (c) details on different components of the inter-

vention and how they were tailored to individual patients;

and (d) how adherence of care providers (health coach,

therapist) with protocol was assessed or enhanced (Stu-

dies: 3, 8). The majority of papers lacked details regarding

randomization procedures (Studies: 2, 3, 5, 6, 7, 9). As

with many psychological interventions, it usually is not

possible for researchers and participants to be blind to

the intervention a participant is receiving. Therefore,

there is a potential risk of reporting or scoring bias that

might mask a real intervention effect. Bias could be mini-

mized by the use of procedures such as blinding of coders

or using web-administered questionnaires as was done in

two of the studies (Studies: 5, 8). Five of the nine studies

did not report sample size calculations (Studies: 2, 4, 6–8).

However, five of the nine studies used an intention-to-treat

analysis that would indicate how the overall result was

affected by participant attrition (Studies: 1, 2, 4, 7, 8).

Finally, while monitoring for adverse effects in psychologi-

cal trials is not common practice, one of the nine studies

mentioned the method used to assess for adverse effects in

intervention and control groups (Study: 4).

Discussion

This study sought to critically appraise the research evi-

dence on the effectiveness of self-management interven-

tions delivered by the internet for youth with health

conditions. In the past 5 years, nine studies using rigorous

study designs (eight RCTs and one quasi-experimental)

have examined the effectiveness of the internet as a

medium for delivering self-management interventions to

youth with health conditions. Most of these were pub-

lished in the past 2 years and it can be expected that the

number of trials will continue to grow exponentially.

Overall, there is consistent evidence that they lead to

improvements in symptom or disease control in four of

the five health conditions. However, there is limited evi-

dence regarding their impact on health care utilization

(some evidence with asthma), knowledge and quality of

life outcomes. We were unable to determine the effective-

ness of internet interventions on self-efficacy, social sup-

port or emotional well-being.

Previous systematic reviews and meta-analyses of psy-

chological interventions for children with chronic illnesses

have found moderate-to-strong evidence for their effective-

ness (Bauman, Drotar, Leventhal, Perrin, & Pless, 1997;

Beale, 2006; Kibby, Tyc, & Mulhern, 1998). However, our

results are best interpreted in the context of findings from

other internet-based psychological intervention studies.

Several recent systematic reviews have found that internet

interventions have been effective in improving symptoms

in adults with mental and other chronic health conditions

(Cuijpers et al., 2008; Griffiths & Christensen, 2006;

Murray et al., 2005; Spek et al., 2007; Wantland et al.,

2004); however, the effects appear to differ across target

conditions (Cuijpers et al., 2008). For example, Cuijper

and colleagues (2008) found that internet interventions

targeting pain and headaches were comparable to face-to-

face treatments; however, they found that the effectiveness

differed across other target populations such as those with

chronic health conditions. In contrast to these findings, we

found improvements in symptoms in four of the five health

conditions. Finally, the studies included in our review did

not allow us to draw definite conclusions about whether

self-management interventions delivered through the

506 Stinson, Wilson, Gill, Yamada, and Holt

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 13: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

internet are as effective as face-to-face therapies as most of

the studies used usual care or wait-list control comparison

groups.

Wantland and colleagues (2004) conducted a sys-

tematic review and meta-analysis of 22 studies that com-

pared web-based to non-web-based self-management

interventions in adults. They found substantial evidence

that use of internet interventions improved behavioral

change outcomes in terms of increased exercise time,

increased knowledge of nutritional status, increased

knowledge of asthma treatment, increased participation

in healthcare, slower health decline, improved body

shape perception, and 18-month weight loss maintenance

compared to non-web-based interventions across a wide

variety of chronic conditions. Those interventions that

directed the participant to relevant, individually tailored

materials reported longer internet usage (session time

and site visits). Additionally, those sites that incorporated

the use of a chat room demonstrated increased social sup-

port scores. Unfortunately, few studies in our review

reported on internet usage or utilized chat rooms. In keep-

ing with our review and others (Murray et al., 2005), they

were not able to determine the durability of treatment

effects or cost-effectiveness of internet therapies.

The internet interventions in our review varied in the

degree of interactivity that was provided through anima-

tions or games, video and audio clips, use of chat rooms

for social support, and feedback and decision-making sup-

port from a therapist. All of these additional features that

enhance interactivity may influence outcomes, dropout

rates, as well as compliance with and use of the internet

site (Wantland et al., 2004). Poor adherence is a significant

issue for most health interventions, including internet

interventions. Therefore, it would be important to identify

characteristics of the interventions as well as end-users

(cognitive development in children and youth) that affect

adherence to internet interventions. Nijland and colleagues

(Nijland, van Gemert-Pijnen, Boer, Steehouder, & Seydel,

2008) recently examined patients’ and caregivers’ percep-

tions of the user-friendliness of internet applications for

supporting self-care. They found that inadequate naviga-

tion structures, search options, and lack of feedback fea-

tures hindered the usability of the applications.

Furthermore, participants felt the information was not suf-

ficiently tailored to meet their individual information needs

and that the language/readability level of the information

provided appeared to be a barrier to using self-care advice.

In addition to conducting usability testing studies,

researchers should use the tools offered by this medium

to track the use of an internet site (visits to pages, dwell

time). Utilization patterns would help to determine what

dose and which components of the interventions are most

effective.

The main aim of these internet interventions was to

produce cognitive and behavior change that leads to symp-

tom improvement (Ritterband et al., 2006). A theory-driven

approach would further our understanding of how inter-

net interventions work. Theory should be used to guide

the development of these interventions as well as study

design and outcome measurement (Sidani & Braden,

1997). Models specific to internet interventions are

needed as there are obvious differences in treatment deliv-

ery from traditional interventions (Ritterband et al., 2006).

Murray and colleagues (2005) have recently developed

a pathway for change whereby internet self-management

interventions are hypothesized to work by combining

information with peer, decision-making, and behavioral

change supports to allow internalization and interpretation

of the information by the user. A combination of enhanced

self-efficacy with motivation and knowledge enables users

to change their health behaviors, leading to changes in

clinical outcomes. The use of a theoretical framework

would help standardize the essential ingredients of self-

management programs (i.e., information, skills training,

social support), interactive features (e.g., feedback), and

use of therapist or health coach as well as help to standar-

dize outcomes for internet interventions (Clarke, 2007).

Future Directions for Research

The present study also revealed areas for future research on

internet self-management interventions. First, more high

quality studies with adequate samples sizes, using both

clinical and community samples of children with a wide

range of health conditions are needed to confirm these

findings and to elucidate the best type and way to deliver

internet interventions, and to conduct cost-benefit analyses

comparing them to standard approaches (e.g., face-to-face,

group formats). Second, due to the cost of developing

internet interventions, it would be beneficial to develop

consensus on the steps to take in the development and

testing of internet interventions to ensure their usability

and feasibility. These steps are crucial to determine that

participants can be identified, recruited and maintained in

the project, that the intervention can be delivered consis-

tently, and that the end-users are satisfied with and recep-

tive to these modes of intervention delivery. Third, greater

uniformity of assessment intervals across studies is recom-

mended in order to more effectively evaluate the long-term

effects of these interventions. Finally, more research is

needed to elucidate the mechanism of action of internet

Systematic Review of Internet Interventions for Youth with Health Conditions 507

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 14: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

self-management and how it differs from traditional meth-

ods of delivering self-management interventions.

Limitations

The present study has several limitations. First, because

few internet interventions targeting self-management in

youth with health conditions have been conducted over

the past 15 years, our findings are based on a very small

number of studies and five illnesses. Future independent

replications will be required to establish whether internet

treatment approaches are reliably efficacious for treating a

wide variety of acute and chronic illnesses. Due to the

limited data reported in the studies included in this

review, we were only able to provide a qualitative synthesis.

Furthermore, conducting systematic and meta-analytic

reviews on internet interventions is challenging due to

the wide variability in the treatment programs, as well as

differences in nature of health conditions and their treat-

ments. In addition, several factors may limit the general-

izability including limiting studies to English and that the

studies found were primarily conducted in North America.

Conclusions

The internet shows great promise as a mode of delivering

self-management interventions for youth with health con-

ditions. In the past, psychological interventions aimed at

promoting self-management in traditional formats (group

or individual formats) have been found to be effective.

Evidence from this review lends support to the delivery

of these types of interventions using the internet to

improve symptoms in youth with health conditions.

Further research is necessary to increase the scope of

these findings using rigorous study designs so we can har-

ness the full potential of the internet as a medium for

delivering self-management interventions.

Acknowledgements

We acknowledge Cheri Nickel, BSW, MLS for assistance

with the electronic searches. We thank the reviewers for

their helpful comments on earlier versions of the

manuscript.

Funding

Canadian Institute for Health Research (CIHR) Post-

Doctoral Fellowship (to J.S.); CIHR Doctoral Fellowship

(to J.Y.); Summer Research Studentship through the

Department of Anesthesia and Pain Medicine (to J.H.).

Conflicts of interest: None declared.

Received March 6, 2008; revisions received October 9,

2008; accepted October 14, 2008

References

Barlow, J. H., & Ellard, D. R. (2004). Psycho-educational

interventions for children with chronic disease, par-

ents and siblings: An overview of the research evidence

base. Child: Care, Health and Development, 30,

637–645.

Barlow, J., Wright, C., Sheasby, J., Turner, A.,

& Hainsworth, J. (2002). Self-management for people

with chronic conditions: A review. Patient Education

and Counselling, 48, 177–187.

Bauman, L. J., Drotar, D., Leventhal, J. M., Perrin, E. C.,

& Pless, I. B. (1997). A review of psychosocial inter-

ventions for children with chronic health conditions.

Pediatrics, 100, 244–251.

Beale, I. L. (2006). Scholary literature review: Efficacy of

psychological interventions for pediatric chronic ill-

ness. Journal of Pediatric Psychology, 31, 437–451.

Boutron, I., Moher, D., Altman, D. G., Schulz, K. F.,

& Ravaud, P. for the CONSORT Group. (2008a).

Extending the CONSORT Statement to randomized

trials of nonpharmacological treatment: Explanation

and elaboration. Annals of Internal Medicine, 148,

295–309.

Boutron, I., Moher, D., Altman, D. G., Schulz, K. F.,

& Ravaud, P. for the CONSORT Group. (2008b).

Methods and processes of the CONSORT Group:

Example of an extension for trials assessing nonphar-

macologic treatments. Annals of Internal Medicine, 148,

W60–W66.

Chan, D. S., Callahan, C. W., Hatch-Pigott, V. B.,

Lawless, A., Proffitt, H. L., Manning, N. E., et al.

(2007). Internet-based home monitoring and educa-

tion of children with asthma is comparable to ideal

office-based care: results of a 1-year asthma in-home

monitoring trial. Pediatrics, 119(3), 569–578.

Clarke, M. (2007). Standardizing outcomes for clinical

trials and systematic reviews. Trials, 8(39), 1–3.

Cuijpers, P., van Straten, A., & Andersson, G. (2008).

Internet-administered cognitive behavior therapy for

health problems: a systematic review. Journal of

Behavioral Medicine, 31, 169–177.

Elgar, F. J., & McGrath, P. J. (2003). Self-administered

psychosocial treatments for children and families.

Journal of Clinical Psychology, 59, 321–339.

508 Stinson, Wilson, Gill, Yamada, and Holt

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 15: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

Fiese, B. H., & Everhart, R. S. (2006). Medical adherence

and childhood chronic illness: family daily

management skills and emotional climate as

emerging contributors. Current Opinion in Pediatrics,

18, 551–557.

Gray, N. J., Klein, J. D., Noyce, P. R., Sesselberg, T. S.,

& Cantrill, J. A. (2005). Health information-seeking

behavior in adolescents: The place of the Internet.

Social Science and Medicine, 20, 1467–1478.

Griffiths, K. M., & Christensen, H. (2006). Review of

randomised controlled trials of internet interventions

for mental disorders and related conditions. Clinical

Psychologist, 10(1), 16-1-29.

Hicks, C. L., von Baeyer, C. L., & McGrath, P. J. (2006).

Online psychological treatment for pediatric recurrent

pain: A randomized evaluation. Journal of Pediatric

Psychology, 31, 724–736.

Jan, R., Wang, J., Huang, M., Tseng, S., Su, H., & Liu, L.

(2007). An internet-based interactive telemonitoring

system for improving childhood asthma outcomes in

Taiwan. Telemedicine and e-Health, 13(3), 257–268.

Joseph, C. L., Peterson, E., Havstad, S., Johnson, C. C.,

Hoerauf, S., Stringer, S., et al. (2007). A Web-based,

tailored asthma management program for urban

African-American high school students. American

Journal of Respiratory Critical Care Medicine, 175,

888–895.

Kibby, M. Y., Tyc, V. L., & Mulhern, R. K. (1998).

Effectiveness of psychological interventions for chil-

dren and adolescents with chronic illness: A meta-

analyasis. Clinical Psychology Review, 18(1), 103–117.

Krishna, S., Balas, A. E., Spencer, D. C., Griffin, J. Z.,

& Boren, S. A. (1997). Clinical trials of interactive

computerized patient education: Implications for

family practice. Journal of Family Practice, 45, 25–33.

Krishna, S., Francisco, B. D., Balas, E. A., Konig, P.,

Graff, G. R., & Madsen, R. W. (2003). Internet-

enabled interactive multimedia asthma education

program: a randomized trial. Pediatrics, 111(3),

503–510.

Last, B. F., Stam, H., Onland-van Nieuwenhuizen, A. M.,

& Grootenhuis, M. A. (2007). Positive effects of a

psycho-educational group intervention for children

with a chronic illness: First results. Patient Education

and Counseling, 65, 101–112.

Marks, R., & Allegrante, J. P. (2005). A review and

synthesis of research evidence for self-efficacy-enhan-

cing interventions for reducing chronic disability:

Implications for health education practice (Part II).

Health Promotion Practice, 6, 148–156.

Moher, D., Schulz, K. F., & Altman, D. G. (2001). The

CONSORT statement: revised recommendations for

improving the quality of reports of parallel-group

randomised trials. Lancet, 357, 1191–1194.

Murray, E., Burns, J., See, T. S., Lai, R., & Nazareth, I.

(2005). Interactive Health Communication

Applications for people with chronic disease. The

Cochrane Database of Systematic Reviews, 4, 1–25.

Newacheck, P. W., Strickland, B., Shonkoff, J. P.,

Perrin, J. M., McPherson, M., McManus, M., et al.

(1998). An epidemiologic profile of children with

special health care needs. Pediatrics, 102, 117–123.

Nijland, N., van Gemert-Pijnen, J., Boer, H.,

Steehouder, M., & Seydel, E. (2008). Evaluation of

internet-based technology for supporting problems

encountered by patients and caregivers when using

self-care applications. J Med Internet Res, 10(2), e13.

Ritterband, L. M., Andersson, G., Christensen, H. M.,

Carlbring, P., & Cuijpers, P. (2006). Directions for the

International Society for Research on Internet

Interventions (ISRII). Journal of Medical Internet

Research, 8(3), e23.

Ritterband, L. M., Cox, D. J., Walker, L. S., Kovatchev, B.,

McKnight, L., Patel, K., et al. (2003a). An internet

intervention as adjunctive therapy for pediatric

encopresis. Journal of Consulting and Clinical

Psychology, 71(5), 910–917.

Ritterband, L. M., Gonder-Frederick, L. A., Cox, D. J.,

Clifton, A. D., West, R. W., & Borowitz, S. M.

(2003b). Internet interventions: in review, in use, and

into the future. Professional Psychology: Research &

Practice, 34(5), 527–534.

Runge, C., Lecheler, J., Horn, M., Tews, J., & Schaefer, M.

(2006). Outcomes of a web-based patient education

program for asthmatic children and adolescents. Chest,

129(3), 581–593.

Sawyer, S. M., & Aroni, R. A. (2005). Self-management in

adolescents with chronic illness. What does it mean

and how can it be achieved? Medical Journal of

Australia, 183, 405–409.

Sidani, S., & Braden, C. J. (1997). Evaluating Nursing

Interventions: a theory-driven approach. Sage Publishing.

Spek, V., Cuijpers, P., Nyklıcek, I., Riper, J., Keyzer, J.,

& Pop, V. (2007). Internet-based cognitive behavior

therapy for mood and anxiety disorders: A meta-ana-

lysis. Psychological Medicine, 37, 319–328.

Stinson, J., McGrath, P. J., & Yamada, J. (2003). Clinical

trials in the Journal of Pediatric Psychology: Applying

the CONSORT Statement. The Journal of Pediatric

Psychology, 28(3), 159–167.

Systematic Review of Internet Interventions for Youth with Health Conditions 509

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from

Page 16: A Systematic Review of Internet-based Self-Management Interventions for Youth with Health Conditions

Wade, S. L., Carey, J., & Wolfe, C. R. (2006). The efficacy

of an online cognitive-behavioral, family intervention

in improving child behavior and social competence

following pediatric brain injury. Rehabilitation

Psychology, 51, 179–189.

Wallander, J. L., & Varni, J. W. (1998). Effects of

pediatric chronic physical disorders on child and

family adjustment. The Journal of Child Psychology

and Psychiatry and Allied Disciplines, 39, 29–46.

Wantland, D. J., Portillo, C. J., Holzemer, W. L.,

Slaughter, R., & McGhee, E. M. (2004).

The effectiveness of Web-based vs. non-Web-based

interventions: a meta-analysis of behavioral change

outcomes. J Med Internet Res, 6(4), e40.

Williamson, D. A., Walden, H. M., White, M., York-

Crowe, E., Newton, R. L., Alfonso, A., et al. (2006).

Two-year Internet-based randomized controlled trial

for weight loss in African-American girls. Obesity, 14,

1231–1243.

Yeo, M., & Sawyer, S. (2005). Chronic illness and dis-

ability. British Medical Journal, 330, 721–723.

510 Stinson, Wilson, Gill, Yamada, and Holt

at Ball State U

niversity on Novem

ber 20, 2014http://jpepsy.oxfordjournals.org/

Dow

nloaded from