i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771

journa l homepage: www. int l .e lsev ierhea l th .com/ journa ls / i jmi

Effectiveness of telemedicine: A systematic review ofreviews

Anne G. Ekelanda,∗, Alison Bowesb, Signe Flottorpc,d

a Norwegian Centre for Integrated Care and Telemedicine, University Hospital of North Norway, P.O. Box 6060, N-9038 Tromsø, Norwayb Department of Applied Social Science, University of Stirling, Scotland, UKc Norwegian Knowledge Centre for the Health Services, Oslo, Norwayd Department of Public Health and Primary Health Care, University of Bergen, Norway

a r t i c l e i n f o

Article history:

Received 23 April 2010

Received in revised form

11 July 2010

Accepted 29 August 2010

Keywords:

Telemedicine

Telecare

Systematic review

Effectiveness

Outcome

a b s t r a c t

Objectives: To conduct a review of reviews on the impacts and costs of telemedicine services.

Methods: A review of systematic reviews of telemedicine interventions was conducted. Inter-

ventions included all e-health interventions, information and communication technologies

for communication in health care, Internet based interventions for diagnosis and treat-

ments, and social care if important part of health care and in collaboration with health care

for patients with chronic conditions were considered relevant. Each potentially relevant sys-

tematic review was assessed in full text by one member of an external expert team, using

a revised check list from EPOC (Cochrane Effective Practice and Organisation of Care Group)

to assess quality. Qualitative analysis of the included reviews was informed by principles of

realist review.

Results: In total 1593 titles/abstracts were identified. Following quality assessment, the

review included 80 heterogeneous systematic reviews. Twenty-one reviews concluded that

telemedicine is effective, 18 found that evidence is promising but incomplete and others

that evidence is limited and inconsistent. Emerging themes are the particularly problem-

atic nature of economic analyses of telemedicine, the benefits of telemedicine for patients,

and telemedicine as complex and ongoing collaborative achievements in unpredictable

processes.

Conclusions: The emergence of new topic areas in this dynamic field is notable and review-

ers are starting to explore new questions beyond those of clinical and cost-effectiveness.

Reviewers point to a continuing need for larger studies of telemedicine as controlled inter-

ventions, and more focus on patients’ perspectives, economic analyses and on telemedicine

innovations as complex processes and ongoing collaborative achievements. Formative

assessments are emerging as an area of interest.

© 2010 Elsevier Ireland Ltd. All rights reserved.

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7372. Objectives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738

∗ Corresponding author. Tel.: +47 952 66791.E-mail address: anne.granstrom.ekeland@telemed.no (A.G. Ekeland).URL: http://www.telemed.no (A.G. Ekeland).

1386-5056/$ – see front matter © 2010 Elsevier Ireland Ltd. All rights reserved.doi:10.1016/j.ijmedinf.2010.08.006

Appendix 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742Appendix B. Supplementary data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 769

. . . . .

1

Pitrttaptclscutbtshcie

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. Introduction

revious reviews of telemedicine have concluded thatrrefutable evidence regarding the positive impact ofelemedicine on clinical outcomes still eludes us. Oneeview [1] of more than 150 articles concluded that poten-ial effectiveness could only be attributed to teleradiology,elepsychiatry, transmission of echocardiographic imagesnd consultations between primary and secondary healthroviders. Another systematic review [2] that assessed morehan 1300 papers making claims about telemedicine out-omes found only 46 publications that actually studied ateast some clinical outcomes. A review that analyzed theuitability of telemedicine as an alternative to face-to-faceare [3] concluded that establishing systems for patient caresing telecommunications technologies is feasible; however,he studies provided inconclusive results regarding clinicalenefits and outcomes. A report on peer-reviewed litera-ure for telemedicine services that substituted face-to-faceervices with ICT-based services at home and in offices or

ospitals [4] identified 97 articles that met the inclusionriteria for analysis. The authors concluded that telemedicines being used even if the use is not supported by high qualityvidence. Reviews on cost outcomes have fared similarly. A

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 769

study focused on cost-effectiveness interventions concludedthat there is no good evidence that telemedicine is or is not acost-effective means for delivering healthcare [5].

The quality of studies is a recurrent concern in thesereviews [1,2,4–6]. There is also a debate about appropriateresearch methodologies. For example, economic analysis oftelemedicine has not yet met accepted standards [5]; there isa relative lack of exploration of the socio-economic impactof telemedicine [7]; evidence on factors promoting uptake oftelemedicine is lacking [8]; there is relatively undeveloped use,at the time, of qualitative methods [9]; many studies have notbeen well-designed [4,10]; and, considering perceived difficul-ties of building a robust evidence base for recent innovations,researchers have argued that simulation modelling needs fur-ther development [11].

The lack of consensus raises questions about the qualityof research evidence in terms not only of the data collectedand analysed, but also in terms of the approaches to evalua-tion, that is, the underlying methodologies used, which maynot be capable of addressing the questions to which differentstakeholders seek answers. Others have noted that evaluation

i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771 737

3. Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7383.1. Inclusion criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738

3.1.1. Population/participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7383.1.2. Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7383.1.3. Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7383.1.4. Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7383.1.5. Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738

3.2. Exclusion criteria. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7383.2.1. Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7383.2.2. Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7383.2.3. Interventions considered not relevant for the review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7383.2.4. Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738

3.3. Information sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7393.4. Search. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7393.5. Study selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7393.6. Data collection process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7393.7. Data items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7393.8. Quality of systematic reviews and risk of bias in individual studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7393.9. Summary measures and synthesis of results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739

4. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7395. Telemedicine is effective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7396. Telemedicine is promising. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7407. Evidence is limited and inconsistent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7408. Economic analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7419. Is telemedicine good for patients? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74110. Asking new questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74111. Reflections on the methodology of our study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74212. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742

traditions do not sufficiently collaborate to cross borders andthat a common language for evaluation is missing [12].

This paper reports on research funded under EU SMART2008/0064, which sought to review the evidence on the

i c a l

738 i n t e r n a t i o n a l j o u r n a l o f m e d

effectiveness of telemedicine with particular reference toboth outcomes and methodologies for evaluation. This paperfocuses mainly on the evidence about effectiveness, andassesses the range of conclusions drawn by reviewers aboutthe effectiveness of telemedicine and the gaps in the evidencebase. A companion paper focuses on the methodologicalissues and recommendations [13].

2. Objectives

The objective of the work was to conduct a review of reviewson the impacts and costs of telemedicine services and con-sider qualitative and quantitative results, with the purposeof synthesizing evidence to date on the effectiveness oftelemedicine. The key questions addressed were firstly, howare telemedicine services defined and described in terms ofparticipants, interventions, comparisons and outcome mea-sures; secondly, what are the reported effects of telemedicine:thirdly which methodologies were used to produce knowl-edge about telemedicine in studies included; fourthly, whatare the strengths and weaknesses of these methodologies,including HTA methodologies; and finally what are the knowl-edge gaps and what methodologies can be recommended forfuture research? The present paper addresses the first two ofthese questions, and identifies assessments of the evidencebase provided within the reviews and knowledge gaps in termsof outcomes.

3. Methods

An initial search identified systematic reviews of telemedicinepublished from 1998. A systematic review was defined as anoverview with an explicit question and a method section witha clear description of the search strategy and the methodsused to produce the systematic review. The review shouldalso report and analyse empirical data. In addition, reviewswhich described or summarised methods used in assessingtelemedicine were included. Because of the large number ofreviews retrieved, a decision was taken to include only reviewspublished from 2005 and onwards in the final review.

3.1. Inclusion criteria

3.1.1. Population/participantsSystematic reviews on patients and consumers, health pro-fessionals and family caregivers, regardless of diagnoses orconditions, were included in the searches for systematicreviews.

3.1.2. InterventionsAll e-health interventions, information and communicationtechnologies (ICT) for communication in health care, Internet

based interventions for diagnosis and treatments, and socialcare if an important part of health care and in collaborationwith health care for patients with chronic conditions wereconsidered relevant.

i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771

3.1.3. ComparisonsReviews of studies comparing telemedicine to standard care orto another type of care, as well as reviews of studies comparingdifferent e-health solutions were included.

3.1.4. OutcomesOnly reviews reporting relevant outcomes were included,specified as health related outcomes (morbidity, mortality,quality of life, patient’ satisfaction), process outcomes (qual-ity of care, professional practice, adherence to recommendedpractice, professional satisfaction) and costs or resource use.Systematic reviews reporting emerging issues, such as anunexpected finding or important new insights were alsoincluded.

3.1.5. LanguagesNo articles were excluded based on language, although themain focus of the project was telemedicine in Europe.

3.2. Exclusion criteria

3.2.1. DesignReviews considered not systematic, including commentariesand editorials, were excluded. Systematic reviews with majorlimitations (low quality reviews) according to a revised check-list for systematic reviews from EPOC (Cochrane EffectivePractice and Organisation of Care Group) were excluded.

If the same authors had produced several publications ofthe same review, the most updated and/or the full report ofthe review was selected, and other versions excluded. Disser-tations, symposium proceedings, and irretrievable documentswere excluded.

3.2.2. ParticipantsStudies with participants considered not relevant for thereview, for instance studies on use of ICT on people outsidehealth care were excluded. Animal studies were excluded.

3.2.3. Interventions considered not relevant for the reviewOther exclusions were studies on interventions considered notrelevant for the review, such as studies on Internet and otherICT media used for information seeking; quality of informa-tion on the Internet; Internet based education of students andhealth professionals, including use of games; medical tech-nology in clinical practice in general, i.e. medical and surgicalexaminations and treatments based on computer technolo-gies, except when used as remote diagnosis and treatment(telehealth); ordinary use of electronic patient records; use oftelephone (including cell phones) only; e-health as only a verylimited part of an intervention; use of Internet for surveys andresearch; online prescriptions; mass media interventions andveterinary medicine.

3.2.4. OutcomesArticles without relevant outcomes, i.e. not on the list of out-comes specified above under inclusion criteria, were excluded.

a l i n

3

LLeComStmE

u

3

Ti

3

BitwAi

3

DtarPtairaAr

3

Digiicat

3i

Ts

i n t e r n a t i o n a l j o u r n a l o f m e d i c

.3. Information sources

iterature searches of the following databases: ACM Digitalibrary (ACM – The Association for Computing Machin-ry), British Nursing Index, Cochrane library (includingochrane database of systematic reviews (CDSR), Databasef reviews of effects (DARE), Health Technology Assess-ent Database (HTA), CSA, Ovid Medline, Embase, Health

ervices/Technology Assessment Text (HSTAT), Interna-ional Network of Agencies for Health Technology Assess-

ent (INAHTA), PsycInfo, Pubmed, Telemedicine Informationxchange (TIE), Web of Science.

The main search was performed in February 2009, and anpdated search was performed in July 2009.

.4. Search

he search strategies are available on the website: (to benserted).

.5. Study selection

ased on the criteria for inclusion and exclusion, AGE and SFndependently screened the lists of titles/abstracts identifiedhrough searches for systematic reviews. Any discrepanciesere solved by discussion with the third member of the team,B. The potentially relevant systematic reviews were retrieved

n full text.

.6. Data collection process

ata collection was carried out online using a data extrac-ion form. Each potentially relevant systematic review wasssessed in full text by one member of an expert panel ofeviewers. A revised check list from EPOC (Cochrane Effectiveractice and Organisation of Care Group) was used to assesshe quality of the systematic reviews. The quality domainsssessed according to this checklist were methods used todentify, include and critically appraise the studies in theeview, methods used to analyse the findings and an overallssessment of the quality of the review. The review team (AGE,B and SF) subsequently checked review reports for agreement

egarding the inclusion and exclusion criteria.

.7. Data items

ata on type of participants, interventions and outcomesncluded in the reviews were collected. Other data items were:eographical coverage of review, time frame of included stud-es, range of data collection methods used in studies includedn the reviews, disciplines/areas covered and methodologi-al traditions included in the review. The reviewers were alsosked to indicate emerging issues identified by the authors ofhe reviews.

.8. Quality of systematic reviews and risk of bias in

ndividual studies

he members of the expert team assessed the quality of theystematic reviews, including questions regarding the degree

f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771 739

to which the systematic reviewers had assessed risk of bias inindividual studies.

Systematic reviews with major limitations were excluded.We assessed the methodological quality of studies in thefield of telemedicine based on the review authors’ assess-ments of risk of bias in the primary studies they hadincluded.

3.9. Summary measures and synthesis of results

The authors analysed the data collected by the members ofthe expert team. Due to the expected heterogeneity of stud-ies, regarding participants, interventions, outcomes and studydesigns, a quantitative summary measure of the results wasnot planned. We did a qualitative and narrative summaryof the results of the systematic reviews. The results of theliterature review were presented and discussed in two work-shops intending to validate results. In the first workshopdifferent user groups took part and in the second workshopmethodology experts participated. The analysis was inspiredby principles of realist review [14], considered appropriate forcomplex interventions.

4. Results

We identified 1593 records through the searches and excluded1419 following screening. We retrieved 174 potentially rel-evant articles in full text. We excluded 94 of these basedon the pre-specified inclusion and exclusion criteria. Thequalitative synthesis below relate to 73 of the 80 includedarticles.

The results of the 80 systematic reviews included are sum-marised in seven tables in Appendix 1. Tables one throughsix list populations, interventions, outcomes, results and con-clusions for the reviews cited in this paper, according to theheadlines presented in the discussion below. Table 7 list theseven included reviews not cited in this paper.

5. Telemedicine is effective

Twenty reviews (Table 1) concluded that telemedicine worksand has positive effects. These include therapeutic effects,increased efficiencies in the health services, and technicalusability.

Types of interventions that were found to be therapeuti-cally effective include online psychological interventions [15];programmes for chronic heart failure that include remotemonitoring [16]; home telemonitoring of respiratory con-ditions [17]; web and computer-based smoking cessationprogrammes [18]; telehealth approaches to secondary preven-tion of coronary heart disease [19]; telepsychiatry [20]; virtualreality exposure therapy (VRET) for anxiety disorders [21];robot-aided therapy of the proximal upper limb [22]; inter-net and computer-based cognitive behavioural therapy for thetreatment of anxiety [23,24]; home telehealth for diabetes,

heart disease and chronic obstructive pulmonary disease [25];and internet based physical activity interventions [26]. Areview comparing telepsychiatry and face-to-face work [27]found no differences between the two, and suggested that

i c a l

740 i n t e r n a t i o n a l j o u r n a l o f m e d

telepsychiatry will increase in use, particularly where it ismore practical.

Interventions that are effective in reducing health serviceuse include vital signs monitoring at home with telephonefollow-up by nurses [28]; computerised asthma patient educa-tion programs [29]; and home monitoring of diabetes patients[30].

Technical effectiveness and reliability are reported inrespect of remote interpretation of patient data [31]; smarthome technologies [32]; and home monitoring of heart failurepatients [33].

One review concluded that home based ICT interventionsin general give comprehensive positive outcomes for chronicdisease management, despite only identifying a small numberof heterogeneous studies [34].

6. Telemedicine is promising

Nineteen reviews (Table 2) were less confident about the effec-tiveness of telemedicine, suggesting that it is promising, or haspotential, but that more research is required before it is pos-sible to draw firm conclusions. In some cases, in which thesame conditions and interventions are discussed, these moretentative conclusions must temper those of authors who findconclusive evidence.

One review [35] for example found internet-delivered CBTto be a ‘promising’ and ‘complementary’ development, but didnot provide the endorsements that others [23,24] did for CBTfor the more specific conditions of anxiety and depression.Similarly psychotherapy using remote communication tech-nologies was seen as promising [36], but still requiring moreevidence.

Areas in which review authors agreed that telemedicineshows therapeutic promise, but still requires further research,include virtual reality in stroke rehabilitation [37,38]; improv-ing symptoms and behaviour associated with and knowledgeabout specific mental disorders and related conditions [39];diabetes [40,83]; weight loss intervention and possibly weightloss maintenance [41]; and alcohol abuse [88].

Other authors found promise in terms of health serviceutilisation. One review [42] for example suggested that asyn-chronous telehealth developments could result in shorterwaiting times, fewer unnecessary referrals, high levels ofpatient and provider satisfaction, and equivalent (or better)diagnostic accuracy. Another [43] found that home telehealthhas a positive impact on the use of many health services aswell as glycaemic control of patients with diabetes.

Positive patient experiences were highlighted as promisingin relation to home telemonitoring for respiratory conditions[17]. There is potential for using Internet/web-based servicesfor cancer patients in rural areas [44], and telemonitoring canempower patients with chronic conditions [45].

Promising impacts on service delivery were identified[46,47] in use of electronic decision support systems andtelemedicine consultations promise to support improved

delivery of tPA in patients with stroke (a treatment whichrequires to be administered within 3 h) [48]. Computerreminders to professionals at the point of care show ‘small tomodest improvements’ in professional behaviour, but studies

i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771

are heterogeneous and interventions complex, making thesedifficult to understand [49].

7. Evidence is limited and inconsistent

Twenty-two reviews (Table 3) however concluded that the evi-dence for the effectiveness of telemedicine is still limited andinconsistent, across a wide range of fields.

In terms of therapeutic effectiveness, there is some lim-ited evidence regarding telemonitoring for heart failure [50];despite reviewers suggesting that electronic transfer of self-monitored results has been found to be feasible and acceptablein diabetes care, they find only weak evidence for improve-ments in HbA1c or other aspects of diabetes management [51];others found only weak evidence of benefit relating to infor-matics applications in asthma care [52]; and no evidence ofimprovement in clinical outcomes following teleconsultationand video-conferences in diabetes care [53].

Frequently, these reviewers call for further research,notably in the form of RCTs. Examples include calls relatingto web-based alcohol cessation interventions [54]; and vir-tual reality in stroke therapy, despite this being found [37] tobe ‘potentially exciting and safe’. More work on telemonitor-ing in heart failure is called for [55]; on e-therapy for mentalhealth problems [56]; on smart home technologies [57]; andon technological support for carers of people with demen-tia [58]. Others [28] underlined that lack of evidence does notimply lack of effectiveness, and that in many cases interven-tions are simply ‘unproven’. Caution is also urged by reviewers[59] who identified small numbers of heterogeneous studiesin relation to chronic disease management. One review [60]found it impossible to draw any significant conclusions aboutthe impact of interventions to promote ICT use by health carepersonnel.

Several reviewers found that research has been somewhatnarrowly focused and suggested further research which takesa broader perspective or a different one. They suggested thattelemedicine researchers have not yet asked all the impor-tant questions, or conducted research in appropriate ways.For example, in the cases of dermatology, wound care andophthalmology, it was argued that evaluation has exploredICT-based asynchronous services for efficacy, but outcomesor access issues have not been considered [61]. In a simi-lar vein, although most of the studies of smart homes foundtechnical feasibility, there remain certain topics that requirefurther research, notably, ‘technical, ethical, legal, clinical,economical and organisational implications and challenges’[32]. Others [44], whilst seeing significant potential for teleon-cology, especially in rural areas, suggested that local studiesmay be needed to confirm this. A further contribution to thedebates about CBT (see above), found that whilst it appears tobe effective for panic disorders, social phobia and depression,its effects on obsessive–compulsive disorder and anxiety anddepression combined remain insufficiently clear [62]. Causalpathways in HbA1c decline in diabetes care remain unclear,

and this conclusion can be linked with the variations in pro-gramme designs [63]. Whilst smoking cessation programmesappear to be effective across a range of studies, neverthelessthe mechanisms of action are not well understood [64].

a l i n

ntiSr[ts[n

8

Ak(

bfecctmaitsldgpa

eototmisw

trblon

9

Att(

i n t e r n a t i o n a l j o u r n a l o f m e d i c

Telemedicine is a dynamic field, and new studies andew systematic reviews are rapidly being published. Aselemedicine extends into new clinical areas, it is unsurpris-ng that reviewers give renewed accounts of limited evidence.ome examples of new areas from our review include littleesearch on health promotion provided through the Internet65]; a Cochrane review that found no studies of smart homeshat met their inclusion criteria [57]; a review of studies onpiritual care that found little systematic research in this area66]; and a review concluding that formative evaluation iseeded for remote monitoring in hypertension [90].

. Economic analysis

n important emerging issue from our review is the lack ofnowledge and understanding of the costs of telemedicine

Table 4).Several reviewers suggested that telemedicine seemed to

e cost-effective, but few draw firm conclusions. One reviewound that 91% of the studies showed telehomecare to be cost-ffective, in that it reduced use of hospitals, improved patientompliance, satisfaction and quality of life [67]. This was thelearest conclusion, with others being much more cautious:elemedicine was found to be cost-effective for chronic disease

anagement, but the authors cautioned that studies were fewnd heterogeneous [34]. A comparison of the costs of telemon-toring and usual care for heart failure patients found thatelemonitoring could reduce travel time and hospital admis-ions, whilst noting that benefits are likely to be realised in theong term [68]. Others found home telehealth for chronic con-itions to be cost saving, though underlining that studies wereenerally of low quality [25]. One review found remote inter-retation in medical encounters to be more expensive than itslternatives [31].

Other reviewers did not find good evidence about cost-ffectiveness; the cost-effectiveness of home telecare forlder people and people with chronic conditions is uncer-ain [28]; there is a lack of consistent results regarding costsf synchronous telehealth in primary care [69]; there is lit-le evidence for the economic viability of home respiratory

onitoring [17]; the cost-effectiveness of IT in diabetes cares undetermined [40]; one review was able to identify only onetudy of the costs of CBT, with significant weaknesses [70],ith another finding little evidence in the same area [62].

A particular limitation identified in terms of costs concernshe wider social and organisational costs of telemedicine. Oneeview found that a societal perspective on costs has not yeteen developed for home telehealth [71] and another high-

ighted the need to consider not only costs to health servicesf interventions, but also costs to service users and their socialetworks [72].

. Is telemedicine good for patients?

second emerging issue concerns patient satisfaction withelemedicine, and indications that telemedicine may alterhe relationships between patients and health professionalsTable 5).

f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771 741

One review found that health service users with ICTused in support, education and virtual consultation feelmore confident and empowered, with better knowledge andimproved health outcomes, as well as experiencing betternurse-patient relationships [73]. The reviewers call for moreresearch on the mechanisms for these changes. Generallythere is evidence of high patient satisfaction ratings for telere-habilitation, but reviewers argue that more process research,case studies and qualitative studies are needed to improveour understanding of these outcomes [74]. Interactive healthcommunication applications (IHCAs) for people with chronicdisease appear to give benefit in terms of improved sup-port, better knowledge and improved health outcomes, butthe authors asked for more larger studies to be conducted[75].

Others found no consistent results regarding user expe-riences, though suggested that access can be improved [69].Alongside development of technologies which aim to ben-efit patients and citizens as well as professionals, we needresearch on the impacts of technologies for these groups [76].An example is that information websites relating to dementiaare geared more to carers than to people with dementia them-selves, and that the websites do not usually offer personalisedinformation [77].

10. Asking new questions

We have already noted the emergence of new topic areas inthis dynamic and complex field. The focus on patient bene-fits however indicates a more basic development, namely thatreviewers are starting to explore new questions beyond thoseof clinical and cost-effectiveness. Our review produced twokey examples (Table 6). Firstly, a review that identified genderdifferences in computer-mediated communications relatingto online support groups for people with cancer cautioned thatstudies are limited and heterogeneous [78]. Nevertheless, theauthors suggested that this issue needs to be considered bythose designing interventions of this kind. This implies a con-sideration that telemedicine is an ongoing intervention whereusers influence its development and hence that effectivenessof outcome is a complex collaborative achievement. Secondly,a review focusing on stroke thrombolysis service configura-tions, their potential impact and ways of recording data toinform which configuration could be most suitable for a partic-ular situation, highlighted the need to consider a wider rangeof service delivery issues [79]. Similarly, it was argued that inpost-stroke patients, the consideration of caregivers’ mentalhealth and high levels of patient satisfaction should be anintegral element of studies [80].

Furthermore, some of the papers included in the reviewexplored issues which can inform the future developmentof telemedicine, that is, they provide formative assessments.Examples include a review of 104 definitions of telemedicine[81] which, in identifying four broad types of definitions,suggested how stakeholder interests can alter perceptions

of priorities in telemedicine interventions, such that somemay focus on delivering healthcare over a distance andothers on the potential of technology per se; and work argu-ing that clinical and technical guidelines can inform the

i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771

Summary points“What was already known on this topic”

• Evidence regarding the effectiveness of telemedicineis patchy and incomplete.

• The quality of much of the research conducted is poor.

“What this study added to our knowledge”

• The evidence base is accumulating robust knowledgeabout the effectiveness of telemedicine.

• As the field is rapidly evolving however, new knowl-edge is constantly needed.

• Continuing areas of weakness but also of great interestinclude economic analyses, understandings of patientperspectives, of effectiveness and outcomes as com-plex and ongoing collaborative achievements, and

742 i n t e r n a t i o n a l j o u r n a l o f m e d

future development of telemedicine and facilitate evaluation[20,82].

11. Reflections on the methodology of ourstudy

Our study is a review of systematic reviews. There are someinherent weaknesses in this approach. In general we have torely on the information in the included reviews. The qualityof the reviews may vary; the reviews may have done a poorjob in specifying their inclusion and exclusion criteria, thesearches may not be comprehensive, the review authors maynot have assessed or extracted data from the primary studiesadequately, nor analysed and synthesised the findings acrossthe studies properly. But even using high quality reviews, wenecessarily lose information and details that we can only findif we go back to the primary studies.

Although we did a thorough job in developing the searchstrategy and identified a vast amount of reviews on the effectsof telemedicine, we might have missed relevant systematicreviews.

Some of the included reviews are probably outdated. Stud-ies that are published after the search date in the reviews arenot included. Ideally we could have supplemented the reviewwith more recent primary studies not included in the reviews,but we did not have the resources to do this.

We did not check whether reviews included the same ref-erences. Several reviews have studied similar or overlappingtopics, and have at least partially included the same studies.It may therefore be that evidence is counted twice, or thatdifferent interpretations of effectiveness are given by reviewauthors. We have not analysed the degree to which there arediscrepancies in the analyses of similar studies, nor the rea-sons for different interpretations of the same findings, forinstance did we not analyse the heterogeneity of the resultsamong the reviews based on the quality of the reviews.

The data collection and assessment of each includedreview was accomplished by one external expert, while twois considered to be optimal in order to reduce risk of bias.We did not train the data extractors, and we did not pilotthe data extraction form. The experts were not completelyconsistent in their judgments. This limitation was partly dueto the resources and organisation of the project, in that twoworkshops were held, intending to validate results. In addi-tion, the review team made a quality check of the reviews bycomparing the reported data with information in the full textpapers. Any unclear themes were discussed in the team toreach consensus.

We have limited information regarding effect sizes and thestrength of evidence for the outcomes that we have studied.

We have however demonstrated that it is possible to makesuch a large overview in quite a short time, involving bothmethodology and content experts. We have used systematicmethods in the literature searches and the assessment of thereviews, and we have excluded reviews of low methodological

quality.

In combining rigorous and systematic methods with apragmatic approach we have produced a relevant and richoverview of the field.

formative assessments.

12. Conclusions

Despite large number of studies and systematic reviewson the effects of telemedicine, high quality evidence toinform policy decisions on how best to use telemedicinein health care is still lacking. Large studies with rigorousdesigns are needed to get better evidence on the effects oftelemedicine interventions on health, satisfaction with careand costs. As the field is rapidly evolving, different kindsof knowledge are also in demand, e.g. a stronger focus oneconomic analyses of telemedicine, on patients’ perspec-tives and on the understanding of telemedicine as complexdevelopment processes, and effectiveness and outcome asongoing collaborative achievements. Hence formative assess-ments are also pointed out as an area of weakness andinterest.

Acknowledgements

The study was funded by the EU under SMART 2008/0064and was conducted as part of the MethoTelemed project. Weacknowledge the support of our MethoTelemed colleagues, thegroup of external review experts, the workshop participants,the project officers at the Norwegian centre for integrated careand telemedicine, and Ingrid Harboe at the Norwegian Knowl-edge Centre for the Health Services, who did the literaturesearches.

Appendix 1.

In Tables 1–7, columns listing results and conclusions quotefrom the authors’ work. Where a review appears in more thanone table, this reflects the range of evidence produced. Full

access to a searchable database of abstracts of items includedin the review will be available on the MethoTelemed web-site, which also includes guidance for evaluating telemedicine.www.telemed.no/MethoTelemed.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

743

Table 1 – Systematic reviews reporting that telemedicine is effective.

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Barak et al. [15] Mental health Not stated Internet basedpsychotherapy

Behavioural,Health, Percep-tion/satisfaction,Social

Sixty-four studies included covering 94services. The overall mean weighted effectsize was 0.53, similar to the average effectsize of traditional, face-to-face therapy.Comparison between face-to-face andInternet intervention across 14 studiesshowed no differences in effectiveness.

Internet based intervention is aseffective as face-to-faceintervention.

Clark et al. [16] Cardio-vascular(CHF)

All countries Remotemonitoring,telephonesupport

Behavioural,Cost/economic,Health

Fourteen studies (RCTs) included. Fourevaluated telemonitoring, nine structuredtelephone support, and one both. Remotemonitoring programmes reduced the rates ofadmission to hospital for chronic heartfailure by 21% and all cause mortality by 20%.Three studies reported quality of lifeimprovements and four, reduced cost, onefound no gain in cost-effectiveness.

Programmes for chronic heartfailure that include remotemonitoring have a positive effecton clinical outcomes incommunity dwelling patients withchronic heart failure.

Jaana et al. [17] Respiratoryconditions

USA, Europe,Israel, Taiwan

Remotemonitoring

Behavioural,Cost/economic,Feasibility/pilot,Health, Percep-tion/satisfaction

Twenty-three studies included. Good levels ofdata validity and reliability were reported.However, little quantitative evidence existsabout the effect of remote monitoring onpatient medical condition and utilization ofhealth services. Positive effects on patientbehaviour were consistently reported. Onlytwo studies performed a detailed costanalysis.

Home telemonitoring ofrespiratory conditions results inearly identification ofdeteriorations in patient conditionand symptom control. Positivepatient attitude and receptivenessof this approach are promising.However, evidence on themagnitude of clinical andstructural effects remainspreliminary, with variations instudy approaches and an absenceof robust study designs and formalevaluations.

Myung et al.[18]

Smokingcessation

Worldwide Web andcomputer-basedprogrammes

Behavioural Twenty-two studies included (RCTs). In arandom-effects meta-analysis of all 22 trials,the intervention had a significant effect onsmoking cessation. Similar findings wereobserved in nine trials using a Web-basedintervention,(and in 13 trials using acomputer-based intervention Subgroupanalyses revealed similar findings fordifferent levels of methodological rigor,stand-alone versus supplementalinterventions, type of abstinence ratesemployed, and duration of follow-up period,but not for adolescent populations.

The meta-analysis of RCTsindicates that there is sufficientclinical evidence to support theuse of Web- and computer-basedsmoking cessation programs foradult smokers.

744in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771

Table 1 (Continued)

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Neubeck et al.[19]

Cardio-vascular(CHD)

USA (3studies),Norway (1),Canada (3),Australia (3),Germany (1)

Communicationusing ICT,patient-professional

Behavioural, Health,psychosocial state,quality of life

Eleven studies included (RCTs).Telehealth interventions were associatedwith non-significant lower all-causemortality than controls. Theseinterventions showed a significantlylower weighted mean difference atmedium long-term follow-up thancontrols for total cholesterol, systolicblood pressure, and fewer smokers.Significant favourable changes atfollow-up were also found in high-densitylipoprotein and low-density lipoprotein.

Telehealth interventions provideeffective risk factor reduction andsecondary prevention. Provision oftelehealth models could helpincrease uptake of a formalsecondary prevention by thosewho do not access cardiacrehabilitation and narrow thecurrent evidence-practice gap.

Pineau et al.[20]

Psychiatricconditions(adult andpaediatric)

Focus onCanada andUSA

Telepsychiatry Cost/economic,Ethical issues, Legal,Organizational,Technology related,clinical guidelinesand technicalstandards

‘About 60’ studies included. The authorsargue that definition of clinical guidelinesand technological standards aimed atstandardising telepsychiatric practice willpromote its large scale implementation.

The review concludes thattelepsychiatry should beimplemented in Québec andprovides detailed clinical andtechnical guidelines forimplementation. They add thattaking into account human andorganizational aspects plays a partin ensuring the success of thistype of activity; that legal andethical aspects must also beconsidered; and that a detailedeconomic analysis should becarried out prior to any largeinvestment in telepsychiatry.Finally, implementation ofpsychiatry should be subjected torigorous downstream assessmentin order to improve managementand performance.

Powers andEmmelkamp[21]

Anxiety(especiallyphobias)

Not stated Virtual realityexposuretherapy

Behavioural, Percep-tion/satisfaction,Psychophysiology,perceived controlover phobias

Thirteen studies included. VRET (Virtualreality exposure therapy) is highlyeffective in treating phobias and more sothan inactive control conditions. VRET isslightly, but significantly more effectivethan exposure in vivo, the gold standardin the field. Advantages of VRET: can beconducted in the therapist’s office, ratherthan in vivo situations, the possibility ofgenerating more gradual assignmentsand of creating idiosyncratic exposure.VRET is cost-effective.

Given the advantages and theefficacy of VRET supported by thismeta-analysis a broaderapplication in clinical practiceseems justified.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

745

Prange et al.[22]

Stroke USA Rehabilitation(robots)

Health Eleven studies included. Robot-aidedtherapy of the proximal upper limbimproves short and long-term motorcontrol of the paretic shoulder and elbow:however, there is no consistent influenceon functional abilities.

This systematic review indicatesthat robot-aided therapy of theproximal upper limb can improveshort and long-term motor controlof the paretic shoulder and elbow.Robot-aided therapy appears toimprove motor control more thanconventional therapy.

Reger andGahm [23]

Mental healthproblems(anxiety)

Not stated Internet/computer-basedtreatment

Behavioural, Health Ninteen studies included (RCTs).Meta-analysis showed that ICT wassuperior to waitlist and placeboassignment across outcome measuresThe effects of ICT were equal totherapist-delivered treatment acrossanxiety disorders. Conclusions werelimited by small sample sizes, the rareuse of placebo controls, and othermethodological problems. The number ofavailable studies limited the opportunityto conduct analyses by diagnostic group.

The results of this meta-analysisprovide preliminary support forthe use of Internet andcomputer-based CBT for thetreatment of anxiety.

Spek et al. [24] Mental health(depressionand anxiety)

Global CBT via internet Health Twelve studies included (RCTs). Authorsconcluded that eCBT was effective, butnoted that there was only a small numberof studies and significant heterogeneity.

Despite study limitations, eCBTseemed to be effective.

Tran et al. [25] Diabetes, heartfailure, COPDand otherchronicdiseases

Canadafocused, butinternationalpublicationsincluded

Home telehealth Cost/economic,Health, Percep-tion/satisfaction

Seventy-nine studies included. Of theincluded studies, 26 pertained todiabetes, 35 to CHF, nine to COPD, andeight to mixed chronic diseases. Thecomparator “no care” was not identifiedin any of the included studies, so usualcare was used as the comparatorthroughout the clinical review. Hometelehealth appeared generally clinicallyeffective and no patient adverse effectswere reported. Evidence on health serviceutilization was more limited, butpromising The economic reviewsuggested cost-effectiveness, but thequality of studies was low.

Conclusions relate to the potentialfor home telehealth in Canadawhich is seen as positive. However,more research, such as multicentreRCTs, is warranted to accuratelymeasure the clinical and economicimpact of home telehealth forchronic disease management tosupport Canadian policy makers inmaking informed decisions.

van den Berg etal. [26]

Internet basedphysicalactivityinterventions

Notmentionedother thanlanguagelimitations

Physical activity Behavioural, Health Ten studies included. The analysisfocused on the methodological quality ofthe studies, which showed variation instudy populations and interventionsmaking generalization difficult.

There is indicative evidence thatinternet based physical activityinterventions are more effectivethan a waiting list strategy.

746in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771

Table 1 (Continued)

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Hyler et al. [27] Mental health France,Australia,Canada,Japan, UKand US

Telepsychiatry Feasibility/pilot, Per-ception/satisfaction,Quality of differentinstruments usedfor consultations

Fourteen studies included. Telepsychiatrywas found to be similar to In person forstudies using objective assessments.Bandwidth was a moderator.Heterogeneous effect sizes for differentmoderators (bandwidth) High bandwidthwas slightly superior for assessmentsrequiring detailed observation of patients.

Only a handful studies haveattempted to comparetelepsychiatry with in-personpsychiatry (IP) directly, usingstandardised assessmentinstruments to permit meaningfulcomparison. According to themeta-analysis, there was nodifference in accuracy orsatisfaction between the twomodalities. Telepsychiatry isexpected to replace IP in certainresearch and clinical situations.

Barlow et al.[28]

Elderly people,chronicdiseases

Worldwide Home telecare Behavioural, Health,Organisational,Safety

Sixty-eight RCTs and 30 observationalstudies with 80 or more participantsincluded. Results show that the mosteffective telecare interventions appear tobe automated vital signs monitoring (forreducing health service use) andtelephone follow-up by nurses (forimproving clinical indicators andreducing health service use). Evidence oncost-effectiveness is less clear, and onsafety and security alert systemsinsufficient.

Having identified where there isevidence of effectiveness, andwhere it is lacking, the authorsconclude that insufficientevidence does not amount to lackof effectiveness: more research isneeded.

Bussey-Smithand Rossen[29]

Asthma USA, Hawaii,Sweden

Computer-basedpatienteducationprogrammes(CAPEPs)

Behavioural,Cost/economic,Health, Percep-tion/satisfaction,Social

Nine studies included. One study eachshowed reduced hospitalizations, acutecare visits, or rescue inhaler use. Tworeported lung function improvements.Four showed improved asthmaknowledge, and five showedimprovements in symptoms.

Although interactive CAPEPs mayimprove patient asthmaknowledge and symptoms, theireffect on objective clinicaloutcomes is less consistent

Jaana et al.[30] Diabetes NorthAmerica,Europe andAsia

Remotemonitoring

Behavioural, Health,Technology related,Structural

Seventeen studies included. Most studiesreported overall positive results inDiabetes mellitus type 2, and found thatIT based interventions improved healthcare utilisation, behaviour attitudes andskills.

Positive effects are reported, butthere is variation in patientcharacteristics (background,ability, medical condition) sampleselection and approach fortreatment of control groups.

Azarmina andWallace [31]

All All countries Remoteinterpretation inmedicalencounters

Cost/economic,Feasibility/pilot,Health,Organisational, Per-ception/satisfaction,Safety

Nine studies included. Results showedthat time between encounters wasreduced, but evidence on consultationlength was not consistent. Good clientand doctor satisfaction was shown, butthose interpreting data preferred to do soface to face. Costs of these interventionsare high, but efficiency gains are possible.

The review suggests that remoteinterpretation is an acceptable andaccurate alternative to traditionalmethods, despite the higherassociated costs.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

747

Demiris andHensel [32]

Older people,people withdisabilities

Europe, USA,Asia

Smart home Behavioural, Health,Safety, Social,Physiological andfunctional

Twenty-one projects included (drawingon 114 publications). A table is presentedwith their technologies, target audience,technologies and different outcome. Alack of evidence on clinical outcomes isidentified.

Most of the studies demonstratedthe feasibility of the technologicalsolution. Technical, ethical, legal,clinical, economical andorganisational implications andchallenges need to be studiedin-depth for the field to growfurther.

Martinez et al.[33]

Heart failure All countries Remotemonitoring(home)

Behavioural,Cost/economic,Feasibility/pilot,Health, Legal,Organizational, Per-ception/satisfaction,Safety, Social,Technology related

Forty-two studies included. (1) Remotemonitoring for cardiac heart failureappears to be technically effective forfollowing the patient remotely; (2) itappears to be easy to use, and it is widelyaccepted by patients and healthprofessionals; and (3) it appears to beeconomically viable.

Evaluating the articles showedthat home monitoring in patientswith heart failure is viable.

Gaikwad andWarren [34]

Chronicdisease

Not stated Home based ICTinterventions

Behavioural,Cost/economic,Health, Percep-tion/satisfaction

Twenty-seven studies included. Thesesystems can improve functional andcognitive patient outcomes in chronicdisease and reduce costs. However, theresearch is not yet sufficiently robust.

Telecare, telehealth etc. havepositive clinical and cost outcomes– although studies are few innumber and heterogeneous. Betterevidence-based outcome measuresare needed, especially regardingcosts and physician perspectives.

748in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771

Table 2 – Systematic reviews reporting that telemedicine is promising.

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Cuijpers et al. [35] Pain andother healthproblems

Global CBT viainternet

Health Twelve studies included (RCTs). Threestudies focused on pain, three onheadache, and six on other healthproblems. Effects found for Internetinterventions targeting pain andheadache were comparable to effectsfound for face-to-face treatments, andthe same was true for interventionsaimed at headache. Otherinterventions also showed someeffects, which differed across targetconditions.

Internet-delivered cognitivebehavioural interventions are apromising addition andcomplement to existingtreatments. The Internet willmost likely assume a majorrole in the future delivery ofcognitive behaviouralinterventions to patients withhealth problems. Moreresearch on eCBT is needed.

Bee et al. [36] Anxiety anddepression

Not stated Psychotherapymediated byremote com-municationtechnology

Behaviouraloutcomes

Thirteen studies included. Pooledeffect sizes for remote vs.conventional services were 0.44 fordepression and 1.15 for anxietyrelated disorders. Few studiescompare remote and face-to-facepsychotherapy. Data suggest thatgood effects may not be dependent onpatient and therapist beingco-located, but the evidence is limited.

Remote therapy has thepotential to overcome some ofthe barriers to conventionalpsychological therapy services,but large scale trials areneeded.

Crosbie et al. [37] Stroke Not stated Virtual reality(VR) in strokerehabilitation

Behavioural,Health, Safety

Eleven studies included. Five coverupper limb rehabilitation, three, gaitand balance, two cognitiveinterventions, and one both upperand lower limb rehabilitation. Threewere AACPDM Level I/Weak, two LevelIII/Weak, three Level IV/Weak andthree Level V quality of evidence.Three RCTs obtained statisticalsignificance, and eight studies foundVR based therapy to be beneficial.None reported any significant adverseeffects.

VR is a potentially exciting andsafe tool for strokerehabilitation but its evidencebase is too limited by designand power issues to permit adefinitive assessment of itsvalue. Thus, while the findingsof this review are generallypositive, the level of evidence isstill weak to moderate, interms of research quality.Further study in the form ofrigorous controlled studies iswarranted.

Henderson et al.[38]

Stroke Global Virtual reality(VR) in strokerehabilitation

Health Six studies included. The results ofthe reviewed studies suggest thatimmersive VR may have an advantageover no therapy in the rehabilitation ofthe upper limb in patients with stroke,but the results are still questionable.

Current evidence on theeffectiveness of VR in therehabilitation of upper limb inpatients with stroke is limitedbut sufficiently encouraging tojustify further research in thisarea.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

749

Griffiths andChristensen [39]

Mental health Global Internetinterventions

Behavioural,Health, Percep-tion/satisfaction

Sixteen papers included (reporting15RCTs). The review demonstrates thatInternet interventions show promiseas a means of improving symptomsand behaviour associated with andknowledge about specific mentaldisorders and related conditions.

Most interventions werereported to be effective inreducing risk factors orimproving symptoms,although many of the studieshad methodologicallimitations. Three of theinterventions that reportedpositive outcomes are availablewithout charges to the public.

Jackson et al. [40] Diabetes Not stated Computerassistedinteractive IT

Behavioural,Cost/economic,Health

Twenty-seven papers included(reporting 26 studies) Significantimpacts on behavioural, clinical andstructural levels.

There is growing evidence thatemerging IT may improvediabetes care. Future researchshould characterize benefits inthe long term, establishmethods to evaluate clinicaloutcomes, and determine thecost-effectiveness.

Azar and Gabbay[83]

Diabetes Not stated Web-basedmanagementof glucoseuploads

Clinical, Health,Behavioural

Eight studies on type 1 diabetes sevenpapers on type 2 (or mixedpopulations). The type 1 studiestended to show equivalent HbA1cimprovements in intervention andcontrol groups. For type B patients,there were statistically significantdifferences for HbA1betweenintervention and control groups.

Patients benefitted fromweb-based diabetesmanagement, through savingsin time and cost. Majorobstacles to widerimplementation are patientcomputer skills, adherence tothe technology, architecturaland technical design and theneed to reimburse providersfor their care.

Weinstein [41] Obesity USA Internetbasedprogrammes

Health Eight studies included. All studiesexamining weight loss via Internetprograms reported positive results,except one investigating a commercialprogram. Results from the threeweight loss maintenance programsconducted on the Internet wereequivocal. Because the subjects of allthese studies were predominatelywhite, educated women,generalisability of findings is limited.

Preliminary studies suggestthat the internet may be anadequate vehicle for weightloss intervention and possiblyfor programs directed at weightloss maintenance.

750in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771

Table 2 (Continued)

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Elliott et al. [88] Alcohol abuse Mostly USstudies

Computer-basedtreatment

Behavioural,Health, Percep-tion/satisfaction,Social

Seventeen studies included (RCTs).The e-interventions emerge as moreeffective than no treatment, andapproximately equivalent toalternative intervention approaches.E-interventions appear to be superiorto assessment-only (AO) controlconditions. In addition, possiblemoderators (e.g. baseline drinkingpatterns) and mediators (e.g. correcteddrinking norms) have emerged.

These studies compared theeffectiveness of e-interventionswith other commonly usedtechniques, reading materials,and assessment-only controlconditions. Overall, findingsprovide some support for suchprograms, especially incomparison withassessment-only controlconditions.

Deshpande et al.(asynchronoustelehealth) [42]

All, butespeciallydermatologi-calconditions

World Store andforward ofclinicallyimportantdigitalsamples forassessmentat aconvenienttime

Cost/economic,Health,Organisational,Safety, Social

Fifty-two studies included (16 of goodquality). An environmental scanfound 39 organisations using both realtime and asynchronous services.Overall study quality is poor, evidencesuggests shorter waiting times, fewerunnecessary referrals, high levels ofpatient and provider satisfaction, andequivalent (or better) diagnosticaccuracy when compared withface-to-face consultations. It isunknown whether the benefits thathave been shown in small localstudies could be realized afterwide-scale implementation.

Pragmatic objectives forimplementation andevaluation are needed,involving policy makers,specifically in Canada.Standardisation procedures arealready in progress and arelikely to improve collaborationbetween providers.

Polisena et al.(diabetes) [43]

Diabetes Worldwide Hometelehealth

Health Twenty-six studies included. Studyresults indicated that home telehealthhelps to reduce the number ofpatients hospitalised, hospitalizationsand bed days of care. Home telehealthwas similar or favourable to UC acrossstudies for quality-of-life and patientsatisfaction outcomes.

In general, home telehealthhad a positive impact on theuse of numerous healthservices and glycaemic control.More studies of highermethodological quality arerequired to give more preciseinsights into the potentialclinical effectiveness of hometelehealth interventions.

Holland et al. [89] Heart failure Europe, USA,Australia,New Zealand,Argentina

Remotemonitoring,telephonesupport

Health Thirty studies included (RCTs).Sensitivity analysis of combined datafrom 30 interventions showedreduction in hospital admission andreduction in all-cause mortality.Comparison of interventions indifferent settings indicated thateffective interventions were deliveredat least partly at home.

Results suggest thatmultidisciplinary interventionsfor patients with heart failurereduce hospital admissionsand mortality. Interventionsinclude telemedicine, but thereis limited evidence in this area.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

751

Jaana et al. [17] Respiratoryconditions

USA, Europe,Israel, TaiwanEnglish-languagepublicationsin peer-reviewedjournals

Remotemonitoring

Behavioural,Cost/economic,Feasibility/pilot,Health, Percep-tion/satisfaction

Twenty-three studies included. Goodlevels of data validity and reliability werereported. However, little quantitativeevidence exists about the effect of remotemonitoring on patient medical conditionand utilization of health services. Positiveeffects on patient behaviour wereconsistently reported. Only two studiesperformed a detailed cost analysis.

Home telemonitoring ofrespiratory conditions resultsin early identification ofdeteriorations in patientcondition and symptomcontrol. Positive patientattitude and receptiveness ofthis approach are promising.However, evidence on themagnitude of clinical andstructural effects remainspreliminary, with variations instudy approaches and anabsence of robust studydesigns and formalevaluations.

Hailey et al. [44] Cancer United States,Canada,Denmark,Finland,Spain, UK

Comparing awide range ofinterventions

Cost/economic,Health, Percep-tion/satisfaction,Social

Fifty-four studies included. Positivefindings from higher-quality studiessuggested that telephone-basedtechnology was effective for promotingmammography and colposcopy in specificpopulations, for increasing fruit andvegetable consumption, and as analternative to in-person support groupsfor women with breast cancer. Eighteconomic assessments provided someindications of cost advantages indiagnosis and treatment, and in palliativecare. 20 papers reported patient andfamily satisfaction. However, thesignificance and generalisability of thesefindings appears limited.

From the perspective of theAlberta Cancer Board, theliterature suggests some usefulpossibilities for developing newservices using internet orweb-based, telephone-based,and video-based technologiesfor cancer patients in ruralareas. However, it seems likelythat these applications wouldneed validation with suitablelocal studies.

Pare et al. [45] Chronicdisease

US, Canada,Europe, Asia

Remotemonitoring

Behavioural,Health,Organisational,Policy,Technologyrelated

Sixty-five studies included. Themagnitude and significance of the effectof (automated) telemonitoring of patientswith four chronic conditions(hypertension, diabetes, COPD andcardiac failure) remains inconclusive.Patients’ compliance is high, The clinicaleffectiveness (e.g. less frequenthospitalisation) is more consistent incardiac and pulmonary studies than indiabetes and hypertension. Economicviability was poorly evidenced.

Telemonitoring seems to be apromising approach to chronicpatient management.Technology is reliable. Itempowers patients, andpotentially improves clinicalconditions.

752in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771Table 2 (Continued)

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Nannings andAbu-Hanna [46]

All (especiallychronic)

Notmentioned

Decisionsupportmodels forprofessionaluse

Technologyrelated,Conceptualmodels

Sixty-five studies included. The objectiveswere to define the term DSTS, to proposea general DSTS model, and to proposemodel-based templates to aid DSTSdevelopment for three medical tasks. Thedefinition, general model andmodel-based templates are based on asystematic literature search. Theapplicability of the templates to newDSTSs found in a separate limitedliterature search was tested. A definitionof DSTSs is provided, a conceptual modelfor understanding DSTSs is proposed anda set of reusable templates, and examplesfor using them are provided.

The conceptual model and thereusable modelling templatesare demonstrated to be usefulin understanding andmodelling DSTSs during theearly stages of theirdevelopment.

Sintchenko et al.[47]

Diabetes,heart failure,and chronicobstructivepulmonarydisease(COPD). Otherchronicdiseases thatcould bemanaged byusing hometelehealth

Not stated Hometelehealth

Clinicaloutcomes; healthservice use, QoL,Patientsatisfaction

Twenty-four studies included. Theauthors feel they have shown that EDSSimproves prescribing practices andtreatment outcomes of patients withacute illnesses, but are less effective inprimary care.

The authors state that furtherresearch is needed to quantifythe range of benefits of EDSS,and to explore newmeasurement metrics andenhance the appropriateclinicaluse of electronicdecision support.

Wu and Langhorne[48]

Stroke Found studiesonly fromGermany, US,France

Acutemedicine

Cost/economic,Feasibility/pilot,Health, Percep-tion/satisfaction,Safety

Seventeen studies included. Studiesreported that telemedicine systems werefeasible and acceptable. Inter-raterreliability was excellent for global clinicalassessments and decisions onradiological exclusion criteria althoughagreement for individual assessmentitems was more variable. Telemedicinesystems were associated with increaseduse of tPA.

Although there is limitedreliable evidence, observationalstudies have indicated thattelemedicine systems can befeasible, acceptable, andreliable in acute-strokemanagement. In addition,telemedicine consultationswere associated with improveddelivery of tPA.

Shojania et al. [49] Medication,vaccinationand tests(variousconditionsunspecified)

Worldwide Decisionsupport forclinicians

Health Twenty-eight studies included. Thestudies tested reminders to prescribespecific medications, to warn about druginteractions, to provide vaccinations, orto order tests. The review found small tomoderate benefits. The remindersimproved physician practices by a medianof 4%. In eight of the studies, patients’health improved by a median of 3%.

Point of care computerreminders generally achievesmall to modest improvementsin provider behaviour. Aminority of interventionsshowed larger effects, but nospecific reminder or contextualfeatures were significantlyassociated with effectmagnitude.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

753

Table 3 – Systematic reviews reporting that evidence on telemedicine is limited and inconsistent.

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Chaudhry et al. [50] Heart failure Telephone-basedmonitoring: Argentina(1 study), USA 4 studies.Automated monitoringUSA 1 study. Automatedphysiologic monitoringUSA 1 study.Comparisons of two ormore methods oftelemonitoring:Germany/theNetherlands and UK:one study, USA 1 study

Remotemonitoring

Cost/economic,Health

Nine studies included. Sixsuggested reduction in all-causeand heart failure hospitalisationswith telemonitoring. Of the threenegative studies, two enrolledlow-risk patients and patientswith access to high quality care,and one enrolled a very high-riskHispanic population. Studiescomparing forms oftelemonitoring demonstratedsimilar effectiveness. Interventioncosts were higher with morecomplex programs.

The evidence base fortelemonitoring in heart failure iscurrently quite limited. Based onthe available data, telemonitoringmay be an effective strategy fordisease management in high-riskheart failure patients.

Farmer et al. [51] Diabetes Not stated Self-monitoring,Data transfer

Cost/economic,Feasibility/pilot,Health,Organizational,Percep-tion/satisfaction

Twenty-six studies included.Electronic transfer of glucoseresults appears feasible in aclinical setting. Only two of theRCTs included more than 100patients, and only three extendedto one year. Only one study wasdesigned to show thattelemedicine interventions mightreplace clinic interventionswithout deterioration in HbAResults pooled from the nine RCTswith reported data did not provideevidence that the interventionswere effective in reducing HbA to0.04%) (p. 1372).

Telemedicine solutions fordiabetes care are feasible andacceptable, but evidence for theireffectiveness in improving HbA orreducing costs while maintainingHbA levels, or improving otheraspects of diabetes management isnot strong. Further researchshould seek to understand howtelemedicine might enhanceeducational and self-managementinterventions and RCTs arerequired to examinecost-effectiveness.

Sanders andAronsky [52]

Asthma USA Diagnostics,preventionandmonitoring,decisionsupport tools,patient-centrededucationtools

Behavioural,Health, Social

Sixty-four studies included, butonly 21 prospective trials. Themean quality score was 6.6 (range:3–10). None of the studies reportedon allocation concealment. Of the13 studies that reported a clinicaloutcome, seven reported a positiveeffect of the computerisedintervention and six reported nosignificant change. Of the eightstudies reporting a non-clinicaloutcome, seven reported astatistically significant positiveeffect of the computerisedintervention.

Most studies took place in theoutpatient clinic environment,with minimal study of theemergency department orinpatient settings. Few studiesdemonstrated evidence ofcomputerised applicationsimproving clinical outcomes.

754in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771Table 3 (Continued)

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Verhoeven et al.[53]

Diabetes Worldwide Teleconsultationand video-conferencing

Cost/economic,Health, Percep-tion/satisfaction,Technologyrelated

Thirty-nine studies included. They foundno significant statistical heterogeneityamong the pooled randomised controlledtrials that they identified within thisgroup. Most of the improvements foundconcerned (a) satisfaction withtechnology, (b) improved metaboliccontrol or (c) cost savings. No significantbenefits were found in relation to (a)quality of life, (b) transparency, or (c)better access to care.

The study did not support any conclusionthat these interventions improved clinicalvalues (e.g. blood pressure). The authorsargued that diversity in design of thestudies meant that strong conclusionswere premature.

Bewick et al. [54] Alcoholabuse

Worldwide Web-basedinterventions

Behavioural,Cost/economic

Ten studies included. Five gave processevaluations and five, some pre- andpost-intervention results. Only one studywas an RCT, and the studies scored lowon quality criteria.

There is inconsistent evidence on theeffectiveness of electronic screening andbrief intervention (eSBI) for alcohol use.Process research suggests that web-basedinterventions are generally well received.However further controlled trials areneeded to fully investigate their efficacy,to determine which elements are keys tooutcome and to understand if differentelements are required in order to engagelow and high-risk drinkers.

Crosbie et al. [37] Stroke Not stated Virtual realityin strokerehabilitation

Behavioural,Health, Safety

Eleven studies included. Five cover upperlimb rehabilitation, three, gait andbalance, two cognitive interventions, andone both upper and lower limbrehabilitation. Three were AACPDM LevelI/Weak, two Level III/Weak, three LevelIV/Weak and three Level V quality ofevidence. Three RCTs obtained statisticalsignificance, and eight studies found VRbased therapy to be beneficial. Nonereported any significant adverse effects.

VR is a potentially exciting and safe toolfor stroke rehabilitation but its evidencebase is too limited by design and powerissues to permit a definitive assessmentof its value. Thus, while the findings ofthis review are generally positive, thelevel of evidence is still weak to moderate,in terms of research quality. Further studyin the form of rigorous controlled studiesis warranted.

Maric et al. [55] Heartfailure

Not stated Remotemonitoring

Cost/economic,Feasibility/pilot,Health,Organisational

Fifty-six papers included. Many of theidentified papers on telemonitoring ofheartfFailure (HF) patients demonstrate atleast some benefit. There is a generaltrend towards improvement associatedwith the use of most modalities. Howeverinconsistent evidence between trials forthe same modality and differencebetween modalities make a definitiveconclusion difficult. Evidence of benefitfrom video consultation alone is lacking.The majority of papers reported trialsthat are non-randomised and manypapers had small sample sizes.

This study reviewed studies conducted inHF telemonitoring, to describe the natureof the modality, the methods and theresults. Telemonitoring appeared to be anacceptable method for monitoring HFpatients. Controlled, randomized studiesdirectly comparing different modalitiesand evaluating their success andfeasibility when used as routine clinicalcare are now required.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

755

Postel et al. [56]. Mental healthproblems

North American andEuropean

E-therapy Health Fourteen studies included.E-therapy seems to be promisingbut study results are still basedonly on small groups.

More research is needed, andshould be better reported.

Martin et al. [57] Physicaldisability,cognitiveimpairment,learningdisability

Not explicitlymentioned

Smart home Cost/economic,health,organizational,social andtechnologyissues

No study identified met theinclusion criteria.

The review does not providesufficient evidence to support orrefute the integration of smarthome technologies into health andsocial care. RCTs should beadopted. International descriptiveterminology is needed. The rate ofabandonment by individuals of thetechnologies needs investigation.

Powell et al. [58] Dementia(carers’ stressanddepression)

North America Five specifictechnologicalinterventions

Behavioural,Cost/economic,knowledge

Fifteen studies included. Theydescribe five interventions:ComputerLink, AlzOnline, Caringfor Others and two studies fromthe REACH project (TLC and CTIS).The interventions reviewed weremultifaceted with elements ofnetworked peer support.Outcomes were inconsistent butsuggested that the interventionshad moderate effects on improvingcarer stress and depression.Treatment effects were found tovary with caregiver characteristicssuch as ethnic groups, formalsupport and baseline burden.

Further evaluation is needed inrobust trials with good follow-up.

Barlow et al. [28] Elderlypeople,chronicdiseases

Worldwide Hometelecare

Behavioural,Health,Organisational,Safety

Sixty-eight RCTs and 30observational studies with 80 ormore participants included.Results show that the mosteffective telecare interventionsappear to be automated vital signsmonitoring (for reducing healthservice use) and telephonefollow-up by nurses (for improvingclinical indicators and reducinghealth service use). Evidence oncost-effectiveness is less clear, andon safety and security alertsystems insufficient.

Having identified where there isevidence of effectiveness, andwhere it is lacking, the authorsconclude that insufficientevidence does not amount to lackof effectiveness: more research isneeded.

756in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771

Table 3 (Continued)

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Garcia-Lizana andSarria-Santamera[59]

Chronicdisease

Nations around theworld

Management,control andprevention

Health, Percep-tion/satisfaction,Safety

Twenty-four studies included.Improvements in clinicaloutcomes were not shown, thoughno adverse effects were identified.In the detection and follow-up ofcardio-vascular diseases however,ICTs provided better clinicaloutcomes, mortality reduction andlower health services utilization.Systems used for improvingeducation and social support werealso shown to be effective.

Although some positive resultswere identified, at present theevidence about the clinicalbenefits of ICTs for managingchronic disease is limited.

Gagnon et al. [60] None (imple-mentationissues)

Not stated Educational Behavioural,Cost/economic

Twenty-seven references listed.The authors find very little data oninterventions to promote ICT use,and argue that more studies areneeded.

Too few studies, which are tooheterogeneous, make it impossibleto draw any significant conclusionsabout the impact of interventionsto promote ICT use by HCPs.

Hersh et al. [4] Variousconditions(Medicarepopulation)

USA, UK, Australia,Europe

Comparingstore andforward,home based,office-hospital-basedservices

Health,Organisational,Concordancestudies,diagnosis andmanagement,access

Ninety-seven studies included.Store and forward services havebeen studied in many specialties,especially dermatology, woundcare, and ophthalmology. Theevidence for their efficacy ismixed, and in most areas, thereare not corresponding studies onoutcomes or improved access tocare. Several limited studiesshowed benefits of home basedtelemedicine interventions inchronic diseases. Theseinterventions appear to enhancecommunication with providersand provide closer monitoring ofgeneral health, but the studieswere conducted in settingsrequiring additional resources anddedicated staff. Studies ofoffice/hospital-based telemedicinesuggest that telemedicine is mosteffective for verbal interactions,e.g. videoconferencing fordiagnosis and treatment neurologyand psychiatry.

Outside of a small number ofclinical specialties, the evidencebase for the efficacy oftelemedicine is weak. Furtherwell-designed and targetedresearch that provides high qualitydata will provide a strongcontribution to understandinghow best to deploy technologicalresources in health care.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

757

Demiris andHensel [32]

Older people,people withdisabilities

Europe, USA, Asia Smart home Behavioural,Health, Safety,Social,Physiological andfunctional

Twenty-one projects included(drawing on 114 publications). Atable is presented with theirtechnologies, target audience,technologies and differentoutcome. A lack of evidence onclinical outcomes is identified.

Most of the studies demonstratethe feasibility of the technologicalsolution. Technical, ethical, legal,clinical, economical andorganisational implications andchallenges need to be studiedin-depth for the field to growfurther.

Hailey et al. [44] Cancer United States,Canada, Denmark,Finland, Spain, UK

Comparing awide range ofinterventions

Cost/economic,Health, Percep-tion/satisfaction,Social

Fifty-four studies included.Positive findings fromhigher-quality studies suggestedthat telephone-based technologywas effective for promotingmammography and colposcopy inspecific populations, for increasingfruit and vegetable consumption,and as an alternative to in-personsupport groups for women withbreast cancer. Eight economicassessments provided someindications of cost advantages indiagnosis and treatment, and inpalliative care. 20 papers reportedpatient and family satisfaction.However, the significance andgeneralisability of these findingsappears limited.

From the perspective of theAlberta Cancer Board, theliterature suggests some usefulpossibilities for developing newservices using internet orweb-based, telephone-based, andvideo-based technologies forcancer patients in rural areas.However, it seems likely that theseapplications would need validationwith suitable local studies.

Linton [62] Anxiety anddepression

Not specified Computer-basedcognitivebehaviouraltherapy (CBT)

Behavioural,Cost/economic,Ethical issues,Health, Percep-tion/satisfaction,self-assessment

Twelve studies included. Thepaper investigates what effectsand costs are associated withcomputer-based CBT in treatingadult patients with anxietydisorders or depression.

There is limited scientific evidenceindicating that computer-basedCBT has favourable, short-termeffects on symptoms in thetreatment of panic disorder, socialphobia, and depression. Thescientific evidence is insufficientto assess the effects of treatmenton obsessive–compulsive disorderand mixed anxiety/depression.The scientific evidence isinsufficient to assess thecost-effectiveness of the method.

758in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771

Table 3 (Continued)

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Mathur et al. [63] Diabetes Not stated Evaluation Feasibility/pilot,Health

Fourteen studies included. Markedvariability in operational design ofprograms and poor rationalizationof choice of outcome metrics withprogram components found.

Causal pathways in decline ofHbA1c remain unclear, eventhough 11 of 14 studies showeddecline. The authors recommend astandardized methodology foranalyzing communicationstechnology use in diabetes care.

Walters et al. [64] Smokingcessation

Worldwide Web andcomputer-basedprogrammes

Behavioural Ninteen studies included. 47% ofincluded studies reportedstatistically significant orimproved outcomes at the longestfollow-up, relative to a comparisongroup.

Few patterns emerged in terms ofsubject, design or interventioncharacteristics that led to positiveoutcomes. The ‘first generation’format, where participants weremailed computer-generatedfeedback reports, was the modalintervention format and the onemost consistently associated withimproved outcomes.

Lintonen et al. [65] Promotion ofhealth

Global Emerginginformationand commu-nicationstechnology,especially theinternet

Classified uses ofIT generally

Fifty-six papers included. Usingthe papers studied, the authorsclassified IT in health promotion:IT as an intervention medium; ITas a research focus; IT as aresearch instrument and IT forprofessional development.

IT in health promotion is only justemerging. While a lot of healthpromotion information is providedover internet, little publishedresearch exists on its use andusefulness.

Van Nooten et al.[66]

All Not stated Spiritual care Information,assessments,theories of role ofspiritual care andinterventions

Eighteen studies included (17articles and one book). There arevery few systematic studies on useof Internet for spiritual care ase-health. Four different categoriesof spiritual and religious care areidentified: information search,assessments, theories andinterventions.

No systematic studies were foundon how ICTs are being used tointegrate religious and spiritualcare into healthcare.

Jaana et al. [90] Cardio-vascular(hyperten-sion)

USA, Europe, Japan,Israel, Malaysia

Remotemonitoring

Behavioural,Cost/economic,Feasibility/pilot,Blood pressurecontrol

Fourteen studies included. Thestudies present evidence on thepositive impacts of telemonitoringon patients and their condition:significant BP control, bettermedication adherence, changes inpatient lifestyle and attitudes. Thestudies were considered limitedbecause of study designs. Little isknown on the effects oftelemonitoring on servicesutilization. Only one studydemonstrated a detailedcost-effectiveness analysis.

Preliminary evidence exists onbenefits. However, limitedinformation exists on effects ofutilization and economic viability.Future studies should include bothRCT’s and research to understandprocessual conditions andmechanisms through which BPcontrol is achieved. (Formativeevaluations).

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

759

Table 4 – Emerging issue 1: economic analysis.

Reference Conditionsincluded

Geographicarea

Service/intervention Outcome Authors’ summary of results Authors’ conclusions

Rojas and Gagnon[67]

All Worldwide Telehomecare(THC)

Cost/economic,Health, Percep-tion/satisfaction

Twenty-three studies included.THC was found to be acost-effective alternative totraditional approaches in 91% ofthe studies. Main benefits includeddecreased hospital utilisation;improved patient compliance withtreatment plans; improved patientsatisfaction with health servicesand improved quality of life.

The authors argue that one of themajor disadvantage is of THCstudies has been the failure toadopt a set of common indicatorsto calibrate theircost-effectiveness.

Gaikwad andWarren [34]

Chronicdisease

Not stated Home basedICTinterventions

Behavioural,Cost/economic,Health, Percep-tion/satisfaction

Twenty-seven studies included.These systems can improvefunctional and cognitive patientoutcomes in chronic disease andreduce costs. However, theresearch is not yet sufficientlyrobust.

Telecare, telehealth etc. havepositive clinical and cost outcomes– although studies are few innumber and heterogeneous. Betterevidence-based outcome measuresare needed, especially regardingcosts and physician perspectives.

Seto [68] Heart failure Worldwide Remotemonitoring

Cost/economic Eleven studies included. Theauthor considered a variety ofdirect and indirect cost categoriesincluding costs to the healthsystem and costs to the patient.These are described.

Telemonitoring has a positive roleto play in reducing costs byreducing re-hospitalisation andtravel costs.

Tran et al. [25] Diabetes,heart failure,COPD andother chronicdiseases

Canada focused, butinternationalpublicationsincluded

Hometelehealth

Cost/economic,Health, Percep-tion/satisfaction

Seventy-nine studies included. Ofthe included studies, 26 pertainedto diabetes, 35 to chronic heartfailure, nine to COPD, and eight tomixed chronic diseases. Thecomparator “no care” was notidentified in any of the includedstudies, so usual care was used asthe comparator throughout theclinical review. Home telehealthappeared generally clinicallyeffective and no patient adverseeffects were reported. Evidence onhealth service utilization wasmore limited, but promising Theeconomic review suggestedcost-effectiveness, but the qualityof studies was low.

Conclusions relate to the potentialfor home telehealth in Canadawhich is seen as positive. However,more research, such as multicentreRCTs, is warranted to accuratelymeasure the clinical and economicimpact of home telehealth forchronic disease management tosupport Canadian policy makers inmaking informed decisions.

760in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771

Table 4 (Continued)

Reference Conditionsincluded

Geographicarea

Service/intervention Outcome Authors’ summary of results Authors’ conclusions

Azarmina andWallace [31]

All All countries Remote inter-pretation inmedicalencounters

Cost/economicFeasibility/pilot,Health,Organizational,Percep-tion/satisfaction,Safety

Nine studies included. Resultsshowed that time betweenencounters was reduced, butevidence on consultation lengthwas not consistent. Good clientand doctor satisfaction wasshown, but those interpreting datapreferred to do so face to face.Costs of these interventions arehigh, but efficiency gains arepossible.

The review suggests that remoteinterpretation is an acceptable andaccurate alternative to traditionalmethods, despite the higherassociated costs.

Barlow et al. [28] Elderlypeople,chronicdiseases

Worldwide Hometelecare

Behavioural,Health,Organizational,Safety

Sixty-eight RCTs and 30observational studies with 80 ormore participants included.Results show that the mosteffective telecare interventionsappear to be automated vital signsmonitoring (for reducing healthservice use) and telephonefollow-up by nurses (for improvingclinical indicators and reducinghealth service use). Evidence oncost-effectiveness is less clear, andon safety and security alertsystems insufficient.

Having identified where there isevidence of effectiveness, andwhere it is lacking, the authorsconclude that insufficientevidence does not amount to lackof effectiveness: more research isneeded.

Deshpande et al.(synchronoustelehealth) [69]

Chronicdisease, CHF,psychologicalandneurologicalproblems

All countries Hometelehealth, inreal time

Behavioural,Cost/economic,Feasibility/pilot,Health, Percep-tion/satisfaction,Safety, Social,Technologyrelated

Thirty-one systematic reviewsincluded (11 of high quality). Mostreviews and most of the studiesthey review are low quality. Theauthors suggest that thisillustrates the resource constraintsfor researchers and policy makers.Nevertheless, weak evidence thatreal-time telehealth can improveservice access, user satisfactionand resource utilization is found.For patients with psychiatric andneurological conditions in remoteareas, evidence of benefit isstronger.

Evidence is generally weak andstudies of poor quality, buttelehealth is neverthelesspromising, especially in aCanadian context over a largegeographical area.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

761

Barak et al. [15] Mental health Not stated Internetbased psy-chotherapy

Behavioural,Health, Percep-tion/satisfaction,Social

Sixty-four studies includedservices. The overall meanweighted effect size was 0.53(medium effect), similar to theaverage effect size of traditional,face-to-face therapy. Comparisonbetween face-to-face and Internetintervention across 14 studiesshowed no differences ineffectiveness.

Internet based intervention aseffective as face-to-faceintervention.

Jackson et al. [40] Diabetes Not stated Computerassistedinteractive IT

Behavioural,Cost/economic,Health

Twenty-seven papers included(reporting 26 studies) Significantimpacts on behavioural, clinicaland structural levels.

There is growing evidence thatemerging IT may improve diabetescare. Future research shouldcharacterize benefits in the longterm, establish methods toevaluate clinical outcomes, anddetermine the cost-effectiveness.

Kaltenthaler et al.[70]

Mental health(depressionand anxiety)

Worldwide Computerisedcognitivebehaviouraltherapy(CCBT)

Cost/economic,Health, Percep-tion/satisfaction,Safety

Twenty-four CCBT studies and oneeconomic analysis included. Thereis some evidence for positiveeffects of CCBT, and ofcost-effectiveness compared withusual care. The authors identify‘significant uncertainty’ regardingwhether the treatments should beadopted.

The authors identify a long list ofneeds for further research,including access to CBT,comparing CCBT with otherinterventions, CCBT via theinternet, more RCT type studies,patient preferences, studies in GPsettings and studies ofco-morbidities.

762in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771

Table 4 (Continued)

Reference Conditionsincluded

Geographicarea

Service/intervention Outcome Authors’ summary of results Authors’ conclusions

Linton [62] Anxiety anddepression

Not specified Computer-basedCBT

Behavioural,Cost/economic,Ethical issues,Health, Percep-tion/satisfaction,Self-assessment

Twelve studies included. Thepaper investigates what effectsand costs are associated withcomputer-based CBT in treatingadult patients with anxietydisorders or depression.

There is limited scientific evidence(indicating that computer-basedCBT has favourable, short-termeffects on symptoms in thetreatment of panic disorder, socialphobia, and depression. Thescientific evidence is insufficientto assess the effects of treatmenton obsessive–compulsive disorderand mixed anxiety/depression.The scientific evidence isinsufficient to assess thecost-effectiveness of the method.

Polisena et al.(chronic disease)[71]

Chronicdisease

Worldwide Hometelehealth

Cost/economic Twenty-two studies included.Home telehealth was found to becost saving from the healthcaresystem and insurance providerperspectives in all but two studies,but the quality of the studies wasgenerally low. An evaluativeframework was developed whichprovides a basis to improve thequality of future studies tofacilitate improved healthcaredecision-making and anapplication of the framework isillustrated using data from andexisting programme evaluation ofa home telehealth program.

Current evidence suggests thathome telehealth has the potentialto reduce costs, but its impactfrom a societal perspectiveremains uncertain untilhigher-quality studies becomeavailable.

Griffiths et al. [72] Manyconditions –all involvingtheintervention

Sweden, UnitedStates, Australia,Canada, UK, Spain,Denmark, Italy,Netherlands, Japan

Internetdelivery ofhealthcareinterventions

Reasons for use,reflections

Thirty-seven studies included.These covered a range of healthconditions. Reasons for Internetdelivery included low cost andresource implications due to thenature of the technology; reducingcost and increasing conveniencefor users; reduction of healthservice costs; overcoming isolationof users; the need for timelyinformation; stigma reduction;and increased user and suppliercontrol of the intervention.

Internet delivery may haveunintended consequences. It mayovercome isolation by time,mobility, and geography, but itmay not be a substitute forface-to-face contact. Costs toservice users and their socialnetworks as well as providers needto be considered. Reasons forchoosing internet delivery must bemade clear. Internet deliveryneeds to be compared with othermodes of delivery.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

763

Table 5 – Emerging issue 2: is telemedicine good for patients?

Reference Conditionsincluded

Geographicarea

Service/Intervention

Outcome Authors’ summary of results Authors’ conclusions

Akesson et al.[73]

All Worldwide Support,education/information/virtualconsultation

Behavioural,Feasibility/pilot,Health, Percep-tion/satisfaction,Quality of life

Twelve studies included. Threethemes identified: support and help,education and information, andtelecommunication instead of on-sitevisiting. Findings show consumersfeeling more confident andempowered, with increasedknowledge and improved healthstatus due to the ICT resources. Lackof face-to-face meetings or privacy didnot appear to be a problem.

ICT can improve the nurse-patient relationshipand augment well-being for consumers. Moreresearch is needed to measure consumers’experiences and factors that influence it.

Kairy et al.[74]

Physicaldisabilities

Not stated Telerehabilitation Behavioural,Cost/economic,Health, Percep-tion/satisfaction,Social

Twenty-eight studies included.Clinical outcomes were generallyimproved; clinical process outcomeswere high; consultation time waslonger; patient satisfaction ratingswere high. Healthcare utilizationevidence was unclear. There is someevidence of potential cost savings.

While evidence is mounting concerning theefficacy and effectiveness of telerehabilitation,high quality evidence regarding impact onresource allocation and costs is still needed tosupport clinical and policy decision-making.

Murray et al.[75]

Chronicdisease

Not stated Interactivehealth com-municationApplications(IHCA)

Behavioural,Health

Twenty-four studies included (RCTs).Computer-based programmes(’Interactive Health CommunicationApplications’) for people with chronicdisease had a significant positiveeffect on knowledge, social support,and clinical outcomes. Results suggestit is more likely than not that IHCAshave a positive effect on self-efficacy.IHCAs had a significant positive effecton continuous behavioural outcomes.Binary behavioural outcomes alsoshowed a positive effect for IHCAs,although this result was notstatistically significant. It was notpossible to determine the effects ofIHCAs on emotional or economicoutcomes.

IHCAs appear to have largely positive effects onusers, in that users tend to become moreknowledgeable, feel better socially supported,and may have improved behavioural and clinicaloutcomes compared to non-users. There is aneed for more high quality studies with largesample sizes to confirm these preliminaryfindings, to determine the best type and bestway to deliver IHCAs, and to establish howIHCAs have their effects for different groups ofpeople with chronic illness.

764in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771

Table 5 (Continued )

Reference Conditionsincluded

Geographicarea

Service/Intervention

Outcome Authors’ summary of results Authors’ conclusions

Deshpande etal. (syn-chronoustelehealth)[69]

Chronicdisease

All countries Hometelehealth, inreal time

Behavioural,Cost/economic,Feasibility/pilot,Health, Percep-tion/satisfaction,Safety, Social,Technologyrelated

Thirty-one systematic reviewsincluded (11 of high quality). Mostreviews and most of the studies theyreview are low quality. The authorssuggest that this illustrates theresource constraints for researchersand policy makers. Nevertheless,weak evidence that real-timetelehealth can improve service access,user satisfaction and resourceutilization is found. For patients withpsychiatric and neurologicalconditions in remote areas, evidenceof benefit is stronger.

Evidence is generally weak and studies of poorquality, but telehealth is nevertheless promising,especially in a Canadian context over a largegeographical area.

Koch [76] Chronicdiseases,elderlypopulation,paediatrics

Country and noof publications:USA 238; UK 52;Japan 39;Germany 31;Greece 22;Australia 20;Canada 18;France 17; Spain17; China 14;Italy 14; Sweden13; Finland 6;other 77

Hometelehealth

Cost/economic,Percep-tion/satisfaction,Feasibility/pilot,Health,Organizational,Technologyrelated

Five hundred and seventy-eightstudies included. 44% of publicationscome from the United States, followedby UK and Japan. Most cover vital signparameter (VSP) measurement andaudio/video consultations.Publications about IT tools forimproved information access andcommunication as well as decisionsupport for staff, patients andrelatives are relatively sparse. Clinicalapplication domains are mainlychronic diseases, the elderlypopulation and paediatrics.

Internationally, we observe a trend towards toolsand services not only for professionals but alsofor patients and citizens. However, their impacton the patient—provider relationship and theirdesign for special user groups, such as elderlyand/or disabled needs to be further explored. Ingeneral, evaluation studies are rare and furtherresearch is critical to determine the impacts andbenefits, and limitations, of potential solutionsand to overcome a number of hinders andrestrictions, such as the lack of standards tocombine incompatible information systems; thelack of an evaluation framework consideringlegal, ethical, organisational, clinical, usabilityand technical aspects; the lack of properguidelines for practical implementation of hometelehealth solutions.

Lauriks et al.[77]

Dementia No limits ICT-basedinterventions

Health, Percep-tion/satisfaction,Social,Technologyrelated

Forty-six publications and 22 websitesincluded. Needs identified aregeneralized and personalizedinformation; support with regard tosymptoms of dementia; social contactand company; health monitoring andperceived safety.

Information is generally not personalised and isnot attuned to the person with dementia; ICTsolutions aimed at compensating for disabilitiesdemonstrate that people with mild to moderatedementia can handle simple electronicequipment and can benefit from it. InstrumentalICT-support for coping with behavioural andpsychological changes in dementia is relativelydisregarded as yet, while support for socialcontact can be effectively realised through, forexample, simplified (mobile) phones. GPStechnology and monitoring systems are provento result in enhanced feelings of safety and lessfear and anxiety.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

765

Table 6 – Asking new questions.

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Mo et al. [78] All Not stated Online supportgroups

Behavioural,Health, Outcomeof postedmessages usedto categorisegenderdifferences

Twelve studies included. Half of thestudies examined gender differencesby comparing male and female cancerdiscussion boards. Some genderdifferences were observed in thesestudies. However, for studies thatanalysed mixed-gender communities,gender differences were less evident.

Results seemed to reveal gender differences incommunications in single-sex online healthsupport groups, and similarities incommunication patterns in mixed-sex onlinehealth support groups. However, findings shouldbe treated with caution due to the diversity instudies and methodological issues highlighted inthe present review.

Price et al.[79]

Stroke Oceania, Asia,Europe, NorthAmerica, Europe

Serviceconfigurations

Safety, Activitylevel andresponse timesfor thrombolysistreatment

Fifty-four studies included. Localservice configuration provides lessthrombolysis activity than widercollaborations and report as manyprotocol violations despite theirsimpler design. Local variations inpopulation density, geography,experts and activity and safety datareporting formats makes it difficult tocompare service configurations.Reporting of activity and safety in astandardised format is urged.

Stroke services should continue to publishthrombolysis activity and safety data in arecommended format in order to determine themost suitable configuration for different settings.

Deshpande etal. (strokemanage-ment)[80]

Stroke US, Canada,Germany, Italy,Netherlands,China

Acute medicine Cost/economic,Feasibility/pilot,Health, Percep-tion/satisfaction,Technologyrelated

Twenty-two studies included (eight ofhigh quality). Results showedimproved access to thrombolysis,acceptable times between hospitalarrival and thrombolysis, anddecreased need to transfer patientsacross institutions. Mortality rates,and three and six month functionaloutcomes were comparable withthose of face-to-face stroke care.Mortality rates were also similar.Patients and provider satisfaction washigh, though not assessed in detail.

Although it is difficult to draw conclusions fromthis small sample of studies, the trend suggeststhat in post-stroke patients, telehealth led toimprovements in caregivers’ mental health andhigh levels of patient satisfaction. There wasminimal evidence regarding the impact onresource utilization.

766in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771

Table 6 (Continued )

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Sood et al.[81]

All Not stated All Definitions oftelemedicine

Hundred and four studies included,which generate 104 definitions. Theauthors identify four types ofdefinition: (1) Medical: mention of“providing healthcare services”. (2)Technological: indication oftechnology’s role. (3) Spatial:“geographical separation of patientand doctor” pertains to or involvesnature of space/distance. (4) Benefits:medical care is brought to peoplewhen it is not feasible to get people tomedical care.

The article provides formative evaluation byclassifying the different approaches to definingtelemedicine and their underlying theoreticalassumptions.

Pineau et al.(telepsychi-atry)[20]

Psychiatricconditions(adult andpediatric)

Not stated Focuson Canada andUSA

Telepsychiatry Cost/economic,Ethical issues,Legal,Organizational,Technologyrelated, clinicalguidelines andtechnicalstandards

‘About 60′ studies included. Theauthors argue that definition ofclinical guidelines and technologicalstandards aimed at standardizingtelepsychiatric practice will promoteits large scale implementation.

The review concludes that telepsychiatry shouldbe implemented in Québec and provides detailedclinical and technical guidelines forimplementation. They add that taking intoaccount human and organizational aspects playsa part in ensuring the success of this type ofactivity; that legal and ethical aspects must alsobe considered; and that a detailed economicanalysis should be carried out prior to any largeinvestment in telepsychiatry. Finally,implementation of psychiatry should besubjected to rigorous downstream assessment inorder to improve management and performance.

Pineau et al.[82]

Multipleconditions

References inEnglish. Focus onCanada and USA

Telerehabilitation Cost/economic,Ethical issues,Legal,Organizational,Technologyrelated, clinicalguidelines andtechnicalstandards

Unspecified number of studiesincluded. The review offers clinicaland technical guidelines fortelerehabilitation implementation.

The review concludes that telerehabilitationshould be implemented in Québec and providesdetailed clinical and technical guidelines forimplementation. They add that taking intoaccount human and organizational aspects playsa part in ensuring the success of this type ofactivity; that legal and ethical aspects must alsobe considered; and that a detailed economicanalysis should be carried out prior to any largeinvestment in telerehabilitation. Finally,implementation of telerehabilitation should besubjected to rigorous downstream assessment inorder to improve management and performance.

int

er

na

tio

na

ljo

ur

na

lo

fm

ed

ica

lin

fo

rm

at

ics

79

(20

10

)736–771

767

Table 7 – Included reviews not cited in the paper.

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Bonacina etal. [84]

Telecardiology No mention Systematicreview(methodologypaper)

Methodologydevelopment

Sixty-one studies included. Mostliterature related to pilot projects,feasibility studies and short-termoutcomes. The analysis is used todevelop a method of classifyingscientific publications to assess theevidence they contain so that it can beused.

This is a methodological study suggesting aframework for evaluation of literature ontelemedicine.

Christensenet al. [85]

Anxiety anddepression

Not mentioned Open accesswebsites whichdeliver cognitiveand behaviouralinterventions

Behaviouraloutcomes

Twenty-three studies included. Allwere RCTs. Data included dropout andadherence, predictors of adherenceand reasons for dropout. Relative toreported rates of dropout from openaccess sites, the study found that therates of attrition in randomizedcontrolled trials were lower, rangingfrom approximately 1–50%. Predictorsof adherence included diseaseseverity, treatment length, andchronicity. Few studies formallyexamined reasons for dropout, andmost failed to use appropriatetechniques to analyze missing data.

Dropout rates from randomized controlled trialsof Web interventions are low relative to dropoutfrom open access websites. The development oftheoretical models of adherence is as importantin the area of Internet intervention research as itis in the behavioural health literature.

Clarke andThiyagara-jan[86]

None(technical)

Not described Evaluation Technologyrelated

Forty-seven studies included. Theanalysis was unable to identify a‘definitive, standards basedtelemedicine technical evaluationframework.

Work needs to be done to address the deficiencyidentified.

Demiris [87] Manyconditions(alsoprofessionalgroups)

USA, UK,Scandinavia

Not aboutinterventions –focus isdevelopment ofelectroniccommunities

Ethical issues,Policy, Definingfeatures of acomplexintervention

Forty-seven studies included. Virtualcommunities in health care cover awide range of clinical specialties,technologies and stakeholders. Theyinclude peer-to-peer networks, virtualhealth care delivery and researchteams. Ethical challenges includeidentity and deception, privacy andconfidentiality. Technical issuesinclude sociability and usability.

Virtual communities may empower patients andenhance coordination of care services; however,there is not sufficient systematic evidence of theeffectiveness of virtual communities on clinicaloutcomes or patient empowerment. Researchersneed to address issues, such as sample sizes andexperimental design to further the research fieldin this domain.

768in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

9(2

01

0)

736–771

Table 7 (Continued )

Reference Conditionsincluded

Geographicarea

Service/intervention

Outcome Authors’ summary of results Authors’ conclusions

Marziali et al.[91]

Older peoplewithunspecifiedconditions

North Americaand Europe

Home telehealth(ethical issues)

Ethical issues Hundred and seven studies included.Studies show lack of informationabout ethics and professional practice.Where these issues are explored, thisis done to a limited extent.Peer-reviewed work is more likely toconsider these issues.

That studies of telemedicine have addressedissues of ethics and professional practice to alimited extent and that these are not welldocumented.

Oh et al. [92] All No mention Evaluation(definitions)

Organizational,Percep-tion/satisfaction,Policy,Technologyrelated,Definitions ofeHealth

Thousand two hundred and nineabstracts scanned, 430 citationsreviewed, 1158 Google sites reviewed.51 unique definitions were retrieved,showing a wide range of themes, butno clear consensus about the meaningof the term eHealth. Two universalthemes (health and technology) wereidentified and six less general(commerce, activities, stakeholders,outcomes, place, and perspectives).

The widespread use of the term eHealth suggeststhat it is an important concept, and that there isa tacit understanding of its meaning. Thiscompendium of proposed definitions mayimprove communication among the manyindividuals and organizations that use the term.

Scott et al.[93]

All Canada, USA,New Zealand,Europe

All (focus is onidentifyingagreed outcomeindicators)

Behavioural,Cost/economic,Ethical issues,Health, Legal,Organizational,Percep-tion/satisfaction,Policy, Safety,Social, Thesewere grouped

Two hundred and eighteen studiesincluded. A key objective of the NTOIPwas to identify and describe outcomeindicators that had been used toevaluate telehealth projects forquality, access, acceptability and cost.The findings show that the situation iscomplex and confusing, but thatrecommendations can be made.

The study was not considered as an end in itself,but rather the beginning of a continuing processof telehealth evaluation and research in Canadaand elsewhere. That is, the ongoing quest foridentification of a small number of appropriateoutcome indicators (as well as their relatedoutcome measures and outcome tools), theirconsistent description and their consistentapplication in future evaluations. This initiativesuggests that the importance of evaluation andof consistent use of indicators is understood.

a l i n

A

Si

r

i n t e r n a t i o n a l j o u r n a l o f m e d i c

ppendix B. Supplementary data

upplementary data associated with this article can be found,n the online version, at doi:10.1016/j.ijmedinf.2010.08.006.

e f e r e n c e s

[1] R. Roine, A. Ohinmaa, D. Hailey, Assessing telemedicine: asystematic review of the literature, CMAJ 165 (September (6))(2001) 765–771.

[2] D. Hailey, R. Roine, A. Ohinmaa, Systematic review ofevidence for the benefits of telemedicine, J. Telemed.Telecare 8 (Suppl. 1) (2002) 1–30.

[3] R. Currell, C. Urquhart, P. Wainwright, et al., TelemedicineVersus Face to Face Patient Care: Effects on ProfessionalPractice and Health Outcomes, Cochrane Library TheCochrane Collaboration, 2002 [Oxford: Update Software](Issue 3).

[4] W.R. Hersh, D.H. Hickam, S.M. Severance, T.L. Dana, K.P.Krages, M. Helfand, Telemedicine for the medicarepopulation: update, Evid. Rep. Technol. Assess (Full Rep.)(February (131)) (2006) 1–41.

[5] P.S. Whitten, F.S. Mair, A. Haycox, C.R. May, T.L. Williams, S.Hellmich, Systematic review of cost effectiveness studies oftelemedicine interventions, BMJ 324 (June (7351)) (2002)1434–1437.

[6] G. Demiris, D. Tao, An analysis of the specialized literaturein the field of telemedicine, J. Telemed. Telecare 11 (6) (2005)316–319.

[7] P. Jennett, L. Affleck Hall, D. Hailey, A. Ohinmaa, C. Anderson,R. Thomas, et al., The socio-economic impact of telehealth:a systematic review, J. Telemed. Telecare 9 (6) (2003) 311.

[8] E. Johnsen, E. Breivik, R. Myrvang, F. Olsen, Benefits fromTelemedicine in Norway: An Examination of AvailableDocumentation, HØYKOM Report No. 2006: 1. HØYKOMReport Series, 2006, ISSN 1504-5048.

[9] E. Murphy, R. Dingwall, D. Greatbatch, S. Parker, P. Watson,Qualitative research methods in health technologyassessment: a review of the literature, Health Technol.Assess. 2 (16) (1998), iii–ix, 1–274.

[10] W. Hersh, M. Helfand, J. Wallace, D. Kraemer, P. Patterson, S.Shapiro, et al., Clinical outcomes resulting fromtelemedicine interventions: a systematic review, BMC Med.Informatics Decis. Making 1 (1) (2001) 5.

[11] J. Barlow, S. Bayer, B. Castleton, R. Curry, Meetinggovernment objectives for telecare in moving from localimplementation to mainstream services, J. Telemed.Telecare 11 (Suppl. 1) (2005) 49–51.

[12] E. Ammenwerth, J. Brender, P. Nykanen, H.U. Prokosch, M.Rigby, J. Talmon, Visions and strategies to improveevaluation of health information systems. Reflections andlessons based on the HIS-EVAL workshop in Innsbruck, Int. J.Med. Inform. 73 (June (6)) (2004) 479–491.

[13] A.G. Ekeland, A. Bowes, S.A. Flottorp, Methodologies forassessing telemedicine: a systematic review of reviews(under review).

[14] R. Pawson, T. Greenhalgh, G. Harvey, K. Walshe, Realistreview – a new method of systematic review designed forcomplex policy interventions, J. Health Serv. Res. Policy (July

(10 Suppl. 1)) (2005) 21–34.

[15] A. Barak, L. Hen, M.B.-N. Shapira, A comprehensive reviewand a meta-analysis of the effectiveness of Internet-basedpsychotherapeutic interventions, J. Technol. Hum. Services26 (2–4) (2008) 109–114.

f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771 769

[16] R.A. Clark, S.C. Inglis, F.A. McAlister, J.G. Cleland, S. Stewart,Telemonitoring or structured telephone supportprogrammes for patients with chronic heart failure:systematic review and meta-analysis, BMJ 334 (7600) (2007May 5) 942.

[17] M. Jaana, G. Pare, C. Sicotte, Home telemonitoring forrespiratory conditions: a systematic review, Am. J. Manag.Care 15 (May (5)) (2009) 313–320.

[18] S.K. Myung, D.D. McDonnell, G. Kazinets, H.G. Seo, J.M.Moskowitz, Effects of Web- and computer-based smokingcessation programs: meta-analysis of randomized controlledtrials, Arch. Intern. Med. 169 (May (10)) (2009) 929–937.

[19] L. Neubeck, J. Redfern, R. Fernandez, T. Briffa, A. Bauman,S.B. Freedman, Telehealth interventions for the secondaryprevention of coronary heart disease: a systematic review,Eur. J. Cardiovasc. Prev. Rehabil. 16 (June (3)) (2009) 281–289.

[20] G.M.K. Pineau, C. St-Hilaire, R. Perreault, E. Levac, B. Hamel,Telehealth: Clinical Guidelines and Technical Standards forTelepsychiatry, Agence d’Évaluation des Technologies et desModes D’intervention en Santé, Montreal, Quebec, 2006.

[21] M.B. Powers, P.M. Emmelkamp, Virtual reality exposuretherapy for anxiety disorders: a meta-analysis, J. AnxietyDisord. 22 (3) (2008) 561–569.

[22] G.B. Prange, M.J. Jannink, C.G. Groothuis-Oudshoorn, H.J.Hermens, M.J. Ijzerman, Systematic review of the effect ofrobot-aided therapy on recovery of the hemiparetic armafter stroke, J. Rehabil. Res. Dev. 43 (March–April (2)) (2006)171–184.

[23] M.A. Reger, G.A. Gahm, A meta-analysis of the effects ofinternet- and computer-based cognitive-behavioraltreatments for anxiety, J. Clin. Psychol. 65 (January (1)) (2009)53–75.

[24] V. Spek, P. Cuijpers, I. Nyklicek, H. Riper, J. Keyzer, V. Pop,Internet-based cognitive behaviour therapy for symptoms ofdepression and anxiety: a meta-analysis, Psychol. Med. 37(March (3)) (2007) 319–328.

[25] K. Tran, J. Polisena, D. Coyle, Home Telehealth for ChronicDisease Management, Canadian Agency for Drugs andTechnologies in Health, 2008 (113).

[26] M.H. van den Berg, J.W. Schoones, T.P. Vliet Vlieland,Internet-based physical activity interventions: a systematicreview of the literature, J. Med. Internet Res. 9 (3) (2007) e26.

[27] S.E. Hyler, D.P. Gangure, S.T. Batchelder, Can telepsychiatryreplace in-person psychiatric assessments? A review andmeta-analysis of comparison studies, CNS Spectr. 10 (May(5)) (2005) 403–413.

[28] J. Barlow, D. Singh, S. Bayer, R. Curry, A systematic review ofthe benefits of home telecare for frail elderly people andthose with long-term conditions, J. Telemed. Telecare 13 (4)(2007) 172–179.

[29] K.L. Bussey-Smith, R.D. Rossen, A systematic review ofrandomized control trials evaluating the effectiveness ofinteractive computerized asthma patient educationprograms, Ann. Allergy Asthma Immunol. 98 (June (6)) (2007)507–516, quiz 16, 66.

[30] M. Jaana, G. Pare, Home telemonitoring of patients withdiabetes: a systematic assessment of observed effects, J.Eval. Clin. Pract. 13 (April (2)) (2007) 242–253.

[31] P. Azarmina, P. Wallace, Remote interpretation in medicalencounters: a systematic review, J. Telemed. Telecare 11 (3)(2005) 140–145.

[32] G. Demiris, B. Hensel, Technologies for an Aging Society: ASystematic Review of “Smart Home” applications. Yearbookof medical informatics, 2008, p. 33.

[33] A. Martinez, E. Everss, J.L. Rojo-Alvarez, D.P. Figal, A.Garcia-Alberola, A systematic review of the literature onhome monitoring for patients with heart failure, J. Telemed.Telecare 12 (5) (2006) 234–241.

i c a l

770 i n t e r n a t i o n a l j o u r n a l o f m e d

[34] R. Gaikwad, J. Warren, The role of home-based informationand communications technology interventions in chronicdisease management: a systematic literature review, HealthInformatics J. 15 (June (2)) (2009) 122–146.

[35] P. Cuijpers, A. van Straten, G. Andersson,Internet-administered cognitive behavior therapy for healthproblems: a systematic review, J. Behav. Med. 31 (April (2))(2008) 169–177.

[36] P.E. Bee, P. Bower, K. Lovell, S. Gilbody, D. Richards, L. Gask, etal., Psychotherapy mediated by remote communicationtechnologies: a meta-analytic review, BMC Psychiatry 8(2008) 60.

[37] J.H. Crosbie, S. Lennon, J.R. Basford, S.M. McDonough, Virtualreality in stroke rehabilitation: still more virtual than real,Disabil. Rehabil. 29 (July (14)) (2007) 1139–1146, discussion47–52.

[38] A. Henderson, N. Korner-Bitensky, M. Levin, Virtual reality instroke rehabilitation: a systematic review of its effectivenessfor upper limb motor recovery, Top. Stroke Rehabil. 14(March–April (2)) (2007) 52–61.

[39] K.M. Griffiths, H. Christensen, Review of randomizedcontrolled trials of Internet intervention for mentaldisorders and related conditions, Clin. Psychol. 10 (1) (2006)16–29.

[40] C.L. Jackson, S. Bolen, F.L. Brancati, M.L. Batts-Turner, T.L.Gary, A systematic review of interactive computer-assistedtechnology in diabetes care. Interactive informationtechnology in diabetes care, J. Gen. Intern. Med. 21 (February(2)) (2006) 105–110.

[41] P.K. Weinstein, A review of weight loss programs deliveredvia the Internet, J. Cardiovasc. Nurs. 21 (July–August (4))(2006) 251–258, quiz 9–60.

[42] A. Deshpande, S. Khoja, J. Lorca, A. McKibbon, C. Rizo, A.R.Jadad, Asynchronous Telehealth: Systematic Review ofAnalytic Studies and Environmental Scan of RelevantInitiatives, Canadian Agency for Drugs and Technologies inHealth, Ottawa, 2008.

[43] J. Polisena, K. Tran, K. Cimon, B. Hutton, S. McGill, K. Palmer,Home telehealth for diabetes management: a systematicreview and meta-analysis, Diabetes Obes. Metab. 11 (October(10)) (2009) 913–930.

[44] D. Hailey, M.-J. Paquin, O. Maciejewski, L. Harris, A. Casebeer,G. Fick, et al., The Use and Benefits of Teleoncology, TheInstitute of Health Economics (IHE), Canada, 2007, January.

[45] G. Pare, M. Jaana, C. Sicotte, Systematic review of hometelemonitoring for chronic diseases: the evidence base, J.Am. Med. Inform. Assoc. 14 (May–June (3)) (2007) 269–277.

[46] B. Nannings, A. Abu-Hanna, Decision support telemedicinesystems: a conceptual model and reusable templates,Telemed. J. E Health 12 (December (6)) (2006) 644–654.

[47] V. Sintchenko, F. Magrabi, S. Tipper, Are we measuring theright end-points? Variables that affect the impact ofcomputerised decision support on patient outcomes: asystematic review, Med. Inform. Internet Med. 32(September (3)) (2007) 225–240.

[48] O. Wu, P. Langhorne, The challenge of acute-strokemanagement: does telemedicine offer a solution? Int. J.Stroke 1 (November (4)) (2006) 201–207.

[49] K.G. Shojania, A. Jennings, A. Mayhew, C.R. Ramsay, M.P.Eccles, J. Grimshaw, The effects of on-screen, point of carecomputer reminders on processes and outcomes of care,Cochrane Database Syst. Rev. (3) (2009) CD001096.

[50] S.I. Chaudhry, C.O. Phillips, S.S. Stewart, B. Riegel, J.A.Mattera, A.F. Jerant, et al., Telemonitoring for patients with

chronic heart failure: a systematic review, J. Card. Fail. 13(February (1)) (2007) 56–62.

[51] A. Farmer, O.J. Gibson, L. Tarassenko, A. Neil, A systematicreview of telemedicine interventions to support blood

i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771

glucose self-monitoring in diabetes, Diabet. Med. 22(October (10)) (2005) 1372–1378.

[52] D.L. Sanders, D. Aronsky, Biomedical informaticsapplications for asthma care: a systematic review, J. Am.Med. Inform. Assoc. 13 (July–August (4)) (2006)418–427.

[53] F. Verhoeven, L. van Gemert-Pijnen, K. Dijkstra, N. Nijland, E.Seydel, M. Steehouder, The contribution of teleconsultationand videoconferencing to diabetes care: a systematicliterature review, J. Med. Internet Res. 9 (5) (2007) e37.

[54] B.M. Bewick, K. Trusler, M. Barkham, A.J. Hill, J. Cahill, B.Mulhern, The effectiveness of web-based interventionsdesigned to decrease alcohol consumption – a systematicreview, Prev. Med. 47 (July (1)) (2008) 17–26.

[55] B. Maric, A. Kaan, A. Ignaszewski, S.A. Lear, A systematicreview of telemonitoring technologies in heart failure, Eur. J.Heart Fail. 11 (May (5)) (2009) 506–517.

[56] M.G. Postel, H.A. de Haan, C.A. De Jong, E-therapy for mentalhealth problems: a systematic review, Telemed. J. E Health.14 (September (7)) (2008) 707–714.

[57] S. Martin, G. Kelly, W.G. Kernohan, B. McCreight, C. Nugent,Smart home technologies for health and social care support,Cochrane Database Syst. Rev. (4) (2008) CD006412.

[58] J. Powell, T. Chiu, G. Eysenbach, A systematic review ofnetworked technologies supporting carers of people withdementia, J. Telemed. Telecare 14 (3) (2008) 154–156.

[59] F. Garcia-Lizana, A. Sarria-Santamera, New technologies forchronic disease management and control: a systematicreview, J. Telemed. Telecare 13 (2) (2007) 62–68.

[60] M.P. Gagnon, F. Legare, M. Labrecque, P. Fremont, P. Pluye, J.Gagnon, et al., Interventions for promoting information andcommunication technologies adoption in healthcareprofessionals, Cochrane Database Syst. Rev. (1) (2009)CD006093.

[61] W. Hersh, D. Hickam, S. Severance, T. Dana, K. Krages, M.Helfand, Diagnosis, access and outcomes: update, J.Telemed. Telecare 12 (2) (2006) 3–31.

[62] S. Linton, Datorbaserad kognitiv beteendeterapi vidångestsyndrom eller depression, 2007.

[63] A. Mathur, J.C. Kvedar, A.J. Watson, Connected health: a newframework for evaluation of communication technology usein care improvement strategies for type 2 diabetes, Curr.Diabetes Rev. 3 (November (4)) (2007) 229–234.

[64] S.T. Walters, J.A. Wright, R. Shegog, A review of computerand Internet-based interventions for smoking behavior,Addict. Behav. 31 (February (2)) (2006) 264–277.

[65] T.P. Lintonen, A.I. Konu, D. Seedhouse, Informationtechnology in health promotion, Health Educ. Res. 23 (June(3)) (2008) 560–566.

[66] J. van Nooten, H. Oh, B. Pierce, F.J. Koning, A.R. Jadad,Spiritual care as eHealth: a systematic review, J. Pastoral.Care Counsel. 60 (Winter (4)) (2006) 387–394.

[67] S.V. Rojas, M.P. Gagnon, A systematic review of the keyindicators for assessing telehomecare cost-effectiveness,Telemed. J. E Health 14 (November (9)) (2008) 896–904.

[68] E. Seto, Cost comparison between telemonitoring and usualcare of heart failure: a systematic review, Telemed. J. EHealth 14 (September (7)) (2008) 679–686.

[69] A. Deshpande, S. Khoja, A. McKibbon, A.R. Jadad, Real-Time(synchrounous) Telehealth in Primary Care: SystematicReview of Systematic Reviews, 2008.

[70] E. Kaltenthaler, J. Brazier, E. De Nigris, I. Tumur, M. Ferriter,C. Beverley, et al., Computerised cognitive behaviour therapyfor depression and anxiety update: a systematic review and

economic evaluation, Health Technol. Assess. 10 (September(33)) (2006), iii, xi–xiv, 1–168.

[71] J. Polisena, D. Coyle, K. Coyle, S. McGill, Home telehealth forchronic disease management: a systematic review and an

a l i n

i n t e r n a t i o n a l j o u r n a l o f m e d i c

analysis of economic evaluations, Int. J. Technol. Assess.Health Care 25 (July (3)) (2009) 339–349.

[72] F. Griffiths, A. Lindenmeyer, J. Powell, P. Lowe, M. Thorogood,Why are health care interventions delivered over theinternet? A systematic review of the published literature, J.Med. Internet Res. 8 (2) (2006) e10.

[73] K.M. Akesson, B.I. Saveman, G. Nilsson, Health careconsumers’ experiences of information communicationtechnology – a summary of literature, Int. J. Med. Inform. 76(September (9)) (2007) 633–645.

[74] D. Kairy, P. Lehoux, C. Vincent, M. Visintin, A systematicreview of clinical outcomes, clinical process, healthcareutilization and costs associated with telerehabilitation,Disabil. Rehabil. 31 (6) (2009) 427–447.

[75] E. Murray, J. Burns, T.S. See, R. Lai, I. Nazareth, Interactivehealth communication applications for people with chronicdisease, Cochrane Database Syst. Rev. (4) (2005) CD004274.

[76] S. Koch, Home telehealth-current state and future trends,Int. J. Med. Inform. 75 (August (8)) (2006) 565–576.

[77] S. Lauriks, A. Reinersmann, H.G. Van der Roest, F.J. Meiland,R.J. Davies, F. Moelaert, et al., Review of ICT-based servicesfor identified unmet needs in people with dementia, AgeingRes. Rev. 6 (October (3)) (2007) 223–246.

[78] P.K. Mo, S.H. Malik, N.S. Coulson, Gender differences incomputer-mediated communication: a systematic literaturereview of online health-related support groups, PatientEduc. Couns. 75 (April (1)) (2009) 16–24.

[79] C.I. Price, F. Clement, J. Gray, C. Donaldson, G.A. Ford,Systematic review of stroke thrombolysis serviceconfiguration, Expert Rev. Neurother. 9 (February (2)) (2009)211–233.

[80] A. Deshpande, S. Khoja, A. McKibbon, C. Rizo, A.R. Jadad,Telehealth for Acute Stroke Management (Telestroke):Systematic Review of Analytic Studies and EnvironmentalScan of Relevant Initiatives, 2008.

[81] S. Sood, V. Mbarika, S. Jugoo, R. Dookhy, C.R. Doarn, N.

Prakash, et al., What is telemedicine? A collection of 104peer-reviewed perspectives and theoretical underpinnings,Telemed. J. E Health 13 (October (5)) (2007) 573–590.

[82] G. Pineau, K. Moqadem, C. St Hilaire, R. Perreault, E. Levac, B.Hamel, et al., Telehealth: Clinical Guidelines and

f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771 771

Technological Standards for Telerehabilitation, Agenced’Evaluation des Technologies et des Modes d’Interventionen Sante (AETMIS), Montreal, 2006, No. 101.

[83] M. Azar, Gabbay R, Web-based management of diabetesthrough glucose uploads: has the time come fortelemedicine? Diabetes Res. Clin. Pract. 83 (1) (2009) 9–17.

[84] S. Bonacina, L. Draghi, M. Masseroli, Pinciroli F,Understanding telecardiology success and pitfalls by asystematic review, Connect. Med. InformaticsBio-Informatics 116 (2005) 373–378.

[85] H. Christensen, K.M. Griffiths, L. Farrer, Adherence ininternet interventions for anxiety and depression, J. Med.Internet Res. 11 (June (2)) (2009).

[86] M. Clarke, Thiyagarajan CA, A systematic review of technicalevaluation in telemedicine systems, Telemed. J. E Health 14(2) (2008) 170–183.

[87] G. Demiris, The diffusion of virtual communities in healthcare: concepts and challenges, Patient Educ. Couns. 62 (2)(2006) 178–188.

[88] J.C. Elliott, K.B. Carey, J Bolles, R Computer-basedinterventions for college drinking: a qualitative review,Addict. Behav. 33 (8) (2008) 994–1005.

[89] R. Holland, J. Battersby, I. Harvey, E. Lenaghan, J. Smith, L.Hay, Systematic review of multidisciplinary interventions inheart failure, Heart 91 (7) (2005) 899–906.

[90] M. Jaana, G. Pare, Sicotte C, Hypertension hometelemonitoring: current evidence and recommendations forfuture studies, Dis. Manag. Health Out 15 (1) (2007)19–31.

[91] E. Marziali, J.M.D. Serafini, McCleary L, A systematic reviewof practice standards and research ethics intechnology-based home health care intervention programsfor older adults, J. Aging Health 17 (6) (2005) 679–696.

[92] H. Oh, C. Rizo, M. Enkin, Jadad A, What is eHealth (3): asystematic review of published definitions’, J. Med. InternetRes. 7 (1) (2005) e1.

[93] R.E. Scott, F.G. McCarthy, P.A. Jennett, T. Perverseff, D.Lorenzetti, A. Saeed, et al., Telehealth outcomes: a synthesisof the literature and recommendations for outcomeindicators, J. Telemed. Telecare 13 (Suppl. 2) (2007)1–38.