asernip-s inventory of current methods and practice · pdf filean inventory of current methods...

Review

Rapid versus full systematic reviews: an inventory of current methods and practice in Health Technology Assessment

ASERNIP-S REPORT NO. 60

July 2007

Australian Safety & Efficacy Register of New Interventional Procedures – Surgical

The Royal Australasian College of Surgeons

ASERNIP S Australian Safety

and Efficacy

Register of New

Interventional

Procedures - Surgical

- ASERNIP-S INVENTORY OF CURRENT METHODS AND PRACTICE IN HTA JULY 2007 -

Rapid versus full systematic reviews: an inventory of current methods and practice in Health Technology Assessment.

ISBN 0909844 82 8

Published July 2007

This report should be cited in the following manner:

Cameron A., et al. Rapid versus full systematic reviews: an inventory of current methods and practice in Health Technology Assessment. ASERNIP-S Report No. 60. Adelaide, South Australia: ASERNIP-S, July 2007.

Copies of these reports can be obtained from:

ASERNIP-S PO Box 553, Stepney, SA 5069 AUSTRALIA Ph: 61-8-8363 7513 Fax: 61-8-8362 2077 E-Mail: [email protected] http://www.surgeons.org/asernip-s


Rapid versus full systematic rmethods and practice in He

was rat

The ASERNIP-S Man

June

a

The Council of the Royal Aust

July

eviews: an inventory of current alth Technology Assessment

ified by:

agement Committee in

2007

nd

ralasian College of Surgeons on

2007


Table of Contents

Executive Summary ........................................................................................... v The ASERNIP-S Review Group ..................................................................... viii 1. Introduction ............................................................................................... 1

Health technology assessment...........................................................................................1 Rapid review.........................................................................................................................2 Objectives .............................................................................................................................3

2. Methods...................................................................................................... 4 2.1. Survey of HTA organisations.....................................................................................4 2.2. Identification of literature on rapid review methodology ......................................5

Literature search protocol ............................................................................................5 Literature search strategies ...........................................................................................6

Databases searched .............................................................................................................6 Methods of the review ..................................................................................................7

2.3. Identification and comparison of rapid reviews and full systematic reviews......8 Rapid review...................................................................................................................8 Full reviews.....................................................................................................................9 Outcome measures..................................................................................................... 10

Methods of the review..................................................................................................... 10 3. Results .......................................................................................................12

3.1. Survey of HTA organisations.................................................................................. 12 Survey response data .................................................................................................. 12 Administration of rapid review process .................................................................. 13 Research strategy used for rapid reviews ................................................................ 15 Composition of rapid reviews .................................................................................. 17 Experts and peer review ............................................................................................ 17 Comparison of rapid review products with full systematic reviews.................... 18

3.2. Literature on rapid review methodology ............................................................... 21 3.3. Identification and comparison of rapid reviews and full systematic reviews... 24

Scoping search for rapid reviews.............................................................................. 24 HTA agency website search for rapid products..................................................... 26 Drug eluting stents ..................................................................................................... 26 Lung volume reduction surgery................................................................................ 31 Living donor liver transplantation............................................................................ 34

i


Hip Resurfacing...........................................................................................................39 4. Discussion ................................................................................................ 43

4.1. Survey of HTA organisations ..................................................................................43 Limitations of the survey............................................................................................46 Summary .......................................................................................................................46

4.2. Literature on rapid review methodology ................................................................47 4.3. Identification and comparison of rapid reviews and full systematic reviews....49

Summary .......................................................................................................................52 Future research ..................................................................................................................53

5. Conclusions and Recommendations ....................................................... 54 Acknowledgments.............................................................................................................55

6. References ................................................................................................ 57 Appendix A – Survey........................................................................................................64 Appendix B – INAHTA members.................................................................................68 Appendix C – Excluded studies......................................................................................70 Appendix D – HTA agencies searched .........................................................................77 Appendix E – Excluded agencies ...................................................................................81 Appendix F – Details of products from included websites ........................................83 Appendix G – Data extraction tables.............................................................................93

ii


List of Tables

Table 1. Databases searched................................................................................................. 6

Table 2. Search terms used for MEDLINE....................................................................... 6

Table 3. Search terms used for EMBASE and AMI......................................................... 6

Table 4. Survey response data ............................................................................................ 12

Table 5. Reason for conducting a rapid review ............................................................... 13

Table 6. Commissioning of rapid reviews by level of decision-maker ......................... 14

Table 7. Level of intended audience of rapid reviews .................................................... 15

Table 8. Full time equivalent (FTE) staff assigned to rapid reviews ............................ 15

Table 9. Search strategy for rapid review products ......................................................... 16

Table 10. Study types excluded in rapid review products .............................................. 16

Table 11. Components of rapid review products............................................................ 17

Table 12. Use of experts and peer review in rapid review products............................. 17

Table 13. Comparison of search strategies for systematic reviews and rapid reviews18

Table 14. Exclusion of study types in systematic reviews compared to rapid reviews................................................................................................................................................. 19

Table 15. Reported components of systematic reviews compared to rapid reviews . 19

Table 16. Use of external experts and peer review in systematic reviews compared to rapid reviews ......................................................................................................................... 20

Table 17. Rapid review methodology: included studies ................................................. 22

Table 18. Summary of HTA website search results for rapid review products .......... 25

Table 19. Basic characteristics of review products for DES.......................................... 27

Table 20. Comparators and outcomes for DES.............................................................. 28

Table 21. Methodology of review products for DES ..................................................... 29

Table 22. Essential conclusions of review products for DES ....................................... 30

Table 23. Basic characteristics of review products for LVRS........................................ 31

Table 24. Comparators and outcomes for LVRS............................................................ 32

Table 25. Methodology of review products for LVRS ................................................... 32

Table 26. Essential conclusions of review products for LVRS.................................... 34

Table 27. Basic characteristics of review products for LDLT....................................... 35

Table 28. Comparators and outcomes for LDLT ........................................................... 36

iii


Table 29. Methodology of review products for LDLT...................................................37

Table 30. Essential conclusions of review products for LDLT.....................................38

Table 31. Basic characteristics of review products for hip resurfacing.........................39

Table 32. Comparators and outcomes for hip resurfacing .............................................40

Table 33. Methodology of review products for hip resurfacing ....................................41

Table 34. Essential conclusions of review products for hip resurfacing ......................42

iv


Executive Summary

Objectives The objectives of this report were:

to assess current practice in the preparation of rapid reviews by HTA organisations nationally and internationally,

to examine the current evidence base pertaining to the methodology of rapid reviews and

to determine if there are differences in the essential conclusions of rapid and full reviews on the same topic.

Methods Three concurrent methodologies were employed to inform this report. A survey tool was developed and distributed electronically to 50 national and international HTA agencies, identified through INAHTA membership records and Review Group advice. Data on a broad range of themes related to the conduct of rapid reviews and their comparison to full reviews were collated via spreadsheet tabulation, discussed narratively and subjected to simple statistical analysis where appropriate.

Systematic literature searches of The Cochrane Database of Methodology Reviews, The Cochrane Methodology Register, EMBASE, MEDLINE and the Australasian Medical Index were undertaken in March 2007 to identify any literature pertaining to methodology developed for undertaking rapid reviews. Comparative studies, guidelines, program evaluations, methods studies, commentaries and surveys were considered for inclusion.

The internet sites of 75 international HTA organisations were searched for rapid reviews meeting pre-defined inclusion criteria. For each rapid review identified, a literature search was undertaken utilising the University of York CRD database to identify full reviews (systematic reviews or HTA reports) published on the same topic within approximately one year of the identified rapid review. Clinical outcomes, the scope of the report, the methodology employed in its production and the essential conclusions of each review were used to compare rapid reviews with full systematic reviews.

Results Survey of HTA organisations – 23 surveys were returned, with 18 agencies reporting the production of 36 rapid review products. Seventeen of these products were completed between one to three months, and a further sixteen between three to six months. Three products did not fit into these categories and were considered separately. Collectively, the most common reason for conducting a rapid review was in response to political urgency and/or to support decisions (44%), and 69% of respondents

v


indicated that macro-level decision makers commissioned rapid reports. Search strategies varied widely; however, there was an overall focus on identifying higher levels of evidence wherever possible. The components of reviews also varied between product types, with full reviews more likely to report clinical outcomes (100% vs. 94%), examine economic factors (92% vs. 72%) and consider social issues (85% vs. 53%).

Literature on rapid review methodology – A total of 11 relevant studies were identified: one guideline abstract; three program evaluations; one comparative study; two methods studies; three commentaries and one survey. None of the included studies detailed guidelines for the methodology underpinning rapid reviews; rather, many offered examples and discussion surrounding the complexity of the area. Authors suggested restricted research questions and truncated search strategies as potential methods by which to limit the time taken to complete a review.

Identification and comparison of rapid reviews and full systematic reviews – A manual search of HTA agency websites identified eight agencies that produced rapid products. Where there was uncertainty surrounding a product, the agency was contacted to provide clarification. Comparisons were carried out between full and rapid reviews on the topics of drug eluting stents, lung volume reduction surgery, living donor liver transplantation and hip resurfacing. Axiomatic differences between the products were identified; however, there were no instances in which the essential conclusions of the different reviews were opposed. The full reviews consistently provided greater depth of information and more detailed recommendations pertaining to the implementation of each particular health technology.

Conclusions and Recommendations This report has identified that the current rapid review products being produced by HTA agencies are not well defined and are highly variable in their methodology. However, it is a reality of the HTA environment that there will continue to be pressure to produce reviews that are both timely and accurate, in order to support the ever-increasing speed of the policy making process in this area.

It is therefore recommended that, rather than developing a formalised methodology by which to conduct rapid reviews, which may inappropriate and oversimplified, agencies work to increase the transparency of the methods utilised for each review. It would thus be useful if HTA agencies could clearly identify their HTA products, with respect to the commissioning group and purpose of the review along with some general details outlining the methodologies used in their preparation. Despite this, it should be appreciated that certain parts of a comprehensive systematic review (such as an independent and complete economic evaluation) may not realistically be completed in a rapid timeframe. Furthermore, methods for incorporating the advice of expert panels in a timely manner need to be developed to ensure that rapid reviews reach appropriate conclusions at both clinical and policy levels.

vi


A rapid review should be written in answer to specific questions, rather than as a quick alternative to a comprehensive systematic review. In this manner, rapid reviews could be used to inform specific policy decisions in a timely manner without losing any of the important information that may be expected from a comprehensive review. It is perhaps the focus on appropriate use, along with suitable methodologies, of a rapid review that requires future consideration.

vii


The ASERNIP-S Review Group

ASERNIP-S Director Professor Guy Maddern Surgical Director ASERNIP-S Royal Australasian College of Surgeons Stepney South Australia

Advisory Panel members Mr Stephen Blamey Chair, Medical Services Advisory Committee (MSAC) Head of Surgery, Monash Medical Centre Melbourne Australia Dr Karen Facey Evidence Based Health Policy Consultant Drymen United Kingdom

Mr Mike Gaucher Vice-President, Health Technology Assessment Canadian Agency for Drugs and Technologies in Health Ottawa, Ontario Canada Dr David Hailey Senior Advisor Institute of Health Economics Edmonton Canada Dr Inger Norderhaug Research Director Norwegian Knowledge Centre for the Health Services Oslo Norway

viii


Dr Wendy Babidge Director, Research and Audit Royal Australasian College of Surgeons Stepney South Australia

ASERNIP-S Researchers Dr Alun Cameron Ms Amber Watt Mr Tim Lathlean Ms Lana Sturm ASERNIP-S Royal Australasian College of Surgeons Stepney South Australia

Conflict of Interest None of the authors have declared a financial conflict of interest.

ix


1. Introduction

Health technology assessment Health technology assessment (HTA) is the systematic evaluation of properties, effects, or impacts of healthcare technology (Goodman and Ahn 1999). HTA presents the opportunity to evaluate one or more of the following attributes of a health technology:

• The safety and efficacy/effectiveness of a health technology compared to existing technologies

• Economic attributes or impacts • Social, legal, ethical and/or political impacts of a technology.

HTA has been defined as “The systematic evaluation of properties, effects, and/or impacts of healthcare technology. It may address the direct, intended consequence of technologies as well as their indirect, unintended consequences. Its main purpose is to inform technology-related policymaking in healthcare. HTA is conducted by interdisciplinary groups using explicit analytical frameworks drawing from a variety of methods” (Jonsson et al. 2002).

With the rapid proliferation of healthcare technologies, there has been an increasing demand from clinicians, consumers and policy makers for evidence-based information and recommendations regarding these technologies. HTA has rapidly become an essential tool for understanding the potential benefits, effects and costs that new technologies present.

HTA reflects a multidisciplinary effort, and is undertaken using explicit methodological and analytical frameworks. While HTA may encompass a number of components and report styles, systematic reviews are frequently employed as a key element.

Systematic reviews are a form of structured literature review that address a specific question which is formulated to be answered by the analysis of available evidence. They involve an objective means of searching the literature, the application of predetermined inclusion and exclusion criteria, critical appraisal and the extraction and analysis of data (NICHSR 2006).

The time taken to complete a full systematic review depends on the reviewing organisation, the complexity of the question being addressed and the scope of available evidence. Inherent delays in completing administrative functions such as organising meetings with advisory panels, or overseas ordering of studies can be a real problem for some HTA programs. It has been documented that the mean total number of hours required to undertake a review process utilising similar methodological principles is 1139 (median 1110), with a wide range from 216 to 2518 hours (Allen and Olkin 1999). The component mean times reported were (1) pre-analysis search, retrieval and database development: 588 (standard deviation (SD)

SECT ION 1 INTRODUCTION 1


337) hours; (2) statistical analysis: 144 (SD 106) hours; (3) report and manuscript writing: 206 (SD 125) hours; and (4) other (administrative): 201 (SD 193) hours. This represents a significant investment of resources on behalf of the individual or organisation undertaking the review, and diminishes the ability of HTA to play a role in the rapid analysis and formulation of recommendations often required by decision-makers.

Rapid review For HTA to have an impact, it should reflect the needs of its users (Paolucci et al. 2005). While detailed assessment of health technologies is an important goal, another reality for HTA is that advice may be required quickly by decision makers. There is a tension between delay in obtaining and analysing data and the imperatives of the policy-making process, where HTA input must be timely if it is to influence decisions (Hailey et al. 2000).

Thus the concept of ‘rapid review’ is increasing, driven primarily by this need to engage with clinicians, consumers and policy makers in a timely manner in order to provide evidence-based recommendations pertaining to healthcare activities and decisions.

Internationally, several organisations now produce accelerated assessments in timeframes ranging from one week to many months. These documents vary widely in the methodology employed to prepare them, the scope of the research question and the intended audience.

This brings to the fore the question of the validity of rapid HTA, given the alterations in methodology required to produce a rapid assessment. These methodological differences could include, but not be limited to:

• Addressing only a narrow research question – e.g. specific to a unique patient group, or single indication

• Restricting study inclusion to only those of the most appropriate methodology for the question (utilising study levels of evidence hierarchies such as those designated by NHMRC in Australia)

• Limiting peer-review • Limiting the number and/or scope of databases searched • Using structured submission forms to generate appropriate research

questions • Limiting grey literature searched • Limiting the use of handsearching • Using stricter language restrictions during searches

However, the impact of these alterations on the validity of each review is yet to be assessed. There is not yet a consensus among the HTA community on the ideal methodology to employ in the production of a rapid review.

2 SECT ION 1 INTRODUCTION


Objectives Rapid reviews of healthcare technologies are gaining prominence, as HTA organisations attempt to keep pace with the introduction of increased volumes of new technologies and the resultant policy cycle. However, there remains a need to determine the current use of rapid reviews, the methodology upon which they are based and the validity of their conclusions.

Hence, the objective of this review is threefold:

To assess current practice in the preparation of rapid reviews by HTA organisations nationally and internationally by attempting to identify: • which HTA organisations are currently producing rapid reviews • the definitions of ‘rapid review’ currently in use • the purposes for which rapid reviews are used.

To examine the current evidence base pertaining to the methodology of rapid reviews, including: • identifying if there is a consistent methodology applied to the preparation

of rapid reviews • highlighting the current state of discourse pertaining to rapid review

methodology.

To determine if there are any differences in essential conclusions between rapid and full reviews of the same topic by identifying: • how essential conclusions of a rapid review compare with those of a full

systematic review on the same topic • whether any identified differences can be accounted for by the differing

methodologies employed by the two products.

SECT ION 1 INTRODUCTION 3


2. Methods Three concurrent methodologies were undertaken for this report. A survey tool was developed to allow a scan of rapid products from HTA agencies; a systematic literature search was undertaken to identify methodological papers pertaining to rapid products; and an internet search of HTA agencies was undertaken to identify rapid reviews which were then matched with a full review written on the same topic at around the same time. As the three methodologies were carried out concurrently, the results from one section could not be used to inform other sections of the report.

For the purposes of this report, the working definition of a rapid review shall be: “any HTA report or systematic review that has taken between 1-6 months to produce which contains the elements of a comprehensive literature search”.

The comparator to the rapid review shall be a comprehensive HTA report or systematic review whose production has not been limited by time. For the purposes of this review, these reports shall be termed ‘full review’.

2.1. Survey of HTA organisations A survey tool (Appendix A) was developed and distributed to all INAHTA members (46 Agencies, Appendix B) in March 2007 to undertake an environmental scan to assess the current status of rapid review production both within Australia and internationally. The following organisations that are not INAHTA members were also contacted:

• NICE (UK) (www.nice.org.uk/) • Malaysian HTA program (www.moh.gov.my/MohPortal/) • Singapore Health Authority (www.hsa.gov.sg/) • Agostino Gemelli University Teaching Hospital

(www.policlinicogemelli.it)

The survey addressed a broad range of themes related to the conduct of rapid reviews and used open-ended narrative questions and specific categorical questions. Respondents were also asked to compare a number of features of their rapid review products with those of full reviews.

Initially, the survey was distributed electronically via the INAHTA Secretariat, with reminder emails after one month. If a response was not received, further follow-up was undertaken.

Data analysis Data was collated via spreadsheet tabulation. The main outcomes were discussed narratively. Simple descriptive statistics were limited to the calculation of percentage values across the data set.

4 SECT ION 2 METHODS


2.2. Identification of literature on rapid review methodology A systematic literature search was undertaken to identify methodological articles pertaining to rapid products, both as stand-alone reviews and part of larger HTAs.

Literature search protocol Inclusion criteria Articles were selected for inclusion in this systematic review on the basis that they reported at least one of the following outcomes in either a qualitative or quantitative manner:

Rapid review initiation, including: • Rationale for undertaking rapid, rather than full, review • Source of request for the rapid review • Type of technology to be assessed (emerging or existing)

Rapid review methodology, including: • Scope of question addressed • Methodologies utilised for increasing speed

Content/completion of the rapid review, including: • Size and composition of the review • Time taken (person hours) for completion of the review • Dissemination and impact of the rapid review • Peer-review procedures

Quality evaluation of the rapid review, including: • Comparison to full systematic review • Further review validation by later studies

Types of studies Studies considered included comparative studies, guidelines, program evaluations, methods studies, commentaries and surveys.

Language restriction Searches were conducted without language restriction. Foreign language articles were subsequently excluded due to time and financial constraints.

SECT ION 2 METHODS 5


Literature search strategies

Databases searched Table 1. Databases searched

Database Platform Edition Cochrane Library - Cochrane Database of Methodology Reviews

Issue 4, 2006

- Cochrane Methodology Register Issue 4, 2006 EMBASE Ovid Week 1 1980 to 19/03/2007 MEDLINE Ovid 1950 to 19/03/2007 Australasian Medical Index (AMI) Informit 2004 to 19/03/2007

Search terms Search terms used for The Cochrane Library:

methodology AND (rapid OR accelerated OR quality assessment)

Table 2. Search terms used for MEDLINE

# Search terms 1 Exp: Evidence-Based Medicine/ classification, methods, education, standards, trends 2 KW: health technology assessment 3 1 OR 2 4 KW: rapid 5 KW: methodology 6 KW: accelerated 7 KW: quality assessment 8 KW: accuracy 9 KW: quality 10 KW: speed 11 4 OR 5 OR 6 OR 7 OR 8 OR 9 OR 10 12 3 AND 11

Table 3. Search terms used for EMBASE and AMI

# Search terms 1 *Evidence Based Medicine 2 KW: health technology assessment 3 1 OR 2 4 KW: rapid 5 KW: methodology 6 KW: accelerated 7 KW: quality assessment 8 KW: accuracy 9 KW: quality 10 KW: speed 11 4 OR 5 OR 6 OR 7 OR 8 OR 9 OR 10 12 3 AND 11



Handsearching The International Journal of Technology Assessment in Healthcare (1998 – March 2007) was handsearched for germane articles, which were retrieved as appropriate.

Methods of the review Literature database Articles were retrieved if they were judged to possibly meet the inclusion criteria based on their abstracts. Two ASERNIP-S Researchers independently applied the selection criteria and any differences were resolved through discussion. In some cases, when the full text of the article was retrieved, closer examination revealed that it did not meet the inclusion criteria specified by the review protocol. Consequently these papers were not used to formulate the evidence base for the systematic review (see Figure 1 and Appendix C). However, relevant information contained in these excluded papers was used to inform and expand the review discussion. The bibliographies of all publications retrieved were manually searched for relevant references that may have been missed in the database search (pearling).

Data analysis The included studies were considered by type of study, with data reported qualitatively.



2.3. Identification and comparison of rapid reviews and full systematic reviews

Rapid review The internet sites of international health technology assessment organisations (including INAHTA members and non-members) were searched for rapid reviews (Appendix D). Agencies that were included after being found to contain rapid reviews are listed in Appendix E.

Inclusion criteria Health technology assessment products which:

• appeared to be produced within one to six months • appeared to contain the elements of a comprehensive or systematic/

quasi-systematic literature search • were available in full, not as a summary • were produced by not-for-profit organisations (i.e. the reviews were

freely accessible) • were published between 2001-2004

The year range 2001-2004 was chosen as this provided a more limited and specific representative sample of the HTA literature. It also allowed scope for the comparator to have been published either before or after the date of the rapid review which may not have been the case with very recent reviews.

Exclusion criteria Health technology assessment products which:

• appeared to be completed in greater than six months or less than one month, regardless of the nomenclature of the product (including Horizon Scanning Reports and Prioritisation Summaries)

• were produced in one to six months but did not appear to contain a systematic search for studies

• were produced by for-profit organisations. Accessibility was a problem to those agencies which charge a fee to view each review.

• were English language summaries of foreign language papers • were produced only in a language other that English • were not primarily concerned with surgical interventions (due to the

remit of ASERNIP-S).

Note When conducting the scoping review of the information of agency websites, every attempt was made to consistently apply the inclusion and exclusion criteria. However, due to the scope and number of HTA products from worldwide agencies, a fully systematic search was not possible. It was often difficult to be absolutely clear regarding the type of HTA product and defining whether a product was completely



systematic, horizon scanning or a rapid review according to our definition. If the information provided on the agency website was ambiguous, the agency was contacted for clarification. When a response provided guidance regarding the interpretation of a particular product, the agency’s advice was heeded.

It is acknowledged that in this search all possible HTA rapid products were not identified. In addition, products may have been included that may not be considered ‘rapid’ products.

Full reviews For each rapid review topic identified, a literature search was undertaken utilising the University of York Centre for Reviews and Dissemination (CRD) database (HTA data set) to identify full reviews (systematic reviews or HTA reports) published on the same topic produced by HTA organisations. CRD hits under the Database of Abstracts of Reviews of Effects (DARE), Economic Evaluation Database (EED), and HTA Projects databases were not included.

The comparators to rapid reviews were full reviews (comprehensive systematic reviews, or the systematic review component of full HTAs). All other identified reviews which complied with the inclusion and exclusion criteria were also included. This was in order to illustrate the range of products which had been written on the same topic in the same approximate time period. It is acknowledged that these HTA products differ in their scope and audience.

Inclusion criteria

• Comprehensive systematic reviews or HTAs which were written on the same subject as the previously identified rapid reviews

• The review was available in full text, not as a summary • The review was published within approximately one year of the identified

rapid review (so that both reports would have had access to a similar group of studies)

To ensure a balanced assessment of products from different HTA agencies, no more than two topics were selected from each agency for the rapid product. Topics which enabled the most accurate reflection of the inclusion criteria for both rapid and full reviews were chosen in preference to other topics.

Exclusion criteria Full health technology assessment products which:

• did not adequately report the methodology undertaken (e.g. databases searched for studies, search criteria used as this precluded proper assessment and inter-review comparison)

• were not available in the English language • were not available in full text • were not published within approximately one year of the rapid review



• were written on the same topic as the rapid review product but contained a diverse variety of interventions and comparators not considered in the rapid review (e.g. in the case of morbid obesity and bariatric surgery where, in some cases, the intervention included non-surgical treatments as well as a wide variety of surgical interventions)

Where suitably matched rapid and full HTA products were identified, the definition of each product, according to the agency which produced it, was given. Correspondence, defining or confirming each product, was sought from each agency where there was uncertainty regarding the nature of the included product.

Note The intention of this scoping was to assemble reviews written on the same topic by various agencies; some being more rapid, others more comprehensive. For each included topic, an attempt was made to have a standard comparator, that is, a comprehensive systematic review or HTA written on that topic, with approximately the same publication dates. Differences in the methodology, results and conclusions of the included reviews were discussed narratively.

Outcome measures Clinical outcomes (safety and efficacy or effectiveness) discussed by each included review were investigated, as well as the overall conclusion, the scope of the report and the methodology employed in its production. Outcome measures used to compare rapid reviews with full systematic reviews were limited to the systematic review component of full reports and included:

• publication and search dates • number of pages (for the whole review) • treatment and comparator intervention • details of literature search • number of authors and use of an expert panel/ peer review • number and hierarchy of studies included plus quality assessment of the

included studies • scope of results (safety, efficacy/effectiveness, cost-effectiveness) • use of advanced methodology (e.g. meta-analysis, economic analysis) • final conclusions

Methods of the review HTA websites Rapid products were retrieved if they were judged to possibly meet the inclusion criteria. One ASERNIP-S researcher independently applied the selection criteria and another researcher was consulted if there was any doubt as to the nature of the HTA rapid product.



Data extraction Data from all included studies were extracted by one researcher and checked by a second using standardised data extraction tables that were developed a priori. Data were only reported if stated in the text, tables, graphs or figures of the rapid product, or could be accurately extrapolated from the data presented. If no data were reported for a particular outcome then this was presented as ‘not reported’.

Data analysis The included products were discussed according to basic characteristics, comparators and outcomes, methodology and essential conclusions. Outcomes, where possible, were grouped according to safety and efficacy and discussed narratively.



3. Results

3.1. Survey of HTA organisations The survey consisted of 17 questions, which have been divided into the following parts:

• Survey response data (questions 1, 2, and 3) • Administration of rapid review process (questions 4, 5, 6, and 7) • Research strategy used for rapid review products (questions 9, and 16) • Composition of rapid review products (questions 8, 10, 11, and 12) • Use of peer review and external experts in rapid review products

(questions 13, 14, and 15) • Comparison of rapid review products with full systematic reviews

(questions 9 – 14, and question 16) • Additional comments (question 17).

Survey response data Survey data are shown in Table 4. Twenty three surveys were returned (46% response); 18 agencies had rapid review products and four did not. One survey was returned blank. The surveys that did not describe rapid review products, and the blank survey, were excluded from the subsequent analyses.

Table 4. Survey response data

Rapid review surveys N Total surveys emailed 50 Total surveys returned 23 Number of agencies who do not conduct rapid reviews 4 Blank survey form 1 Number of agencies who conduct rapid reviews 18 Number of rapid review products 36

Less than 1 month 2 1 – 3 months 17 3 – 6 months 16 1 – 3 & 3 – 6 months 1

Eighteen of the 23 agencies who returned the survey collectively described 36 rapid review products. For the purposes of this analysis, these products were divided into the length of time taken to complete. In particular, they were divided into rapid products that took between one and three months to complete, and rapid products that took between three and six months to complete. Three products did not fit into these categories; one agency indicated that their rapid review product could fit into both categories, and two agencies indicated that their rapid review products took less than one month to complete. Although these products did not fit into the time categories specified, they were included in the results, but were detailed separately.

SECT ION 3 RESULTS 12


The rapid review products of one to three and of three to six months duration have received the greatest focus in this review.

Administration of rapid review process Rationale for conducting rapid review The reasons given by respondents for conducting a rapid review varied. The results were grouped and the five most reported categories are given in Table 5.

Table 5. Reason for conducting a rapid review

Study type

< 1 month 1 - 3 months 3 - 6 months 1 - 3 & 3 - 6 Total, n (%) Reason for review N = 2 N = 17 N = 16 N = 1 N = 36 Political urgency/ to support decisions, n

2 9 4 1 16 (44.4)

Limited time and resources, n 1 4 7 0 12 (33.3) Question raised in previous HTA/ to answer specific question, n

1 4 5 1 11 (30.6)

Clinical urgency, n 0 4 1 1 6 (16.7) Uptake of technology, n 0 2 4 0 6 (16.7) NOTE Respondents were able to give more than one response. The results were grouped and the top five categories are listed.

Collectively, the most commonly stated reason for conducting a rapid review was in response to political urgency and/or to support decisions (44%). Limited time and resources (33%) and to answer a question raised in a previous HTA and/or to answer a specific question (31%) were the next most frequent reasons given for conducting a rapid review. This was then followed by clinical urgency and uptake of technology (17% for both).

In the one to three month, and three to six month categories, the number of responses to this question were similar (n = 23 and n = 21 respectively), but the main reasons for conducting a rapid review differed. For the shorter category, the greatest emphasis was placed on political urgency and/or to support decisions. For the three to six month category, there was more spread of results, with limited time and resources being the main reason given, followed by the other reasons in a downward fashion.

Reasons that respondents gave for conducting rapid reviews which did not fit into the top five collated reasons were:

• Epidemiological aspects • New technologies with potential pressure for incorporation • Rapid review undertaken when policy questions cover multiple areas and

technologies • No analysis needed • Candidate technologies are reviewed annually for consideration for public

funding in accordance with National Health Insurance Law • Too little data for a full review



Authorisation to write rapid reviews There was variation in the specific responses relating to the commissioning of reports. Decision making has therefore been grouped into macro, meso and micro levels of healthcare (van Velden and Severens 2005), as described below.

• Macro level - national or regional regulatory bodies that coordinate activities outside of healthcare organisations, and are responsible for the availability of effective and affordable healthcare programs for the whole population. Examples include governments, health ministries and health authorities.

• Meso level - providers with a local perspective that provide specific programs. Examples include healthcare organisations, hospitals, and community health agencies.

• Micro level - healthcare at the individual level. Examples include medical professionals, clinicians, patients, and individual projects.

The level of decision making of the organisations that commissioned the rapid review products are reported in Table 6.

Table 6. Commissioning of rapid reviews by level of decision-maker

Study type < 1 month 1 - 3 months 3 - 6 months 1 - 3 & 3 - 6 Total, n (%) Level of authority N = 2 N = 17 N = 16 N = 1 N = 36 Macro level, n 3 12 9 1 25 (69.4) Meso level, n 0 7 10 2 17 (47.2) Micro level, n 1 7 6 0 14 (38.9) Total, n 4 26 25 3 NOTE Respondents were able to give more than one response.

When examined collectively, many (69%) respondents indicated that macro decision makers commissioned the rapid reports. Meso decision makers were the commissioners in nearly half (47%) of the cases and micro decision makers in 39%. Although the total number of responses for the one to three and three to six categories were similar (n = 26 and 25 respectively), within the one to three month category nearly half (46%) of the responses indicated commissioning at a macro level. For the three to six month category, the responses were spread more between the three levels, with the highest being 40% at the meso level.

Intended audiences Data relating to the intended audience of the rapid review products varied, and were grouped into macro, meso and micro audiences (as described previously). The intended audiences of the reports are tabulated in Table 7.



Table 7. Level of intended audience of rapid reviews

Study type < 1 month 1 - 3 months 3 - 6 months 1 - 3 & 3 - 6 Total, n (%) Level of audience N = 2 N = 17 N = 16 N = 1 N = 36 Macro level, n 3 10 12 1 26 (72.2) Meso level, n 0 8 3 0 11 (30.6) Micro level, n 2 9 10 0 21 (58.3) Total, n 5 27 25 1 NOTE: Respondents were able to give more than one response.

Overall, the macro level decision makers were most often (72% cases) the intended audience. This was followed by micro level audiences (58%) and then, considerably less, the meso level audiences (31%). Within the one to three month category, there were only small differences between the numbers of responses for the three levels of audiences. For the three to six month category, the majority of the responses fell into the macro and micro levels, with only 12% of responses falling into the meso level.

Staff requirements Respondents were asked how many full time researchers work on each rapid product. The mean number of staff involved (including information resource staff) is described in Table 8. The mean number of full-time equivalent staff for rapid reviews was similar across all the timeframes, with the highest number in the three to six month category.

Table 8. Full time equivalent (FTE) staff assigned to rapid reviews

Study type < 1 month 1 - 3 months 3 - 6 months 1 - 3 & 3 - 6 N = 2 N = 17 N = 16 N = 1

Number FTE (mean) 2.0 2.0 2.6 1.5

Research strategy used for rapid reviews Database searches Respondents were asked whether their products used systematic searches of many databases, systematic searches of restricted databases, hand searching, and grey literature. The search strategy used for the rapid review products is shown in Table 9. Many respondents ticked more than one response.

As a whole, a systematic search of many databases was reported by half of the respondents. A similar number (56%) reported a systematic search of restricted databases. The searching of grey literature was reported by 42% of respondents, and hand searching was reported to occur less often (by 25% of respondents).



Table 9. Search strategy for rapid review products

Study type < 1 month 1 - 3 months 3 - 6 months 1 - 3 & 3 - 6 Total, n (%) Type of databases searched N = 2 N = 17 N = 16 N = 1 N = 36 Systematic search of many databases, n (%)

0 (0) 6 (35.3) 12 (75.0) 0 (0) 18 (50.0)

Systematic search of restricted databases, n (%)

2 (100) 13 (76.5) 4 (25.0) 1 (100) 20 (55.5)

Hand-searching, n (%) 0 (0) 5 (29.4) 4 (25.0) 0 (0) 9 (25.0)

Grey literature, n (%) 1 (50.0) 9 (52.9) 4 (25.0) 1 (100) 15 (41.7) NOTE Respondents were able to tick more than one response.

Within the categories, the length of time of the review tended to indicate the level of literature searching. In the one to three month category, a systematic search of many databases was reported to occur by only 35% of the responses. In the longer category, three quarters of the respondents reported the systematic search of many databases. Hand searching was not often conducted in either category, while the use of grey literature was reported to be used more often by the shorter category of review than the longer category (53% vs. 25%).

Exclusion of studies The responses in relation to whether the rapid review products excluded types of studies are shown in Table 10. No rapid review product excluded systematic reviews.

Table 10. Study types excluded in rapid review products

Study type < 1 month 1 - 3 months 3 - 6 months 1 - 3 & 3 - 6 Total, n (%) Study type N = 2 N = 17 N = 16 N = 1 N = 36 Systematic review, n (%) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) RCT, n (%) 0 (0) 2 (11.8) 0 (0) 0 (0) 2 (5.6) Non-randomised controlled trial, n (%)

0 (0) 2 (11.8) 4 (25.0) 0 (0) 6 (16.7)

Case series, n (%) 0 (0) 7 (41.2) 6 (37.5) 1 (100) 14 (38.9)

Case report, n (%) 1 (50.0) 13 (76.5) 15 (93.8) 1 (100) 30 (83.3) Total responses per category 1 24 25 2 52 NOTE Respondents were able to tick more than one response.

RCTs were reported to be excluded by 6% of respondents, non-randomised controlled trials by 17%, and case series by 39%. The largest proportion (83% of respondents) excluded case reports from their rapid review products. Thus, as the level of evidence decreased, it was more likely to be excluded. When examining within the categories, 77% of respondents in the one to three month category indicated that they excluded case reports in their rapid reviews. This was higher for the three to six month category, at 94%.

Seven agencies indicated in the additional comments section (question 17) that the exclusion of studies was dependent on the availability of published data, and that



lower levels of evidence would be used if there was nothing else available (results not shown).

Composition of rapid reviews The components included in the rapid review products are shown in Table 11. Across all study types, clinical outcomes were included in nearly all rapid review products (94%). Economic factors were included in 78% of products, and quality assessment in 72% of products. Social factors were included less often, in just over half (53%) of the products.

Table 11. Components of rapid review products

Study type < 1 month 1 - 3 months 3 - 6 months 1 - 3 & 3 - 6 Total, n (%) Component of review N = 2 N = 17 N = 16 N = 1 N = 36 Quality assessment, n (%) 1 (50.0) 12 (70.6) 12 (75.0) 1 (100) 26 (72.2) Clinical outcomes, n (%) 2 (100) 15 (88.2) 16 (100) 1 (100) 34 (94.4) Economic factors, n (%) 1 (50.0) 12 (70.6) 14 (87.5) 1 (100) 28 (77.8)

Social issues, n (%) 1 (50.0) 9 (53.0) 8 (50.0) 1 (100) 19 (52.8) NOTE the remaining studies either indicated that they did not conduct a quality assessment, or did not respond to the question

When comparing the one to three month category with the three to six month category, the longer timeframe rapid reviews included more detail than the shorter ones, with the exception of social issues.

Experts and peer review The use of external experts and peer review for rapid reviews is summarised in Table 12.

Table 12. Use of experts and peer review in rapid review products

Study type < 1 month 1 - 3 months 3 - 6 months 1 - 3 & 3 - 6 Total, Use of experts N = 2 N = 17 N = 16 N = 1 N = 36 Use external experts, n (%) 0 (0) 10 (58.8) 14 (87.5) 0 (0) 24 (66.7) Use peer review, n (%) 0 (0) 8 (47.1) 13 (81.3) 1 (100) 22 (61.1) Of those using peer review, number using external experts for peer review, n (%)

NA 5 (62.5) 9 (69.2) 1 (100) 15 (68.2)

Impact of peer review (including external peer review) on reports that use it lengthens time, n (%) NA 7 (87.5) 6 (46.1) 0 (0) 13 (59.1) no impact, n (%) NA 0 (0) 7 (53.8) 1 (100) 8 (36.4) NR NA 1 (12.5) 3 (23.1) 0 (0) 4 (18.2) NOTE the remaining studies indicated that they did not use external review or peer review; NA – not applicable; NR – not reported

In total, more than half (67%) of respondents reported the use of external experts for their rapid review products. Peer review was reported to be used in 61% of rapid products. Of the group using peer review, 68% indicated that they used external peer review; 59% indicated that this lengthened the time the report took to complete, and



36% indicated that the use of external peer review did not impact on the timeframe of the report. It is interesting to note that within the categories, peer review had a much greater impact on timeframes for the one to three month category than the three to six month category, with the timeframes reported to be lengthened in 88% and 46% of cases respectively.

Comparison of rapid review products with full systematic reviews In addition to asking questions related to rapid reviews, questions 9 to 14, and question 16 also asked respondents to answer in relation to full systematic reviews so that a comparison between the two types of reviews could be made. Thirteen agencies (72%) responded to these questions. It was unclear from the surveys whether the five agencies that did not respond to these questions conducted full systematic reviews or not.

The combined total results for rapid reviews reported above have been duplicated to aid with the comparison of full systematic reviews.

Database searches The databases that agencies used to search the literature for systematic reviews are shown in Table 13.

Table 13. Comparison of search strategies for systematic reviews and rapid reviews

Study type Systematic review Rapid review Search strategy N = 13* N = 36 Systematic search of many databases, n (%) 13 (100) 18 (50.0) Systematic search of restricted databases, n (%) 4 (30.8) 20 (55.5) Hand-searching, n (%) 9 (69.2) 9 (25.0)

Grey literature, n (%) 10 (76.9) 15 (41.7) NOTE: Respondents were able to tick more than one response; * 5 agencies did not respond to this question When compared with rapid reviews, systematic reviews were twice as likely to utilise systematic searches of many databases than rapid reviews (100% vs. 50%), were more likely to conduct hand searches (69% vs. 25%), and were more likely to use grey literature (77% vs. 42%). For full systematic reviews, the systematic search of many databases was conducted by all of the agencies, this dropped to 75% for reviews of three to six months in length, and to below 40% for reviews of less than three months in length.

Exclusion of studies



Table 14 details the types of studies excluded for use in systematic reviews. When compared with systematic reviews, rapid reviews had a higher rate of excluding RCTs (6% vs. 0%), non-randomised controlled trials (17% vs. 8%), case series (39% vs. 31%), and case reports (83% vs. 77%).

Table 14. Exclusion of study types in systematic reviews compared to rapid reviews

Study type Systematic review Rapid review Excluded study type N = 13* N = 36 Systematic review, n (%) 1 (7.7) 0 (0) RCT, n (%) 0 (0) 2 (5.6) Non-randomised controlled trial, n (%) 1 (7.7) 6 (16.7)

Case series, n (%) 4 (30.8) 14 (38.9) Case report, n (%) 10 (76.9) 30 (83.3) NOTE: Respondents were able to tick more than one response; * 5 agencies did not respond to this question

Components of systematic reviews Table 15 details the composition of systematic reviews reported by the agencies that responded to the survey. All agencies indicated that clinical outcomes were part of their systematic reviews. Economic factors and social issues also featured prominently (92%, and 85% respectively). The survey did not ask agencies whether the systematic reviews utilised quality assessment or critical appraisal of the included studies.

When compared with rapid reviews, systematic reviews were more likely to have clinical outcomes (100% vs. 94%), and examine economic factors (92 vs. 78%), or social issues (85% vs. 53%). When comparing systematic reviews with rapid reviews of different durations, it was found that full systematic reviews were the most comprehensive review type (assuming the inclusion of quality assessment), followed by reviews of three to six months duration and, reviews of one to three months duration (Table 11 and 15).

Table 15. Reported components of systematic reviews compared to rapid reviews

Study type Systematic review Rapid review Component of review N = 13* N = 36 Quality assessment, n (%) NA 26 (72.2) Clinical outcomes, n (%) 13 (100) 34 (94.4) Economic factors, n (%) 12 (92.3) 28 (77.8)

Social issues, n (%) 11 (84.6) 19 (52.8) NOTE: NA- not applicable. Question was not included in survey; * 5 agencies did not respond to this question

Use of external experts and peer review The use of external experts and peer review in full systematic reviews and rapid reviews is shown in Table 16. All agencies that responded to this question in relation



to full systematic reviews used external experts, and most (85%) used peer review. Of the 11 agencies that used peer review, 10 (91%) of these used external peer review.

Table 16. Use of external experts and peer review in systematic reviews compared to rapid reviews

Study type Systematic review Rapid review Use of experts N = 13* N = 36 Use external experts, n (%) 13 (100) 24 (66.7) Use peer review, n (%) 11 (84.6) 22 (61.1)

Of these, how many use external experts for peer review, n (%)

10 (90.9) 15 (68.2)

NOTE:* 5 agencies did not respond to this question

Compared with systematic reviews, rapid reviews were less likely to use external experts than full systematic reviews (67% vs. 100%), and were also less likely to use peer review (61% vs. 85%). Of the groups that used external experts, rapid reviews were less likely to use external experts for peer review than full systematic reviews (68% vs. 91%).

The duration of a review determined the likelihood that it used external experts, peer review, and external peer review. As the duration of the reviews increased, the likelihood that these factors were incorporated also increased, with reports of one to three months having the lowest likelihood (external experts 59%; peer review 47%) and full systematic reviews the highest likelihood (external experts 100%; peer review 85%) (Table 12 and Table 16).



3.2. Literature on rapid review methodology A total of 11 relevant studies were identified: one guideline abstract; three program evaluations; one comparative study; two methods studies; three commentaries and one survey (Figure 1 and Table 17).

Figure 1. Process for the selection of studies retrieved from the literature database

Duplicate references removed: 238 references deleted

Remaining 2522 studies evaluated on title: 2421 articles deleted as not pertaining to the review

Remaining 101 references evaluated on abstract: 60 references determined not to meet inclusion criteria & excluded after evaluation of abstract (Appendix C)

Full text of remaining 41 references evaluated: 30 references determined not to meet inclusion criteria

& excluded after evaluation of full text (Appendix C)

References discussed in the final review: 11 references determined to meet inclusion criteria

Foreign language articles removed: 375 references deleted

Potentially relevant citations identified as a result of electronic and internet searches:

3135 references identified



There was a paucity of information identified in the literature regarding methodology pertaining to the undertaking of rapid reviews of health technologies. Studies identified relevant to this aspect of rapid reviews are listed in Table 17.

Table 17. Rapid review methodology: included studies

Authors Study type Subject Aidelsburger et al. 2002 Guideline (abstract) Recommendations for undertaking rapid health-

economic HTA’s Best et al. 1997 Program evaluation Commentary on development of HTA program in

the United Kingdom Corabian & Harstall 2002 Comparative study (abstract) Comparison of rapid versus full assessment with

respect to outcomes Egger et al. 2003 Methods study Evaluation of necessity of comprehensive

literature searches Eisenberg & Zarin 2002 Commentary Stakeholders and process of HTA in United

States Hailey et al. 2000 Program evaluation Commentary on the use and impact of rapid

HTAs undertaken by AHFMR Hailey 2006 Commentary Methods and purpose of HTA Lehoux et al. 2004 Survey HTA products in Canada McGregor & Brophy 2005 Program evaluation Creation of local HTA unit Royle & Milne 2003 Methods study Effective methods of database searching to

identify randomised controlled trials Stevens et al. 1999 Commentary Methods to evaluate rapidly emerging

technologies

A number of authors acknowledge the complexities inherent in conducting rapid reviews, suggesting that it is difficult to accomplish the triad of responsiveness to short timeframes, scientific rigor and transparency in a manner consistently acceptable to all stakeholders (Eisenberg 2002).

However, despite these challenges, some HTA organisations produce a considerable number of rapid reviews. An overview of the Canadian HTA products produced between 1995 and 2001 (Lehoux 2004) indicated that 17% of the total HTA output was ‘short documents’, indicating the increasing importance placed upon responsiveness to short timeframes. Furthermore, the increased opportunity presented by rapid reviews to contextualise them within specific healthcare settings is an acknowledged strength of this style of product (McGregor & Brophy 2005; Hailey 2000).

Other authors urge caution in the production of rapid assessments, while acknowledging the influence that they can have in the context of policy making. Rapid review has been proposed as an important intermediary step in the assessment of emerging technologies, to be followed by a more comprehensive assessment (Stevens 1999). However, detailed HTAs, undertaken over months or even years, will remain essential in order to address certain sorts of questions (Hailey 2006). A comparative study exploring the differences in the complexity and findings of rapid and full reviews (Scott & Harstall 2003; identified via personal communication)



evaluated the minimum information response required to accurately answer a specific policy question. It was found that the extra clinical information included in a full assessment may be extremely valuable to clinicians, but is of far less importance to the healthcare decision maker, highlighting the need to have a thorough understanding of the requirements of the intended audience before undertaking a rapid review.

Methodological alterations that may distinguish a rapid review from a full HTA as discussed generally in published literature include the development of a limited research question and truncated literature searching.

Aidelsburger (2002) proposed that a rapid HTA should focus on the study question by exactly defining the technology, outcomes and study population to be examined, thereby facilitating an accurate literature search and reducing the number of identified studies. Furthermore, in an examination of the use and impact of rapid HTAs (Hailey 2000), it was stated that the rapid reviews under consideration provided somewhat restricted advice, given that they were generally confined to addressing questions of efficacy or effectiveness. This contrasts with the commonly broader research questions addressed by full HTAs, where issues of efficacy/effectiveness are frequently considered in conjunction with those of safety, economic viability, legality and ethics.

Rapid reviews may also utilise a different search strategy to those employed in full systematic reviews, by reducing the number of databases searched, not searching grey literature or restricting hand-searching. The potential impact that this may have on the validity of the review findings has been assessed by two groups. After examining the indexing characteristics of a variety of clinical trials, Egger (2003) concluded that “systematic reviews that are based on a search of English language literature that is accessible in the major bibliographic databases will often produce results that are close to those obtained from reviews based on more comprehensive searches that are free from language restrictions”. The Cochrane Controlled Trials Register (CCTR) was found to be the single best source of randomised controlled trial references, with additional database searching retrieving only a small percentage of extra trials (Royle & Milne 2003). Best (1997) also acknowledged that exhaustive data collection may have little effect on the final recommendation, although did not provide any empirical evidence to support this claim.

Only one study could be identified that evaluated the validity and reliability of rapid reviews compared to more extensive follow-up reports (Corabian & Harstell 2002). In five of six cases, the conclusions reached by the rapid review product (‘Technotes’) was confirmed by later peer-reviewed reports. The only disagreement in conclusions was a Technote concluding that an intervention was experimental, while a larger cost-effectiveness study indicated the intervention to be reported as safe and efficacious in the literature.



3.3. Identification and comparison of rapid reviews and full systematic reviews

Scoping search for rapid reviews The intention of this scoping was to assemble a number of reviews written on the same topic by different agencies; some being more rapid, others more comprehensive. For each included topic, an attempt was made to have a standard comparator (a full review), that is, a comprehensive systematic review, or the systematic review component of a HTA written on that topic, with approximately the same publication dates. In line with the inclusion/exclusion criteria, any comparator not published within a year of the rapid review product was excluded.

A manual search of HTA agency websites (Appendix D), mostly by following publication links or carrying out a website search, identified eight agencies that produced rapid products (Table 18). Excluded agencies are listed in Appendix E.

Initially, the Institute of Health Economics / Alberta Heritage Foundation for Medical Research (IHE/AHFMR) website was searched by following publication links and nine products were found. Of these, ‘Technotes’ appeared to fit the inclusion criteria for a rapid review product.

The only product produced by the Australian Safety and Efficacy Register of New Interventional Procedures, Surgical (ASERNIP-S) that was considered to fit the definition of a rapid product was the ‘Accelerated Systematic Review’.

The Canadian Agency for Drugs and Technologies in Health (CADTH, formerly the Canadian Coordinating Office for Health Technology Assessment, CCOHTA) website was searched. A table demonstrating the differences between various products was found. Initially, ‘Issues in Emerging Technologies’ appeared to fit the inclusion criteria; however, following correspondence with the agency it was found that it was not a rapid product. The Agency identified two rapid review products in the HTA report series that originate from their Health Technology Inquiry Service (THIS). However, since these products were first published in 2006, they could not be included.

The Monash Centre for Clinical Excellence (CCE) website was searched and ‘Evidence Reports’ (formerly ‘Critical Appraisals’) appeared to fit the definition of a rapid product.

Search of the Institute of Clinical Systems Improvement (ICSI) website revealed a product called ‘Technology Assessments’ that appeared to fit the inclusion criteria. Correspondence with the agency confirmed this, and information was obtained concerning the timeframe for the review and whether an expert panel was used.

The McGill University Health Center (MUHC) website contained one specific product called ‘Reports’. This product appeared to meet the definition of a rapid product. Correspondence with the agency revealed that it considered this report to



contain elements of a full systematic review (such as policy recommendations), and is completed within three months. It was therefore included as a rapid review product.

Three products were identified on the National Institute for Clinical Excellence (NICE) website. ‘NICE Overviews’ from the Interventional Procedures Programmes (IPP) were confirmed through correspondence with NICE as being a rapid review product. The product included the relevant components of a rapid review (i.e. searches, critical appraisal, summary of evidence, and appropriate time frame).

A search of the New Zealand Health Technology Assessment (NZHTA) website identified eight products. Correspondence with NZHTA indicated that ‘Technical Briefs’ fitted the criteria of a rapid review product.

Following searching the Blue Cross Blue Shield (BCBS) website ‘Technology Evaluation Centre Assessments’ were identified to fit the criteria of a rapid review product. As the BCBS website contained only reviews published since 2004, only reviews published in this year were included.

Table 18. Summary of HTA website search results for rapid review products

Agency HTA products identified AHFMRM HTA initiatives

HTA series A HTA series B Information papers International Reports Presentation Report of activities Technotes* Techwise newsletter

ASERNIP-S General information and reports Systematic Reviews Accelerated systematic reviews Technology overviews Clinical practice guidelines General guidelines for assessing, approving and introducing new procedures into a hospital or health service Other publications Peer reviewed publications

CCE Literature Searches Critical Appraisals/ Evidence Reports* Full Reports

ICSI USA Technology Assessments* MUHC Canada Reports* NICE UK Systematic Review

Overview* Guidance

Continued over page…



Table 18. Summary of HTA website search results for rapid review products continued

NZHTA Systematic Reviews Medical Services Advisory Committee NZHTA Technical Briefs* NZHTA Evidence Tables NZHTA Health Services Evaluation Research NZHTA Health Services Research Annotated Information Packages Horizon Scanning Network Reports

BCBS US (post 2004 included) Technology Evaluation Centre Assessments*

NOTE: Product names in bold denote rapid review products; * Agency was contacted to confirm the product was a rapid product

HTA agency website search for rapid products Eight rapid products were identified on the following topics (details of topics identified for possible inclusion are listed in Appendix F):

• Drug eluting stents • Lung volume reduction surgery • Living donor liver transplantation • Hip resurfacing

Full extraction tables for each topic are included in Appendix G.

Drug eluting stents Coronary artery disease (CAD) is the largest single cause of death and most common cause of sudden death in Australia, claiming 24,576 lives in 2004 (Australian Institute of Health and Welfare 2006). Treatment protocols for this disease include medical management, percutaneous transluminal coronary angioplasty (PTCA) with or without stenting and surgical treatment with coronary artery bypass grafting (CABG), depending on the severity of the disease.

It has been suggested that drug-eluting stents (DES) used in conjunction with PTCA may reduce the magnitude of restenosis and improve the effectiveness of the intervention. These stents are coated with a polymer that acts as a drug reservoir, which allows controlled local release of a drug directly at the site of the injured endothelium, thus avoiding systemic side effects. Two of the key pharmacologic agents used in drug eluting stents for the treatment of CHD are paclitaxel and sirolimus, both of which work to prevent cell proliferation.

Two rapid review products (Product A and Product B) and a full review (Product C) were examined in this assessment. In addition to the comparison between bare-metal stents and DES, Product C also included data comparing PTCA or CABG to bare-metal stents which will not be considered further in this assessment of the products.



Table 19. Basic characteristics of review products for DES

Date published

Literature search dates

Size of document

Number of authors

External review

Product A (rapid review)

Dec 2003 NR 14 pages 1 Yes

Product B (rapid review)

Jul 2003 NR 33 pages 1 2 external reviewers TAU Committee

Product C (full HTA)

Feb 2003 Dec 2004 257 pages 11 3 expert reviewers

* TAU= Technology Assessment Unit; NR=not reported

Each of the three products compared DES to bare metal stents, with Products A and B also comparing high-dose DES to low-dose DES. Outcomes assessment varied widely among the products with both narrative description and meta-analysis employed (Table 20). Products A and B both examined the outcomes of stents eluting different pharmaceutical agents separately, while Product C did not attempt this comparison. Product A made no attempt to synthesise any novel results, simply reporting a narrative summary of the results of each key trial included. Product B included a narrative summary of each of the trials along with a summary compiled by the authors and a meta-analysis. Product C focussed entirely on meta-analysis. Product C had removed a variety of commercial-in-confidence (CIC) information from the outcomes analysis, making it difficult to fully evaluate the depth of the analysis.

Product A did not fully address economic considerations, with only a description of the prices of various stents. Product B reported considerable cost effectiveness data, with decision analysis and sensitivity analysis calculations shown. Product C presented an extremely detailed cost effectiveness review (of 14 pages), along with a critical appraisal of the economic models submitted by device manufacturers, and independent analysis and a budget impact analysis.



Table 20. Comparators and outcomes for DES

Intervention Comparator Outcomes reported

DES (sirolimus) Bare metal stents DES (paclitaxel) Bare metal stents High-dose DES (paclitaxel) Bare metal stents Low-dose DES (paclitaxel) Bare metal stents


Slow release DES Fast release DES

Narrative summary of each included trial. Efficacy: ISR, TLR, stent thrombosis Safety: MACE

DES (sirolimus) Bare metal stents Fast release DES (sirolimus) Slow release DES (sirolimus) DES (sirolimus) - DES (paclitaxel) Bare metal stents DES (paclitaxel, 4 doses) Bare metal stents Moderate/slow release DES (paclitaxel)

Bare metal stents

DES (QP2) Bare metal stents


QP2 -

Narrative summary of each included trial: Efficacy: ISR, TLR, stent thrombosis Safety: MACE

Summary of safety/efficacy evidence 3 meta-analyses*: 6 month revascularisation

Cost-effectiveness Product C (full HTA)

DES (paclitaxel, QP2, sirolimus, everolimus, actinomycin)

Bare metal stents 12 meta-analyses†: Event rate Mortality AMI Binary restenosis

Cost-effectiveness NOTES: MACE – major adverse cardiac events (composite endpoint); ISR – in-stent restenosis; TLR – target lesion revascularisation; AMI – acute myocardial infarction; QP2 – 7 hexanolytaxol; * sirolimus, paclitaxel and all stents revascularisation rates; † all stents considered together, 4 outcomes analysed three times each – at up to 36 days, 6 months and 12 months

The search methodology employed was not described by either Product A or B. Product C reported an extensive description of the search strategy, inclusion/exclusion criteria and critical appraisal (Table 21).

There was substantial variation in the evidence base of each review (Table 21). Product A appeared to include only published randomised comparative trials (RCTs), which may be the reason for the smaller evidence base. However, the rationale behind the inclusion of an additional two observational studies was not clearly stated in the methodology. While the search strategy was not stated explicitly in Product B, the review included a number of abstracts and unpublished RCTs, in addition to a number of non-randomised studies. Again, without explicit methodology, it was difficult to determine if this evidence base resulted from an exhaustive search, or was compiled from other sources. Product C limited the review to considering only RCTs, both published and unpublished, found through database searches, handsearching, conference proceedings and contact with trial coordinators. The potential for bias/inaccurate data was clearly addressed in the detailed critical appraisal which was undertaken.



Table 21. Methodology of review products for DES

Systematic search Critical appraisal of evidence

RCTs included Other evidence included


Databases: NR Search terms: NR Date limits: NR Restrictions: NR

Evidence grading undertaken. No summary of quality of overall evidence base.

Total: 4 Paclitaxel: 2 Sirolimus: 2

Non-randomised trials/case series: 2 Siroluimus: 2


Databases: NR Search terms: NR Date limits: NR Restrictions: NR

NR Total: 9 Paclitaxel: 6 Sirolimus: 3

(7 unpublished / 9 abstracts)

Non-randomised trials/case series: 7 Paclitaxel: 1 Sirolimus: 3 QP2: 3


Databases: MEDLINE EMBASE Science Citation Index CCTR CDSR HTA DARE Search terms: explicit Date limits: 1990-2002 Restrictions: English

Explicit and detailed. Authors suggest caution in results interpretation due to poor quality of evidence.

Total: 12 Paclitaxel: 9 Sirolimus: 3 QP2: 1 Everolimus: 1 Actinomycin: 1

(10 unpublished/ 10 abstracts)

Review inclusion criteria limited to RCT evidence

NOTES: NR - not reported

The general conclusions of the reviews were superficially similar, but varied in depth and focus. All of the reviews stated that there was no evidence of the safety or efficacy of DES beyond 12 months (Table 22).

Product A specified efficacy outcomes for which DES have shown benefit, but did not address specific safety outcomes. Additionally, this product very clearly defined the sub-groups of patients in which benefit had been shown.

Product B had a significant focus on pragmatic economic distribution of this technology, revealed by the essential conclusions of the review. Efficacy outcomes enumerated through meta-analysis were reiterated. There was no essential safety conclusion pertaining to the use of DES. The authors also acknowledged the rapidly evolving nature of this technology – “The evidence on which this policy recommendation is based is likely to be very time sensitive. The decision should be frequently reviewed and modified of necessary in the light of such evidence”.

The conclusions of Product C were reflective of the more detailed assessment of the evidence that was undertaken in the review process. While the reduction in restenosis rates with the use of DES is acknowledged, the authors placed a caveat on interpretation of these results, as they found clinical definitions to vary widely across the included trials. Product C was also cautious in interpreting any safety data, given the poor quality evidence available. The review team “stress that these results are



interim and incomplete, and… await definitive publication of studies confirming patient numbers and outcomes”.

Table 22. Essential conclusions of review products for DES

Safety Efficacy Essential conclusions


Cited FDA adverse event reports. Long term effects of DES are unknown at this time; short term side effects have been similar to bare metal stents.

DES significantly decreased the rate of restenosis, repeat revascularisation procedures and cardiac events up to one year post treatment in specific sub-groups.

No evidence of safety and efficacy of DES in other patient groups* Long term safety and efficacy of DES are unknown.


NR Repeat revascularisation rates may be reduced by 65% with DES (83% with sirolimus). No reduction in mortality, AMI or bypass surgery rates demonstrated with the use of DES.

Long term safety and efficacy are unknown. “Despite good evidence supporting the efficacy of DES to reduce the rate of restenosis, the current budget of the hospital should not be redistributed to permit the routine acquisition of DES”


The available data do not allow for any conclusions to be made with regard to the effect of DES on mortality or in the case of AMI.

Results indicate that DES decrease rates of restenosis and therefore revascularisation following placement. The exact rate of lowering of revascularisation seems to be by approximately 60-70% at 12 months, but there are difficulties in definitions of how many of these were clinically driven.

“Data are limited by the lack of reporting of longer term outcomes” “…a substantially higher cost with a very small benefit outcome, so that drug-eluting stents would not normally be considered a cost-effective alternative”

NOTES: AMI acute myocardial infarction; * other than de novo or restenotic lesions; NR – not reported

In an area of such nascent and rapidly evolving technology, even the few months separating publication of each of these reviews may have influenced the evidence base available to each agency. However, without explicit details of the methodology pertaining to each review, the impact of this is impossible to assess.

Furthermore, each review had a slightly different focus, possibly due to variations in the source of request for the review, which influenced the outcomes reporting particularly in regard to economic evaluation. However, despite this, each review reached essentially the same conclusions in regard to the safety and efficacy of the DES. The economic analyses are also important to consider, given the substantial potential cost impact of this technology. All three reviews investigated costs to some extent; however Product C analysed costs in a much more complete and detailed manner than either of the two rapid reports.



Lung volume reduction surgery Chronic lung disease affects a large proportion of the older population. It has a major impact on community health resources, is responsible for a high rate of hospital admissions and contributes significantly to rising mortality rates due to chronic obstructive pulmonary disease (COPD), especially in women (Rowett 2005). Emphysema, a subtype of COPD, is characterised by the destruction of alveolar walls and hyperinflation of the lungs, which results in shortness of breath (dyspnoea), decreased exercise capacity and reduced quality of life (QOL) (Beauchamp 2000; Benditt & Albert 1997; Celli 2000).

There are a variety of interventions for chronic lung disease ranging from rehabilitation programs (involving exercise and education), oxygen therapy and, in some cases, surgery. Lung volume reduction surgery (LVRS) is a palliative treatment that aims to remove the least functional part of the lungs in order to improve airflow, diaphragm and chest wall mechanics and alveolar gas exchange in the remaining portion of the lung (Cooper et al 1995).

Three products were identified examining LVRS. Product A was identified by a HTA website search for ‘rapid reviews’. Following this, a York CRD search was conducted, using the search term “lung volume reduction surgery”, and Products B and C were identified. Basic characteristics of these products are listed in Table 23.

Table 23. Basic characteristics of review products for LVRS

Date published

Date of literature search

Size of document

Number of authors

Expert review panel


Sept 2004 May 2004 19 pages NR Yes Society of Cardiothoracic Surgeons of Great Britain and Ireland and the British Thoracic Society


Oct 2003 1997-2003 9 pages (Revised 1997 report is 16 pages)

3 Yes Technology Assessment Committee


Dec 2004 Mar 2004 72 pages 7 Yes 3 external reviewers 2 scientific advisory panel members

NOTE: NR – not reported

The treatment intervention was consistent between all three products, as was the comparator (medical management). Therefore, as far as can be deduced, all three reviews were addressing the same research question. This is detailed in Table 24.



Table 24. Comparators and outcomes for LVRS

Intervention Comparator Outcomes Product A (rapid review)

LVRS Other medical intervention 19 outcomes discussed narratively


LVRS Medical therapy 6 outcomes discussed narratively according to study


LVRS Medical management 5 outcomes summarised according to type with meta-analysis: QOL, dyspnoea, pulmonary function, exercise tolerance, blood gases

NOTE: LVRS – lung volume reduction surgery, QOL – quality of life

The outcomes of the products were analysed and reported in different ways. For Product A and Product B, outcomes were discussed narratively for each included study and there was no collective summary of these outcomes. Product C computed summary estimates (weighted mean difference (WMD) or relative risk (RR)) using a random effects model. For some of these outcomes (e.g. QOL), more than one outcome measure was analysed (Table 25).

Table 25. Methodology of review products for LVRS




Searched MEDLINE, PREMEDLINE, EMBASE, Cochrane Library Science Citation Index Date limits and search terms not reported

Brief - Reported variability in validity between studies and many of the studies were found to be of a poor quality.

4 1 systematic review 1 case series


NR No 3 3 case series 1 cost effectiveness analysis


Searched MEDLINE, EMBASE, BIOSIS Previews, PASCAL, Current Contents Search From 1992 onwards Described search terms

Yes Strategy for quality assessment detailed but actual quality assessment not reported

7 59 case series

NOTE: NR - not reported

Product A and C detailed the databases that they used for their respective searches and carried out a critical appraisal of the evidence, although this was brief for Product A. Product A found many of the studies to be of poor quality. Product B did not carry out a critical appraisal and Product C detailed the method of critical appraisal (using the Jahad scoring system), but not the results of this quality rating system. Despite this, Product C seemed to be much more comprehensive, which was demonstrated by the difference in the number of included studies (seven RCTs



compared to four and three in the other two products). It seems that this difference can be explained by the fact that two of the included studies in Product C were unpublished RCTs that are not included in A and B. Furthermore, an early RCT (Criner et al. (1999)) was not included in Products A or B. Both these reviews may have excluded this RCT due to its year of publication, but this is uncertain.

Product A included one systematic review, which was not reported in the other products. The reason for exclusion from the two other products may have been because the systematic review was based on non-comparative data and was published in 1999.

Product B did not specify the inclusion criteria for studies. Therefore, the reasons for the inclusion of no more than three case series in Product B are unclear. Product A stated that only good quality comparative studies were included, whereas Product C reported a more open inclusion criteria, including case series of greater than 20 patients. This is one reason why the evidence base for this review is greater than that of the other two.

In terms of safety, there are a variety of complications that could arise from LVRS and all three products seemed to report on approximately the same number and type of complications. All three products discussed efficacy/effectiveness narratively. Product C conducted a meta-analysis of five outcomes (death, dyspnoea, lung volume, pulmonary function and exercise capacity).

The general conclusions of the three reviews were similar, each suggesting that further research, particularly RCTs, are required. Products B and C both suggest that the overall results should not be attributed to the general population, but that the procedure may be more effective in a subgroup of patients. Each study seemed to describe the safety, in terms of complications and morbidities, in reasonable detail; however, Product C highlighted the need for more data on the risks of LVRS in greater detail.



Table 26. Essential conclusions of review products for LVRS

Safety Effectiveness Essential Conclusions


Discusses complications such as persistent air leak etc. Complications include those that may arise from already present co-morbidities

Outcomes improve in short term Specialist advisors considered that, with proper selection, efficacy is well established

Overall LVRS does not appear to have any effect on long-term survival


Reports on morbidity and mortality

Discussed narratively in each study

LVRS effective in improving both objective and subjective short-term measures of functional ability. Results cannot be extended to general population of those with emphysema Long term follow up data unknown Wide variation in literature to date making generalisations difficult- need national collaborative study.


Complications and mortality discussed

Outcomes improved after LVRS

Reports “functional benefits of LVRS come at price of increased short-term mortality” Data on risks associated with LVRS poorly documented RCTs needed to confirm group of patients who would gain most benefit RCTs needed to further support safety issues and provide evidence that will help to prevent the occurrence of adverse events

Product A produced no overall conclusion for the safety and effectiveness for LVRS, other than that it does not appear to have any effect on long-term survival. Product B concluded that LVRS was effective in the short term but unclear in the long term. Product C focused additionally on the risks/safety as the extra case series data gave a great deal of safety information, with particular emphasis on long-term safety.

Living donor liver transplantation The liver is the largest internal organ of the human body, and is responsible for a number of functions including the processing of fats, carbohydrates and amino acids, the production of essential proteins such as enzymes and clotting factors, immunological defence and the breakdown of potentially toxic substances such as alcohol. End-stage liver disease (ESLD) occurs when functional liver cell mass falls below a critical level (Said et al. 2007). Liver transplantation may be the only chance of survival for patients with end-stage liver disease, but the lack of cadaveric livers continues to be an increasing and pressing issue. Live donor liver transplantation (LDLT), where up to 70% of the donor’s liver can be transplanted, is one alternative which provides similar recipient outcomes to cadaver liver transplantation. However, live liver donation, particularly the right lobe of the liver, constitutes major surgery which exposes donors to morbidity and mortality risks (Middleton et al. 2004).



Therefore, the safety and efficacy explored in the HTA products, investigated here, is of utmost importance.

Basic characteristics are described in Table 27. Although there is a difference in time between the publication dates of both products, the literature search date was the same for each product (January 2004).

Table 27. Basic characteristics of review products for LDLT

Date published

Literature search dates

Size of document

Number of authors

Expert review panel


Mar 2004 Jan 2004 22 pages 1 NR

Product B (full HTA)

Oct 2004 Jan 2004 673 pages 9 (2 Reviewers) Expert Panel

The treatment intervention discussed was adult-to-adult living donor liver transplantation for Product A and any surgical technique transplanting a liver from a liver adult donor to an adult or child recipient for Product B. Product B was more comprehensive as it also examined a broader topic by dividing the review into donor and recipient outcomes.



Table 28. Comparators and outcomes for LDLT

Intervention Comparator Outcomes


Living donor liver transplantation

Cadaveric transplant Recipient: Hepatic complications Biliary complications Hospital stay Patient survival Graft survival Waiting time

Donor: Hospital Stay Recovery time Mortality rate Morbidity QOL


Living donor liver transplantation to adult or child

Cadaveric transplant Recipient : Survival (short/ long term) Graft-related complications Graft rejection or dysfunction Graft survival (short/ long term) Other complications (short/ long term) Regeneration time Time in hospital Time to return to normal functioning Quality of life Psychosocial outcomes

Donor: Death Complications (short/ long term) Need for blood transfusion Regeneration time Time in hospital Time to return to normal functioning Quality of life Psychosocial outcomes

Other: Costs, resource utilisation

Although both reported donor and recipient outcomes, Products A and B differed in how they reported these outcomes. Product B tabulated and discussed outcomes in the results, whereas Product A discussed the outcomes narratively. There seemed to be some common outcomes discussed in each of the products (for example mortality and other complications such as hepatic and biliary complications). The variability in which both reports discussed outcomes was probably related to the variability of outcomes reported in the included primary studies. Data on included studies is shown in Table 29.



Table 29. Methodology of review products for LDLT




Searched Cochrane Library CINAHL, EBM Reviews- ACP Journal Club EMBASE HealthSTAR PubMed Science Citation Index NHS CRD (UK) HTA Agencies Date limits and search terms included

No 0 Donor: 1 systematic

review Recipient: 3 comparative

studies Financial: 1 comparative

study


Searched Medline, Premedline, Embase, Australasian medical index, current contents, Cochrane library Conference proceedings Date limits and search terms were included

No 0 Donor: 79 comparative

studies 330 case series

Recipient: 89 comparative

studies 149 case series

The literature searches of both Product A and B included a range of databases. The methodology of both products reported date limits and search terms in some depth. No Level II evidence (RCTs) was identified by either review. Neither review undertook a critical appraisal of the identified literature. Product A included a systematic review published in 2002 which reported on donor outcomes (surgical case series and survey results from a number of countries). Product A reported that the results from this systematic review were outdated and therefore this may be why it was not included in Product B. In addition to this, it seemed that all of the studies included in this systematic review were also included individually in Product B.

Both studies discussed ethical considerations and economics in similar detail. The products examined one cost-effectiveness study each and despite this not being the same study, found similar results.



Table 30. Essential conclusions of review products for LDLT

Safety Efficacy Essential Conclusions Product A (rapid review)

Discusses complications such as postoperative biliary and hepatic complications; mortality; graft survival for recipient and recovery time, morbidity and quality of life for donors.

Discussed narratively- Recipients: Better overall condition compared to cadaveric graft recipients Waiting list- 140 days for a liver graft compared to 406 days for cadaveric Donors: Reports time lost- average 13 weeks to recover Long term outcomes unknown Costs: Donor evaluation Right hepatectomy Donor follow-up

• Evidence base for LDLT is incomplete

• LDLT is still undergoing active development, so safety and efficacy relative to CLT and impact on waiting lists whilst seemingly efficacious are still relatively unknown


Discusses complications such as infection, hepatic and multiple organ failure, recurrent disease

Discussed narratively Hospital stay varied from study to study Waiting time significantly shorter in LDLT than CLT No significant differences in ICU stay, regeneration and liver function

• LDLT not shown to be superior to CLT

• Is one method of increasing organ availability with small but real risks

• Most literature grade IV evidence (case-series) therefore much concern over comprehensiveness and reliability of published information and concerns about donor safety

• Require higher level of evidence, especially RCTs and systematic reviews, and need to focus on long term data

NOTES: LDLT – living donor liver transplantation; CLT – cadaveric liver transplantation; ICU – intensive care unit

The overall conclusions of the reviews were similar; however, Product B provided greater detail. Each of the reviews stated that there was a need for more long term results for both safety and efficacy.

Product A stated that the evidence base for LDLT was incomplete but undergoing active development with more research needed for both safety and efficacy. Due to the nature of this topic it is accepted that this does not necessarily have to be in the form of RCTs.

Product B appears to be much clearer in its safety and efficacy outcomes and thus essential conclusions, detailing concerns about safety, especially those for the donor in the long term.

It is important to recognise that the more detailed conclusions reported from Product B was a result of a much more comprehensive review that included a much broader range of literature for both recipient and donor outcomes. Despite this, LVLT continues to be a complex topic which involves quite complex ethical issues and requires considerably more research.



Hip Resurfacing Arthritis and related disorders represent the highest number of people with a disability associated with chronic diseases and conditions, with approximately 550,000 cases in Australia per year; almost 6 times that for asthma, stroke or depression (Australian Institute of Health and Welfare (AIHW) 2005). Osteoarthritis is the primary cause for disease affecting the hip joint but other conditions, such as rheumatoid arthritis, avascular necrosis, congenital dislocation, Paget’s disease, ankylosing spondylitis and traumatic arthritis, also play a role (Vale et al. 2002). These conditions are collectively referred to as degenerative hip disease.

Degenerative hip disease causes functional and symptomatic problems (such as progressively intense chronic pain and stiffness) and people suffering from it may have difficulty carrying out normal daily activities and may lose their independence (AHFMR 2002). Palliative treatment, such as modification of activity, analgesic and anti-inflammatory medication, exercise and weight reduction, may often deal with the pain and reduced joint mobility, and prevent further damage. When conservative options have failed, surgery may be an appropriate measure.

Total hip replacement, the most common surgical intervention for degenerative hip disease, has relatively good outcomes for the elderly but has problems for younger, more active adults including short device survival, device dislocation and loosening. It has been found that 15-20% of young, active patients require revision surgery within 15 years, whereas less than 5% of older adults need such surgery at 10 years (AHFMR 2002). To address these issues of total hip replacement in younger adults, alternative forms of surgery are often pursued, one of which is hip resurfacing.

Two rapid review products (Products A and B) and a full HTA (Product C) were identified on hip resurfacing. In addition to comparing with total hip replacement, Product C also utilised watchful waiting, osteotomy, arthrodesis and arthroscopy as comparators. For the purposes of this assessment, only the comparison to total hip replacement was examined.

Table 31. Basic characteristics of review products for hip resurfacing

Date published

Literature search date

Size of document

Number of authors

Expert review


Mar 2002 Feb 2002 16 pages NR No


Nov 2002 Oct 2002 9 pages 1 NR


Jun 2002 2001 117 pages 6 Yes- 4 panels

NOTES: NR – not reported.

The three products included in this review were of varying scope. Product A was a summary of five previous HTA reports, whereas Product C was a more comprehensive HTA which included one RCT, a number of observational studies and an industry-submitted report. Product B was a brief summary of two previously



published HTAs, namely Product A and Product C, and as a result, its inclusion criteria were quite brief. Neither Products A or B involved expert advice or peer review, while Product C included advice from four panels.

Table 32. Comparators and outcomes for hip resurfacing

Intervention Comparator Outcomes reported Product A (rapid review)

Hip Resurfacing (Birmingham, Conserve Plus, Buechel- Pappas, Cormet 2000)

Total hip replacement Safety outcomes: Post-operative

complication rate Translocation of metal

ions/risk for cancer 6 efficacy outcomes: Revisions Length of hospital stay Recovery time Return to work time Capability to perform

daily tasks Activity level


Birmingham Hip Resurfacing Total hip replacement Efficacy: Improved mobility Pain control


Metal-on-metal hip resurfacing arthroplasty (Birmingham Hip resurfacing)

Total hip replacement Watchful waiting Osteotomy Arthrodesis Arthroscopy

7 Outcomes: Safety: Complications

Efficacy: Duration of operation Time to return to

normal activities Revision surgery Functional result Patients pain free Conversions

Product A searched for four different technologies for metal-on-metal hip resurfacing (Table 32), but did not identify any studies focusing on hip resurfacing using the Buechel-Pappas or the Cormet 2000 implants. Products B and C predominantly focused on the use of the Birmingham implant. This difference may have led to differences in the number of included HTA reports due to a broader range of evidence available (based on the variety of indications) in Product A.

Most of the evidence in Product C was from single prosthesis observational studies; however, three HTAs were included and these were critically appraised using a form specific to the methodological quality of systematic reviews.



Table 33. Methodology of review products for hip resurfacing




Databases: EMBASE PubMed NHS HTA TRIP database Search terms: explicit Date limits: 1997-2002 Restrictions: humans, age≥18years, English and controlled clinical trials

Evidence grading undertaken. No summary of quality of overall evidence base.

0 HTA reports: 5


Databases: Cochrane Library MEDLINE PREMEDLINE Current Contents CINAHL AMI Search terms: explicit Date limits: 2002 Restrictions: NR

Evidence summaries produced of each HTA

0 2 HTA reports


Databases: Cochrane Library DARE CCTR MEDLINE and PREMEDLINE EMBASE HealthSTAR CINAHL NHS EED AMED Audit databases Worldwide web Search terms: explicit Date limits: NR Restrictions: NR

Quality assessment using form specific to systematic reviews. No attempt was made to synthesise the studies that were identified quantitatively

1 3 HTA Reports Published single observation studies: 18 Industry submissions: 3 Unpublished single prosthesis observational data: 23

The economics of hip resurfacing was discussed in briefly in Product A, as two of the five included HTAs provided costing data (one of these being that of Product C). Product C discussed an included economic study, as well as conducting an independent analysis of the economics, giving this product a strong economic focus. Ethics, equity, fairness and accessibility of the product were discussed briefly in Product C but were not considered by Products A and B.

Overall, each of the studies concluded that there was a potential benefit to metal-on-metal hip resurfacing for degenerative hip disease. All products in this assessment highlighted the lack of long term data and suggest that when more of this evidence becomes available, firmer conclusions can be drawn with respect to safety and efficacy.



All three HTAs found limited evidence for metal-on-metal hip resurfacing and a great deal of the evidence was in the form of single prosthesis observational studies. Product A stated that the most relevant data ‘were derived from observational studies (case series with no controls) with short-to mid-term follow-up (less than 10 years) conducted in specialist centres’ and that this is ‘likely to limit the generalisability of the reported results’. These observational studies were reported in the five HTAs included in Product A.

Table 34. Essential conclusions of review products for hip resurfacing

Safety Efficacy Essential conclusions Product A (rapid review)

Discusses safety narratively in terms of complications:

Broadly comparable to that after conventional total hip replacement

6 Efficacy outcomes discussed narratively in the text

Most HTA reports identified poor quality of primary studies being reviewed

Most relevant data from observational studies

May be viable and bone-conserving option

Promising interim procedure in younger, active patients

Brief report largely based on other HTAs


Evidence summaries of other two HTAs

Evidence summaries of other two HTAs

May be viable and bone-conserving option for those likely to out-live hip replacement


Discusses safety narratively in terms of complications

Discusses efficacy narratively according to 6 outcomes

Revision surgery seems to be a predominant outcome

Evidence is limited Has potential to be

effective for hip disease Lack of long term-data More research required

Further to the uncertainty of the safety and efficacy data for hip resurfacing, especially in the long term, each product discussed the lack of evidence regarding the outcomes of subsequent procedures after the primary procedure of hip resurfacing. Despite Product C being more detailed, particularly in regard to economics and ethics, all of the products reflected this gap in current evidence.



4. Discussion

4.1. Survey of HTA organisations For the purpose of this project, ‘rapid reviews’ were classified as health technology assessment products that took between one and six months to complete, which utilised a comprehensive, systematic or quasi-systematic search strategy. The majority of agencies who responded to the survey indicated that they did conduct rapid reviews, and most were able to categorise them into products that took between one and three months to complete, and products that took between three and six months to complete. It is likely that Products of very short duration exist (i.e. less than three months), such as horizon scanning reports and summary reports, and also that full systematic reviews of less than six months exist. This highlights the difficulties inherent in categorising reports by the length of time taken to complete. The fact that some respondents included Products of very rapid duration in the current survey indicates that there is no uniform description of what constitutes a rapid review product. During the Adelaide HTAi Conference in 2006, INAHTA surveyed conference attendees about their rapid review processes (I. Norderhaug, personal communication 2007), and classified rapid reviews into less than one month in duration, one to three months, and three to six months. Horizon scanning reports and prioritising summaries were included within these categories, so no direct comparisons have been made with the current survey.

The current survey showed that no systematic reviews reportedly excluded RCTs, and rarely excluded non-randomised comparative studies. Rapid reviews did on occasion exclude these study types, although the reasons for this were unclear. However, it may be that some rapid reviews are based solely on existing systematic reviews, thus creating a rapid overview of reviews, as opposed to a rapid review of primary evidence. It is also interesting to note that other systematic reviews were sometimes excluded in full systematic reviews, but never in rapid reviews. A well conducted systematic review would be an excellent resource on which to base a rapid review after appropriate quality assessment. The overall conclusions of the full review could be utilised by the rapid review, perhaps for a local perspective or national costing implication. Case reports were excluded from most rapid review products, and also from a high proportion of systematic reviews. Agencies indicated that lower levels of evidence were excluded unless they made up the majority of the evidence base. This indicates that the highest level of evidence is preferable but is not always available.

Rapid review products were less likely to utilise systematic searches of many databases, conduct hand searches and use grey literature than full systematic reviews. This result is not surprising and is likely due to the time constraints faced when preparing rapid reviews. Within the rapid review categories, the reviews of longer duration tended to use systematic searches of many databases more often than the

SECT ION 4 D ISCUSS ION 43


shorter category. It must be noted that the categories used for this question were not clearly defined (eg ‘many’ vs. ‘restricted’) which may have resulted in different interpretations between respondents, with some respondents ticking both responses to this question. Future surveys should aim to find out exactly which databases are searched for each product, to allow for more accurate comparisons.

When compared across the review types, there was a trend showing the length of the review and the more databases searched, with longer reviews having a more rigorous search strategy. This may be of little surprise as database searching can be very time consuming (Allen and Olkin 1999). It has been suggested that exhaustive data collection may have little effect on the final recommendation of a report (Best et al. 1997), and that extended database searching may only retrieve a small percentage of extra studies (Royle & Milne 2003).

Rapid reviews were also less likely to examine economic factors and social issues than full systematic reviews. The longer category of rapid review (reviews of three to six months in duration) tended to be more comprehensive in this respect than the shorter category of review. Although the inclusion of these components would provide little value if based on poor clinical data, they could provide an opportunity to draw attention to the lack of, or limitations of, the evidence.

Rapid reviews were less likely to use external experts, and were also less likely to use peer review than full systematic review. Of the groups that used experts for peer review, all systematic reviews used external experts, compared with 67% for rapid reviews. The rapid reviews of longer duration tended to have a higher rate of external and peer review than the shorter rapid reviews. This is likely to be due to the extra time needed for peer review (as indicated by the responses), and may result in these very rapid products receiving little scrutiny by content and other experts. The impact of this on review quality is unknown and deserves investigation as high level decisions may be made based on these reviews. It would be desirable for rapid review products to include some notification of this limitation so that people are aware that they have not received a high level of scrutiny. The use of external guidance need not always lead to increased timelines. Although it lengthens the time for the shorter rapid reviews, there is no impact to over half of the longer (3-6 month) rapid reviews.

Although the full systematic reviews of all the agencies included in this study have been described as a group, like rapid reviews, they are likely to constitute distinct review products, encompassing differences in search strategies, composition, and included study types. The survey did not ask whether systematic reviews utilised quality assessment or critical appraisal. Although these may be considered fundamental elements of a systematic review, confirmation of their existence would have strengthened the results. Full systematic reviews, like rapid reviews are not homogenous, which emphasises the importance of detailing research methodologies for these reviews (as well as any other type of review). In the case of rapid reviews, this should be done in such a way as to avoid making them appear more valid than



they are. Failure to do this could result in full systematic reviews appearing redundant by those who do not understand the important differences between the two types of products. This may result in decision makers choosing to not progress to the use of full systematic reviews, but rather commission a series of rapid reviews. This may not be appropriate in some instances, particularly when considering complex interventions when safety is a major concern and when risks need to be minimised.

From the survey results, it appeared that commissioning of rapid reports was mainly done at the macro level of decision making. The intended audiences of rapid reviews were similar to the commissioning group, except at the micro level. Nearly 60% of responses indicated that audiences were at the micro level, but these audiences commissioned reports considerably less often. This is not unexpected, as individuals would not normally have the opportunity to commission work, being usually done at a higher level e.g. government. These results were supported by the responses regarding the rationale for conducting a rapid review, which tended to reflect political urgency or a need to support particular decisions/answer specific policy or clinical questions. Shorter rapid reviews were more likely to be written as a result of political urgency, whereas larger rapid reviews were more frequently completed due to limited time and resources.

The use of different types of HTA reviews in determining and implementing policy decisions was beyond the scope of this review, but cannot be ignored. Health technology assessment falls in between two policy arenas: policy making at the macro level, which defines the general scope of government obligations to cover healthcare services, and how health technologies will be assessed, and decision making at the meso level, which determines whether specific technologies fall within the coverage policies (Abelson et al. 2007). In addition to this, there are organisations, such as ASERNIP-S, who have mandates focussed towards clinicians making decisions at the micro level, with less regard to overall policy.

To effectively influence policy makers (as well as other decision makers), review products must take into account issues relating to the timing of the release of the document, political sensitivity, their intended audience, and the most appropriate way to reach that audience (Lomas 1993). In some cases, these factors may conflict with the need of the report to be highly accurate and written quickly (Rotstein and Laupacis 2004). Further research into the impact of various types of HTA reviews (such as rapid, comprehensive, and horizon scanning reports) into the policy making process would be worthwhile, especially in relation to specific topics and different types of technologies.

Due to the limited survey group, the results were not grouped by country or type of healthcare system. Exploring whether there are any links between healthcare systems and the use of rapid reviews for policy making may also provide interesting results, and could contribute to revealing the international impact of these products.



Limitations of the survey The survey group was made up of preselected agencies that were members of INAHTA, or were suggested as likely groups to conduct rapid reviews rather than those identified by a comprehensive and systematic search. As a result only a relatively small number of agencies were contacted, and some groups that do conduct such reviews may have been missed. In some cases, the differences in results between the rapid review products may potentially have been be no more than a reflection of the sampling in the survey, as the number of survey responses was relatively small. It was noted in some instances that questions were misinterpreted by respondents whose first language was not English. Clearer direction regarding the intent of each question would have minimised this problem. In addition to this, the level of detail given by respondents varied greatly. The open-ended narrative questions may have favoured agencies that gave multiple responses: all of the responses given to a particular question were included in the analyses, and as such an agency providing multiple responses to a question would have contributed a larger proportion of the results than an agency providing only one answer.

Although the survey tool did address the number of staff (FTE) working on rapid reviews it did not address the duration and length of each agency’s systematic review, or the number of staff working on each systematic review. The annual number of each type of review conducted by each agency was also not collected. Capturing this data during the current survey could have given an indication of the proportion of each type of review, and provided more current insight into the necessity and importance placed on these rapid reports by the responding agencies.

Summary Our survey indicates that many HTA organisations around the world are currently conducting rapid reviews. These reviews are mainly commissioned by high level decision makers in response to political urgency or the need to support a decision. These decision makers are also the reports’ intended audience, in addition to individuals involved in healthcare. In general, the duration of the review determined the level of database searching, the use of external guidance, the extent of study exclusion, and the components included in the review, with rapid reviews generally being done with less rigor, and containing less detail than full systematic reviews. Alternatively, the methodology of the review may have determined the duration of time required for its completion. It is still unclear whether these aspects have any detrimental effects on policy decisions made using rapid reviews. Further research is needed to explore the use, outcomes, and importance of rapid reviews in the current HTA environment.



4.2. Literature on rapid review methodology Evident from the peer-reviewed literature is a lack of standardised, explicit methodology for the undertaking of rapid reviews. It may be that these products defy the description of prescriptive, a priori methodology due to their flexible and adaptive nature, which allows them to be targeted to particular audiences (e.g. clinicians, policy makers or administrators) within specific healthcare contexts. Hence, the development of explicit, standardised methodology regarding these products may be counter-productive, as it would limit one of their key strengths, their adaptiveness.

Further preventing the development of standardised methodology may be the wide variety of interventions and technologies that rapid reviews consider – from nascent, emerging interventions through to those that are well-established. To some extent, the evidence base available will prescribe the precise methodology employed by each individual review. For example, an emerging technology may demand more extensive data searches in order to identify sufficient literature, while a more established technology requiring contextualisation to a specific healthcare setting may allow less intensive searching. Additionally, consideration of the interventions under investigation will further influence the methodology – those requiring extensive consideration of economics and ethics, for example, may not be as suited to rapid review as more straightforward interventions.

Proposed methods for conducting rapid reviews, such as limited searching and developing highly refined research questions are yet to be adequately validated in the literature. Clearly, development of a limited research question will reduce the generalisability of the review findings to smaller patient populations or certain comparator interventions, which may be more appropriate for some topics than others.

Truncated literature search strategies may often lead to uncertainty surrounding the conclusions of a review. Conversely, however, it has been suggested that extensive literature searching can actually introduce bias into an evidence base rather than limiting it, as trials that are harder to locate can be of lower quality (Egger et al. 2003). It may be that resources would be more appropriately directed to thorough quality assessments of evidence available in major databases, rather than extensive searching.

Given the inherent variability of the size and composition of the evidence base for any one intervention, it may not be possible to validate methodological strategies for conducting rapid reviews and apply them to every subject. Rather, each topic must be evaluated by thorough scoping, and appropriate methodology defined. This may highlight a greater role for a panel of expert advisors to ensure that the nuances of each topic have been adequately considered.

Rapid reviews are gaining increasing prominence within the HTA community, as new technologies emerge rapidly and the demands of policy makers increase to keep pace. These products may continue to defy the definition of explicit a priori methodology; however, each agency producing rapid reviews must be transparent about the



methods used for each individual review that they produce, thereby allowing all users to appreciate the limitations of the product and apply its recommendations appropriately.



4.3. Identification and comparison of rapid reviews and full systematic reviews The internet scoping search for rapid reviews identified a large variety of HTA literature on agency websites. There are many different HTA products varying in size from a few pages to many hundreds of pages, depending on the complexity of the topic or technology, scope/comprehensiveness of the research and the timelines of the agency.

Clearly, different agencies prescribe different times in which a rapid review is to be completed. Some agencies take over six months to complete a rapid review, while others may require substantially less time. For the purposes of this report the working definition of a rapid review was ‘any HTA product produced between one and six months with a comprehensive/systematic literature search’. It is also important to note that there may be significant dissonance between a rapid systematic review and a rapid (or mini) HTA. A mini HTA may be much more comprehensive in the elements that it includes, including policy and budget considerations contextualised to a specific healthcare setting. Consideration must also be given to the impact of the publication cycle on perceived timeframes for the completion of reviews. While the research component of a rapid review may only take a few months, procedures such as guidance, appeals and publishing may add significantly to the time elapsed between the commissioning of a review and its availability for use in decision making.

When carrying out a search utilising the relevant HTA agencies’ websites, a fully systematic search was not possible for a number of reasons, primarily the varied nature and sheer volume of the HTA literature. There was a lack of consistency between websites in how the publications were listed and it was frequently a challenge to distinguish between products that were full systematic reviews, horizon scanning reports or rapid reviews. Navigation of HTA websites would have been easier if all the agencies had detailed the purpose of, and methodology used for, each of their products. This would enable the user (be it healthcare professional or a consumer) to fully understand the intention and limitations of each product.

A further limitation of this study was that translation of non-English reviews was not possible, resulting in a number of rapid reviews that could not be used in this study. Such reviews may have been produced for the local context. It would be of interest to examine these foreign language products as they could possibly provide alternative methodologies. In addition, this review has focussed on surgically-related topics. HTA reports have been published in many other areas such as pharmaceuticals, diagnostic devices and workforce-related issues. A rapid review produced on these topics may differ to a review written on surgical topics for various reasons such as the availability of studies, budget and policy implications, as well as the utilised methodology.



The reports examined comparatively in this study demonstrated that one of the key ways in which to increase the viability of producing a rapid review on a specific topic is to formulate a specific research question of limited scope. For example, in the case of drug-eluting stents the full review looked at two other health technologies as comparators in addition to the use of stents in the treatment of coronary artery disease (coronary artery bypass grafting and percutaneous transluminal coronary arthroplasty). These alternative comparators were not considered in the rapid reviews, so the review questions could be much more specific.

Rapid reviews are often written as an alternative to full reviews due to limited time. The reasons for the lack of long timelines may be varied. The agency may not have the resources to spend many months writing a single review, or the review may be required urgently as a result of a policy requirement. There are advantages and disadvantages to this. Of benefit is the fact that the results of the available evidence are published much more swiftly. An example of where the time taken for completion of a full review differed significantly from that for a rapid review was with living donor liver transplantation. Searching for both products took place in January 2004, so both would have had access to the same evidence base. However, the rapid product was published a full seven months before the full review.

Alternatively, a full review which has a larger and more comprehensive literature search can identify many more studies that the rapid reviews would not have had access to. An example of this is with lung volume reduction surgery where the full review had great access to unpublished manuscripts and grey literature. These additional resources may have been available because of the use of an expert advisory panel.

There was variability in the types of included studies used in the rapid reviews. Some used only published systematic reviews, some used RCTs, and some also used selected case series evidence. The methodology of reviews should set out the rationale for the use of levels of evidence so that it is clear why certain information is included and others excluded.

The availability of different literature also plays an important role in determining the outcomes and essential conclusions of each topic. Case series may be very useful as a source of long-term safety evidence, so may be valuable in addition to RCTs. However, for some topics, RCT evidence may be very difficult to obtain; such is the case of live donor liver transplant. It would be ethically very difficult to randomise participants to this kind of surgery and so RCTs may be difficult, if not impossible, to conduct. In instances such as this, the absence of comparative evidence should not prevent the production of a comprehensive review. The next best available evidence should be used. At the very least, this would provide evidence regarding short and long-term safety of the health technology. With rapid reviews, the inclusion of large numbers of case series may not be possible so the issue would be how the studies are selected to provide safety information.



In addition to safety outcomes varying according to topic and the literature available, the inclusion of other issues such as ethics are very much dependent on the topic. In the case of living donor liver transplantation, where there effectively are no gains for the donor, ethical issues were important to consider. Such themes as coercion and informed consent are pivotal and both the rapid and full reviews discussed these. This is considerably different to a topic like hip resurfacing where informed consent is still an issue but the consequences are of more obvious benefit to the participant. Rapid reviews may not have the remit to comprehensively consider complex issues such as these in appropriate depth. Careful consideration, possibly with the input of external experts, is required to ensure that the nuances of any topic that forms the basis of a rapid review are adequately addressed.

For many health technologies, cost can be a significant issue. For example, studies have shown that drug eluting stents are considerably more expensive per unit than a bare metal stent. If these new technologies were widely adopted simply because of a slight increase in effectiveness over their current counterparts, the budget impact could be very large. In these cases, a detailed cost-effectiveness analysis would be of great importance. This may highlight any specific potential indications and populations for the health technology. In the case of drug eluting stents the full review carried out a cost-effectiveness analysis, an independent evaluation, a budget impact analysis and also critically appraised economic models submitted by device manufacturers. One of the rapid reviews (product B), in this instance, carried out a brief cost-effectiveness analysis showing that despite the limited timeframe economic evaluations are still possible and useful for rapid reviews to consider. In this instance, the overall conclusions of both the full and rapid reviews were the same.

The depth of economic evaluation may be dependent on the commissioner of the review and its ultimate target audience. Detailed costing may be required if the review is intended to guide policy, in which case a rapid review with limited timelines may not be possible. Alternatively, a review compiled in response to a question by a clinician may not be required to deal with costs, but only safety and effectiveness outcomes.

In order to ensure quality control of their reviews, some HTA agencies routinely employ the expertise of an advisory panel/review group in order to provide comprehensive expert opinion with which to inform the review. For the topics examined in this report, all of the full systematic reviews utilised advisory panels but only two of the rapid products did. Advisory panels offer a number of significant advantages such as guidance on the direction of the review, greater access to the relevant literature and expert advice which would result in greater rigour in the final product. An expert clinical panel could suggest whether any specific issues such as cost, ethics, or long-term safety, are important to consider and help choose appropriate clinical outcomes, and guide the conclusion. This is evident in the two rapid reviews (drug eluting stents product B and lung volume reduction surgery



product B) which included more elements of a systematic review without exceeding the allocated timeline, despite the involvement of an expert review panel.

Each of the review styles were commissioned by a variety of bodies, including healthcare plans, hospital administrators, clinicians and governments. Within the data set of the eleven reviews considered, there was no clear correlation between the commissioning group and the use of the review findings in policy making. For the majority of reviews, whether they were rapid or full, the overall guidance of the review was followed in the application of the review findings to policy making at a variety of levels. The most notable deviation from this was in the case of the full review on drug eluting stents, where the recommendations of the review were not followed in policy making regarding this technology.

The impact of reviews on policy seemed to be more closely related to the intended audience/the requestor than the nature of the review, but given the relatively small sample size, definitive conclusions regarding the impact that rapid reviews have on policy making cannot be drawn.

Summary Overall, the products examining each intervention agreed with one another in their essential conclusions, regardless of whether they were rapid or full reviews. However, the full reviews consistently provided greater depth of information and more detailed recommendations pertaining to the implementation of each particular health technology.

Depending on the scope of the topic, some reviews may be better suited to a rapid methodology whereas others require the depth and rigour of a comprehensive review (e.g. lung volume reduction surgery) to highlight the safety and risk factors of an intervention (even if the evidence is observational). Additionally, some topics are clearly better suited to full reviews, due to their complexity and need to consider issues such as ethics in greater depth. Topics that are potentially controversial, and the findings of a report therefore more liable to challenge, may also require the undertaking of a full review to ensure the capacity of the findings to withstand any contention. New health technology interventions are almost always more expensive than the current option, and economics may play a significant role in certain topics. Comprehensive reviews have the capacity to carry out detailed and robust cost effectiveness analyses that rapid reviews, due to time limitations, cannot undertake. Whatever the methodology undertaken this should be stated clearly and transparently in the review so that readers can understand any limitations. Advisory panels also add further to the robustness of full systematic reviews and it may be worthwhile considering how the use of advisory panels can be tailored to meet the requirements of rapid reviews.



Future research In such a rapidly developing area, there are a number of avenues of research that will provide valuable contributions to developing the body of evidence pertaining to the use and production of rapid reviews.

An exploration of the differences between the different types of rapid review products (production taking one to three months and production taking three to six months) and full HTAs would further inform HTA agencies and commissioning bodies about the most appropriate product for their specific needs.

A more in-depth examination of the characteristics of the groups commissioning rapid reviews and their purpose would allow HTA agencies to tailor their product appropriately. Additionally, exploring the perceptions of end-users of rapid review products would allow development in this area to be appropriately focussed, by examining questions surrounding whether decision makers trust full HTAs more than rapid reviews, and what factors influence their trust in products. This information could be further complemented by additional research into the impact of various types of HTA reviews (such as rapid, comprehensive, and horizon scanning reports) into the policy making process.

Reviews written in languages other than English and those pertaining to non-surgical topics have been excluded from this review. A collaborative effort with HTA organisations publishing rapid and full reviews in different languages and other HTA areas may be worthwhile in order to examine the applicability of the extrapolation of these findings to other countries and areas of specialisation.



5. Conclusions and Recommendations

The multi-faceted approach taken to investigation of rapid reviews by this report has identified that the current products being produced by HTA agencies are not well defined and are highly variable in both their methodology and composition.

The majority of agencies which responded to the survey reported undertaking some form of rapid review, utilising a variety of methodologies to increase the speed of the review: highly defined research questions; limited searching; highly specific inclusion and exclusion criteria; limited use of external peer review (expert panel) and restriction of the components of the final review. These products may defy the development of explicit a priori methodology; the focus of the review will play a large part in defining the exact methodology used. However, it is critical that each agency producing rapid reviews is transparent about the methods used for each individual review, allowing users (whether consumers or healthcare professionals) to appreciate the limitations of the review and apply its findings appropriately. It may indeed be inappropriate to define a specific template for the production of rapid reviews. The methodology used for an individual review can depend on a large variety of factors, such as who commissioned the review, what the topic and the specific review questions are, which comparators are appropriate, quality of the available studies and so on. The development of a formal outline by which to conduct rapid reviews may justify their use to decision makers who may request them in preference to more comprehensive reviews simply due to the speed of completion. This perceived increase in ‘bang for the buck’ may not always be appropriate.

Commissioning of these reviews was found to be undertaken primarily at the macro level of decision making, with the audience frequently at a micro decision making level. The commissioning of the review and its intended use should provide appropriate direction to the scope of the document. A review which is intended for a purely clinical audience could limit the work compared to that which may be required for a macro policy decision if this required a detailed and time-consuming economic evaluation.

The overall influence of rapid reviews on policy decisions has not yet been adequately investigated, but deserves greater attention. It may be that decision makers implicitly trust a comprehensive review more than a rapid review, in which case it would be interesting to define which factors influence their trust in a product.

While the comparison between full and rapid reviews on a number of individual topics did not identify any instances where the various products reached fundamentally different conclusions, it was found that rapid reviews may be inappropriate in a number of circumstances. Procedures that require extensive consideration of safety outcomes may require the preparation of a full review, to enable an adequate number of case series to be assessed for safety data where

SECT ION 5 CONCLUS IONS AND RECOMMENDATIONS 54


necessary. Additionally, those procedures and devices that require a substantial cost-effectiveness analysis relevant to a specific healthcare context may not be suited to rapid review, nor are those that demand complex consideration of ethical issues pertaining to their establishment.

Rapid reviews may benefit from the rigor of external peer review through the use of expert advisory panels/review groups. Preliminary data suggests that this does not necessarily impact adversely on review timelines, and may be useful in directing the scope of the review appropriately. External advice may also assist in the completion of an appropriate review conclusion at both clinical and policy levels.

In summary, a rapid review should be written in answer to specific questions, rather than as a quick alternative to a comprehensive systematic review. The speed of review completion should be due to a specified direction, rather than as a result of a curtailed and perhaps overly simplified methodology. In this manner, rapid reviews could be used to inform specific policy decisions in a timely manner without losing any of the important information that may be expected from a comprehensive review. It is perhaps the appropriate use, as opposed to the exact methodology, of a rapid review that requires future consideration.

Acknowledgments The authors wish to acknowledge Dr Ann Scott and Anne Parkhill for their assistance during the preparation of this review. We are also indebted to all of the Agencies and individuals who responded to the survey and provided valuable information and ideas to guide the preparation of this report.

The ASERNIP-S project is funded by the Australian Government Department of Health and Ageing and the South Australian Department of Health.

SECT ION 5 CONCLUS IONS AND RECOMMENDATIONS 55


6. References Abelson J, Giacomini M, Lehoux P, Gauvin F-P. Bringing 'the public' into health

technology assessment and coverage policy decisions: From principles to practice. Health Policy 2007; 82: 37-50.

Aidelsburger P, Felder S, Wasem J. Guidelines for rapid health-economic HTAs. 2002.

Alberta Heritage Foundation for Medical Research (AHFMR). Adult to Adult Living Donor Liver Transplantation. AHFMR.Technote TN 45. Last updated March 2004. http://www.ihe.ca/hta/hta_unit.html [Accessed March 2007].

Alberta Heritage Foundation for Medical Research (AHFMR). Metal-on-metal hip resurfacing for young, active adults with degenerative hip disease. AHFMR.Technote TN 33. Last updated March 2002. http://www.ihe.ca/hta/hta_unit.html [Accessed March 2007].

Allen E and Olkin I. Estimating time to conduct a meta-analysis from number of citations retrieved. JAMA, Journal of the American Medical Association 1999; 282(7): 634-635.

Australian Institute of Health and Welfare (AIHW). Coronary Heart Disease. AIHW website. Last updated May 2006. http://www.aihw.gov.au/cvd/majordiseases/coronary.cfm [Accessed May 2007].

Australian Institute of Health and Welfare (AIHW). Disability associated with chronic diseases. AIHW website. Last updated June 2005. http://www.aihw.gov.au/cdarf/data_pages/disability/index.cfm [Accessed May 2007].

Banjeree S, Babidge W, Miller J, Smith J, Noorani HZ, Cuncins-Hearn A, and Mensinkai S. Comparison of Lung Volume Reduction Surgery with Medical Management for Emphysema. Canadian Coordinating Office for Health Technology Assessment (CCOHTA).Technology Report.Issue 48. Last updated December 2004. http://www.cadth.ca/media/pdf/176_lvrs_tr_e.pdf [Accessed March 2007].

Beauchamp G. Lung volume reduction surgery: evidence for the resurrection of an old operation? Canadian Journal of Surgery 2000; 43(5): 328-329.

Benditt JO and Albert RK. Surgical options for patients with advanced emphysema. Clinical Chest Medicine 1997; 18(3): 577-593.

Bernath V. Hip resurfacing in patients with osteoarthritis. Centre for Clinical Effectiveness. Last updated 2002. http://www.med.monash.edu.au/healthservices/cce/evidence/ [Accessed March 2007].

REFERENCES 57


Best L, Stevens A, Colin-Jones D. Rapid and responsive health technology assessment: The development and evaluation process in the South and West region of England. Journal of Clinical Effectiveness. 1997; 2(2): 51-56.

Brophy J. An evaluation of drug eluting (coated) stents for percutaneous coronary interventions: What should their role be at the McGill University Health Centre (MUHC)? Technology Assessment Unit of the McGill University Health Centre (MUHC). Last updated July 2003. http://www.mcgill.ca/tau/publications/ [Accessed March 2007].

Canadian Agency for Drugs and Technologies in Health (CADTH). Drug Eluting Stents: Managing Coronary Artery Stenosis following PTCA. Issues in Emerging Technologies. Issue 40. Last updated October 2002. http://www.cadth.ca/media/pdf/212_drugelutingstents_cetap_e.pdf [Accessed March 2007].

Celli BR. Lung volume reduction surgery, where are we now? European Respiratory Review 2000; 10(74): 343-349.

Cooper JD, Trulock EP, Trintafillou AN et al. Bilateral pneumectomy (volume reduction) for chronic obstructive pulmonary disease. Journal of Thoracic and Cardiovascular Surgery 1995; 109:106-116.

Corabian P and Harstall C. Rapid assessments provide acceptable quality advice. 2002.

Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technology Assessment 2003; 7(1)

Eisenberg JM and Zarin D. Health technology assessment in the United States. Past, present, and future. International Journal of Technology Assessment in Health Care 2002; 18(2): 192-198.

Goodman C and Ahn R. Methodological approaches of health technology assessment. International Journal of Medical Informatics 1999; 56: 97-105.

Hailey D and Juzwishin D. Managing external risks to health technology assessment programs. International Journal of Technology Assessment in Health Care. 2006; 22(4): 429-435.

Hailey D, Corabian P, Harstall C, Schneider W. The use and impact of rapid health technology assessments. International Journal of Technology Assessment in Health Care 2000; 16(2): 651-656.

Hailey D. Health technology assessment. Singapore Medical Journal 2006; 47(3): 187-192.

Higgins S. The effectiveness of vacuum assisted closure (VAC) in wound healing. Centre for Clinical Effectiveness (CCE). Last updated 2003. http://www.med.monash.edu.au/healthservices/cce/evidence/ [Accessed February 2007].

58 REFERENCES


Hill RA, Bagust A, Bakhai A, Dundar Y, Mota RM, Reaney A, Roberts DH, Walley T, Dickson R, Williamson PR, and Haycox A. Coronary Artery Stents: Rapid Systematic Review and Economic Evaluation. Health Technology Assessments. Last updated February 2003. http://www.hta.ac.uk/execsumm/summ835.htm [Accessed March 2007].

Jonsson E, Banta H, Henshall C, Sampietro-Colom L. Summary report of ECHTA/ECAHI project. International Journal of Technology Assessment in Health Care 2002; 18(2): 218-237.

Lefevre F. Extracorporeal Shock Wave Treatment for Chronic Plantar Fasciitis. Bluecross Blueshield Association. TEC Assessment 19(18). Last updated March 2005. http://www.bcbs.com/betterknowledge/tec/vols/19/19_18.html [Accessed March 2007].

Lefevre F. Extracorporeal Shock Wave Treatment for Chronic Tendinitis of the Elbow (Lateral Epicondylitis). Bluecross Blueshield Association. TEC Assessment 19(16). Last updated 2005. http://www.bcbs.com/betterknowledge/tec/vols/19/19_16.html [Accessed March 2007].

Lehoux P, Tailliez S, Denis J-L, Hivon M. Redefining health technology assessment in Canada: Diversification of products and contextualization of findings. International Journal of Technology Assessment in Health Care. 2004; 20(3): 325-336.

Lomas J. Diffusion, dissemination, and implementation: Who should do what? Annals of the New York Academy of Sciences 1993; 703: 226-235.

Lynch P, Raya T, Mathew V, Soni B, Laroia S, and Rahman M. Drug-Eluting Stents for the Prevention of Restenosis in Native Coronary Arteries. Institute for Clinical Systems Improvement. Technology Assessment Report. Last updated December 2003. [Accessed March 2007].

Martin CW. Extracorporeal shock wave therapy for treating musculoskeletal. WCB Evidence Based Practice Group. Review. Last updated September 2004. http://www.worksafebc.com/health_care_providers/related_information/evidence_based_medicine/default.asp [Accessed March 2007].

McGregor M and Brophy JM. End-user involvement in health technology assessment (HTA) development: a way to increase impact. International Journal of Technology Assessment in Health Care 2005; 21(2): 263-267.

Mickman J, Harmon K, and Drage C. Lung Volume Reduction Surgery for Emphysema. ICSI: Technology Assessment Report. Last updated October 2003. http://www.icsi.org/index.asp [Accessed March 2007].

Middleton P, Duffield M, Lynch S, Verran D, House T, Stanton P, Stitz R, Padbury R, and Maddern G. Live donor liver transplantation- adult outcomes: a systematic review. ASERNIP-S Reports. Last updated October 2004. http://www.surgeons.org/asernip_s/publications_livedonorlivertransp.htm [Accessed March 2007].

REFERENCES 59


NICE. Interventional Procedures Overview of Lung Volume Reduction Surgery for Advanced Emphysema. NICE: Interventional Procedure Guidance 114. Last updated September 2004. http://www.nice.org.uk/ipcat.aspx?c=90647 [Accessed March 2007].

NICHSR: National Information Centre on Health Services Research and Health Care Technology. HTA 101: Glossary. United States National Library of Medicine. Last updated 2006. http://www.nlm.nih.gov/nichsr/hta101/ta101014.html [Accessed November 2006].

Paolucci E, Donnon T, and Poulin P. An overview: What is health technology assessment (HTA)? Canadian Coordinating Office for Health Technology Assessment and University of Calgary. Last updated 2005. http://www.calgaryhealthregion.ca/surgery/officesurgicalresearch/ssp_pdf_doc/Manual%20Module%201%20(FINAL).pdf [Accessed November 2006].

Pham C, Middleton P, and Maddern G. "Vacuum-Assisted Closure for the Management of Wounds: An Accelerated Review". ASERNIP-S: Accelerated Review. Last updated 2003. http://www.surgeons.org/asernip_s/publications_vacuum.htm [Accessed February 2007].

Rotstein D and Laupacis A. Differences between systematic reviews and health technology assessments: A trade-off between the ideals of scientific rigor and the realities of policy making. International Journal of Technology Assessment in Health Care 2004; 20(2): 177-183.

Rowett D, Simmons S, Cafarella P, Frith P, editors. Informed Partnerships for Effective Self-Management of Chronic and Complex Lung Diseases. 2005.

Royle P and Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searching. International Journal of Technology Assessment in Health Care. 2003; 19(4): 591-603.

Said A, Einstein M and Lucey MR. "Liver transplantation: an update 2007". Current Opinion in Gastroenterology 23(3):292-8.

Samson D, Lefevre F, and Aronson N. "Wound-healing Technologies: Low-level Laser and Vacuum-Assisted Closure". AHRQ: Evidence Report/ Technology Assessment. Last updated December 2004. http://www.ahrq.gov/clinic/tp/woundtp.htm [Accessed February 2007].

Scott A and Harstall C. Brief reports versus full systematic reviews: when is enough really enough? Annu Meet Int Soc Technol Assess Health Care Int Soc Technol Assess Health Care Meet 2003; 19(8)

Stevens A, Milne R, Burls A. Health technology assessment: history and demand. Journal of Public Health Medicine 2003; 25(2): 98-101.

Vale L, Wyness L, McCormack K, McKenzie L, Brazelli M, and Stearns S. A systematic review of the effectiveness and cost-effectiveness of a metal-on-

60 REFERENCES


metal hip resurfacing arthroplasty for treatment of hip disease. NCCHTA: Health Technology Assessment. Last updated 2002. http://www.hta.as.uk/execsumm/summ615.htm [Accessed March 2007].

van Velden ME, Severens JL, Novak A. Economic evaluations of healthcare programmes and decision making. The influence of economic evaluations on different health care decision-making levels. Pharmacoeconomics 2005; 23(11): 1075-1082.

REFERENCES 61

APPENDIX A – SURVEY


Appendix A – Survey ASERNIP-S survey on Rapid HTA

Please return the completed survey to:

e-mail: [email protected]

Fax: +61 8 8362 2077

Postal address:

ASERNIP-S PO Box 553

STEPNEY SA 5069 AUSTRALIA

An important part of the health technology assessment process is the timely production of high quality reviews that investigate all the available evidence on a particular health technology. Over the past few years, there has been an increased interest in the production of rapid reviews, in which the timeliness of the review may take precedence over the quality of the evidence used. At ASERNIP-S we are currently conducting a study on ‘Rapid verus full systematic reviews’. The purpose of this is to establish the current role and perception of rapid reviews within the HTA community. We shall also try to assess the quality and conclusions of rapid reviews and compare them to full systematic reviews or full HTA reports. This survey follows on from a survey conducted by INAHTA in 2006. Agency name: Respondent name: Please answer the following questions with regard to your own agency’s experience of conducting rapid HTA reviews: 1. Does your agency

undertake rapid reviews? (Please check YES or NO)

YES NO

If NO then please return the survey to the above address If YES then please answer the following questions:

Product 1 Name:

Product 2 Name:

2. What kind of rapid review products do you have, which are prepared in between 1-6 months? (Please list each product separately)

Product 3 Name:

1-3 months 3-6 months Product 1 Product 2

3. What timeframe do you have for your rapid review products?

Product 3 Product 1 Product 2

4. How many researchers (full time equivalent) work on each product (include information resource staff)? Product 3

5. Who commissions each Product 1

64

mailto:[email protected]

Product 2 rapid review product? (e.g. internal question, government, hospital, charity, individual)

Product 3

Product 1

Product 2

6. What is the intended audience of each rapid review product? (e.g. consumer, health professional, hospital policy, government policy)

Product 3

Product 1

Product 2

7. What is the rationale for undertaking rapid, rather than full, reviews? (e.g. clinical urgency, local hospital urgency, political urgency, cost of review process, other HTA agency has already done a full review, answer to a specific question)

Product 3

Sy

stema

tic

sear

ch of

ma

ny

datab

ases

Syste

matic

se

arch

of

restr

icted

da

tabas

es

Hand

se

arch

ing

Use o

f gre

y lite

ratur

e

Product 1

Product 2

Product 3

8. Do your HTA products use an exhaustive search strategy? (Please check box if you use the following searches)

Full systematic review

YES NO

Product 1

Product 2

Product 3

9. Does your rapid review examine clinical outcomes?


Product 1

Product 2

Product 3

10. Does your rapid review examine economic factors?


Product 1

Product 2

Product 3

11. Does your rapid review deal with social issues? (e.g. equity, ethical or legal)



YES NO Product 1

Product 2

Product 3

12. Do your HTA products use external experts (e.g. an advisory panel)?


Product 1 External?

Product 2 External?

Product 3 External?

13. Do your HTA products use peer review. Please check the second box if this is external review.


External?

Product 1 Product 2

14. If you have answered YES to questions 12 or 13, how does this impact on your timeframe? Product 3

Exist

ing

syst

emat

ic re

view

RCT

Non-

rand

omise

d co

ntro

lled

trial

Case

serie

s

Case

repo

rt

Product 1

Product 2

Product 3

15. Would you exclude any of these studies from your HTA products?


Product 1

Product 2

16. Does your rapid product utilise quality assessment or critical appraisal of the included studies? Please give details. Product 3

17. Any additional comments:

66

APPENDIX B – INAHTA MEMBERS

67


Appendix B – INAHTA members AETMIS (Canada)

http://www.aetmis.gouv.qc.ca/site/index.php?accueil

AETS (Spain)

http://www.isciii.es/htdocs/index.jsp

AETSA (Spain)

http://www.juntadeandalucia.es/salud/orgdep/aetsa/

AHFMR (Canada)

http://www.ahfmr.ab.ca/ (former INAHTA member; information on HTA activities now available through IHE website)

AHRQ (USA)

http://www.ahrq.gov/

AHTA (Australia)

http://www.health.adelaide.edu.au/publichealth/consult/health_techn_assess.html

AHTAPol (Poland)

http://www.aotm.gov.pl/#

ASERNIP-S (Australia)

http://www.surgeons.org/Content/NavigationMenu/Research/ASERNIPS/default.htm

AVALIA-T (Spain)

http://avalia-t.sergas.es/

CADTH (Canada)

http://www.cadth.ca/ http://dev.ccohta.ca/entry_e.html

CAHTA (Spain)

http://www.aatrm.net/html/ca/Du8/index.html

CEDIT (France)

http://cedit.aphp.fr/

CENETEC ( Mexico)

http://www.cenetec.gob.mx/htmls/cenetec.html#

CMT (Sweden)

http://www.cmt.liu.se/

CRD (UK)

http://www.york.ac.uk/inst/crd/

CVZ (The Netherlands)

http://www.cvz.nl/

DACEHTA (Denmark)

http://www.sst.dk/Planlaegning_og_behandling/Medicinsk_teknologivurdering.aspx?lang=en

DAHTA @ DIMDI (Germany)

http://www.dimdi.de/dynamic/de/index.html

DECIT-CGATZ (Brazil)

http://portal.saude.gov.br/portal/saude/area.cfm?id_area=1088

DSI (Denmark)

http://www.dsi.dk/

FinOHTA (Finland)

http://finohta.stakes.fi/FI/index.htm

GR The Netherlands)

http://www.gr.nl/index.php

HAS (ANAES) (France)

http://www.has-sante.fr/anaes/anaesparametrage.nsf/HomePage?ReadForm

HunHTA (Hungary)

http://hecon.uni-corvinus.hu/

IAHS (UK)

http://www.abdn.ac.uk/iahs/

ICTAHC (Israel)

http://www.health.gov.il/english/pages_e/default.asp?pageid=28&parentid=15&catid=13&maincat=2

IECS (Argentina)

http://www.iecs.org.ar/

IHE (Canada)

http://www.ihe.ca/hta/index.html

IMSS (Mexico)

http://www.imss.gob.mx/IMSS/IMSS_SITIOS/DPM/Informacion/Tecnologia/Principal.htm

KCE (Belgium)

http://kce.fgov.be/index_en.aspx?SGREF=5211

LBI @ ITA (Austria)

http://hta.lbg.ac.at/de/index.php

MAS/OHTAC http://www.health.gov.on.ca/english/providers/program/mas/mas_mn.html

68

http://www.aetmis.gouv.qc.ca/site/index.php?accueil

http://www.isciii.es/htdocs/index.jsp

http://www.ahfmr.ab.ca/

http://www.health.adelaide.edu.au/publichealth/

http://www.surgeons.org/Content/Navigation


http://dev.ccohta.ca/entry_e.html

http://www.aatrm.net/html/ca/Du8/index.html

http://www.sst.dk/Planlaegning_og_behandling


http://finohta.stakes.fi/FI/index.htm

http://www.gr.nl/index.php

http://www.has-sante.fr/anaes/anaesparametrage.nsf/

http://www.health.gov.il/english/pages_e/default.asp

http://www.ihe.ca/hta/index.html

http://kce.fgov.be/index_en.aspx?SGREF=5211

http://hta.lbg.ac.at/de/index.php


(Canada) MSAC (Australia)

http://www.msac.gov.au/

MTU-SFOPH (Switzerland)

http://www.bag.admin.ch/

NCCHTA (UK)

http://www.ncchta.org/

NHSC (UK)

http://www.pcpoh.bham.ac.uk/publichealth/horizon/

NHS QIS (UK)

http://www.nhshealthquality.org

NOKC (Norway)

http://www.kunnskapssenteret.no/

NZHTA (New Zealand)

http://nzhta.chmeds.ac.nz/

OSTEBA (Spain)

http://www.osanet.euskadi.net/r85-2905/es/contenidos/informacion/presentacion_osteba/es_1199/indice_c.html

SBU (Sweden)

http://www.sbu.se/www/index.asp

UETS (Spain)

http://www.madrid.org/lainentralgo/estudios/marcevalua/ffevalua.htm

VATAP (USA)

http://www.va.gov/vatap/

VSMTVA (Latvia)

http://www.vsmtva.gov.lv/v/en/v/en/agentura/struktura/index.aspx

ZonMW (The Netherlands)

http://www.zonmw.nl/

69

http://www.nhshealthquality.org/

http://www.sbu.se/www/index.asp


70

APPENDIX C – EXCLUDED STUDIES


Appendix C – Excluded studies Studies excluded on abstract

Reference Focus of paper/ basis for exclusion Proposed methodology for establishing priorities for healthcare technology assessments--AHCPR. Federal Register 1994; 59(79:Pt 1): t-6.

Outline of future research agenda – not rapid review specific

Report from the Canadian Coordinating Office for Health Technology Assessment (CCOHTA). International Journal of Technology Assessment in Health Care. 1995; 11(4): 796-797.

General HTA information/discussion

Using evidence to guide practice. Merec Briefing 2005;(30): 1-8. Concepts/methodology pertaining to EBM/HTA – not rapid review specific

Ahern F and O'Doherty N. Health technology assessment in Ireland. International Journal of Technology Assessment in Health Care 2000; 16(2): 449-458.

Development of EBM/HTA General HTA information/discussion

Alam M and Talha M. Evidence-based medicine: medical practice in the third millennium. Jcpsp, Journal of the College of Physicians & Surgeons - Pakistan 2005; 15(1): 57-59.

Concepts/methodology pertaining to EBM/HTA – not rapid review specific

Antes G, Sauerland S, Seiler CM. Evidence-based medicine - From best research evidence to a better surgical practice and healthcare. Langenbecks Archives of Surgery. 2006; 391(2): 61-67.

General EBM information/discussion

Atkins D and Diguiseppi CG. Broadening the evidence base for evidence-based guidelines: A research agenda based on the work of the U.S. preventive services task force. American Journal of Preventive Medicine. 1998; 14(4): 335-344.


Atkins D, Briss PA, Eccles M, Flottorp S, Guyatt GH, Harbour RT, Hill S, Jaeschke R, Liberati A, Magrini N, Mason J, O'Connell D, Oxman AD, Phillips B, Schunemann H, Edejer TT, Vist GE, Williams JW, Jr., GRADE Working Group. Systems for grading the quality of evidence and the strength of recommendations II: pilot study of a new system. BMC Health Services Research 2005; 5(1): 25.


Aveyard P. Evidence-based medicine and public health. Journal of Evaluation in Clinical Practice 1997; 3(2): 139-144.


Awonuga AO, Dwarakanath LS, Johanson R, Hyde C, Taylor R, Khan KS. Critical appraisal workshops to promote evidence-based healthcare. Journal of Obstetrics & Gynaecology. 2000; 20(1): 10-14.

EBM/HTA critique

Babidge W and Maddern G. Evidence-based surgery at ASERNIP-S. Can this improve quality in surgical practice? Journal of Quality in Clinical Practice. 2000; 20(4): 164-166.


Berguer R. The evidence thing. Annals of Vascular Surgery. 2004; 18(3): 265-270.


Bhandari M and Giannoudis PV. Evidence-based medicine: What it is and what it is not. Injury. 2006; 37(4): 302-306.


Bie NO. The lay perspective in health technology assessment. International Journal of Technology Assessment in Health Care. 1996; 12(3): 511-517.


Black N. Evidence-based surgery: A passing fad? World Journal of Surgery. 1999; 23(8): 789-793.

General EBM information/discussion EBM/HTA critique

Blanpain JE. Health technology assessment in Belgium, France, Germany and Japan. Acta Hospitalia 1983; 3: 37-44.

Development of EBM/HTA

Boba A and Mills FG. On evidence-based medicine. CMAJ: Canadian Medical Association Journal. 1998; 159(7): 758.


Bos M. Health technology assessment in The Netherlands. International Journal of Technology Assessment in Health Care 2000; 16(2): 485-519.


Bruckel J. Evidence-based medicine and rapid response team implementation. Mcgill Journal of Medicine. 2006; 9(1): 5-7.


Burger I, Sugarman J, Goodman SN. Ethical issues in evidence-based surgery. Surgical Clinics of North America. 2006; 86(1): 151-168.


Cleemput I and Kesteloot K. Health technology assessment in Belgium. International Journal of Technology Assessment in Health Care 2000;


71


16(2): 325-346. Cox K. Evidence-based medicine and everyday reality. Medical Journal of Australia 2001; 175(7): 382-383.


Daya S. Evidence-based medicine. Middle East Fertility Society Journal. 1998; 3(1): 4-8.


De Vet HCW, Kroese MEAL, Scholten RJPM, Bouter LM. A method for research programming in the field of evidence-based medicine. International Journal of Technology Assessment in Health Care. 2001; 17(3): 433-441.


Diamond LH. Technology assessment: Relationship to coverage issues - A case for temporary coverage decisions. American Journal of Kidney Diseases. 1999; 33(1): 208-210.

Not rapid review specific

Djulbegovic B, Hozo I, Lyman GH. Linking evidence-based medicine therapeutic summary measures to clinical decision analysis. Medscape General Medicine 2000; 2(1): E6.


Doherty S. History of evidence-based medicine. Oranges, chloride of lime and leeches: barriers to teaching old dogs new tricks. Emergency Medicine Australasia 2005; 17(4): 314-321.


Doherty S. Evidence-based medicine: arguments for and against. Emergency Medicine Australasia 2005; 17(4): 307-313.


Eitel F and Steiner S. Evidence-based learning. Medical Teacher. 1999; 21(5): 506-512.

Medical education

El Ansari W and Weiss ES. Quality of research on community partnerships: Developing the evidence base. Health Education Research. 2006; 21(2): 175-180.

Research partnerships Not rapid review specific

Eldar R. Health technology: Challenge to public health. Croatian Medical Journal. 2002; 43(4): 470-474.


Facey K. Health technology assessment in Scotland. Journal of Clinical Excellence. 2002; 4(1 suppl.): 100-101.


France G. Health technology assessment in Italy. International Journal of Technology Assessment in Health Care 2000; 16(2): 459-474.


Georgiou A. Data, information and knowledge: The health informatics model and its role in evidence-based medicine. Journal of Evaluation in Clinical Practice. 2002; 8(2): 127-130.


Hailey DM and Roseman C. Healthcare technology in Australia and New Zealand: Contrasts and cooperation. Health Policy. 1990; 14(3): 177-189.


Halley D and Menon D. A short history of INAHTA. International Journal of Technology Assessment in Health Care. 1999; 15(1): 236-242.


Jacob R and McGregor M. Assessing the impact of health technology assessment. International Journal of Technology Assessment in Health Care 1997; 13(1): 68-80.


Juzwishin D, Olmstead D, Menon D. Hospital-based technology assessment programmes: two Canadian examples. World Hospitals & Health Services 1996; 32(2): 2-9.


Kazanjian A and Green CJ. Health technology assessment within a public accountability framework. Clinical Governance: an International Journal. 2004; 9(1): 51-58.


Lauslahti K, Roine R, Semberg V, Kekomaki M, Konttinen M, Karp P. Health technology assessment in Finland. International Journal of Technology Assessment in Health Care 2000; 16(2): 382-399.


Liaropoulos L and Kaitelidou D. Health technology assessment in Greece. International Journal of Technology Assessment in Health Care 2000; 16(2): 429-448.


Lohr KN, Eleazer K, Mauskopf J. Health policy issues and applications for evidence-based medicine and clinical practice guidelines. Health Policy. 1998; 46(1): 1-19.


Maddern GJ, Middleton PF, Tooher R, Babidge WJ. Evaluating new surgical techniques in Australia: the Australian Safety and Efficacy Register of New Interventional Procedures-Surgical experience. Surgical Clinics of North America 2006; 86(1): 115-128.


Melnyk BM. Strategies for overcoming barriers in implementing evidence- Outline of future research agenda – not

72


based practice. Pediatric Nursing 2002; 28(2): 159-161. rapid review specific Menon D and Marshall D. The internationalization of health technology assessment. International Journal of Technology Assessment in Health Care. 1996; 12(1): 45-51.

Development of EBM/HTA Outline of future research agenda – not rapid review specific

Menon D and Topfer LA. Health technology assessment in Canada. A decade in review. International Journal of Technology Assessment in Health Care 2000; 16(3): 896-902.


Oortwijn WJ, Vondeling H, Bouter L. The use of societal criteria in priority setting for health technology assessment in the Netherlands: Initial experiences and future challenges. International Journal of Technology Assessment in Health Care. 1998; 14(2): 226-236.


Orvain J, Xerri B, Matillon Y. Overview of health technology assessment in France. International Journal of Technology Assessment in Health Care. 2004; 20(1): 25-34.


Perleth M and Busse R. Health technology assessment in Germany. Status, challenges, and development. International Journal of Technology Assessment in Health Care 2000; 16(2): 412-428.


Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, Woolacoot N, Glanville J. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technology Assessment 2004; 8(36): iii-iiv.

Guideline development Not rapid review specific

Philips Z, Bojke L, Sculpher M, Claxton K, Golder S. Good practice guidelines for decision-analytic modelling in health technology assessment: a review and consolidation of quality assessment. Pharmacoeconomics 2006; 24(4): 355-371.

Guideline development Not rapid review specific

Pinto MM, Ramos F, Pereira J. Health technology assessment in Portugal. International Journal of Technology Assessment in Health Care 2000; 16(2): 520-531.


Pivik J, Rode E, Ward C. A consumer involvement model for health technology assessment in Canada. Health Policy. 2004; 69(2): 253-268.


Rutten F. Health technology assessment and policy from the economic perspective. International Journal of Technology Assessment in Health Care 2004; 20(1): 67-70.


Sigmund H and Kristensen FB. Does health technology assessment benefit health services and politics? The experiences of an established HTA institution: The Danish centre for evaluation and HTA. European Journal of Health Economics. 2002; 3(1): 54-58.


Stein K, Dalziel K, Garside R, Castelnuovo E, Round A. Association between methodological characteristics and outcome in health technology assessments which included case series. International Journal of Technology Assessment in Health Care 2005; 21(3): 277-287.


Stevens AJ, Raftery J, Roderick P. Can health technologies be assessed using routine data? International Journal of Technology Assessment in Health Care. 2005; 21(1): 96-103.


Wild C. Health technology assessment in Austria. International Journal of Technology Assessment in Health Care. 2000; 16(2): 303-324.


Williams AH and Cookson RA. Equity-efficiency trade-offs in health technology assessment. International Journal of Technology Assessment in Health Care. 2006; 22(1): 1-9.


Wilson M. Evidence-Based Medicine - Would you rather read 50 000 articles or 120? Evidence Based Medicine. 2006; 11(2): 40.


73


Studies excluded after full text appraisal

Reference Focus of paper/basis for exclusion Abrams HL and Hessel S. Health technology assessment: problems and challenges. AJR: American Journal of Roentgenology 1987; 149(6): 1127-1132.

Development of EBM/HTA Evaluation of primary research

Antes G and Diener MK. The role of systematic reviews in evidence-based healthcare. Chinese Journal of Evidence-Based Medicine. 2006; 6(7): 467-468.


Atkins D, Fink K, Slutsky J, Agency for Healthcare Research and Quality, North American Evidence-based Practice Center. Better information for better healthcare: the Evidence-based Practice Center program and the Agency for Healthcare Research and Quality. Annals of Internal Medicine 2005; 142(12:Pt 2): t-41.

Details of AHRQ reports and experiences with HTA (report timeframe: within 12 months)

Banta D. The development of health technology assessment. Health Policy. 2003; 63(2): 121-132.


Battista RN, Lance JM, Lehoux P, Regnier G. Health technology assessment and the regulation of medical devices and procedures in Quebec. Synergy, collusion, or collision? International Journal of Technology Assessment in Health Care 1999; 15(3): 593-601.

Details of The Quebec Health Technology Assessment Council’s experiences with HTA – not rapid review specific Policy implications and healthcare reform

Battista RN. Expanding the scientific basis of health technology assessment: A research agenda for the next decade. International Journal of Technology Assessment in Health Care. 2006; 22(3): 275-280.


Behney CJ. Some theories to explain the origin and rapid growth of health technology assessment. International Journal of Technology Assessment in Health Care. 1991; 7(1): 116-118.


Berwick DM. Broadening the view of evidence-based medicine. Quality & Safety in Health Care. 2005; 14(5): 315-316.

EBM/HTA critique

Buetow S, Upshur R, Miles A, Loughlin M. Taking stock of evidence-based medicine: Opportunities for its continuing evolution. Journal of Evaluation in Clinical Practice. 2006; 12 (4): 399-404.

General HTA information/discussion EBM/HTA critique

Busse R, Orvain J, Velasco M, Perleth M, Drummond M, Gurtner F, Jorgensen T, Jovell A, Malone J, Ruther A, Wild C. Best practice in undertaking and reporting health technology assessments: Working group 4 report. International Journal of Technology Assessment in Health Care. 2002; 18(2): 361-422.


Caine N. Health technology assessment - Principles, pointers and problems. Perfusion. 1997; 12(4): 269-272.

Details of NHS HTA experiences – not rapid review specific

Carlsson P, Jonsson E, Werko L, Banta D. Health technology assessment in Sweden. International Journal of Technology Assessment in Health Care. 2000; 16(2): 560-575.

Details of HTA in the Swedish healthcare system (SBU) (report timeframe: 2-3 years)

Carlsson P, Hultin H, Törnwall J. The early experiences of a national system for the identification and assessment of emerging healthcare technologies in Sweden. International Journal of Technology Assessment in Health Care. 1998; 14(4): 687-694.

Focus on ‘SBU Alert’. Does not meet definition of rapid review.

Cookson R and Maynard A. Health technology assessment in Europe. Improving clarity and performance. International Journal of Technology Assessment in Health Care 2000; 16(2): 639-650.

Details of HTA experience in Europe Outline of future research agenda – not rapid review specific

Dixon S, Coleman P, Nicholl J, Brennan A, Touch S. Evaluation of the impact of a technology appraisal process in England: the South and West Development and Evaluation Committee. Journal of Health Services & Research Policy 2003; 8(1): 18-24.

Evaluation of impact of NHS South and West region Development and Evaluation Committee (DEC) technology appraisal reports

Douw K and Vondeling H. Selection of new health technologies for assessment aimed at informing decision making: A survey among horizon scanning systems. International Journal of Technology Assessment in Health Care 2006; 22(2): 177-183.

Survey of methods used for prioritising technologies for assessment

Draborg E and Gyrd-Hansen D. Time-trends in health technology assessments: an analysis of developments in composition of international health technology assessments from 1989 to 2002. International Journal of Technology Assessment in Health Care 2005; 21(4): 492-498.

Assessment of full HTA reports only

Draborg E and Andersen C. What influences the choice of assessment methods in health technology assessments? Statistical analysis of

Assessment of full HTA reports only

74


international health technology assessments from 1989 to 2002. International Journal of Technology Assessment in Health Care 2006; 22(1): 19-25. Elphick HE and Smyth RL. Research: The principles of evidence-based medicine. Current Paediatrics. 2002; 12(4): 325-330.


Goodman CS and Ahn R. Methodological approaches of health technology assessment. International Journal of Medical Informatics. 1999; 56(1-3): 97-105.

HTA methodology not pertaining to rapid review

Hailey D and Juzwishin D. Managing external risks to health technology assessment programs. International Journal of Technology Assessment in Health Care. 2006; 22(4): 429-435.

Risk management – does not pertain specifically to rapid review

Hailey DM, Cowley DE, Dankiw W. The impact of health technology assessment. Community Health Studies. 1990; 14(3): 223-234.

Does not address rapid review

Liberati A, Sheldon TA, Banta HD. EUR-ASSESS Project Subgroup report on Methodology. Methodological guidance for the conduct of health technology assessment. International Journal of Technology Assessment in Health Care 1997; 13(2): 186-219.


Milne R, Clegg A, Stevens A. HTA responses and the classic HTA report. Journal of Public Health Medicine. 2003; 25(2): 102-106.


Perry S, Gardner E, Thamer M. The status of health technology assessment worldwide. Results of an international survey. International Journal of Technology Assessment in Health Care 1997; 13(1): 81-98.

Does not address rapid review

Rotstein D and Laupacis A. Differences between systematic reviews and health technology assessments: a trade-off between the ideals of scientific rigor and the realities of policy making. International Journal of Technology Assessment in Health Care 2004; 20 (2): 177-183.

Does not consider the impact of rapid reviews on policy

Stevens A, Colin-Jones D, Gabbay J. 'Quick and clean': authoritative health technology assessment for local health care contracting. Health Trends 1995; 27(2): 37-42.

Does not consider time-frame of report preparation

Stevens A, Milne R, Burls A. Health technology assessment: history and demand. Journal of Public Health Medicine 2003; 25(2): 98-101.

General HTA information/discussion Concepts/methodology pertaining to EBM/HTA – not rapid review specific

Topfer L-A, Parada A, Menon D, Noorani H, Perras C, Serra-Prat M. Comparison of literature searches on quality and costs for health technology assessment using the MEDLINE and EMBASE databases. International Journal of Technology Assessment in Health Care. 1999; 15(2): 297-303.

Focus on cost of data searches – not rapid review specific

Vallance-Owen A and Warren V. Evaluation of rapid assessment of interventional procedures in the UK independent sector. 2004.

Comparison of rapid assessments (within 48 hours) to NICE IP reports

75


APPENDIX D – HTA AGENCIES SEARCHED

76


Appendix D – HTA agencies searched The websites of health technology agencies visited to identify rapid reviews are detailed below.

INAHTA member organisations AETMIS (Canada) http://www.aetmis.gouv.qc.ca/site/index.php?accueil AETS (Spain) http://www.isciii.es/htdocs/index.jsp AETSA (Spain) http://www.juntadeandalucia.es/salud/orgdep/aetsa/ AHFMR (Canada) http://www.ahfmr.ab.ca/

(former INAHTA member; information on HTA activities now available through IHE website) AHRQ (USA) http://www.ahrq.gov/ AHTA (Australia) http://www.health.adelaide.edu.au/publichealth/

consult/health_techn_assess.html AHTAPol (Poland) http://www.aotm.gov.pl/# ASERNIP-S (Australia)

http://www.surgeons.org/Content/NavigationMenu/Research/ASERNIPS/default.htm

AVALIA-T (Spain) http://avalia-t.sergas.es/CADTH (formerly CCOHTA) (Canada)

http://www.cadth.ca/ http://dev.ccohta.ca/entry_e.html

CAHTA (Spain) http://www.aatrm.net/html/ca/Du8/index.html CEDIT (France) http://cedit.aphp.fr/ CENETEC (Mexico) http://www.cenetec.gob.mx/htmls/cenetec.html# CMT (Sweden) http://www.cmt.liu.se/ CRD (UK) http://www.york.ac.uk/inst/crd/ CVZ (The Netherlands)

http://www.cvz.nl/

DACEHTA (Denmark) http://www.sst.dk/Planlaegning_og_behandling/Medicinsk_teknologivurdering.aspx?lang=en

DAHTA @ DIMDI (Germany)


DECIT-CGATZ (Brazil)

http://portal.saude.gov.br/portal/saude/area.cfm?id_area=1088

DSI (Denmark) http://www.dsi.dk/ FinOHTA (Finland) http://finohta.stakes.fi/FI/index.htm GR (The Netherlands) http://www.gr.nl/index.php HAS (ANAES) (France)

http://www.has-sante.fr/anaes/anaesparametrage.nsf/HomePage?ReadForm

HunHTA (Hungary) http://hecon.uni-corvinus.hu/ IAHS (UK) http://www.abdn.ac.uk/iahs/ ICTAHC (Israel) http://www.health.gov.il/english/pages_e/default.asp

?pageid=28&parentid=15&catid=13&maincat=2 IECS (Argentina) http://www.iecs.org.ar/ IHE (Canada) http://www.ihe.ca/hta/index.html IMSS (Mexico) http://www.imss.gob.mx/IMSS/IMSS_SITIOS/DPM/Informacion/Tecnologia/Principal.htm KCE (Belgium) http://kce.fgov.be/index_en.aspx?SGREF=5211 LBI @ ITA (Austria) http://hta.lbg.ac.at/de/index.php MAS/OHTAC (Canada)

http://www.health.gov.on.ca/english/providers/program/mas/mas_mn.html

MSAC (Australia) http://www.msac.gov.au/ MTU-SFOPH (Switzerland)

http://www.bag.admin.ch/

NCCHTA (UK) http://www.ncchta.org/ NHSC (UK) http://www.pcpoh.bham.ac.uk/publichealth/horizon/ NHS QIS (UK) http://www.nhshealthquality.org NOKC (Norway) http://www.kunnskapssenteret.no/ NZHTA (New Zealand)

http://nzhta.chmeds.ac.nz/

OSTEBA (Spain) http://www.osanet.euskadi.net/r85-2905/es/contenidos/informacion/presentacion_osteba/es_1199/indice_c.html

SBU (Sweden) http://www.sbu.se/www/index.asp UETS (Spain) http://www.madrid.org/lainentralgo/estudios/

marcevalua/ffevalua.htm

77

http://www.ahfmr.ab.ca/

http://www.health.adelaide.edu.au/publichealth/

http://www.surgeons.org/Content/Navigation


http://www.sst.dk/Planlaegning_og_behandling

http://www.has-sante.fr/anaes/anaesparametrage.nsf/

http://www.health.gov.il/english/pages_e/default.asp


VATAP (USA) http://www.va.gov/vatap/ VSMTVA (Latvia) http://www.vsmtva.gov.lv/v/en/v/en/agentura/struktura/index.aspxZonMW (The Netherlands)

http://www.zonmw.nl/

Other international HTA organisations AAS Austria http://www.oeaw.ac.at/ BCBS US http://www.bcbs.com/tec/index.html CCE Australia http://www.mihsr.monash.org/cce/ CEA USA http://www.tufts-nemc.org/cearegistry/publications.html CHE Monash Australia

http://www.buseco.monash.edu.au/centres/che/

CHEPA Canada http://www.chepa.org/Home/tabid/36/Default.aspx CHSPR Canada http://www.chspr.ubc.ca/publications CTFPHC Canada http://www.ctfphc.org/ ECRI US http://www.ecri.org/ ETESA Chile http://www.minsal.cl/ HAYES Inc. USA http://www.hayesinc.com/ HSA Singapore http://www.hsa.gov.sg/ HUI Canada http://www.fhs.mcmaster.ca/hug/ ICES Canada http://www.ices.on.ca/webpage.cfm ICSI USA http://www.icsi.org/index.asp Kaiser Permanente http://www.kaiserpermanente.org/ MHH Germany http://www.mh-hannover.de/ MHTAU Malaysia http://www.moh.gov.my/ MUHC Canada http://www.mcgill.ca/tau/ NICE UK http://www.nice.org.uk/ NHS HTA UK http://www.hta.nhsweb.nhs.uk/ OHTAC Canada http://www.health.gov.on.ca/english/providers/program/mas/ohtac_about.html SNHTA Switzerland http://www.snhta.ch/home/portal.php TA-SWISS Switzerland

http://www.ta-swiss.ch/framesets/home-e.htm

TNO The Netherlands

http://www.tno.nl/tno/index.xml

UHC USA http://www.uhc.edu/ WIHRD UK http://www.wihrd.soton.ac.uk/ WMHTAC UK http://www.pcpoh.bham.ac.uk/publichealth/wmhtac/index.htm WorkSafe BC Canada

http://www.worksafebc.com/

WSDLI USA http://lni.wa.gov/

78


APPENDIX E – EXCLUDED AGENCIES

79

Appendix E – Excluded agencies Agency Reason for exclusion AAS Austria Foreign language AETMIS Other type of product AETS Spain Foreign language AETSA Spain Foreign language AHRQ Agency was contacted and was there was no reply AHTA Australia Other type of product AHTA Pol Issues with access to website AVALIA-T Foreign language CADTH Other type of product* CAHTA Spain Foreign language CEA USA Issues with access to website CEDIT Other type of product CENETEC Foreign language CHE Monash Australia Other type of product CHEPA Canada Other type of product CHSPR Canada Other type of product CMT Sweden Foreign language CRD UK Other type of product, topics not relevant to surgical interventions CTFPHC Canada Other type of product CVZ The Netherlands Foreign language DACEHTA Foreign language DAHTA @ DIMDI Germany Foreign language DECIT-CGATZ Foreign language DSI Denmark Foreign language ECRI US Other type of product ETESA Chile Foreign language FinOHTA Finland Foreign language GR The Netherlands Other type of product HAS France Foreign language HAYES Inc. USA Fee For Product HUI Canada Other type of product HunHTA Hungary Foreign language IAHS UK Other type of product ICES Canada Other type of product, topics not relevant to surgical interventions ICTAHC Israel Foreign language IECS Argentina Foreign language KCE Belgium Issues with access to website LBI @ ITA Austria Foreign language MAS Canada Other type of product MHH Germany Other type of product MHTAU Malaysia Foreign language MSAC Australia Other type of product


MTU-SFOPH Foreign language NCCHTA UK Other type of product NHS HTA UK Other type of productNHS QIS UK Other type of product NHSC UK Other type of product NOKC Norway Foreign language OSTEBA Spain Foreign language SBU Sweden Foreign language SNHTA Switzerland Issues with access to website TA-SWISS Switzerland Foreign language TNO The Netherlands Other type of product UETS Spain Issues with access to website UHC USA Issues with access to website VATAP Topics not relevant to surgical interventions VSMTVA Foreign language WIHRD UK Other type of product WMHTAC UK Other type of product WorkSafe BC Canada Other type of product WSDLI USA Other type of product ZonMW The Netherlands Other type of product

82

APPENDIX F – PRODUCT DETAILS


Appendix F – Details of products from included websites The tables in appendix F summarise the titles of the topics scoped for by publications and the results of the CRD search used to identify comparators for rapids with these titles. In the case of topics, four were included; however, others were initially looked at. These are described in the table notes and were excluded in most cases where topics did not match criteria as well as others.

Alberta Heritage Foundation for Medical Research Alberta Heritage Foundation for Medical Research (AHFMR) was scoped according to the inclusion criteria. Eight topics were identified. Issues regarding identifying comparator reviews included foreign language reviews, accessibility to fee-for-product agencies or no matches were found. Institute of Clinical Systems Improvement (ICSI) were contacted regarding their product “Technotes”. This was defined as a rapid review.

Year of Publication Title of Publication Search terms for CRD Database

Excluded Result of CRD Database Search

Technotes 2001 Off-Pump Coronary Artery Bypass

Surgery* Off pump coronary artery bypass surgery

No (ICSI Product appears to be rapid)

ICSI (2003) ASERNIP-S (2001)

2002 Metal-on-metal hip resurfacing for young, active adults with degenerative hip disease*

Hip resurfacing No (CCE appears to be rapid)

CCE (2002) NHSHTA (2002)

2002 Surgical treatment for Chronic Venous Insufficiency

Chronic venous insufficiency

Yes No matches found

2002 Cryosurgery for Prostate Cancer cryosurgery Yes Foreign language Hayes Accessibility

2002 Intracoronary Brachytherapy for the treatment of In-stent restenosis

brachytherapy Yes (ICSI product seems to be rapid)

BSBS (2002) Accessibility

2002 Treatment of Thoracic Insufficiency Syndrome with The Vertical Expandable Prosthetic Titanium Rib

Thoracic insufficiency syndrome Prosthetic titanium rib


2002 Trigger point injections for non-malignant chronic pain

Trigger points injections Chronic pain


2003 Sclerotherapy for Varicose Veins of the Legs

sclerotherapy Yes Hayes (2004) Accessibility ANAES (2004) Foreign language

2004 Adult to Adult Living Donor Liver Transplantation*

Living donor liver transplantation

No ASERNIP-S (2004) CCOHTA (2003)

Note: * Included as topics for comparison

84

Agency for Healthcare Research and Quality All topics scoped from AHRQ were excluded on the basis that no matches for pairing were found.

Year of Publication

Title of Publication Search terms for CRD Database


Technology Assessment 2002 Liver Transplantation for Patients

with Hepatobiliary Malignancies other than Hepatocellular Carcinoma

Liver transplantation

Yes No Matches Found

2002 Morbidity and Mortality Among Adult Living Donors Undergoing Right Hepatic Lobectomy for Adult Recipients

Right Hepatic Lobectomy


2003 Acupuncture for Fibromyalgia Acupuncture for Fibromyalgia


2003 Acupuncture for Osteoarthritis Acupuncture for Osteoarthritis


2004 Diagnosis and Treatment of Vulnerable Plaques

Vulnerable Plaques Yes No Matches Found

2004 Percutaneous Myocardial Revascularization/ Transmyocardial Revascularization

Myocardial Revascularisation/ Myocardial Revascularization


2004 Treatments for Benign Prostatic Hyperplasia

Benign Prostatic Hyperplasia


Australian Safety and Efficacy Register of New Interventional Procedures, Surgical Four Accelerated Systematic Reviews were listed on the ASERNIP-S website published during 2001 to 2004.

Year of Publication


Excluded: Other Rapid Products?

Result of CRD Database Search

Technotes 2003 Vacuum-Assisted Closure for

the Management of Wounds* Vacuum assisted closure

No CADTH CCE

AHRQ (2004)

2003 Spinal Cord Stimulation Spinal Cord Stimulation

Maybe (depth of WCB Full Systematic Review)

HAYES (2003) WCB Evidence Based Group (2003)

2003 Implantable Spinal Infusion Devices for Chronic Pain and Spasticity

Spinal Infusion Devices

Yes No results found

2004 Laparoscopic Ventral Hernia Repair

Laparoscopic Ventral Hernia

Yes No results found

NOTE: * Included as topic for comparison


Blue Cross Blue Shield Year of Publication


Excluded Other Rapid Products?

Result of CRD Database Search

TEC Assessments 2004 Percutaneous vertebroplasty

for vertebral fractures caused by osteoporosis and malignancy, or hemangioma

Percutaneous verterbroplasty


2004 Carotid artery with distal embolic protection of the cerebral circulation

Carotid distal embolic protection


2004 Extracorporeal shock wave treatment for chronic tendonitis of the elbow (lateral epicondylitis)†

Extracorporeal shock wave treatment

No WCB Evidence Based Group (2004)

2004 Extracorporeal shock wave treatment for chronic plantar fasciitis†

Extracorporeal shock wave treatment

No ICSI (2004) WCB Evidence Based Group (2004)

NOTE: † included as topic for comparison, however, not in final four topics

86

Monash Centre for Clinical Excellence Three different types of report were identified when searching the CCE website: Literature searches; Critical Appraisals and Full Reports. Critical Appraisals seemed to be the most appropriate as an example of rapid review. There were 88 Critical Appraisals in total; however, after these were narrowed to the years 2001-2004, 56 critical appraisals were left. These were narrowed down according to topic (i.e. surgical intervention) and 7 critical appraisals were remaining.

Year of Publication


Excluded: Result of CRD Database Search

Critical Appraisals 2001 Needleless versus conventional

intravenous systems Needleless intravenous systems


2001 Electroconvulsive therapy for bipolar mood disorders during pregnancy

Electroconvulsive Therapy for Bipolar


2001 Heel prick versus venipuncture for blood sampling in neonates

Blood Sampling in Neonates


2002 Flushing of PICC lines Flushing Yes No Matches Found

2002 ‘Open’ versus ‘closed’ systems for enteral feeding

Enteral Feeding Yes NICE (2002) not full text No suitable Matches Found

2002 Hip resurfacing in patients with osteoarthritis*

Hip resurfacing No AHFMR (2002) NHSHTA (2002)

2003 Vasospasm following subarachnoid haemorrhage: papaverine infusion, nimodopine infusion or balloon angioplasty?

Vasospasm Yes No Matches Found

NOTE Due to difficulty further classifying only these 7 were included at this stage, however, there may be several others that would be appropriate as comparable products. *topic included for comparison

After correspondence with CCE, it was discovered that the products titled ‘Critical Appraisals’ were no longer conducted and that ‘Evidence Reports’ were now the ‘rapid-like’ products produced by CCE.


Institute for Health Economics A range of products were found when searching the IHE website: 1) HTA Initiatives 2) HTA series A 2) HTA series B 3) HTA Info Paper 4) Presentations 5) Report of Activities 6) Technotes and 7) Techwise Newsletter. Results were limited to 23 Technotes when restricting to years 2001-2004 and after restricting according to type of intervention, that is, merely surgical, 7 Technotes were highlighted.

The resources of IHE and AHFMR were integrated and as such the products of interest for both AHFMR and IHE are listed earlier under AHFMR.

Institute of Clinical Systems Improvement

Year of Publication



Tech Assessments 2002 Pancreatic islet transplantation for

Patients with Type 1 Diabetes Mellitus†

Pancreatic islet transplantation

No MAS (2003) AHRQ (2003) Rapid: NICE Overview (2003)

2003 Uterine Artery Embolization for Uterine Fibroids

Uterine artery embolisation/ embolization


2003 Pancreas transplant for Insulin-Dependent Diabetes†

Pancreas transplant

No MAS (2003) AHRQ (2003)

2003 Lung Volume Reduction Surgery for Emphysema*

Lung volume reduction surgery

No CCOHTA (2005) Rapid: NICE (2004)

2003 Drug-Eluting Stents for the Prevention of Restenosis in Native Coronary Arteries*

Drug eluting stents No NICE (2003) Rapid: CADTH (2003)

2004 Vertebroplasty and Balloon-assisted vertebroplasty for the treatment of osteoporotic compression fractures†

Vertebroplasty No AHRQ (2005) Rapid: BCBS (2005)

2004 Fluoroscopically guided transforaminal epidural steroid injections for lumbar radicular pain

Transforaminal epidural steroid injections


NOTE: * Topic(s) included for comparison; † Topics included for pairing but not in final four

88

McGill University Health Center Year of Publication


Excluded

Rapid Result of CRD Database Search

Reports 2003 Should the MUHC approve the use

of colorectal stents? Colorectal Stents Yes No Matches

Found 2003 Should the MUHC approve the vide

capsule endoscopy system in the diagnosis of small bowel abnormalities?

Video Capsule Endoscopy

Yes CADTH (2003)

No Matches Found

2003 An evaluation of drug eluting (coated) stents for percutaneous coronary interventions; What should their role be at the McGill University Health Centre (MUHC)?*

Drug Eluting Stents No CADTH (2002) ICSI (2003)

NICE (2003)

2003 The use of Self-expanding Metallic Stents in the Palliation of Dysphagia in Patients with Malignant Esophageal Strictures

Metallic Stents Yes No Matches Found

2004 The use of Matrix Coils in the treatment of Cerebro-vascular Aneurysms

Matrix Coils Yes No Matches Found

2004 The Gastric Banding Procedure. An Evaluation†

Gastric Banding No BCBS (2005) NZHTA (2005)

MAS (2005) IHE (2005)

2004 Use of carmustine implants (Gliadel wafer) in patients with malignant glioma at the McGill University Health Centre

Carmustine implants/ malignant glioma


NOTE: * Topic(s) included for comparison; † Topic(s) included for pairing but not in final four


National Institute for Clinical Excellence Year of Publication


Excluded: Result of CRD Database Search

Overviews 2003 Laparoscopic hysterectomy Laparoscopic

hysterectomy Yes No Matches

Found 2004 Arteriovenous sheathotomy for

branch retinal vein occlusion Arteriovenous sheathotomy


2004 Endovascular closure of atrial septal defect

Endovascular closure/ atrial septal defect


2004 Endovascular closure of patent ductus arteriosus

patent ductus arteriosus


2004 Photodynamic therapy for high-grade dysplasia in Barrett’s oesophagus

Photodynamic therapy

Yes No Suitable Matches Found

2004 Wireless capsule endoscopy for investigation of the small bowel‡

Wireless capsule endoscopy

Maybe MAS (2003)

2004 Percutaneous disc decompression using coblation for lower back pain

Percutaneous disc decompression/ coblation


2004/5? Coil embolisation of ruptured intracranial aneurysms‡

ruptured intracranial aneurysms

Maybe Year matching

2004 Transobturator foramen procedures for stress urinary incontinence

stress urinary incontinence


2004 Gastroelectrical stimulation for gastroparesis

gastroparesis Yes Hayes (Accessibility)

2004 Percutaneous closure of patent foramen ovale for the prevention of cerebral embolic stroke

Percutaneous closure


2004 Overview of carotid artery stent placement for carotid stenosis‡

Carotid stenosis Maybe 2x Rapid products found BCBS (2005) & CCOHTA (2005)

2004 Minimally Invasive two-incision surgery for total hip replacement

Total hip replacement Yes No Matches Found

2004 Mini-incision surgery for total knee replacement‡

Total knee replacement

Maybe- date of full review

MAS (2005)

2004 Artificial metacarpophalangeal and interphalangeal joint replacement for end-stage arthritis‡

Metacarpophalengeal and interphalangeal joint replacement

Maybe MAS (2004) Replacement product (ceramic vs. silicon base) Time comparison

2004 Lung reduction volume surgery for advanced emphysema*

Lung reduction volume surgery

No CCOHTA (2005)

2004 Endoluminal gastroplication for gastro-oesophageal reflux disease

Endoluminal gastroplication


2004 Artificial trapeziometacarpal joint replacement for end-stage osteoarthritis

trapeziometacarpal joint replacement


2004 Low dose rate brachytherapy for localised prostate cancer

Brachytherapy Yes No Matches Found

90

2004 Interspinous distraction procedures for spinal stenosis causing neurogenic claudication in the lumbar spine

Interspinous distraction procedures/ neurogenic claudication


2004 Cryoablation for atrial fibrillation as an associated procedure with other cardiac surgery

Cryoablation for atrial fibrillation


NOTE: * Topic(s) included for comparison; ‡ Topics for possible inclusion but not used for comparison

New Zealand Health Technology Assessment

Year of Publication



Technical Briefs 2004 Review of the effectiveness of

infrared thermal imaging (thermography) for population screening and diagnostic testing of breast cancer

Thermography and breast cancer



92

APPENDIX G – DATA EXTRACTION TABLES

Appendix G – Data extraction tables Drug eluting stents

Type RAPID RAPID FULLAgency and product information Name of agency ICSI MUHC NICEProduct Technology Assessment Report Systematic Review Year of publication Dec 2003 Jul 2003 2003 (Date completed 18 Feb 2003) Title “Drug-Eluting Stents for the Prevention of

Restenosis in Native Coronary Arteries” “An evaluation of drug eluting (coated) stents for percutaneous coronary interventions; What should their role be at the McGill University Health Centre (MUHC)?”

“Coronary artery stents: Rapid Systematic Review and Economic Evaluation” (a previous NICE Review on this topic was completed 2000)

Review methodology Search Dates

NR NR(the request for the review was received in March 2003)

Dec 2002

No. of Pages 14 33 257

Treatment intervention Coated stent implantation Sirolimus-eluting stents Pacliitaxel- eluting stents

Coated stent implantation Sirolimus-eluting stents Paclitaxel-eluting stents

Coated stent implantation: Sirolimus Paclitaxel Actinomycin QP2 (7-hexanolytaxol) Everolimus

Comparator intervention Bare Metal Stent

slow dosage vs high dosage coated stent Bare metal stent Uncoated stent

3 separate comparators in whole review: - stent vs percutaneous transfemoral coronary angioplasty - stent vs CABG - stent vs drug-eluting stent

Detail of systematic literature search given

No NR Search terms and databases listed Separate strategies were used for clinical studies and economic studies

Study inclusion criteria: NR NR RCTs: Patients with: CAD in native/ graft vessels And: - stable angina/ acute coronary syndrome (ACS) and unstable angina Full economic evaluations that compared two or more options and consider both costs and consequences including cost-effectiveness, cost-utility analysis or

93

94

cost-benefit analysis undertaken in the context of high quality randomised controlled trials were included in the review.

Study exclusion criteria: NR NR NR

No. of authors 1 + 5 clinical working group members 1 11 Use of expert panel or peer review YES (expert panel) YES

(2 external reviewers and Technology Assessment Unit Committee)

3 expert clinical panel members

Type of evidence grading NR NR Centre for Reviews and Dissemination, York, Report 4

Evidence base used Systematic reviews 0 0 0 RCTs Sirolimus= 2

Paclitaxel= 2 Sirolimus= 2 Paclitaxel= 6

Sirolimus= 3 Paclitaxel= 9 QP2 (7-hexanolytaxol)=1 Everolimus=1 Actinomycin=1

Comparative (non-randomised) 2 NR Stradia=1 Tacrolimus=2 Paclitaxel=2 Dexamethasone=1

Case studies/ case reports 0 Sirolimus=4 Paclitaxel= 4

Other studies 0 0 0 Studies excluded from review: 4 5 NR Total meeting criteria 6 16 68 Outcomes RCTS:

Study 1: Sirolimus= 3 outcomes Study 2: Sirolimus= 4 outcomes Study 3: Paclitaxel=8 Study 4: Paclitaxel= 5 NON-RCTS: Study 1: Sirolimus= 4

Sirolimus= 8 outcomes Paclitaxel= 4 outcomes

Meta-analysis conducted on 5 outcomes

Collective number of patients included: 1622 Sirolimus= 2362 Paclitaxel= 2054

4367 +

Critical appraisal of included studies No No (studies discussed but not critically appraised)

Yes

Results Safety Yes

(1 para) - morbidity rate - mortality rate - training and experience required to perform procedure safely - where the procedure should be performed - co-morbidities that increase the risk assoc. with procedure - potential for inappropriate use of the technology

Yes - no reduction in mortality, myocardial infarction or bypass surgery rates with coated stents

Yes Major Adverse Cardiac Events:

- mortality - acute myocardial infarction - revascularisation

Effectiveness: No. of outcomes

Yes (discussed narratively for each study) Yes - discussed narratively relevant outcomes

Overall conclusions of studies are discussed narratively Meta-analysis on 5 outcomes

Economics 2 sentences discussing prices 2.5 pages cost effectiveness - decision analysis and sensitivity analysis calculations shown

Cost-effectiveness Review (14 pages) Also a critical appraisal of economic models submitted by device manufacturers (10 pages) Independent evaluation (33 pages) Budget impact analysis (6 pages)

Ethics NR NR NR Use of Meta-analysis NR YES Yes- 5 outcomes Overall conclusions 1. DES shown in prospective, RCT to significantly

decrease rate of restenosis, repeat revascularisation procedures and cardiac events up to 1 year post-treatment as compared with bare metal stents in selected native coronary lesions of a particular size that have not been previously stented 2. Presently no data from peer-reviewed RCTs to permit a conclusion about safety and efficacy of drug-eluting stents for other patient populations, other lesions sizes or other vessel diameters 3. Long-term safety and efficacy of DES unknown at this time.

Despite good evidence supporting efficacy of coated stents to reduce rates of restenosis, cost-effectiveness and safety is not desirable. Coated stents suggested for special cases only.

Studies not powered to measure mortality. Not adequate data on QOL. Cost of stents increasing for drug eluting stents. Long term data for drug eluting stents not clear. Careful patient selection and follow-up and re-appraisal and effectiveness of devices will be essential.

95

96

Lung volume reduction surgery Type RAPID RAPID FULLAgency and product information Name of agency NICE ICSI CCOHTA Product Interventional Procedures Programme Overview Technology Assessment Report (an Update of a

report published in 1997) Technology Report

Year of publication Sept 2004 Oct 2003 Dec 2004 Title “Interventional procedures overview of lung

volume reduction surgery for advanced emphysema”

“Lung volume reduction surgery for emphysema” “Comparison of Lung volume Reduction Surgery with Medical Management for Emphysema”

Review methodology

Search Dates

May 2004

From 1997 - 2003 Mar 2004

No. of Pages 19 (also a guidance document of 2 pages) 9 (The revised 1997 report is 16 pages, and the 1997 committee summary is 8 pages in length)

72

Treatment intervention Lung Volume Reduction Surgery Lung Volume Reduction Surgery Lung Volume Reduction Surgery

Comparator intervention Other Medical Intervention Medical therapy Medical Management


Yes (Lists databases but not search terms) NR Yes (Lists databases and search terms)

Study inclusion criteria: 1. Clinical studies- evidence on identifying good quality comparative studies 2. Patients with COPD 3. Intervention: LVRS (by any method) 4. Outcome: articles were retrieved if abstracts contained info relevant to safety and/or efficacy

NR RCTs: patients with emphysema Comparing LVRS with medical management Reported data on QOL, mortality, complications, dyspnoea, pulmonary function, exercise tolerance, blood gas levels Case Series: case studies were included if they reported data on complications of surgery or mortality Only case-studies published in English and involving 20 or more patients were considered.

Study exclusion criteria: 1. Abstracts with no reported clinical outcomes 2. Conference abstracts 3. Non-English language articles

NR NR

No. of authors NR 3 7 Use of expert panel or peer review Specialist advice sought from: Society of

Cardiothoracic Surgeons of Great Britain and Ireland and the British Thoracic Society

Technology Assessment Committee 5 (3 external reviewers, 2 scientific advisory panel members)

Type of evidence grading NR None Jahad 5-point scale for RCTs

Evidence base used Systematic reviews 1 0 0 RCTs 4 3 7Comparative (non-randomised) 0 0 0 Case studies/ case reports 1 (250 consecutive patients with long-term

follow-up) 3 59

Other studies 0 1 (cost effectiveness analysis) 0 Studies excluded from review: 8 NR NR Total meeting criteria 6 7 66 Outcomes Outcomes discussed narratively Outcomes discussed narratively Outcomes discussed narratively

- QOL - Dyspnoea - Pulmonary function - Exercise Tolerance - Blood Gases

Collective number of patients included: 1631 1930 RCTs: 1412 Case Series: 5000+ patients

Critical appraisal of included studies Yes (brief) No YES

Results Safety Yes

Complications: Persistent air leak from lung Pneumonia In-hospital mortality Myocardial Infarction Deep vein thrombosis Small bowel obstruction Phrenic Nerve Injury

Yes Morbidity; mortality; training required; where the procedure should be performed; co-morbidities that increase the risk associated with the procedure; potential for inappropriate use of the technology.

Yes Complications: Prolonged air leaks Infection Bleeding Pneumonia TIA Transient confusion Transient Horner syndrome Small Bowel Ileus Benign Dysrhythmia Atrial Fibrillation Mortality

Effectiveness: No. of outcomes

Yes - improvements in lung function, exercise

performance and quality of life are improved

- Specialist Advisors considered efficacy to be well established for a select group of

Each study discussed narratively Yes - improvements in QOL, pulmonary function, exercise tolerance

97

98

patients ~19 outcomes

Economics NR Economic data from 1 study discussed narratively

(<0.5 page) Economic data from 1 study are discussed narratively (1 para)

Ethics NR NR NRUse of Meta-analysis NR NR YES

outcomes meta-analysed include: - death - dyspnoea - lung volume (3 outcomes) - pulmonary function - exercise capacity

Overall conclusions More RCTs required. There is no overall conclusion regarding the safety and effectiveness of LVRS.

LVRS may be an improvement on medical therapy for some patients, but that the results cannot be extended to the general population of patients with emphysema. The authors do not provide a comment on the requirement of further research data.

LVRS is a palliative treatment that improves QOL, pulmonary function and exercise tolerance. Data on risks associated with LVRS is poorly document. RCTs need to confirm a group of patients that are likely to benefit from LVRS.

Living donor liver transplantation Type RAPID FULL

Agency and product information Name of agency AHFMR ASERNIP-S Product TechNote Systematic reviewYear of publication Mar 2004 Oct 2004 Title “Adult to living donor liver transplantation” “Live donor liver transplantation – adult outcomes: a

systematic review” Review methodology

Search Dates

Jan 2004

Jan 2004

No. of Pages 22 673

Treatment intervention Adult-adult living donor liver transplant Any surgical technique transplanting liver from live adult donor to adult or child recipient

Comparator intervention Cadaveric transplant Some studies reported comparison with cadaveric liver transplants but most were case series


Yes (websites and search terms) Yes (websites and search terms)

Study inclusion criteria Only systematic reviews or comparative studies, with at least 10 patients in each study arm, published in English from 1995 onwards were included for analysis. Participants: Data were collected on adult (≥18 years of age) patients receiving a liver transplant from an adult liver donor for any indication.

Recipient: Patients 18 years and over, needing a liver transplant Donors: People aged 18 years and over who have given informed consent to donate part of their liver (to adult or child recipients) Interventions: Recipient: any surgical technique for transplanting a liver from a live adult donor to an adult recipient Donor: any surgical technique for transplanting a liver from a live adult donor to a recipient Study design: Any comparative studies, registry data or case series. In addition, case studies of AA LDLT were reviewed for reports of outcomes related to safety issues.

Study exclusion criteria: Patients receiving any other organ transplant in addition to the liver graft. Animal studies.

Foreign languages only if findings supported those reported in well-designed studies in the English language

No. of authors NR 9 (2 reviewers) Use of expert panel or peer review One surgeon Expert panel Type of evidence grading NR NHMRC

Evidence base used Systematic reviews 1 (donor) NR RCTs 0 0Comparative (non-randomised) 3 (recipient)/ 1 (financial) Donor: 79/ Recipient: 89 Case Series 0 Donor: 330/ Recipient: 149 Other studies 0 3 surveys Studies excluded from review: NR 95 Total meeting criteria 5 409/231 Outcomes 11 19Collective number of patients included: 973 NR (>1,000)

Critical appraisal of included studies No No

Results Safety Yes (mortality; postoperative biliary

complications, graft survival rates, or length of Yes (mortality, graft loss, complications, psychiatric)

99

100

hospital stay) Effectiveness: No. of outcomes

Yes 8

Yes 5

Economics Yes1 study: LDLT vs cadaveric liver transplantation Local costs not discussed

Yes Discussion of one economic analysis study (< 1 page) Local costs not discussed

Ethics Yes (autonomy of donor and the recipient; environment where donor is not subject to coercion; informed donor consent supported by a true estimate of the utility of LDLT and donor complication rates, and the identification of any future physical and financial risks for the donor) > 1 page

Yes (justice, nonmaleficence, patient autonomy and beneficence; nonmaleficence remains to be validated both in physiologic and quality of life respects; coercion; shielding; informed patients; surgeon’s moral responsibility; informed consent) > 1 page

Use of Meta-analysis NR NR Overall conclusions Evidence base for LDLT is incomplete. LDLT is

still undergoing active development, so safety and efficacy relative to CLT and impact on waiting lists are unknown.

LDLT not shown to be superior to CLT One way to increase organ availability- small but real risks Concerns about comprehensiveness and reliability of much of published info and concerns about donor safety Require more long-term outcomes

Hip Resurfacing

Type RAPID RAPID FULL

Agency and product information

Name of agency AHFMR CCE NHSHTA

Product Technote Critical Appraisal HTA Programme Report

Year of publication 2002 2002 2002

Title “Metal-on-metal hip resurfacing for young, active adults with degenerative hip disease”

“Hip resurfacing in patients with osteoarthritis” “A Systematic Review of the Effectiveness & Cost-Effectiveness of Metal-on-Metal Hip Resurfacing arthroplasty for treatment of hip disease”

Review Methodology

Search Dates

Feb 2002 October 2002 2001

No. of Pages 16 9 117

Treatment intervention Hip Resurfacing “Birmingham Hip” resurfacing Metal-on-metal hip resurfacing arthroplasty

Comparator intervention Total Hip Replacement Total Hip Replacement Watchful waiting

Total hip replacement

Osteotomy

Arthrodesis

Arthroscopy


Search Terms and Databases used Provided Search Strategy Detailed (Search terms and databases) List of Search Terms and Databases Used

Study inclusion criteria: Young, active adults (≤65 years of age) with degenerative hip disease

Search limited to humans aged ≥18 years, English language and controlled clinical trials

“If sound, relevant material of a high level of evidence is identified, the search stops”

RCTs: those that provided patient outcome data and compared metal-on-metal hip resurfacing arthroplasty with other interventions

Comparative studies: those with concurrent controls, in which revision rates, clinical assessment or patient-based outcomes were

101

102

provided

Single prosthesis observational studies: those in which revision rates, clinical assessment or patient-based outcomes were provided

Study exclusion criteria: NR NR Studies not reporting patient outcome data, such as laboratory only studies, were excluded.

Studies published in a language other than English

No. of authors NR 1 6

Use of expert panel or peer review

No (not subject to an external review process) NR YES

Diagnostic Technologies and Screening Panel/ Pharmaceuticals Panel/ Therapeutic Procedures Panel/ Expert Advisory Network

Type of evidence grading NR NHMRC Yes

Checklist developed by Morris & colleagues

Evidence Base Used

HTA Reports 5 2 3

RCTs 0 0 1

Comparative (non-randomised)

0 0 0

Published Single Prosthesis Observational

0 0 18

Industry Submission 0 0 3

Unpublished single prosthesis observational data

0 0 2

Case studies/ case reports 0 0 0

Other studies 0 0 0

Studies excluded from NR NR NR

review:

Total meeting criteria 5 2 27

Outcomes Overall conclusions of the HTA reports are discussed narratively

Overall conclusions of the HTA reports are discussed narratively

7

Collective number of patients included:

NR NR NR

Critical appraisal of included studies

No Yes Yes

103

104

Results

Safety YES

Overall conclusions of HTA reports are discussed narratively

Complications:

- blood loss - infection - blood clotting in legs - blood clots moving to lungs - nerve injury - hip dislocation - concerns over toxic metals in body

Evidence summaries of both studies provided Yes.

Complications discussed narratively.

Effectiveness:

No. of outcomes

YES

2 (Revision rates and post-surgery complication rates)

Overall conclusions of HTA reports discussed narratively

Evidence summaries of both studies provided Yes

7:

- Duration of op - Time to return to normal activities - Revision surgery - Functional result - Patients pain free - Complications - Conversions

Economics Costing data from 2 out of the 5 HTA included HTA reports are discussed narratively (<0.5 page)

NR The review discusses the economics section of one industry submission to NICE (5 pages)

Also have conducted an independent evaluation (16 pages)

The authors comment on the lack of good data.

Ethics NR NR Implications for other parties; access, equity, resource and budget considered

Use of Metanalysis No No NR

Overall conclusions Metal-on-metal hip resurfacing may be a viable and bone-conserving option for adults with degenerative hip disease who would otherwise

Available evidence, whilst not of a high level and lacking long term follow up, suggests metal-on-metal resurfacing of hip may be viable and bone-conserving option for

Evidence is limited but-

Metal-on-metal hip resurfacing has the potential to be an effective technique for management of

receive and are likely to outlive conventional THR

There is a lack of long term data

adults who are likely to outlive total hip replacement. hip disease.

Lack of long-term data.

More research required.

105

asernip-s inventory of current methods and practice · pdf filean inventory of current methods...

Documents