2

Section 505(d) of the Act requires “substantial evidence” of effectiveness before a drug can be placed on the market and sold to consumers. In most instances, the Agency has interpreted this to mean that a drug manufacturer must provide adequate and well-controlled clinical studies distinguishing a drug’s effectiveness “from other influences, such as spontaneous change in the course of the disease, placebo effect, or biased observation.” 21 C.F.R. § 314.126. The Act’s language makes clear that homeopathic drugs fall under the definition of the term “drug” as used broadly in the Act. Section 201(g)(1) of the Act defines the term “drug” as used in the Act to include “articles recognized in the official United States Pharmacopeia [“USP”], the official Homeopathic Pharmacopeia of the United States [“HPUS”], or official National Formulary [“NF”], or any supplement to them; and articles intended for use in the diagnosis, cure, mitigation, treatment, or the prevention of disease in man or other animals; articles (other than food) intended to affect the structure or any function of the body of man or other animals; and articles intended for use as a component of any articles” so specified. Whether or not they are official homeopathic remedies, those products offered for the cure, mitigation, prevention, or treatment of disease conditions are regarded as drugs within the meaning of Section 201(g)(l) of the Act. Section 400.400 of the FDA Compliance Policy Guidelines, “Conditions Under Which Homeopathic Drugs May be Marketed,” states that a product’s “compliance with requirements of the HPUS, USP, or NF does not establish that it has been shown by appropriate means to be safe, effective, and not misbranded for its intended use.” The plain meaning of the Act therefore permits the Agency to subject OTC homeopathic drugs to the same effectiveness standards that apply to other OTC drugs. The Agency has broad authority over the labeling of drugs. See Section 502 of the Act and 21 C.F.R. Part 201. Through its rulemaking authority, the Agency has required both testing for effectiveness and appropriate labeling of certain classes of drugs. E.g., Labeling and Effectiveness Testing; Sunscreen Drug Products for Over-the-Counter Human Use, 76 Fed. Reg. 35620 (June 17, 2011).

2. The collective weight of the scientific evidence establishes that homeopathic drugs are no more effective than placebos.

Homeopathic drugs are ineffective for at least two independent reasons, either of which is sufficient to establish homeopathic drugs’ inefficacy in light of basic, well-established principles of biochemistry and pharmacology. First, there is no reliable scientific evidence that the alleged active ingredients in homeopathic drugs cure the diseases and conditions for which they are indicated. Indeed, as discussed further below, homeopathic “theory” is premised on the magical notion that very small doses of substances that cause disease symptoms can alleviate those same symptoms. Second, producers of homeopathic drugs typically dilute the drugs’ alleged active ingredients in solution to the point that the solution contains, on average, not a single molecule of the alleged

3

active ingredient.1 In other words, what the producers and retailers of homeopathic drugs market as “medications” are nothing more than solutions that are devoid of active ingredients (often the solutions are simply water), which then have been combined with various inactive ingredients, such as balls of ordinary sugar. Homeopathy, which was developed and popularized in the late 18th century by the German physician Samuel Hahnemann, purports to stimulate the body’s ability to heal itself by administering vanishingly small doses of highly diluted substances.2 Homeopathy is based on two principles, both of which contradict basic, well-established laws of modern chemistry and pharmacology: the “principle of similars,” which holds that a disease can be cured by a substance that produces similar symptoms in healthy people; and the “principle of dilutions,” which holds that the lower the dose of the medication, the greater its effectiveness.3 Pursuant to Hahnemann’s teachings, most homeopathic remedies sold today have been diluted repeatedly to the extent that not a single molecule of the alleged healing substance remains.4 As one would expect, the scientific evidence establishes that any apparent benefits from homeopathic drugs are mere placebo effects. The medical journal The Lancet published a comprehensive review of homeopathic treatments in 2005.5 The authors analyzed every clinical investigation then published on the effects of homeopathy. Their analysis identified 110 well-designed investigations into its effects. The results of those trials were then compared to 110 well-designed investigations of conventional medical therapies for the same ailments. The authors concluded:

Biases are present in placebo-controlled trials of both homoeopathy and conventional medicine. When account was taken for these biases in the analysis, there was weak

1 Homeopaths denote a 1-to-100 dilution by the letter “C,” and a 1-to-10 dilution by the letter “X.” For example, a “6X” homeopathic preparation is a 1-to-10 dilution repeated six times, leaving the alleged active ingredient as one part per million. Homeopathic drugs designated as 24X, 12C or higher contain, on average, no molecules of the original substance from which they were prepared. See generally Stehlin, Isadora. “Homeopathy: real medicine or empty promises?” FDA Consumer, Dec. 1996. Available online at http://findarticles.com/p/articles/mi_m1370/is_n10_v30/ai_18979004/. 2 National Institutes of Health (“NIH”) National Center for Complimentary and Alternative Medicine Backgrounder: Homeopathy, An Introduction (July 2009). Available online at http://nccam.nih.gov/health/homeopathy/. 3 Id. 4 Id. 5 Shang A, Huwiler-Müntener K, Nartey L, et al. “Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy.” Lancet 2005; 366: 726-732. Available online at http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(05)67177-2/fulltext. (Attached hereto as Exhibit A.)

4

evidence for a specific effect of homoeopathic remedies, but strong evidence for specific effects of conventional interventions. This finding is compatible with the notion that the clinical effects of homoeopathy are placebo effects.6

The highly esteemed Cochrane Collaboration has published reviews of homeopathic treatments for the treatment of a number of diseases and medical conditions, including, among others, dementia, chronic asthma, attention deficit hyperactivity disorder (ADHD), and influenza. In each instance the reviews concluded that evidence for homeopathic drugs’ efficacy is nonexistent, unreliable, or too scarce to warrant recommendations regarding homeopathic treatment.7 Although most analyses have concluded that there is little evidence to support homeopathy as an effective treatment for any specific condition, some studies have reported positive findings.8 Studies reporting positive findings, however, are generally poorly controlled and/or have not been replicated.9 6 Id. 7 See, e.g., McCarney, R.W., et al., “Homeopathy for chronic asthma,” Cochrane Database of Systematic Reviews 1999, Issue 1. Art. No.: CD000353. DOI: 10.1002/14651858.CD000353.pub2 (available online at http://www2.cochrane.org/reviews/en/ab000353.html); Heirs, M. and Dean, M.E., “Homeopathy for attention deficit/hyperactivity disorder or hyperkinetic disorder,” Cochrane Database of Systematic Reviews 2007, Issue 4. Art. No.: CD005648. DOI: 10.1002/14651858.CD005648.pub2 (available online at http://www2.cochrane.org/reviews/en/ab005648.html); McCarney, R.W., et al., “Homeopathy for dementia,” Cochrane Database of Systematic Reviews 2003, Issue 1. Art. No.: CD003803. DOI: 10.1002/14651858.CD003803 (available online at http://www2.cochrane.org/reviews/en/ab003803.html); Vickers, A. and Smith, C., “Homoeopathic Oscillococcinum for preventing and treating influenza and influenza-like syndromes,” Cochrane Database of Systematic Reviews 2009, Issue 3. Art. No.: CD001957. DOI: 10.1002/14651858.CD001957.pub4 (available online at http://www2.cochrane.org/reviews/en/ab001957.html). 8 NIH National Center for Complimentary and Alternative Medicine Backgrounder: Homeopathy, An Introduction (July 2009). Available online at http://nccam.nih.gov/health/homeopathy/. 9 For example, a 1997 meta-analysis of placebo-controlled trials of homeopathic drugs concluded that patients receiving homeopathic drugs were more likely than patients receiving placebos to show signs of improvement. Linde, K., et al., “Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials.” Lancet, 1997; 350 (9081): 834-43. Available online at http://www.ncbi.nlm.nih.gov/pubmed/9310601. The authors of the report later re-examined their meta-analysis. In a follow-up paper that accounted for new, high-quality trials showing negative results, the authors concluded that “in the study set investigated, there was clear evidence that studies with better methodological quality tended to yield less positive results . . . It seems likely, therefore, that our [1997] meta-analysis at least over-estimated the

5

The collective weight of scientific evidence demonstrates that homeopathic drugs yield no benefit beyond the placebo effect. In addition, the premise upon which the effectiveness of “homeopathic medicine” is founded—that highly diluted preparations of substances that cause symptoms in healthy individuals will reduce similar symptoms in patients—has no basis in reality and has been disproved repeatedly.10 Accordingly, major health and medical organizations, including the American Medical Association, have concluded that there is no convincing scientific evidence to support the use of homeopathic treatments in medicine.11

3. The Agency’s failure to regulate homeopathic products has enabled a multi-million dollar industry to market ineffective products to unsuspecting, ill consumers.

Although the Agency has the authority to require homeopathic products to undergo the effectiveness testing that applies to prescription and new OTC drugs, it has to date declined to exercise that authority. The Agency’s permissive regulatory policy has enabled the growth of a multi-million dollar market for products for which there is no credible scientific evidence of effectiveness beyond what is expected from the placebo effect. Consumer sales of homeopathic treatments in the U.S. reached $870 million in 2009.12 According to a 2007 study conducted by the Center for Disease Control and Prevention (“CDC”), an estimated 3.9 million U.S. adults and approximately 900,000 children used homeopathic products in the previous year.13 Ill consumers who desire effective medical treatment turn, unwittingly, to homeopathic drugs, using them for a broad range of health concerns, including the treatment of diseases and conditions such as allergies, asthma, chronic fatigue syndrome, depression, digestive disorders, ear infections, headaches, and skin rashes.14 effects of homeopathic treatments.” Linde, K., et al. “Impact of Study Quality on Outcome in Placebo-Controlled Trials of Homeopathy,” Journal of Clinical Epidemiology 1999:52 (7): 631–6. Available online at http://www.ncbi.nlm.nih.gov/pubmed/10391656 (Attached hereto as Exhibit B.) 10 See, e.g., Ernst, E. “A systematic review of systematic reviews of homeopathy,” Br. J. Clin. Pharmacol. V.54(6): 577-582 (Dec. 2002). Available online at http://www.ncbi.nlm.nih.gov/pubmed/12492603. (Attached hereto as Exhibit C.) 11 AMA Council on Scientific Affairs. “Alternative Medicine: Report 12 of the Council on Scientific Affairs (A–97)” page 8. American Medical Association (1997). 12 “Homeopathy prospers even as controversy rages.” J. Deardorff, Chicago Tribune, March 6, 2011. Available online at http://articles.chicagotribune.com/2011-03-06/news/ct-met-0306-homeopathy-20110306_1_homeopathy-oscillococcinum-products. 13 NIH National Center for Complimentary and Alternative Medicine Backgrounder: Homeopathy, An Introduction (July 2009). Available online at http://nccam.nih.gov/health/homeopathy/. 14 Id.

6

Homeopathic drugs are typically marketed in ways that lead consumers to believe that they are effective for preventing and treating illnesses and symptoms, some of which can lead to severe complications, hospitalization, or death if improperly treated. By way of example, the retail giant Wal-Mart Stores, Inc. (“Wal-Mart”) promotes the sale of the homeopathic drug Boiron Oscillococcinum, a homeopathic drug produced by Boiron USA (“Boiron”), through Wal-Mart’s website, www.walmart.com. The website indicates that Boiron Oscillococcinum is indicated for the treatment of “flu-like symptoms.”15 The website16 also features an image of the product’s package, which indicates that the product “Reduces [the] Duration and Severity of Flu Symptoms,” including “Fever, Chills, Body Aches and Pains.” Boiron itself is even bolder, engaging in advertising that is highly misleading as it seeks to suggest this Agency has approved its product. Borion’s website for Oscillococcinum, http://www.oscillo.com/about/facts-about-oscillo , maintains that four clinical studies show the effectiveness of its product, and this claim is then immediately followed by the assertion that “Oscillo complies with a well-established framework of guidelines, regulations, and quality standards established by the FDA.” Consumers undoubtedly purchase the product based on Wal-Mart’s and Boiron’s assurances that Boiron Oscillococcinum is, in fact, effective for preventing and treating the flu—especially as Boiron implies its product has been tested by this Agency. Yet there is no credible scientific evidence to support the effectiveness of Boiron Oscillococcinum beyond what is expected from the placebo effect.17 This is hardly surprising, as Boiron Oscillococcinum’s alleged ingredient—liquefied duck liver and heart—was mistakenly identified in the early 20th century as a possible homeopathic agent on the basis of the false

15 See http://www.walmart.com/ip/Boiron-Oscillococcinum-Homeopathic-Flu-Medicine-6ct/10316942#ProductDetail. 16 Wal-Mart’s misleading promotion of this homeopathic drug as a treatment for flu is not limited to the webpage on which the product is displayed. Consumers will reach this page only after visiting Wal-Mart’s “Medicine Cabinet” page, http://www.walmart.com/cp/Medicine-Cabinet/976798, which assures customers that the products Wal-Mart carries will “fight colds and the flu.” From there, website visitors will navigate to the “Cough, Colds & Flu Wellness Shop” page, http://health.walmart.com/health-tips/cough-cold-flu/20, which promises to help the customer “Stay on top of cold and flu season by learning about products that can help you and your family stay well, relieve symptoms and recover fast.” In its “Cough, Cold, and Flu Buying Guide,” http://health.walmart.com/health-advice/cough-cold-flu-buying-guide/689/, Wal-Mart asserts that its products will provide the customer “with everything you and your family need for battling a cold or the flu.” 17 Guo, R., et al., “Complementary Medicine for Treating or Preventing Influenza or Influenza-like Illness,” Am. J. Med., Vol. 120 Issue 11, 923-929 (2007). Available online at http://www.amjmed.com/article/S0002-9343%2807%2900876-5/abstract.

7

belief that it contained “oscillococci,” microscopic bacteria that proved to be imaginary.18 Moreover, like most producers of homeopathic drugs, Boiron dilutes its alleged active ingredient to the point that the likelihood that even a single molecule of the substance remains is vanishingly small.19 The homeopathic industry’s marketing of homeopathic drugs, aided by major retailers such as Wal-Mart, is a profound disservice to the public. Beyond leading consumers to squander hundreds of millions of dollars annually on ineffective remedies, homeopathic drug producers and retailers place the public health at risk by displacing proven, effective medical treatments. Influenza, for example, is a serious illness that can lead to complications resulting in hospitalization or even death, especially among the elderly, the very young, and individuals with certain health conditions.20 Yet U.S. sales of the wholly ineffective homeopathic flu remedy Oscillococcinum reached $15 million in 2008 alone.21 The Agency should act to ensure that consumers are not led to believe that effective preventive and therapeutic measures can be ignored in favor of products that amount to mere placebos, at best.

4. As the Agency recognizes, accurate labeling is critical if consumers are to make an informed choice.

As demonstrated by the Agency’s recently issued rule on labeling of sunscreen products, Labeling and Effectiveness Testing; Sunscreen Drug Products for Over-the-Counter Human Use, 76 Fed. Reg. 35620 (June 17, 2011), this Agency recognizes its responsibility to ensure that consumers are provided with all material information regarding a product so they can make an informed choice. The choice of a sunscreen can be very important. At least equally important, however, are choices relating to the treatment of serious diseases such as influenza. Homeopathic drug producers and retailers are misleading consumers into believing that their products are effective. As indicated by Boiron’s promotion of Oscillococcinum, manufacturers of homeopathic products are not averse to implying that this Agency has approved their product.

18 Nienhuys, J. “The True Story of Oscillococcinum” (2003). Available online at http://www.homeowatch.org/history/oscillo.html. 19 Boiron Oscillococcinum uses a “200C” homeopathic preparation, indicating a 1-to-100 dilution repeated two hundred times. By way of comparison, a mere “30C” preparation is so dilute that a single molecule of the original substance dissolved would occupy a volume of solution that is more than 30 billion times the size of the earth. See Barret, S. “Homeopathy: The Ultimate Fake.” Available online at http://www.quackwatch.org/01QuackeryRelatedTopics/homeo.html. 20 See, e.g., Centers for Disease Control and Prevention, “Seasonal Influenza: The Disease.” Available online at http://www.cdc.gov/flu/about/disease/. 21 Park, Robert L. “Superstition: Belief in the Age of Science,” Princeton University Press. ed., pp. 146 (2008).

-

EXHIBIT A

-

I on,o*

Ld^.et 2oo5; f66: 726-j2

5e€comnrertPage69l

Departm en t of social and

Preventive Medi.in€, Univebityof B€rne, B€.ne. switu €rl.nd

(A5han9MD,

K HuwileFMiintend MD,

LNarteyM0,

PJlniMD,5Doig,D PsnerMD,

PrcfMEggerMD); Medical

Re5ea,(h Coun.il H.althS€ruicsResear.h

Collabo.,tion, D€partdenl ofso. i.l Medicine, U n ive6 itY of

B itol, Brirtol, uK(PJ!ni,

J A C sten€ PhD, PotM E9ga4;

D€p:nment of Pharm:<olo9y,

UniveEity of Zitich, Zijtnh,

switu€ and(5 Dodg);and

Pr:di.e B ickfeld. Medix

G€n€lal Practi.€ Netwolk,

B.rn.,switzedand (D Pesner)

Corspondence to

Prof Manhes E99er, D€partnent

.f s..ial and Prekntlv€

Medicine, UniBityof Be..e,

8erne, switzedand

eqqer@ispm,unibe.ch

\ Are the clinical effects of homoeopathy placebo effects?

Comparative study of placebo-controlled trials ofhomoeopathy and allopathyAijingShong,KorinHu\Niter-Minte et, LindaNoftey,Petet)ini,StephanDotig,Jonotfion A a stcrte, DonicJPevJrner, Motthios Egqer

5ummaryBackground Homoeopathy is widely used, but specific effects ofhomoeopathic remedies seem implausible. Bias inthe conduct and reporting of trials is a possible explanation for positive findings oftrials ofboih homoeopathy and

conventional medicine. We analysed trials of homoeopathy and conventional medicine and estimated treatment

elTects in trials least likely to be affected by bias.

Methods placebo.controlled trials of homoeopathy were identified by a comprehensive litenture seatch, which

covered 19 electronic databases, reference lists ofrelevant papers, and contacts with experts. Trials in conventional

medicine matched to homoeopathy trials for disorder and gpe of outcome were randomly selected from the

Cochrane Controlled Trials Register (issue 1, 2003). Data were extracted in duplicate and outcomes coded so that

odds ratios below 1 indicated benefit. Trials described as double.blind, with adequate nndomisation, were assumed

to lre ofhigher methodological quality, Bias effects were examined in funnel plots and meta'regrcssion models.

Findjngs 110 homoeopathy trials and 110 matched conventional-medicine trials were analysed. The median study

size wls 65 participants (range ten to 1573). 21 homoeopathy trials (19%) and nine (8%) conventional'medicine trials

were ofhigher quility. In both groups, smaller trials and those oflower quality showed more benefrcial treatment

effects thai larger and hlgh"r-qn"lity ttiul". When the analysis was restricted to large trials ofhigher quality, the odds

ratio was 0.8E (95% CI 0.65-i.19) for homoeopathy (eight trials) and 0 . 58 (0 39-4 .85) for conventional medicine

(six trials).

lnterpretation Biases are present in placebo-controlled tlials ofboth homoeopathy and conventional medicine. \Xrhen

account was taken for thise biases in the analysis, there was weak evidence for a specific e{fect ofhomoeopathic

remedies, but strong evidence for specific effecrs ofconventional interventions. This finding is compatible with the

notion that the clinical effects ofhomoeopathy are placebo effects.

lntroduction to affect small than large studies;the smaller a study, the

Homoeopathy is a widely used but controversial larger the treatment eflect necessary for the results to be

complementary or aliernative therapy.'' The basic statistically signifrcant, whereas lalge studies are more

premi." is that like is cured by like (sinilia simi,Iibus likely to be ofhigh methodological <luality and published

curenfur)--iiseases can be treated by substances that even if their results are negative We examined the

produce the same signs and symptoms in a healthy effects of homoeopathy and conventional .medicineindividual.., The preplration ofremedies involves serial observed in matched Pails of Placebo-conholled t als,

dilution, commo;ly io the extent that no molecules of assessed trial quality and the probability of publication

the original substance temain, and vigorous shaking and lelated biases, and estimated results oflarge trials

between dilutions (potentisation). During this process least affected by such biases

informaiion is thought to be transferred from the diluted

substance to the soivent,' which in the light of current Methodsknowledge seems implausible. Many people theiefore Literature search and data sources

""",r-" ih"t any effecis of homoeopathy must be non- we updated a prenous comprehensive search for

specific placebo effects.' placebo'controlled tlials ofhomoeopathy, which covered'Bias irithe conduct and rePorting oftrials is a possible Publications up to August' 1995 " We searched

explanation for positive 6"il"g. of placebo'controlled 19 electronic databases, including specialised

trials of both homoeopathy and allopithy (conventional homoeopathic and comPlementary-medicine registries,

medicine).'" Publication bias is defined as the covering the Period from 1995 to |anuary' 2003:

preferential and more rapid publication of trials with MEDLINE, PIe-MEDLINE' EMBASE' DARE' CCTR'

statisticaliy significant and bineficial results than of CDSR, CINAHL, AMED, MANTIS, Toxline' PASCAL'

trials -itf,oit significant results.,o The low BIoL, science Citation Index, clscoM, British

methodological quality of many trials is another Homeopathic Library, the HomeoPathy A!:tracJ Pagel

important s'ource of bias." These biases are more likely Homlnform Homoeopathic library' NCCAM' and

726vol365 Auqu5t27,2005

Arai.b, I

SIGLE. The search terms in MEDLINE were (homeop'kOR homoeop* OR llomeopathy (MeSH)) AND (placebo*OR placebos {MeSH) OR placebo effect (MeSH) ORsham). Search terms for the other databases were muchthe same. We also checled the reference lists ofrelevantpapers, including teviews and meta-analyses ofhomoeopathic interventions, and contacted experts inthe specialty. There were no language restrictions.

We searched the Cochrane Controlled Trials Registetto identify placebo-controlled trials of conventionalmedicine. This bibliographic database of controlled trialsis maintained by the Cochrane Collaboration. As part ofan international effort to search systematically health-care journals worldwide and other sources ofinformation, the collaboration has combined results ofelectronic searches and searches by hand io create a

comprehensive database oftrials.'r We searched issue 1,

2003, ofthe Cochrane Controlled Triais Register, whichincluded 353 809 billiographic references.

studyseledionWe defined inclusion and exclusion criteria a priori andapplied the same criteria to trials ofhomoeopathy and ofconventional medicine. lnclusion c te a were: that thetrial was controiled and of treatments o. preventivemeasures with clinical outcomes; that it had a parallel-group design with placebo control; that there wasnndom o! quasirandom assignment to treatment andplacebo groups; and t}lat a wyitten report (eg, journalpublication, abstract, thesis, conference proceeding,unpublished repolt, book chapter, monograph) wasavailable with sufficient data to aliow the calculation ofodds ratios. Ve excluded trials of homoeopathic"provings" in which remedies are given to healthyindividuals to assess thef effects, cross'over trials, andN-of-1trials.

ProceduresVe used prespecified criteria to identify outcomes forinclusion in the analyses. The first choice was the mainoutcome measure, defined as the outcome used forsample-size calculations. If no main outcome wasspecified, we selected other outcomes, in the order:patients' overall assessment of improvement;physicians' overali assessment of improvement; and theclinically most relevant other outcome measule {forexample, the occurrence or duration of an illness).Outcomes were selected iandomly if several were

ludged equally relevant. For each homoeopathy trial, we

identified matching trials of conventional medicine thatenrolled patients with similar disorders and assessed

simijar outcomes. We used computer-generatedrandom numbers to select one from several eligibletrials ofconventional medicine. Outcomes were selectedand triais matched without knowiedge oftrial results.

We used a piloted data-ertraction sheet, whichcovered descriptive information on the trial and study

wthelancet.com Vol356 AugUst27, 2oo5

figure 1j ldentifcation of 110 eligibl€ placebo<ontrolledtdakofhomoeopathyth.t.ould b€ makhed to an equal number of pla(ebo-

controlled tdak of conventional medicine

population, intenertion, outcome measures, and trialquality. Data wele ertracted independently by twoobsewers, and discrepancies were resolved byconsensus,

Homoeopathic interventions were defined as

classical, clinical, or compiex homoeopathy, or as

isopathy. Classical homoeopathy was defined as

comprehensive homoeopathic history-taking, followedby the prescription of a single individualised remedy,possibly with subsequent change ofremedy in responseto changing symptoms. If no comprehensivehomoeopathic history was taken and all patienisreceived a single. identical remedy, inreraenlions were

classified as clinical homoeopathy. Complexhomoeopathy was defined as the prescription of a

mixture of several different remedies. Interventionswere classified as isopathy if the agent that was judgedto be the cause of the disorder was used (for example,pollen in pollinosis). hldications fo! tleatment wereclassified as acute or chronic or primary prevention orprophylaxis {interventions with the intention of

Respnatory-tn.t infe.tions

6ynae(olog, and obneti65urqeryand ana€sthet.s

M!sculdkeletaldnode6

21(19%)

16(15%)

14(13%)

12(11%)

72177%)

11(10%)

10 (9*)14{11%)

pu blicatio.5 ld.n tilied

17 inj0fficient inlomation14 inellgible study design

2 rc homoeoprthlc inleNent on

rbl" t: Dlrtrlbrtb*I p""-f plac

-

I o*,o*

Homoeopathytrials Conventional-m.dicin.triak

{n=11o) (n-110)

Median ye& of publicaiion (.ange)

lnEnglish

MEDLINE-indered jounal

ovedll a$e$ment of rerponse

o..urcn.e ordu.aiion of d sodetA$e$m€ntofsrmptomtMearurcment of function or nate

A$e$nent of dini(al 5i9n,

55.5 (10-1573) 65 02 1t67)777 (111) 131(216)

1992(1966-2003) 1994(1974-2002)

58{53%) 94(8s%)

94(85%) 110{1oo%)

45 &1%) 95 (85%)

54149%1 49145t476124%) 26 (24. )77179%) 26 Q4%)6(sv.) 6{5%)

D6dibed ar double'blind 101{92%)

Adeq u ate gene Etiof of allo.ation seq! en ce 27 (25%)

Adequate co..ealm6t of alloGtion 49 (45%)

Analysi, by intention to t@at 33 (30%)

Hiqherquality' 21(19%)

96187%)

to t)7%)21(19%)

40 (36%)

e (8'")

'fiah de{ibed 6 doubh-blind, with ad€qste 9eh!6ton ofall@tion 5eqwne and adequate rcn.ealment ot all(ation.

T06r. 2: chah<teri5ticr ofiola<abo-controlled t alr othom@opathy and conventional medicine

See r r'.i . .r r'{orwebappendi(es 1 and 2

preventing the occurrence of a disorder orcomplication). The dwation of followup was measuredin weeks from the start of the treatment to theassessment of outcomes.

Assessment of study quality focused on three keydomains of internal vaiidiry:'r'' randomisation(generation of allocation sequence and concealment ofallocation), masking (of patients, therapists, andoutcome assessors), and data analysis (by intention totreat or other). Random-number tables, computer-generated random numbers, minimisalion. coin-tossing, caid'shuffling, and lot-drawing were classifiedas adequale methods for the generation ofthe allocationsequence. Sealed, opaque, secluentially numberedassignment envelopes, central randomisatron,independendy prepared and coded drug packs ofidentical appearance, and on-site computerisedrandomisation systems were classifled as adequatemethods of allocation concealment. Analysis byintention to treat was assumed ifthe reported number ofpalticipants randomised and the number analysed wereidentical. Descriptions of other methods wele codedeither as inadecluate or unclear, depending on theamount of detail provided. Trials described as double-blind, with adequate methods for the generation ofallocation sequence and adequate concealment ofallocation, were classified as of higher methodologicalquality.

G raphical an d statistical a n alysisWe expressed results on the odds ratio scale and usedthe method descdbed by Hasselblad and Hedges" toconvert difTerences in continuous outcomes to oddsratios. We lecoded outcomes if necessary, so that odds

ratios below 1.0 indicated a beneficial effect oftreatmentin ali cases. We used descriptive analyses to comparecharacteristics of homoeopathy and conventional-medicine trials. We examined heterogeneity betweentrials with standard 1' tests and calculated I': statistics,which measure the proportion ofvadation in treatmenteffect estimates due to between'strldy heterogeneity.r6We investigated the association between study size andtrial results in funnel plots, by plotting odds ratios on thehorizontal aJds (on a logarithmic scale) against thei. SEon the vertical axis.'7 The extent to which studylevelvariables were associated wit}l log odds ratios wasexamined by itting of univariable and multivariablemeta-regression models." The following variables wereconsidered: SE oflog odds ratio, Ianguage ofpublication,indexing of the publication in MEDLINE, trial quality

{masking, generation of allocation se<1uence, conceal-ment of allocation, intention-tolreat analysis), durationoffollow-up, and clinical topic. For homoeopathy trials,we also examined whether effects varied between t,?esofhomoeopathy and tlpes ofindications {acute, chronic,primary prevention, or prophylaxis).

We combined treatment effects from larger trials ofhigher quality by use of standard random'effects meta-analysis and used meta-regression analysis to predicttreatment effects in trials as large as the largest trialsincluded in the study. Trials with SE in the lowestquartile were defined as largertrials. Results are given as

odds ratios, raiios of odds ratios, or asymmetrycoefficients with 95% CL Ratios of odds ratios of less

than 1 0 corespond to a smaller odds ratio for trialswith the chalacteristic and hence a larger apparentbenefit of the inteflention. Funnel-plot asymmetry wasmeasured by the asymmetry coemcient: the ratio ofoddsratios per unit increase in SE of log odds ratio.'' Allanalyses were done in Stata version 8.2.

Role ofthe funding sourceThe funding sources had no role in the study design;collection, analysis, or interpretation of data; or thewriting of the report. The corresponding author had fuliaccess to all the data in the study and had finalresponsibility for the decision to submit the paper lbrpublication.

ResultsWe identified 165 potentially eiigible reports ofplacebo-controlled trials of homoeopathy and excluded60 reports. The commonesl reasons for exdusion wereinsumcient information (precluding the calculation ofodds ratios), ineligible study design, multiplepublication, and inability to identify a matching trial ofconventional medicine (figure 1). We induded105 publications that repor(ed on a total of110 independent trials ofhomoeopathy (webappendix 1)

and 110 publications of 110 matched trials ofconventional medrcine {webappendix 2).

{wwrhelancet.com vol355 Alqust27,2oo5

7

O,ai.l", I

The clinical topics shrdied in paits oftdals ranged fromrespiGtory infections to surgery and anaesthesiology

{table 1). The outcomes studied were closely matched;overall assessments ofresponse were analysed in 49% ofhomoeopathy trials and 45% of trials of conventionalmedicine (table 2). More detailed information onoutcomes is given in the webtable. The average sh-rdy size

was similar for the two groups, with a median of around65 paticipants. Overall, str.rdy size ranged from ten to1573 padicipants. Among homoeopathy tlials 48 (44%)

concemed dinical homoeopathy, 35 (32%) complexhomoeopathy, 18 (16%)dassical homoeopathy, and eight

\7%J isopathy. For the remaining t al, the nature of thehomoeopathic intervention was unclear. 101' 192%) ofrheconventional-medicine trials investigated drugs, eight(7oZ) immunotherapy, and one a vaccine. The drugs mostfrequently tested were non-steroidal anti"inflammatoryagents (11 trials), anti-allergy drugs (11 tiials), virostaticdrugs (11 trials), and antibiotics (seven trials).

53% of homoeopathy trials were published in English

compared with 85% of trials in conventional medicine.50 homoeopathy trials were published in German orFrench. The two groups of trials aiso differed in theproponion published in MEDLINE indexed journals.

The two groups had similar methodological quality interms ofmasking, generation ofallocation sequence, and

analysis according to intention to treat, but a highe!proportion of homoeopathy trials reported adequate

concealment of patients' allocation. 21 (19%) homo-

eopathy trials and nine (8olo) conventional-medicine trialswere ofhigher quality (table 2).

Most odds ratios indicated a beneficial effect of the

intervention {frgure 2). SE ranged from 0 12 to 1 65 forhomoeopathy trials and 0.11 to 1.52 for conventional-

medicine h'ials. Heterogeneity of trial results was less

plonounced for homoeopathy (heterogeneity x'=309, dfi09, p<0.0001) than for conventional medicine(heterogeneity x'=481, df 109, p<0 0001) Thisdifference is unlikely to be due to chance {p:0 011 by

F test). The proportion oftotal variation in the estlmates

of treatment effects due to between-study heterogeneity(I?) '" was 6 5 % for hom oeoparhy aad 77/o for conventional

medicine.Funnel plots were asymmetrical, with smailer trials

(largo SE) in the lower part of the plot showing morebenefrcial treatment effects than iarger ttials (smaller SE,

figure 2). In meta-regression models, the association

between SE and treatment effects was similal for trials ofhomoeopathy and conventional medicinel the respective

asymmetry coefficients were 0 17 l95Yo Cl 0 10-0 12)

and 0 21 {0 11-0 40). Therefore, with each unit increase

in the SE, the odds ratio decreased by a factor of0 17 forhomoeopathy and 0.21 for conventional medicine

{table 3).

Other sources of heterogeneity between homoe-

opathy trials included the language of publication(more beneficial effects in trials published in languages

vvw.thelan@t.con vol365 August27,2oos

other than English), indexing in MEDLINE {mo!ebeneficial effects in trials not indexed in MEDLINE),and indicators oftriai quality (more beneficial effects intrials of lower quality). The effects of these valiableswere generally similar fot conventional-medicine trialsbut did not reach statistical significance (table 3). Therewas little evidence that treatment effects variedaccording to duration of follow'up (p:0.862 fothomoeopathy, p=0 594 for conventional medicine) orclinical topic {p=0 660 for homoeopathy, p=0'360 forconventional medicine) or that effects differed betweendifferent lypes of homoeopathy (p=0.636) or type ofindication (p=0 487). In multivariable analyses, the SE

of the log odds raiio (asymmetry coefEcient) was thedominant variable in both groups. Coefficients ofother

l

:o

03

06

g 09

7)

15

06

g 09

\2

15

18

rO'a'a

a

Figurc 2: Funn.lplotof 11o homo.oP.thytials and 110 matched convehtionalmedkin€ t ak

solid lines indicate predktedt,eatmenteffectsfrom metajeg.ession, with dotted lines rcPr€senting the95% cl

I o*,0*

(9516 (DR.tl"

"f "dd-"ii.* p

(95%Ct)

Arymmetry .o€f6. .ntt

Con..alment of allo.arion

0.17(0.10-0.32)

073 (o 53-1oo)0.69(0.50 0.94)

o 44 (o.11-o.81)

067(028-0.95)

0.78(0.57-1.07)

1.25(0.87-1.80)

0.62 {043-0.90)

<0.0001 0.21{0.11-0.40)

005 0 67 io 40-1 14)

oo19 103 (061-1.75)

0017 061(0.36-1.11) 0.107

o o?4 098(065 146) o91l

0117 076(o,E-116) 0193

0.225 1.14(0.78-1.56) o 506

0.011 0.61(0.34-1.09) 0.095

'od6atiowilhchac.te sti. dividrd by od& Elio witho'rt {haad.'i3tk. ^atior

belw 1.o @respondto a 5mall€r odris 6tiofor lrirlr with .haadennk and hen(e a larqer apparenr h€n.fit ofihraNeniions. T'ials pubkh.d in lanluaqB other than E.qlish

show-r moft 6ei.n.ial trcatment elt .t th:n th6e publish ed in Ens li$, lor erample. rRario of odds htio p( lnit in.rca* in st

I"bt"homoeopathy rnd 110 matchid trials otronventioftl medicine

variables, including srudy quality, were attenuated and

became non-signi6cant.When the analysis was rest cted to the larger trials of

highe! repoded methodological quality, the odds ratiofrom random-effects meta-analysis was 0 88

(0.65-1.19) based on eight trials of homoeopathy and

0 58 (0 19-0.85) based on six trials of conventionalmedicine. Similarly, for prediction of treatment effects

in trials as large as the largest trials, the odds ratio was

0 96 {0.73-1 25) for homoeopathy and 0.67(0 48-4.91) for conventional medicine.

Discussionwe compared the effects of homoeopathy and

conventional medicine that are seen in placebo'

controlled tials, examined the presence of bias

resulting from inadequate methods and selectivepublication, and estimated results in trials least affected

by these biases. We assumed that the effects observed inplacebo-controlled trials of homoeopathy could be

explained by a combination of methodologicaldeficiencies and biased reporting. Conve6ely, we

postulated that the same biases could not explain theeffects observed in comparable placebo.controlled t als

of conventional medicine. Our results confirm these

hypotheses: when analyses were restricted to large trialsofhigher quality there was no convincing evidence thathomoeopathy was superior to placebo, whereas forconventionai medicine an important effect remained.Our results thus provide support for the hypothesis thatthe clinical effects of homoeopathy, but not those of

conventional medicine, are unspecific placebo orconterl effects.

In 1991, Kleilnen and colleagues')o argued that there isno reason to believe that compared with homoeopathy"the influence of publication bias, data massage, badmethodology, and so on is much less in conventionaimedicine". Indeed, we fourd that trials ofhomoeopathytended to be of higher methodological quality thanconventional-medicine tlials, altlough most t als ofeither type of medicine were oflow or uncertain quality.In both groups, smaller trials and those ofiower qualityshowed more beneficial treatmeni effects than largertrials and those of higher quality. Between-trialheterogeneity was less pronounced among homoe-opathy trials. This finding might be expected ifheterogeneity between homoeopathy trials is essentiallydue to biased repoding and conduct oftdals, whereas inthe conventional-medicine sample treatment effectsrepresented an additional relevant source of hetero-geneity. When we discussed resuits with practitioners ofhomoeopathy, they contended that classicai homoe-opathy and homoeopathic tteatment of chronicdisorders, in tials with longer follow-up, would yieldspecific effects. We addressed these points in additionalanalyses but found no strong evidence in support ofthese hypotheses.

This study directly compared the presence of biasesand their influence on effect estimates in homoeopathyand conventional-medicine trials. Identical definitionswere used, and data were abstracted independently bytwo obseflers. The search ofhomoeopathic publicationswas comprehensive, and we are confident that weidentified a near-complete set of published placebo-

controlled trials of homoeopathy. The identification ofunpublished studies is notoriously difhcrit, and weprobably missed some of these t als. Conventional-medicine t als were randomly selected from the largestexisting database of clinical triais (the CochnneControlled Trials Register) and were carefully matchedto homoeopathy triais fo! clinical subject and type ofoutcome.

Diffelent sources of bias are difficult to disentangle.The methodological quality of randomised rtials cannotbe reiiably assessed from published articles because

reporting on impofiant features of the methods isincomplete in many cases." Indeed, deficiencies inmethods of smaller trtals that were either not reported ornot assessed by us could also have conhibuted to theasymmetrical shape of the funnel plot. We have argued

elsewhere that the funnel plot should be seen not oniy as

a means of detecting publication bias, but also as a

generic tool for examination of small-study effects*thetendency for the smailer studies to show largertreatment effects.l'] Ifreporting is inadequate, study size

can be a more plecise measure of t al quality thanformal assessments of trial quality. Ve addressed thispossibility by modelling the effects expected in t als as

ww.thelancet.(om Vol366 August27,20os

O,ai.b, I

large as the largest trial included in our study; again, wefound litde evidence for an effect of homoeopathy butstlonger evidence for conventional medicine. Anotherlimitation of our study is the exclusive focus on thebeneficial effects of homoeopathy and conventionalmedicine, nther than on both benefits and risks.However, the trials included in the study were smali andlacked the powe! to reveal infrequent but importantadve$e effects. Futhermore, leporting on adverseeffects is inadequate even in larger trials.,' Acomprehensive and vaiid assessment of adverse effectswouid probably not have been possible within theframework of this study.

A previous review, which did not include a meta-analysis, also found that many trials of homoeopathyshow beneficiai effects but are of low methodologicalquality.'o A meta-analysis by Linde and co-workets', wasbased on an extensive literahrre search, which weupdated for ou! study, but it did not include trials ofconventiooal medicine. These lesearchers concludedthat theii results were "not compatible with thehypothesis that the clinical effects of homoeopathy arecompleteiy due to placebo". However, in a subsequent,more detailed analysis of the same data,,, they observedthat mole ligorous trials yielded smaller effect sizes andtlat thefu meta-analysis" ptobably "at leastoverestimated the ellects of homoeopathic treatmenls".ln a separate study, the same group obse ed that manytrials in complementary medicine have importantmethodological weaknesses.'zt Finally, a study of23 trialsof homoeopathy that were considered to be of highmethodological quality found that the few trials tharused objective endpoints were all negative.To

Our study has implications beyond the question ofwhether homoeopatlic remedies have specific effects.First, an important point to keep in mind is that mostsystematic reviews and meta-analyses ale based onrelativ+ few tdals. Simulation studies have shown thatdetecdon of bias is difhcult when meta,analyses arebased on a small number oftrials." For example, for theeight t als ofhomoeopathic remedies in acute infectionsof the upper lespiratory tract that were included in oursample, the pooled effect indicated a substantialbeneficial effect (odds ratio 0.36 195% CI 0.2 .500

and there was neither convincing evidence offunnel-plotasymmetry nor evidence that ttre effect di{fered betweenthe trial classified as of higher leporled quality and theremaining trials. Such sensitivity analyses might suggestthat there is robust evidence that the treatment underinvestigation works. However, the biases that areprevalent in these publications, as shown by our study,might plomote the conclusion that the results cannot betrusted. We submit that similar studies shouid be done inother 9?es of both complementary and conventionalmedicine. Such shrdies would "borow strength" from a

large number oftrials and provide empirical informationto assist reviewers and reade$ in the interyretatio4 of

wwwthelancet.com Vol 355 August27, 2oo5

findings from small meta-analyses that focus on aspecifrc intervention and disorder. Second, althoughimpotant progress has been made lately,'r,7 furtherresearch is needed to identify the dimensions ofmethodological quality that aie important in differentclinical contexis, diflerent outcomes, and different typesol trials. Finally, the relation between the probability ofpublication of a study and its methodological <lualityshould be examined in more detail.

We emphasise that our study, and the trials weexamined, exclusively addressed the narrow question ofwhether homoeopathic remedies have specific effects.Conte* effects can influence the effects ofinterventions,and the relationship between patient and cater might bea$ important ?athway mediating such effects.!*,,Practitionels of homoeopathy can form powelfulalliances with their patients, because patients and carerscommonly shale strong beliefs about the treatment'se{fectiveness, and other cultural beliefs, which might beboth empowering and restorative.b For some people,therefore, homoeopathy could be another tool thatcomplements conventional medicine, whereas othenmight see it as pwposeful and antiscientific deception ofpatients, which has no place in modern health care.Clearly, rather than doing further placebo-controlledtrials of homoeopathy,' futr-rre research efforts shouldfocus on the nafure ofcontext effects and on the place ofhomoeopathy in health-cale systems.

Our study powerfully illustates the interplay andcumulative effect of different sources of bias. Weacknowledge that to prove a negative is impossible,', l:utwe have shown that the effects seen in placebo-controlled tdals ofhomoeopathy are compatible with theplacebo hypothesis. By contnst, with identical methods,we found that the beneits ofconventional medicine afeunlikely to be explained by unspeciEc effects.

M Eggerconceived the studyand wrote the 6rst draft oftherelort.All the authors contributed to the final draft. A ShanBK HuwilerMiintener, L Naltey, S Dijrig, and P riini did the literaturesearches, ideniified eligibie studies, and ext.acteddata. P jiini advisedon data enraction and quality assessnent. D Pewsner helped with daiaefiraction and classifcarion ofhomoeopathy irials. A Shang,

I A C Sterne, P Jiini, and M Egger did the siatistjcal analyses andconlributed to daia inierpretation.

We dedare thatwehave no conflict ofinterest.

We thank lritz Grossenbacher iorvaluable helpwith literaturesearches. This studywas funded by the Conrplementary MedicineEvaluation Progiam (Programm €valuation der Komplementrrmedizin

IPEKI) of the Swiss Federal office ror Public Heahh. we thankMaiianne Amiet and Florian Mitscherlich from the PIK coordinatingoltrceand Felir Curtne! froft the Fede.al Oifice ofPublic Health forlheir suppon. Peter r0ni was supported by sranis from the SwissNationalScience Foundalion (grants no. 3231066177 and 1200-066178).

1 risenberg DM, Davis RB, Ettner SL. etal. Trends in aLtemativemedicine use in the United States,1990-1997:results ofa follo*up national suney. _IAMA 1998t 260: 1561-75.

7

I o*n,*

2 Emsl E. The role ofcomplementary and altemative medicine. BM.l2000:321: l1l3-ls

3 Vandenbroucke ,P. Homoeoparhy t.ials:going nowhere. tara!1997:350:824

4 Viclers A. Zouman C ABC of(omptemFnrary medkine:homo€opafiy. 8Ml 1999: ll9. l1l5-18.

5 lonas w, jdcobs l. Heal.ns wifi horreopath]. New york. w"rrerBo.k( 190a.

6 Schulte J. tfie.ts of porenrizarion in aqueous sotufions.tu Honeopath I 1999. EE: t5540.

7 Skrabanek P.Is homoeopathya placebo responsel I'bc.tt9g6tz.1107.

8 Gotzsche PC. T.ials ofhomeopathy. tana, 1991; t4t: 1513.9 Rennie D. Fan conduct and fair repolting of dinicat trials. JAMA

1999: 242: 17664a.10 Egger M, Davey Smiti c. Meta-analysis: bias in locationand

sel€ction of studies. BMJ 1998j 116: 61-{6.11 Schulz KI, Chalme6 I, Hayes Rl. Altman D. Empirical evidence of

bias: dimensions of helhodological qualiry a$ociated withestimates of treatment effects in conbolled trials..IAMA 1995;273:408-1,2.

12 Linde K, clausius N, Ram;ez c, et al. Are the clinicalellects ofhonoeopathy placebo effectsi A meta-anelysjs ofplacebo,controlled trials. tan rr l997i 350: 834-41.

ll Di&ersin K, Manheimer E. Wieland 5. Robinson KA. Lefebqe C.McDonald S. Development ofthe Cochnne Collaboration'sCENTRAL Register of contolled clinicaltrials. E al H.atrt p/otZOO2: 25. 1844.

14 Jijnr P. Alrman DC. Egger M. Assesrng the qualiry oj .onrrothdclnical trials. 8Ml 2001:lZl: a2-a6.

15 Hasselblad v, Hedges Lv. Meta-analysis ofscreening anddiagnostic tests. Prydrrol ErlL t995; t77: t67-78.

16 HiggiDs lPT, Thompson SG. Quantjrying hererogeneityin a meta-analylis. Sr4t Mrl 2002; 21: l519-58.

17 Steme tAC. Egger M. Funnelplots for deteding bias in meta-analysis: guid€iines on cho\.e of aas. I clin Epidemiol 2oot: S4l1046_55.

l8 Thompson SC. Sharp sJ Er?leining helerogenFiry in mera-.nalysis:a comp!rison o[merhods. Srar M.d tr9q; lE:2693-708.

t9 StemerAC, EggerM. Davey-Smjth C. lnvesligating add dealingwith publication andotherbiases in meta-anatysis. BMJ2001; t3:101-05.

20 Kleijrcn l, Knipschild p, ter Rier c. CliDicaltrials ofhomoeopathy.BMI 1991,t 302:31613.

2l Schulz KF. Randomised trials, human narure, and reponinSguidelines. lare, 1996; 3.1E: 596-98.

22 StemeJAC. Cavaghan Dt. Egger M. publ'cation and retated bias inmela{nalysis. po$e! ol sr!-rsdcel lesrs dnd prevrten(e rn rhelle?turc J Clin Epd.tuial 2000: 53: Ilt !2i

23 JoaDnidis tP, ku t. completeness of safery.eporting inrandomized trials:an €valuation of7 medical areas. JAMA 20011zES:4J743.

24 Linde K. Scholz M. R2mire c, Clausius N, Melchalt D,lonasWB.lmpa.r ofstudy quahry on ourome in Dlrcebo-conLrolted lnats ofhomeopathy.I Clin Eyid,hol 1999, 52: 6tl-36.

25 Linde K, ronas wB. Melchan D, wiilich S. The nethodologicalquality of randomized controlled trials of homeopathy, her6almedicines and acupuncture . Itt J Epid.hinl lWL 30:526-31.

16 Morison B. trlfold R,. Imsi E. Mehodologka nsour end rpsutrsof clinical lrials of homoeoparh'c remedies. p.tlstoa,,000j ll.132-38.

27 Moher D. Pham B. lones A. fl al. Does queliry or relons of!Jndomised rruls affed estrmales ofintFtuenl'on efficacy leportedin meta-analysesf lrrrr, 1998; 3 52: 609-13.

28 Di Blasi z, Harkness €, Ernst E, ceorsiou A, Kteiinen J. jnfluence

ofcontext effecis on health oulcomes: a systematic review. l4rrrt20011 357, 75742.

29 KleiinenJ, de Craen Al. van Everdingen l, KJol L. placebo effecr indoubl€-blind clinical bials: a rwiN of inleractio,s wirhmedications. bnet 1994: !4L rJ4749.

l0 KaplLhukTJ. Fiserberg DM. Thp pF (uanvF " ppea ofahernarve

ntedicne Ann t^km M.d1998. l29 t0bl-6531 Allrnan DC, Bland JM- Absence of evidence is nor evidence of

absence. BMl 1995; 311 a85.

732 M.thelancet.coh vol366 Auqun27,zoos

EXHIBIT B

I Clin EpidemiolVol. 52, No. ?, pp. 631-616, 1999Copyright @ 1999 Eisevier Science Inc. All righb reserved. ffi

ET.sEVIER

0895-4356i99/$-see 6ont marterP1l 50895.4356(99)00048-?

Impact of Study Quality on Outcome inPlacebo-Controlled Trials of Homeopathy

Klnus Linde,l '* Michnel SchoLT,z Gilbert Ramirez,3 Nicola Clausius,l Dieter Mebhart,landWayne B. lonasa

IMUNcHENER MooELL CeNrR-E eon CoN{prsrlrNrany Mrorcrns Rrsranca, Depanrr.asxr or lNrgRNaL MrotcrNr Il, TecuNrscuoUNlvEFsrrAT MiNcHeN, MuNrcn, GeRMeNy; 2lls,rt].urr ron Msorcer Srerrsrlcs awo ErtoevloLocv, Trcnr.rIscH! IJNIVERSITiT

MuNcasN, Muwrcn, Genuerqv; lDeeexturur or PugI-rc HeeLtH aNo PnrvrNrve Meotcrvs, l-lvlvensl,tv or Nonrntnr TrxasHsarrg ScrrNcr Cruren, Fo*r Woern, Trxes; ar.ro 4Onrtce oe Arrsrwarrvp Meotctle, NartouaL Insrlrures or HeaLttt,

Be.Iursoe, M.rnvu+r.ro

ABSTRACT. !7e investigated the influence of indicarors of methodological quality on study outcome in a set

of 89 placebo-controlled clinical trials of homoeopathy in rhree dif{erenr ways: (l ) The results ofstudies meeting

single crireria (explicit statement of random allocation, allocation concealment, double,blinding, compieteness

of follow-up) of methodological quality were compared with those of studies not meeting the criteria inunivariate and multivariate analyses; (2) The results ofstudies scoring above and below predefined scores in two

quality assessment scales were compared; (3) Primary studies were consecutively entercd into a cumulative meta-

analysis according to the summary scores derived from the quaiity assessment scales. All analyses were performed

lsing meta-regression methods. Studies that were explicitly €ndomized and were doubie-blind as well as studies

scoring above the cut-points yielded significantly less positive resuks than studies not meeting the criteria. ln rhe

cumulative meta-analyses, there was a rrend for increasing effect sizes when more studies wich lower-qualiry

scores were added. However, there was no linear relationship between quaLity scores and study outcome. !0e

conclude that in the study ser investigared, rhere was clear evrdence that studies with better methodological

quality cended to yield less positive resuits. Because summarizing disparate study features into a singl€ score is

problemaric, meta-regression methods simultaneously invesrigating the influence of single study features seem

rhe best method {or investigating the impact of study quality on outcome. J cLrN EprDEMloL 52:7:631-636, 1999.

@ 1999 Elsevier Science Inc.

KEYWORDS. Study quality, meta,analysis, homeopathy, randomized controlled crials, bias

INTRODUCTION

There is increasing evidence that more rigorous trials tend

ro yield less optimistic results than trials with less precau'

tions against bias [1-J]. ln a controversial area like comple-

mentary and alternative medicine, in which many trials are

performed by proponents with little research experience

and published in non-peer-reviewed journals, one might

expect that this problem could be even more pronounced.

Quality assessment, therefore, should be a mandatory ele-

ment ot !yslematic revrews ofsuch rherapies

However, the assessment of clinical trial quality is prob-

lematic. Although a number of instruments designed for

this purpose have been used (see [4] for an overview), there

*Addres conespondence to: Dr. K. Linde, Centre for ComplementarvMedicine Research, Depanment o( Internat Medicine ll, Technische Uni-vesiriir Miinchen, Kaiserstt. 9. 80801 Munich, Gerrnanv.

Accepted for publicarion on B March 1999.

is no widely accepted single method. There is not even a

uniform definition of quality. Most assessment methods fo-

cus on criteria dealing with the likelihood of bias such as

randomization, blinding, completeness of follow-up, and

intent-to-treat analysis. The "dimension" assessed withsuch criteria is that of "methodological quality" or "internalvalidity." There is also substantial controversy over whether

qualiry can be assessed with scoring systems that combine,

weight, and sum a catalogue of critetia, or whether the in'fluence of single-quality components should be investi-gated separately [5,6].

We compared three different approaches to investigate

rhe impact of quality aspects on outcome in a published

meta,analysis of placebo-controlled trials of homeopathy

[7]: ( I ) investigating the influence of single-quality compo-

nents on the outcome, (2) using cut-off points in quality

scores as inclusion criterion, and (3) entering trials intometa-analysis consecutively according to quality scores

(cumulative meta-analysis).

632

METHODS

The general methods of our review have been described indetail elsewhere [7] and are reported here only in a cursorymanner. In this article, we focus on the specific methods re-garding study quality and outcome.

Selection Ctitetia ond Literatute Seatch

\fle included all available double'blind and/or randomizedclinical trials in which a homeopathic intervention and aplacebo had been compared for preventive or therapeuticpurposes. Eligible trials were identified *tough multiple sources

inciuding MEDLINE, Embase, complementary medicinedatabases, contacts with researchers, and checking bibliog-raphies of identified articles. A rotal of 1 19 studies meetingthe inclusion criteria were identified, and 89 presented suf-ficient data to be included in the quantitative meta-analy-sis. Study characteristics and results were extracted by twoindependent reviewers using a pretested form.

As s es sment of Methodotogical QtalityFor assessing the methodological quality, the scale by Jadadet ai. [8] and a system developed by one of us (KL) were

used. The Jadad scale is a scoring system that has threeitems adding up to a maximum score offive points. Zero, 1,

or 2 points can be given for randomization (explicit state-

ment that allocation was randomized and description of an

adequate generation of the random sequence), 0, 1, or 2

points for double-blinding (explicit statement that patients

and evaluators were blinded and thar treatments were in-distinguishable), 0 or 1 point for the description of drop-outs and withdrawals (numbers and reasons in all comparedgroups given separately).

The second scale (lnternal Validity Scale = IVS; de-

tailed description submitted for publication), which has

TABLE l. Component and minimum 6core analysis

K. Linde er al.

been used in a slightly modified version in other rcviews oncomplementary medicine also [9,10], has seven items scor-ing 0,0.5, or 1 point each. ltem I is a statement ofrandomallocation; item 2 assesses the adequacy of randomizationconcealment; item 3, baseline comparability; irem 4, blind-ing of patients; item 5, blinding of evaluators; item 6, thelikelihood of selection bias after allocation; and item 7, theadequacy of the statistical analysis. Both scales were opera-tionalized with detailed instructions (available from the au-thors). Al1 trials were assessed by at least two independentreviewers. Each reviewer used both quality scales.

Stctisticol Anafses

Odds ratios (OR) and their respective 95olo confidence in-tervals (957o CI) were calculated for each trial, with values )I indicaring rhar the homeopathic rnterventron was moreeffective than placebo. For this analysis, the data from theoriginal meta-analysis l7l were reanallzed by using meta-regression methods allowing arnong-trial heterogeneity.First, ORs were naturai log-transformed before analysis.

This results in a treatment effect y, : ln(OR') ofstudy i, i =1,...,n. \7e assumed the meta-rcgression model [11,12]

Y = x;F + bi+ ei, (1)

where x, denotes a vector of known covariates for study i,b, - N(0,r2) is a study-specific random effect reflecting theresidual among-trial heterogeneity and e, - N(0,o'z) withthe individual within-trial variance o,2. The variance o,z

was estimated by the known sample variance sj of the ithtrial. For b,, e, independent var(y,) equals .r2 * s'2. Underthis model assumption, the estimated linear predictor r,prcpresents the mean effect ofall triaLs having covariate vec-

tor x,, and b, is the ith rrial's deviation from this mean. Pa-

rameter estimares were obtained via a likelihood-basedmethod (resrricted maximum likelihood, REML [13]) im-

n OR (95olocl)

Not met

oR (95ol.Cr) ROR(95ol.CI) Heterogeneity

Component analysis (univariate)Explicirly randomizedAdcquate concealmentDouble-blindingComplete follow-up

Component analysis (multivariate)Explicitly randomizedAdequate concealmentDouble-blindingComplete follow-up

Minimrlm score analysis (univariate)

Jadad score > 2

lV scale > 4.5Both

z.z3 (1.81-2.75) 252.00 (1.50-2.65) 552.18 (1.83 2.60) 83.03 (2.124.33) 6l

2.rs (t.78-1.59)z.tt (t.62-2.74)2.08 (t.7 5-1.47 )1.70 (t.97-3.72)

r.81 (1.4r-2.32) 491.95 (1.50,2.59) 55t.'tz (1.28-2.3t\ 63

64348178

4034z6

3.4A (2.324.97)2.83 (1.23-3-58)9.14 (4.81-17.4)2.29 (1_84-2.86)

3.36 (2.374.78)2.s0 (2.00-l.l l )

7.85 (4.10-15.0)2.21 (1.81-2.69)

3.23 (2.534.13)2.92 (2.29-3.73)2.95 (2.37-3.67)

0.66 (0.43-1.0r)0.71 (0.49-1.02)0.24 (a.t74.46)r.r I (0.88-2.00)

0.64 (0.43 {.94)0.84 (0.60-r.18)0.26 (0.14-O.51)1.73 (0.85-1.77)

0.56 (0.40-0.79)0.6? (0.474.97)0.58 (0.40-0.79)

0.41 (0.21-0.8?)0.r9 (0.22-0.87)0.31 (0.18-0.71)0.44 (0.26-0.91)

0.28 (0.15-0.69)

0.34 (0.19-0.78)o.4o (0.23-o.86)0.36 (0.20-{.82)

Impact of Study Quality on Outcomes in Placebo.Controlled Trials ofHomeoparhy 633

TABLE 2. Pooled odds ratios of shrdies with the same quality scores and cumulative meta.analysis according to quality

IV Score(n studies) oR (9solocr)

IV score(n studies )

CumulativeoR (95%Cr)

7.0 (5)6.5 (2)

6.0 (4)

5.5 (13)

5.0 (10)

4.s (e)

4.0 (13)

3.5 (9)

3.0 (B)

2.5 (5)

2.0 (9)

1.0 (2)

Jadad

5 (10)

4 (ll)3 (1e)

z (32)

1(r5)0 (2t

:7.0 (5)

=6.5 (7 )

>6.0 (11)

>5.5 (24)>5.0 (34)

-_4.5 (43)>4.0 (56)

=t.5 (65)

>3.0 (73)

>25 (78\>2.0 (87)

all (89)

Jadad

=5 (10)

>4 (Zt)>3 (40)

>z (72)> I (8?)

all (89)

1.55 (0.77-J.r0)2.AZ (r .06-3 .85\1.67 (1.tL254t1.7 | (1.36-2.15)

1.89 (1.s3-2.35)

2.09 (r.69 2.59)2.29 (1.88-2.80)

2.35 (1.9+233)2.29 (1.92_2.'t 4)2.4212.00-2.9r)2-44 (Z .02-2.93)2.47 Q.a6-2.97\

z.0o (t.37,2.9t)1.68 (r.29-2.r8)r.7311.44-2.08)2.26 (1.87,2.',l3)

2.44 (Z.OZ-2.93)

2.47 (2.06-2.9?)

1.55 (0.?7-3.10)

4.6r (0.9\-23.4)

t -35 (0-74-Z.4sll.?4 (1.38-2.19)

2.95 (2.404.34)4.33 (1.86-r0.1)

3.16 (2.00+.98)2.82 (t.57-5.05)1.97 (l.r+3.40)3.91 (0.97-15.8)

2.47 (r.rt-5.52)6.92 (t.53-3t.41

7.00 (1.3?-2.91)

t .42 (t .02-t .99)t .83 (t A+2 .34)

3.24 (2.274.63)3.58 (2.11-6.08)

6.92 (r.53,31.4)

plemented in PROC MIXED of the SAS stem release 6.12(SAS Institute, Cary, NC).

A likelihood ratio test was used to examine the null hy-pothesis of no residual among trial heterogeneity (r2 : 0)by comparing the likelihood of this restricted model withthat of a model where rz is estimated. lf the restrictionholds, model (1) reduces to the fixed-effects model. The re-

sults are given either on the original OR-scale or in terms ofratios of odds ratios (ROR). A ROR of 1.0 for a dichoto-mous quality criterion indicares no difference between thetwo ORs. ln our coding lcheme, a ROR < 1,0 always re-flecs that trials meeting the quality criterion had a smaller ORcompared with the trial not fulfilling this quality standard.

Approaches to Test the I'7'pact of Q1/,d,LitJ on Outcome

The relationship between study quality indicators and out-come was tested by three different approaches:

1. Cornponent analysis: Comparison of pooled effect sizes

of studies meeting a criterion and of studies not meetinga criterion (explicit statement that allocation to groups

was randomized, adequate concealment of randomiza-tion, double-blinding, complete follow-up, or intent-to-treat analysis). Both univariate and multivariate analy-ses were performed.

2. Minimum score analysis: Comparison of pooled effectsizes of studies scoring above or below predefined cut-off-points (>3 in the Jadad score, >5 in the IVS score,

separately and combined).3. Cumulative meta-analysis: Primary studies were entered

consecutively into meta-analysis according to the qual-ity scores received (Jadad and IV scale).

RESULTS

The mean quality score of the 89 trials was 2.58 (SD 1.29)from a maximum offive on the Jadad scale and 4.20 (1.46)

from a maximum of seven on the IV scale. Forty (45%) ofthe trials scored over the predefined quality cut-off (three

or higher) on the Jadad scale, and 34 (38%, five or higher)on the IV scale. Twenty,six (29o/o\ vials scored above thecut-off in both scales. Sixty-four (72%) trials explicitlystated that allocation to groups was randomized; 21 (24olo)

made no ctear statement (only double-blinding mentioned);and 4 (4ok) trials had used quasi-random methods. For 34(38%) trials, an adequate method of allocation conceal-ment (mostly consecutively numbered drug containers) was

reported; 50 (56010) did not describe the method; and in 5(6%) concealment was inadequate. Eighty-ooe (917o) stud-ies were double-blind;3 (3o ), single-blind; and in 5 (6Eo)

there was no statement about blinding. Twenty-eight

634

(37olo) trials either appeared to have included all patientsrandomized into the analysis or did an intent-to-treat anal-ysis; in 20 studies (22%), major bias seemed unlikely; in 19(210,6) trials major bias seemed possible or likely; and in 22(25olo), the item could not be rated owing to insufficient in-formation.

The pooled mean random effects OR all 89 studies in-cluded was 2.45 (95Vo CI 2.05-2.93\. Because there was

strong heterogeneity Gt : 0.43;95okCI4.254.90; P = 2.4 xl0-rl), only random effects analyses are reported here. [fthe analysis was restricted to trials that were explicitly ran-

domized, adequately concealed, or double-blinded, the re-

sulting mean ORs were 2.23 (a decrease of 9olo comparedwith when all 89 trials were pooled), 2.00 ( 18% decrease),

and 2.18 (11"/" decrease) respectively (see Table 1). The117o reduction of the overall OR by excluding eight non-double-blind trials indicates that these studies had yieldedgreatly inflated effect sizes and their removal considerablydecreased heterogeneity (see Table 1, last column). Trialswith complete follow-up yielded a slightly larger mean ORrhan did rrials not meering rhis criterion.

Double.blinding had the strongest impact on outcome in

both the univariate and the multivariate analysis. The ratioof odds ratios (ROR) were 0.24 (g5yocl 0.12-0.46; P <0.0001) and 0.26 (95%CI 0.14-0.51; P = 0.0002), respec-

tively. ln the mulitvariate analysis also the explicit state-

ment of randomization reached significance (ROR 0.64;

95'kCl 0.434.94; P = 0.03). The influence of allocation

OR100.0

10.0

K. Linde er al.

concealment and complete follow-up was not statisticallysignificant.

If the analysis was restricted to trials that scored three orhigher on the Jadad scale, five or higher on the lV scale, orboth, the mean OR was decreased ro I.8I (1.41-2.32,28Vodecrease), 1.97 (1.50-2.59,229o decrease), and 1.72 (1.28-231, 3AVo decrease), respectively. The heterogeneity was

reduced most (350/o) in the multivariate component analysis.

\X4ren the primary studies were entered into the cumula-tive meta-analysis in descending order of IVS scores, therewas a trend for increasing effect sizes when studies withlower quality were added (see Table 2, upper part). If onlythe five studies scoring highest (seven points) were pooled,the resulting OR was 1.55 (0.79-3.01), and for the studies

scoring more than two thirds of the iterns (>5 points) 1.89(I.53-2.35\. When using the Jadad scale, the mean OR ofthe highesr scoring trials was more positivc (2.00; 1.3?-2.91) than if trials scoring four or three of the possible fivepoints were added (see Table 2, lower part).

When the OR of primary studies are plotted against qual-

ity scores, it becomes obvious that there is no clear linearrelationship between these two panmeters (Fig. I and 2).

DISCUSSION

Our analyses provide clear evidence that in the study set in-vestigated more rigorous trials tended to yield smaller effect

sizes. The most plausible explanation ofthis finding is bias.

2345Jadad score

FIGURE l. Scatterplot of odds ratios of 89 trials of homeopathy plotted against Jadad score (circle areas are scaled propordonalto the variance of the trials odds ratio).

eI

B

6H6Afo,

lmpact ofStudy Qualrry on Outcomes in Placebo-Controlled Trials of Homeopathy

The results are comparable to those from similar analyses inconventional medicine. However, the influence of dif{erentquality factors depends on the context. Schulz et al. [1]found in their analysis covering 250 trials from 33 meta'analyses from the Cochrane Pregnancy and Childbirth da-

tabase that aLlocation concealment was the most relevantqualiry factor. Double-blinded trials also yielded signifi-cantly smaller effect sizes; however, the method ofsequencegeneration for the allocation had no influence, and trialswith complete follow-up had slightly more positive out,comes. In an analysis of 127 trials from I I meta-analyses ona variety ofconditions by Moher et al. [3], the findings were

similar to those of Schulz et aI. for allocation concealmentand sequence genemtion, whereas double-blinding had noeffect. ln the homeopathy studies evaluated here, double-blinding proved to be the most relevant influence factor. Apossible explanation is that bias from this factor might beparticularly important, as in these trials homeopathy was

mainly used for mild and chronic conditions for whichthere are few objective outcome measures.

The evidence ofbias weakens the findings ofour originalmeta-analysis [7]. Since we completed our literature search

in 1995, a considerable number of new homeopathy trialshave been published. The fact that a number of the newhigh-quality trials (e.c. [14,15]) have negative results, and atecent update of our review for the most "original" subtype

of homeopathy (classical or individualized homeopathy

116]), seem to confirm ahe finding that more rigorous trials

635

have less,promising results. lt seems, therefore, likely thatout meta-analysis [7] at least overestimated the effects ofhomeopathic treatments.

The use of the predefined cut-off points to separatehigher- and lower-quality trials of homeopathy producedmo.e conservative effect estimates than the use of singlecomponents in the univariate analysis. Also, in the cumula'tive meta-analyses, pooled effect sizes tended to increasewhen studies with lower scores were entered. However, thescatterplots reveal that a convincing linear relationshipdoes not exist. There are several possible explanations forthis result. ( 1 ) lf the overall hypothesis that all homeopathyis placebo is inco[ect, our study set is clinically extremelyheterogeneous and the "true" effect sizes might vary stronglybetween treatment and conditions. This heterogeneitymight obscure the slight correlation suggested by the cumu-lative meta analysis. Other explanations are (2) that the re-

lationship between quality and outcome is rather weak or(l) that the scores used ale not suitable to detect an exist-ing relationship. ln a set of seven (clinically more homoge.neous) meta.analyses applying the widely used scale byChalmers er al, [17], Emerson and coworkers [18] also failedto find a clear relationship between quality and outcome.This would support the explanations (2) and (3).

Quality scores have been criticized for mixing disparate\rudy features rnto a nontranqparent quantitative e5timate

and inrroducing an arbitrary element into the analysis {191.

Empirical research providing sound evidence on the valid-

OR100.0

10.0

internal validity score

FIGURE 2. Scatterplot of odds ratios of 89 trials of homeopathy plotted against intemal validity (IV) score (crfcle areas arescaled proportional to the variance ofthe trials odds ratio).

1.0

0.1

o (o)€.-,eo6 o i:t.o,to_)

ao

oco^o (.J

-id(i,g /-'jc\r'H UO

o

636

ity and comparability of the various instruments is almostnonexistent [20]. Our results indicate that quality scores

might be useful and can lead to similar results as compo-nent analysis. Nevertheless, if feasible, meta-regressionwith multivariate modeling of the influence of single com-ponents seems to us the first choice to test the influence ofquality on outcome, as this approach is transparent and has

a straighdotward rationale.The smal number of primary studies in most meta-analyses

strongly limits the power of meta-regression. lt seems reason.

able to use minimum scores in such situations to investigatewhether high-quality studies yield resuls that are different fiomlower-quality srudies. This approach has been used in the litera-ture (e.g. [2,21]). A look on our scatter plot and the results byEmerson et ai. [18] suggest *ut in small study sets dris might be

haz.ardous. Also, using quality scores to weight primary studies

[3] in meta-analysis seems problernatic based on such results.'Whether

using summary scores or components, readersshould be aware that all quality assessments based on pub'Lished reports are very crude. For example, a single phrase(which even mighr have been inch.rded by the authors butdeleted after editorial review because of lack of space)might be used to decide whether a trial is assessed as ade-quately concealed or not. Even if we assume that allocationconcealment is a relevant indicator, the presence or lack ofa statement on it does not necessary mean that a trial is

high or low quality. Guidelines such as the CONSORTstatement [22] should improve the reporting of key meth.odological issues in future years.

There is a clear need to investigate the influence ofstudydesign factors and aspects of methodological quaLity onstudy outcomes in more detail. The available tools for qual-ity assessment are far from being perfect. In our opinion,there is no reason why single-quality criteria and summary

scores should not be used in parallel. However, in thepresent situation we recommend that investigators shouldnot rely on quality scoring methods alone when perforrning

sensitivity analyses. Such analyses should be interpretedwith great caution, as their accuracy and relevance may be

greatly influenced by context and reporting factors.

This studl was partiall:1 suppoftea bJ the Karl and Yetonica Cartens-Srrfr*ng and the NIAMS {ant 5 U24 -AR43346-a2. The Centre fmCompbmenttrl Medicire Research is fundzd 11 *e Bavarian Parlanmt.

References1. Schulz KF, Chalmers l, Hayes RJ, Altman DG. Empirical evi-

dence of bias. JAMA 1995 ; 27 3 : 40B4lZ.2. Khan KS, Daya S, Jadad AR. The importance of quality of

primary studies in producing unbiased sysremalic reviews.Arch lntern Med 1996;156 661 666.

l. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et

al. Does quality of reports ofrandomised trials affect estimates

K. Linde er dl

of intervention efficacy reported in meta-analysesl l,ancet1998t 352: 649-613.

4. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, WalshS. Assessing rhe quality of randomized controlled trials: Anannotated bibliography ofscales and checklisrs. Conrrol ClioTrials 1995: 16:62-73.

5. Greenland S. Quality scores ate useless and polentially mis-leading. Am J Epidemiol 1994; 140: 100-301.

6. Olkin l. lnvited commentary: Re: "A crirical look at somepopular meta-analytic methods." Am J Epidemiol 1994; 140:297 -299.

7. Linde K, Clausius N, Ramirez G, Melchart D, Eitel F, HedgesLV, et al. Are rhe clinical effects of homoeopathy placebo ef-fects? A meta-analysis of placebo-controlled nials. l,ancet1997;350, 83+843.

8. Jadad AR, Moore RA, Carrol D, Jenkinson C, Reynolds DJM,Lavaghan DJ, et al. Assessing the qLrality ofreports ofrandom-ized clinical rrials: is blinding necessaryl Control Clin Trials1996: 17: 1 12.

9. Linde K, Worku F, Srdr \f, lyiesner-Zechweister M, Pothen-ceuu R, Veinschitz T. Randomized clinical trials of acupunc-ture for asthma-a systematic revie\r. Fortschr Komple-mentiirmed 1996; 3: 148-155.

10. Linde K, Ramirez G, Mulrow CD, Pauls A, WeidenhammerV, Melchart D. Sc John's wort for depression-an overviewand mera-analysis of randomised clinical ftials. BMJ 1996;313:253 Z5B.

11. DerSimonian R, Laird N. Meta-analysis in clinical trials.Control Clia Trials 1986;7: 177-188.

12. Berkey CS, Hoaglin DC, Mosteller F, Coldicz GA. A random-effecrs regression model for mera-analysis. Stat Med 1995; 14:

39541r.13. Laird NM, Ware JR. Random.effects models of longitudinal

data. Biometrics 1982; 38 963-97 4.14. \0alach H, Haeusler W, Lowes T, Mussbach D, Schawell U,

Springer \0. Classical homeopathic trealment of chronicheadache. Cephalalgia 1997; 17: 119-126.

15. Whrrmarsh TE, Colescon-Shields DM, Sreiner TJ. Double-blind randomized placebo-controlled study of homoeopathicprophylaxis of migraine. Cephalalgia 1997; 1?: 600-604.

16. Linde K. Melcharr D. Randomized controlled trials of individ-ualized homeopathy-a state-of,the-art review. J Alt Com-plement Med 1998; 4: 171-388.

17. Chalmers TC, Smith H, Blackbum B, Silverman B,Schroeder B, Reirman D. A method for assessing the qualityof a randomized control crial. Control Clin Trials 19811 2:

I1-J9.18. Emerson jD, Burdick E, Hoaglin DC, Mosteller F, Chalmers

TC. An empirical study of the possible relation of trearmenrdifferences to quality scores in controlled randomized clinicaltrials. Conttol Clin Trials 1990; I l, 339-352.

19. Greenland S. Mera,analysis. In: Rothman K, Greenland S.

Modern Epidemiology. Philadelphia, PA: Lippincott Raven;199& 643473.

20. Moher D, Jadad AR, Tugwell P. Assessing the quality of ran-domized controlled crials. Current issues and future directions.Int J Technol Assessment Health Care 1996; 12:195-208.

21. Nurmohamed MT, Rosendaal FR, Bueller HR, Dekker E,Howmes DV, Vandenbroucke JP. Low.molecular weight hep-arin versus standard heparin in general and orthopedic sur.gery: A meta-anaLysis. Laacet 1997;340, 152-156.

17.. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin J.Improving rhe quality of reporting of randomized trials: TheCONSORT statement. JAMA 1996; 27 6: 637 439.

EXHIBIT C

A systematic review of systematic reviews of homeopathy

E. Ernst

I lomcopachy renrains onc' of tht' urost controversial subjcccs in therapcutics. l"hisarticle is an irttenlpt lo clarifi its efkctiveness basecl on recent systemille revlcws.Illectrouic dltabase's rvere searched l?cr syscenatic rt'views/rletn analysis on thc sub-jcct. Seventecn ar:ticles frrlfllled the irclusion/exclusion criceria. Six of thern relatedto re-analyses of one lardnrark meta-analysis. Collectively they implied that theoverall positive result of tbis rreta-analysis is not supporred by a critical analysis ofthe data. Eleven indcperdent systematic reviews rvere located. Collectively theyfailcd to provide strong evidcrcc in favour of ho:ncopatirv In particular, tllcrc \4?J

no conclition which rt'sponds convincingly bctter to horlcopathic treatnrent thaoto pilcebo or other cortrcl interventions. Similarl_v, chcle rv.rs no honeopathic:remcclv that w?s detrrolrstntecl ro vield clinical effects that are couvilcingly differeltfronr piacebo. lt is corcluded thrt rhc best clinical evidence for honrcopathy available

to date does not warrant positivc r€conrmendations tbr its use in clinical practice.

Keytr,ords: akernative rnedicine, clinic:rl trials, horneopathy, nteta lnalysr'. sy\tcnrtic

Intloduction

Honeopathy is a therapeutic mcthod using prepantior.sof substalces whose effects when administered to healthv

subjects corresponcl to the marifestatiorx of the disorder(symptoms, clinical signs, pathological states) il rhe indi-vidual patient. The method r'vas developed by Sanrnel

Hahncrnann (1755 1843) and is norv practised through-our the werld f 1]. Homeoparhy is based orr trvo nainprincipals [1-3]. According to thr' 'Like cures like' prin-ciple, parients with particular sillDs xnd s),mptoms can be

helped by a homeopathic remedy that produces these

signs and syinpto*rs iu healthy individuals. Accorcling tothe second principle, homeopathic renreclies retain bio-Jogical activitl, after repeacecl cliltrtion lrrd sLrcussion even

rvhen cliluted beyoncl Avogaclrols nLrnrber.

Few therapies have atuacteci nrorc debate and contlo-vctsy thau homc'opathy.

-i"hrougbout lts 200-year history,

critics have pointed out that its vely grriociplcs fly in tJ:e

face of science, while proponents have nairtained thatit is nar row minded to reject;ur overtly he$ful approach

to healing only because ooe canrrot explair how it rnigJrt

work [2|. Similarly, proponcnts bave quoted seemitrglv

Coresponden.er Professor E. Ernst, Deparrmenr of Complementary l4edicine,

Schooj ot Spon and Health scien.es, Universiry of Exeter, 25 Victoria Park

Road, Exeter EX2 4NI UK. E-mail: e.Ernsr@exererac.uk

Re.etued - 20U, accepred " 2002.

@ 2002 Bhckwell Science Ltd W J Ah Phomo.ot.54.571-582

rigorous trials that snggest eflicacy, while critics had linletrouble citing equally rigorous studies that implied theopposite.

"fhc existence of contraclicring eviclence is not unusualin therapeutics. Oue soltrtion to resoh.e srrch contradic-tion-\ is to c:onduct s),slcmatic levie\vs rnc{ nreta-analvses

of rrgorous studies. ln I997, Lindc et oi. [-jl dicl just thar.fhe conclusions of this techr.rically superb neta-aoalysis

expressed the notion that homeopathic nrcclicines are

more than mere placebos. The authors also stated that noindication was identified in u,hich horncoprthy is clearlysupcrior to piaccbo. Despite this ard other caveatr,

homcopaths wor-lelrvidc celcbntcd thrs publicarion as theultirnate proof of their oeatnent. Silcc- thcn, a ilurry ofinceresl ilr honreopathy has errrerged, anti ser,'eral flrtlrcrsvstr:nrntic rsviews havc bren pLrblishcd.This article is an

atempt to criticalJy eraluatc' all such pepers published

since 1997 rvith a vic'w to defining the clinical effecrive-ness of hou-reopatbic rnedicines.

Methods

l-iterarun sealchers weLe carricd out iLr the lbllor'virrg

databases: Metllirc (via Pubmc'cl), L.mbase, Amed,CTSCOM (tiorr inception to October 2001). lhe serrch

terms used rvere hon-reopath..., homoeopath ...,clinical trial, meta-analysis, s)/stenatic rcview, etlicacy,cllectiveness. Io addition, other experts ir thc field

t_

E. Ernsr

(t = 5) lvere consulted and my owtt, extensive files wcrestudied. The bibliographies of all articles thus iocaredwere scamed for firther relevant teferences. No larrguagerestr ic!ions rvere applied.

Onl), systentatic reviervs (including rneta-anal1'ses) ofcontrolled clinical tlials of homcopachy r"r,ith hunuupatienls or voluntee$ were includecl. Non-sysrematicreviews, overvielvs, clinical trials and reviervs of non-clinical investigations were excluded. All arricles rvercevaluated by the present author. The folLowine informa-tion rvas extracted fiom the original artidesi il)chrsion/exclusioo criteria, cotal sanple size, assessnrcuc of rnethodoiogical qualiry results of mcta-analyses. ovcrall corr,clusicln of the arrrhors.

Results

Six re-analyses of Linde ef a/.t or:iginal me'ta-analysis [3]u,'ere located [4-9]. Tlble 1 suturarizes kev dare frolnthese prrblications. Tire results of these rc-anaiyses tlenr,onstnte thar the more rigQrous trials arc associatcd rvithsrnaller effect sizes which. in turn, render the o\-emll cllectinsignificant 15, 6, 81. One re*analysis sussests rhar rbe

initial positive meta aralytic result [3'] was largely clue topublication bias [9], a notion that had been considercdby the original authors brlt rvas rejected by theli. Mosrnotably, pr:rhaps, the authors of thc originrl nrera,arulysis

l3l conclrrded that their rc' analysis 'weakent'cl the findingsof cheir original meta-analvsis' [6]. Coliectivel,v these re-analyses imply tl.ur the initial conclusions of L-irrde cl al.

f3l u'as not sr.rpportc'd by critical evaluation of rheir data.

lrr additrou. ll irrdepcndenr sysrenr.rtic r'(\r,.\v5 wirllocated i10 20]. Table 2 sumrnarizes key data fiom thesepublications. CoJlectively tlre findings do not pror.ide'

strong evidence iu favour of homeopathy'. Wirh theexceptioD of postopc'rative iler.rs [:10] and influer.rza li7](see bclorv) tl.rerc is no condition for which honeopathyis corvincingiy cfictive [1.0, 11, 13, 18-201. Arnica, thcnlost fJequendy tested lloneopathic retrledy, is notdemonstrab)y diferent from pJaccbo f12, 151. Onchon.reopathic remecly (oscilJococcinun.r) rvas fbund to be

superior to placebo as a treahleDt lnd prcr.ention ofinflucnza but thc eflect size r.vas small and thercfore ofdebarablc clinical teJcv*lcc 117]. Morcover, rhc \'ololneolthc evidence for osciilococcirrurr-r is srrall ancl rlrert'forcDot lirily collclurrvc. ()ttr rystclrrati.' r('vr'\\' ot \.,r'!,'u\bon-reopathic m.:dicines for postoperative ileus prodlrccdan over:all positivc result [10].Yet ser''erai caveats need tobe taken into account, most irnpor:tant)y the fict rhat thcr

clcfilitive study designed as a Drulticentre trial ro lcp-licatcscveral of snrallcr: studies failed to dcmonstr:ltc a positive:

effecr f101. Onc indeperdent revierv of all honreopathicl\C'l's regardlcss of indication or rype of rcrred), ),ieldeila positive result [16l.Yet the statisticnl appro:rch to gen-

578

eete this r€sult r,r.'as of debatable va.lidity and the authorsare keen to point out that rheir overa.ll result is weak audnot sufficieDt lbr definitivc recontmendations

Discrrssion

CoLlectivcly rhese data do rrot provide sound cvidenccthat homeopathic remedies are clinically diffcrcnt fromplacebos. Horvcver, the present analysis has scveral limi-tatior'ls that should be kept in rnind when interprering itsconcL"rsions. Iiven thorrgh a thoroirgh sealch strategy was

adoptcd. tJrer:e is no absolute quaranree that all rclevanrartjclcs wen' locltecl. Mrrry of tlre inclLrded reviews arefrom thc present author:s team, ahd tbis c<luld havc inrnr-duced bias. Finally the validity of conducrlng a sysrernaticreview of systetuatic rcviews has its limitations; mostimporcantly it does not create any ilrforrnation that wasnot available before.

The clinical cvidence suormarized abovc is not dissir-r'l-

ilar 1ion.r thc prcclirical data.Vickers rccently conducLeda $ysternadc review of preclinical invcsrigations of home,opachy l2l I. Er,en rhough 120 papers could be ilciudedin the evaluation. tbis aurhor found that lack of indepcn-dent rcplications, sc'rious methodological flaws, and con,tradicrory results precluded any firrn conclusion. Thissysternatic revierv therefore casts considerirble doubt ononc' of thc main assumprior-rs of honeoparhy, namelv thathoureopatiric rernedics tetain biological activity evenu,hcn dilurecl bc.vond Avogadro'.s ninber (sec abovc).

Pcrhaps tire nlost ri'cent trial cviclcncc, rror vetincludecl in systcnlatic reviervs, helps clarify rhe questionrvherher horreopathic remedies are more dran placebos-

Since the publication of the systematic reviervs. bothpositive, e.g. 122 241. zs well as negative cLinical crialsc.g. [25-27] have eurerged. It seems drerefore unlikelvtlut tllesc lew findings lr'ould substantially change rhe

results of an1, of the systernatic rsviews rvcrc they ro bcup-dated.

The rccent observation of sqhrtc clusrcls in highlydiluted water has been inrerprcted by several homeopathsas iDcreasing the plausibiliry oL bomeopathy [28]. Thisnovel finding rcqrires indcpendent replication. Furthcr-nrore, this obscrl,atiolr (if confirmed) d(res not lend itselfro explainirig hovu, solute clusters colrld havr' anv efictsou hun):lr healtb. TlrtLs both the clinical cvidc-nce rndtlre basic research rrnclcrpinnirg homeofathv rcntlrnunconviocirrg-

lfone accepts this conclusion, one mrghr ask what itsinlplicitions for future research may be. Tu,o opposiugvieu,s edst. One holds that the dcfinitive trial of hone ,opathy should be conductecl to once and for all settle tht:cluesrion [29].'l lle other slales thal 'ne\,v trials . . . are nolonger a research priorify' and advocatcs'outcomr' srudiesto ev:llunte the individual treatment decisions . . . and

@ 2002 Blackwell Science Ltd B( J Clin Pharmo.ol, 54,577-Sa2

A systenatic rcview of systenoti. rcviews of homeopdthy

':=

I

II

t)

?

5

lt

+

e

I

a?ti a,9c>.;!

|2l:

!;i : 3- ti iz : =i i'iEEi i=22 iE i! x 7'a! iiE.E:;: iii|z !i- Zt ; ?:; i<;;;;€ tiFld E!* ;"vi i iii Liec g r i S;: E t 3= q l-: = = |:i€e= iiE:= :;: =s:

s? iz, a''ciz.? 5'FZ:i tiZo ::Z !: ,:E ;ZElfii '^"?Z<i ti=t 6.-. E: !=' i-*{ti22==i?2r:et2 i*z*;E "&1,i E:;E;:Zi ;si=4 ::ie :.i <' y, i" ?7i rid:. y 2 j.t i Ea ?E: *_ - _l=nA = Z F J

;i ,j :1,, ,l;.ttj=:ii:::i;: il . 7:? i ,1i: i; i;: i ;iB*€; lzi ;ia niiti;='.itzzi'tiia3i ; '€ E4i :iili:E ii1;; j ?i1i

9,8-

'",:,"-3

., 1 2t ai : "n a==- 3,i: 4 . '

-, !!:i,:E E i !:1.+ .I c ='\).- a ii 3aj a !4.=':?i-!. ai.t = =a. e t\ i.i f ! = 2_i 3l= i : . a

=L;a : =:. !. - F V

i."7..

! i o= !E

1 = a a a ",? = tL-z a

-- I .- | . ^ E

t =. i a ,,oc

g

579@ 2002 Bla(kwell Science Ltd * I An PhdtnacoL 5 4, 377 -587

ii

!!c

E

t::

.:':*

I!g

-:'

a

5

F

!F

E

IP

c

7

I

tl

ts

Elt

d.ie

:s

lt

c

1.6

i :i: r,ii:t :.i:,-= iria ii " iii l?zi 4i; i i=iil ti i:=7 ii ;+'t a=2 .ii! EiE i 'Li:t: az;;€r .;; 7i: izz tZaE :ii i -"'|=iE iz liZi ii 2

E F :zz clF:-iez jr 4.zziz*3 F{:E'i=re Ee7y= ..E= t22".>i4a EE Li4E2-E-.. Et:rE?Zi?ii.. 3;g ;: g=i; E ! s: ,r7=-t=i!1 ;ziZ2illir;F a F F z E e i: ZZ Z

,i9i

; r€ts - * -{

't G - ?!. 2 ! J

ilil^ 't:.i2 n.e:;d:- t;4x !I_;e.:.ii :i*z h3

o::: b

= l!^-{ ; i

-t;:; L .' H.:r a aZ a -

T;:Ei i P. i; r'l^: P ; i!--: ?- - o ; t! -.r.s t1 l! E + Ias = t,4. ., aa tc z c z

i".ii- ,;r ,.rr+ ,=''-lZ J =i Za; i!{?.+j:;?; le at" 4.,TEttEr=i !iE== i?55i :iif :,E:-i?3j'r* -a;i=- 2i;ei:l2z:::ii ;Sf9;+ ii.. z 5 u = 4 !. ),

= -! , . -Jl-a!iL iL

?ii1==,i7i.a:.i I': ;l : ! - iz+Jr-iEi;114 !_ 1E;iiE

=

jrF-trzz

49i:= Y- it ";s;i =f 2? ;E,7'.i,; -!." 4a =^8. : 4::= vi in ?."n 2" v.:! ' t^::. o= L==:l:: :i i= i*l i 'j

9 :-c i.! i= -; P = a ? -.;' ? z:4: "= Tc '-t4 i Ta- 2 :: :: =, r , 2i=

^ 2-n '2 7 !-!+ 2 ',!

- :: c5 E; !:; E r>zzzzz

@ 2002 Bla.kwell Science Lvl k I Ain F'lrcrnacol, 54.577-582

l-

:d

!c

t:! i::

*

!1

d

€

E

580

compare outconles to orthodox trcetntelrt' f3l]1. Sr"rch

outcome studies exist. They are btrdcned rvitb a :rvriadof methodological w.eaknt'sses, rnost importanrly a pronc,ness to selcction bias, ancl r:suallv rcporr findings r,vhichare convincinglv in fivour of the horneoparhic apprr>ach

[31j.Thrs could inrplv rh.rt rhc iudiviclua]izc'd. cnrFarhticand tinre-intensir,c rpproach nrosr hortreepat]rs rclopr tohealthcare viclds eood clinicel resr.rlrs.This onrphasizt's rheimportance ofthe therapcuric elrcouDtcr and is in accor-dance with a wc'aith of inlbrrnation in this area f321. lrdoes not, howevcr, ansrver the'placebo qLrestion'. I insisc

that this quesLioo cloes re'cluirc an answcr - fbr thc' sake

of scielltific honestv arrtl possibly in the namc ofcliuicrlProgress.

In conclusion. the hyporhcsis thlt any givcn homccrpathic rc'Drecly lcads co clinicll t'llects th:rt ale rclcvrrltivclil-ierent fronr placcbo or superior to ofher ccurtrol ir)ter-ventiom for any Dledicil conilitioD, is not suPported byevidence lion systematic rcvicws. Until nrore cornpcllinElresults arc available. honrcopltlly c:lrr)ot be vie,.vcel as an

evidcncc basecl lourr oi rhcral.y.

(irrllrrt of irrtllrrt: Thc rLrtlror ir .r trainccl horrrc.opirth: he hrs uoIlnitnci intcrcrts in this.rrca.

References

I Sw:ryn. J. I turrtia , D,.liutaty ol llowopdthy Erlnbureh:(lhurchll Llvi Stort, ?ll(r0

2 Ernst E. Hor!reop!L,'. Fi\t prcscr:t hrtLrrc. Ilr I Clin Phdrndan

1991.44 135-43i3 Linde K, Clrusrus N, R.ur)irez (]. fr d/ Ar. thc cli lc:rl clla'cts

of homoeopathy placebo ell'ects7 A urcie ilnel_\,sis of p[crt]oconlrolled lrialr. Inrcr 1')97;350: 8-14 843.

4 Ernst E. Arc lighly dihtc horlocopathic remedies phccbos?PerJ :ion 1<)98; 11 2L)1-292.

5 L;rdc K. Mclclnr! I) lllrdornrzcd controllcd trals ofindivrdLralired honrcoprtlrv: .r \!arr of:thc irt rc!ic\v../,4,0\nt)le tttlt r\tud 1r)98:4: -l:is.

(, Lrnde K. Scholz M, l\rnlrez (1, (lhLrsrLrs N, Mclch;rlt I).Jona\ WB. lr4);rct oi stud1, qrrrlitl' on outcomc Ln placeboconuollcd trirls ol homoeoprthv.J Clut Epn!etnd l')99;52:631-63(r.

7 Morrison B. Lilfi)rd I\J, Ir.rlst E. Mcrhodoloeic.rl rigour rndrc-sults of clirucrl lri;rls of h(nrocol):rthic r(xllcdics. Pl'4;/r,r,20001 13: 132 liS.

3 Ernst E, ['littlrr M]i i\c':urJysis of prcvior.rs rncLr rn.Ll]:ris ofclinrcal tli.rh ofhomcop.rrhy J C/rr i+i,/,'rnn l 2{)tl{l; 53: 1188(lerer).

9 Sterne J, Egger M. Snrrrh (lD. Invcstrq;rring ancl de:rling withpublication end otlrcr brirscs. In .S),-vorlrtir' Rd,t.,r,r i,'Heahhcare: Ment attulj\ii i Ci tt:xt, c'ds Eggcr M, Snith GD,Altmu D(i, pp. 189-20u. l-ordon: ISMJ Publishing ciroup,2001.

1() llitrnc\ J. Rcsrjr KL. E|rrsr [. I Ionrcopirthv tb. Po\topcr;uivr:Jlcus.J Clrr Cnnnc ri/Jl 19r)7; 25: 62ti {r3-1.

A syslemoti. review ofsystematjc .ewews of homeopathy

Erlst E, l3arncs.f Arc homoeoprrhic rcrrrcc|cs r{lcctile lirrJrLy.'l-orr- nr,rsl. sorcn..- A \\:,.,, ,r, (v,('. otphccbo-corrnallcd trials. P./ftq.,,r 1993t 1 1 : :1*8.

Ernsr l, Pitdcr MrI. [f]icac,v ol-honlcoparhic arnicr. Asystematic ,clirjN of plact-bo corrtlollcd clinical trials ,4rd.S,iU 1!)')ili 133: llll7 1190.Lntst E. HoruocoprrhJc plophvlrls ol Icrd.r,-lrt's rLrd

nigrainci A lvst!'nuti. rcvrli\\'. I l' h Sy,l1pt i\,Idnaf(n 1t).)t):

18: -353 357Ernst E. Classieal homoeoir.rthv vt'rsus conventror:rltrertnrents: a sy\tenmti. rcvr.w Pcrli:ion 1999; 12:13 15.Liidtke I\, Wilkcns J. I{liriscire Wirksrr[bcitsstudien zuArnica i:r hornocopathischcn Zubercurrngcn. In (-ir,5lr,?r

StiJiunq Oanpanl' Rq,orr. Escn. Gernuny: Cirrstcns Srifrurg,i999.Cuchcrrt M. ll.rugh Nt(]. (;oorh M. tloissel l 1, Eridenceof ciinical eflic:rc1 oi:lmnrcop:rthy A rrct.r-,rralyrLs oi clLnrcal

irirh. It,r / OIn Phannatol 2o01)i 56: 27 :l.iVickers AJ. Snrith Cl. lJomcoprthic oscrlloco..rlrLlrn tirprevcnting :rnd rreature inlluc-nza ud inllutnz;r Lkcsyndrorrres. Corhntnt' l-ibnt11 2001; 1: 1 10.

Linde I{.Jobst KA. Hon!'oprtlw fi)r chronir asthma. Clodrraltl-tlr#l l99ti: l: I 7.

-kurs Wll, Linde I{, Rrnrirtz (1. l-lornrop:rthv rnd rhcrurr:rticdrscre. ,lllrcrar .1)ii a,lnr \i,(lr .1n 2t)ltt):26: 111 123Long L, err\i Ii. Honrroprthrc rcrDedrrs lbr tht rrL,,liurcnr ofosr.orrthritrs: .r \ysld1$tL. rcvrr:rv Ilr Harrrr-opatlr i l1)0li 90:37 43.Vickers AJ. hrdcpeldent rcpllc:rtion ofpre clinic;11 rescltrch inhomeopathv:,r systen.tii. rer,it'rv. I:orrb Konplc'ktltannd1999: 6:311-12().Tirlbt MA. Rcil[l l). L.1t wellyn-Joncs ILII, McSharrv C,Ai!cl)i\on TC. l{audou)rscd contK)ll.d trial ol homcopath,v

versus phc'bo rn percurri.rl rllergic rhinitrs lith orcrlrov offtiur rfid serres b lltd.l )t)|r):721: It'1 416.

Jacobs -i, Springcr 1)A, CrorL!'r ll Homcoprrhi,: rlcatmentof.rcr.rte otitjs uredu rn childrcrr: .r prc'lLnrrrrirry randonizeciplacebo conlrolled tnll. Pt:JiM lifut Dis J 2li\)1.20 117-183.()bc'rbaurn M, Vrrrii l, tlerr Gal Y, cr a/. A nndonriscd,controlled clinital tli,rl cif thc honrcoprtlric rrcdic,rtionTl\AUMEEI- Si irr thc trc,rtrlrcrrt oi circr othcrrpl-indLrcectstoDl:rn!is ir) .hiLlrel Lrudcrgoing stcnr li:ll trLnsl,l.rnr.rnon.( )|L@ 2l)t )1. 92: 68.1-6'l(lWahch l{,l.orves T, Musl';i(h l), r,1 The long rcrnr c*lc$of homcop,rrhi. trertnrcrr ol chronrc herd.rchc,i: one ye;rl

fo]lor-r-uf and snglc c.rse tirrc sclies analvsir. I3r'IlonreopatlJ2()01:90:63-12.Flicsc K-i-I, Fcuchrer U. l,iid*c ll, Morllcr Il. ll-csLrlts of .r

rrrrdoruscrl prcspci'ti!r doublt lnind clinrcal trr:rl or thchonrcop:rthic trernrcnr of .rdcnoicl vrqctanorrs. Iff -,1

(;ca

.P,11,r 2al00i 7: 48' 54.

Lewith (il-, V,'rtknN AIl. llvl.rrrds ML, rt,r/. Usc olultrarnoleculrr potcrri:ies ofaliergen to uret rsthruh. propld;rll(rqic to housc dust rrritcr do hie blilrd r.rndonrrscd

co[trollcd dirucr] trirl. B/ l,/r(/ / 2002t 32: 520 523.\.rr.r.rl S. (;rrlcl, r l{1.I rr. r1'cir"d'ol.r. i& n.$", ^n rrrvater on riiluriorr. Chtn OantmLr ?0t)1; l)ct:2221-2223.l'anccstcr T, Vickers A. I-:rger nials r.cdcd. 8r.\4cdJ 2ir00;321: 4'16

lt

23

24

t5

21

11

12

13

1:l

l5

16

Iri

@ 2002 Blackweil Science Ltd Bt I Clin Phomacot, 54.577 5A7

2tt

58r

E. Ernsl

3tl Feder G, Katz T. I{xndotriscd co troiled triils for 32 Di Illasi Z. Halkncss F.. Ernst Ii, Ccorgiorr A, Klerjncn J.horlcopirthy. Ilr Med J 2Ot)2; 324 4t)8-41)t). InilLreni:c ofcorrrert effccrs on hcalth ourconrcs: r \trr.nrrrir

31 f\iley D, Fisher M, Singh 13, l'lridvogl M, I{cgi.r M. rcview. l*rcr 20011357:731-762.llorlcopathy i:nd conveutional nrfdicinc: .rn outcornes strrdvconrparing effectiveness in r prxr.rri,carc sertnlit. -/.4llCotry emenrar A4cd 2tJiI;1:149 159.

582 O 2002 Blackwell Science Ltd B.l Ain Phamacal, 54.577-SA1