Epilepsia, 44(Suppl. 7):51–54, 2003Blackwell Publishing, Inc.C© International League Against Epilepsy

What We Don’t Learn from Clinical Trials in Epilepsy

Frank Gilliam

Department of Neurology, Washington University, St. Louis, Missouri, U.S.A.

Summary: The randomized, double-blind trial design offers themost accurate data regarding the efficacy of antiepileptic treat-ments. However, translating the results of a trial into clinicalcare can be complex due to the intrinsic tension between the re-quirements for scientific methods that minimize systematic andrandom error, and the need for clinically relevant and gener-alizable data. The interpretation of the trial results is compli-

cated further by the probable inaccuracy of self-reported seizurerates and spontaneously reported adverse effects in most trials.Patient preference may be a feasible outcome measure that al-lows patient-oriented validation of the results, and also inherentlyweighs the positive and negative effects of a treatment in a singleendpoint. Key Words: Epilepsy—Seizures—Clinical trials—Antiepileptic drugs—Trial design—Ethics—Treatments.

The randomized, double-blind trial is the most ac-curate mechanism to determine the efficacy of a treat-ment for an illness. Randomization limits the potentialfor selection bias, and blinding (masking) minimizes in-formation and investigator bias. Sufficient sample sizeto ensure adequate power, and accurate instruments tosystematically assess outcome endpoints, determine theprecision of the trial. Although randomization, blinding,and precision are the fundamental components of accept-able study design, careful consideration of all aspectsof data collection, analysis, and presentation is requiredto determine the quality of the results and the validityof the investigators’ conclusions. Of equal importanceare the ethical considerations involved in possibly ran-domizing patients to suboptimal treatments in superi-ority trials and the potential for erroneous conclusionsfor efficacy in equivalence trials. Thorough understand-ing of all aspects of clinical trials and methods to col-lect and analyze data facilitates appropriate interpretationand successful translation of results into improved patientcare.

This article does not attempt to review the entirety ofimportant considerations for the design and interpretationof clinical trials in epilepsy, but rather emphasizes severalessential issues that require further exploration to refinesubsequent trials and also improve the process of conver-sion of results into clinically useful information.

Address correspondence and reprint requests to Dr. F. Gilliam atWashington University Epilepsy Program, Campus Box 8111, 660South Euclid Avenue, St. Louis, MO 63110, U.S.A. E-mail: gilliamf@neuro.wustl.edu

THE INHERENT “TENSION”IN CLINICAL TRIALS

The primary purpose of an efficacy trial is to determinethe effectiveness of a treatment in the clinical care of pa-tients. If the intervention under study did not have thepotential to improve health outcomes for at least a subsetof patients, there would be no function for the trial. One ofthe explicit goals of a trial is to provide as much clinicallyrelevant data as possible, such as efficacy, dosing, and ad-verse effects, to allow improvement in the medical careof persons with the condition under study. For optimalepilepsy management, medical providers desire a treat-ment that is effective for a wide variety of seizure types, iswell tolerated by sensitive patients such as seniors, and issafe for children and women of childbearing age. Patientsand their caregivers generally want to become completelyfree of seizures so that they can drive a car and live withoutsocial and vocational disability (1).

In contrast to the goals of patients and providers, clini-cal scientists desire a protocol that provides reproducibledata based on appropriate validity and precision (i.e., min-imal systematic and random error), as presented in Fig. 1(2). This requires a clearly defined sample based on spe-cific enrollment criteria, as well as adequate sample size,to ensure that a relevant difference between treatmentscan be identified if one exists. For patients with pharma-coresistent epilepsy, the proportion that will be renderedseizure free is small, so the required sample size will bevery large. This would increase the cost and duration of en-rollment to a prohibitive degree for most new treatments.Alternative endpoints, such as a 50% reduction in seizurefrequency, are often used to reduce the required sample

51

52 F. GILLIAM

Scientific Method Validity

(- systematic error) Precision

(- random error)

Clinical Practice Relevance

Generalizability

FIG. 1. Competing concerns of scientific methodology and clin-ical relevance create inherent tension in clinical trials in epilepsy.

size, and the sample is usually defined by specific inclu-sion criteria developed in consideration of subject avail-ability. A majority of efficacy trials in epilepsy have subse-quently enrolled predominantly young adults with partialseizures. The result is scientifically valid, reproducibledata that can support a Food and Drug Administration in-dication but do not answer many important questions re-garding optimal use of the drug in many common clinicalsettings.

This inherent difference between the scientific man-dates of clinical trial design and the generalizability andclinical relevance of results creates tension in the data thatis unavoidable. As Dr. Alexandre Jadad recently summa-rized, “you should consider the information provided by asingle randomized controlled trial as an important piece ina puzzle with many empty spaces. This information willhave to be assessed and used in conjunction with othertypes of information (for example, data from other ran-domized controlled trials or from other study designs, andyour clinical experience), and the values and preferencesof the people involved in the decisions” (2).

COMPLEXITY OF DATA ANALYSISAND INTERPRETATION

The translation of data from a clinical trial into clini-cally useful information can be a complex task, as sum-marized in Fig. 2. As an example of the difficulty in-volved in this important process, we will review the firstVeteran’s Administration Cooperative trial (VA-118) thatcompared carbamazepine (CBZ), phenobarbital, pheny-toin (PHT), and primidone in patients with recently diag-nosed epilepsy (3). The primary outcome measure wasa composite score that integrated seizure control andantiepileptic drug (AED) side effects, which allowed ef-

Trial Results

Seizure Control Tolerability

Safety

Optimal Care and Patient Preference?

Clinician’s Interpretation

FIG. 2. Model of the traditional application of trial results to clini-cal care that is based on potentially incorrect self-reported seizurerates and spontaneously reported side effects in most studies.

TABLE 1. Mean composite scores at interval assessments inthe Veterans Administration (VA-118) comparative trial (3)

12 Month 24 Month 36 Month

Carbamazepine 22 27 31Phenobarbital 26 31 33Phenytoin 21 24 29Primidone 31 35 36

0–20, good; 21–34, satisfactory; 35–49, poor.

fectiveness to be the principal outcome variable. How-ever, specific data regarding seizure control and adverseAED effects were also evaluated during the trial and pre-sented in the summary manuscript. The proportion of pa-tients who were “successfully treated” on each drug is pre-sented in Fig. 3. These survival curves have been widelyinterpreted by the neurology community to indicate thatmost patients have acceptable outcomes on either CBZor PHT. However, the conclusion in the discussion of thepublished summary was that “The outcome of this projectunderscores the unsatisfactory status of antiepileptic ther-apy with the medications currently available. Most pa-tients whose epilepsy is reasonably controlled must tol-erate some side effects. These observations emphasizethe need for new AEDs and other approaches to treat-ment”(3). Despite the apparent “successful treatment” inmost patients, the study findings that support the authors’conclusions are apparent when examining the compositescore, medication toxicity, and seizure-free outcome as-sessments. The composite score results are summarizedin Table 1. At no point in the study did the mean compos-ite score for any drug fall within the range designated as“good,” and the final scores were closer to the poor than thegood range. Similarly, medication toxicity contributed towithdrawal from the study in 40–50% of patients on CBZand PHT, the better-tolerated drugs, as shown in Table 2.In a subsequent reanalysis of the original data to evaluatethe probability of complete seizure control, only 35–40%of patients with predominantly partial seizures remainedin remission after achieving optimal monotherapy, as pre-sented in Fig. 4 (4).

Again, the purpose of this review is not to criticizethe study design, analysis, or interpretation of this re-markable study, but rather to emphasize the complexityof considerations involved in the translation of results of

TABLE 2. Causes of antiepileptic drug failure in the VACooperative trial (VA-118)

AllCBZ PB PHT PRM patients

(N = 101) (N = 101) (N = 110) (N = 109) (N = 421)

Toxicity alone 12 19 18 36 85Toxicity plus 30 33 29 35 127

seizuresSeizures alone 3 4 1 3 11Total failures 45 56 48 74 233

Epilepsia, Vol. 44, Suppl. 7, 2003

WHAT WE DON’T LEARN FROM CLINICAL TRIALS 53

FIG. 3. Cumulative percentage of pa-tients successfully treated with carba-mazepine (CBZ), phenytoin (PHT), prim-idone (PRM), and phenobarbital (PB) inthe Veterans Administration Trial (VA-118) (3).

a trial into clinical practice. The overall distribution ofprescriptions for epilepsy in the United States, in whichPHT, CBZ, and the barbiturates remained among the mostfrequently prescribed AEDs in 2001(5) despite eight newAEDs in the market for >5 years, suggests that the medi-cal community did not receive or believe the investigators’interpretation of the results of the VA Cooperative Trials.

INACCURACY OF SELF-REPORTED SEIZURERATES IN CLINICAL TRIALS?

The inaccuracy of self-reported seizure rates reportedin prior studies (6,7) creates a dimension of complexitythat may be unique to epilepsy. Available evidence indi-cates that many patients are not aware of or are amnesticfor a significant proportion of their seizures. Three stud-ies using various methods of continuous EEG monitoringhave suggested that ∼40% of seizures are not identifiedby patients (6–8).

In a study intended to validate computer-assistedseizure detection with continuous EEG monitoring,Gotman found that 69 of 179 (39%) scalp-recordedseizures did not have a patient alarm activation (6). Tatumet al. reported that 18 of 47 (38%) seizures recorded dur-ing ambulatory scalp EEG studies did not have correlative

Treatment A

Treatment B

Treatment A

Treatment B

1 2 3 4

Assessments at time 1–4: patient preference, seizure rate, adverse events profile, mood,brief cognitive battery.

Randomized/Blinded

FIG. 4. Schematic of a crossover patient preference trial design.

patient alarms or notation in the seizure log to indicatethat the patient or witnesses had identified the seizure (8).Although it is possible that some of these seizures wereelectrographic only, without clinical symptoms or signs,this seems unlikely given the extent of cortical regions re-quired to generate enough current to be detected by scalpEEG.

A more detailed study by Blum et al. (7) provided ad-ditional information on the inaccuracy of self-reportedseizure occurrence. They found that 30% of a sample of pa-tients evaluated in their video/EEG monitoring unit wereunaware of all of their recorded seizures. Only 23% wereaware of all of their seizures. Approximately 90% of lefttemporal and 50% of right temporal lobe seizures were de-nied by patients. Of particular concern was the inverse cor-relation between self-reported seizure rates before seizuremonitoring and proportion of correctly identified seizures;the lower the self-reported seizure rate in the outpatientclinic, the greater the inaccuracy of seizure identificationduring video/EEG (7).

The 30–50% underreporting of seizures by patientspresents major difficulty for the interpretation of clini-cal trials. Although seizure-free rates could be overesti-mated in both equivalence and conversion to monother-apy trials, the increased vigilance that patients and fam-ily members may provide after randomization to a newtreatment could potentially increase seizure identificationand decrease the actual difference from the baseline rate.Also, sample size calculations for comparison of meanseizure response are dependent on the standard deviationof the estimated seizure rate, which could be artificiallysmall due to unidentified seizures. Underestimating therequired sample size required to determine a significantdifference between groups will bias the study towards thenull, which would lead to false-negative results in con-version to monotherapy trials and false-positive results inequivalence trials.

Epilepsia, Vol. 44, Suppl. 7, 2003

54 F. GILLIAM

INACCURACY OF SPONTANEOUSLYREPORTED ADVERSE EVENTS

IN CLINICAL TRIALS?

Accurate determination of medication side effects oradverse events has also been a limitation of prior trials ofnewer AEDs. Most studies have monitored spontaneouslyself-reported adverse effects without systematic assess-ments. If some patients may be reluctant to “complain”about side effects to the investigator or fear prematurewithdrawal when they desire additional treatment, the re-ported rates of some adverse effects may be artificiallylow. The VA Cooperative studies (3,9), which used a stan-dardized questionnaire for systematic assessment of ad-verse medication effects, found that medication toxicitycontributed to exit from the trial in 40–60% of subjects,as summarized in Table 2; this is substantially higher thanmost studies using only spontaneous self-reporting. Usinga valid and reliable instrument called the Adverse EventsProfile, Baker et al. (10) found similar rates of impor-tant side effects such as sedation and fatigue with PHTand CBZ in a survey of >5,000 patients from Europeancountries. Gilliam et al. (11) subsequently showed that theAdverse Events Profile scores are strong predictors of aperson’s perception of their overall health with epilepsy,independent of seizure rate. The Commission on Out-come Measurement of the International League AgainstEpilepsy consequently has recommended the inclusion ofreliable and valid instruments in clinical trials to more ac-curately assess subjective adverse effects rates in clinicaltrials of AEDs (12).

IS PATIENT PREFERENCE A FEASIBLEOR DESIRABLE OUTCOME MEASURE

IN CLINICAL TRIALS?

In a recent position paper in Science (13), Clancy andEisenberg stated that “outcomes research—the study ofthe end results of care that takes patients’ experiences,preferences, and values into account—is intended to pro-vide scientific evidence relating to decisions made by allwho participate in health care.” The emphasis on patients’preferences by these administrators of the Agency forHealth Care Policy and Research arises in part from thesubstantial variations in medical practices in the UnitedStates and abroad (14). Lack of uniformity of medicalcare has stimulated growing inquiry into the relation be-tween medical decision-making and the end results ofinterventions.

The difficulty and complexity of translating the po-tentially inaccurate estimates of effects of newer AEDson seizure rates and adverse events may make patientpreference a particularly valuable outcome measurementin epilepsy. Evaluating patient preference of a treatmentcompared to a control would allow comparison to the clin-ician’s interpretation of the relative value of reported effi-

cacy and tolerability data from a clinical trial. As we havefound in prior studies of patient concerns of living withepilepsy (1), patients and clinicians may emphasize dif-ferent aspects of the effects of epilepsy and its treatment.

A trial that uses patient preference for a treatment as aprimary endpoint must be based on several assumptions:(1) patients value the option to compare treatments, (2)the study treatments have equal efficacy and safety, (3)most patients experience some adverse effects, and (4) thetreatments have differential adverse effects. The primaryoutcome analysis would compare the proportion of sub-jects who favor treatment A over treatment B. A schematicof a crossover patient preference study design is presentedin Fig. 4. Such a study would allow the evaluation of clin-icians’ interpretation of efficacy and tolerability resultsfrom a clinical trial compared to the actual patient prefer-ences of the treatments that inherently weigh the relativepositive and negative effects of the treatments. This wouldminimize the guesswork intrinsic to the translation of rela-tively inaccurate seizure and adverse effects outcomes intoclinical decision-making in the management of epilepsy.

Acknowledgment: Supported by National Institutes ofHealth grants NS01794 and NS40808, and a grant from theEpilepsy Foundation of America.

REFERENCES

1. Gilliam F, Kuzniecky R, Faught E, Black L, Carpenter G, SchrodtR. Patient-validated content of epilepsy-specific quality-of-life mea-surement. Epilepsia 1997;38:233–6.

2. Jadad A. Randomized controlled trials. London: BMJ Books, 1998.3. Mattson RH, Cramer JA, Collins JF, et al. Comparison of car-

bamazepine, phenobarbital, phenytoin, and primidone in partialand secondarily generalized tonic-clonic seizures. N Engl J Med1985;313:145–51.

4. Mattson RH, Cramer JA, Collins JF. Prognosis for total control ofcomplex partial and secondarily generalized tonic clonic seizures.Department of Veterans Affairs Epilepsy Cooperative Studies No.118 and No. 264 Group. Neurology 1996;47:68–76.

5. National Ambulatory Medical Care Survey. Centers for DiseaseControl National Center for Health Care Statistics Web Database2000.

6. Gotman J. Automatic seizure detection: improvements and evalua-tion. Electroencephalogr Clin Neurophysiol 1990;76:317–24.

7. Blum DE, Eskola J, Bortz JJ, Fisher RS. Patient awareness ofseizures. Neurology 1996;47:260–4.

8. Tatum WOt, Winters L, Gieron M, et al. Outpatient seizure identifi-cation: results of 502 patients using computer-assisted ambulatoryEEG. J Clin Neurophysiol 2001;18:14–9.

9. Mattson RH, Cramer JA, Collins JF. A comparison of valproatewith carbamazepine for the treatment of complex partial seizuresand secondarily generalized tonic-clonic seizures in adults. The De-partment of Veterans Affairs Epilepsy Cooperative Study No. 264Group. N Engl J Med 1992;327:765–71.

10. Baker GA, Jacoby A, Buck D, Stalgis C, Monnet D. Quality of life ofpeople with epilepsy: a European study. Epilepsia 1997;38:353–62.

11. Gilliam F. Optimizing health outcomes in active epilepsy. Neurology2002;58(suppl 5):S9–19.

12. Baker GA, Camfield C, Camfield P, et al. Commission on out-come measurement in epilepsy, 1994–1997: final report. Epilepsia1998;39:213–31.

13. Clancy CM, Eisenberg JM. Outcomes research: measuring the endresults of health care. Science 1998;282:245–6.

14. Wennberg J, Gittelsohn. Small area variations in health care delivery.Science 1973;182:1102–8.

Epilepsia, Vol. 44, Suppl. 7, 2003