safety surveillance of longitudinal databases: methodological considerations

pharmacoepidemiology and drug safety 2011; 20: 714–717Published online 2 June 2011 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/pds.2151

COMMENTARY

Safety surveillance of longitudinal databases: methodologicalconsiderations

G. Niklas Norén1,3*, Johan Hopstadius1, Andrew Bate2,4,5 and I. Ralph Edwards1

1Uppsala Monitoring Centre, WHO Collaborating Centre for International Drug Monitoring, Uppsala, Sweden2Pfizer Inc., New York, NY, USA3Department of Mathematics, Stockholm University, Stockholm, Sweden4School of Information Systems, Computing & Mathematics, Brunel University, London, UK5Dept of Clinical Pharmacology, New York University, New York, USA

Received 18 January 2011; Revised 10 March 2011; Accepted 23 March 2011

We read with interest the paper by Schuemie1 describ-ing an implementation of Bayesian disproportion-ality analysis (Gamma Poisson Shrinker (GPS)) forlongitudinal data and emphasizing the importanceof graphical representation. In this commentary, wedescribe a previously published related approachto safety signal detection in longitudinal medicalrecords,2,3 which we argue has important advantages.We describe some details of the method that can serveas an extension of the approach described by Schuemie.We believe this makes the case for graphical repre-sentation coupled with Bayesian analysis in safetysurveillance of longitudinal data more convincingly.As with Schuemie’s method, the method for temporal

pattern discovery in Norén et al.2,3 uses Bayesianshrinkage to protect against spurious associations,contrasts event rates in different periods to filter outindications for treatment, and proposes a graphicalstatistical approach to characterize temporal patternsand facilitate clinical interpretation. In addition, itcontrols for time‐constant confounders through a self‐controlled design while incorporating information onunexposed patients separately to account for systematicvariability in event rates over time. It contrasts fourdistinct periods relative to treatment initiation tohighlight time‐varying confounding by underlyingdisease. Its computational framework has been evalu-ated in the UK IMS Disease Analyzer data set of more

* Correspondence to: G. Niklas Norén, Uppsala Monitoring Centre, Box 1051,751 40 Uppsala, Sweden.E‐mail: niklas.noren@who‐umc.org

Copyright © 2011 John Wiley & Sons, Ltd.

than two million electronic patient records, for whichthe empirical results include the timely identificationof an association between terbinafine and angioedemain longitudinal data.2,3 Schuemie’s longitudinal GPS(LGPS) method on the other hand computes expectednumbers of medical events during drug prescriptionbased on an aggregate of unexposed patient time inever‐exposed and unexposed patients, where the latterwill tend to dominate so long as the majority of patientsin a population are not treated with a specific medicine.Because unexposed time in patients who never havetaken, and perhaps never will take, the medicine ofinterest is less likely to contain events that are related tothe indication for treatment and underlying disease,LGPS does not effectively protect against such con-founding. Instead, Schuemie proposes the LongitudinalEvaluation of Observational Profiles of Adverse eventsRelated to Drugs (LEOPARD) filter, which eliminatesmedical events with higher prescription rates of the drugof interest in the month after than in the month beforethe medical event. To eliminate events associated withthe indication for treatment by reversing the perspectivefrom the drug to the medical event is innovative andinteresting. However, a comparison of absolute pre-scription rates may be subject to bias (for example,prescription rates for chronic treatment might increaseover time regardless of the medical event). The“reversed LGPS” approach on the other hand employsthe principle previously described in Norén et al.2,3 ofcontrasting observed‐to‐expected ratios in distinctperiods relative to treatment initiation, even though thedetails of the implementation are different.

safety surveillance of longitudinal observational databases 715

The visualization of temporal associations is funda-mental to effective screening of patient records. It canhighlight unexpected temporal variability or con-founding that render summary statistics misleadingand can help identify subtle temporal variabilitythat would otherwise go undetected. The graphicalrepresentation of temporal patterns in Norén et al.2,3

differs from that advocated by Schuemie in severalrespects. We have displayed event rates rather thantheir cumulative function, which we consider tofacilitate interpretability. More importantly, we havenot relied on event rates in isolation as they are proneto censoring and recording biases. Instead, we havecontrasted them to expected rates, in order to reducedistortion from, for example, age gradients in therates of certain medical diagnoses, loss to follow‐up,and tendencies of doctor visits to cluster together intime. Although we have primarily visualized rates ofmedical diagnoses relative to first prescriptions ratherthan rates of prescriptions relative to first medicaldiagnoses, the two perspectives are ideally combinedfor a more complete view of the temporal association ofinterest. See, for example, the analysis of nifedipineand diltiazem in Norén et al.2,3

As an illustration, consider the temporal associationbetween nifedipine and flushing, in the UK IMSDisease Analyzer collection of patient records.2,3

Figure 1 displays a graphical representation referred toas a chronograph for the registration of flushing relativeto first prescriptions (in 13months) of nifedipine.

Figure 1. Chronograph for flushing diagnoses relative to first nifedipine prescripinitiation is clear. The top panel displays the logarithm of a shrunk observed‐to‐eand expected (line) numbers of patients with a flushing diagnosis in different pecorresponds to the day of treatment initiation, and the other points on the x‐axis a1–30 days prior to a first prescription of nifedipine). Reproduced from Norén et


Although both the observed and expected countsincrease near the first prescription, the observed‐to‐expected ratio remains close to one for most periods,indicating that patients prescribed nifedipine are aslikely as the population in general to suffer fromflushing. However, a transient increase in the registra-tion of flushing in the first 2 or 3months after treatmentinitiation is clear and reflects a known pharmacologicaleffect. Figure 2 displays an event‐centered chronographfor the same pattern: the number of patients with anifedipine prescription in different periods relative tofirst diagnoses of flushing. Overall, patients who sufferflushing are less likely than patients in general to beprescribed nifedipine. However, a transient increase inthe rate of nifedipine prescriptions 2months prior to theflushing diagnoses mirrors the pattern in Figure 1. Thedrop to below baseline again suggests that patients dostop nifedipine treatment after flushing. The lower rateof nifedipine prescriptions long after flushing mayreflect that this treatment is avoided in patients with ahistory of flushing, whereas the lower rate of nifedipineprescriptions long before flushing may reflect depletionof susceptible patients. Figures 1 and 2 clearly docontribute complementary perspectives on the associ-ation of interest. The event‐centered visualization inFigure 2 lends itself more naturally to interpretation ofthe effects of medical events on prescriptions than theother way around, as is also illustrated by Schuemie’sexamples for upper gastrointestinal bleeding, whichappears to induce pantoprazole and terminate naproxen

tions. A transient increase in diagnoses of flushing subsequent to nifedipinexpected ratio. The bottom panel displays the corresponding observed (bars)riods relative to the first prescription of nifedipine. The zero on the x‐axisre months relative to treatment initiation (for example, “−1” corresponds toal.3 with permission from Springer.

Pharmacoepidemiology and Drug Safety, 2011; 20: 714–717DOI: 10.1002/pds

Figure 2. Chronograph for nifedipine prescriptions relative to first flushing diagnoses. A transient increase in prescriptions of nifedipine in the 2monthsprior to first flushing diagnoses is clear. In all other periods patients who experience flushing are less likely to have nifedipine prescriptions.

g. n. norén ET AL.716

prescriptions.1 As a direct comparison, Figure 3 pre-sents a visualization of the temporal associationbetween flushing and nifedipine in a graph such as thatpresented by Schuemie. The steeper slope in thecumulative number of prescriptions before than afterflushing suggests that patients stop nifedipine afterflushing, in line with the interpretation of the periodsimmediately close to zero in Figure 2. The use ofcumulative rates can facilitate visualization for rareevents but makes interpretation more challenging asillustrated by a direct comparison of Figures 2 and 3.Schuemie provides some references to earlier work

on spontaneous reports, but there exist severalpublications on signal detection in electronic patientrecords in addition to those cited in the precedingparagraphs, including the adaptation of a proportionalreporting ratio for adverse drug reaction (ADR)surveillance in administrative health databases,4 animplementation of the GPS algorithm on medical

Figure 3. Cumulative graph of nifedipine prescriptions relative to first flushing dcumulative number of nifedipine prescriptions can be observed.


claims data,5 and an information‐theoretic approach todetect medically similar cases in longitudinal data.6

Specific studies have covered the evaluation oftemporal data mining for the MMR vaccine in Danishhealthcare databases,7 a characterization of the tempo-ral association between antipsychotics treatment andpneumonia,8 and ADR surveillance for cardiovasculardevices.9 The promising performance of Schuemie’sadapted GPS methodology on simulated data inwinning the “OMOP cup” is encouraging. As high-lighted by the author, the evaluation in real data is animportant next step, in particular because it will allowthe comparison against methods for screening obser-vational data that were not entered into the OMOP cupevaluation.The development of methods for signal detection and

refinement in patient records and medical claimsremains an important challenge. Methods for explor-atory analysis must be carefully devised in order not to

iagnoses. A steeper slope before than after first diagnoses of flushing in the


safety surveillance of longitudinal observational databases 717

yield large numbers of false positives. At the sametime, false negatives are perhaps an underestimatedchallenge with methods for broad surveillance thatcannot account for the intricacies of each specific issueof potential interest. Research in this area is in itsinfancy, and we believe that there is potential forsignificant innovation as the field matures.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

REFERENCES

1. Schuemie MJ. Methods for drug safety signal detection in longitudinalobservational databases: LGPS and LEOPARD. Pharmacoepidemiol Drug Saf2011; 20: 292–299.


2. Norén GN, Bate A, Hopstadius J, Star K, Edwards IR. Temporal pattern discoveryfor trends and transient effects: its application to patient records. In Proceeding of the14th ACM SIGKDD international conference on Knowledge discovery and datamining, 24–27 August 2008, Las Vegas, NV. ACM: Rochester, IL, 2008; 963–71.

3. Norén GN, Hopstadius J, Bate A, Star K, Edwards IR. Temporal pattern discovery inlongitudinal electronic patient records.DataMin KnowlDiscov2010; 20(3): 361–387.

4. Jin HW, Chen J, He H, Williams GJ, Kelman C, Keefe CM. Mining unexpectedtemporal associations: applications in detecting adverse drug reactions. IEEETrans Inf Technol Biomed 2008; 12(4): 488–500.

5. Curtis JR, Cheng H, Delzell E, Fram D, Kilgore M, Saag K, et al. Adaptation ofBayesian data mining algorithms to longitudinal claims data: coxib safety as anexample. Med Care 2008; 46(9): 969–975.

6. Walker AM. Signal detection for vaccine side effects that have not been specifiedin advance. Pharmacoepidemiol Drug Saf 2010; 19(3): 311–317.

7. Svanström H, Callréus T, Hviid A. Temporal data mining for adverse eventsfollowing immunization in Nationwide danish healthcare databases. Drug Saf2010; 33(11): 1015–1025.

8. Star K, Bate A, Meyboom RHB, Edwards IR. Pneumonia following antipsychoticprescriptions in electronic health records: a patient safety concern? Br J Gen Pract2010; 60(579): 385–394.

9. Resnic FS, Gross TP, Marinac‐Dabic D, Loyo‐Berrios N, Donnelly S, NormandS‐LT, et al. Automated surveillance to detect postprocedure safety signals ofapproved cardiovascular devices. JAMA 2010; 304(18): 2019–2027.


safety surveillance of longitudinal databases: methodological considerations

Documents