8/4/2019 ADESSA AMIA Presentation Jon Duke

1/29

ADESSA:AReal-TimeDecisionSupport

ServiceforDeliveryofSemancally

CodedAdverseDrugEventData

JonD.Duke,MD

AMIAAnnualSymposium2010

8/4/2019 ADESSA AMIA Presentation Jon Duke

2/29

Overview

Introduc@ontoADESSA ConceptandDesign Evalua@onStudy ResultsDiscussion/FutureWork

8/4/2019 ADESSA AMIA Presentation Jon Duke

3/29

ADESSAisawebservicedesignedto

providelabeledadversedrugeventdata

incoded,structuredform

8/4/2019 ADESSA AMIA Presentation Jon Duke

4/29

OriginsofADESSA

8/4/2019 ADESSA AMIA Presentation Jon Duke

5/29

TheProblem

8/4/2019 ADESSA AMIA Presentation Jon Duke

6/29

Whataboutthe

StructuredProductLabel?

CentralandPeripheralNervousSystemDisorders

Frequent: hypertonia,hypoesthesia;Infrequent:twitching,confusion,hyperkinesia,vergo,ataxia,

migraine,abnormalcoordinaon,hyperesthesia,legcramps,abnormalgait,nystagmus,

hypokinesia;Rare:dysphonia,coma,dyskinesia,

hypotonia,ptosis,choreoathetosis,hyporeflexia.

8/4/2019 ADESSA AMIA Presentation Jon Duke

7/29

OpenSourcesofCoded

AdverseEventData NDFRT

Physiologiceffects SIDER

CO-STARTTerminologyLimiteddrugsetInfrequentlyupdated

AdverseEventRepor@ngSystemSpontaneousreports,notlabeleddata

8/4/2019 ADESSA AMIA Presentation Jon Duke

8/29

Op@malSolu@on

ComprehensivesourceoflabledADEdata MedDRAcoded

usefulforpharmacovigilance/ADErepor@ngnaturalphrasingsofadverseevents,human-readable

SNOMED-CTcodedusefulforintegra@onwithclinicaldecisionsupport

Webservicecapableofretrievingdataforoneormoredrugsusingstandardmedica@oniden@fiers

8/4/2019 ADESSA AMIA Presentation Jon Duke

9/29

SystemComponents

StructuredProductLabelInforma@onCoderandExtractoR(SPLICER)

Quan@ta@veAdverseReac@onKnowledgebase(QuARK) AdverseDrugEventSupportService(ADESSA)

8/4/2019 ADESSA AMIA Presentation Jon Duke

10/29

GeneralOverviewofSPLICERArchitecture

Processors

List

MedDRA term identifier

Table

Section Extraction

XML tag removal

Text Normalization

Sentence Detection

Processors

ExactMatch

Synonym

StopStemTransform

Similarity

Tokenizer

LaboratoryMapper

SPL

~~~~~~~~~~

~~~~~~~~~~

~~~~~~~~

SPL Parser

ADE

ExtractorMedDRA

Mapping

8/4/2019 ADESSA AMIA Presentation Jon Duke

11/29

GeneralOverviewofSPLICERArchitecture

Processors

List

MedDRA term identifier

Table

Section Extraction

XML tag removal

Text Normalization

Sentence Detection

Processors

ExactMatch

Synonym

StopStemTransform

Similarity

Tokenizer

LaboratoryMapper

SPL~~~~~~~~~~

~~~~~~~~~~

~~~~~~~~

SPL Parser

ADE

ExtractorMedDRA

Mapping

StructuredProductLabel

XMLDocument

HL7V3Specifica@on

Mandatedsince2006

8/4/2019 ADESSA AMIA Presentation Jon Duke

12/29

GeneralOverviewofhowSPLICER

ProcessesSPLs

Processors

List

MedDRA term identifier

Table

Section Identification

XML tag removal

Text Normalization

Sentence Detection

Processors

ExactMatch

Synonym

StopStemTransform

Similarity

Tokenizer

LaboratoryMapper

SPL

~~~~~~~~~~

~~~~~~~~~~

~~~~~~~~

SPL Parser

ADE

ExtractorMedDRA

Mapping

Cleansandprepares

SPLforextrac@on

8/4/2019 ADESSA AMIA Presentation Jon Duke

13/29

GeneralOverviewofhowSPLICER

ProcessesSPLs

Processors

List

Table

MedDRA term sweeper

Section Extraction

XML tag removal

Text Normalization

Sentence Detection

Processors

ExactMatch

Synonym

StopStemTransform

Similarity

Tokenizer

LaboratoryMapper

SPL

~~~~~~~~~~

~~~~~~~~~~

~~~~~~~~

SPL Parser

ADE

Extractor MedDRAMapping

ExtractsrawADEs

fromcleanSPL

8/4/2019 ADESSA AMIA Presentation Jon Duke

14/29

GeneralOverviewofhowSPLICER

ProcessesSPLs

Processors

List

MedDRA term identifier

Table

Section Extraction

XML tag removal

Text Normalization

Sentence Detection

Processors

ExactMatch

Synonym

StopStem

Transform

Similarity

Tokenizer

LaboratoryMapper

SPL

~~~~~~~~~~

~~~~~~~~~~

~~~~~~~~

SPL Parser

ADE

ExtractorMedDRA

Mapping

Mapsrawtermsto

MedDRAconcepts

8/4/2019 ADESSA AMIA Presentation Jon Duke

15/29

8/4/2019 ADESSA AMIA Presentation Jon Duke

16/29

DatabaseCrea@on

Retrieved5782SPLsfromDailyMed UsedSPLICERtoextractandmaptheADEsfromtheselabelstoMedDRApreferredterms

LinkedSPLstoRxCUIsusingRxNormviaSPL_SET_IDviaNDCcodes

MappedMedDRAtermstoSNOMED-CTviaUMLSCUIsynonymy

8/4/2019 ADESSA AMIA Presentation Jon Duke

17/29

DatabaseCrea@on

Successfullyprocessed5602SPLs180notprocessedduetoXMLvaria@on1706genericmedica@ons

Extracted545602ADEs22097uniquerawphrases8709uniqueMedDRAterms

PerLabelMeanADEs:70

MedianADEs:49Range:0to525

8/4/2019 ADESSA AMIA Presentation Jon Duke

18/29

ADESSA

RxCUI

String

CCD

Level3

Labeled

ADEs

ADESSAOverview

PHP/MySQL

NuSOAP

QuARK

RxCUI MedDRA

SNOMED-CT

8/4/2019 ADESSA AMIA Presentation Jon Duke

19/29

CCDMedica@onExample

Promethazine25MGOralTablet

8/4/2019 ADESSA AMIA Presentation Jon Duke

20/29

ExampleXMLResponse

8/4/2019 ADESSA AMIA Presentation Jon Duke

21/29

Evalua@onGoals

Assesstheaccuracyoftheadverseeventextrac@onprocess

Assessthecomprehensivenessandperformanceoftheserviceusingreal-worlddata

8/4/2019 ADESSA AMIA Presentation Jon Duke

22/29

NLPEvalua@on

IndependentmanualreviewofSPLICERoutputbyaMedDRAspecialist

For100randomlabels,reviewermarkedADEsTruePosiveADEinbothoutputandlabelFalsePosiveADEclaimedbySPLICERbuteither

notpresentinlabelornotatrueadverseevent

FalseNegaveADEpresentinlabelbutnotfoundbySPLICERornotcorrectlymapped

8/4/2019 ADESSA AMIA Presentation Jon Duke

23/29

NLPEvalua@on

Outof9529ADEs:9064TruePosi@ves

465FalsePosi@ves706FalseNega@ves

Recall:92.8%

Precision:95.1%

F-Measure:0.94

8/4/2019 ADESSA AMIA Presentation Jon Duke

24/29

WebServiceEvalua@on

PassedADESSA30randomlyselectedanonymizedCCDs

Measuredaccuracyofdrugcapture(howmanydrugswerecorrectlyiden@fiedandavailableinQuARK)

@mefromsubmissiontoresponse

8/4/2019 ADESSA AMIA Presentation Jon Duke

25/29

Results:DSS

For30CCDs,drugcountranged4-37,mean12.2 ADEssuccessfullyreturnedfor157/181(88%)ofthesedrugs

Meanresponse@me620ms Reasonsforfailuretoreturnresults(n=24)

7duetoCCDswithoutRxCUIcodes17duetoincompleteRxCUItoSPLmappingsinQuARK Innocasewasthetargetdrugmissingfromthedatabase

8/4/2019 ADESSA AMIA Presentation Jon Duke

26/29

Discussion

SPLICERextractedlabeledadverseeventdatawithhighprecisionandrecall

ADESSAreturnedpa@ent-specificADEdatarapidlyuponreceiptofaCCD/RxNormcodes

Poten@alvaluein:adverseeventrepor@ng(checkingagainstknownadversereac@ons)

clinicaldecisionsupport(associa@ngpa@entcondi@onswithpoten@allycausa@veagents)

8/4/2019 ADESSA AMIA Presentation Jon Duke

27/29

Limita@ons

Evenwith95%precision,s@ll~25,000incorrectasser@onsindatabase

Sta@csetdoesnotaccountfornewandupdatedlabels

EncounteredsomeissueswithmappingofRxCUIstoSPLs

SNOMEDconceptsmaynotreflectoriginalnuancedADEconcept

8/4/2019 ADESSA AMIA Presentation Jon Duke

28/29

Conclusions

Extrac@onviaNLPanddissemina@onviaDSScanprovidecodedADEdataforuseinCDS

FutureworkincludesSearchbyspecificADEorADEcategorySupportforreal-@meupdatesofQuARKusingFDAsRSSfeedfornew/updatedlabels

BroadeningofADESSAforuseoutsidefirewall

8/4/2019 ADESSA AMIA Presentation Jon Duke

29/29

Thanks!

Ques@ons?

JonDuke

jdukeregenstrief.org