Marie-Christine Jaulent Iulian Alecu

SPIM- UMRS872,EQ20 - France

MIE 2009 – Sarajevo - 31 aout 2009

Agenda

  History of the project   Pharmacovigilance and Signal detection   The ontological resource: OntoEIM   The web service PharmARTS

  Evaluation of the ontological resource   Context   Design of the study   Methodology of evaluation   Current results   Evolution and maintenance of the ontology

Pharmacovigilance and Signal Detection   Pharmacovigilance

  Set of procedures for the identification, evaluation and prevention of ADR (adverse drug reaction) risk

  ADR are coded in databases •  terminologies WHO-ART (WHO) & MedDRA (FDA – MSSO)

  Signal detection   Signal ↔ possible cause {Drug → ADR} (OMS)   Data mining - automatic analysis need prior automatic grouping

  Limits of terminology’ structure for grouping [Bousquet et a. 2005]   Lack of polyhierarchy   MSSO : manual answer to the problem, SMQ

•  Example: « acute pancreatitis » : 47 prefered terms, 320 terms including synonyms (Cullen's sign, Hereditary pancreatitis,Ischaemic pancreatitis, Blood amylase increased, Pancreatic enzymes increased, Urine amylase abnormal, Blood bilirubin increased, Gastrointestinal pain, Nausea, …)

•  75 SMQs in march 2009

An ontology to group cases « on the fly »  OntoEIM [Alecu et. Al., 2008]*

 90000 concepts (45000 SNOMED_CT concepts = 10% of all SNOMED_CT)

WHO-ART MedDRA Snomed_CT

* Alecu I, Bousquet C, Jaulent MC. A case report: using SNOMED CT for grouping Adverse Drug Reactions Terms. BMC Med Inform Decis Mak. 2008 Oct 27;8 Suppl 1:S4.

PharmARTS [Alecu et. Al., Medinfo 2007], US patent in 2007   Browse the resource for coding purpose,   Build queries to group terms   Retrieve cases in a pharmacovigilance database

Evaluation of OntoEIM   Context: Define « on the fly » a medical condition by

querying the ontology and grouping concepts   Question: Is the ontology appropriate to propose relevant

set of concepts for a given medical condition?

  Evaluation of ontologies [Portzel 2004; Rogers 2006; Cornet 2008] internal and external   the relevance of the vocabulary describing the concepts   the relevance of the “is-a” hierarchy   the relevance of the semantic relations  OntoEIM is evaluated according to its purpose

  Rogers 2006:   One significant problem for ontology quality assurance is the

lack of a gold standard against which to determine the correctness of the ontology and its suitability for the given purpose

 Definition of a precise gold standard: SMQ

Design of the study

SMQ

Medical condition

Built query

return

OntoEIM

COMPARE?

Gold Standard

Evaluation of the query   Modelisation for each medical condition

  Group of individuals: Set of possible terms (MedDRA)   Definition of the condition: term present in SMQ (SMQ

+)   Definition of the test (a query). We say that the test is:

•  Positive if the term is returned by the query (Q+) •  Negative if the term is not returned by the query (Q-)

SMQ + SMQ -

Q+ a b

Q- c d a b c

Sensibility: a / (a+c) Specificity: d / (d+b)≈1

Construction of the query A two steps automatic process  Selection of « candidate concepts »

in OntoEIM  Construction of an optimal query

Selection of « candidate concepts » in OntoEIM

Correspondance Example •  SMQ = ACUTE RENAL FAILURE •  C = ACUTE RENAL FAILURE

Composition

Example : 1) SMQ = Asthma/bronchospasm 2) C1 = Asthma 3) C2 = bronchospasm

C2 C1

Enlargment Example : 1)  SMQ = CONVULSIONS 2)  C = CONVULSIONS 3)  C’ = Seizures_incl_subtypes

C

C’

C

Construction of an optimal query   For each candidate concept, we calculate

a, b and c

  If there are several candidate concepts, we build as queries as possible using the union operator   C1, C2   Q1=C1; Q2=C2; Q3=C1+C2

  Final query   Q = Q such that max (a) and min (b+c) < λ   λ is an heuristic

a b c

Results Results for 24 SMQs

Number of terms in the SMQ Sensitivity

Mean Range Mean Range

23,5 [7 ; 47] 0,82 [0,45 ; 1]

  List of 24 SMQs Angioedema ; Acute central respiratory depression ; Acute pancreatitis; Acute renal failure ; Agranulocytosis ; Anaphylactic reaction; Asthma/bronchospasm ; Cardiac failure; Haemorrhagic cerebrovascular condition; Convulsions; Guillain-Barre syndrome ; Dyslipidaemia ; Hostility/aggression ; Interstitial lung disease; Lack of efficacy/effect; Neuroleptic malignant syndrome; Pseudomembranous colitis ; Peripheral neuropathy ; Pulmonary hypertension ; Retroperitoneal fibrosis ; Rhabdomyolysis/myopathy ; Severe cutaneous adverse reactions; Torsade de pointes/QT prolongation ; hyperglycemie/ diabete mellitus

Discussion Typology of errors   b≠0. The term is missing in the SMQ.

  SMQ « angioedema », term W:OEDEMA_CIRCUMORAL.

  b≠0. There is a taxonomic error in the ontology.   SMQ “acute pancreatitis”, term

W :PANCREATITIS_CHRONIC,   c≠0. Formal concept definitions are not

complete.   c≠0. Semantic relations are missing in

the ontology   the relation “evoke” between an exam result

and a diagnosis does not exist in the ontology

Conclusion   Evaluation of an existing resource

  Specific purpose : prior grouping of terms to improve the performances of signal detection algorithms in pharmacivigilance

  Evaluation by comparison with the existing SMQs   Implementation of the approach   Reusability

  Ontology evolution : The method does not support the development of the ontology but allows identifying how it has to evolve by finding the corrections that have to be made (missing definitions or missing relations, wrong definitions or wrong relations)

  Ontology maintenance: measure if the ontology remains adequate when there is a new context (construction of new SMQs by the MSSO)

Perspectives

  Continue the evaluation for the other SMQs

  Collect manually the classification and corrections of errors done during the evaluation in order to extract, if possible, some rules that could help the correction process.

  Design tools and interfaces to assist the resolution of the errors revealed by the method

  Define medical conditions « on the fly » (design of new SMQs)