matching key safety questions with appropriate algorithms final
TRANSCRIPT
![Page 1: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/1.jpg)
Matching key safety questions with
appropriate algorithms for
appropriate corrective actions
Lionel Van Holle (GSK Vaccines)
DSRU 10th June 2015
![Page 2: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/2.jpg)
Table of content
� Introduction
�The key Safety questions
�The appropriate corrective actions
�The role of algorithms screening SRDs�Disproportionality
�Time-to-onset signal detection
�Logistic regression
�Tree-based scan statistics
�The full quantitative signal detection toolkit
�Future developments needed
�Conclusion
![Page 3: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/3.jpg)
Introduction
• Disproportionality algorithms have been used for screening spontaneous report databases (SRDs) for more than a decade.
• Are they enough or should we develop otheralgorithms?
• What for?
– Better answering the same safety question as before?
– Answering other safety question?
![Page 4: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/4.jpg)
The Key Safety Questions
1. Does the product cause adverse reactions?
�Product-related safety issues
2. Does an ingredient of my product cause adverse reactions?
�Ingredient-related safety issues
3. Does a subset of manufactured products cause adverse reactions?
�Manufacturing-related safety issues
![Page 5: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/5.jpg)
Safety issues Examples
Product-related Thalidomide drug for morning sickness -> birth
defects, limb malformations in ~ 10,000 children
worlwide
Ingredient-related E-ferol: an injectable preparation of alpha-tocopherol
(vitamin E) for parenteral nutrition was recalled from
the market because of unusual liver and kidney
syndromes with 38 deaths reported among treated
low birthweight infants -> syndrome most likely due
to a combination of alpha-tocopherol, polysorbates,
contaminant.
Manufacturing-related Cutter incident -> some lots of the Cutter vaccine
(polio) were not properly inactivated and contained
live polio vaccine -> 120,000 doses distributed;
40,000 developed abortive poliomyelitis; 56
paralytic; 5 deaths
![Page 6: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/6.jpg)
The appropriate corrective actions
• ? Safety profile re-evaluation
• ? B/R re-evaluationProduct-related
safety issue
• ? Strategy of ingredientsubstitution/removal
Ingredient-relatedsafety issue
• ? Recall of manufacturing lot(s)
• ? Development of new QC/QA tests
Manufacturing-related safety issue
![Page 7: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/7.jpg)
The role of algorithms screening SRDs
Due to high number of spontaneous reports
preventing individual medical assessment
All product-event pairs
First-pass
screening
Causality
Assessment
Algorithms that do not require
prior medical assessment
Association
Temporality
Specificity
Consistency
Biological gradient
Experimentation
Plausibility
Analogy
SAFETY SIGNALS
![Page 8: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/8.jpg)
Disproportionality
When disproportionality is used in routine, it
compares the observed number of reports for a
given product-event to what is expected from
other/all products.
Event of interest Other (or all) events
Product of interest A B
Other (or all) products C D
��� �
�
��
�∗�
∗��
�∗�
����
���
![Page 9: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/9.jpg)
Time-to-onset signal detection
Kolmogorov-Smirnov tests
If the distance between
Cumulative distributions is
unexpected
Van Holle L et al, Using time-to-onset for detecting safety signals in spontaneous reports of adverse events
following immunization: a proof of concept study. PDS 2012; 21: 603-610. DOI: 10.1002/pds.3226
![Page 10: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/10.jpg)
Logistic regression
• Determines if the reporting pattern (in terms
of causality criteria/strength of evidence) of a
product-event pair is similar or not to the
reporting pattern of a positive reference set
(i.e emerging signals or listed events)
Van Holle L et al, Use of logistic regression to combine two causality criteria for signal detection in vaccine spontaneous
report data. Drug Safety (2014) 37:1047-1057.
Caster O et al, Improved statistical signal detection in pharmacovigilance by combining multiple strength-of-evidence
aspects in vigiRank. Drug safety (2014) 37: 617-628.
![Page 11: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/11.jpg)
• Integrate more causality criteria in the first-passscreening
BUT
• Routine signal detection methods (DPA or more advanced ones: TTO, LogReg, Supervised methods) use other products as comparator and are consequentlyinappropriate for detecting ingredient-based or manufacturing-based safety issues.
• It leads to potential non-detection of these issues or worse, detection at the wrong level (product).
Role of these methods?
![Page 12: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/12.jpg)
Relevant Action performed ?Safety profile update
• LIKELY if product related
• POSSIBLE if large fraction of lots
Ingredient substitution
• UNLIKELY if ingredient shared(Rely only on qualitative assessment or ad hoc analysis)
Lot(s) recall
• UNLIKELY if small fraction of lots (Rely only on qualitative assessment or ad hoc analysis)
Algorithms
Routine Disproportionality [+ TTO, LogReg]
Safety issue
Product-related Ingredient-related Manufacturing-related
![Page 13: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/13.jpg)
Tree-based scan statistic
Originally used in disease surveillance: e.g.
investigating ‘death from silicosis’ (event of
interest) incidence among different occupations
or group of occupations (exposure).
Potential cuts in the tree structure
symbolizing a scanning window
investigating combinations of
occupations
![Page 14: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/14.jpg)
• For each tree cut, the rate of the event of interest (λ)
in the window scan (G) or not (R) is calculated. Total
number of cases of interest if fixed (c).
• Null hypothesis (H0) is that the rate of the event-of-
interest is the same in the window scan than outside.
• Alternative hypothesis (H1) is that the rate of the
event-of-interest is higher in the window scan than
outside.
• A likelihood ratio can be built (H1/H0)
Kulldorf M et al, A tree-based scan statistic for database disease surveillance. Biometrics (2003) 59, 323-331.
![Page 15: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/15.jpg)
• The cut with the maximal likelihood ratio constitutes
the test statistic
• Significance is measured through Monte Carlo
simulations
• It adjusts for multiple testing across the multiple cuts
in the tree structure.
It can be adapted from disease surveillance to adverse
reaction surveillance if we link spontaneous report data
with hierarchical data representing an exposure of
safety relevance.
Kulldorf M et al, A tree-based scan statistic for database disease surveillance. Biometrics (2003) 59, 323-331.
![Page 16: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/16.jpg)
Has the potential for being an algorithm for detecting
manufacturing-related issues & identifying most likely
manufacturing step
Specific
to SRDs
![Page 17: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/17.jpg)
Has the potential for being an algorithm for detecting
ingredient-related issues & identifying most likely
ingredient
![Page 18: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/18.jpg)
Relevant Action performed ?
Safety profile update
LIKELY if product related
Ingredient substitution
LIKELY if ingredient-related
Lot(s) recall
LIKELY if manufacturing-related
Algorithms
Routine Disproportionality [+ TTO, LogReg]
Tree-based scan (linked to product dictionary)
Tree-based scan (linked to manufacturing data)
Safety issue
Product-related Ingredient-related Manufacturing-related
Full quantitative SD toolkit
![Page 19: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/19.jpg)
Future developments needed?
• Spontaneous report database (SRD) need to be linked to external information:
– A product dictionary with a hierarchical structure allowing to see the different ingredients of the product (non-independence of the products)
– A manufacturing database with a hierarchicalstructure allowing to see the differentmanufacturing steps (non-uniformity in production)
>< DPA approach with standalone SRD, flexible enough softwares?
![Page 20: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/20.jpg)
• Define which events to monitor for manufacturing-related or ingredient-relatedsafety issues:– All MedDRA PTs as for product-related safety? (no a
priori)
– A subselection based on biological plausibility?
– A grouping of terms?
– …
• Extend the scope of tree-based scan statistic to allow integration of other causality criteria (thannumbers) as for product-related safety issues?
• Need a zero-pass screening to determine the most likely scenario (product, manufacturing, ingredient)?
![Page 21: Matching key safety questions with appropriate algorithms Final](https://reader031.vdocuments.site/reader031/viewer/2022030314/589ed1b71a28ab47138b6e5d/html5/thumbnails/21.jpg)
Conclusion
• Quantitative signal detection toolkit shouldcontain algorithms of signal detection able to detect different types of safety issues (product, ingredient, manufacturing)
• The tree-based scan statistic is a good candidate for filling the current gap that preventsappropriate corrective actions
• Creation of a product dictionary & a manufacturing hierarchy database will berequired.