using social media to support safety surveillance efforts
TRANSCRIPT
Nabarun Dasgupta, MPH, PhDEpidemico, Inc.
September 11, 2015 – M-CERSI Workshop
Using Social Media to Support Safety
Surveillance EffortsSlides
available now
@epidemico
R E A L -W O R L D E X P E R I E N C E
00Disclosur
e
13Patient Benefits
06Supply Chain
16DEMO
10Post-
marketing
01Data
Taming
09Clinical Trials
11Space +
Time
14Limitatio
ns
20Minutes
01. USAA public-private partnership entering Year 2. The WEB-RADR project is supported by the Innovative Medicines Initiative Joint Undertaking (IMI JU) under grant agreement n° 115632, resources of which are composed of financial contribution from the European Union's Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution.
02. EUROPEGSK has been a development partner to extend the technology platform to meet the standards of regulated software for the last year and half. Epidemico provides data to other pharmaceutical companies and CROs.
03. INDUSTRY
Disclosures Funding Sources
FDA has been funding the research into the technology for 3 years.
Office of the Chief Scientist (OCS)
Center for Tobacco Products (CTP)
Office of International Programs (OIP)
EPIDEMICO, Inc. + EPIDEMICO, Ltd.Authors are employees or consultants. Epidemico is commercializing aspects of this software platform. Epidemico is a wholly own subsidiary of Booz Allen Hamilton.
How to Tame Social Media
The Drug I’m TakingThe Drug I’m Responsible for Monitoring
Remove false
positives(optional)
Hypothesis generation
Automated processes to
identify events and clean data
Collect publicsocial
media data
Synthesis of
information from other
sourcesAcquire Curate Causation
StatisticsFilter
DATA P R O C E S S OV E RV I E W
SC
RA
NTO
N B
, DE
VR
IES
J, B
ER
INAT
O S
, BO
RTZ
C. H
AR
VAR
D B
US
INE
SS
RE
VIE
W,
JUN
E 2
013
Geotag
Geography identified if available usingGoogle API
4
32
1
ConsolidateMultiplicate copies & posts from same user consolidated(PII removed)
Translate
Internet vernacular to MedDRA v18 preferred terms
Identify Events
Automated NLP Bayesian classifierMachine learning
A U T O M AT E D P R O C E S S E S
Spelling VariationsCombined or invented wordsImplied phrasesInternet Vernacular Translation
lost their eyesight
seeing weird colorseeing weird colour
doublevision
couldn’t seedouble vision
blindgoogley eyed
blurry vision
changes in visioncross eyedseeing weird
vision changeblindness
cross visionvisual snow
googly eyedseeing double
making me eat like a mouseanorexic
lost appetite
#notevenhungryappetite is nonexistent
apetite surpressed
didn’t get hungry
dont want to eat
killed my apetite
miss feeling hungry
killed my appetitecan’t eatlost apetite
lost my appetiteno appetitey
lack of apetitestomach small
lost teh appetite
never hungrynever want to eat
cant eat
couldnt seecrosseyed
blurry
apetite surpressed
killed my apetitelost apetite
lack of apetite
lost teh appetite
Typos
seeing weird colorseeing weird colour
googley eyed
googly eyed
#notevenhungry
no appetitey
making me eat like a mouse
never want to eat
Visual impairmentMedDRA 10047571
Visual impairmentSNOMED 397540003
Decreased appetiteMedDRA 10061428
Loss of appetiteSNOMED 79890006
making me eat like a mouseanorexic
lost appetite
#notevenhungryappetite is nonexistent
apetite surpressed
didn’t get hungry
dont want to eat
miss feeling hungry
killed my appetitecan’t eatlost apetite
lost my appetiteno appetitey
lack of apetitestomach small
lost teh appetite
never hungrynever want to eat
cant eat
lost their eyesight
seeing weird colorseeing weird colour
doublevision
couldn’t seedouble vision
blindgoogley eyed
blurry vision
changes in visioncross eyedseeing weird
vision changeblindness
cross visionvisual snow
googly eyedseeing double
couldnt seecrosseyed
blurry
killed my apetite
P ro t o -A EP o s t s w i t h R e s e m b l a n c e t o A d v e r s e E v e n t s
ML classifier
trained by manual
review of posts
Precision-
Positive predictive
value
Recall-
Sensitivity
Posts mentioning
medical products
60M 380k
68% 88%
S Y S T E M P E R F O R M A N C E~100% with manual curation
Supply Chain Safety for Drugs & Food
OIP investigated further on Tingyi Holding Corp:- The company actively ships to the U.S. with increasing shipments from 2011-2014, two refused entry lines in 2012 and four detained or reconditioned lines from 2011-2013- Products include instant noodles, tea, fruit juices, and prepared baked goods- Multiple distributors of its products with the U.S.- OIP presented its findings to the Centers with a mapping of potential networks of suppliers identified using FDA internal systems.
Counterfeit Products: Weibo
Consumer purchased extremely cheap ointment sold under brand name Bactroban but was not manufactured by GSK.
GSK’s Bactroban
Clinical Trial Unblinding and Drop-out
18
CLINICAL TRIAL DISCUSSION
I'm in a [Company] double blind ######### trial and could receive the placebo up until week 16 at which point I'm guaranteed to receive the drug.
What I don't understand is that I can't be told if I was on the placebo for the first 16 weeks. I'm not seeing any positive benefit from whatever I've been taking for 8 weeks so far, and I would really like to know at week 16 if it's the drug because then I can bail from the study at that time and move on to something else…
19
CLINICAL TRIAL DISCUSSION
It sounds like [your father] very well may be on the real thing. Those are definite improvements.
…I rather doubt my bro is on the real thing, even with the physical things that have happened to him (constipation, UTI and prostate infections.) Either that, or it's not a medication that's going to work for him. I'm so afraid the latter is the case. If so, what next??
…If he's been on it, I can't see any reason to continue…If he hasn't been getting it, then certainly it's worth continuing with the real thing. Problem is we won't find out until this study is completed for everyone currently in it…
20
CLINICAL TRIAL DISCUSSION
He started on Thursday. He told me yesterday that he tried to chew a capsule (I'm not sure why, I think he wanted to see if it had a flavor) & it was hard as a rock. I'm assuming maybe it was eneric coated or something. He took the no flavor to mean he's on the placebo, but I actually wonder if they would bother enteric coating a placebo. Probably doesn't matter & I know we're not supposed to know anyway, just wondered if anyone else's was enteric coated.
Rectifying A False Negative in Spontaneous
Report Data
Submit to Regulator
Review Company’s Safety Database
Routine Portfolio Surveillance
Twitter Post Identified
“Did this product cause x, y, z? I’ve had 3 administrations of this product and ever since then have felt a, b, c.”
SOCIAL DATAHELPING SPONTANEOUS DATA
172
Real-time Geo-temporal Surveillance for Adverse Events
Asking New Kinds of Questions in Pharmacovigilance
Remove Spam, news, coupons67% of posts were spam
Consolidate multiplicate copies35% Excluding literal duplicates and retweets
Keep posts with in-scope geographyKeep posts from US, UK, CA, AUIdentifiable geography for 55% of Proto-AEs
Import Twitter PostsAll DTC products
Limit to English LanguageOnly 6 posts with unknown
language
Identify Posts with Proto-AEsAutomated
classifier
Exclude Chantix/ChampixDTC advertising allowed in
non-US countries
1,245,530
408,867
338,263
218,512
11,265
5,535
4,499
DATA P R O C E S S I N G18 month study period 58 Rx Drugs w/DTC ads
18 month study period 58 Rx Drugs w/DTC ads
ResultsW h a t H a p p e n s a t 1 0 O ’ C l o c k ?
"Most AEs occurred between 08:00 and 12:00a.m. with a peak around 10:00 a.m."
“The majority of postings [about migraine] occurred in the United States, peaking on weekdays at 10:00h and then gradually again at 22:00h…”
Can we quantify patient benefit?
12 month study period 15 GSK Rx & OTC Products
Source: Powell GE, Duke SP, Bell HG, Anderson LS, Metcalf MA. In Their Own Words: Social Listening for “Real-World Benefits” from Prescription and OTC Products. Poster, ISPOR 2015.
DEMO
LIMITATIONS OF SOCIAL MEDIACAUSALITY & VERACITYPatients may not correctly assess causality. Can system be gamed?Develop tools to assess manipulation.VOLUMEVolume of reports likely to be large. Reduce false positives and create automated tools to triage information.SIGNAL DETECTIONVery limited statistical methods to detect problems. Collaborate with academia, industry and regulators to refine methods.PRIVACY & GENERALIZABILITYPatient privacy expectations and fear of government oversight. Use publicly available data only.REGULATION UNCLEARWhen is there an obligation to monitor or report? Work with regulators and industry to clarify guidance.
01
02
03
04
05
CarrieMike Aranka Ciara Nabarun Wenjie
Harold John Robin Chi Sue
Kelly Kate Clark Chris Robin
Derek
Matt
Essam
Melissa
Kyra
Nico
Lou
Carly
Lucas
