1

Ontology-enabled Healthcare Applications Exploiting Physical-

Cyber-Social Big DataOntology Summit for the Health Care Track on Semantic Integration, 7 April 2016

Amit ShethKno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing:

An Ohio COE on BioHealth InnovationWright State University

Special thanks: Sujan Parera

33%

35%

32%

Kno.e.sis: Ohio Center of Excellence in Knowledge-enabled Computing

DoD & Industry• Metabolomics & Proteomics• Medical Info Decisions• Human Detection

on Synthetic FMV• Sensor & Information• Material Genomics• Cardiology Semantic Analysis

NIH: National Inst. of Health• kHealth - Asthma• eDrug Trends• Depression on Social Media• Drug Abuse Early Warning

NSF: National Science Foundation• Harassment on Social Media• Citizen & Physical Sensing• Twitris - Collective Intelligence• Aerial Surveillance• Visual Experience• Web Robot Traffic

Kno.e.sis’ research in World Wide Web ranks Wright State University among the top 10 organizations in the world based on 10-yr impact. Its total budget for currently active projects is $11,443,751, with $5,912,162 for new projects starting after July 2015. The significant majority of funds are highly competitive federal grants. World-class research is complemented by exceptional student outcomes and commercialization with local economic impact.

As an Ohio COE on Bio Health Innovation, Kno.e.sis conducts research leading to building intelligent systems for clinical, biomedical, policy, and epidemiological applications.

Example clinical/healthcare applications include major diseases such as asthma, depression, cardiology, dementia and GI.

This is complemented by social and development challenges such as marijuana legalization policy, harassment on social media, gender-based violence, and disaster coordination.

60+ Funded Students• 40 PhD• 16 MS• 5 BS

3

Collaborators

Projects @ Kno.e.sis

Image Credits: https://www.boxchilli.com/wp-content/uploads/2014/07/socialmediamarketing.png , http://rotwnews.com/wp-content/uploads/2014/04/DRUG-TRENDS-Talk.jpg, http://slapheadmarketing.com/wp-content/uploads/2012/05/Forum-Marketing.jpg

eDrugTrends is social media data analytics platform to monitor the cannabis and synthetic cannabinoids usage. It uses social media and Web forums data to: 1) Identify and compare trends in knowledge, attitudes, and behaviors related to cannabis and synthetic cannabinoid, and 2) Identify key influencers in cannabis and synthetic cannabinoid-related discussions on Twitter.

eDrugTrends

Data Sources

Project Wiki

Daily average content: Tweets: 135,553Forum Posts*: 8,899Total: 144,452* Bluelight, Drugs-forum, and Reddit

Projects @ Kno.e.sis

Image Credits: https://i.ytimg.com/vi/GOK1tKFFIQI/maxresdefault.jpg, http://slapheadmarketing.com/wp-content/uploads/2012/05/Forum-Marketing.jpg, https://www.boxchilli.com/wp-content/uploads/2014/07/socialmediamarketing.png, https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcSucmCuyWvX4dFHv5XvS3KzjvD11hC8HwK9N4004LnBZOGLOgf6, http://www.crmchealth.org/sites/default/files/images/medical-records/Medical_Records_0.png?1314713869

Identifying combinations of online socio-behavioral factors and neighborhood environmental conditions that can enable detection of depressive behavior in communities and studying access and utilization of healthcare services

Depression Behavior

Data Sources

Electronic Medical RecordsPublic Surveys

Project WikiSearch Log

Depending on collection method,We get 7-17K tweets per day, and

Have 800K to 18M total tweets in several months.

Projects @ Kno.e.sis

This project seeks to understand and satisfy users’ need for keeping track of new information in healthcare and well-being. The project harvest collective intelligence to identify high quality, reliable and informative healthcare content shared over social media based on following analysis: Text Analysis, Semantic analysis, Reliability analysis, Popularity Analysis.

Social Health Signals

Data Sources

Image Credits: https://www.boxchilli.com/wp-content/uploads/2014/07/socialmediamarketing.png, https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcS6qI3Z_Y0Uh0sPNCgy0J_0d66-5NsCwK3VqWsIkAKRmqjTSXK0uA

Project Wiki

kHeath analyzes both active and passive observations of the patients to generate the alarms that helps to improve health, fitness, and wellbeing of the patient. It uses Semantic Sensor Web technology, Semantic Perception, and Intelligence at the Edge to enable sophisticated analysis of personal health observations.

kHealth

Projects @ Kno.e.sis

Data Sources

image credits: https://www.boxchilli.com/wp-content/uploads/2014/07/socialmediamarketing.png, https://www.cooking-hacks.com/media/cooking/images/documentation/e_health_v2/e_health_sensors_small.png, http://www.co.freeborn.mn.us/ImageRepository/Document?documentID=483

Public Health APIs

Project Wiki

Projects @ Kno.e.sis

Monitor the health status of the military personnel in training period through self-reported fitness notes and continuous monitoring with body sensors. The collected data is used to assess the health status of the person and suggest exercise regimen change or treatment plans if needed.

MIDAS

Data Sources

image credits: https://www.cooking-hacks.com/media/cooking/images/documentation/e_health_v2/e_health_sensors_small.png, https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcROFoUIaLPDWcvcCmoi1-sl8Bl3CPUtZooX5HHPuDiQKGI7oFZfuQ

Self-reported Data

Project Wiki

Projects @ Kno.e.sis

PREDOSE developed techniques to facilitate prescription drug abuse epidemiology, related to the illicit use of pharmaceutical opioids. PREDOSE is designed to capture the knowledge, attitudes and behaviors of prescription drug abusers through the automatic extraction of semantic information from social media.

PREDOSE

Data Sources

image credits: https://www.boxchilli.com/wp-content/uploads/2014/07/socialmediamarketing.png, http://slapheadmarketing.com/wp-content/uploads/2012/05/Forum-Marketing.jpg, https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTrMsVTVc6RJrWZtst5ZTILWoD83HO0DPbj3I89YSqMiNRdwI7S

Project Wiki

Projects @ Kno.e.sis

The scientific analysis of the parasite Trypanosoma cruzi (T. cruzi), the principal causative agent of human Chagas disease, is the driving biological application of this project. We developed and deployed a novel ontology-driven semantic problem-solving environment (SPSE) for T.cruzi

SPSE – T.cruzi

Data Sources

image credits: https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRB-YQT3LWMXm9vfv3IdclcMDjP-_ChizcFMw53OAkptnHdaUAn6w, http://www.clfs.umd.edu/biology/machadolab/images/trypanosoma.jpg, http://web.eecs.umich.edu/~dkoutra/courses/W16_484/

Public & Private Databases (Uniprot, GO, KEGG, TriTrypDB

Project Wiki

experimental data from mass spectrometry and microarray experiments

Textual Data

Ontologies Developed at Kno.e.sis

• Drug Abuse Ontology – 83 classes, 37 properties• Depression Insight Ontology – ongoing work• Healthcare Ontology/ezDI Knowledge Graph – proprietary• Human Performance and Cognition “Ontology” – 2 million entities, 3

million facts (HPCO)• Ontology for Parasite Lifecycle – 360 classes, 12 properties (BioPortal)• Parasite Experiment Ontology – 142 classes, 40 properties (BioPortal)• Provenir Ontology - 88 classes, 23 properties (Provenir) – a key input to

W3C provenance work

Earlier at UGA: ProPreO (500+ classes), GlycO,…

Semantic Filtering

Data Integration

Knowledge Enrichment

Entity Annotation

Triple Extraction

Sentiment Analysis/Intent Mining/User Modeling

r1

Search/Browsing/Summarization/Trend/Analysis/Prediction

Knowledge Base Usages @ Kno.e.sis

Data alone is not enough.KB+NLP+ML

Explanation Module

Explained?

Yes

NoHypothesis

FilteringHypothesis Generation

Hypothesis with High

Confidence

D

D D

DD

D

Patient Notes

UMLS

Knowledgebase Enrichment

Sujan Perera, Cory Henson, Krishnaprasad Thirunarayan, Amit Sheth, and Sahas Nair. "Semantics driven approach for knowledge acquisition from EMRs." Biomedical and Health Informatics, IEEE Journal of 18, no. 2 (2014): 515-524.

Knowledgebase Enrichment

● Knowledge in a given knowledge base may not always sufficient● Acquiring required knowledge in some domains is a tedious task● Data available for a particular domain may contain required knowledge● Partial knowledge about the domain can be used to efficiently acquire

domain knowledge from data that can fill existing gaps in a knowledge base

Data

Data Integration with Ontologies

UniPort

Internal Lab Data

T.cruzi DBNCBI Data Sources

PubMed

T.cruzi immunology ontology Parasite Experiment ontology T.cruzi life cycle ontology

Aligned Ontologies

● Qualitative studies such as telephonic survey which suffer from limited population coverage and large temporal gaps.

● To address limitations of the qualitative studies, researchers have used various data sources such as social media (e.g. Twitter), web search logs, and neighborhood factors.....but in silos

Depression

Social Media

Web Search log

Neighborhood factors

EHR data

Depression Behavior

Depression Behavior

PREDOSE

Population Level

Personal

Wheeze – YesDo you have tightness of chest? –Yes

Observations Physical-Cyber-Social System Health Signal Extraction Health Signal Understanding

<Wheezing=Yes, time, location><ChectTightness=Yes, time, location>

<PollenLevel=Medium, time, location>

<Pollution=Yes, time, location><Activity=High, time, location>

Wheezing

ChectTightness

PollenLevel

Pollution

Activity

Wheezing

ChectTightness

PollenLevel

Pollution

Activity

RiskCategory

<PollenLevel, ChectTightness, Pollution,Activity, Wheezing, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory>

.

.

.

Expert Knowledge

Background Knowledge

tweet reporting pollution level and asthma attacks

Acceleration readings fromon-phone sensors

Sensor and personal observations Signals from personal,

personal spaces, and community spaces

Risk Category assigned by doctors

Qualify

Quantify

Enrich

Outdoor pollen and pollution

Public Health

Well Controlled - continueNot Well Controlled – contact nursePoor Controlled – contact doctor

kHealth

SSN

Dealing with Heterogeneity

He showed shortness of breath in last visit

Dyspnea was observed in his last visit

It is observed that patient has labored breathing

The patient was breathing comfortably in room air

He showed short of breath in last visitC0013404

shortness of breath dyspnea

Labored or difficult breathing associated with a variety of disorders, indicating inadequate

ventilation or low blood oxygen.

rdfs:labelrdfs:label

is_defined_as

Expressing the Shortness of Breath

explicit mention

syntactic variation

synonym

positive implicit mention

negative implicit mention

individual literal

I was sent home with 5 x 2 mg Suboxones. I also got a bunch of phenobarbital (I took all 180 mg and it didn't do shit except make me a walking zombie for 2 days). I waited 24 hours after my last 2 mg dose of Suboxone and tried injecting 4 mg of the bupe. It gave me a bad headache, for hours, and I almost vomited. I could feel the bupe working but overall the experience sucked.

Of course, junkie that I am, I decided to repeat the experiment. Today, after waiting 48 hours after my last bunk 4 mg injection, I injected 2 mg. There wasn't really any rush to speak of, but after 5 minutes I started to feel pretty damn good. So I injected another 1 mg. That was about half an hour ago. I feel great now.

Codes Triples (subject-predicate-object)Suboxone used by injection, negative experience Suboxone injection-causes-Cephalalgia

Suboxone used by injection, amount Suboxone injection-dosage amount-2mg

Suboxone used by injection, positive experience Suboxone injection-has_side_effect-Euphoria

experience sucked

feel pretty damn good

didn’t do shit

feel great

Sentiment Extraction

bad headache

+ve

-ve

Triples

DOSAGE PRONOUN

INTERVAL Route of Admin.

RELATIONSHIPS SENTIMENTS

DIVERSE DATA TYPES

ENTITIES

I was sent home with 5 x 2 mg Suboxones. I also got a bunch of phenobarbital (I took all 180 mg and it didn't do shit except make me a walking zombie for 2 days). I waited 24 hours after my last 2 mg dose of Suboxone and tried injecting 4 mg of the bupe. It gave me a bad headache, for hours, and I almost vomited. I could feel the bupe working but overall the experience sucked.

Of course, junkie that I am, I decided to repeat the experiment. Today, after waiting 48 hours after my last bunk 4 mg injection, I injected 2 mg. There wasn't really any rush to speak of, but after 5 minutes I started to feel pretty damn good. So I injected another 1 mg. That was about half an hour ago. I feel great now.

I was sent home with 5 x 2 mg Suboxones. I also got a bunch of phenobarbital (I took all 180 mg and it didn't do shit except make me a walking zombie for 2 days). I waited 24 hours after my last 2 mg dose of Suboxone and tried injecting 4 mg of the bupe. It gave me a bad headache, for hours, and I almost vomited. I could feel the bupe working but overall the experience sucked.

Of course, junkie that I am, I decided to repeat the experiment. Today, after waiting 48 hours after my last bunk 4 mg injection, I injected 2 mg. There wasn't really any rush to speak of, but after 5 minutes I started to feel pretty damn good. So I injected another 1 mg. That was about half an hour ago. I feel great now.

Buprenorphine

subClassOf

bupe

Entity Identification

has_slang_term

SuboxoneSubutex

subClassOf

bupey

has_slang_term

Drug Abuse Ontology (DAO)

83 Classes37 Properties

33:1 Buprenorphine24:1 Loperamide

Ontology Lexicon Lexico-ontology Rule-based Grammar

ENTITIESTRIPLES

EMOTIONINTENSITYPRONOUN

SENTIMENT

DRUG-FORMROUTE OF ADM

SIDEEFFECT

DOSAGEFREQUENCY

INTERVAL

Suboxone, Kratom, Herion, Suboxone-CAUSE-Cephalalgia

disgusted, amazed, irritatedmore than, a, few of

I, me, mine, myIm glad, turn out bad, weird

ointment, tablet, pill, filmsmoke, inject, snort, sniff

Itching, blisters, flushing, shaking hands, difficulty breathing

DOSAGE: <AMT><UNIT> (e.g. 5mg, 2-3 tabs)

FREQ: <AMT><FREQ_IND><PERIOD> (e.g. 5 times a week)

INTERVAL: <PERIOD_IND><PERIOD> (e.g. several years)

Smarter Data Generated with Ontologies

Thank You

Visit Us @ knoesis.org Follow us @ facebook.com/Kno.e.sis

One example of commercial applications: ezdi.com

with additional background at http://knoesis.org/amit/hcls

Ohio Center of Excellence in Knowledge-enabled Computing -An Ohio Center of Excellence in BioHealth Innovation

Wright State University