personalised terms derivative- semantic stemming

Personalized Terms DerivativeBeyond Stemming

Root

Word 1

Word 2

Word 3Root

Word 4

Word 5

What is PTD?

Semantic approach for Stemming

Capabilities to include Ontology and Semantic dictionary like wordnet

Extends the meaning of stemming to hypernym and user defined root word

Includes linguistic and morphological stemming provided by Porter-Stemmer and K-Stem

Algorithm Steps

Preprocessing Word Sense Disambiguation Sense Annotation Relationship K-Stem

Architecture Flow Diagram

Extract

Sentences

Lemmatized each word in the Sentences

Named Entity Extraction , Tokenization

Annotation

For Each Sentences

For Each Tokens

If Term = N-E

Word Sense Disambiguation for each

word in a sentence

Right sense of each word ,

POS Identification

TaggingIf

POS =

NN

For Each Tokens

YES

YES

NO

Find Hypernym/Root word

NO (Till all terms are processed once)

Knowledge Data Sources

Thesaurus

Dictionary,Ontology

Create Exclusion List

K-Stemmer

If Root word not found

Documents

Application Area

Search, Index Engine(Solr, ES)

Personalized Search(Dictionary)

WEB Search(Google)

Targeted Stemming

Text Summarization(Abstractive)

ComparisonWords Porter

StemmerPTD with Wordnet

PTD with Ontology

Source Link

oncology oncolog oncology medical specialty

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3596793/mutation mutation organism genetic

variationhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC3596793/apoptosis apoptosi cell death cell

physiological processes

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3596793/

biological biolog biological discipline https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3596793/immune immun immune pathogeneses https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3596793/cytarabine cytarabin cytarabine pyrimidine

nucleosideshttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC3596793/metastasis metastasi pathological

processcancer progression

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2927383/

Demo

Demo : http://apis.tekstosesense.com Source Code : https://github.com/TekstoSense/word-root-finder Home : http://www.tekstosense.com Source Code Home :

https://github.com/TekstoSense/about-tekstosense