Identification of clinically Identification of clinically important PECODR elements important PECODR elements

within medical journal abstractswithin medical journal abstracts

PPatient-population-problem, atient-population-problem, EExposure-intervention, xposure-intervention, CComparison, omparison,

OOutcome, utcome, DDuration, & uration, & RResultsesults

Martin Dawes, Pierre Pluye, Laura Shea, Roland Grad, Arlene Martin Dawes, Pierre Pluye, Laura Shea, Roland Grad, Arlene Greenberg, Jian-Yun NieGreenberg, Jian-Yun Nie

Evidence-Based PracticeEvidence-Based Practice

See a patientSee a patient Have a questionHave a question Sometimes seek an answer Sometimes seek an answer Firstly in secondary sources Firstly in secondary sources

E-B synopses eg GAC guidelinesE-B synopses eg GAC guidelines Secondly in primary sources Secondly in primary sources

MedlineMedline

QuestionsQuestions

Unstructured = UnanswerableUnstructured = UnanswerableMake them structuredMake them structured PP Do women > 50 yearsDo women > 50 years EE taking calciumtaking calcium CC compared to women taking nothingcompared to women taking nothing OO have fewer hip fractureshave fewer hip fractures DD over 5 yearsover 5 years RR NNT of 30NNT of 30

PROBLEMS - SOLUTIONS - QUESTIONPROBLEMS - SOLUTIONS - QUESTION

ProblemsProblems Too much informationToo much information Information keeps changingInformation keeps changing

SolutionsSolutions Filters for evidence Filters for evidence MESH & PubMed clinical queriesMESH & PubMed clinical queries

QuestionsQuestions1.1. Would PECODR indexing the clinical literature Would PECODR indexing the clinical literature

help me find information for my patient?help me find information for my patient?2.2. Can we identify PECODR elements in abstracts?Can we identify PECODR elements in abstracts?3.3. Would the words or groups of words used help us Would the words or groups of words used help us

to automatically do this?to automatically do this?

METHODSMETHODS

Pilot studyPilot study

Convenience sample (N = 40)Convenience sample (N = 40) 20 research articles (Evidence Based Medicine 20 research articles (Evidence Based Medicine

synopsessynopses))

20 corresponding 20 corresponding abstractsabstracts (PubMed)(PubMed)

Converted the information into a text fileConverted the information into a text file

Analyse the content: Qualitative and QuantitativeAnalyse the content: Qualitative and Quantitative

MethodsMethods 6 themes defined: P, E, C, O, D, R6 themes defined: P, E, C, O, D, R Extract text and allocate to a themesExtract text and allocate to a themes

extracts made from a word or words, extracts made from a word or words, made up from a series of characters (the gold made up from a series of characters (the gold

standard denominator)standard denominator)

Independently by two researchersIndependently by two researchers 10 rules developed iteratively10 rules developed iteratively

eg exclusive assignment « 1 extracteg exclusive assignment « 1 extract 1 theme » 1 theme »

CodingCoding

<R> <R> Despite major differences Despite major differences <O> <O> in blood pressure lowering, in blood pressure lowering, <R> <R> there were nothere were no<O> <O> outcome outcome <R> <R> differences differences <E> <E> between atenolol between atenolol <C> <C> and placebo in the four studies, and placebo in the four studies, <P> <P> comprising 6825 patients, comprising 6825 patients, <D> <D> who were followed up for a mean of who were followed up for a mean of

4·6 years 4·6 years

Should this be coded under C?

Quantitative AnalysisQuantitative Analysis

Characters Characters assigned to assigned to

PubMed Abstracts# %

EBM Synopses# %

P 5,450 17,0% 13,104 31,8%

E 7,354 22,9% 8,168 19,8%

C 3,426 10,7% 4,417 10,7%

O 6,649 20,7% 8,751 21,2%

D 1,058 3,3% 1,473 3,6%

R 8,010 25,0% 5,350 13,0%

Total 31947 41263

Quantitative AnalysisQuantitative Analysis

6 « PECODR » themes6 « PECODR » themes

40 documents40 documents73,315 characters73,315 characters

20 AbstractsCharacters # %

20 SYNOPSESCharacters # %

Agreement 27,360 85,4% 35,716 86,6%

Consensus 3,808 11,9% 4,873 11,8%

Arbitration 884 2,8% 674 1,6%

Disagreement 105 0,3% 0 0,0%

So the method of coding

seems to work

QuantitativeQuantitative AnalysisAnalysis1. 1. For each theme PECODRFor each theme PECODR

Identification (inductive) of Identification (inductive) of potential potential patterns or series of words patterns or series of words

Compare theme patterns Compare theme patterns comparison, compare, compared, comparing, comparison, compare, compared, comparing, etc,etc,

2. For each pattern2. For each patternOnly for those with a pattern frequency of Only for those with a pattern frequency of > 70% found in the themes> 70% found in the themes

Results: PECODR by Results: PECODR by abstracts abstracts & & synopsessynopses

docsdocs extractsextracts docsdocs extractsextracts

P P 1919 8989 2020 116116 E E 2020 163163 2020 180180C C 1818 9292 1919 120120 O O 2020 169169 2020 187187D D 1515 3636 1818 4545 RR 2020 210210 2020 187187

TotalTotal 759759 835835

PECODR exists in abstracts

RESULTSRESULTS

P = Patient-population-problemP = Patient-population-problem

No pattern: Example, diabeticNo pattern: Example, diabetic

E = Exposure-interventionE = Exposure-intervention

No pattern: Example, AmoxilNo pattern: Example, Amoxil

RESULTSRESULTSC = Comparison: C = Comparison: number of extracts assigned to C with C pattern and % of total C extracts found not assigned to C

C Patterns ABSTRACTSABSTRACTS

# extracts C# extracts C

(92)(92)

% C% C

SYNOPSESSYNOPSES

# extracts C# extracts C

(120)(120)

% C% C

Comparing 2 100% 4 50%

Compared 19 83% 20 86%

Placebo 47 83% 20 84%

Standard 3 80% 4 75%

Versus 2 71% 5 100%

Than 42 67% 18 81%

All were in C extracts

4 more occurrences of the word “comparing” were in parts of the text that were

not C extracts

RESULTSRESULTS

O Patterns ABSTRACTS ABSTRACTS (169)(169)

# extracts O# extracts O% O% O

SYNOPSES SYNOPSES (187)(187)

# extracts O# extracts O% C% C

End point 3 100% 0 NA

Mortality 35 85% 50 93%

Death 5 83% 8 89%

Incidence 12 75% 5 83%

Outcome 14 70% 34 76%

O = « Outcome »: O = « Outcome »: number of extracts assigned to O with O pattern and % of total O extracts found not assigned to O

RESULTSRESULTSR = Results: R = Results: number of extracts assigned to R with R pattern and % of total R extracts found not assigned to R

R patterns ABSTRACTSABSTRACTS

(210)(210)

# extracts R# extracts R

% R% R

SYNOPSESSYNOPSES

(187)(187)

# extracts R# extracts R

% R% R

Frequent 1 100% 0 NA

Correlated 1 100% 2 100%

Strongly 2 100% 0 NA

Closely 1 100% 0 NA

Cast doubt 1 100% 0 NA

RESULTSRESULTS challenge 1 100% 0 NA

superiority 1 100% 0 NA

replicate 1 100% 0 NA

chance 1 100% 0 NA

gradient 1 100% 0 NA

fewer 3 100% 7 88%

better 3 100% 5 71%

likely 1 100% 2 67%

decrease 3 100% 2 50%

differ 11 92% 24 83%

confidence interval 9 90% 0 NA

increase 8 89% 13 76%

significant 25 86% 5 83%

difference 6 86% 6 75%

odds ratio 5 83% 1 100%

occur 4 80% 1 50%

associated 7 78% 5 83%

greater 3 75% 3 100%

higher 5 71% 2 67%

ruling 0 NA 1 100%

highest 0 NA 1 100%

lowest 0 NA 1 100%

R R (more)

Significance

Results SummaryResults SummaryPresence of text or text patternsPresence of text or text patterns

PECODR elements are present in the PECODR elements are present in the majority of abstracts with the exception of majority of abstracts with the exception of Duration (75%)Duration (75%)

Patterns of PECODRPatterns of PECODR

P & E = Medical or Drug Terms (thesaurus)P & E = Medical or Drug Terms (thesaurus) The other element related extracts contain The other element related extracts contain some patternssome patterns

DISCUSSIONDISCUSSIONLimitsLimits

Pilot Study (N=40) Pilot Study (N=40) Exploratory study (rules developed Exploratory study (rules developed iteratively for text extraction/ qualitative iteratively for text extraction/ qualitative pattern identification)pattern identification)

Further Research Further Research Textual Analysis using clever software Textual Analysis using clever software with UdM to better identify patternswith UdM to better identify patternsLarger sample of papersLarger sample of papersDo GP’s want a PECODR search engine?Do GP’s want a PECODR search engine?

CONCLUSIONCONCLUSION1.1. Possible standard textual patterns (lexico-Possible standard textual patterns (lexico-

semantics) may help automatically determine semantics) may help automatically determine PECODR elementsPECODR elements

2.2. If this is possible we may be able to develop a If this is possible we may be able to develop a prototype of new technology for automatic prototype of new technology for automatic information retrieval (aspectual Search information retrieval (aspectual Search Engine) Engine)

3.3. To index the clinical information according to To index the clinical information according to needs' for the users (eg family practitioners)needs' for the users (eg family practitioners)

4.4. That will help people answer clinical That will help people answer clinical questions…maybequestions…maybe

Martin DawesMartin Dawes

DIABETIC Therapy 28,000

Prognosis 2,000

Harm 1,780

Aetiology 4000

Oral Hypoglycaemics 1,780

Heart attacks 2,300

Problem/Patient Select

ExposureSelect Outcome

Number under each heading represents articles

Cost Effectiveness 1,300

Diagnosis12,000

Insulin 12,000

Antihypertensive 6,000

Mortality 60 >3 years

n=20

Select Duration

Male/Female

Old/Adult/Young

1. NNT 25 4.6 years

2. NNT 34 4.2 years

3. N/A

4. NNT 54 1.8 years

5. NNT 35 N/A