cullen: why most published research is wrong
DESCRIPTION
Question everything! Louise cullen examines the minefield of published research and importance of reading around topics, not articles.TRANSCRIPT
“Why most published
research is wrong.”
Louise Cullen(Clinician researcher)
Disclosure Information
“It is everyone’s responsibility to find out how to
ask questions systematically, find answers from searching the
literature, critically appraise the literature
and apply the results to practice.”
Rinaldo Bellomo
“It is everyone’s responsibility to find out how to
ask questions systematically, find answers from searching the
literature, critically appraise the literature
and apply the results to practice.”
Rinaldo Bellomo
40 ingredients associate with cancer
Most single studies showed implausibly large effects.
The p value
The p value
Observed size of Effect
p=0.01
p=0.01
There is a 1% chance of results as extreme as these would occur when there is really no difference occurring in the experiment.
1000 hypotheses
Replication of studies
Replication of studies
Problems with the study itself.
Wrong question
Wrong Theory
Wrong population studied
2
ACS: Trial and community populations
Circulation. 115(19):2549-69, 2007 May 15.
n=2
Wrong design
• Greater the flexibility in– designs – definitions – outcomes– analytical modes
• Greater the flexibility in– designs – definitions – outcomes– analytical modes
• Hotter a scientific field with more teams involved.
Wrong Endpoints
Ad and high dose Ad
Ca++ in cardiac arrest
COX-2 inhibitors
Milrinone
Methodology
Statistical hypothesis inference testing
Problems with reporting
Interpretation
• “a little significance”
• “a definite trend is evident”
• “a clear tendency”
• “almost achieved significance”
• “a little significance”
• “a definite trend is evident”
• “a clear tendency”
• “almost achieved significance”
The data is practically
meaningless
• “In my experience”• “In case after case”• “In a series of cases”
• “It is generally believed that..”
• “A highly significant area for exploratory study”
• Once• Twice• Three times
• A couple of others think so too
• A totally useless topic in my underpowered study…….
Omitting facts deliberately
….!
Why?
Incentives
Pharma
Pharma
Why?
Ethical practice of researchers
Problems with publishing
Don’t believe in the review process
Journal publishing practices
• 2004 “original articles” in NEJM– 363 tested an established therapy– 146 (40%) reversed that practice– 138 (38%) reaffirmed it
What can you do about it?
Read more than the title!
Reporting Framework
CONSORT (http://bit.ly/14qUNEF)– Standards for reporting of trials
STARD– Standards for the Reporting of diagnostic accuracy
studies
Biases
Be sceptical!
Thank you
@louiseacullen