shelley hurwitz medicres world congress 2014
DESCRIPTION
Biostatistics and Ethics Shelley Hurwitz, PhD Brigham and Women’s Hospital Harvard Medical School Fellow, American Statistical Association Advisory Board on Ethics, International Statistical InstituteTRANSCRIPT
Biostatistics and Ethics
Shelley Hurwitz, PhD
Brigham and Women’s Hospital Harvard Medical School
Fellow, American Statistical Association
Advisory Board on Ethics, International Statistical Institute
MedicReS World Congress on Good Medical Research
New York October 17, 2014
1
Outline
• Reputation of statistics
• Statisticians respond
• Ethical guidelines
• Reproducibility
2
• “Figures will not lie, but liars will figure.”
• “There are three kinds of lies: Lies, damned lies, and statistics"
• “Facts are stubborn things, but statistics are pliable” • “How to lie with statistics”
• "Statistics are no substitute for judgment.”
Reputation of Statistics
3
• “Figures will not lie, but liars will figure.” (1889)
• “There are three kinds of lies: Lies, damned lies, and statistics"
• “Facts are stubborn things, but statistics are pliable”
• “How to lie with statistics”
• "Statistics are no substitute for judgment.”
Reputation of Statistics
4
• “Figures will not lie, but liars will figure.”
• “There are three kinds of lies: Lies, damned lies, and statistics” (1906)
• “Facts are stubborn things, but statistics are pliable”
• “How to lie with statistics”
• "Statistics are no substitute for judgment.”
Reputation of Statistics
5
• “Figures will not lie, but liars will figure.”
• “There are three kinds of lies: Lies, damned lies, and statistics"
• “Facts are stubborn things, but statistics are pliable”
• “How to lie with statistics”
• "Statistics are no substitute for judgment.”
Reputation of Statistics
6
• “Figures will not lie, but liars will figure.”
• “There are three kinds of lies: Lies, damned lies, and statistics"
• “Facts are stubborn things, but statistics are pliable”
• “How to lie with statistics” (1954)
• "Statistics are no substitute for judgment.”
Reputation of Statistics
7
• “Figures will not lie, but liars will figure.”
• “There are three kinds of lies: Lies, damned lies, and statistics"
• “Facts are stubborn things, but statistics are pliable”
• “How to lie with statistics”
• "Statistics are no substitute for judgment.”
Reputation of Statistics
8
Outline
• Reputation of statistics
• Statisticians respond
• Ethical guidelines
• Reproducibility
9
Series of articles in British Medical Journal, 1980-1
“Statistics and Ethics in Medical Research” by Douglas G. Altman
• Misuse of statistics is unethical • Study design • How large a sample? • Collecting and screening data • Analysing data • Presentation of results • Interpreting results • Improving the quality of statistics in medical journals
10
John C Bailar III "Science, Statistics, and Deception"
Annals of Internal Medicine 1986;104:259-60.
• Summarizes important role of statistics in scientific research
• Practices that can distort scientific inferences:
– Failure to deal honestly with readers about non-random error (bias) – Post hoc hypothesis – Inappropriate statistical tests and other statistical procedures – Fragmentation of reports – Low statistical power – Suppressing, trimming, or “adjusting” data – Undisclosed repetition of “unsatisfactory” experiments – Selective reporting of findings
11
David L. DeMets “Statistics and Ethics in Medical Research”
Science and Engineering Ethics. 1999; 5(1) 97-117.
• Statistical methods are powerful tools but can easily be
misused. • Pressures to get “right” results can be difficult, especially for
junior statisticians. • Community must be vigilant; misuse is hard to detect. • Just as no one may know if you disobey a stop sign late at
night, rarely will anyone be able to double check in detail our statistical analyses and design.
• Ultimately, proper use of statistics in medical research rests with the integrity of the individual.
12
John S. Gardenier and David B. Resnik “The Misuse of Statistics: Concepts, Tools, and a Research Agenda”
Accountability in Research: Policies and Quality Assurance. 2002 (9) 65-74.
• Misconduct
• Incompetence
• Negligence
• Honest error
• Deliberate deception
– Pressures to publish, produce results, or obtain grants
– Career ambitions or aspirations
– Conflicts of interest and economic motives
– Inadequate supervision, education, or training
13
Outline
• Reputation of statistics
• Statisticians respond
• Guidelines for Ethical Statistical Practice
• Reproducibility
14
The Statistical Society of Australia Inc. Code of Conduct (1998)
American Statistical Association
Ethical Guidelines for Statistical Practice (1999)
Statistical Society of Canada Code of Ethical Statistical Practice (2004)
International Statistical Institute
Declaration on Professional Ethics (2010)
Royal Statistical Society Code of Conduct (2014)
15
American Statistical Association Ethical Guidelines for Statistical Practice
1949: Commission on Statistical Standards to develop
a code of ethical practices 1954: 2 surveys 1956: Interest was low. Idea was tabled. 1977: Interest increased, new committee formed. 1980: Interim Code of Conduct published, requesting
comments. There was extensive input. 1983: Final version published. 1996: The Committee on Professional Ethics started
drafting a revision. 1998: Draft published. Invited comments. Open
session at annual meeting. 1999: Current version approved by Board of Directors.
16
Domains of Ethical Responsibility in ASA’s and other societies’ codes of conduct
• Society
• Public interest
• The profession
• Funders
• Clients
• Employers
• Research subjects
• Research team colleagues
• Other statisticians
17
15 years later New ethical concerns for statisticians
• Massive amounts of data
• Complex data structure
• High dimensionality
• De-identification
• Security & encryption
• Increasingly multidisciplinary studies
18
Outline
• Reputation of statistics
• Statisticians respond
• Ethical guidelines
• Reproducibility
19
“How Science Goes Wrong” The Economist, 10/19/2013
• Half of published research cannot be replicated.
• 2000-10 roughly 80,000 patients took part in clinical trials based on research that was later retracted because of mistakes or improprieties
• Lost opportunity, lost money
• Obligation to publish or perish
• Honest confusion “sweet signal of a genuine discovery” and “freak of the statistical noise”
• Spurious correlations in newspapers, morning TV
• Failure to report failure leads to more researchers exploring blind alleys already explored.
20
“Fraud in Medical Research: An International Survey of Biostatisticians”
Ranstam, Buyse, George, Evans, Geller, Scherrer, Lesaffre, Murray, Edler, Hutton, Colton, and Lachenbruch,
for the ISCB Subcommittee on Fraud Controlled Clinical Trials 21:415–427 (2000)
• 51% of biostatisticians know of at least one fraudulent project.
• 31% have been involved in fraudulent projects.
• 13% have been directly asked to support fraud.
– Fabrication or falsification of data
– Suppression or selective deletion of data
– Deceptive design or analysis
– Deceptive reporting of results
21
The Duke Saga
2006: Nature Medicine “Genomic signatures to guide the use of chemotherapeutics”
2009: The Annals of Applied Statistics “Deriving chemosensitivity from cell lines: Forensic
bioinformatics and reproducible research in high- throughput biology”
2011: Nature Medicine “Retraction: Genomic signatures to guide the use of
chemotherapeutics”
22
Nature Medicine 2006, Vol. 12, 1294 – 1300
“Genomic Signatures to Guide the use of Chemotherapeutics”
By
Anil Potti, Holly K Dressman, Andrea Bild, Richard F Riedel, Gina Chan, Robyn Sayer, Janiel Cragun, Hope Cottrill, Michael J Kelley,
Rebecca Petersen, David Harpole, Jeffrey Marks, Andrew Berchuck, Geoffrey S Ginsburg, Phillip Febbo, Johnathan
Lancaster & Joseph R Nevins
23
Corrigenda: Genomic signatures to guide the use of chemotherapeutics
By Anil Potti, Holly K Dressman, Andrea Bild, Richard F Riedel,
Gina Chan, Robyn Sayer, Janiel Cragun, Hope Cottrill, Michael J Kelley, Rebecca Petersen, David Harpole, Jeffrey Marks, Andrew Berchuck, Geoffrey S Ginsburg, Phillip Febbo,
Johnathan Lancaster & Joseph R Nevins
• Nature Medicine 13, 1388 (2007)
• Nature Medicine 14, 889 (2008)
24
The Annals of Applied Statistics
2009, Vol. 3, 1309 – 1334
“Deriving Chemosensitivity From Cell Lines: Forensic Bioinformatics and Reproducible
Research in High-throughput Biology”
By Keith A. Baggerly
and Kevin R. Coombes
25
Nature Medicine 2011, Vol.17, 135
“Retraction: Genomic Signatures to Guide the use of Chemotherapeutics”
By
Anil Potti, Holly K Dressman, Andrea Bild, Richard F Riedel, Gina Chan, Robyn Sayer, Janiel Cragun, Hope Cottrill, Michael J Kelley,
Rebecca Petersen, David Harpole, Jeffrey Marks, Andrew Berchuck, Geoffrey S Ginsburg, Phillip Febbo, Johnathan
Lancaster & Joseph R Nevins
26
Uninteresting Work
• Replication of positive results is not interesting work. • Negative studies are not that interesting. • Open access to all data • Clinical trial registration • Journals should set aside room for
– Replication studies – Negative studies – Expanded methods sections
• Funders should set aside funding for this uninteresting work.
27
“Lies, Damned Lies, and Medical Science”
by
David H. Freedman October 4, 2010
The Atlantic
28
“Why Most Published Research Findings are False”
by
Professor John P. A. Ioannidis
2005, PLOS Medicine
1,138,461 on-line views
1,413 citations 10,464 shares on Facebook and Twitter
29
What reduces the likelihood a scientific finding is true?
1. Studies conducted in the scientific field are small. 2. Effect sizes in the field are small. 3. Number of studies is high, pre-tested relationships are not
selected for study. 4. Highly flexible designs, definitions, outcomes, analyses. 5. Greater financial and other interests and prejudices in a
scientific field. 6. Greater “hot-ness” of the field, with more teams involved.
Ioannidis, JPA (2005) “Why Most Published Research Findings are False”
30
Biostatistics and Ethics Conclusion
Updating Wright, 1889: It is our duty, as practical statisticians, to prevent the investigator from figuring;
in other words, to prevent him or her from perverting the truth, in the interest of some theory he or she wishes to establish.
DeMets, 1999: Ultimately proper use of statistical methods must rest with the integrity of the
individual statistician.
• Integrity • Transparency • Reproducibility • Accountability • Guidelines
31