How to smell a rat: Examples of scientific and regulatory failure in medicine

Dr Aubrey Blumsohn

Presentation at RadstatsBritish Library, London

24 February 2012

Nature of science – and how we define scientific fraud

Data – and access to data

Regulators (watchdogs)

Individual responsibility in science

Science

Data

Regulators Individual responsibility

History

• Einstein– Insanity: doing the same thing over and over

again and expecting different results.

What is Science?

• UK Science Council (2009)– Spent a year working out a new definition of science.

– Claim as the first "official definition of science" ever published.

•“Science is the pursuit of knowledge and understanding of the natural and social world following a systematic methodology based on evidence”

What is Science? – Mertonian Norms

• Communalism– Everyone can see what’s going on

• Universalism – Doing of it not restricted to special people

• Disinterestedness– An arms length attitude.

• Skepticism (Organized Skepticism) - – Claims exposed to critical scrutiny. Awkward questions.

Merton, Robert K. (1973), "The Normative Structure of Science",

Science demands awkward people

It takes all sorts of people

Example 1: “Mr Nobody”and Journal of the American Medical Association (JAMA)

Letter #1 to JAMA

Letter #2 To BMJ

Scientific deception- and its mis-definition

The Classics

William Summerlin (Sloan Kettering, 1984) - faked transplantation experiments in white mice by blackening patches of their skin with a pen

Vijay Soman (Yale, 1980) a diabetologist - wrote 12 papers based on invented data. His co-author Felig had to resign

The Classics

Robert Slutsky (Radiologist, University of California). Published 137 papers between 1978 and 1985 -- one every 20 days. Many faked.

John Darsee (Cardiologist, Harvard, 1981). Observed falsifying data. His boss, Eugene Braunwald ignored the problem. Later investigations discovered more than a 100 fraudulent papers

In these odd cases, the stories, as told, usually minimise the involvement of senior colleagues, University administrators, Hospitals and regulatory bodies who hid the misconduct before it was eventually exposed

The impression

• The choice of cases implicitly defines what we mean by scientific fraud/misconduct– Fraud is rare– It involves a particular type of activity

(painting a mouse)– Usually involves an aberrant (male) acting alone

(or duping otherwise honourable collaborators)

“Problems of scientific misconduct are rare - the product of psychopathic behaviour’ originating in temporarily deranged minds” President of the National Academy of Sciences (at Congressional Hearings)

And the solution

• Is to create yet more procedures and guidance

• And for Universities to create robust, secret, legalistic procedures to investigate these rare rogues

• Only one problem. As Arnold Schwarzenegger once said: “it's bullshit—all of it”

The reality

• Not rare• Scientific misconduct

often not an individual activity – but a group activity (or involves group collusion)

Group-think

• Hannah Arendt (1965) – "rule by nobody".

• No single individual feels responsible for what is ultimately done.

• Indeed many people have gone to great lengths to make sure that this exact state of affairs achieved– Legislators– Ethics Committees– Trade bodies– Regulators

• System designers responsible for fraud?

The reality

• The offender is often not a lone white male psychopath

• Corporate fraud

The reality

• Corporation acting at the University interface

The reality

• or a journal editor

The reality

• or a regulator ….

• or government itself

The mechanisms also differ

• The mechanism of misconduct is also usually not direct fabrication of findings but something far more nuanced

Defining pornography

• "hard-core pornography" was hard to define, but "I know it when I see it“

Potter Stewart(Associate Justice of the US Supreme Court)

Misconduct• Misconduct is nothing to do with the correctness of

conclusion– To err is human, and scientific!!!

• It has to do with behaviours• Behaviours which could disturb the scientific record• Not binary• We need to get away from ludicrous discussion about

restrictive definitions and start discussing behaviours– Openly

Example 2: The fearful junior

• Approached by a Junior Doctor• In order to gain her pathology registration she had to submit her

dissertation• She could not write up her dissertation or prepare the main area

of work for publication because her supervisor had made this impossible – for years…..

• She could only present the secondary aspects• The work involved looking at physiological changes in healthy

volunteers randomised to atorvastatin or placebo• The study showed that atorvastatin induced a diabetes like state• Her supervisor was funded by the manufacturer of atorvastatin

Example 3: Does Seroquel cause diabetes and weight gain

• But there was a cloud on the horizon. The F.D.A. was beginning to look into the issue of Diabetes in patients on Atypical Antipsychotics and wrote to Astrazeneca

• Then somebody noticed the obvious that they had more than just 18 months of data in these subjects

Example 4 – Famovir and Herpes

• Famvir is used for treatment of herpes zoster

• There was a nine year delay before partial publication of findings which had adverse implications for the sponsor

What are data?

Data is not the same as representation of

that data

Getting a hamburger when what you need is the original cow

New York Times May 31, 2006

Example 5: Vioxx APPROVE Trial

Look at the language

Merck & Co. acknowledged that it misidentified a statistical method

Merck has contended that the study shows The company said in a statement

yesterday

Merck research chief Peter Kim said the company stood by its analysis

Merck has insisted that its 18-month argument holds up

while outside scientists have cast doubt on the company's interpretation.

Example 6: Vioxx AVANTAGE Trial

• On Oct. 21, 1999 a 73-year-old female participant dies in a clinical trial

• The trial was Merck’s Advantage trial of Vioxx

• She died a few minutes after calling her son to tell him she felt short of breath

• The Advantage trial was a very short (12-week) clinical trial

• eight people taking Vioxx suffered heart attacks or sudden cardiac death in the trial (compared with only one not on the drug)

• The difference was statistically significant, but Merck never showed the data that way

• In the published study, the “author” (Dr. Lisse, University of Arizona), the 8 deaths became` 5, a difference that did not reach statistical significance.

• The paper never mentioned the other cardiac deaths of patients taking Vioxx, including the 73-year-old woman.

• Dr. Lisse said that while he was listed as the paper's first author, Merck actually wrote the report.

"Merck designed the trial, paid for the trial, ran the trial," Dr. Lisse said. "Merck came to me after the study was completed and said, 'We want your help to work on the paper.' The initial paper was written at Merck, and then it was sent to me for editing."

• Results from the trials were also supposed to be blinded when coded

• During coding, Dr. Eliav Barr, a Merck employee judged that the woman “had probably died of a heart attack."

• Dr. Reicin a Merck executive immediately shot back :

"I think this should be called an unknown cause of death." A few hours later, she wrote, "I would prefer unknown cause of death so we don't raise concerns."

• In November 2001, an F.D.A. reviewer who had examined all the company's data concluded in a report to other agency officials that Merck had cheated. But the F.D.A. never told anyone either.

• Dr. Lisse, the study "author" said he had never heard of the case of the woman who died, until told of it by the New York Times.

Example 7: Vioxx - Fake Journals

• Merck paid Elsevier to create a number of fake scientific journals

• This came to light 2009 in the context of a civil suit filed by Graeme Peterson, who suffered a heart attack in 2003 while on Vioxx

• These were designed in such a way that the average medical reader would “mistake the publication for a genuine peer reviewed medical journal”

• The articles supported Merck Drugs

• One of the members of Australasian Journal of Bone and Joint Medicine's "Honorary Editorial Board," Peter Brooks, a rheumatologist said:

he didn't recall who asked him to serve on the board, …. "You get involved in a whole bunch of things at this level".

• Elsevier issued at Press release and an apology– We “published a series of sponsored article compilation

publications, on behalf of pharmaceutical clients, that were made to look like journals and lacked the proper disclosures. This was an unacceptable practice, and we regret that it took place.”

• But what of the academics involved, and their institutions?

REGULATORS AND SYSTEMS

• Medical Writers (Ghost writers)

• Medical Journals

• MHRA (UK Drug regulator)

• Medical Ethics system

• UK Research Integrity Office

• General Medical Council

The illusion of due diligence

• "We're looking for the illusion of due diligence, Mr. Pope. Two criminal acts successfully prosecuted -- it gives us that illusion."

Medical Writers(and Ghostwriters)

“When the time comes for publication, the medical writer writes thepublication in consultation with the named author(s). This usually beginswith discussions between the medical writer and authors(s) about the mainfeatures of the paper, after which the medical writer writes the firstdraft. The draft is then revised (sometimes many times) after review bythe author(s). Thus although the authors do not actually write the paper,they do have control over its content, which is far more important.Naturally, they have access to the report of the study and its datatables, so they can verify that the paper is accurate. They often don'thave access to the raw data, but it is worth repeating that raw data areof very little use when writing a paper. Results of a study needsummarising and analysing before they mean anything, and it is thosesummaries and analyses that are important for proper interpretation of astudy, not the raw data themselves. “

Adam Jacobs – European Medical Writers AssociationBMJ 21 Nov 2004

Journals

Motto – Which? Magazine

No advertising, no bias, no hidden agenda. Just expert advice from anindependent source

Motto – BMJ

“We protect our reputation by careful balance of editorial and advertising, which means your messages will always stand out.”(http://group.bmj.com/group/advertising)

The Medicines and Healthcare Regulatory

Agency (MHRA)

MHRA

• Has absolutely nothing to say on the problem of ghostwriting

• Has nothing to say on the problem of denial of access to data (to authors or the scientific community in general)

• Have no apparent concerns nor any apparent remit to deal with events at Sheffield University

• Even suggested that it might be “illegal” for authors to obtain the randomisation codes in a study they are authoring because commercial companies “own” those codes.

Risedronate

• FOI: – Did the MHRA receive and scrutinise in any

way the raw data relating to the three key clinical trials which led to approval of Risedronate

– Has the MHRA got the randomisation codes

Risedronate

• FOI response: – We cannot tell you because part of the response is

commercially confidential

• Reply: The questions required answers of “yes” or “no”. Which parts of the Yes or No are commercially confidential

• Eventual FOI response: The MHRA did not scrutinise or retain any actual raw data before licensing Risedronate. They simply accepted what the company told them.

The MHRA (HOC SC Report 2005)

• "practices have developed which act against the public interest."

• The MHRA relies on company data presented as reports, in its decision whether or not to licence a drug. Raw data is very rarely analysed.

The MHRA (HOC SC Report 2005)

• “The MHRA Chairman suggested that trust underpinned the stance of the MHRA towards the companies they regulate. We inferred that this extended to the routine acceptance of companies’ summaries of the results of tests on their drugs as true reflections of the raw data on which they were based.”

• “Evidence that regulatory agencies are acquiescing in biased later publication of the information they hold.”

The MHRA (HOC SC Report 2005)

• “Lack of transparency has played a major part in allowing failings to continue.”

• “100% of the funding for the MHRA comes from the industry it regulates”

• "In view of the failings of the MHRA, we recommend a fundamental review of the organisation."

The General Medical Council

• Body that “regulates” doctors• A science-based profession• Might have thought they would take

scientific integrity seriously• Long history of a very lax approach to

scientific fraud– Particularly when it involves very senior

academics

Dr David Hurlstone• Sheffield Teaching Hospitals NHS Trust

• Bowel cancer research

• 2006 BUPA Foundation Research Award

• 2007 Medical Futures Innovation Oncology & Cancer Care Award

• GMC - dishonesty in three research studies published in 2007 & 2008 (not retracted)

• Suspended from Medical Register for 3 months

• On return to work retained Clinical Excellence Award

What did he do?• In three publications

– Submitted authors’ consent forms to journal with false signatures of collaborators

– Studied patients without ethical approval– Created consent forms for patient records in retrospect– Stated falsely that patients had been randomised– Made false statements about microscopy having been carried

out– Stated falsely that two blinded specialists had exampled

samples – they did not exist– Stated falsely that an independent nurse participated in the

study and that the study endoscopist was blinded• And so on…..

GMC and Richard Eastell

• 3 publications intended by Procter and Gamble (1992-1995)

• Authors actively denied randomisation codes by sponsor

• Eastell prepared to go along regardless• Prepared to induce a colleague to do the same• False conclusions in all manuscripts• Research Dean of a Medical School

R&D Director of Sheffield NHS Trust

How to get out of this?

• Didn’t write the false statementThe “medical writer” wrote that as well

• I don’t need raw data as a scientist because I would not know how to analyse it (and presumably neither would my coauthors). It is very hard to analyse data.

• The statement was in 2002. – In 2002 I had a watershed moment when I realised this kind of

activity was inappropriate. – I would never have done it again after 2002, and I will never do

this sort of thing again.• Legal manoeuvres to avoid having the GMC panel

examine any information subsequent to this 2002 paper and repeated events

The GMC response

• 2002 statement– Data are raw data– Data are having the data– It was false– It was misleading– It was negligent

• Epiphany in 2002• 2003/2004/2005???• No punishment - “it would interfere

with his ability to get research funding”!

• This type of behaviour has no impact on patient care (according to the GMC)

Dr Guirong Jiang

• Radiologist

• Working as a postdoctoral fellow at the Universityof Sheffield

• One of the most experienced radiologists in the world in terms of diagnosis and definition of vertebral fracture

• Several complaints about Richard Eastell 2007-2010– Including his wish to author in the absence of data

– Apparent secret contracts involving Eastell, Sheffield University and Commercial Companies preventing publication

– Jiang not permitted to see contracts apparently preventing her right to publish findings she believed to be true based on studies in which she was involved