Salvador Peiró Fundación para el Fomento de la Investigación Sanitaria y Biomédica de la Comunidad Valenciana (FISABIO) Red de Investigación en Servicios de Salud en Enfermedades Crónicas (REDISSEC).

… in his very well argued reasoning about the ”hijacking of the EBM" (by the pharmaceutical and biomedical industry, by medical "eminences”, by “eminent” researchers) and about the low quality of most of the “science” currently published. Although the relationship between EBM and the industry is complex, several factors (including EBM's own success) tend to transform it into a symbiotic relationship (I will sometimes refer to biomedical industry as the “symbiont” and the EBM as the “host”). Because of the regulatory requirements for authorization, most drugs have RTCs “evidencing” higher (at least in terms of surrogate endpoints) or non-inferior efficacy (in a more or less tiny range, but always with p>0.05). The symbiont, largely thanks to the “host”, multiply the impact of these RTCs through meta-analysis, reviews, CPGs … and “evidence” becomes the main promotional mechanism of a drug.

Increasingly, only the large pharmaceutical firms have the resources (and the know-how) to finance the trials (and mega-trials) necessary to detect the desired effects (sometimes irrelevant, but with p-values <0.05). Clinicians and patients only have the "evidence" that industry has funded, and the firms that develop “evidences” studies have a competitive advantage over other firms, and alternative therapeutic options (e.g. exercise). We have (and we will have) more (and more) “evidence”, but also a more scotomic and skimmed perspective (more scotomic the greater the impact of evidence on prescription). The pharmaceutical industry (and its contracted CROs) do RTCs well: the randomization sequence is perfect, the endpoints have been validated, they score better than independent RCTs in the quality scales and their results are published faster and in “better” journals.

Results of these RTCs will be disseminated in each congress (international, national, regional, local), conference, symposium (satellite or planetary), workshop, seminar, meeting, convention and similar by reputable experts. A reprint of the RTC (original) will arrive -delivered by hand- to each potential prescriber. These renowned experts will be those who "participate" (most of those with little real contribution, beyond putting their name in the cover page) in mega-trials, meta-analysis, observational studies, economic evaluations and CPGs, generating a vicious circle of funding (private and public), "prestige" and debatable clinical recommendations.

…, burden of disease studies will be published months before, so that we can get an idea of the importance of the problem (previously forgotten or “orphan”) now that someone has found an expensive (but necessary) solution. … Cost-effectivenness evidence will appear -in each country- at the exact moment when national authorities are discussing price and inclusion in the public portfolio. Invariably they will make “evident” how foolish we would be to lose the savings that would produce to pay half a million euros for a month of a treatment (of uncertain effectiveness). mmm … I hate economic evaluations. The authors use economic models developed in their headquarters and, changing to local prices, they make an article (with the same model in different countries) in 2 days. I take 2 weeks to evaluate it, largely trying to decipher the validity of the values used in the model (years ago they refuse to provide the model to the editor, but not actually)

… that, paraphrasing Ioannidis, many of these RCTs asked for the wrong question, and they answered it through the wrong analysis with the wrong surrogate endpoint and the wrong criterion for success (the legendary p value <0.05 for null hypothesis rejection) . … that the original purpose of EBM (to use the best evidence from RTCs and meta-analysis of RTCs to improve patient care) has been transformed into a big promotional strategy for the prescription of drugs (or diagnostic tests, procedures and -more or less disparate- biomarkers).

The pharmaceutical industry has also fueled EBM ... and there are many "evidenciologists" who (legitimately) have found work in the industry or in scientific consulting firms (who work for the industry).

… Ioannidis does not "blame” the biomedical industry for this situation but the science and technology system. He argues that the science-technology system -including scientific publication as an essential component- is perfectly designed to produce "poor science". And -although painful for those of us who are part of this system- “evidence based” it is also difficult not to agree with him (again).

Scarcity (and disincentivation) of validation and replication studies (“reproducibility crisis”) Use of p-values as synonymous of causal association: Is the Brexit a late effect of the Bovine Spongiform Encephalopathy? A Creutzfeldt–Jakob symptom? The massive production of “spurious, false-positive, or confounded putative risk factors”

Studies adding little or nothing to patient care, poorly designed, poorly performed or simply fraudulent: cherry picking (selective reporting of favorable outcomes), p-hacking (construct the design according significant p-values), HARKing (hypothesizing after results are known), …

Why Most Published Research Findings Are False? Why Most Discovered True Associations Are Inflated? Why Current Publication Practices May Distort Science? Are Medical Conferences Useful? And for Whom? and …

How to survive the medical misinformation mess?

Protect us from poor-quality medical research. An Epidemic of False Claims. Competition and conflicts of interest distort too many medical findings Clinical trials: what a waste. Trials that are unregistered, unfinished, unpublished, unreachable, or simply irrelevant Head-to-head randomized trials are mostly industry sponsored and almost always favor the industry sponsor Waste, Leaks, and Failures in the Biomarker Pipeline The scientists who publish a paper every five days

Register all before starting (protocols, sample size, hypothesis) and register (and justify any change). Transparency, openness, sharing. More replication studies. Publish negative and replication studies. More public funding for science Ban funding that has overt conflicts of interest (Journals should not accept research with CoI)

Use a lottery to decide which grant applications to fund. Shifting the funds from senior people to younger researchers. Create better infrastructure for enabling transparency, openness and sharing. Make transparency a prerequisite for funding. Universities and research institutes could preferentially hire, promote or tenure those who are champions of transparency. Funding agencies must pay for replication studies. Scientists’ advancement should be based not only on their discoveries but also on their replication track record. Universities should try to shift the aging distribution of their faculty by hiring more young investigators.

More funds should be earmarked for new fields and fields that are high risk. Researchers should be encouraged to switch field, whereas currently they are incentivized to focus in one area. We should reward scientists for high-quality work, reproducibility and social value rather than for securing funding. Excellent research can be done with little to no funding other than protected time. Institutions should provide this time and respect scientists who can do great work without wasting tons of money. Fund scientists rather than projects. Judge success on the total agenda, not a single experiment or result. Invest in studying how to get the best science and how to choose and reward the best scientists

Research projects become more expensive (in terms of researcher time). Not problematic for those who can finance the additional costs (industry), but disruptive for the small and medium groups of independent researchers. Reduction in the number of projects and associated funding (an important aspect for groups that work primarily with competitive funding grants ... not so much for industry-financed research). Reduction of the "quality" of the research group in the current metrics (certainly erroneous, but in force) of impact factor and financing capacity: Loss of “prestige”, no attraction of doctoral students and young researchers, loss of competitive (Not problematic for the industry because it hires researchers already trained by higher salaries).

The 4 mega-trials of the International Study of Infarct Survival (ISIS 1 to 4) were carried out at the end of the 80s and the beginning of the 90s of the last century and had some characteristics that today would be extraordinarily rare:

The trials corresponded to an initiative of the researchers themselves (not from a pharmaceutical company), The Case Report Form was limited to a single page, No center was monitored or audited, No event was "adjudicated", Neither centers nor investigators received payments for their participation in the trials. The follow-up was very long (eg, 10 years in ISIS-2)

More than 100,000 people in several hundred hospitals were randomized in these trials that, at an incredibly low cost to our current standards, offered “evidence” useful for many patients with myocardial infarction ( including information about a non-patented drug such as aspirin). These trials were simple (how I miss the 2x2 factorial designs!) and relied in randomization and “hard” outcomes (relevant, but easy to identify, as death). Heterogeneity between centers (inclusion criteria, ancillary treatments, …) was not so important because randomization within each center and the inclusion of a large number of patients facilitated that potential variability was distributed more or less homogeneously between the active and control groups. A breach of protocol in some cases may not severely affect the results of a study that was carried out in hundreds of hospitals.

With the current European regulation (2004 RCT directive) these studies would not meet -not by far- quality standards.

The high workload of the current protocols would require remuneration of the participating professionals (In addition, they are so accustomed to being paid for participation in industry (competitive) RCTs that it would be difficult to involve them in independent altruistic unpaid studies). Their centers would also want to recover the extra costs of the (mostly unnecessary) tests carried out and professional time We also would need reference labs, monitors, auditors and several committees Obviously we need a professional research organization to handle this complexity. And it is very doubtful that we would find funding, including the public one, to assess an out-of-patent drug (except for repositioning with a new price).

Therefore, it is unlikely that the ISIS trials (or similar trials) would have been carried out today.

Probably this is what has been happening since the increase in RCTs quality control and associated costs (in time and money) produced the expulsion of independent researchers from the clinical trials arena. All of these regulatory built-in barriers (primarily designed to improve the quality of studies and to control some inadequate industry behaviors) restricted substantially the scientific capacity to address relevant questions independently of the pharmaceutical industry. And, although I am sure that the quality of the RTCs (measured by the scales of quality) has improved remarkably (regardless the low value of most of them), I am not sure of the value of what we have lost. But I am sure that, at least in the Spanish context, higher formal exigency will result in lesser independent researchers (and more researchers working for scientific consultings and pharmaceutical firms).

I suspect that many of the actions proposed to improve the quality of the science will expel independent researchers from other fields of clinical research. I also suspect that smaller groups, from smaller research centers, will be more affected than large universities. More researchers with clinical dedication than "professional researchers". More Europe than the United States. More Spain than Europe. I think that these “policies” should be evaluated. But, nevertheless, I think it is a road that needs to be traveled. I do not see how a scientist (or a doctor) could defend low-quality science, not transparent or not useless for patients, even though their impact factor "suffer" with these policies.

Maybe scientific and academic scientific policies (more public funding, funding researchers instead of projects, …) could be developed to ”smooth" the potential impact of a more strict process research on independent research, but according to our experience with research policies in Spain, it is highly improbable. In fact, the contrary is highly probable. Therefore …

peiro_bor@gva.es

La mayor parte de los proyectos de nuestro grupo de investigación han sido financiados por ayudas competitivas públicas y, por acuerdos institucionales con firmas farmacéuticas y tecnológicas o directamente por la fundación FISABIO (dependiente de la Generalitat Valenciana). Los patrocinadores nunca han tenido ningún papel en el diseño de los estudios, la adquisición de datos, el análisis o su interpretación. Tampoco han tenido acceso a las fuentes de datos y nunca influyeron en la decisión de publicación. Los proyectos desarrollados desde 2013 han recibido fondos de la Red de Investigación en Servicios de Salud en Enfermedades Crónicas (REDISSEC; proyectos RD12/0001/0005 y RD16/0001/0011 del Instituto de Salud Carlos III, cofinanciados con Fondos FEDER).