The Peer Review Process for Manuscripts

Gary Curhan, MD, ScD

Editor-in-Chief emeritus

Clinical Journal of the American Society of Nephrology (CJASN)

Disclosures

• UpToDate author and section editor

• Research funding: NIH

• Consultant: Allena, AstraZeneca, Decibel Therapeutics

• Spouse: Associate Editor Ear and Hearing

Overview

• Pre-submission considerations

• Manuscript preparation

• Submission

• Peer-review process

• Response to review and resubmission

“Don’t believe everything you read”

Types of Articles

• Original investigation

• Research letter

• Case reports

• Review articles

• Commentaries

• Editorials

• Special Features

• Supplements

• Meeting abstracts

Components of the Manuscript Submission Process

• Pre-submission

• Submission

• Decision

• Resubmission

Pre-Submission

• Starts with:

– Original hypothesis

– Study design

– Study execution

– Data analysis

• Manuscript preparation

• Internal review (co-authors)

– Iterative process

– Technical review

Manuscript Preparation

• The goal is to help readers understand:

– What was the research question?

– Why is this question important?

– How was it examined?

– What were the findings?

– What are the implications?

• Provide enough information for other investigators to reproduce the study

• Tailor for the target journal

What NOT to Do

• Errors in abstract

– Numbers transposed

• Typographical errors in abstract or text

• Grammatical errors in abstract or text

“The alternative route of administration may illicit a response…”

• Errors in tables, graphs or figures

Make a Good First Impression

Title

• Be sure it describes the study being presented in the manuscript

• Not declarative or focused on the conclusions

• Choose keywords from MeSH

– Medical Subject Headings in NLM

– Thesaurus for indexing in PubMed

– Aids retrieval and citations

Abstract

• First and possibly only impression

• Structured vs. unstructured

• Major findings

– Quantitative results

• Major implications

– Appropriate and relevant conclusions

Structured abstract• Background

In patients with chronic kidney disease (CKD), as in other populations, elevations in cardiac biomarker levels predict increased risk of cardiovascular events. We examined the value of troponin T (TnT) and N-terminal pro–brain natriuretic peptide (NT-pro-BNP) in assessing the risk of developing end-stage renal disease (ESRD) in diabetic patients with CKD.

• Study DesignProspective cohort study nested within a randomized clinical trial.

• Setting & ParticipantsPatients with type 2 diabetes, CKD (estimated glomerular filtration rate [eGFR], 20-60 mL/min/1.73 m2), and anemia enrolled in TREAT (Trial to Reduce Cardiovascular Events With Aranesp Therapy).

• PredictorsSerum levels of the cardiac biomarkers TnT and NT-pro-BNP.

AJKD, 2011

• Outcomes

Incidence of ESRD and the composite of death or ESRD.

• Measurements

We measured TnT and NT-pro-BNP in baseline serum samples from the first 1,000 patients enrolled in TREAT. The relationship of these cardiac biomarker levels to the development of ESRD and death or ESRD was analyzed in multivariable regression models.

• Results

Detectable TnT (≥0.01 ng/mL) was present in 45% of participants, and median NT-pro-BNP level was elevated at 605 pg/mL. Higher levels of both cardiac biomarkers were associated independently with higher rates of ESRD, as well as death or ESRD, and remained prognostically important after adjustment for eGFR, proteinuria, and other known predictors of CKD progression. The addition of cardiac biomarkers to a multivariable model for prediction of ESRD improved discrimination of those with and without an event by 16.9% (95% CI, 6.3%-27.4%).

• Limitations

Observational study in a clinical trial cohort; results require validation.

• Conclusions

In ambulatory patients with type 2 diabetes, anemia, and CKD, TnT and NT-pro-BNP levels frequently are elevated. These cardiac-derived biomarkers enhance prediction of ESRD beyond established risk factors. Measurement of TnT and NT-pro-BNP may improve the identification of patients with CKD who are likely to require renal replacement therapy, supporting a link between cardiac injury and the development of ESRD.

Unstructured Abstract

In most patients with hypertensive nephropathy and low glomerular filtration rate (GFR), the kidney function progressively declines despite the adequate control of the hypertension with angiotensin-converting enzyme inhibition. Previously we found that 2 years of oral sodium citrate slowed GFR decline in patients whose estimated GFR (eGFR) was very low (mean 33 ml/min). This treatment also slowed GFR decline in an animal model of surgically reduced nephron mass. Here, we tested if daily oral sodium bicarbonate slowed GFR decline in patients with hypertensive nephropathy with reduced but relatively preserved eGFR (mean 75 ml/min) in a 5-year, prospective, randomized, placebo-controlled, and blinded interventional study. Patients matched for age, ethnicity, albuminuria, and eGFR received daily placebo or equimolarsodium chloride or bicarbonate while maintaining antihypertensive regimens (including angiotensin-converting enzyme inhibition) aiming for their recommended blood pressure targets. After 5 years, the rate of eGFR decline, estimated using plasma cystatin C, was slower and eGFR was higher in patients given sodium bicarbonate than in those given placebo or sodium chloride. Thus, our study shows that in hypertensive nephropathy, daily sodium bicarbonate is an effective kidney protective adjunct to blood pressure control along with angiotensin-converting enzyme inhibition.

KI, 2010

Unstructured Abstract

In most patients with hypertensive nephropathy and low glomerular filtration rate (GFR), the kidney function progressively declines despite the adequate control of the hypertension with angiotensin-converting enzyme inhibition. Previously we found that 2 years of oral sodium citrate slowed GFR decline in patients whose estimated GFR (eGFR) was very low (mean 33 ml/min). This treatment also slowed GFR decline in an animal model of surgically reduced nephron mass. Here, we tested if daily oral sodium bicarbonate slowed GFR decline in patients with hypertensive nephropathy with reduced but relatively preserved eGFR (mean 75 ml/min) in a 5-year, prospective, randomized, placebo-controlled, and blinded interventional study. Patients matched for age, ethnicity, albuminuria, and eGFR received daily placebo or equimolarsodium chloride or bicarbonate while maintaining antihypertensive regimens (including angiotensin-converting enzyme inhibition) aiming for their recommended blood pressure targets. After 5 years, the rate of eGFR decline, estimated using plasma cystatin C, was slower and eGFR was higher in patients given sodium bicarbonate than in those given placebo or sodium chloride. Thus, our study shows that in hypertensive nephropathy, daily sodium bicarbonate is an effective kidney protective adjunct to blood pressure control along with angiotensin-converting enzyme inhibition.

KI, 2010

Introduction

• Justification

– Set up justification for current study

• Context

– Concise review of relevant existing literature

• Purpose

– What this study is attempting to address

• Plan

– Succinct statement of how this will be examined

Methods

• Clear and detailed

• Sufficient information for reader to assess methodologic quality

• If methods not adequately described, manuscript cannot be evaluated

• Reproducible

• Registration information for clinical trials

• Ethical Committee authorization

Results

• Thoughtful reporting of pertinent findings

• Tables and figures should be relevant and self-explanatory

– Descriptive title

– Easy to read

– Legend

Discussion

• Concise yet complete

• Integrate findings into the broader literature

• Include a paragraph on limitations

– But…

• Conclusions need to be supported by study design and findings

• The authors should help the reader understand the study and the implications

References

• Should support statements in text

• Readily available on Medline/PubMed (Main online libraries, catalogues, databases)

• What factors influence which reference is cited?

• How can a reader assess the appropriateness of a citation?– Access to abstract for each reference

• Meeting abstracts– Should not be cited

– Vast majority are never published as full articles

Components of the Peer-Review Process

• Pre-submission

• Submission

• Decision

• Resubmission

Selecting a Journal for Submission

• How to assess the ‘quality’ and appropriateness of a journal?

– Impact factor

– Past experience

– Style guidelines

– Circulation?

– Website?

Impact Factor

“In a given year, the impact factor of a journal is the average number of citations received per paper published in that journal during the two preceding years. For example, if a journal has an impact factor of 3 in 2008, then its papers published in 2006 and 2007 received 3 citations each on average in 2008.”

• Short-term assessment

• Average can be influenced by a few papers (good or bad)

• Authors tend to cite their own work

How to Influence the Impact Factor

• Review articles

• Self citation

• Publish likely high citation articles in January

• Negotiate which articles will not count in the denominator

• Probably many other ways

Other Factors

• Journal: Eigenfactor

– ‘total importance’ of a journal

– Weights origin of citations

• Individual: h-index

– Productivity and impact of author

Past Experience

• Personal preferences

• Previous submission experience

• But:

Journals do change

• Editor-in-Chief

• Editorial Team

• Editorial Board

• Policies

• Priorities

Journal Selection

• Aim high

– But be reasonable

• Scan the journal for similar types of studies

• Ask colleagues for advice

• Some journals will give a ‘curb-side’ opinion

• Other considerations

– Duration of review process

– Other restrictions (length, tables, etc)

– Animal vs human research focus

Instructions to Authors

• Read them

• They are not just suggestions

• Follow the format

• Word counts can be a bit flexible

– Include on title page

– These do not include abstract or references

– If excessive, shorten the manuscript

Components of the Peer-Review Process

• Pre-submission

• Submission

• Decision

• Resubmission

Peer Review

• The aim of peer review is two-fold:

(1) Evaluate scientific rigor

(2) Assess contribution to the literature

Review Process

• Initial assignment by Editor-in-Chief

• Initial decision

– Review

– Triage

• First impression is very important

• Selection of reviewers

– Who?

– How many?

– “preferred” vs “non-preferred”

Review Process

• What is expected of the reviewers?

• How long to wait for a review?

• Associate Editor’s recommendation

• Editorial Decision

– Minor revision

– Major revision

– Reject

Reviewer’s Responsibilities

• Structured review

– Helps focus the comments

– Encourages complete review

• Constructive suggestions

• Timely response

• Disclosure of COI

• Be aware of personal bias

Editorial Decisions

• Significance

• Design

• Relevance

• Generalizability

• Power (for null studies)

Major Concerns• Study hypothesis unclear

• Inadequate literature review

• Inappropriate citations

• Problematic study design– Inappropriate study population

– Suboptimal measurements

– Insufficient power

– Lack of generalizability

• Over-interpretation of results

• Poorly written manuscript

• Excessive length

Conflict of Interest

• Variety of potential conflicts

– Financial

– Intellectual

– Institutional

• Sources of potential conflicts

– Editorial team members

– Editorial board members

– Reviewers

– Authors

– Journals

• How can the reader assess potential conflicts?

COI--Intellectual

• Definition of hypercalciuria

“The classic definition of hypercalciuria, an upper normal limit of 200 mg/day, is based on a constant diet restricted in calcium, sodium, and animal protein.”

KI, 2011

COI--Intellectual

• “Defining hypercalciuria in nephrolithiasis”

• Summarized predominantly their previous studies

• Variety of definitions in their past studies

• Trying to justify outdated terminology

– “absorptive hypercalciuria Type I”

• Ignored huge existing literature

KI, 2011

COI--Intellectual

• How should these issues have been detected?

– Co-authors

– Reviewers

– Readers (i.e. letters to the editor)

• Why did it get published in this form?

– Reputation

– Reviewer inattention

• Will a letter to the editor make a difference?

COI--Intellectual

• Reviewer 1: “Reject”

• Reviewer 2: “Accept”

• How to resolve these differences of opinion?

Editorial Decision

• Not all rejection letters are the same

• Should you challenge a decision?

– Factual errors in the letter from Editor and comments from Reviewers

• Should you write to the Editor expressing your disappointment?

Careful with Reply All

“Let us celebrate the longest and hardest to reason with review notable for a complete lack of substantive gravitas”

“I think the questions are easy to answer; some are pretty silly!”

“The perils of multi-authorship are on full display in the series of e-mails around this last letter from the JOURNAL to me.

I trust that you accept the comments of the corresponding author, rather than those of some of my co-authors. I apologize for them sending these comments to you. I consider the comments very unprofessional, and have told them so.”

Careful with Reply All

“Great news! Sorry that the journal is so anal.”

Careful with Criticism

• “I would expect open-minded reviewers to look at my statements, particularly when they simply do not agree and to give us their own opinion instead of a critical scientific judgment.

• I do believe that we do not need to be offensive each time we disagree, especially in a scientific debate. I truly hope that for the future manuscripts I will have a less prejudiced and less bigoted open- evaluation.”

• “Thank you very much for your reply. However I do not agree with what you say!!.

• If you say that there is no need to be an expert in an area to be able to deal with, it's like you say that you epidemiologist are able to evaluate papers on clinical nephrology or on dialysis.”

“I'm an editor in many journals I cited "Hepatoma research, World Journal of Nephrology, World Journal of Transplantation Clinical Reviews and Opinions, Journal of Infectious Diseases and Therapeutics, and others….” and I know what I’m saying.”

Components of the Peer-Review Process

• Pre-submission

• Submission

• Decision

• Resubmission

Responding to Reviewers’ Comments

• Point-by-point response is ideal

• Make it as easy as possible for the editor and reviewers

• You don’t have to agree and don’t have to make all the changes

– But you need to justify in your response

• Take the same care and steps as if it were the initial submission

Evaluation After Publication

• Letters to the Editor

– Limited window

– Not a discussion

• Journal Club

– Substantial time and effort to prepare

– Thoughtful discussion

– But the authors are not involved

• Electronic discussions

Proliferation of Journals

• Invited submissions to new journals

– Be cautious

• Many are for-profit

• May not be listed on Pubmed

• Invitation to join editorial boards of new journals

– Be cautious

Summary

• Peer-review process is not perfect

• Manuscripts almost always benefit from the review process

• First impressions are very important

• There is always more than one journal for your manuscript

• Don’t take it personally