fate of manuscripts declined by the british journal of surgery

Original article

Fate of manuscripts declined by the British Journal of Surgery

B. P. L. Wijnhoven1 and C. H. C. Dejong2

1Department of Surgery, Erasmus University Medical Centre, Rotterdam, and 2Department of Surgery, Maastricht University Medical Centre andNutrition and Toxicology Research Institute Maastricht, Maastricht, The NetherlandsCorrespondence to: Dr B. P. L. Wijnhoven, Department of Surgery, Erasmus University Medical Centre, PO Box 2040, 3000 CA Rotterdam, TheNetherlands (e-mail: [email protected])

Background: The fate of papers submitted and subsequently rejected by the British Journal of Surgery(BJS) is currently unknown. The present study was designed to investigate whether, when and wherethese papers are published following rejection.Methods: All rejected manuscripts in the year 2006 were identified from the Manuscript Centralelectronic database. Between December 2008 and February 2009, a PubMed search was conductedspanning the period 2006-2009 using the corresponding author’s last name and initials to identifywhether and when manuscripts had been published elsewhere.Results: From the 926 manuscripts rejected by BJS, 609 (65·8 per cent) were published in 198 differentjournals with a mean(s.d.) time lapse of 13·8(6·5) months. Some 165 manuscripts (27·1 per cent) werepublished in general surgical journals, 250 (41·1 per cent) in subspecialty surgical journals and 194(31·9 per cent) in non-surgical journals. The mean(s.d.) impact factor of the journals was 2·0(1·1). Only14 manuscripts (2·3 per cent) were published in journals with a higher impact factor than that of BJS.Conclusion: Rejection of a manuscript by BJS does not preclude publication, but rejected manuscriptsare published more often in surgical subspecialty journals and journals with a lower impact factor,although the occasional exception exists.

Paper accepted 22 September 2009Published online 22 January 2010 in Wiley InterScience (www.bjs.co.uk). DOI: 10.1002/bjs.6880

Introduction

The British Journal of Surgery (BJS) receives more than1100 article submissions each year. The acceptance rate ofsubmitted manuscripts ranges between 15 and 18 per cent.The editors of BJS accept manuscripts for publicationbased on various factors, including originality of theresearch, clinical importance and usefulness of the findings,methodological quality, and supposed interest of thereadership of the journal. Selection of manuscripts forpublication is a difficult and complex process. Manuscriptsundergo a peer review process that is supposed to beable to identify their strengths and weaknesses. Althoughrecommendations made by reviewers have considerableinfluence on the fate of papers submitted to journals,agreement between reviewers appears to be little greaterthan would be expected by chance alone1,2. Other criticshave also argued that the process of peer and editorialreview can be subjective, biased, and fails to preventpublication of methodologically weak studies3. All of thissuggests that manuscripts rejected by BJS are not always

of inferior quality and may well be published in otherhigh-quality journals.

Hitherto it has not been known what happens tomanuscripts rejected by BJS. In particular, informationis lacking with respect to how many rejected articles aresubsequently published elsewhere. Equally, no informationis available regarding the type of journal in which suchrejected papers may be published and how these journalscompare with BJS in terms of impact factor. The authorsbelieve that an answer to these questions might helpresearchers when choosing a journal for resubmissionof a manuscript. Furthermore, this could provide moreinformation on the quality of the BJS review process. Aretrospective cohort study was therefore undertaken totrack the publication fate of rejected BJS manuscripts.

Methods

All manuscripts submitted to BJS between 1 January and31 December 2006 were identified from the ManuscriptCentral electronic database. This list contained manuscript

Copyright 2010 British Journal of Surgery Society Ltd British Journal of Surgery 2010; 97: 450–454Published by John Wiley & Sons Ltd

Fate of declined manuscripts 451

number, title, corresponding author (name and ini-tials), date of submission, handling editor and rejectionnotes. Abstracts of the manuscripts were also available.Manuscripts that were rejected by the editors in 2006 wereincluded in the study; book reviews and letters (corre-spondence section) were excluded. To identify whetherrejected manuscripts had been published in another jour-nal, a PubMed search (http://www.ncbi.nlm.nih.gov) wasperformed between December 2008 and February 2009,spanning the time period 2006–2009 and using the cor-responding author’s last name and initials. Obviously, thesearch was limited to manuscripts published from theyear of submission onwards. The title and abstract of theretrieved manuscripts for each author were compared withthe title and abstract in the BJS 2006 submission database.In a few cases, where the title and the authors had changedconsiderably, some judgement had to be used to deter-mine whether the published article was the one that hadoriginally been submitted to BJS.

The proportion of declined manuscripts publishedelsewhere was calculated by dividing the number ofrejected manuscripts subsequently published in otherjournals by the total number of rejected BJS manuscripts.The journals in which the rejected manuscripts werepublished were categorized into general surgical journals,subspecialty surgical and ‘surgical interest’ journals,and other (non-surgical) journals. The 2006 impactfactor of the journals in which the rejected BJSmanuscripts were published was retrieved from the Webof Science of the Institute for Scientific Information(ISI) (http://www.isiwebofknowledge.com) and comparedwith the BJS impact factor for 2006. The time intervalbetween date of submission of the manuscript to BJS and(e-)publication elsewhere was calculated.

Papers that are rejected by BJS and subsequentlypublished elsewhere may be of good quality and citedfrequently. It is of interest to compare the number ofcitations to manuscripts rejected by BJS with manuscriptsaccepted by BJS. On 31 March 2009, the number ofcitations to 20 papers rejected by BJS and subsequentlyaccepted by other journals with high impact factors wasscored by using data from the ISI Web of Science, andcompared with the number of citations from 20 randomlyselected manuscripts accepted for publication in BJS in2006. The time interval between publication and websearch was calculated.

Statistical analysis

A database of all rejected BJS papers was generated inMicrosoft Excel 2004 for Mac (Microsoft Corporation,

Redmond, Washington, USA). Mean(s.d.) and median(range) values with standard deviation or range werecalculated. Analysis of categorical data was done with thePearson’s χ2 test. P ≤ 0·050 was considered statisticallysignificant.

Results

Proportion of manuscripts published elsewhere

A total of 926 submitted manuscripts could be identifiedthat were subsequently rejected in 2006 by BJS. The major-ity of rejected manuscripts were original articles (n = 780,84·2 per cent), followed by reviews (n = 52, 5·6 per cent),randomized controlled trials (n = 37, 4·0 per cent), system-atic reviews (n = 24, 2·6 per cent), leading articles (n = 20,2·2 per cent) and meta-analyses (n = 13, 1·4 per cent).Some 496 manuscripts (53·6 per cent) were rejected afterpeer review, whereas 430 (46·4 per cent) were rejected afterearly editorial manuscript screening for appropriatenesswithout being sent out for review.

The PubMed search revealed that, as of March 2009, 609(65·8 per cent) of these 926 manuscripts were published inother journals and 317 (34·2 per cent) were not published.The mean(s.d.) time lapse between submission to BJSand (e-)publication elsewhere was 13·8(6·5) months. Therewas no relationship between the type of manuscript andlikelihood of publication elsewhere. A higher proportionof manuscripts rejected by BJS after peer review werepublished elsewhere compared with manuscripts rejectedafter editorial screening alone: 71·0 per cent (352 of 496)versus 59·8 per cent (257 of 430) (P<0·001, χ2 test).

Type of journal and impact factor

The 609 manuscripts submitted to BJS, then rejected andsubsequently accepted for publication by other journals,were published in 198 different journals. Some 165manuscripts (27·1 per cent) were published in generalsurgical journals, 250 (41·1 per cent) in specialty surgicaljournals and 194 (31·9 per cent) in other journals. Table 1shows the distribution of the 165 papers published ingeneral surgical journals, and Table 2 the 250 paperspublished in subspecialty surgical and ‘surgical interest’journals.

No impact factor was listed in the ISI Web of Knowledgefor 51 (25·8 per cent) of 198 journals. The mean(s.d.)impact factor for the remaining 147 journals was 2·0(1·1).Only 14 manuscripts (2·3 per cent) were published injournals with an impact factor higher than that of BJS(Table 3).

Copyright 2010 British Journal of Surgery Society Ltd www.bjs.co.uk British Journal of Surgery 2010; 97: 450–454Published by John Wiley & Sons Ltd

452 B. P. L. Wijnhoven and C. H. C. Dejong

Table 1 General surgical journals in which manuscripts rejectedby the British Journal of Surgery were published

JournalNo. of

manuscriptsImpactfactor*

World Journal of Surgery 46 (27·9) 1·765American Journal of Surgery 19 (11·5) 2·101Annals of the Royal College of

Surgeons of England15 (9·1) 0·720

ANZ Journal of Surgery 15 (9·1) 0·881Langenbeck’s Archives of Surgery 14 (8·5) 1·496Archives of Surgery 9 (5·5) 3·058Surgery 9 (5·5) 2·977Journal of the American College of

Surgeons7 (4·2) 2·813

Surgeon 6 (3·6) NASurgery Today 6 (3·6) 0·698Acta Chirurgica Belgica 3 (1·8) 0·348American Surgeon 2 (1·2) 2·101Annals of Surgery 1 (0·6) 7·678Other† 13 (7·9) –

Total 165 (100) –

Values in parentheses are percentages. *Impact factor 2006 derived fromthe Web of Knowledge of the Institute of Scientific Information(http://www.isiwebofknowledge.com). †Other journals in which no morethan one manuscript was published. NA, not applicable (no impact factorlisted for 2006).

Table 2 Manuscripts rejected by the British Journal of Surgery andpublished in subspecialty surgical and ‘surgical interest’ journals

JournalNo. of

manuscriptsImpactfactor*

Annals of Surgical Oncology 23 (9·2) 3·329European Journal of Surgical

Oncology22 (8·8) NA

Surgical Endoscopy 22 (8·8) 1·969Colorectal Disease 21 (8·4) NAEuropean Journal of Vascular and

Endovascular Surgery21 (8·4) 2·156

International Journal of ColorectalDisease

18 (7·2) 2·006

Journal of Surgical Oncology 16 (6·4) 2·183Diseases of the Colon and Rectum 13 (5·2) 2·442Digestive Surgery 12 (4·8) 1·397Hepatogastroenterology 11 (4·4) 0·756Journal of Surgical Research 10 (4·0) 2·038HPB 7 (2·8) NAJournal of Laparoendoscopic &

Advanced Surgical Techniques6 (2·4) 0·718

Hernia 6 (2·4) NAJournal of Vascular Surgery 5 (2·0) 3·311Other† 37 (14·8) –

Total 250 (100) –

Values in parentheses are percentages. *Impact factor 2006 derived fromthe Web of Knowledge of the Institute of Scientific Information(http://www.isiwebofknowledge.com). †Other journals in which no morethan five manuscripts were published. NA, not applicable (no impactfactor listed for 2006).

Table 3 Journals with a high impact factor in which 14manuscripts rejected by the British Journal of Surgery werepublished

JournalType of

manuscriptImpactfactor*

Gut 1 original‡ 9·002Annals of Surgery 1 original† 7·678Critical Care Medicine 1 original‡ 6·599Clinical Infectious Diseases 1 original† 6·186Journal of Thrombosis and

Haemostasis1 RCT†, 1 review‡ 5·138

Journal of Nuclear Medicine 1 original‡ 4·986Psychoneuroendocrinology 1 original† 4·850Breast Cancer Research and

Treatment3 original†‡‡ 4·671

British Journal of Cancer 2 original†‡, 1systematicreview‡

4·459

*Impact factor 2006 derived from the Web of Knowledge of the Instituteof Scientific Information (http://www.isiwebofknowledge.com). RCT,randomized controlled trial. †Manuscript rejected after editorial reviewwithout external review; ‡manuscript rejected after external peer review.

Number of citations of rejected manuscripts

The median number of citations for 20 manuscriptspublished in journals with a high impact factor (between3·900 and 9·000) as of March 2009 was 3 (range 0–22). Themedian time elapsed since publication of these manuscriptswas 22 (range 1–30) months. The median number ofcitations for 20 papers accepted by BJS in 2006 was 6(range 1–24), although the time elapsed since publicationwas longer at 24 (range 20–31) months.

Discussion

This study investigated the fate of manuscripts rejectedby BJS. Most (65·8 per cent) were eventually publishedjust over a year (mean 13·8 months) after rejection. Theproportion published elsewhere is somewhat higher thanthat for other journals that have reviewed the outcome ofsubmissions declined by their editorial team. These studieshave reported rates between 41 and 58 per cent4–8. Nosimilar studies have been undertaken for general surgicaljournals.

The number of manuscripts accepted elsewhere isprobably an underestimate, because PubMed does notcover all sources. Only about a quarter of all medicalpublications are listed in PubMed5. Furthermore, thetime between rejection by BJS and the date the PubMedsearch was performed may not have allowed sufficienttime for some rejected manuscripts to reach the stage ofsubsequent publication. However, most articles rejected by


Fate of declined manuscripts 453

the Annals of Internal Medicine were eventually publishedwithin 2·5 years (mean latency 552 days), so it is probablethat those manuscripts in the present study not publishedafter 38 months are inactive. The editorial turnaround timeof BJS (interval between submission of the manuscript andfirst decision) is, on average, 22 days, which helps theauthor to resubmit quickly to another journal. Finally,manuscripts may evolve in terms of author order or title,making the paper difficult to identify through searches.This may also lead to an underestimation of the proportionof rejected manuscripts subsequently published elsewhere.

Early editorial screening of manuscripts for appropriate-ness for publication is a BJS policy. In a randomized trialby Johnston and colleagues9, this method was found to beappropriate as it decreased the time between manuscriptsubmission and publication decision, and reduced the bur-den on reviewers while having a minimal impact on thequality of accepted manuscripts. The manuscripts that wererejected solely after editorial review, that is, without beingsent for peer review, had a small chance of being pub-lished elsewhere. This probably reflects their poor quality.It supports the current BJS policy of editorial screening.

Only 14 manuscripts (2·3 per cent) were published injournals with a higher impact factor than that of BJS (4·304at the time of the study). Eight of these 14 papers werepublished in journals with an impact factor only marginallyhigher (range 4·459–4·986). This is lower than the 7·5 and10 per cent rates of publication in higher-impact journalsreported for other journals6,10. Most authors probably feelthat they should first attempt publication in a journalwith a relatively high impact factor. Upon rejection, themanuscript is then usually sent to a journal with a lowerimpact factor6,7,11,12. This is likely to be because authorsassume that such journals have higher acceptance rates.The mean impact factor of the journals that publishedthe 609 manuscripts rejected by BJS was 2·0, significantlylower than the BJS impact factor of 4·304 (currently 4·921).

Most manuscripts were published in surgical specialtyjournals. Perhaps many papers that are rejected by ageneral surgical journal such as BJS meet this fate becausethey are judged to be too specialized. The subsequentshift to a subspecialty journal may therefore be desirableand rational. The data support previous observations thatmanuscripts rejected by general medical journals are alsopublished more often in journals that serve a smallerreadership or research community and are cited lessfrequently7,13.

It is difficult to draw firm conclusions about thepreference of authors to publish their work in a particularjournal, although the data in Tables 1 and 2 give someinsight. Among the 165 papers accepted for publication

in general surgical journals, most were published in USA-based general surgical journals such as the World Journalof Surgery and the American Journal of Surgery. Thereseems to be no clear preference of authors for a particularsubspecialty surgical and surgical interest journal. Mostof these journals are gastrointestinal or surgical oncologyoriented. This probably reflects the fact that the number ofvascular and trauma papers submitted to BJS is relativelysmall.

The reasons for rejection of the 14 manuscripts thatwere published in journals with a higher impact factorwere reviewed. It was felt by the handling editors thatsome papers dealt with topics that were beyond the scopeof BJS and/or the topic was supposed not to be of interestto a typical BJS reader. Inferior methodological qualitywas another feature. This has been studied by Lee andco-workers14, who demonstrated that manuscripts of highmethodological quality, such as randomized controlledtrials and those with larger sample sizes, are more likelyto be published. Finally, novelty and originality is deemedto be important by BJS editors, and others15. Six of the14 manuscripts were rejected after editorial review withoutexternal review, which is roughly the same proportionas for all papers submitted to BJS. Further researchinto the factors that relate to publication or rejectionof manuscripts might help both authors and editors toimprove the scientific quality of journals.

A journal’s impact factor is often regarded as associatedwith the scientific quality and impact of the paperspublished. However, scientific impact is a multidimensionalconstruct that cannot be measured adequately by any singleindicator such as impact factor16. To estimate the scientificimpact of papers rejected by BJS, the number of citationsfrom 20 papers were retrieved and compared with thatof 20 papers accepted by BJS in 2006. Papers rejectedby BJS were cited less frequently than those accepted byBJS, despite publication in journals with a relatively highimpact factor (3·900–9·000). It is important to realize thata high citation count for an article is not necessarily a signof quality or approval of the results or interpretation ofa study, as citations may be critical of, or contradict, anarticle16,17.

Authors of manuscripts that are rejected by BJS mayfind reason for hope in the finding that rejected articleshad a good chance of being published in other peerreview journals. BJS encourages authors to incorporatesuggestions made by reviewers and editors into theirmanuscript. This may improve the quality of papers and,even if a paper is rejected, this process may add value interms of future publication of rejected papers in differentjournals.


454 B. P. L. Wijnhoven and C. H. C. Dejong

It would be interesting now to evaluate the trajectoryof those papers that were published in BJS. Howmany of them were first submissions? How many hadpreviously been rejected by direct competitors of BJS?Such information would allow the editors to assess therelative prestige of surgical journals. This informationmight be useful not only for the editors but also for thereaders and potential authors.

Acknowledgements

The authors acknowledge Mrs Harriet MacLehose, MrGavin Stewart and Mrs Bryony Urquhart from the BJSeditorial office for their help with retrieving the data andcompleting the database. B.P.L.W. received a BJS EditorAssistant Bursary in 2008 and expenses to permit travel tothe monthly editorial meetings. The authors declare noother conflict of interest.

References

1 Ernst E, Resch KL. Reviewer bias: a blinded experimentalstudy. J Lab Clin Med 1994; 124: 178–182.

2 Rothwell PM, Martyn CN. Reproducibility of peer review inclinical neuroscience. Is agreement between reviewers anygreater than would be expected by chance alone? Brain 2000;123: 1964–1969.

3 Kassirer JP, Campion EW. Peer review. Crude andunderstudied, but indispensable. JAMA 1994; 272:96–97.

4 Armstrong AW, Idriss SZ, Kimball AB, Bernhard JD. Fate ofmanuscripts declined by the Journal of the American Academyof Dermatology. J Am Acad Dermatol 2008; 58:632–635.

5 Liesegang TJ, Shaikh M, Crook JE. The outcome ofmanuscripts submitted to the American Journal ofOphthalmology between 2002 and 2003. Am J Ophthalmol2007; 143: 551–560.

6 Nemery B. What happens to the manuscripts that have notbeen accepted for publication in Occupational andEnvironmental Medicine? Occup Environ Med 2001; 58:604–607.

7 Opthof T, Furstner F, van Geer M, Coronel R. Regrets orno regrets? No regrets! The fate of rejected manuscripts.Cardiovasc Res 2000; 45: 255–258.

8 Silberzweig JE, Khorsandi AS. Outcomes of rejected Journalof Vascular and Interventional Radiology manuscripts. J VascInterv Radiol 2008; 19: 1620–1623.

9 Johnston SC, Lowenstein DH, Ferriero DM, Messing RO,Oksenberg JR, Hauser SL. Early editorial manuscriptscreening versus obligate peer review: a randomized trial. AnnNeurol 2007; 61: A10–A12.

10 McDonald RJ, Cloft HJ, Kallmes DF. Fate of manuscriptspreviously rejected by the American Journal of Neuroradiology:a follow-up analysis. AJNR Am J Neuroradiol 2009; 30:253–256.

11 Hall SA, Wilcox AJ. The fate of epidemiologic manuscripts:a study of papers submitted to Epidemiology. Epidemiology2007; 18: 262–265.

12 Ray J, Berkwits M, Davidoff F. The fate of manuscriptsrejected by a general medical journal. Am J Med 2000; 109:131–135.

13 Bollen J, Van de Sompel H, Hagberg A, Chute R. A principalcomponent analysis of 39 scientific impact measures. PLoSOne 2009; 4: e6022.

14 Lee KP, Boyd EA, Holroyd-Leduc JM, Bacchetti P,Bero LA. Predictors of publication: characteristics ofsubmitted manuscripts associated with acceptance at majorbiomedical journals. Med J Aust 2006; 184: 621–626.

15 Gupta P, Kaur G, Sharma B, Shah D, Choudhury P. What issubmitted and what gets accepted in Indian Pediatrics: analysisof submissions, review process, decision making, and criteriafor rejection. Indian Pediatr 2006; 43: 479–489.

16 Anon. Citation data: the wrong impact? Nat Neurosci 1998; 1:641–642.

17 Ioannidis JP. Contradicted and initially stronger effects inhighly cited clinical research. JAMA 2005; 294:218–228.


fate of manuscripts declined by the british journal of surgery

Documents