Skip to main content

A proposal for reducing the effect of one of many causes of publication bias

Abstract

In order to avoid publication bias, all trials should be registered at initiation and their results made easily accessible. However, some trial results are more difficult to publish than others. This report describes one such trial and highlights the need for a way of making results of trials widely available even if not presented in the traditional format. Until such time as it is required by law both to register all trials and enter their final results into the database, a lack of resources will mean that some trial results are never published. The scale of the problem of non-publication is unknown and for valid trial results any form of publication is better than none. Therefore it is essential that a quick and easy way is available to act as a safety net to catch trial results that would otherwise be lost.

Peer Review reports

Background

The problem of publication bias in scientific research has been recognized for many years, and was drawn to the attention of the medical community in the 1980s [1]. Since that time many studies have shown that lack, or delay, of publication is related to the statistical significance of the results [2–7]. Awareness of this has been increased by the rise in systematic reviews, and methods to assess the degree of publication bias in these have been developed [8]. These methods provide a rating of the quality of the evidence but do not help in determining a corrected effect estimate.

Some measures have already been put in place to address the problem, such as the requirement that trials must have been registered in a recognized public trials registry at initiation as a condition of consideration for publication in a journal that is a member of the International Committee of Medical Journal Editors [9]. Even for trials which have been registered, publication rates are low [10].

We discuss here one example that illustrates that there may be additional ways of reducing this problem.

Main text

The Medical Research Council funded a randomized trial in polycythaemia (Figure 1), which recruited between 1974 and 1993, before the days of trial registries. Due to the low mortality rate from this disease, with a median survival of 13 years, follow-up continued until 2003. During this time personnel working on the trial changed, including the departure of the statistician. This trial then came under the remit of the remaining statistician in the Clinical Trial Service Unit responsible for leukemia trials, who reran analyses and wrote a skeleton paper. The introduction, methods and results sections of the paper were drafted, but the discussion section was incomplete and the clinical lead then retired. The computer system on which the analysis programs ran has now been superseded. Although programs and data have been archived it would take a considerable amount of work to do any further analyses.

Figure 1
figure 1

Randomization structure showing the number of patients allocated to each of the three treatments (Venesection, P32 and Busulphan) and the method of treatment allocation in each phase of the trial. V n1 , P n2 and B n3 indicate that n 1 patients were allocated to Venesection, n 2 to P32 and n 3 to Busulphan.

The paper remains without an abstract, discussion section or references and there are no resources available for any further work to be done. The trial was completed and we strongly believe that the results should be made publicly available. However, we have not found any journal that would accept the paper in this format (see Additional file 1: Medical Research Council randomized Polycythaemia trial results: long term outcome after busulphan, radioactive phosphorous or venesection).

Discussion

Much recent discussion has focused on the issue of competing, particularly financial, interests, and the role of the pharmaceutical industry. This has led to a new US law requiring both the registration of trials and the entry of final results into a database [11], and the suggestion that legislation should be expanded internationally [12]. However, there are other reasons behind non-publication, including a lack of resources, as in the example presented here.

Conclusions

The scale of the non-publication of trials is unknown, but providing a medium for reporting unpublished trials, together with any results that are available from them, would provide further information on this subject.

References

  1. Begg CB, Berlin JA: Publication bias: a problem in interpreting medical data. J Roy Stat Soc A. 1988, 151: 445-463.

    Google Scholar 

  2. Dickersin K: The existence of publication bias and risk factors for its occurrence. JAMA. 1990, 263: 1385-1389. 10.1001/jama.1990.03440100097014.

    Article  CAS  PubMed  Google Scholar 

  3. Dickersin K, Min YI, Meinert CL: Factors influencing publication of research results. Follow up of applications submitted to two institutional review boards. JAMA. 1992, 267: 374-378. 10.1001/jama.1992.03480030052036.

    Article  CAS  PubMed  Google Scholar 

  4. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR: Publication bias in clinical research. Lancet. 1991, 337: 867-872. 10.1016/0140-6736(91)90201-Y.

    Article  CAS  PubMed  Google Scholar 

  5. Dwan K, Altman DG, Arnalz JA, Bloom J, Chan A-W, Cronin E, Decullier E, Easterbrook PJ, von Elm E, Gamble C, Ghersi D, Ioannidis JPA, Simes J, Williamson PR: Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008, 3: e3081-10.1371/journal.pone.0003081.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Burrett JA, Lunn D: Analysis of characteristics of randomized clinical trials in leukaemia that are associated with how results are reported. JEBM. 2011, 4: 217-224.

    PubMed  Google Scholar 

  7. Paulson K, Saeed M, Mills J, Cuvelier GDE, Kumar R, Raymond C, Robinson T, Szwajcer D, Wall D, Seftel MD: Publication bias is present in blood and marrow transplantation: an analysis of abstracts at an international meeting. Blood. 2011, 118: 6698-6701. 10.1182/blood-2011-08-367466.

    Article  CAS  PubMed  Google Scholar 

  8. Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, Alonso-Coello P, Djulbegovic B, Atkins D, Falck-Ytter Y, Williams JW, Meerpohl J, Norris SL, Akl EA, Schünemann HJ: GRADE guidelines 5: rating the quality of evidence - publication bias. J Clin Epidemiol. 2011, 64 (12): 1277-82. 10.1016/j.jclinepi.2011.01.011. Dec, Epub 2011 Jul 30

    Article  PubMed  Google Scholar 

  9. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJPM, Schroeder TV, Sox HC, Van Der Weyden MB: Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Lancet. 2004, 364: 911-912. 10.1016/S0140-6736(04)17034-7.

    Article  PubMed  Google Scholar 

  10. Ross JS, Mulvey GK, Nissen SE, Krumholz HM: Trial publication after registration in ClinicalTrials.gov: a cross-sectional analysis. PLoS Med. 2009, 6 (9): e1000144-10.1371/journal.pmed.1000144.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Tuma RS: New law may be having some effect on publication bias. JNCI. 2010, 102: 290-292. 10.1093/jnci/djq051.

    Article  PubMed  Google Scholar 

  12. Bian Z-X, Wu T-X: Legislation for trial registration and data transparency. Trials. 2010, 11: 64-66. 10.1186/1745-6215-11-64.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Thank you to Iain Chalmers and Doug Altman for encouragement to report this trial, and for the suggestion to write this commentary.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Julie A Burrett.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

SR wrote the first draft of the commentary, JAB revised it, and both authors read and approved the final manuscript.

Electronic supplementary material

13063_2012_1234_MOESM1_ESM.doc

Additional file 1: Medical Research Council randomized Polycythaemia trial results: long term outcome after busulphan, radioactive phosphorous or venesection.(DOC 176 KB)

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Richards, S.M., Burrett, J.A. A proposal for reducing the effect of one of many causes of publication bias. Trials 14, 41 (2013). https://doi.org/10.1186/1745-6215-14-41

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1745-6215-14-41

Keywords