Skip to main content

Prospective reporting of statistical analysis plans for randomised controlled trials

Peer Review reports

In 2017, JAMA published a statistical analysis plan (SAP) guidance document for randomised clinical trials (RCTs). This guidance is part of the EQUATOR Network of reporting resources and includes a checklist of minimum items for reporting details of statistical analysis of RCTs [1]. While the clinical trial protocol should describe the principal features of the statistical analysis, a separate detailed SAP containing sufficient information to support replication by an independent statistician may be needed [2,3,4].

Given the influence of statistical decisions on trial conclusions, well-documented statistical conduct is essential for transparency and reproducibility of the research. Additionally, pre-specification is important to reduce the occurrence of, and facilitate the detection of, bias particularly in relation to selective analysis and reporting [5, 6]. While guidance exists on the content of protocols [2] and final reports for clinical trials [7], both of which require at a minimum a summary of the statistical analyses, there was until recently no guidance on SAP content. Consequently, there is marked variation in the level of detail provided to allow full replication of statistical analysis. The development of guidance for the content of SAPs aims to reduce this variation and to promote full and transparent reporting of pre-specified analyses [1].

Trials therefore encourages the submission for publication of SAPs, which are in line with the guidance document [1], in any of the following formats:

  1. (i)

    A section directly integrated in the protocol being submitted for publication (for uncomplicated RCTs this might be sufficient);

  2. (ii)

    An appendix within the protocol being submitted for publication;

  3. (iii)

    An addendum submitted subsequent to the published protocol paper in Trials as a newly peer-reviewed update and which subsequent to publication would be treated much like an appendix to the protocol;

  4. (iv)

    A separate stand-alone, fully published article, with its own author list and DOI, referencing an already-published protocol (in Trials or elsewhere).

These options underscore the importance of the publication of SAPs but also ensure that the reporting of SAPs as separate publications is not made mandatory. For many RCTs, the statistical analysis plan is likely to be uncomplicated and feasibly be defined in full at the time of writing the protocol. For more complex trials, such as those with adaptive designs or those using new methodologies, the development of the SAP might take longer than that of the protocol. Trials is actively encouraging researchers to publish their SAPs in the format that fits the complexity of their analysis plan. Whatever option is selected, all authors are encouraged to use the SAP reporting guideline [1]. Both authors and reviewers are therefore tasked with ensuring both reporting according to the SAP reporting guideline [1] as well as the SPIRIT guidance for the protocol as a whole [2].

Different approaches to peer review are anticipated depending on the style of submission and consist of the following:

  1. (i)

    If the SAP is submitted as part of the protocol, or an appendix to the protocol, peer review of the SAP will be undertaken in conjunction with review of the protocol following current Trials guidance for protocols which includes checking against the SAP reporting standard [8].

  2. (ii)

    If the SAP is submitted as an addendum to an already published protocol in Trials, authors will be expected to verify and demonstrate reporting against the SAP reporting guideline and the submission will undergo review as deemed appropriate by the handling editor and which might consist of review by the editor only.

  3. (iii)

    If the SAP is submitted as a separate publication, the submission will undergo peer review (for those that contain novel or more complicated statistical analysis proposals this would include statistical review) and the SAP would be checked against the SAP reporting standard.

By allowing a number of approaches, Trials encourages proportionate review. High-quality peer review has to be balanced against the availability of the academic community to serve as reviewers in times of increasing demands. Thus, peer review might be treated somewhat like a scarce resource. Mandating that every trial must have a published protocol and a separate published SAP may well be beyond the timely reviewing capacity of the trial’s community and so should be reserved for those trials for which the level of detail required in the SAP cannot be incorporated into the protocol. Authors should be mindful that these approaches do not distract from relevant legislation and guidelines on when SAPs should be competed.

Trials would thus like to encourage the early publication of SAPs and the use of the SAP guidance and checklist for peer reviewing SAPs put forward for publication. Full details on the submission process for the different types of submissions, along with examples, are available [8]. Regardless, we encourage journals to include the protocol and SAP as online addenda to each peer reviewed results paper they publish [9]. SAPs, like protocols, are living documents: SAPs may need to be updated, perhaps in response to new methods or emerging external evidence. In these instances, amendments can be made by (i) submitting an amendment as an update to either the protocol paper or SAP paper, or (ii) documenting changes to the SAP as part of the published final report (likely to be appropriate for minor changes), as per CONSORT guidance for changes which occurred during the trial [10]. Amendments should also be recorded on the relevant clinical trials registration site and with other relevant bodies as appropriate. Authors are requested to ensure clarity when publishing the SAP within a protocol as to whether it the preliminary or final version.

These different approaches to publication not only reflect differences in nature of complexity of SAPs but also promote opportunities for statisticians to take lead authorship positions. The approach endorsed by Trials will allow appropriate reflection of the level of academic credit that should be accredited to statisticians for their role in planning of randomised trials. Trials also encourages that funders support requests for publication costs associated with this publication.

References

  1. 1.

    Gamble C, Krishan A, Stocken D, Lewis S, Juszczak E, Doré C, Williamson PR, Altman DG, Montgomery A, Lim P, Berlin J, Senn S, Day S, Barbachano Y, Loder E. Guidelines for the content of statistical analysis plans in clinical trials. JAMA. 2017;318(23):2337–43. https://doi.org/10.1001/jama.2017.18556 PubMed PMID: 29260229..

    Article  PubMed  PubMed Central  Google Scholar 

  2. 2.

    Chan A-W, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, Hróbjartsson A, Mann H, Dickersin K, Berlin J, Doré C, Parulekar W, Summerskill W, Groves T, Schulz K, Sox H, Rockhold FW, Rennie D, Moher D. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200–7.

    Article  PubMed  PubMed Central  Google Scholar 

  3. 3.

    Spence O, Hong K, Onwuchekwa Uba R, Doshi P. Availability of study protocols for randomized trials published in high-impact medical journals: a cross-sectional analysis. Clin Trials. 2020;17(1):99–105. https://doi.org/10.1177/1740774519868310.

    Article  PubMed  PubMed Central  Google Scholar 

  4. 4.

    [ICH E9 1998] International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH harmonised tripartite guideline: statistical principles for clinical trials E9. London, England: European Medicines Agency; 1998.

    Google Scholar 

  5. 5.

    Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, Williamson PR, Kirkham JJ. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11(6):e1001666.

    Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365.

    Article  Google Scholar 

  7. 7.

    Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. Consolidated Standards of Reporting Trials Group. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8):e1–37.

    Article  Google Scholar 

  8. 8.

    [Trials submission site] (https://trialsjournal.biomedcentral.com/submission-guidelines/preparing-your-manuscript/update). Accessed 26 Oct 2020.

  9. 9.

    [ICJME site] http://www.icmje.org/recommendations/browse/roles-and-responsibilities/responsibilities-in-the-submission-and-peer-peview-process.html. Accessed 26 Oct 2020.

  10. 10.

    Schulz KF, Altman DG, Moher D, for the CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. Trials. 2010;11:32 PMID: 20334632.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Affiliations

Authors

Contributions

KH lead the development of the first draft with signficant contribution from all authors. The authors have read and approved the final manuscript.

Corresponding author

Correspondence to Karla Hemming.

Ethics declarations

Competing interests

The authors declare they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hemming, K., Kearney, A., Gamble, C. et al. Prospective reporting of statistical analysis plans for randomised controlled trials. Trials 21, 898 (2020). https://doi.org/10.1186/s13063-020-04828-8

Download citation