Commentary | Open | Open Peer Review | Published:
Promoting public access to clinical trial protocols: challenges and recommendations
Trialsvolume 19, Article number: 116 (2018)
Recognizing the value of promoting public access to clinical trial protocols, Trials pioneered the way for their publication over a decade ago. However, despite major advances in the public accessibility of information about trial methods and results, protocol sharing remains relatively rare.
Protocol sharing facilitates the critical appraisal of clinical trials and helps to identify and deter the selective reporting of outcomes and analyses. Challenges to the routine availability of high quality trial protocols include the gaps in incentives and adherence mechanisms, limited venues for sharing the original and final protocol versions, and the need for mechanisms to ensure transparent and complete protocol content.
We propose recommendations for addressing key challenges to protocol sharing in order to promote routine public access to protocols for the benefit of patients and other users of evidence from clinical trials.
As the cornerstone of medical evidence, clinical trials have been instrumental to major healthcare advances over the past decades. However, the impact and value of clinical trials have been limited by inaccessible or selectively reported information about their methods and results, leading to substantial research waste with direct implications for healthcare costs and patient outcomes .
Public access to study protocols is fundamental to the societal value of clinical trials. The trial protocol is the core document reporting the study background, relevance, methods, administration, and ethical considerations . The protocol guides the study team to help ensure that the trial is implemented in a manner that is consistent with the research objectives and the intentions of the steering group. Prior to recruiting study participants, the protocol serves as the basis for trial registration and for external review by research ethics committees and regulators. Increasingly, the protocol is also reviewed after trial completion to place its results into proper context.
In 2006, Trials pioneered the way for public access to trial protocols by providing a venue for their publication . Over 10 years later, the value of public access has become generally accepted by trialists, sponsors, funders, journal editors, regulators, healthcare providers, and patients. This acceptance has manifested in the growing number of protocols made available in journals as stand-alone publications or as web supplements accompanying published final reports . In Trials alone, the number of published protocols has increased from 46 in 2008 to 167 in 2012 and 359 in 2016.
However, despite repeated calls for increased protocol sharing [5,6,7], most clinical trials do not have a publicly accessible protocol [8,9,10]. In this commentary we review the rationale and challenges, and propose recommendations for improving public access to protocols.
Benefits of protocol sharing
Public availability of study protocols helps to facilitate detailed assessments of the internal validity of a trial, deter selective reporting of outcomes and analyses, and improve understanding of external validity. This key role is becoming even more relevant with the introduction of policies requiring the registration of summary results on trial registries as well as the expanded sharing of participant-level data [11,12,13].
The protocol contains information that is essential to appraising internal validity (i.e., risk of bias). It is otherwise difficult to evaluate and interpret the results of a trial without access to sufficient information on its design, conduct, and analysis. Published final reports often lack adequate descriptions of important design elements such as the methods of randomization and blinding [14,15,16,17,18]. Combined with the lack of publicly available protocols, these deficiencies in published final reports contribute to the finding that 89% of randomized trials included in systematic reviews had an ‘unclear’ rating for at least one domain of the Cochrane risk of bias tool, which is widely used for evaluating the internal validity of trials . In contrast, a high-quality protocol is not subject to word count restrictions and provides a comprehensive view of the pre-specified study methods.
Access to a greater level of detail in the protocol also provides a key mechanism to identify and deter the selective reporting of outcomes and analyses. Unacknowledged discrepancies in the primary outcomes, sample size calculations, and analysis plans are often found in published final reports when compared with the protocol [20,21,22,23,24,25]. Selective reporting within published reports acts in the same direction as publication bias of whole trials. Collectively, these reporting biases tend to inflate the efficacy estimates of individual trials and meta-analyses.
The protocol can also help to clarify important aspects of trial organization related to the roles of individuals, sponsors, and funders in trial design, conduct, and reporting. Protocols provide information on who controls or owns the trial data, who makes the decision for early trial stoppage, and the relationship between protocol contributors, professional medical writers, and authors of the final study report [26,27,28,29]. Protocols will also often describe any restrictions to the publication of trial results . This type of information is essential for adequate assessment of the risk of bias.
Availability of the protocol facilitates an understanding of the external validity (i.e., generalizability) of trial results. To determine how the findings should be applied in practice to individual patients, it is necessary to evaluate whether a given clinical scenario differs substantively from how the trial was conducted. Relevant considerations include the study setting, selection of trial participants and centers, details of the interventions and their administration, follow-up intensity, and concomitant care . Many of these trial elements are inadequately described in published final reports.
The protocol, in contrast, conveys much more information about these clinically relevant elements. For example, the eligibility criteria defined in the protocol are often not fully described in the published final report, which tends to portray a broader eligible study population than the protocol [31,32,33]. Further, the trial interventions are more completely described in protocols than in the published final reports, which is relevant when applying the interventions in clinical practice .
Complement to trial registries and results databases
While major progress in trial registration has been made over the past decade, the adherence levels and quality of registered information are highly variable [22, 35,36,37,38,39,40,41]. The Trial Registration Data Set defined by the World Health Organization and endorsed by the International Committee of Medical Journal Editors provides a brief outline of a trial’s topic and design . However, the methodological information in registries is usually insufficient to appraise the merits of the study design or identify selective reporting of analyses . For example, a recent study of registered and published oncology trials found that, due to incompletely or inaccurately registered information, the registry record enabled detection of only 75% of cases of discrepant primary outcomes in the published final report compared with using the full protocol . These limitations of trial registration highlight the important complementary role of having access to full protocols.
Challenges and recommendations
Despite its important benefits, public access to protocols is not yet widespread. Challenges to the routine availability of high quality trial protocols include the gaps in incentives and adherence mechanisms, limited venues for sharing the original and final protocol versions, and the need for mechanisms to ensure transparent reporting and completeness of protocol content. We propose recommendations to help address each of these challenges (Table 1).
In the absence of a universal adherence mechanism, it is important to provide sufficient incentives for trial investigators to share protocols as part of a broader dissemination plan that includes the full reporting of study results and participant-level data . Protocol publication provides credit in the form of a citable paper. Having a protocol that is open to public vetting also provides a degree of transparency that can benefit the trial and its investigators by boosting public awareness and the perceived trustworthiness of a trial.
Public sharing of protocols should be explicitly recognized by academic institutions and funders as a meritorious component of research performance evaluations. Rewarding the dissemination of full trial information helps to recognize its value in reducing research waste and increasing the impact of a study. There is a growing movement towards evaluating researchers based on comprehensive qualitative and quantitative indicators of impact rather than solely relying on traditional bibliometric and funding indices [45,46,47,48].
A clear disincentive for protocol sharing is when investigators have signed agreements with sponsors or funders that inappropriately restrict their freedom to disseminate the protocol or other essential information related to the trial [26, 49,50,51]. It is critical for sponsors, funders, and investigators to avoid placing such restrictions when agreeing to collaborate on a trial.
Comprehensive adherence mechanisms
Journal editors, regulators, sponsors, and funders can play a vital role by implementing policies mandating protocol availability . Because each stakeholder has a limited scope of trials within its purview, there is a need for broad participation across and within all stakeholder groups. Requiring the submission of protocols to journals along with manuscripts was suggested (and rejected) as far back as 1990 . Although most journals still do not routinely publish trial protocols, some major journals, such as The BMJ, Lancet, PLoS Medicine, and Annals of Internal Medicine, have taken the lead by requiring the submission of protocols and posting them online alongside trial manuscripts [53,54,55,56].
Regulatory authorities also have a critical role. In 2015, after Gøtzsche and Jørgensen’s pioneering efforts to obtain access , the European Medicines Agency implemented a policy to grant online access to clinical study reports (including protocols) that it reviewed as part of marketing authorization submissions . In the United States, a recent clarification of the Food and Drug Administration Amendments Act of 2007 affirmed the vital role of protocols and required that they be submitted at the time of registration of summary results on ClinicalTrials.gov . Although these major advances have not yet had sufficient time to demonstrate their impact, a substantial limitation is that almost half of non-industry sponsored trials fall outside of the scope of legislative requirements . Legislation in the United States and European Union excludes phase 1 trials or those evaluating interventions other than regulated drugs and devices [12, 58].
Additional key stakeholder groups include industry and non-industry sponsors and funders. In 2013, the European Federation of Pharmaceutical Industries and Associations endorsed the voluntary sharing of trial protocols, results, and participant-level data for research purposes . An online industry portal has been established to provide access to this information for researchers, subject to approval of the specific request . However, there is substantial variability across participating companies in terms of the scope of disclosure policies as well as the adherence to them [62, 63]. Some companies, such as GlaxoSmithKline, have gone further by voluntarily posting a subset of their protocols on their own publicly accessible website .
Since the patchwork of existing editorial, regulatory, and sponsor policies capture only a fraction of trials conducted around the world, it is important that journal editors, regulators, sponsors, and funders implement measures to improve adherence on a broader scale. Following the lead of major journals, all journals that publish a clinical trial report should either post the protocol as a web supplement or post a link to the online protocol publication. Regulatory policies mandating public access to protocols should be adopted by governments as a condition of marketing authorization. Both industry and non-industry sponsors should make their protocols available for all trials under their stewardship, while funders should require protocol sharing as a condition for grant approval.
Prospective access to the original protocol
Current journal policies and legislation call for protocol sharing prior to the completion of data collection or at the time of results reporting or regulatory submission [12, 58, 65]. This delayed timing allows the final protocol version to be shared, including a list of any amendments, since protocols often evolve with several formal versions over the course of a trial .
However, a major concern is the potential for biased, undisclosed amendments to the pre-specified trial outcomes or analyses based on interim examination of the data. While it is expected that amendments would be transparently listed in each protocol version , we and others have found that even revisions to the protocol-defined primary outcomes are common and not acknowledged in the published final reports or latest protocol version [20, 22,23,24, 68, 69].
Public access to both the original protocol version dated prior to participant enrolment, as well as the final protocol version with a list of amendments, would provide a verifiable record to help identify and evaluate any unacknowledged amendments. Although concerns have been raised over competitive advantage associated with earlier disclosure of detailed protocol information [70,71,72,73], the European Medicines Agency and legal rulings in the United States have concluded that very little of the information contained in a trial protocol constitutes commercially confidential information that would confer a competitive advantage if disclosed to other sponsors [74,75,76]. Earlier protocol sharing helps to assert the intellectual origin of the trial idea. The originators would have a substantial head start with funding, ethics approval, and logistical implementation of the trial. Further, the basic trial description should already be publicly documented on a trial registry prior to participant enrolment. The widespread adoption of trial registration policies reflects broad acceptance that the benefits of public access outweigh the potential risks.
In rare cases where legitimate concerns remain about prospectively sharing the full protocol prior to trial inception, a potential solution would be to offer a lockbox whereby the original protocol receiving ethics approval is submitted to a registry or journal prior to participant enrolment, but not made publicly available until a later time prior to data unblinding. For example, in the context of certain trials evaluating complex interventions where participants are blinded to the true nature of the placebo arm or study hypotheses, a lockbox can help to address concerns about unblinding when participants have access to the protocol while the trial is ongoing .
Universal venue for sharing protocols
Trial registries already provide an established, unrestricted, searchable online mechanism for recording basic protocol information for a trial. A natural extension of this important role would be for registries to serve as the standard repository for housing the original and final trial protocols alongside each registry record. ClinicalTrials.gov and the Australian New Zealand Clinical Trials Registry have recently allowed the full protocol and related documents to be uploaded [12, 78]. To ensure that protocols are captured for all trials regardless of where they are registered, it is important to build protocol uploading capacity for all registries in the World Health Organization Registry Network.
Another key venue for protocol dissemination is journals. Protocol publication offers citable indexing in Medline or other bibliographic databases. There is also the opportunity for additional explanation and discussion of topical issues, beyond the protocol text. However, it is usually not possible to implement major design modifications in response to peer reviewer comments, given that the protocol has already been approved by a research ethics committee, funder, or regulator. Another limitation is that the protocol published in journals is often an abbreviated version of the original protocol, which can reduce transparency if the published version omits relevant details about the trial. Further, given the small number of journals that currently publish protocols, there would be limited capacity to efficiently handle the potential volume of protocol submissions for the thousands of trials initiated monthly .
As protocol publication becomes more widespread, it will be important for journals to commit the necessary resources to review and make decisions on submissions in a timely manner. Journals that publish abridged versions of trial protocols should include the full protocol as an online appendix (for example, see ), similar to how some journals post the full protocol alongside the final report of trial results. After trial completion, the final protocol version could also be posted as a supplementary appendix to the original published protocol.
Complete protocol content
The sharing of a protocol is only useful if the document adequately details the key features of a trial. Protocols are usually more informative about study design and organization than published trial reports, but many protocols still lack information about important methodological, ethical, and administrative trial elements . Missing or unclear information in the protocol makes it difficult to know whether or not the issues were adequately handled by the trial investigators. Poor quality protocols also lead to increased time and costs required for trial completion .
To help promote high-quality and complete content, protocols should adhere to the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) 2013 Statement and explanatory paper, which define a minimum checklist of items to address [2, 67, 81]. SPIRIT has been endorsed internationally by over 100 medical journals, including Trials , as well as research organizations and funders. An online, SPIRIT-based protocol authoring tool is being developed to make it easier to create, manage, and register high-quality protocols .
Ensuring public access to protocols adds considerable value and reduces research waste by providing important study details that are often not found in other sources of information. The routine sharing of protocols with pre-defined outcomes and analysis plans is a simple, low cost, and feasible way to deter and identify selective reporting. However, key challenges must be addressed to fully realize the positive impact of protocol availability. With ongoing and expanded support from journal editors, trial registries, sponsors, funders, regulators, and legislators, public access to study protocols can become standard practice for the benefit of patients and other users of evidence from clinical trials.
Standard Protocol Items: Recommendations for Interventional Trials
Chan A-W, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, et al. Increasing value and reducing waste: Addressing inaccessible research. Lancet. 2014;383(9913):257–66. https://doi.org/10.1016/S0140-6736(13)62296-5
Chan A-W, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, et al. SPIRIT 2013 Statement: Defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200–7. https://doi.org/10.7326/0003-4819-158-3-201302050-00583
Altman D, Furberg C, Grimshaw J, Rothwell P. Trials - using the opportunities of electronic publishing to improve the reporting of randomised trials. Trials. 2006;7:6. https://doi.org/10.1186/1745-6215-7-6
van Rosmalen BV, Alldinger I, Cieslak KP, Wennink R, Clarke M, Ali UA, et al. Worldwide trends in volume and quality of published protocols of randomized controlled trials. PLoS One. 2017;12(3):e0187389.
Chalmers I, Altman DG. How can medical journals help prevent poor medical research? Some opportunities presented by electronic publishing. Lancet. 1999;353(9151):490–3. https://doi.org/10.1016/S0140-6736(98)07618-1
Lassere M, Johnson K. The power of the protocol. Lancet. 2002;360(9346):1620–2. https://doi.org/10.1016/S0140-6736(02)11652-7
Chan A-W. Bias, spin, and misreporting: Time for full access to trial protocols and results. PLoS Med. 2008;5(11):e230. https://doi.org/10.1371/journal.pmed.0050230
Hamm MP, Hartling L, Milne A, Tjosvold L, Vandermeer B, Thomson D, et al. A descriptive analysis of a representative sample of pediatric randomized controlled trials published in 2007. BMC Pediatr. 2010;10:96. https://doi.org/10.1186/1471-2431-10-96
Chan A-W, Upshur R, Singh JA, Ghersi D, Chapuis F, Altman DG. Research protocols: waiving confidentiality for the greater good. BMJ. 2006;332(7549):1086–9. https://doi.org/10.1136/bmj.332.7549.1086
Lucey M, Clark J, Glasziou P. Public availability of trial protocols. Lancet. 2017;390(10113):e54–5. https://doi.org/10.1016/S0140-6736(17)33255-5
Taichman DB, Backus J, Baethge C, Bauchner H, de Leeuw PW, Drazen JM, et al. Sharing clinical trial data – a proposal from the International Committee of Medical Journal Editors. N Engl J Med. 2016;374(4):384–6. https://doi.org/10.1056/NEJMe1515172
Zarin DA, Tse T, Williams RJ, Carr S. Trial Reporting in ClinicalTrials.gov - The Final Rule. N Engl J Med. 2016;375(20):1998–2004. https://doi.org/10.1056/NEJMsr1611785
European Commission. Guidance on posting and publication of result-related information on clinical trials in relation to the implementation of Article 57(2) of Regulation (EC) No 726/2004 and Article 41(2) of Regulation (EC) No 1901/2006. Official Journal of the European Union. 2012. https://ec.europa.eu/health/files/eudralex/vol-10/2012_302-03/2012_302-03_en.pdf. Accessed 12 Feb 2018.
Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;(Issue 11):MR000030. https://doi.org/10.1002/14651858.MR000030.pub2.
Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–6. https://doi.org/10.1016/S0140-6736(13)62228-X
Hróbjartsson A, Boutron I. Blinding in randomized clinical trials: imposed impartiality. Clin Pharmacol Ther. 2011;90(5):732–6. https://doi.org/10.1038/clpt.2011.207
Hróbjartsson A, Pildal J, Chan A-W, Haahr MT, Altman DG, Gøtzsche PC. Reporting on blinding in trial protocols and corresponding publications was often inadequate but rarely contradictory. J Clin Epidemiol. 2009;62(9):967–73. https://doi.org/10.1016/j.jclinepi.2009.04.003
Pildal J, Chan A-W, Hróbjartsson A, Forfang E, Altman DG, Gøtzsche PC. Comparison of descriptions of allocation concealment in trial protocols and the published reports: Cohort study. BMJ. 2005;330:1049–52. https://doi.org/10.1136/bmj.38414.422650.8F
Jørgensen L, Paludan-Muller AS, Laursen DR, Savovic J, Boutron I, Sterne JA, et al. Evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews. Syst Rev. 2016;5:80. https://doi.org/10.1186/s13643-016-0259-8
Chan A-W, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–65. https://doi.org/10.1001/jama.291.20.2457
Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, et al. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11(6):e1001666. https://doi.org/10.1371/journal.pmed.1001666
Chan A-W, Pello A, Kitchen J, Axentiev A, Virtanen JI, Liu A, et al. Association of trial registration with reporting of primary outcomes in protocols and publications. JAMA. 2017;318(17):1709–11. https://doi.org/10.1001/jama.2017.13001
Redmond S, von Elm E, Blümle A, Gengler M, Gsponer T, Egger M. Cohort study of trials submitted to ethics committee identified discrepant reporting of outcomes in publications. J Clin Epidemiol. 2013;66(12):1367–75. https://doi.org/10.1016/j.jclinepi.2013.06.020
van den Bogert CA, Souverein PC, Brekelmans CT, Janssen SW, Koëter GH, Leufkens HG, et al. Primary endpoint discrepancies were found in one in ten clinical drug trials. Results of an inception cohort study. J Clin Epidemiol. 2017;89:199–208. https://doi.org/10.1016/j.jclinepi.2017.05.012
Dwan K, Altman DG, Cresswell L, Blundell M, Gamble CL, Williamson PR. Comparison of protocols and registry entries to published reports for randomised controlled trials. Cochrane Database Syst Rev. 2011;(Issue 1):MR000031. https://doi.org/10.1002/14651858.MR000031.pub2.
Gøtzsche PC, Hróbjartsson A, Johansen HK, Haahr MT, Altman DG, Chan A-W. Constraints on publication rights in industry-initiated clinical trials. JAMA. 2006;295(14):1645–6. https://doi.org/10.1001/jama.295.14.1645
Gøtzsche PC, Hróbjartsson A, Johansen HK, Haahr MT, Altman DG, Chan A-W. Ghost authorship in industry-initiated randomised trials. PLoS Med. 2007;4(1):e19. https://doi.org/10.1371/journal.pmed.0040019
Vera-Badillo FE, Napoleone M, Krzyzanowska MK, Alibhai SM, Chan AW, Ocana A, et al. Honorary and ghost authorship in reports of randomised clinical trials in oncology. Eur J Cancer. 2016;66:1–8. https://doi.org/10.1016/j.ejca.2016.06.023
Lundh A, Krogsboll LT, Gotzsche PC. Sponsors’ participation in conduct and reporting of industry trials: a descriptive study. Trials. 2012;13:146. https://doi.org/10.1186/1745-6215-13-146
Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?”. Lancet. 2005;365(9453):82–93.
Zhang S, Liang F, Li W, Tannock I. Comparison of eligibility criteria between protocols, registries, and publications of cancer clinical trials. J Natl Cancer Inst. 2016;108(11):djw129. https://doi.org/10.1093/jnci/djw129
Blümle A, Meerpohl JJ, Rucker G, Antes G, Schumacher M, von Elm E. Reporting of eligibility criteria of randomised trials: cohort study comparing trial protocols with subsequent articles. BMJ. 2011;342:d1828. https://doi.org/10.1136/bmj.d1828
Gandhi M, Ameli N, Bacchetti P, Sharp GB, French AL, Young M, et al. Eligibility criteria for HIV clinical trials and generalizability of results: the gap between published reports and study protocols. AIDS. 2005;19(16):1885–96.
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687. https://doi.org/10.1136/bmj.g1687
van Lent M, IntHout J, Out HJ. Differences between information in registries and articles did not influence publication acceptance. J Clin Epidemiol. 2015;68(9):1059–67. https://doi.org/10.1016/j.jclinepi.2014.11.019
Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302(9):977–84. https://doi.org/10.1001/jama.2009.1242
Walker KF, Stevenson G, Thornton JG. Discrepancies between registration and publication of randomised controlled trials: an observational study. JRSM Open. 2014;5(5):2042533313517688. https://doi.org/10.1177/2042533313517688
Hannink G, Gooszen HG, Rovers MM. Comparison of registered and published primary outcomes in randomized clinical trials of surgical interventions. Ann Surg. 2013;257(5):818–23. https://doi.org/10.1097/SLA.0b013e3182864fa3
Fleming PS, Koletsi D, Dwan K, Pandis N. Outcome discrepancies and selective reporting: Impacting the leading journals? PLoS One. 2015;10(5):e0127495. https://doi.org/10.1371/journal.pone.0127495
You B, Gan HK, Pond G, Chen EX. Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: a systematic review. J Clin Oncol. 2012;30(2):210–6. https://doi.org/10.1200/JCO.2011.37.0890
Viergever RF, Karam G, Reis A, Ghersi D. The quality of registration of clinical trials: still a problem. PLoS One. 2014;9(1):e84727. https://doi.org/10.1371/journal.pone.0084727
World Health Organization. International Clinical Trials Registry Platform (ICTRP). 2018. http://www.who.int/ictrp/en/. Accessed 2 Jan 2018.
Reveiz L, Chan A-W, Krleža-Jerić K, Granados CE, Pinart M, Etxeandia I, et al. Reporting of methodologic information on trial registries for quality assessment: a study of trial records retrieved from the WHO search portal. PLoS One. 2010;5(8):e12484. https://doi.org/10.1371/journal.pone.0012484
Perlmutter A, Tran VT, Dechartres A, Ravaud P. Statistical controversies in clinical research: comparison of primary outcomes in protocols, public clinical-trial registries and publications: the example of oncology trials. Ann Oncol. 2017;28(4):688–95. https://doi.org/10.1093/annonc/mdw682
Wilsdon J, Allen L, Belfiore E, Campbell P, Curry S, Hill S, et al. The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management. Bristol: Higher Education Funding Council for England; 2015. https://doi.org/10.13140/RG.2.1.4929.1363
Hicks D, Wouters P, Waltman L, de Rijcke S, Rafols I. Bibliometrics: The Leiden Manifesto for Research Metrics. Nature. 2015;520:429.
Benedictus R. From Output to Impact. 2017. https://www.nrin.nl/ri-collection/rcr-practice/policy/output-impact/. Accessed 2 Jan 2018.
Dijstelbloem H, Huisman F, Miedema F, Mijnhardt W. English: Science in Transition. 2015. http://www.scienceintransition.nl/english. Accessed 8 Jan 2018.
Stretton S, Lew RA, Ely JA, Snape MJ, Carey LC, Haley C, et al. Sponsor-imposed publication restrictions disclosed on ClinicalTrials.gov. Account Res. 2016;23(2):67–78. https://doi.org/10.1080/08989621.2015.1020375
Schulman KA, Seils DM, Timbie JW, Sugarman J, Dame LA, Weinfurt KP, et al. A national survey of provisions in clinical-trial agreements between medical schools and industry sponsors. N Engl J Med. 2002;347(17):1335–41. https://doi.org/10.1056/NEJMsa020349
Kasenda B, von Elm E, You JJ, Blümle A, Tomonaga Y, Saccilotto R, et al. Agreements between industry and academia on publication rights: A retrospective study of protocols and publications of randomized clinical trials. PLoS Med. 2016;13(6) https://doi.org/10.1371/journal.pmed.1002046.
Siegel JP. Editorial review of protocols for clinical trials. N Engl J Med. 1990;323(19):1355. https://doi.org/10.1056/NEJM199011083231921
Jones G, Abbasi K. Trial protocols at the BMJ. BMJ. 2004;329(7479):1360. https://doi.org/10.1136/bmj.329.7479.1360
The Lancet. Types of Article and Manuscript Requirements. 2017. http://thelancet.com/lancet/information-for-authors/article-types-manuscript-requirements. Accessed 8 Jan 2018.
PLOS Medicine. Submission Guidelines. 2017. http://journals.plos.org/plosmedicine/s/submission-guidelines. Accessed 2 Jan 2018.
Annals of Internal Medicine. Information for Authors. 2018. http://annals.org/aim/pages/authors. Accessed 2 Jan 2018.
Gøtzsche PC, Jørgensen AW. Opening up data at the European Medicines Agency. BMJ. 2011;342:d2686. https://doi.org/10.1136/bmj.d2686
Davis AL, Miller JD. The European Medicines Agency and Publication of Clinical Study Reports: A Challenge for the US FDA. JAMA. 2017;317(9):905–6. https://doi.org/10.1001/jama.2017.0918
Zarin DA, Tse T, Ross JS. Trial-results reporting and academic medical centers. N Engl J Med. 2015;372(24):2371–2. https://doi.org/10.1056/NEJMc1505965
European Federation of Pharmaceutical Industries and Associations (EFPIA) and The Pharmaceutical Research and Manufacturers of America (PhRMA). Principles for Responsible Clinical Trial Data Sharing. 2013. http://phrma-docs.phrma.org/sites/default/files/pdf/PhRMAPrinciplesForResponsibleClinicalTrialDataSharing.pdf. Accessed 2 Jan 2018.
Clinical Study Data Request. 2018. https://clinicalstudydatarequest.com/Default.aspx. Accessed 2 Jan 2018.
Clinical Study Data Request. Study Sponsors. 2018. https://clinicalstudydatarequest.com/Study-Sponsors-Info.aspx. Accessed 2 Jan 2018.
Boutron I, Dechartres A, Baron G, Li J, Ravaud P. Sharing of data from industry-funded registered clinical trials. JAMA. 2016;315(24):2729–30. https://doi.org/10.1001/jama.2016.6310
GlaxoSmithKline. Clinical Study Register. 2017. https://www.gsk-clinicalstudyregister.com. Accessed 2 Jan 2018.
Li T, Boutron I, Salman RA, Cobo E, Flemyng E, Grimshaw JM, et al. Review and publication of protocol submissions to Trials - what have we learned in 10 years? Trials. 2016;18:34. https://doi.org/10.1186/s13063-016-1743-0
Getz KA, Stergiopoulos S, Short M, Surgeon L, Krauss R, Pretorius S, et al. The impact of protocol amendments on clinical trial performance and cost. Ther Innov Regul Sci. 2016;50(4):436–41. https://doi.org/10.1177/2168479016632271
Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586. https://doi.org/10.1136/bmj.e7586
Chan A-W, Krleža-Jerić K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004;171:735–40.
Al-Marzouki S, Roberts I, Evans S, Marshall T. Selective reporting in clinical trials: analysis of trial protocols accepted by The Lancet. Lancet. 2008;372(9634):201. https://doi.org/10.1016/S0140-6736(08)61060-0
Ross S, Magee L, Walker M, Wood S. Protecting intellectual property associated with Canadian academic clinical trials – approaches and impact. Trials. 2012;13:243. https://doi.org/10.1186/1745-6215-13-243
Reveiz L, Krleža-Jerić K, Chan A-W, de Aguiar S. Do trialists endorse clinical trial registration? Survey of a PubMed sample. Trials. 2007;8:30. https://doi.org/10.1186/1745-6215-8-30
Lurie P, Zieve A. Sometimes the silence can be like the thunder: access to pharmaceutical data at the FDA. Law Contemp Probl. 2006;69(3):85–97.
Krleža-Jerić K. Clinical trial registration: the differing views of industry, the WHO, and the Ottawa Group. PLoS Med. 2005;2(11):e378. https://doi.org/10.1371/journal.pmed.0020378
European Medicines Agency EMA/240810/2013. European Medicines Agency policy on publication of clinical data for medicinal products for human use, POLICY/0070. 2016. http://www.ema.europa.eu/docs/en_GB/document_library/Other/2014/10/WC500174796.pdf. Accessed 2 Jan 2018.
Eichler HG, Abadie E, Breckenridge A, Leufkens H, Rasi G. Open clinical trial data for all? A view from regulators. PLoS Med. 2012;9(4):e1001202. https://doi.org/10.1371/journal.pmed.1001202
US District Court for the District Court of Columbia. Public Citizen Health Research v. Food and Drug, 964 F. Supp. 413 (D.D.C. 1997). 1997. https://law.justia.com/cases/federal/district-courts/FSupp/964/413/1410001/. Accessed 8 Jan 2018.
Basu AP, Pearse JE, Rapley T. Publishing protocols for trials of complex interventions before trial completion - potential pitfalls, solutions and the need for public debate. Trials. 2017;18:5. https://doi.org/10.1186/s13063-016-1757-7
Askie L. Trial registration records, updates, and protocols. Lancet. 2016;388(10042):341–2. https://doi.org/10.1016/S0140-6736(16)30965-5
Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. https://doi.org/10.1371/journal.pmed.1000326
Bouton TC, Phillips PJ, Mitnick CD, Peloquin CA, Eisenach K, Patientia RF, et al. An optimized background regimen design to evaluate the contribution of levofloxacin to multidrug-resistant tuberculosis treatment regimens: study protocol for a randomized controlled trial. Trials. 2017;18(1):563. https://doi.org/10.1186/s13063-017-2292-x
Chan A-W, Tetzlaff JM, Altman DG, Dickersin K, Moher D. SPIRIT 2013: New guidance for content of clinical trial protocols. Lancet. 2013;381:91–2. https://doi.org/10.1016/S0140-6736(12)62160-6.
SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials). Implementation Tools. 2017. http://www.spirit-statement.org/implementation-tools/. Accessed 2 Jan 2018.
No specific funding.
Availability of data and materials
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.