Skip to main content

How do trial teams plan for retention during the design stage of the trial? A scoping review protocol

Abstract

Background

Retention remains a major challenge for many clinical trials. The SPIRIT guidelines state the following information on retention should be included in the trial protocol “Plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols”. This guidance shows the importance of planning retention methods and handling missing data as this can impact how the results of the trial are interpreted. The most recent Cochrane review of strategies to improve retention in clinical trials highlighted that some trials implemented multiple retention strategies and we questioned whether the use of multiple strategies was planned at the design stage and included in the protocol or are strategies implemented when retention becomes an issue within the trial. The purpose of our scoping review is to establish if and how trial teams prepare for retention at the design phase of clinical trials.

Methods and analysis

We will follow the methodological framework and guidelines for scoping reviews outlined by the Joanna Briggs Institute. We will search MEDLINE/PubMed, Scopus, EMBASE, CINAHL (EBSCO), and Web of Science. A comprehensive search strategy for PubMed was developed in collaboration with an experienced research librarian. We will include protocols for phase 2, 3, and 4 RCTs as well as pilot and feasibility studies. The screening process will involve two reviewers. EM will independently screen all titles and abstracts. FS will screen 10% of the overall search output, and where necessary full protocol texts will be screened to determine eligibility. We will randomly sample eligible protocols to ensure the protocols represent a variety of trial and intervention types. Data will be extracted from each protocol and the results will be synthesised. The analysis will be qualitative using a narrative summary and descriptive statistics where appropriate.

Discussion

The scoping review will help trial methodologists better understand if retention strategies are planned for during the design stage of the trial contributing to the PRioRiTy II unanswered question “How should people who run trials plan for retention during their funding application and creation of the trial (protocol development)?”.

Peer Review reports

Background/rationale

Randomised controlled trials (RCTs) are conducted in accordance with a trial protocol. Having a protocol that is incomplete and non-transparent makes it difficult to critically appraise the trial [1]. Protocols are needed for the readers of the corresponding paper to be able to fully appraise and interpret the results of the trial [2]. As per the 2013 SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) definition, a protocol is “a document that provides sufficient detail to enable understanding of the background, rationale, objectives, study population, interventions, methods, statistical analyses, ethical considerations, dissemination plans, and administration of the trial, replication of key aspects of trial methods and conduct; and appraisal of the trial’s scientific and ethical rigour from ethics approval to dissemination of results” (3;202). Prior to the introduction of SPIRIT, the content and quality of protocols differed greatly [3, 4], but the 33-item SPIRIT checklist for the minimum recommended protocol items improved this. Having a clearly written protocol increases transparency in trial conduct. Protocols are usually published prior to the trial paper publication [5]. However, this is not always the case, and some protocol publications will require payment for access. For example, of the cancer clinical trials published in January of 2020 (n = 113), only 11.3% had a publicly accessible protocol that was not behind a paywall [2]. This further limits transparency and hinders replication in trial methods and conduct, which has been recommended for trial retention strategies in order to improve the evidence base for their effectiveness [6].

Retention remains a major challenge for many clinical trials [7]. The SPIRIT guidelines recommend the following information on retention be included in the trial protocol in Sect. 18b “Plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols” (8;3). These of course are minimum requirements but the information can impact how missing data is dealt with and interpreted in the analysis of the trial [8].

A recent Cochrane systematic review of strategies to improve retention in clinical trials found that there were no strategies for which the quality of evidence was high that showed improved retention. The review also highlighted that some trials implement multiple retention strategies [6], and thus we questioned whether the use of multiple strategies is planned at the design stage and included in the protocol or are strategies implemented when retention becomes an issue within the trial. Evidence from a small number of interviews with trial staff is variable [9]. Prospective retention planning informs appropriate costing and implementation of retention strategies but also increases transparency in trial conduct.

Objective

The purpose of our scoping review is to establish if and how trial teams prepare for retention at the design phase of clinical trials. This will contribute to the evidence base for the PRioRiTy II (Prioritising Retention in Randomised Trials) unanswered question “How should people who run trials plan for retention during their funding application and creation of the trial (protocol development)?” [10].

Methods

The scoping review will be conducted in accordance with the guidelines and framework outlined by the Joanna Briggs Institute (JBI) [11], the most recent framework for scoping reviews which builds on prior work [12,13,14]. The framework consists of the following nine steps:

  1. 1.

    Defining and aligning the objective/s and question/s.

  2. 2.

    Developing and aligning the inclusion criteria with the objective/s and question/s.

  3. 3.

    Describing the planned approach to evidence searching, selection, data extraction and presentation of the evidence.

  4. 4.

    Searching for the evidence.

  5. 5.

    Selecting the evidence.

  6. 6.

    Extracting the evidence.

  7. 7.

    Analysis of the evidence.

  8. 8.

    Presentation of the results.

  9. 9.

    Summarising the evidence in relation to the purpose of the review, making conclusions and noting any implications of the findings.

The guidance from the Joanna Briggs Institute [11] along with the newly developed reporting guidelines for scoping reviews: the Preferred Reporting Items for a Systematic Review and Meta-Analysis Protocols Extension for Scoping Reviews (PRISMA-ScR) [15] has been consulted in the development of this protocol (Additional File 1).

Data sources and search strategy

A comprehensive search strategy for PubMed was developed in collaboration with an experienced research librarian at University College Cork and is shown below. The following electronic databases will be searched for relevant protocols, MEDLINE/PubMed, Scopus, EMBASE, CINAHL (EBSCO), and Web of Science. The search will be adapted as appropriate for each database using the software Polyglot which translates search strategies across databases. The search and screening process will take place over a six-week period.

Example of PubMed search

(“randomised controlled trial”[Title/Abstract]) OR (“randomized controlled trial”[Title/Abstract])) OR (“randomised clinical trial”[Title/Abstract])) OR (“randomized clinical trial”[Title/Abstract])) OR (“randomized controlled trials as topic”[MeSH Terms])) AND (“protocol”[Title/Abstract])) OR (“clinical trial protocols as topic”[MeSH Terms]).

This is the full search strategy developed for PubMed. The limits that will be applied include published in the English language between the years of 2014 and 2019 (inclusive). No other limits will be applied to the search.

Inclusion criteria

Population

The population include the following: protocols of RCTs that include adults and/or children of any age.

Concepts

The concepts includes the following: protocols of RCTs investigating any treatment/intervention type for any disease area. Randomisation can be at the cluster or individual level. The concepts also include protocols of RCTs investigating any comparator including placebo and examining any outcome.

Context

The context of this scoping review is open sources of evidence pertaining to any contextual setting.

Types of evidence sources

We will include trial protocols published in the English language. We will include protocols from 2014 to 2019 (inclusive), giving sufficient time to see the effect of the SPIRIT guidelines, which were published in 2013. We will include protocols for phase 2, 3, and 4 RCTs as well as pilot and feasibility studies.

Screening and selection process

EM will systematically collate and import titles and abstracts of all electronically sourced search results to EndNote, grouping results separately for each database. Duplicates will be removed, and the remaining results will be exported to the Rayyan QCRI software for screening. The screening process will involve two reviewers (EM and FS). EM will independently screen all titles and abstracts. FS will screen a random selection of 10% of the overall search output. If there is less than 80% agreement on the random 10% of the search output that is double screened, we will undertake screening of a further 10% validation sample. We will reiterate this until > 80% agreement on the validation sample is reached. If disagreement arises between these two reviewers regarding the eligibility of a protocol, a third reviewer KG will be consulted, and where necessary full protocol texts will be screened to determine eligibility. The protocol screening and selection process will be presented both narratively and graphically using a PRISMA (Transparent Reporting of Systematic Reviews and Meta-Analyses) flow diagram [16]. Details of excluded sources at full-text review will be outlined and included in the review with reasons for the protocol exclusion.

Sampling

We will select a random sample of 10% of eligible protocols for data extraction; the random sample will be chosen by including blocks of 10 protocols from the list of eligible protocols for inclusion. A randomly chosen 10% will provide a sufficient representation of all eligible protocols.

Data management and data charting (extraction) process

Following screening for eligibility all data on retention plans will be extracted. Data to be extracted is outlined in Table 1. The main outcome of interest is whether SPIRIT item 18b is satisfied (Table 1, variable 7), since this statement has sub-requirements during the data charting process we will divide the statement into three aspects “plans to promote participant retention” and “plans to complete follow-up, including list of any outcome data to be collected for participants who discontinue from intervention protocols” and “plans to complete follow-up, including list of any outcome data to be collected for participants who deviate from intervention protocols”. Dividing the statement into sub-statements will more accurately reflect whether SPIRIT item 18b is satisfied or not. With regard to retention strategies we specifically mean an action/activity that is conducted with the purpose of reducing missing data/improving data completeness, we will not include activities to improve adherence or compliance to an intervention. Where there is ambiguity on the purpose of specific actions, i.e. if they are for participant retention or not, the protocol authors will be contacted. Where contact is not possible/a reply is not received, the ambiguous information will be highlighted and included in the data extraction. Prior to the full data charting process, the data extraction form will be piloted using a sample of 10 protocols as the variables to be extracted may need to be refined and improved to best meet the objectives of the scoping review. Data charting will be carried out by one reviewer (EM) and a random sample (10%) of the protocols will be double extracted and checked for consistency by (FS) to ensure consistency and improve the reliability of the data extraction process. If there is less than 80% agreement regarding the data extracted in the 10% random sample of protocols, we will undertake data extraction of a further 10% validation sample. We will reiterate this until > 80% agreement on the validation sample is reached. Data charting will be conducted, and information will be entered into a Microsoft Excel file.

Data items

Table 1 Variables to be extracted

Data synthesis

We will synthesise the data in a narrative synthesis with descriptive statistics where appropriate. The descriptive statistics that will be generated will include the frequencies and percentages of the trial characteristics such as the percentage of trials conducted in different disease areas, percentage of trials conducted among vulnerable populations, percentage of trials that are publicly or privately funded, and the percentage of drug trials, surgical trials, non-drug/behavioural intervention, and medical device trials, modes of follow-up methods used by trials, and number of trials with patient-reported primary outcomes vs non-patient-reported outcomes. We will also generate the percentage of trial protocols that have satisfied the SPIRIT item 18b statement.

Since we will include pilot and feasibility trial protocols, we plan to conduct a sub-analysis of this group of studies as one of the purposes of pilot and feasibility trials may be to develop and test retention strategies.

The content will be analysed to determine if the protocol complies with the SPIRIT guidelines for retention strategies Sect. 18b- “Plans to promote participant retention and complete follow-up including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols” (8;3). We will divide the statement into three aspects “plans to promote participant retention” and “plans to complete follow-up including list of any outcome data to be collected for participants who discontinue from intervention protocols” and “plans to complete follow-up including list of any outcome data to be collected for participants who deviate from intervention protocols”. This will be recorded as either “yes” or “no” for each separate aspect. If the protocols include information regarding planned retention strategies, this information will be analysed in the narrative synthesis. The narrative synthesis approach will be based on the Guidance on the Conduct of Narrative Synthesis in Systematic Reviews [17].

Presentation of findings

We will present the search results in a PRISMA flow diagram illustrating the total number of protocols generated by the search strategy and the number of protocols excluded following the application of the inclusion/exclusion criteria and ultimately the number of protocols included in the scoping review. Summary tables will depict the study characteristics described in the included protocols. Additional tables, figures, and narrative descriptions will illustrate the data addressing our research question. The findings of the scoping review will be disseminated via publication in a peer-reviewed journal.

Discussion

Conducting a scoping review of randomised controlled trial protocols to investigate if retention strategies are planned by trial teams during the design stage of the trial will contribute to the PRioRiTy II unanswered question “How should people who run trial plan for retention during their funding application and creation of the trial (protocol development)?” [10]. The strength of this review is the intention to include a variety of trial types as described in the methods section. The review will highlight the gaps that exist in the planning and communicating of retention strategies in clinical trial protocols, which may have the potential to increase transparency in trial conduct and make trials more efficient as it will highlight to researchers the lack of consideration and/or communication of trial retention strategies during the initial planning and design stages of the trial. Foreplaning could increase retention strategy success as trial teams will be able to spend greater time researching the most effective strategies in terms of retaining participants and carefully consider the resources required as retention strategies can be very expensive to implement often with little reward in terms of retaining participants [6, 18, 19]. The findings of this review have the potential to inform trialists to plan retention strategies during the design stage of the trial rather than when retention issues appear. The review will also help identify questions for future research projects and findings from this review can be explored in more depth in future research investigating retention strategies in clinical trials.

Availability of data and materials

Not applicable as this is a protocol for a scoping review.

Abbreviations

RCTs:

Randomised controlled trials

SPIRIT:

Standard Protocol Items: Recommendations for Interventional Trials

PRioRity:

Prioritising Retention in Randomised Controlled Trials

PRISMA-ScR:

Preferred Reporting Items for a Systematic Review and Meta-Analysis Protocols Extension for Scoping Reviews

PRISMA:

Transparent Reporting of Systematic Reviews and Meta-Analyses

References

  1. Tetzlaff JM, Moher D, Chan A-W. Developing a guideline for clinical trial protocol content: Delphi consensus survey. Trials. 2012;13(1):1–10.

    Article  Google Scholar 

  2. Babu C, Mell L, Lee N, Zakeri K. Public access to protocols of contemporary cancer randomized clinical trials. Trials. 2021;22(1):1–4.

    Article  Google Scholar 

  3. Chan A-W, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200–7.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Tetzlaff JM, Chan A-W, Kitchen J, Sampson M, Tricco AC, Moher D. Guidelines for randomized clinical trial protocol content: a systematic review. Syst Rev. 2012;1(1):1–11.

    Article  Google Scholar 

  5. Chan A-W, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–65.

    Article  CAS  PubMed  Google Scholar 

  6. Gillies K, Kearney A, Keenan C, Treweek S, Hudson J, Brueton VC, et al. Strategies to improve retention in randomised trials. Cochrane Database Syst Rev. 2021;3(3):MR000032.

    PubMed  Google Scholar 

  7. Walters SJ, Bonacho Dos AnjosHenriqus-Cadby I, Bortolami O, Flight L, Hind D, Jacques RM, et al. Recruitment and retention of participants in randomised controlled trials: a review of trials funded and published by the United Kingdom Health Technology Assessment Programme. BMJ open. 2017;7(3):e015276-e.

    Article  Google Scholar 

  8. Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, SPIRIT, et al. explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;2013:346.

    Google Scholar 

  9. Daykin A, Clement C, Gamble C, Kearney A, Blazeby J, Clarke M, et al. ‘Recruitment, recruitment, recruitment’ – the need for more focus on retention: a qualitative study of five trials. Trials. 2018;19(1):76.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Brunsdon D, Biesty L, Brocklehurst P, Brueton V, Devane D, Elliott J, et al. What are the most important unanswered research questions in trial retention? A James Lind alliance priority setting partnership: the PRioRiTy II (Prioritising Retention in Randomised Trials) study. Trials. 2019;20(1):1–12.

    Article  Google Scholar 

  11. Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil H. Chapter 11: scoping reviews (2020 version). JBI manual for evidence synthesis, JBI. 2020;2020.

  12. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

    Article  Google Scholar 

  13. Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):1–9.

    Article  Google Scholar 

  14. Khalil H, Peters M, Godfrey CM, McInerney P, Soares CB, Parker D. An evidence-based approach to scoping reviews. Worldviews on Evidence-Based Nursing. 2016;13(2):118–23.

    Article  PubMed  Google Scholar 

  15. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

    Article  PubMed  Google Scholar 

  16. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic reviews. 2021;10(1):1-11.

  17. Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme Version. 2006;1:b92.

  18. Kearney A, Daykin A, Shaw ARG, Lane AJ, Blazeby JM, Clarke M, et al. Identifying research priorities for effective retention strategies in clinical trials. Trials. 2017;18(1):406.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Murphy E, Shiely F, Treweek S. How much is the lack of retention evidence costing trial teams in Ireland and the UK? Trials. 2022;23(1):396.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Patient and public involvement

No patient or public involvement.

Amendments

The protocol will be closely followed during the scoping review; due to the iterative nature of scoping reviews, any amendments that are made to the published protocol will be reported in the final publication.

Funding

This research forms part of a doctoral research project supported by the Health Research Board Trial Methodology Research Network (HRB-TMRN) PhD scholarship awarded to EM. The funder had no role in the design, data collection, and analysis or preparation of the protocol.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to developing the review question and the review design. EM designed and drafted the protocol and is the guarantor of this protocol. FS and KG reviewed the protocol. All authors read and approved the final version of the protocol.

Corresponding author

Correspondence to Ellen Murphy.

Ethics declarations

Ethics approval and consent to participate

As this will be a scoping review of previously published protocols, no ethical approval or consent to participate is required.

Consent for publication

Not applicable as this protocol does not contain any individual persons’ data.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Murphy, E., Gillies, K. & Shiely, F. How do trial teams plan for retention during the design stage of the trial? A scoping review protocol. Trials 23, 944 (2022). https://doi.org/10.1186/s13063-022-06866-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-022-06866-w

Keywords