- Study protocol
- Open Access
How do trial teams plan for retention during the design stage of the trial? A scoping review protocol
Trials volume 23, Article number: 944 (2022)
Retention remains a major challenge for many clinical trials. The SPIRIT guidelines state the following information on retention should be included in the trial protocol “Plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols”. This guidance shows the importance of planning retention methods and handling missing data as this can impact how the results of the trial are interpreted. The most recent Cochrane review of strategies to improve retention in clinical trials highlighted that some trials implemented multiple retention strategies and we questioned whether the use of multiple strategies was planned at the design stage and included in the protocol or are strategies implemented when retention becomes an issue within the trial. The purpose of our scoping review is to establish if and how trial teams prepare for retention at the design phase of clinical trials.
Methods and analysis
We will follow the methodological framework and guidelines for scoping reviews outlined by the Joanna Briggs Institute. We will search MEDLINE/PubMed, Scopus, EMBASE, CINAHL (EBSCO), and Web of Science. A comprehensive search strategy for PubMed was developed in collaboration with an experienced research librarian. We will include protocols for phase 2, 3, and 4 RCTs as well as pilot and feasibility studies. The screening process will involve two reviewers. EM will independently screen all titles and abstracts. FS will screen 10% of the overall search output, and where necessary full protocol texts will be screened to determine eligibility. We will randomly sample eligible protocols to ensure the protocols represent a variety of trial and intervention types. Data will be extracted from each protocol and the results will be synthesised. The analysis will be qualitative using a narrative summary and descriptive statistics where appropriate.
The scoping review will help trial methodologists better understand if retention strategies are planned for during the design stage of the trial contributing to the PRioRiTy II unanswered question “How should people who run trials plan for retention during their funding application and creation of the trial (protocol development)?”.
Randomised controlled trials (RCTs) are conducted in accordance with a trial protocol. Having a protocol that is incomplete and non-transparent makes it difficult to critically appraise the trial . Protocols are needed for the readers of the corresponding paper to be able to fully appraise and interpret the results of the trial . As per the 2013 SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) definition, a protocol is “a document that provides sufficient detail to enable understanding of the background, rationale, objectives, study population, interventions, methods, statistical analyses, ethical considerations, dissemination plans, and administration of the trial, replication of key aspects of trial methods and conduct; and appraisal of the trial’s scientific and ethical rigour from ethics approval to dissemination of results” (3;202). Prior to the introduction of SPIRIT, the content and quality of protocols differed greatly [3, 4], but the 33-item SPIRIT checklist for the minimum recommended protocol items improved this. Having a clearly written protocol increases transparency in trial conduct. Protocols are usually published prior to the trial paper publication . However, this is not always the case, and some protocol publications will require payment for access. For example, of the cancer clinical trials published in January of 2020 (n = 113), only 11.3% had a publicly accessible protocol that was not behind a paywall . This further limits transparency and hinders replication in trial methods and conduct, which has been recommended for trial retention strategies in order to improve the evidence base for their effectiveness .
Retention remains a major challenge for many clinical trials . The SPIRIT guidelines recommend the following information on retention be included in the trial protocol in Sect. 18b “Plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols” (8;3). These of course are minimum requirements but the information can impact how missing data is dealt with and interpreted in the analysis of the trial .
A recent Cochrane systematic review of strategies to improve retention in clinical trials found that there were no strategies for which the quality of evidence was high that showed improved retention. The review also highlighted that some trials implement multiple retention strategies , and thus we questioned whether the use of multiple strategies is planned at the design stage and included in the protocol or are strategies implemented when retention becomes an issue within the trial. Evidence from a small number of interviews with trial staff is variable . Prospective retention planning informs appropriate costing and implementation of retention strategies but also increases transparency in trial conduct.
The purpose of our scoping review is to establish if and how trial teams prepare for retention at the design phase of clinical trials. This will contribute to the evidence base for the PRioRiTy II (Prioritising Retention in Randomised Trials) unanswered question “How should people who run trials plan for retention during their funding application and creation of the trial (protocol development)?” .
The scoping review will be conducted in accordance with the guidelines and framework outlined by the Joanna Briggs Institute (JBI) , the most recent framework for scoping reviews which builds on prior work [12,13,14]. The framework consists of the following nine steps:
Defining and aligning the objective/s and question/s.
Developing and aligning the inclusion criteria with the objective/s and question/s.
Describing the planned approach to evidence searching, selection, data extraction and presentation of the evidence.
Searching for the evidence.
Selecting the evidence.
Extracting the evidence.
Analysis of the evidence.
Presentation of the results.
Summarising the evidence in relation to the purpose of the review, making conclusions and noting any implications of the findings.
The guidance from the Joanna Briggs Institute  along with the newly developed reporting guidelines for scoping reviews: the Preferred Reporting Items for a Systematic Review and Meta-Analysis Protocols Extension for Scoping Reviews (PRISMA-ScR)  has been consulted in the development of this protocol (Additional File 1).
Data sources and search strategy
A comprehensive search strategy for PubMed was developed in collaboration with an experienced research librarian at University College Cork and is shown below. The following electronic databases will be searched for relevant protocols, MEDLINE/PubMed, Scopus, EMBASE, CINAHL (EBSCO), and Web of Science. The search will be adapted as appropriate for each database using the software Polyglot which translates search strategies across databases. The search and screening process will take place over a six-week period.
Example of PubMed search
(“randomised controlled trial”[Title/Abstract]) OR (“randomized controlled trial”[Title/Abstract])) OR (“randomised clinical trial”[Title/Abstract])) OR (“randomized clinical trial”[Title/Abstract])) OR (“randomized controlled trials as topic”[MeSH Terms])) AND (“protocol”[Title/Abstract])) OR (“clinical trial protocols as topic”[MeSH Terms]).
This is the full search strategy developed for PubMed. The limits that will be applied include published in the English language between the years of 2014 and 2019 (inclusive). No other limits will be applied to the search.
The population include the following: protocols of RCTs that include adults and/or children of any age.
The concepts includes the following: protocols of RCTs investigating any treatment/intervention type for any disease area. Randomisation can be at the cluster or individual level. The concepts also include protocols of RCTs investigating any comparator including placebo and examining any outcome.
The context of this scoping review is open sources of evidence pertaining to any contextual setting.
Types of evidence sources
We will include trial protocols published in the English language. We will include protocols from 2014 to 2019 (inclusive), giving sufficient time to see the effect of the SPIRIT guidelines, which were published in 2013. We will include protocols for phase 2, 3, and 4 RCTs as well as pilot and feasibility studies.
Screening and selection process
EM will systematically collate and import titles and abstracts of all electronically sourced search results to EndNote, grouping results separately for each database. Duplicates will be removed, and the remaining results will be exported to the Rayyan QCRI software for screening. The screening process will involve two reviewers (EM and FS). EM will independently screen all titles and abstracts. FS will screen a random selection of 10% of the overall search output. If there is less than 80% agreement on the random 10% of the search output that is double screened, we will undertake screening of a further 10% validation sample. We will reiterate this until > 80% agreement on the validation sample is reached. If disagreement arises between these two reviewers regarding the eligibility of a protocol, a third reviewer KG will be consulted, and where necessary full protocol texts will be screened to determine eligibility. The protocol screening and selection process will be presented both narratively and graphically using a PRISMA (Transparent Reporting of Systematic Reviews and Meta-Analyses) flow diagram . Details of excluded sources at full-text review will be outlined and included in the review with reasons for the protocol exclusion.
We will select a random sample of 10% of eligible protocols for data extraction; the random sample will be chosen by including blocks of 10 protocols from the list of eligible protocols for inclusion. A randomly chosen 10% will provide a sufficient representation of all eligible protocols.
Data management and data charting (extraction) process
Following screening for eligibility all data on retention plans will be extracted. Data to be extracted is outlined in Table 1. The main outcome of interest is whether SPIRIT item 18b is satisfied (Table 1, variable 7), since this statement has sub-requirements during the data charting process we will divide the statement into three aspects “plans to promote participant retention” and “plans to complete follow-up, including list of any outcome data to be collected for participants who discontinue from intervention protocols” and “plans to complete follow-up, including list of any outcome data to be collected for participants who deviate from intervention protocols”. Dividing the statement into sub-statements will more accurately reflect whether SPIRIT item 18b is satisfied or not. With regard to retention strategies we specifically mean an action/activity that is conducted with the purpose of reducing missing data/improving data completeness, we will not include activities to improve adherence or compliance to an intervention. Where there is ambiguity on the purpose of specific actions, i.e. if they are for participant retention or not, the protocol authors will be contacted. Where contact is not possible/a reply is not received, the ambiguous information will be highlighted and included in the data extraction. Prior to the full data charting process, the data extraction form will be piloted using a sample of 10 protocols as the variables to be extracted may need to be refined and improved to best meet the objectives of the scoping review. Data charting will be carried out by one reviewer (EM) and a random sample (10%) of the protocols will be double extracted and checked for consistency by (FS) to ensure consistency and improve the reliability of the data extraction process. If there is less than 80% agreement regarding the data extracted in the 10% random sample of protocols, we will undertake data extraction of a further 10% validation sample. We will reiterate this until > 80% agreement on the validation sample is reached. Data charting will be conducted, and information will be entered into a Microsoft Excel file.
We will synthesise the data in a narrative synthesis with descriptive statistics where appropriate. The descriptive statistics that will be generated will include the frequencies and percentages of the trial characteristics such as the percentage of trials conducted in different disease areas, percentage of trials conducted among vulnerable populations, percentage of trials that are publicly or privately funded, and the percentage of drug trials, surgical trials, non-drug/behavioural intervention, and medical device trials, modes of follow-up methods used by trials, and number of trials with patient-reported primary outcomes vs non-patient-reported outcomes. We will also generate the percentage of trial protocols that have satisfied the SPIRIT item 18b statement.
Since we will include pilot and feasibility trial protocols, we plan to conduct a sub-analysis of this group of studies as one of the purposes of pilot and feasibility trials may be to develop and test retention strategies.
The content will be analysed to determine if the protocol complies with the SPIRIT guidelines for retention strategies Sect. 18b- “Plans to promote participant retention and complete follow-up including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols” (8;3). We will divide the statement into three aspects “plans to promote participant retention” and “plans to complete follow-up including list of any outcome data to be collected for participants who discontinue from intervention protocols” and “plans to complete follow-up including list of any outcome data to be collected for participants who deviate from intervention protocols”. This will be recorded as either “yes” or “no” for each separate aspect. If the protocols include information regarding planned retention strategies, this information will be analysed in the narrative synthesis. The narrative synthesis approach will be based on the Guidance on the Conduct of Narrative Synthesis in Systematic Reviews .
Presentation of findings
We will present the search results in a PRISMA flow diagram illustrating the total number of protocols generated by the search strategy and the number of protocols excluded following the application of the inclusion/exclusion criteria and ultimately the number of protocols included in the scoping review. Summary tables will depict the study characteristics described in the included protocols. Additional tables, figures, and narrative descriptions will illustrate the data addressing our research question. The findings of the scoping review will be disseminated via publication in a peer-reviewed journal.
Conducting a scoping review of randomised controlled trial protocols to investigate if retention strategies are planned by trial teams during the design stage of the trial will contribute to the PRioRiTy II unanswered question “How should people who run trial plan for retention during their funding application and creation of the trial (protocol development)?” . The strength of this review is the intention to include a variety of trial types as described in the methods section. The review will highlight the gaps that exist in the planning and communicating of retention strategies in clinical trial protocols, which may have the potential to increase transparency in trial conduct and make trials more efficient as it will highlight to researchers the lack of consideration and/or communication of trial retention strategies during the initial planning and design stages of the trial. Foreplaning could increase retention strategy success as trial teams will be able to spend greater time researching the most effective strategies in terms of retaining participants and carefully consider the resources required as retention strategies can be very expensive to implement often with little reward in terms of retaining participants [6, 18, 19]. The findings of this review have the potential to inform trialists to plan retention strategies during the design stage of the trial rather than when retention issues appear. The review will also help identify questions for future research projects and findings from this review can be explored in more depth in future research investigating retention strategies in clinical trials.
Availability of data and materials
Not applicable as this is a protocol for a scoping review.
Randomised controlled trials
Standard Protocol Items: Recommendations for Interventional Trials
Prioritising Retention in Randomised Controlled Trials
Preferred Reporting Items for a Systematic Review and Meta-Analysis Protocols Extension for Scoping Reviews
Transparent Reporting of Systematic Reviews and Meta-Analyses
Tetzlaff JM, Moher D, Chan A-W. Developing a guideline for clinical trial protocol content: Delphi consensus survey. Trials. 2012;13(1):1–10.
Babu C, Mell L, Lee N, Zakeri K. Public access to protocols of contemporary cancer randomized clinical trials. Trials. 2021;22(1):1–4.
Chan A-W, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200–7.
Tetzlaff JM, Chan A-W, Kitchen J, Sampson M, Tricco AC, Moher D. Guidelines for randomized clinical trial protocol content: a systematic review. Syst Rev. 2012;1(1):1–11.
Chan A-W, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–65.
Gillies K, Kearney A, Keenan C, Treweek S, Hudson J, Brueton VC, et al. Strategies to improve retention in randomised trials. Cochrane Database Syst Rev. 2021;3(3):MR000032.
Walters SJ, Bonacho Dos AnjosHenriqus-Cadby I, Bortolami O, Flight L, Hind D, Jacques RM, et al. Recruitment and retention of participants in randomised controlled trials: a review of trials funded and published by the United Kingdom Health Technology Assessment Programme. BMJ open. 2017;7(3):e015276-e.
Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, SPIRIT, et al. explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;2013:346.
Daykin A, Clement C, Gamble C, Kearney A, Blazeby J, Clarke M, et al. ‘Recruitment, recruitment, recruitment’ – the need for more focus on retention: a qualitative study of five trials. Trials. 2018;19(1):76.
Brunsdon D, Biesty L, Brocklehurst P, Brueton V, Devane D, Elliott J, et al. What are the most important unanswered research questions in trial retention? A James Lind alliance priority setting partnership: the PRioRiTy II (Prioritising Retention in Randomised Trials) study. Trials. 2019;20(1):1–12.
Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil H. Chapter 11: scoping reviews (2020 version). JBI manual for evidence synthesis, JBI. 2020;2020.
Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.
Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):1–9.
Khalil H, Peters M, Godfrey CM, McInerney P, Soares CB, Parker D. An evidence-based approach to scoping reviews. Worldviews on Evidence-Based Nursing. 2016;13(2):118–23.
Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic reviews. 2021;10(1):1-11.
Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme Version. 2006;1:b92.
Kearney A, Daykin A, Shaw ARG, Lane AJ, Blazeby JM, Clarke M, et al. Identifying research priorities for effective retention strategies in clinical trials. Trials. 2017;18(1):406.
Murphy E, Shiely F, Treweek S. How much is the lack of retention evidence costing trial teams in Ireland and the UK? Trials. 2022;23(1):396.
Patient and public involvement
No patient or public involvement.
The protocol will be closely followed during the scoping review; due to the iterative nature of scoping reviews, any amendments that are made to the published protocol will be reported in the final publication.
This research forms part of a doctoral research project supported by the Health Research Board Trial Methodology Research Network (HRB-TMRN) PhD scholarship awarded to EM. The funder had no role in the design, data collection, and analysis or preparation of the protocol.
Ethics approval and consent to participate
As this will be a scoping review of previously published protocols, no ethical approval or consent to participate is required.
Consent for publication
Not applicable as this protocol does not contain any individual persons’ data.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Murphy, E., Gillies, K. & Shiely, F. How do trial teams plan for retention during the design stage of the trial? A scoping review protocol. Trials 23, 944 (2022). https://doi.org/10.1186/s13063-022-06866-w
- Scoping review
- Randomised controlled trial protocols
- Retention strategies
- Trial design stage