How do trial teams plan for retention during the design stage of the trial? A scoping review

Abstract

Background

Retention to trials is important to ensure the results of the trial are valid and reliable. The SPIRIT guidelines (18b) require “plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols” be included in trial protocols. It is unknown how often protocols report this retention information. The purpose of our scoping review is to establish if, and how, trial teams report plans for retention during the design stage of the trial.

Materials and methods

A scoping review with searches in key databases (PubMed, Scopus, EMBASE, CINAHL (EBSCO), and Web of Science from 2014 to 2019 inclusive) to identify randomised controlled trial protocols. We produced descriptive statistics on the characteristics of the trial protocols and also on those adhering to SPIRIT item 18b. A narrative synthesis of the retention strategies was also conducted.

Results

Eight-hundred and twenty-four protocols met our inclusion criteria. RCTs (n = 722) and pilot and feasibility trial protocols (n = 102) reported using the SPIRIT guidelines during protocol development 35% and 34.3% of the time respectively. Of these protocols, only 9.5% and 11.4% respectively reported all aspects of SPIRIT item 18b “plans to promote participant retention and to complete follow-up, including list of any outcome data for participants who discontinue or deviate from intervention protocols”.

Of the RCT protocols, 36.8% included proactive “plans to promote participant retention” regardless of whether they reported using SPIRIT guidelines or not. Most protocols planned “combined strategies” (48.1%). Of these, the joint most commonly reported were “reminders and data collection location and method” and “reminders and monetary incentives”. The most popular individual retention strategy was “reminders” (14.7%) followed by “monetary incentives- conditional” (10.2%). Of the pilot and feasibility protocols, 40.2% included proactive “plans to promote participant retention” with the use of “combined strategies” being most frequent (46.3%). The use of “monetary incentives – conditional” (22%) was the most popular individual reported retention strategy.

Conclusion

There is a lack of reporting of plans to promote participant retention in trial protocols. Proactive planning of retention strategies during the trial design stage is preferable to the reactive implementation of retention strategies. Prospective retention planning and clear communication in protocols may inform more suitable choice, costing and implementation of retention strategies and improve transparency in trial conduct.

Background

Retention of participants to trials is an ongoing challenge with little evidence to support what works and what does not work [1]. The most recent Cochrane systematic review of strategies to improve retention in trials found that there were no strategies that improved retention for which the quality of evidence was high. Despite this, many are used in trials frequently [1, 2] with some trials evaluating multiple retention strategies simultaneously [1]. We recently conducted a study to investigate how much the most routinely used trial retention strategies cost trial teams in the UK and Ireland. Even when calculated conservatively, the financial cost is staggering [3]. Even more staggering, it is estimated that roughly 50% of trials experience loss-to-follow-up of at least 11% with some experiencing loss-to-follow-up rates as high as 20% [1, 4]. Higher rates of loss-to-follow-up are shown to be associated with longer length of follow-up [5]. These statistics suggest that trial teams should consider retention strategies at the design stage, before the trial begins, particularly for trials at higher risk of loss-to-follow-up.

Poor retention causes bias to be introduced into the trial [5] and reduces the power of the trial which means the ability to detect significant findings and the confidence in the conclusions drawn from the trial are both affected [5,6,7]. Poor retention also results in incomplete data, it can delay the delivery of interventions and increase the costs associated with running trials [8]. This contributes to research waste [8,9,10]. Missing data and poor retention can be dealt with by statistical techniques in the analysis of the trial [5, 6], but no missing data technique is as good as retaining the participant and having complete data. Recruiting larger numbers of participants to counteract the expected dropout rate is also used to mitigate missing data, but this is more expensive and exposes more people to the risks associated with trial participation [11]. Rather than dealing with the problem after it occurs, trial teams could/should be looking to factor in plans to mitigate poor retention at design stages—a question identified as a priority for research by the trials community [12].

Trial protocols are an essential document for planning and conducting the trial. Protocols are reviewed and approved by ethics committees before the trial begins to ensure the trial team has fully accounted for any potential issues that may arise during the course of the trial [13]. Having a comprehensive clearly written protocol increases the transparency in trial conduct [13, 14] and allows for the replication of trial methods [14]. Protocols need to be published and be freely assessable for the readers of the corresponding results paper to fully appraise and interpret the results of the trial [15, 16]. Despite the importance of trial protocols, research shows that the content of protocols varies greatly [13, 14]. They often fail to report, in sufficient detail, some key trial design elements such as the primary outcome of the trial [17], statistical methods [18], and allocation concealment [19]. Deficiencies in protocol content may result in trial teams seeking ethical amendments, and may lead to poor trial conduct [13].

According to ICH GCP guidance, there is no requirement or recommendation that retention strategies be included in trial protocols but it does recommend that protocols should specify “the type and timing of the data to be collected for withdrawn subjects” ([20]:40). The new Clinical Trial Regulation [21] makes no comment on retention either. However, the U.S. Food and Drug Administration (FDA) recommend that preventing poor retention needs to be dealt with by improving trial design and trial conduct [22]. One document, developed to improve the completeness and reporting of content of trial protocols, SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) was developed in 2013. The SPIRIT statement is a 33-item checklist for minimum protocol content that aims to promote and improve the transparency and description of trial activities by encouraging trial teams to consider potential and important issues during the design stage of the trial [13]. One of these issues is retention, and as per the SPIRIT 2013 statement it is recommended that the following be included “plans to promote participant retention and complete follow-up including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols” ([14]:3). Existing evidence, from interviews with trial staff, as to whether trial teams actually prospectively plan retention strategies during the trial design stage is variable [23].

The purpose and primary aim of our scoping review is to establish if, and how, trial teams report plans for retention at the design stage of clinical trials by examining a body of trial protocols. Our secondary aim is to compare the reported retention strategies with their evidence of effect. This will contribute to the evidence base for the PRioRiTy (Prioritising Retention in Randomised Controlled Trials) unanswered question “How should people who run trials plan for retention during their funding application and creation of the trial (protocol development)?” [12].

Materials and methods

This scoping review has been conducted using the guidelines and framework outlined by the Joanna Briggs Institute [24], the most recent framework for scoping reviews [25,26,27]. This scoping review was reported using the Preferred Reporting Items for a Systematic Review and Meta-Analysis Protocols Extension for Scoping Reviews (PRISMA-ScR) [28] (Supplementary File 1). The protocol for this review is published in Trials and is also available in Supplementary File 2.

Data sources and search strategy

The search strategy was developed in collaboration with a research librarian at University College Cork and is shown in Table 1. The following electronic databases were searched for relevant protocols, PubMed, Scopus, EMBASE, CINAHL (EBSCO), and Web of Science. The search was adapted as appropriate for each database using the software Polyglot [29] which translates search strategies across databases.

Table 1 PubMed search strategy

Inclusion and exclusion criteria

We included the following: protocols for phase II, phase III and phase IV randomised controlled trials (RCTs), pilot and feasibility trials, and mixed methods studies that included a RCT element; protocols published between 2014 and 2019 (inclusive)—we chose this timeline to allow for sufficient time for the uptake of the SPIRIT guidelines published in 2013 [14] and then included a 6 year time-horizon, as this would provide a sufficient sample size; protocols of RCTs from any setting, that involved adults and/or children of any age, investigating any treatment/intervention type for any disease area/clinical specialty, investigating any comparator including placebo and examining any outcome; protocols for trials randomised at the cluster or individual level; protocols published in the English language. Excluded were as follows: non-protocol papers; protocols for non-randomised trials; protocols for quasi/partially randomised trials; protocols for single-arm trials; protocols for studies within a trial (SWATs); protocols for statistical analysis plans; protocols for phase 1 trials; protocols for process evaluations; protocols for economic evaluations; protocols for N-of-1 trials.

Screening and selection process

EM imported titles and abstracts of all electronically sourced search results to EndNote, grouping results separately for each database. Duplicates were removed and the remaining results were exported to Rayyan QCRI software for screening. The screening process involved two reviewers (EM and FS). EM independently screened all titles and abstracts. FS screened a random selection of 10% of the overall search output, this random 10% was selected using a random number generator. Where disagreement arose, a third reviewer KG was consulted, and when necessary, full protocol texts were obtained to determine eligibility. We set ourselves a target of 10% of the eligible protocols (n = 8244). We wanted a sample that was large enough to say something meaningful, but small enough to facilitate completion. Ten percent (n = 824) seemed a reasonable compromise between size and feasibility. See Fig. 1 for The PRISMA flow diagram.

Fig. 1

PRISMA flow diagram. Diagram showing the number of protocols included at each stage of the screening process of the scoping review

Data management and data extraction process

The full list of extracted variables, discussed and agreed upon by all authors, are outlined in the protocol (Supplementary File 2). The data extraction form was piloted by EM using a sample of 10 protocols and was reviewed by FS and KG to ensure the variables extracted best met the objectives of the scoping review. Data extraction was performed by EM and a random sample (10%) of the protocols was selected by FS using a random number generator, and checked to ensure consistency and improve the reliability of the data extraction process. All extracted information was entered into a Microsoft Excel file.

For the purposes of this scoping review, we defined a retention strategy as an action/activity that is conducted, in addition to usual follow-up procedures, with the purpose of retaining participants in a trial, reducing missing data or improving data completeness. We were not concerned with extracting information regarding activities to improve adherence or compliance to an intervention.

The outcome of interest was adherence to SPIRIT item 18b, “Plans to promote participant retention and complete follow-up including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols” ([14]:3). We divided this statement into three categories; 18b(i) “plans to promote participant retention”, 18b(ii) “plans to complete follow-up including list of any outcome data to be collected for participants who discontinue from intervention protocols” and 18b(iii) “plans to complete follow-up including list of any outcome data to be collected for participants who deviate from intervention protocols”. For 18b(i), we defined this as proactive plans outlined in the protocol that aim to actively promote participant retention in the trial. For 18b(ii) and 18b(iii), we defined these as reactive plans outlined by the trial team to complete outcome data collection and to complete follow-up of participants who have withdrawn/discontinued or deviated from the intervention protocols.

Regardless of whether the protocol reported using the SPIRIT guidelines in the protocol, we analysed all protocols for information that we could map to SPIRIT item 18b “Plans to promote participant retention and complete follow-up including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols” ([14]:3). Which as described above we divided into three categories.

Data synthesis

We produced descriptive statistics on the characteristics of the trial protocols and on those adhering to SPIRIT item 18b. A narrative synthesis of the retention strategies was also conducted. The Guidance on the Conduct of Narrative Synthesis in Systematic Reviews were referred to during this process [30]. The retention strategies were coded by EM based on the type of retention strategy, i.e., reminders, prompts, monetary incentives. We mapped each strategy to the ORRCA (Online Resource for Research in Clinical triAls) retention domains [31] (ORRCA_Retention_Domains.pdf) as has previously been done in the Cochrane systematic review of strategies to improve retention in randomised trials [1]. The data items were mapped to 16 out of the 44 ORRCA domains, the most popular category being “B. Participants”. A total breakdown of the number of protocols mapped to each ORRCA domain can be seen in Table 4 along with example quotes that were mapped to each domain.

When it was unclear which ORRCA domain, a retention strategy mapped to FS and KG were consulted and a joint decision was made. Fifty of 307 data items were consulted upon. We made the following assumptions when mapping the retention strategies.

• (1) Following the approach taken by the Cochrane review [1], ORRCA domain “B1 Reminders” was divided into reminders (sent after a missed data collection time point) and prompts (sent before the data collection time point). In some cases, it was not clear if the strategy was intended as a reminder or prompt therefore based on the wording of the surrounding text we made a judgement call as to whether it was a reminder or prompt. If it was still unclear from this and the strategy included the word reminder/prompt, we mapped it to ORRCA as such.

• (2) When a protocol outlined the use of more than one retention strategy, we created a new category called “combined strategies”. We have detailed the most commonly combined strategies in Table 4.

• (3) Regarding monetary and non-monetary compensation for participants, we made the assumption that all monetary compensation functioned as incentives rather than rewards. Our reasoning for this is that ethically, patient information leaflets must disclose to participant information about receiving monetary compensation [20]. This prior knowledge of receiving monetary compensation means the compensation functions as an incentive rather than a reward.

• (4) We further classified monetary and non-monetary incentives as either conditional (based on the participant completing a task) or non-conditional (not based on participants completing a task).

Since we included pilot and feasibility trial protocols, we conducted a sub-group analysis of the retention strategies outlined in these protocols. The results from the pilot and feasibility protocols will be reported separately (“Analysis of pilot and feasibility trial protocols”). All other protocols that were not pilot and feasibility trials are included in what we will refer to as RCT protocols throughout the paper and Supplementary File 3.

Results

Protocol characteristics

Table 2 displays the characteristics of the protocols included in our analysis. In summary, of the 824 protocols included in the 6-year period: 26.6% (n = 219) were published in 2019; most trials were non-commercial, i.e. publicly funded trials that did not receive funding/donations from private for-profit companies (80.8%, n = 666); tested non-drug interventions such as diet, exercise, therapy, and educational interventions (72%, n = 593). Individually randomised designs dominated (84%, n = 692). Thirty-point-three percent (n = 250) of protocols were for trials conducted in vulnerable populations and 22% (n = 181) were for trials conducted among populations consisting of both vulnerable and non-vulnerable individuals. Vulnerable populations were defined by this reviews’ authors via local ethics committee [32] and ICH GCP definitions [20] these included; infants and children aged 17 and under, pregnant women, institutionalised individuals, adults aged 60 and over, critically ill patients not able to consent for themselves, homeless individuals and refugees, see Table 2 for full definition of all included populations. The protocols covered a wide range of clinical specialties, 38 in total, including oncology, musculoskeletal diseases, cardiology, neurology, nephrology, and obstetrics and gynaecology. The topic of public health was the most common (15.9%, n = 131). This category included trials evaluating interventions targeting for example physical activity, nutrition, smoking cessation, alcohol/drug misuse, gambling, obesity, sleep disorders, and family planning and contraception.

Table 2 Protocol characteristics (total protocols = 824)

Compliance with the SPIRIT 2013 Statement

Table 3 reports the key findings relevant to the use of the applicable retention items from the SPIRIT 2013 statement for RCT protocols (n = 722). We report separately on the pilot and feasibility protocols (n = 102). (A more detailed breakdown is provided in Supplementary File 3).

Of the 35% (n = 253) of RCT protocols that reported using the SPIRIT guidelines when developing the protocol, 9.5% (n = 24) fully complied and included all aspects of item 18b (18b(i) and 18b(ii) and/or 18b(iii)), and 41.5% (n = 105) included item 18b(i) “plans to promote participant retention” (proactive rather than reactive plans).

Table 3 Key SPIRIT 2013 statement results^a

Plans to promote participant retention

Of the RCT protocols (n = 722) regardless of reporting using the SPIRIT guidelines, 7.3% (n = 53) included all aspects of item 18b (18b(i) and 18b(ii) and/or 18b(iii)), “plans to promote participant retention and complete follow-up, including list of any outcome data to be collected from participants who discontinue or deviate from intervention protocols” ([14]:3).

SPIRIT item 18b(i)

SPIRIT item 18b(i) “plans to promote participant retention” (proactive plans) was included in 36.8% (n = 266) of the RCT protocols, regardless of whether they reported using SPIRIT guidelines or not. The most common retention strategy was the use of “combined strategies” used in 48.1% of protocols (n = 128). The joint most popular combined retention strategies were the use of “reminders and data collection location and method” (e.g. use of return postage such as pre-paid stamped return envelopes, options of home visits/telephone/postal data collection versus clinic visits), and “reminders and monetary incentives”. The median number of retention strategies used in a singular protocol was 3. The highest number of strategies reported in any one protocol was 9.

In terms of individual retention strategies, the most common was “reminders” (14.7%, n = 39) followed by “monetary incentives-conditional” (10.2%, n = 27). Some of the least popular methods included “maintaining staff engagement” (0.4%, n = 1), and “monetary incentives – unconditional” (0.4%, n = 1).

Table 4 summarises the ORRCA domains mentioned in the protocols along with sample quotes from protocols. The most frequently used combined strategies are provided at the bottom of Table 4. A full list of all combinations of combined retention strategies can be found in Supplementary File 3.

Table 4 Plans to proactively promote participant retention (n = 266, 36.8%)^a

SPIRIT items 18b(ii) and 18b(iii)

A combined total of 13.7% (n = 99) of the 722 RCT protocols considered a reactive plan to collecting outcome data (SPIRIT item 18(ii) and/or item 18b(iii)), “plans to complete follow-up including list of any outcome data to be collected for participants who discontinue and/or deviate from intervention protocols”, regardless of reporting using SPIRIT guidelines or not. No strategy actively targeted those that might withdraw from the trial, i.e. strategies typically employed in 18b(ii) and 18b(iii) were seeking consent early in the trial for continued use of the data if they withdrew or deviated from the protocol. A full breakdown of results can be found in Supplementary File 3.

Analysis of pilot and feasibility trial protocols

Of the 824 trial protocols, 12.4% (n = 102) were for pilot and feasibility trials. Of these, 34.3% (n = 35) used the SPIRIT statement during protocol development. Of those 35 protocols, 11.4% (n = 4) included all three parts of SPIRIT item 18b (18b(i) and 18b(ii) and/or 18b(iii)) [14].

Overall, 40.2% (n = 41/102) included item 18b(i) “plans to promote participant retention”, a proactive plan to promote retention, regardless of whether they reported using SPIRIT guidelines or not. A combined total of 14.7% (n = 15) of protocols included a reactive plan (SPIRIT items 18b(ii) and/or 18b(iii)) “plans to complete follow-up for those who discontinue and/or deviate from the intervention protocols”. A total of 6.9% (n = 7) protocols reported all aspects of SPIRIT item 18b (18b(i) and 18b(ii) and/or 18b(iii)) [14].

A total breakdown of the number of pilot and feasibility protocols mapped to each ORRCA domain can be seen in Table 5 along with examples of quotes that were mapped to each domain. The top combined strategies are also shown in Table 5. See Supplementary File 3 for full details of all combinations of combined retention strategies.

Table 5 Analysis of Pilot and Feasibility trial protocols

Evidence to support the use of retention strategies

Table 6 displays the top 10 most popular retention strategies identified in this review mapped to the evidence of their effectiveness from the Cochrane review of strategies to improve retention in randomised controlled trials [1]. The Cochrane review defines these strategies as “those designed to generate maximum data return or compliance and follow-up procedures that aim to collect data from participants” ([1]:22). The evidence to support the use of these strategies is either lacking entirely or in the majority of cases has a low-GRADE certainty rating. GRADE (Grading of Recommendations, Assessment, Development and Evaluations) is the most widely adopted tool for grading the quality of evidence and for making clinical practice recommendations and is endorsed by Cochrane.

Table 6 Most popular retention strategies compared against evidence for effectiveness

Discussion

The protocols included in this review covered a wide variety of clinical specialties, intervention types, sample sizes, patient populations, numbers and modes of participant follow-up. The overall reporting of the use of SPIRIT guidelines [14] during protocol development was low, with only 35% of RCT protocols and 34.3% of pilot and feasibility trial protocols reporting its use when developing the protocol. The SPIRIT guidelines were published in 2013 and taking this into consideration our search started in 2014 allowing a year for guideline uptake. Despite this, our results show the reporting of the use of SPIRIT guidelines when developing protocols is still low. Given the endorsement of SPIRIT by many journals such as BMJ, The Lancet, and JAMA, and by Biomed Central Journals, we believe this is a reporting issue rather than an implementation issue. Though the level of endorsement varies, either through general support for SPIRIT, encouraging protocol authors to use SPIRIT when developing protocols or explicitly requiring protocols to adhere to SPIRIT [33], which is seen in journals such as Trials and PLOS ONE, trialists are inevitably aware of SPIRIT, thus confirming our view that reporting of SPIRIT is poor, whilst implementation of SPIRIT is undoubtedly better than we were able to provide evidence for in this scoping review.

We were particularly interested in the reporting of item 18b which relates to trial retention. Adherence to this was quite low, suggesting that though trialists report using SPIRIT, reporting on retention is very poor. Of the 35% of RCT protocols that reported using SPIRIT, there was incomplete reporting of item 18b. Only 9.5% (n = 24) of these protocols fully complied and included all aspects of this item, “plans to promote participant retention and to complete follow-up, including list of any outcome data for participants who discontinue or deviate from intervention protocols” ([14]:3). The poor reporting of plans to promote participant retention in trial protocols could be because trial teams are initially worried about recruitment rates meaning retention is not a priority during planning but a reaction during conduct. It is also difficult to plan retention strategies when there is no evidence to support using most strategies [1], possibly lending itself into the issue of retention being considered after the fact. Additionally, strategies used to promote retention such as building relationships and maintaining rapport between trial staff and participants [23, 34] may not be reported in protocols as they may be considered more informal strategies [23] that may be difficult to plan, report and evaluate [34]. Furthermore, the lack of reporting poses issues for replication, trial teams may have plans to actively promote participant retention however due to poor reporting, these plans cannot be replicated for evaluation in the future.

Regardless of reporting using SPIRIT guidelines, out of the total 722 protocols for RCTs, only 36.8% reported a proactive plan to actively promote participant retention, meaning 63.2% of protocols did not consider proactively tackling the issue of retention during protocol development. Of the 102 pilot and feasibility protocols, only 40.2% outlined a proactive plan to promote participant retention. This lack of consideration for retention strategies during the design stage of the trial could be due to the emphasis on recruitment or other research priorities. Previous research shows there is still a stronger emphasis on recruitment more so than retention in trials [23, 35]. Trial staff believe reasons for this include funders and research networks place more emphasis on recruitment targets as trial performance is often based on recruitment rates [23]. Additionally, there are statistical methods used in trials to predict outcomes for individuals who have not been retained based on the available data for these participants [6], a possible factor as to why retention may not be considered as important as recruitment. This emphasis means that recruitment is incorporated into specific staff roles and trial teams may not be sufficiently informed about retention strategies [35]. Retention is a widespread issue of concern within trials [1, 10] and poor retention rates should not be a surprise to trial teams; therefore, trial teams should be considering retention strategies when they are designing the trial and developing the protocol.

Not considering retention during trial design and protocol development may lead to protocol amendments, getting further ethical approval, implementing the amendments may require more time and more personnel time. Additionally, not considering retention strategies from the outset can have budget implications as some of the most routinely used retention strategies by CTUs (clinical trial units) in the UK can be expensive to implement [3], the evidence to support their use is lacking [1] and few retention strategies show evidence of cost effectiveness [36]. Therefore, careful consideration and foreplaning is needed to ensure resources are utilised in the best way possible to yield the highest chances of successfully retaining trial participants. We acknowledge that trial teams may have planned retention strategies but failed to report them in the protocol. This lack of communication can lead to implementation issues if there is no clear plan outlined in the protocol document as trial staff use protocols for trial conduct throughout all stages of the trial [14]. A lack of information in the protocol also reduces transparency in trial conduct [14] and limits the replicability of retention strategies which has been recommended to investigate their effects on retention rates [1].

The use of “combined strategies” was the most popular among trial teams (of those that reported a retention strategy) in protocols for both RCTs and pilot and feasibility trials. This concurs with the Cochrane review evaluating strategies to improve retention in randomised trials [1]. Currently, the evidence to demonstrate that retention strategies are effective at retaining participants is either weak or entirely lacking with low to moderate GRADE ratings and no retention strategy has a high certainty GRADE rating to support their use [1]. Therefore, many trial teams are using strategies that may or may not improve retention rates, reinforcing the need for trial teams to plan, report in advance and evaluate the strategies used. This will help to generate evidence to determine which strategy to implement to maximise participant retention rates, whilst also weighing up the cost and resources required to implement the strategy. Implementing multiple strategies also needs further consideration in terms of evaluating their individual effectiveness, as this may be problematic if interaction effects between the different retention strategies are not considered.

Reducing the burden on participants to participate and to provide follow-up data is important in terms of retention [37] and identifying how best to retain participants will save time and trial costs [1]. Patient and public involvement (PPI) in research is important and varying levels of involvement exist. Sometimes PPI members are involved in one specific aspect of the trial or they can be involved for the trial duration [38]. PPI involvement [39] as well as involvement from healthcare professionals [40] during the initial stages of the trial development is important as it can help optimise the relevance of the research to the participant [39, 40] and once the trial is developed and the research question is decided it becomes harder for PPI members to influence key trial aspects [39, 41, 42]. Despite the importance of PPI involvement in the early stages of the trial such as trial development, there tends to be limited PPI input at this stage [42]. This review found a lack of PPI involvement in trial protocols that reported “plans to promote participant retention”. Therefore, we can only assume that PPI input was minimal at best. Thus, trial teams have lost an opportunity to ascertain if their chosen retention strategies are acceptable and suitable to their target population. This is another example of a chronic waste of participants’ time, and undoubtedly adds unnecessarily to trial costs. Another important note is that the majority of retention strategies in our scoping review were generic trial population level strategies and did not make recommendations about target groups within the trial for whom retention may be poorer. There is often also an overreliance on blanket approaches to improve retention with little evidence to support their use [43, 44]. Within trials there may be specific groups of individuals who are more likely to dropout of trials than others and trial teams may need to consider this when planning retention strategies to ensure the strategies they choose target these individuals who are at a higher risk of changes to participation status. One such group would be those participants who actively withdraw from a trial.

Plans to collect outcome data for those who withdraw/discontinue or deviate from the trial protocol (SPIRIT items 18b(ii) and 18b(iii)) were also lacking. There were no strategies that actively targeted withdrawers. Instead, the strategies were either passive—standard practice regarding continued use of collected data, or a more active plan asking participants for consent to continue data collection, despite discontinuation or deviation from the intervention protocol.

Recommendations for future research

Going forward, trial teams need to consider plans to promote participant retention during protocol development, and these should be developed with PPI input. As part of this research programme, we will be conducting qualitative research to investigate why this currently does not seem to be the case, and to further delve into the nuances of these review findings. Retention strategies should be evidence-based strategies that are financially viable, operationalizable, implementable and, importantly, relevant for patients. It is also important for trial teams to consider the environmental sustainability of the retention strategies they choose to implement.

If existing evidence-based retention strategies are not suitable, or trial teams wish to use an alternative strategy, these should be evaluated alongside the trial or within the trial as a SWAT (study within a trial) to contribute to the evidence to support or not support their use. Most of the existing evidence is weak, or entirely lacking, regarding the effectiveness of retention strategies [1]. The Northern Ireland SWAT Repository (https://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/SWATSWARInformation/Repositories/SWATStore/) contains protocols for SWATs that have a retention focus and would provide the much needed evidence needed to decide if the most commonly used retention strategies are effective [45]. We would encourage all trial teams to look at this repository and utilise it. Additionally, the Cochrane review outlines specific priorities for the evaluation of retention strategies which we urge trial teams to take guidance from [1]. To ensure that resources are optimised to retain participants, we need evidence to guide the decision-making process when choosing retention strategies, without this evidence resources are potentially being wasted on strategies that may or may not improve retention rates in trials.

We recommend improved communication of plans to promote participant retention. It was difficult in some cases to distinguish between the use of reminders and prompts due to the language used in some descriptions. We direct trial teams to the ORRCA retention domains [31] and to the most recent Cochrane review of strategies to improve retention in trials, with no high certainty evidence of improvements on trial retention [1], to better communicate their retention strategies. This will assist the conduct of meta-analyses in the future.

Meaningful involvement of members from PPI groups and healthcare professionals [40] is important and valuable at the planning and design phase of a trial [38,39,40]. Currently, not only is there a lack of planning and/or communication of plans to promote participant retention in protocols, but of those that do report a retention strategy, there appears to be little input from PPI colleagues to indicate if these methods are suitable and acceptable to use among the target audience. We need the perspectives and opinions of these individuals to ensure that the strategies being planned are well received by the participants to have the best chance of success. In the UK the National Institute of Health Research now expects active PPI involvement in the research it funds [38, 46], but is it important this is not tokenistic [47]. The Health Research Board in Ireland also recommends working with PPI colleagues in the research it funds. We also direct trial teams to read Trial Forge Guidance 3 which is available as an open access document, to ensure they are taking steps to help recruit and retain individuals from under-served groups and that members of these groups are included in PPI groups [48].

Strengths and limitations

The main strength of this review is the large sample size (n = 824) which includes a wide variety of trial protocols covering different clinical specialties and intervention types. This means that the results are generalizable, representative of RCT protocols and are relevant to a wide variety of trial teams and researchers.

There are some limitations in this review. As mentioned in the “Materials and methods” section, we had to make assumptions regarding some of the reported retention strategies due to a lack of detailed reporting in protocols. We assumed, based on standard ethics committees applications, that all monetary compensation would be disclosed to participants via patient information leaflets [20]. This prior knowledge means that all monetary compensation acted as a monetary incentive rather than a monetary reward. Due to the use of the words reminder and prompt interchangeably, we made assumptions based on the wording of the surrounding text indicating timing of delivery whether it was a reminder or prompt. Therefore, based on these assumptions, we may have misclassified certain retention strategies. We do not believe this has interfered with the overall findings and conclusions however we cannot state this for certain.

We are also aware that the published protocols in the review may not be the first iteration of the protocol, but it was not practical within the scope of our review to track down all versions of the trial protocols. However, due to excluding PsycINFO from our search which specifically specialises in behavioural and mental health trials, these trial protocols may be underrepresented in this review.

Whilst EM screened and data extracted all included protocols, 10% were double screened and double data extracted by FS, and a third reviewer (KG) was consulted where disagreements arose between EM and FS during these processes. This is a limitation as there is a higher possibility of error in the screening (missed protocols/incorrect inclusion of protocols) and the data extraction (relevant data not extracted) processes than if we had double screened and double data extracted all 824 included protocols. This may have impacted the results as relevant protocols and data may have been excluded; therefore, our results may be an underestimation of the reporting of retention strategies in trial protocols.

We also acknowledge that by the time this review is complete and published the timeline may seem out of date as it includes trials from 2014 to 2019 (inclusive), but we sought to establish if planning and reporting of retention plans occurs since the relevant SPIRIT 2013 guidelines were introduced and our inclusion criterion for a 6-year period post 2013 was suitable for that. We still recognise however that the findings may not as accurately reflect protocols written today.

Conclusion

The purpose of our review was to establish if and how trial teams plan for retention at the design stage of clinical trials. Results show that trial teams often do not report plans to prospectively promote participant retention at the design stage of the trial, indicating that the SPIRIT 2013 guidelines item 18b is not being fully considered by trial teams. A greater focus on prospectively planning proactive of retention strategies may inform more suitable choice of strategies and may help lay the groundwork for improving retention rates throughout the course of the trial. Reporting these strategies in protocols also will increase replicability and transparency in trial conduct. Due to the widespread issue of poor retention in clinical trials, trial teams need to pay attention to retention.