Surveys of current practices and a Delphi survey have highlighted routinely used approaches within the UK, the lack of evidence informing practice and future methodological research priorities within retention.
A strong focus has been placed on improving recruitment in clinical trials and this is routinely monitored and accepted as a key performance indicator of sites impacting funding received [20]. However, attention needs to be extended to cover the retention of randomised participants with sites monitored and rewarded accordingly. Whilst HTA chief investigators are aware of retention issues and registered CTUs are regularly revising sample sizes and proactively implementing a broad range of strategies to maintain contact with patients, improve questionnaire and data return, minimise patient burden and incentivise patients, retention is currently not used as a key performance indicator within UK research networks and is seldom linked to per-patient payments for research costs.
Whilst survey formats differed, the perspectives of chief investigators and registered CTUs were similar. Chief investigator’s recommendation of good monitoring processes to identify and address any problems with data collection was congruent with registered CTUs’ routine use of strategies that might facilitate this such as telephone reminders and routine site visits. Both registered CTUs and chief investigators also placed a strong emphasis on training and working with local research site staff to minimise missing data, with six of the top 10 strategies routinely used by registered CTUs focussed in this area.
However, many strategies continue to lack evidence for their effectiveness. Whilst 61% (36/59) of strategies had some evaluation reported by registered CTUs, existing evidence was typically low level: 21 strategies had only one nested RCT, and only five strategies had two evaluations that could be combined in a meta-analysis assuming that there was sufficient homogeneity. It is important to replicate findings in multiple trials to ensure the applicability and generalisability and because results of nested RCTs may be more convincing if they span multiple trials.
Many of the research strategies prioritised within the Delphi survey relate to methods routinely used by registered CTUs. Methods all have implications for resource use and despite their frequent use in current practice there was an absence of evidence to support them. Their prioritisation within the Delphi survey likely reflects the desire of registered CTUs to have their practices supported by evidence to ensure that they direct their limited resources to effective methods. More embedded trials (SWATs: Studies Within A Trial) [21] are needed to further the evidence base and avoid wasting resources on unproven and potentially ineffective retention strategies. The MRC-funded START [22] project focusses on SWATs targeting improved recruitment and a similar initiative could be beneficial to improve the evidence base for retention. The results of the Delphi survey provide a list of priorities for future SWATs.
Site initiation training was routinely used within registered CTUs and a top practice recommended by chief investigators, but was one of the strategies with no reported evaluations. It is, therefore, unsurprising that this strategy was identified as the top priority for future methodological research during the Delphi survey. Triggered site training also reached consensus criteria in round 2 reiterating the need to examine how multicentre trials address attrition when patient contact and outcome measurement are often delegated to sites.
Research into site training is supported by the results of recent workshops reviewing recruitment and retention strategies [16, 23]. Anecdotal evidence suggests that coordinated approaches with local sites and strong clinical buy-in can help to retain patient populations perceived to be at risk of attrition [23]. However, analysis of the methods and content of site training have largely focussed on the impact on informed consent and recruitment [24,25,26,27,28], rather than retention of patients and collection of outcome data where there is a paucity of academic literature. Lienard conducted a nested RCT investigating the effect of training and subsequent monitoring visits on data return and quality but found no difference because the study was terminated early [29].
As the Delphi survey topic titles were broad, further work is needed to explore how site training might impact on retention. Chief investigators commented on both content and methods for delivering training within their survey responses. They described using training to communicate data collection priorities and processes, including the opportunity to develop bespoke processes for individual sites. Suggestions of training methods included interactive presentations, newsletters and refresher training. Existing trial processes (e.g. routine use of newsletters, regular contact with sites) may be able to support such site training as well as enhancing or maintaining the buy-in of research site staff. Berger supports this, communicating how training was important in maintaining the continuity of research staff and equipping them to monitor and address withdrawal reasons, negotiate complaints and reiterate the importance of data collection with patients to improve retention [30].
The choice of outcomes for future SWATs are an important consideration for future research. A Cochrane review identified 38 nested randomised studies of retention interventions of which 34 aimed to improve questionnaire return [17]. In our survey of chief investigators, patient withdrawal was the most widely reported cause of missing data and yet there is little published evidence for effective strategies. The four studies measuring patient retention assessed behavioural motivation, case management and a factorial design of a trial certificate and gift. No effect was found and our research shows that these strategies are, at best, infrequently used by registered CTUs.
Future research
The ranked list of research priorities from the Delphi survey provides a ‘roadmap’ to address uncertainties and systematically evaluate key strategies. Six topics were deemed critically important and should be a primary focus of future retention research.
We strongly recommend that future studies take account of the range of causes of missing data reported by chief investigators and previous studies. Retention strategies applicable to a range of trial designs, such as site training, frequency of patient contact and the frequency and timing of reminders, should assess impact on patient withdrawal, patients lost to follow-up and clinical staff failing to record primary-outcome measurements as well as questionnaire response rates.
Many Delphi survey participants commented on the challenges of scoring the importance of retention strategies when these are often chosen to address challenges within specific trial designs or populations. Considering the number of strategies discussed it was not feasible to explore this and participants were encouraged to score strategies based on the trials frequently delivered within their CTU. Consequently, the results reflect a ranked list of priorities that will have the broadest impact across the range of trials currently undertaken in the UK. Future research is needed to explore the effectiveness of different strategies for specific trial designs, settings and patient populations.
Strengths and limitations
To our knowledge this is the first study to develop a research agenda for evaluating the effectiveness of retention strategies and so provides a valuable reference point for future methodological work.
A systematic review and surveys of registered CTUs and HTA chief investigators were used to compile the initial list of retention strategies for the Delphi survey. The two different surveys sought to capture both the practical experience of chief investigators within specific trials and the broader experience of registered CTUs that work across multiple trials and different trial designs. Registered CTUs were invited to complete the Delphi survey in order to create a priority list that was applicable to the broad range of trials represented within the UK.
Whilst the study was researcher-led and did not include public or patient perspectives, all three surveys benefited from strong response rates and the Delphi survey retained the majority of participants. Only one person failed to complete round 2 suggesting that the results are unlikely to be affected by attrition bias. The high response rate is attributed to engaging an active network of registered CTUs that were invested in the topic having previously agreed that it was of critical importance [5].
Analysis of the number of retention strategies evaluated by registered CTUs should be considered with some caution as missing responses could not be identified. However, this information did not influence the Delphi process or the ranking of retention strategies for future evaluation.