We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
Improving recruitment to a study of telehealth management for COPD: a cluster randomised controlled ‘study within a trial’ (SWAT) of a multimedia information resource
Trialsvolume 20, Article number: 453 (2019)
Good quality information is critical for valid informed consent to trials, but current paper-based consent procedures are potentially unwieldy and can be difficult to comprehend, which may deter people from participating. Multimedia resources may be able to provide more accessible and user-friendly information.
We aimed to test whether offering access to a multimedia information resource alongside standard, printed patient information impacted on recruitment rates by conducting a pragmatic ‘study within a trial’ (SWAT) embedding a trial of a multimedia resource within an existing trial.
The PSM COPD study involved people with mild symptoms of chronic obstructive pulmonary disease (COPD) recruited from primary care being randomised to a nurse-delivered telephone health coaching intervention, or usual primary care.
For the SWAT of recruitment procedures, practices recruiting participants were cluster randomised to use either the standard printed patient information materials or standard printed patient information materials with access to a multimedia information resource.
The multimedia resource was developed by patient and public involvement (PPI) contributors and researchers, and included study-specific information (e.g. study purpose, risks), and generic information about trials (e.g. confidentiality, randomisation). We developed a list of components and used animations as well as video clips of patients discussing their experiences of participation, matched to these components.
The primary outcome was the proportion of participants randomised.
Nine point six percent of those receiving standard printed patient information materials and access to the multimedia information resource were recruited, compared to 10.8% in those receiving standard printed materials alone (odds ratio (OR) = 0.844, 95% confidence interval (CI) 0.58 to 1.22).
We also found no effects on the proportion of people responding to the invitation (OR = 1.02, 95% CI 0.79 to 1.33) or retention in the trial at 6 (ORs 0.84, 95% CI 0.57 to 1.22) and 12 months after randomisation (ORs 0.80, 95% CI 0.54 to 1.18), respectively.
The study suggests no benefits of access to a multimedia information resource alongside patient information materials on recruitment. This may reflect the limited engagement of patients with the multimedia resource. Further uses of multimedia resources will need to explore how content can be explicitly matched to user needs and preferences and methods to encourage engagement to see if effects can be enhanced. More SWATs of multimedia into ongoing trials will provide a more precise estimate of effect, and explore further how effects vary by trial context and recruitment process, intervention, and patient population.
Current controlled trials ISRCTN 06710391. Registered on 21 November 2013.
SWAT 23: Systematic Techniques for Assisting Recruitment to Trials (MRC START). Registered on 11 January 2012.
Participant recruitment is essential for the delivery of trials, but many trials fail to recruit to time and target [1, 2]. Despite these problems and their effects on trial validity and costs, there is little rigorous quantitative research to support recruitment efforts. Embedding trials of different recruitment methods in trials (so called ‘studies within a trial’ or SWATs) is an effective way of testing methods  and is increasingly supported by funders , but a recent Cochrane review identified only 68 studies of this type .
Enhancement to patient information – the role of multimedia
A conventional method of recruitment is providing information to potential participants to help them make an informed decision. Conventional methods are largely paper-based. There are concerns about the quality and comprehensibility of standard paper presentations [6, 7], which in part reflect their lack of flexibility – as there are far fewer options to present information in ways that are engaging and informative, or matched to the needs and preferences of users.
Multimedia interventions may offer a useful way forward, as they provide a useful platform for health communication, including allowance for self-directed and tailored learning [8, 9], greater user choice and potential for personalisation, and may better meet the needs of an audience increasingly used to obtaining information digitally.
Reviews of the impact of multimedia interventions on research participation have explored a variety of outcomes, including knowledge and understanding, recall, willingness to participate, perceptions of the value of research, as well as decision-making outcomes. Only a small number of studies explored the effects of multimedia materials in the Cochrane review on improving recruitment to trials , and the overall conclusion was of uncertainty concerning the effects. Given the limited evidence base and the ubiquity of multimedia, further research is clearly warranted.
Testing the effects of multimedia interventions
The ‘Systematic Techniques for Assisting Recruitment to Trials’ (START) programme seeks to increase the evidence base in this area by developing a platform to encourage the rapid and rigorous testing of recruitment interventions by conducting SWATs in host trials .
As part of the START programme, we recruited trials funded by the UK National Institute of Health Research Health Technology Assessment Programme or registered with the Primary Care Research Network portfolio. Host trials were offered access to one of two interventions: optimised participant information materials  or multimedia information presented via the Internet, both intended to improve communication of trial information to potential participants.
This study aimed to determine whether access to a multimedia information resource alongside standard printed patient information improved recruitment, compared to standard printed patient information alone.
The study was reported in line with published guidance .
Description of the PSM host trial
The host trial was called ‘PSM COPD’, and was a pragmatic multicentre trial of telephone health coaching to support self-management compared with usual care for people with COPD with mild dyspnoea. The protocol and main results paper for the host trial have both been published [13, 14]. Patients were recruited from 71 general practices around Birmingham, Oxford, Manchester, and Stoke-on-Trent. Patients had to be aged 18+ years and to meet the following eligiblity criteria: (1) on the practice COPD register, (2) experience mild dyspnoea (Medical Research Council (MRC) grades 1 or 2), (3) had an FEV1/FVC < 0.7 after post-bronchodilator spirometry.
Eligible patients were sent a letter from their general practitioner, with a slip for return to the research team. Interested patients were telephoned by the research team for further assessment and informed consent.
Description of the SWAT
To ease the logistics of the trial, practices (cluster level) in the host trial were randomised using stratified (by area – Birmingham, Oxford, Manchester, and Stoke-on-Trent) block randomisation (ratio 1:1, with three varying block sizes selected randomly by the computer) to access to the multimedia information resource or only the printed patient information sheet (i.e. all patients from a particular practice were sent the same invitation letter). All patients identified as potentially eligible for the PSM host trial were eligible for the SWAT; there were no additional eligibility criteria for the SWAT. To ensure allocation concealment, the allocation sequence was generated centrally by VM (who had no other involvement in the running of the host trial) using the ‘ralloc’ command in Stata. Although informed consent was gained from patients in the host trial, patients were not aware that they were being randomised within the SWAT and no formal consent was taken. As noted above, we provided a link to the multimedia resource, but the decision to access the resource was entirely that of the patient. No changes to methods were made after commencement.
Due to the logistics of the study, only 58/71 (82%) of the practices taking part in the host trial undertook the SWAT. The host trial ran from 2013 to 2016. There was no pre-study sample size calculation for the SWAT.
Development of the multimedia SWAT intervention
Intervention content was informed by four elements:
Core components for the multimedia information resource were generated by team members
A review of factors identified by patients as determinants of decisions about trial participation was undertaken
The multimedia information resource had input from members of a patient and public involvement (PPI) forum
The multimedia information resource had input from qualitative experts on patient health experiences (http://www.healthtalk.org/)
Multimedia interventions offer a platform for learning which can include study-specific information (e.g. study purpose, risks), and generic information (e.g. confidentiality). Patient and public involvement (PPI) forum members and qualitative experts developed study-specific components involving bespoke themes such as investigator details and benefits of participation. Generic information components included information on informed consent, randomisation, and confidentiality. Existing video clips of patients discussing their experiences of participation were edited for length and carefully matched to these components. The multimedia intervention was developed by a commercial company for use on a range of platforms including desktops and smartphones. Additional file 1 shows screenshots from the multimedia intervention, showing the introduction screen, and the screens related to study-specifc and generic information.
Access to the multimedia resource was provided as part of the patient information sheet, with a URL link and QR code to assist with easy access (see Additional file 2 for the presentation of the resource to patients). However, accessing the multimedia information resource was entirely voluntary.
Analyses were conducted in line with a standard statistical plan developed at Barts and the London Pragmatic Clinical Trials Unit by SE and VM (details available from the authors). Preliminary graphical and tabular examination of the data explored baseline comparability of trial arms and representativeness of the sample in terms of the overall eligible population. The primary outcome was recruitment rate, defined as the proportion of patients actually recruited to the host trial following an invitation and randomised to each group. Analysis was by intention-to-treat. The numbers responding to the trial invitation, as well as 6 and 12 months’ retention rates were secondary outcomes. Outcomes were first described separately by arm, and then compared using logistic regression to estimate the between-group odds ratio and corresponding 95% confidence interval on the basis of the intention-to-treat principle. All analyses took account of the clustering of data due to the allocation procedure by incorporating a dispersion parameter in the model and were conducted using Stata version 12.1. The stratification factor (area) was included as a fixed effect variable in the model.
Post hoc, we estimated the cost per additional participant associated with the intervention. We estimated the cost of the intervention itself based on the price paid by the research team. We estimated the potential effectiveness of the intervention from the upper limit of the 95% confidence interval. We calculated the cost per person approached, and the additional cost associated with this benefit.
Research Ethics Committee approval
The START programme and the individual SWATs within it were approved by the National Research Ethics Service (NRES) Committee, Yorkshire and the Humber – South Yorkshire (Ref: 11/YH/0271) on 5 August 2011. As noted earlier, although informed consent was gained from patients in the host trial, we obtained ethical approval such that patients were not aware that they were being randomised within the SWAT and no consent was taken.
The flow of patients through the trial is shown in Fig. 1. Due to the cluster design, baseline data on patients were not available for comparison. Over 4000 patients were approached, and the rates of response to invitation, randomisation and retention over the 12 months are shown. Analyses (Table 1) showed that randomisation and retention rates were lower in the multimedia information resource group, although none of the differences reached statistical significance.
We estimate that the cost of the multimedia was around £2500. We approached 4223 people (approximately £0.60 per person). Assuming the best estimate of effectiveness (a 0.6% increase), the intervention would recruit an additional six people per 1000 approached at a cost of £100 per additional patient.
Despite the ubiquity of multimedia and digital information, there is limited evidence so far that its benefits can be harnessed to improve patient understanding of trials, or improve randomisation and retention [5, 15]. We tested the effects of access to a multimedia information resource on recruitment and retention to a host trial evaluating a health coaching intervention for COPD. Access provided in addition to a standard patient information sheet did not improve rates of randomisation and retention. Even assuming the most favourable impacts, the cost per additional patient was around £100, which is unlikely to be cost-effective compared to simply increasing the overall numbers of patients mailed.
As with most SWATs, there was no formal sample size calculation, and we undertook the SWAT on the basis of the maximum number of patients and practices possible given study logistics. The cluster design would also potentially have reduced precision, although this was considered a reasonable compromise, given the potential for mistakes in allocation that may have occurred with individual randomisation across multiple sites. Although we planned to assess use of the multimedia, an error in the web-hosting software meant that we were unable to collect accurate data on use. Although this makes it impossible to assess use of the multimedia, this pragmatic trial was designed to assess the offer of access to multimedia within routine trial informed consent procedures.
Baseline data were not available for patients invited to the SWAT, as opposed to those in the trial itself. As there were no additional eligibility criteria for the SWAT, the characteristics of groups randomised in the SWAT should be similar to those in the main trial, which are detailed in the main publication . In summary, the population was mostly male, with a mean age of 70 years, with limited educational qualifications and with most retired from work. We were unable to assess whether those recruited by multimedia differed in characteristics from those who were recruited using conventional methods . Such analyses are not routinely done in published SWATs [11, 17]. The likely modest impact of many recruitment interventions means that the effects on the types of patients recruited is likely to be minimal, although this may be important to report in SWATs where possible.
Interpretation of the findings in the context of the wider literature
The SWAT has been conducted in the context of a specific patient population and a specific intervention, and care must be taken in generalising the results. As part of the START programme we have embedded the same intervention in multiple host trials to better assess the effects through pooling of data, and new trials will report in due course. It is possible that the effects of multimedia information are dependent on population age and other demographic factors. As noted earlier, the population in the host trial was largely retired males aged over 70 years. The evidence suggests that Internet non-users are more likely to be women, but that non-users are also much more prevalent in those aged over 75 years . The host trial population might not have been optimal for testing of this intervention. A related study is being conducted which is exploring multimedia in children and adolescents with long-term conditions. This population may be especially amenable to multimedia . Future studies might also include more proximal outcomes of the multimedia intervention, such as knowledge or understanding of the trial. A recent systematic review of audio-visual information to inform potential trial participants reported small beneficial effects on patient knowledge and understanding of the trial, but no effects on trial recruitment rate, although half of included studies concerned hypothetical not real trials .
Implications for recruitment practice
Although the multimedia information resource was potentially more accessible and engaging than the printed information, it would potentially take a patient more time to understand than the printed resource. If patients already find the conventional patient information sheet to be complex and take a long time to read, they may not find additional information useful, even if the presentation is more engaging.
As noted, the trial procedures meant that we were only able to provide patients with a link to the multimedia resource and not more actively encourage its use. Due to an error in coding, data on uptake or use of the multimedia resource were not available. Therefore, it is not clear whether the resource was not accessed at all, or whether it was accessed and ineffective, or whether it had variable effects (increasing participation in some patients, and reducing it in others).
Another important consideration is the context of the study and the methods of recruitment. Mailing letters from primary care is a common and reliable strategy, but means that getting multimedia into the consent process is difficult. Studies where the initial contact is face to face or by telephone with a researcher may be much more fruitful contexts for testing the results of multimedia.
It is also possible that access to accurate information leads to positive benefits on patient understanding , but that this does not translate to improved recruitment (or even reduces it). In-depth qualitative work alongside the SWAT would have been useful to explore whether it matched the needs of users, as well as use and interpretation of the resource, but this was beyond the resources of the project.
Access to a multimedia information resource had no important effect on recruitment or retention to a host trial. Further SWATs of this technology, exploring effects in different population, are required, alongside innovation in the ways in which patients can access and use such resources.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Medical Research Council
Systematic Techniques for Assisting Recruitment to Trials
McDonald A, Knight R, Campbell M, Entwistle V, Grant A, Cook J, Elbourne D, Francis D, Garcia J, Roberts I, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7:9.
Sully B, Julious S, Nicholl J. A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies. Trials. 2013;14:166.
Treweek S, Bevan S, Bower P, Campbell M, Christie J, Clarke M, Collett C, Cotton S, Devane D, El Feky A, et al. Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)? Trials. 2018;19(1):139.
Treweek S, Parker A, Torgerson D, Clarke M.:Why the NIHR’s new funding stream for ‘Studies Within A Trial’ (SWATs) is potentially game-changing. 2018. https://www.nihr.ac.uk/blogs/why-the-nihrs-new-funding-stream-for-studies-within-a-trial-swats-is-potentially-game-changing/8259. Accessed 17 June 2019.
Treweek S, Pitkethly M, Cook J, Fraser C, Mitchell E, Sullivan F, Jackson C, Taskila TK, Gardner H. Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018;(2).
Terblanche M, Burgess L. Examining the readability of patient-informed consent forms. J Clin Trials. 2010;2:157.
Knapp P, Raynor D, Silcock J, Parkinson B. Can user testing of a clinical trial patient information sheet make it fit-for-purpose? – a randomised controlled trial. BMC Med. 2011;9:89.
Antoniou EE, Draper H, Reed K, Burls A, Southwood TR, Zeegers MP. An empirical study on the preferred size of the participant information sheet in research. J Med Ethics. 2011;37(9):557–62.
Shneerson C, Windle R, Cox K.:Innovating information-delivery for potential clinical trials participants. What do patients want from multi-media resources? Patient Education and Counseling 2012, EPub ahead of print.
Rick J, Graffy J, Knapp P, Small N, Collier D, Eldridge S, Kennedy A, Salisbury C, Treweek S, Torgerson D, et al. Systematic Techniques for Assisting Recruitment to Trials (START): study protocol and preliminary findings on a platform for nesting studies of recruitment interventions across multiple trials. Trials. 2014;15:407.
Man MS, Rick J, Bower P, Group HS, Group MS. Improving recruitment to a study of telehealth management for long-term conditions in primary care: two embedded, randomised controlled trials of optimised patient information materials. Trials. 2015;16:309.
Madurasinghe V, Eldridge S, Group MS, Forbes G, Group SEC. Guidelines for reporting embedded recruitment trials. Trials. 2016;17(1).
Sidhu MS, Daley A, Jordan R, Coventry PA, Heneghan C, Jowett S, Singh S, Marsh J, Adab P, Varghese J, et al. Patient self-management in primary care patients with mild COPD – protocol of a randomised controlled trial of telephone health coaching. BMC Pulm Med. 2015;15(1):16.
Jolly K, Sidhu MS, Hewitt CA, Coventry PA, Daley A, Jordan R, Heneghan C, Singh S, Ives N, Adab P, et al. Self management of patients with mild COPD in primary care: randomised controlled trial. BMJ. 2018;361:k2241.
Brueton V, Tierney J, Stenning S, Harding S, Meredith S, Nazareth I, Rait G. Strategies to improve retention in randomised trials. Cochrane Database Syst Rev. 2013;(12):MR000032. https://doi.org/10.1002/14651858.MR000032.pub2.
Loban A, Mandefield L, Hind D, Bradburn M. A randomized trial found online questionnaires supplemented by postal reminders generated a cost-effective and generalizable sample but don’t forget the reminders. J Clin Epidemiol. 2017;92:116–25.
Parker A, Knapp P, Treweek S, Madhurasinghe V, Littleford R, Gallant S, Sullivan F, Schembri S, Rick J, Graffy J, et al. The effect of optimised patient information materials on recruitment in a lung cancer screening trial: an embedded randomised recruitment trial. Trials. 2018;19(1):503.
Office for National Statistics. Exploring the UK’s digital divide. London: Office for National, Statistics; 2019. https://www.ons.gov.uk/peoplepopulationandcommunity/householdcharacteristics/homeinternetandsocialmediausage/articles/exploringtheuksdigitaldivide/2019-03-04/pdf. Accessed 15 May 2019
Martin-Kerry J, Bower P, Young B, Graffy J, Sheridan R, Watt I, Baines P, Stones C, Preston J, Higgins S, et al. Developing and evaluating multimedia information resources to improve engagement of children, adolescents, and their parents with trials (TRECA study): study protocol for a series of linked randomised controlled trials. Trials. 2017;18(1):265.
Synnot A, Ryan R, Prictor M, Fetherstonhaugh D, Parker B. Audio‐visual presentation of information for informed consent for participation in clinical trials. Cochrane Database of Systematic Reviews. 2014;Issue 5. Art. No.: CD003717. https://doi.org/10.1002/14651858.CD003717.pub3.
We would also like to acknowledge the contribution of all the applicants on the original proposal, as well as Judith Hogg and Ailsa Donnelly from PRIMER (the Primary Care Research in Manchester Engagement Resource) who contributed patient and public involvement to the MRC START team.
MRC START Group
Sandra Eldridge, Blizard Institute, Barts and The London School of Medicine and Dentistry, 4 Newark Street, London, E1 2AT, UK email@example.com
Jonathan Graffy, Department of Public Health and Primary Care, The Primary Care Unit, University of Cambridge, Cambridge, CB2 OSR, UK firstname.lastname@example.org
Adwoa Parker York Trials Unit, Department of Health Sciences, University of York, York, UK email@example.com
Peter Knapp, Department of Health Sciences, University of York, and the Hull York Medical School University Road, Heslington, York, YO10 5DD, UK firstname.lastname@example.org
David Torgerson, York Trials Unit, Department of Health Sciences, University of York, University Road, Heslington, York, YO10 5DD, UK email@example.com
Shaun Treweek, Health Services Research Unit, University of Aberdeen, Fosterhill, Aberdeen, AB25 2ZD, UK firstname.lastname@example.org
PSM COPD Group
Professor Peymane Adab, Institute of Applied Health Research, University of Birmingham, UK, email@example.com
Amy Blakemore, Centre for Primary Care: Institute of Population Health, University of Manchester, UK, firstname.lastname@example.org
Dr. Peter Coventry, Department of Health Sciences, University of York, York, UK email@example.com
Professor Amanda Daley, School of Sport, Exercise and Health Sciences, Loughborough University, Loughborough, UK
Professor David Fitzmaurice, Warwick Primary Care, University of Warwick, UK firstname.lastname@example.org
Professor Carl Heneghan, Nuffield Department of Primary Care Health Sciences, University of Oxford, UK email@example.com
Catherine A Hewitt, Medical Statistician, Birmingham Clinical Trials Unit, University of Birmingham UK C.A.Hewitt@bham.ac.uk
Natalie Ives, Birmingham Clinical Trials Unit, University of Birmingham UK, N.J.Ives@bham.ac.uk
Rachel Jordan, Institute of Applied Health Research, University of Birmingham, UK
Sue Jowett, Institute of Applied Health Research, University of Birmingham, UK
David Nunan, Nuffield Department of Primary Care Health Sciences, University of Oxford, UK.
Sally Singh, Centre for Exercise and Rehabilitation Science, Biomedical Research Centre (Respiratory), University Hospitals of Leicester NHS Trust, Glenfield Hospital, Leicester UK
The authors wish to acknowledge the MRC Methodology Research Programme which funded the embedded trials (MRC Grant Ref: G1002325). The MRC has no role in study design; collection, management, analysis, and interpretation of data; writing of the report; or the decision to submit the report for publication.
The PSM COPD study was funded by the National Institute for Health Research (NIHR) School for Primary Care Research and the NIHR Collaborations for Leadership in Applied Health Research and Care (CLAHRC) West Midlands. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Ethics approval and consent to participate
The START programme of recruitment interventions was approved by the National Research Ethics Service (NRES) Committee, Yorkshire and the Humber – South Yorkshire (Ref: 11/YH/0271) on 5 August 2011. Although informed consent was gained from patients in the host trial, patients were not aware that they were being randomised within the SWAT and no formal consent was taken.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.