Lessons learnt during a complex, multicentre cluster randomised controlled trial: the ProAct65+ trial
© Stevens et al.; licensee BioMed Central Ltd. 2013
Received: 5 February 2013
Accepted: 7 June 2013
Published: 1 July 2013
Failure to recruit to target or schedule is common in randomized controlled trials (RCTs). Innovative interventions are not always fully developed before being tested, and maintenance of fidelity to the intervention during trials can be problematic. Missing data can compromise analyses, and inaccurate capture of risks to participants can influence reporting of intervention harms and benefits.
In this paper we describe how challenges of recruitment and retention of participants, standardisation and quality control of interventions and capture of adverse events were overcome in the ProAct65+ cluster RCT. This trial compared class-based and home-based exercise with usual care in people aged 65 years and over, recruited through general practice. The home-based exercise participants were supported by Peer Mentors.
(1) Organisational factors, including room availability in general practices, slowed participant recruitment so the recruitment period was extended and the number invited to participate increased. (2) Telephone pre-screening was introduced to exclude potential participants who were already very active and those who were frequent fallers. (3) Recruitment of volunteer peer mentors was difficult and time consuming and their acceptable case load less than expected. Lowering the age limit for peer mentors and reducing their contact schedule with participants did not improve recruitment. (4) Fidelity to the group intervention was optimised by introducing quality assurance observation of classes by experienced exercise instructors. (5) Diaries were used to capture data on falls, service use and other exercise-related costs, but completion was variable so their frequency was reduced. (6) Classification of adverse events differed between research sites so all events were assessed by both sites and discrepancies discussed.
Recruitment rates for trials in general practice may be limited by organisational factors and longer recruitment periods should be allowed for. Exercise studies may be attractive to those who least need them; additional screening measures can be employed to avoid assessment of ineligible participants. Enrolment of peer mentors for intervention support is challenging and needs to be separately tested for feasibility. Standardisation of exercise interventions is problematic when exercise programmes are tailored to participants’ capabilities; quality assurance observations may assure fidelity of the intervention. Data collection by diaries can be burdensome to participants, resulting in variable and incomplete data capture; compromises in completion frequency may reduce missing data. Risk assessments are essential in exercise promotion studies, but categorisation of risks can vary between assessors; methods for their standardisation can be developed.
KeywordsRandomised controlled trial General practice Recruitment Adverse events Exercise Peer mentors Postural stability instructors Data collection Quality assurance
The ProAct65+ trial is a three-arm parallel design cluster randomised controlled trial (RCT) of class-based exercises (Falls Management Exercise (FaME)), home-based exercises (Otago Exercise Programme (OEP)) facilitated by peer mentors, and usual care among people aged 65 years and over recruited from general practice . The intervention ran for 24 weeks and in the initial research plan participants were to be followed up for 2 years from the end of the intervention. The trial’s primary outcome is the achievement of the current physical activity recommendations , at 1 year after the intervention ceased. Secondary outcomes include functional ability, falls and fear of falling, and quality of life. The trial is based in north and west London and Nottinghamshire/Derbyshire.
The unique characteristics of this trial are its recruitment base in general practice, its use of volunteer peer mentors to motivate participants in the OEP arm, its three-arm comparison in a UK setting, and its long follow-up period. This update paper describes the challenges faced during the ProAct65+ trial and modifications made to the trial protocol , during the trial.
The protocol was amended to recruit the target number of participants by extending the recruitment period and increasing the numbers of practices and patients per practice invited. To reduce the numbers excluded at baseline assessment, patient telephone screening prior to this initial assessment was introduced. Due to low recruitment and enrolment of peer mentors in the OEP arm the criteria for their recruitment and the intensity of their role were changed. A quality control system was incorporated into the FaME arm to aid standardisation of class activities. To reduce the burden of diary completion and optimise data collection on falls, service use and costs, the number of diaries during the follow-up period was reduced. To ensure governance of risks to participants an adverse event typology was developed and a system for checking it was applied consistently between sites.
The knowledge gained during the ProAct65+ trial and the modifications to its protocol may inform both future exercise trials and public health interventions with older people in general practice.
Recruitment of general practices and participants
Stratified random sampling was planned, whereby eligible patients would be stratified into age groups 65 to 74 years and 75+ years. To simplify the tasks for the practices and to encourage their cooperation this stratified sampling approach was abandoned and patients were sampled from one list of patients aged 65 years and over.
Adding an eligibility screen
Although there were multiple screening steps for eligible patients (by electronic and manual patient searches by the general practices), initially researchers met some patients at the baseline assessment/consent stage who were ineligible due to meeting exclusion criteria, principally falling more than three times in the previous 12 months or already exercising at the target level of five sessions of 30 min of moderate exercise per week. To limit assessment of the ineligible, when arranging the baseline assessment appointment, researchers asked questions over the telephone about falls in the last year and current levels of exercise. Only if the patient was still potentially eligible were they invited to the baseline assessment.
Implementing the interventions
Otago home-based exercise programme (OEP)
Participants were taught the exercise programme in a single group session at a community venue by an exercise instructor trained in the OEP and given a tailored manual of exercises to do at home. In London, 31 sessions were taught to a mean of seven participants per session over 15 months. In Nottinghamshire/Derbyshire, 18 sessions were taught to a mean of seven participants per session over 10 months. Participants were given ankle weights starting at 1 kg and progressing up to 4 kg. Some participants were unable to attend the session due to other personal commitments and were visited and trained by the exercise specialist in their home.
Peer mentors recruitment, training and deployment
Time spent on recruitment by staff (months)
Expressed interest (n)
Time from trained to deployed (days)
Mean 132 (range, 21-255)
Mean 155 (range, 75-257)
Each peer mentor in the trial mentored a mean of three participants (range, 1-13) in London, and a mean of three participants (range, 1-5) in Nottinghamshire/Derbyshire. Overall, both sites therefore fell short of the target of four to five participants per peer mentor. All participants, regardless of their peer mentor support, received the initial exercise training session and a booklet with tailored exercise instructions. Not all participants received a peer mentor because of the difficulties recruiting them. In London 123 (53%) participants and in Nottinghamshire/Derbyshire 21 (12%) participants had a peer mentor. Despite using the same recruitment methods, recruitment difficulties were greater in Nottinghamshire/Derbyshire. This may have been because the trial was competing with existing peer mentor physical activity programmes for older people in Nottinghamshire/Derbyshire. Due to the difficulty recruiting sufficient peer mentors for each participant, we debated using either Postural Stability Instructors (PSIs), already working on and familiar with the study, or volunteers aged < 50 years, who would be trained in the delivery of the home based programme. The Trial Steering Committee advised to keep the intervention true to usual practice in the NHS, that is, one instruction session plus a manual of exercises. Therefore, where there were insufficient peer mentors for all participants, they were not supplemented by an alternative person and some participants had no peer mentor support at all.
In another attempt to increase the number of peer mentors and encourage them to support more participants, the number of their supportive contacts with participants was reduced. Initially peer mentors were scheduled to visit participants in their home on four occasions and telephone them 12 times during the 24-week intervention. This was reduced to two visits and eight telephone calls. Over both sites, the number of home visits ranged from zero to five (mean two), and the number of telephone call contacts ranged from zero to 18 (mean six). Modification of the number of contacts did not increase peer mentor recruitment or their case load.
Falls management exercise programme (FaME)
The FaME intervention is a weekly group-based exercise session, supplemented with additional home exercises (modified from the OEP) described in a booklet. PSIs were recruited to lead the classes. The trial aimed to recruit 12 PSIs per site. In London 16 PSIs were recruited with a total of seven working on the trial. As there were few qualified PSIs available to recruit in Nottinghamshire/Derbyshire, the trial recruited and trained physiotherapists and exercise professionals who were interested in becoming a PSI and working on the trial. Sixteen individuals embarked on the PSI training course (15 completed the training), and seven of them worked on the trial. Some PSIs were not employed on the trial due to their limited availability. Additionally the complex and lengthy process of completing research governance approvals resulted in losing some available PSIs. The recruitment target was reached with 32 PSIs recruited and trained over both sites. Of these 14 (44%) delivered the intervention, enabling the intervention to be fully staffed.
Measuring falls, service use and exercise
Questionnaires used during the trial
End of intervention
6-, 12-, 18-, 24-month follow-up
Face to face
Because non-monetary incentives are known to assist retention in trials , small incentives were sent to participants to encourage completion of postal questionnaires. With diary six and 12, participants received a ProAct65+ pen, and with diary eight and 10, they received a ProAct65+ cotton shopping bag. Participants were also sent an annual Christmas card and brief newsletters with each diary they received.
Response rates at 12-month follow-up
The self-completion diaries requested information on participants’ health and social service use, falls and current exercise levels . Initially it was planned for participants to receive monthly prospective diaries to complete throughout the full length of the trial. When participants said that they wished to withdraw from the trial due to the quantity and frequency of questionnaires they were offered the opportunity to remain in the trial but complete only the six-monthly questionnaires, and not receive further diaries. By doing this the trial retained 52 participants in London, and 28 in Nottinghamshire/Derbyshire (6% of total trial participants) who would otherwise have withdrawn from the trial. To further limit the number of participants who withdrew from the trial due to the burden of the questionnaires and diaries the frequency of the diaries sent during the 2-year follow-up phase was reduced from monthly to quarterly. The diaries sent during the follow-up phase required the participants to recall their service use and falls from the last 3 months and record a 1-week prospective ‘snap-shot’ of their exercise activities.
Capturing adverse events
A comparison of all events between trial sites was carried out towards the end of the trial’s intervention phase. There were noticeable differences in the numbers of Adverse Reactions recorded between sites with London categorising 5%, and Nottinghamshire/Derbyshire categorising 16% of their total events as Adverse Reactions.
A cross-checking system was therefore implemented between sites in an attempt to standardise categorisation. All events from each site, except ‘Adverse Incidents’ , were checked by the other site. If the other site’s categorisation was different to the original categorisation, this was deemed a ‘mismatch’. Mismatches between sites were identified, and blinded forms then passed to the Principal Investigators who discussed and agreed a final categorisation. The initial calculation of mismatches was performed towards the end of the intervention phase, when there were 51 mismatches, giving a mismatch rate between sites of 19%.
The decision on whether an event is ‘Possibly Related’ to the trial is open to subjective interpretation. Consequently, 45 of the 51 (88%) discrepancies in the categorisation of events recorded at each site were between Adverse Events and Adverse Reactions. The category ‘Possible Adverse Reaction’ (Possible AR) was therefore added. After the introduction of the Possible Adverse Reaction category, the mismatch rate (prior to discussion between Principal Investigators) fell to 2.6%.
After advice from the Trial’s Steering Committee, the categorisation was further modified to enable Serious Unrelated Adverse Events to be distinguished from Non-Serious Adverse Events. The final categories applied to the trial’s events were therefore: Serious Adverse Events; Unrelated Serious Adverse Events; Adverse Events; Adverse Reactions; Possible Adverse Reactions; Adverse Incidents.
Discussion and implications
The challenges faced during the ProAct65+ trial and solutions to these challenges are summarised below. Other research which has faced similar challenges is discussed here for comparison. Implications for future research and public health work are suggested.
Although the trial over-achieved its recruitment target, the recruitment process was more difficult and slower than anticipated. The time needed to recruit participants was underestimated and an extension in recruitment time was vital. Other trials recruiting from general practice have found similar slow and difficult recruitment, with lower than anticipated numbers recruited and required time extensions [7–9]. In ProAct65+, the recruitment phase was extended, more general practices were recruited, and more patients at each practice were invited to participate. Trials in general practice should expect the recruitment phase to be longer than initially anticipated, and for more general practices and participants to be invited, to achieve the target numbers. It is advisable to keep recruitment as straight forward as possible and to minimize the work needed by general practices [9, 10].
Expressions of interest were received from patients already exercising at the target level of 150 min of moderate activity per week, and from frequent fallers. It is likely that most exercise trials receive interest from a proportion of the population which is atypical, (that is, more active), than the general population . Telephone pre-screening was introduced to exclude such ineligible patients before they reached the baseline assessment appointment.
The use of volunteers to act as peer mentors proved complicated. As found with other interventions using volunteers, recruitment was slow and the numbers deployed were low . Additionally peer mentors had accepted a lower than expected case load, and consequently support for participants was diluted [13–15], which threatens the OEP intervention’s effectiveness. The lower age limit for peer mentors was reduced, as was the frequency of their contacts with participants, but with only limited benefit. It is important for future interventions to allow enough time and resources (human and financial) when planning peer mentor programmes. In order to minimize the time from training a peer mentor to deployment and to retain interested volunteers, attention needs to be focused on speeding up the process of gaining Criminal Record Bureau checks and Research Management and Governance approvals. Strategies to optimise peer mentor motivation and involvement need further investigation. Additionally, the number of supportive contacts between peer mentors and participants varied and often differed from the number of contacts advised by the research team, which may reflect the needs of the individual participants. Future projects implementing peer mentor support should be aware of participants needs for more or less support, which may lead to varied numbers of contacts with peer mentors.
Failure to ensure the fidelity of interventions is an important source of variation affecting the credibility and utility of research . Trial specific ‘top-up’ training was delivered to instructors in advance of the interventions. Quality assurance observation visits to classes were carried out by expert instructors, with verbal and written feedback on performance. Exercise instructors may not always achieve a balance between tailoring exercise and providing a standardised programme, and observations of intervention delivery and receipt are recommended .
Participants can be burdened by frequent data collection which causes variable response rates to the self-completion questionnaires . Compromises in the frequency of data collection were made, and the frequency of the self-completion diaries was reduced. It has however been suggested that maintaining between-assessment contacts is important to reduce attrition . Personal contact with the research team is said to improve response rates , as do reminders, incentives and printed educational materials [20, 21]. Although more time-consuming and costly than postal self-completion questionnaires, it has previously been shown that home visits to collect follow-up data are useful and can reduce attrition bias in longitudinal studies . Alternatively, higher response rates to postal questionnaires have been found when they are sent directly by the general practices rather than by the research team; this may also be a method to aid retention of participants during a trial .
The classification of safety events between sites was variable, so a method of cross-checking and standardisation was developed. Both site Principal Investigators reviewed and discussed discrepancies in categorisation and a new Possible Adverse Reaction category was introduced to reduce variability. This method of cross checking and the classifications of safety events used in ProAct65+ could be applied to future exercise or indeed any multisite trials.
The ProAct65+ trial is a large pragmatic randomised controlled trial. Despite difficulties the trial reached its recruitment target, making it the largest exercise trial to date in British general practice. The trial’s flexibility in being able to adapt to unexpected problems may have led to the trial’s success . The lessons learnt during the ProAct65+ trial have been valuable and have potential implications for similar trials in general practice.
We thank Ann Bowling; Heather Gage; Mirilee Pearl; Susie Dinan-Young for making substantial contributions to the conception and design of the trial; Cate Barlow, Tessa Hill, Kalpa Kharicha, Tracey Mc Cauley, Marie Ashmore, Philip Prah and Mark Griffin for the acquisition, management, analysis and interpretation of data; Bob Laventure, British Heart Foundation National Centre for Physical Activity and Health and Later Life Training, for the training of Peer Mentors at both research sites; Rachael Taylor for the recruitment of Peer Mentors in Nottinghamshire/Derbyshire; the volunteer participants in this trial for their effort and enthusiasm.
This project was funded by the National Institute for Health Research Health Technology Assessment (NIHR HTA) Programme (project number 06/36/04) and will be published in full in Health Technology Assessment.
The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HTA programme, NIHR, NHS or the Department of Health.
- Iliffe S, Kendrick D, Morris R, Skelton DA, Gage H, Dinan S, Stevens Z, Pearl M, Masud T: Multi-centre cluster randomised trial comparing a community group exercise programme with home based exercise with usual care for people aged 65 and over in primary care: protocol of the ProAct 65+ trial. Trials. 2010, 11: 6-10.1186/1745-6215-11-6.PubMed CentralView ArticlePubMed
- Chief Medical Officers: Start Active Stay Active: A report on exercise for health from the four home countries' Chief Medical Officers. 2011, London: Department of Health
- Booker CL, Harding S, Benzeval M: A systematic review of the effect of retention methods in population-based cohort studies. BMC Publ Health. 2011, 11: 249-10.1186/1471-2458-11-249.View Article
- Perry L, Kendrick D, Morris R, Dinan S, Masud T, Skelton D, Iliffe S, ProAct65+ study team: Completion and return of fall diaries varies with participants' level of education, first language, and baseline fall risk. J Gerontol A Biol Sci Med Sci. 2012, 67: 210-214.View ArticlePubMed
- Ebrahim S, Thompson PW, Baskaran V, Evans K: Randomized placebo-controlled trial of brisk walking in the prevention of postmenopausal osteoporosis. Age Ageing. 1997, 26: 253-260. 10.1093/ageing/26.4.253.View ArticlePubMed
- Lawton BA, Rose SB, Elley CR, Dowell AC, Fenton A, Moyes SA: Exercise on prescription for women aged 40-74 recruited through primary care: two year randomised controlled trial. BMJ. 2008, 337: a2509-10.1136/bmj.a2509.View ArticlePubMed
- Davey R, Edwards SM, Cochrane T: Recruitment strategies for a clinical trial of community-based water therapy for osteoarthritis. Br J Gen Pract. 2003, 53: 315-317.PubMed CentralPubMed
- Campbell MK, Snowdon C, Francis D, Elbourne D, McDonald AM, Knight R, Entwistle V, Garcia J, Roberts I, Grant A, the STEPS group: Recruitment to randomised trials: strategies for trial enrollment and participation study. The STEPS study. Health Technol Assess. 2007, 11: iii-ix. 105View Article
- Skoro-Kondza L, See Tai S, Gadelrab R, Drincevic D, Greenhalgh T: Community based yoga classes for type 2 diabetes: an exploratory randomised controlled trial. BMC Health Serv Res. 2009, 9: 33-10.1186/1472-6963-9-33.PubMed CentralView ArticlePubMed
- Ross S, Grant A, Counsell C, Gillespie W, Russell I, Prescott R: Barriers to participation in randomised controlled trials: a systematic review. J Clin Epidemiol. 1999, 52: 1143-1156. 10.1016/S0895-4356(99)00141-9.View ArticlePubMed
- Hillsdon M, Foster C, Thorogood M: Interventions for promoting physical activity. Cochrane Database Syst Rev. 2005, 1: CD003180-PubMed
- Personal Social Services Research Unit: National Evaluation of Partnerships for Older People Projects. 2009, London: Department of Health
- Hooker SP, Seavey W, Weidmer CE, Harvey DJ, Stewart AL, Gillis DE, Nicholl KL, King AC: The California Active Aging Community Grant Program: translating science into practice to promote physical activity in older adults. Ann Behav Med. 2005, 29: 155-165. 10.1207/s15324796abm2903_1.View ArticlePubMed
- Murphy C, Cupples M, Percy A, Halliday H, Stewart M: Peer-mentoring for first-time mothers from areas of socio-economic disadvantage: a qualitative study within a randomised controlled trial. BMC Health Serv Res. 2008, 8: 46-10.1186/1472-6963-8-46.PubMed CentralView ArticlePubMed
- Dale J, Caramlau I, Sturt J, Friede T, Walker R: Telephone peer-delivered intervention for diabetes motivation and support: the telecare exploratory RCT. Patient Educ Couns. 2009, 75: 91-98. 10.1016/j.pec.2008.09.014.View ArticlePubMed
- Carroll C, Patterson S, Wood S, Booth A, Rick J, Balain S: A conceptual framework for implementation fidelity. Implement Sci. 2007, 2: 40-10.1186/1748-5908-2-40.PubMed CentralView ArticlePubMed
- Resnick B, Inguito P, Orwig D, Yahiro JY, Hawkes W, Werner M, Zimmerman S, Magaziner J: Treatment fidelity in behavior change research: a case example. Nurs Res. 2005, 54: 139-143.View ArticlePubMed
- Davis LL, Broome ME, Cox RP: Maximizing retention in community-based clinical trials. J Nurs Scholarsh. 2002, 34: 47-53. 10.1111/j.1547-5069.2002.00047.x.View ArticlePubMed
- Patel M, Doku V, Tennakoon L: Challenges in recruitment of research participants. Adv Psychiatr Treat. 2013, 19: 229-238. 10.1192/apt.bp.111.009050.View Article
- Motzer SA, Moseley JR, Lewis FM: Recruitment and retention of families in clinical trials with longitudinal designs. West J Nurs Res. 1997, 19: 314-333. 10.1177/019394599701900304.View ArticlePubMed
- Foy R, Parry J, Duggan A, Delaney B, Wilson S, Lewin-Van Den Broek NT, Lassen A, Vickers L, Myres P: How evidence based are recruitment strategies to randomized controlled trials in primary care? Experience from seven studies. Fam Pract. 2003, 20: 83-92. 10.1093/fampra/20.1.83.View ArticlePubMed
- Peterson JC, Pirraglia PA, Wells MT, Charlson ME: Attrition in longitudinal randomized controlled trials: home visits make a difference. BMC Med Res Methodol. 2012, 12: 178-10.1186/1471-2288-12-178.PubMed CentralView ArticlePubMed
- Williamson MK, Pirkis J, Pfaff JJ, Tyson O, Sim M, Kerse N, Lautenschlager NT, Stocks NP, Almeida OP: Recruiting and retaining GPs and patients in intervention studies: the DEPS-GP project as a case study. BMC Med Res Methodol. 2007, 7: 42-10.1186/1471-2288-7-42.PubMed CentralView ArticlePubMed
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.