- Open Access
Identifying important barriers to recruitment of patients in randomised clinical studies using a questionnaire for study personnel
Trials volume 20, Article number: 618 (2019)
Many randomised controlled trials (RCT) fail to meet their recruitment goals. Study personnel play a key role in recruitment. The aim of this study was to identify successful strategies that study personnel consider to be important in patient recruitment to RCT.
We constructed a questionnaire based on the literature, discussions with colleagues and our own experience as trialists. The survey was named “What is Important for Making a Study Successful questionnaire” (WIMSS-q). Our target group was the study personnel in the ongoing EFFECTS study. The questionnaire was sent out electronically to all physicians and nurses (n = 148). Success factors and barriers were divided according to patient, centre and study level, respectively.
Responses were received from 94% of the study personnel (139/148). The five most important factors at centre level for enhancing recruitment were that the research question was important (97%), a simple procedure for providing information and gaining consent (92%), a highly engaged local principal investigator and research nurse (both 87%), and that study-related follow-ups are practically feasible and possible to coordinate with the clinical follow-up (87%). The most significant barrier at the local centre was lack of time and resources devoted to research (72%). Important patient-related barriers were fear of side effects (35%) and language problems (30%).
For recruitment in an RCT to be successful, the research question must be relevant, and the protocol must be simple and easy to implement in the daily routine.
The protocol for this study was registered at the Northern Ireland Hub for trials methodology research (SWAT ID 64). The EFFECTS study has EudraCT number 2011–006130-16 and was registered 17 February 2016 at ClinicalTrials.gov number NCT02683213.
A common problem with randomised controlled trials (RCT) is that the recruitment is slow and fails to meet the recruitment goal in time . In 2007 Campbell found that less than one-third of trials achieved their original recruitment target goal in time . Poor recruitment can lead to underpowered study results. It can also have practical, financial and ethical consequences, as it may prolong the trial. Furthermore, the study subjects consent to participate in a trial to help answer an important health question when, in fact, the question is not answered due to insufficient power.
Clearly there is a need for more research in this area [3, 4] and in the UK finding methods to enhance recruitment in RCTs has been identified as being of the highest priority [5, 6]. It is considered important to try to predict early on in the planning phase of a study what problems are likely to arise when it comes to the recruitment of patients . Study personnel at centres play a crucial role in recruiting and retaining patients in clinical trials. Little is known about what people involved in clinical trials think are relevant barriers and solutions [8, 9]. Workshop and in-depth interview studies have investigated barriers to and opportunities for recruitment in clinical studies and found that it is still a challenge; new recruitment interventions are needed [10, 11].
Kaur et al.  were the first to develop a survey tool to capture the recruitment experience of clinical teams regarding facilitators and barriers to recruitment in clinical trials. The authors identified six important categories: (i) trial, (ii) site, (iii) patient, (iv) clinical team, (v) information and consent, and (vi) the study team.
The aim of this study was to identify successful strategies that study personnel consider to be important for patient recruitment to RCT.
The questionnaire was developed in several steps. First, we performed a literature search (February 2017) in PubMed using the following search terms: survey OR questionnaire OR recruitment challenges OR randomised controlled trial. We searched for original studies and systematic reviews in English. In addition to the literature search, we found information at the Trial Forge initiative [13, 14].
Secondly, we discussed this topic with colleagues from Edinburgh University (acknowledged below) with significant experience of conducting multicentre RCTs.
Finally, we used our own experience of conducting multicentre clinical trials in Sweden.
Construction of the questionnaire
Figure 1 presents an overview of how we constructed the questionnaire. We decided to divide the questionnaire into success factors and barriers to recruitment in an RCT at patient, centre and study level, respectively.
In March 2017, we pre-tested the questionnaire using a think-aloud procedure . One researcher (EI) sat side by side with the test person and wrote down all their comments while they performed the questionnaire. Four test persons with varying experience of working with clinical trials were chosen: one medical doctor with 25 years of experience with trials in neurology, one research nurse with 20 years of experience of research, one nurse with a PhD who had worked with research for 30 years, and the fourth was a nurse who had worked for 6 months as a research nurse. The questions were adapted based on what emerged from the pre-test; it appeared to be difficult for the participants to formulate five free-response alternatives indicating significant barriers and factors important for inclusion, so we reduced the number of alternatives to two. Further, some questions were reformulated. We named the questionnaire “What is Important for Making a Study Successful questionnaire” (WIMSS-q).
To test the method, questions and the response rate, a pilot study was carried out March 2017. The questionnaire was sent electronically by SurveyMonkey  to all physicians and nurses active, listed in the delegation log at the time (n = 136) in Efficacy oF Fluoxetine—a randomised Controlled Trial in Stroke (EFFECTS) , an ongoing RCT in Sweden. We had a 67% response rate (92/136) after three reminders. The questionnaire was adjusted according to what was found (Additional file 1, Changes made in the questionnaire). In summary, we rephrased some of the questions and grouped them differently and removed the link to the EFFECTS study, which meant that the responders should answer the questionnaire with all their accumulated knowledge and experience of working with RCTs.
Description of the final version of the questionnaire
WIMSS-q began with some general questions: age, gender, the role in EFFECTS, and how accustomed the participants were taking part in RCTs.
The general questions were followed by two open questions about barriers and important factors for inclusion in RCT. The purpose of these questions was to capture what we may have missed when we constructed the questionnaire.
After these open questions, we asked questions with predefined response options. We provided a preformed list of potential factors affecting recruitment as well as facilitators for three levels: patient, local centre- and study-related barriers (Additional file 2, final version of the questionnaire). We used a five-point Likert scale [18, 19] ranging from 1 (completely disagree) to 5 (completely agree). One of the questions is illustrated in Fig. 2.
There were questions about ways to succeed with inclusion. We stated 16 factors and the participants were asked to grade them by using the Likert scale, 1 to 5. The questionnaire also had questions about differences between academic-driven studies and industry-financed studies and whether co-authorship or involvement in a sub-trial in both academic and industry-initiated studies was likely to influence the recruitment rate. Academic-driven studies are initiated, led and conducted by investigators from universities or hospitals, and no pharmaceutical company is involved in the trial. The survey ended with a question about the importance of communication about the trial in social media: web-sites, digital newsletters, twitter, Facebook, YouTube, Instagram and blogs.
In January 2018, the final WIMSS-q (Additional file 2) was sent electronically by SurveyMonkey, an online survey tool, to all physicians and nurses who were actively involved in the EFFECTS trial, i.e., those listed in the delegation log at the time (N = 148). It was largely speaking the same group of people that had previously participated in the pilot, with the addition of a few who became active in the study after the pilot phase and the removal of a few since they had stopped working in the study before the final version of the questionnaire was sent out.
They were instructed to answer using all their accumulated knowledge and experience of RCT; the questions were not specific to the EFFECTS trial. Participation in the survey was voluntary. None of the questions required mandatory answers. We informed the participant on the first page of the questionnaire that it was voluntary and that their decision would not influence our contact with them; if they said no we did not send any reminders.
Everyone who participated was given financial compensation in the form of a cinema voucher (worth approximately 11 Euros).
We sent three consecutive reminders to non-responders within a three-week period using the SurveyMonkey system and an additional ten reminders to the non-responders within a period of five weeks. In addition to that, we sent a personal email to non-responders.
Number of questionnaires, data analyses and statistical methods
We had a 94% (139/148) response rate. The WIMSS-q took on average 13 min to complete.
The proportion of missing data within the questionnaires was low, below 3%. All data were exported from SurveyMonkey and entered into the SAS system from the SAS Institute (Cary, NC, USA) and descriptive statistics and graphical methods have been used to characterise the data.
Of 139 responders, 71% were women and 29% men (Table 1). Their mean age was 47 years (SD 11 years). There were 53% physicians and 47% nurses. Two responders (1%) did not state their occupation. Figure 3 is the participant flow diagram.
Sixty-six percent were not accustomed to working with clinical trials; EFFECTS was the first trial in which they have participated (Table 1). Nine percent were very experienced, which means that they had been involved in five or more trials or had carried out their own research. Most of the participants worked at an acute stroke unit (84%).
To succeed with inclusions in trials
The ways of ensuring that recruitment can meet its target are shown in Table 2. The most important factor is that the study personnel consider the research question being studied is relevant (97%). A simple consent procedure is also of importance (92%). Another highly valued factor is that the local principal investigator and the research nurse are highly engaged (87%) and that study-related follow-ups are simple and can be coordinated with the clinical follow-up (87%).
Other important factors are that the central team leading the trial is very enthusiastic (79%), and that the support team responds quickly to questions (89%) and that those leading the trial provide regular information about what is happening in the trial by means of digital newsletters or website updates (66%).
Eighty-four percent said that it was important that involvement in the trial is fun. Intentionally, we did not specify what fun was. Each participant had to interpret the question based on their own experience.
It is of less importance that the steering committee consists of well-known researchers (21%) or that one’s own centre contributes many patients or participants to the trial (39%). Fifty-three percent thought that it was important that there are regular investigation meetings and 51% wanted meetings with only research nurses.
Fifty percent agreed that it was relevant in some way to have co-authorship in a scientific article and that it would influence their dedication to the trial. Of these, 10% said that this would influence them very much.
Fifty percent agreed that the opportunity to propose an idea for a sub-trial or an article once the main trial has been completed would influence their inclusion of individuals. Among them, 23% agreed a lot or completely agreed (4 or 5 according to the Lickert scale).
The most important patient-related barrier for inclusion according to the study personnel shown in Table 3 was fear of side effects of the trial drug (35%). The second most important barrier was that the patient had language problems, aphasia/dysarthria (30%). Twenty percent mentioned that they thought that the patients had difficulties in understanding the importance and need for randomisation.
Table 4 shows the most important centre-related barriers to recruitment. Lack of time and resources devoted to research at the clinic (72%) were the most important. The absence of a research nurse was also an important factor (31%). A few thought that insufficient training in Good Clinical Practice affected inclusion in trials (5%) or feared that participation in the trial might harm the patient (3%). Insufficient financial compensation was not an important factor (5%).
Study-related barriers and complicating factors for inclusion of patients were: narrowly defined criteria for inclusion and exclusion (31%), complex and lengthy procedures for inclusion (17%) and comprehensive monitoring (12%) (Table 5).
Regarding social media, the most important factor was the presence of a study-specific website (79%) where you can provide information about the study, give access to essential documents and study specific tools such as the randomisation system.
Weekly digital web-letters from the central team were found to be of importance (51%). Twitter, Instagram, blog, YouTube or Facebook were not stated as being important.
Offering substantial financial compensation to the study team at the clinical and the university departments is more important in the case of an industry-funded study (73%) than that of an academic-driven study (45%).
In the two open questions about barriers and important measures for inclusion in RCTs at the beginning of the questionnaire we found that the screening procedure to find possible subjects for the trial was an important issue. Many stated that having a structured and organised screening process is the key to success.
In our study we found several factors that can be used by researchers running RCTs to improve the recruitment of patients. The most important factor for recruitment was that the study personnel consider the research question that is being studied to be relevant, which is in agreement with previous reports . This might seem obvious, but it is important to have a strong “why” for people to engage in a study. Having a dedicated local principal investigator and a committed research nurse is a prerequisite [21,22,23].
In the two open questions, many of the respondents stated that an organised screening process to identify patients is vital. This was something that the personnel found challenging in everyday clinical activities. Our own experience as trialists is that a structured and organised screening process is the key to success and you should have time in the daily routine to do this [10, 24].
When planning a study, it is important to understand what the recruitment difficulties can be and then preferably implement strategies before the problems arise. Donovan et al.  interviewed trialists, study personnel and patients in the early phase of a study to learn about recruitment difficulties and made subsequent changes to overcome problems. Doing this will ensure more efficient and effective recruitment. With this study we have made it clearer what the barriers for recruitment can be and that it is of importance to address the problems early in the process, when writing the protocol and planning the study.
Participants in our survey found it important that trials are simple and managed skilfully to support the local centres. Close contact and regular information from an experienced and flexible trial office team would seem to be important. These factors were also found in a similar study, a survey of a paediatric trial in the acute setting by Kaur et al. .
This can be achieved with weekly letters, emails and personal contact. The most commonly reported strategies to improve recruitment were newsletters, mail, regular visits and phone calls. The use of posters or placards at the clinic for patients, relatives and staff reminding them of the study can be one way to do this. In centres with inexperienced personnel this is even more important. It is imperative that those who lead the study consider site-specific issues and work individually with each site based on what their requirements are [26, 27].
Our study found some of the keys to reducing waste in future studies and simplifying research-related procedures both for recruitment and follow-up . The informed consent process must be comprehensible, which is especially important when it is the patient or the next of kin who make the decision, the entry criteria should be adjusted and appropriate to the group one intends to study, and it should be possible to coordinate these follow-ups with the clinical follow-up. Research must be integrated into the day-to-day work of the clinic .
The budget for a study, irrespective of whether it is academic or industry driven, must be large enough to allow activities such as trial meetings, training in trial-specific topics and participation in congresses. It is important to have a stable budget to achieve this. Also, small things such as shortbread or a cinema voucher can encourage study staff to go the extra mile as well as prioritise time for the study [30, 31].
The survey among the study personnel identified many barriers but also several promising methods by which to approach recruitment problems. As an example, the protocol and informed consent process should be simple and the study-related follow-up should be coordinated with the clinical follow-up. They also highlighted the importance of the availability and encouraging support of the central team in the event of questions.
The literature search made it clear that most of these strategies have never been the subject of research. We chose to focus on what is important for recruitment and have not investigated how to retain patients in studies, obtain compliance with a study drug or data quality.
The result of our WIMSS-q study could probably be generalised to other similar trials even outside the stroke area. We believe a relevant research question, a simple protocol and that it is easy to implement research in the daily clinical routine applies for all studies. There are, however, instances where these findings may not directly apply, for example, in patients with stroke which affects the brain and entails specific problems such as aphasia and fatigue, or other diseases, e.g., terminal pancreatic cancer, has its own recruitment problems that our survey certainly does not cover.
The study will add to knowledge about recruitment and management of RCTs. When planning a study, trialists should include recruitment strategies and evaluation of how they proceed in their trials. Finding which strategies are effective would be beneficial to the research community and to the society . Treweek et al. state in their Cochrane analysis that rather than developing and testing new strategies, the evidence base should be improved by replicating evaluations of existing strategies .
The strength of this study is that inexperienced personnel have expressed their thoughts about participating in an RCT. We found that if you design pragmatic studies, they can also be performed advantageously in inexperienced centres.
Our study has some limitations that may have influenced the results obtained. The people who responded to the survey form part of a network with links to those of us who are responsible for the survey. This may have influenced both the response rate and the content of the responses. Another possible confounding factor is that EFFECTS is a study with broad criteria and simple protocols that can also affect the answers. Many in the study are unaccustomed to studies, this being the first study they have ever worked with. This can thus affect the results of the study. Unfortunately, we were not aware of the article of Kaur et al. before developing our questionnaire. This oversight may have affected the design. Interestingly, it turned out that our thoughts were similar, which can be interpreted as we have drawn the same conclusions about what is important when recruiting patients in studies.
Further, if we had combined a mixed model design we would probably have gained deeper knowledge .
This study may not be generalised to all RCTs, but for academic-driven RCTs its relevance and simplicity is of significance.
For recruitment in an RCT to be successful, the research question must be relevant, and the protocol must be simple and easy to implement within the daily routine.
We registered the protocol at the Northern Ireland Hub for trials methodology research .
Availability of data and materials
The dataset for WIMSS-q will be made available by the corresponding authors on reasonable request.
Efficacy oF Fluoxetine – a randomisEd Controlled Trial in Stroke
Randomised controlled trial
What is Important for Making a Study Successful questionnaire
Al-Shahi Salman R, Beller E, Kagan J, Hemminki E, Phillips RS, Savulescu J, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014;383(9912):176–85.
Campbell MK, Snowdon C, Francis D, Elbourne D, McDonald AM, Knight R, et al. Recruitment to randomised trials: strategies for trial enrollment and participation study. The STEPS study. Health Technol Assess. 2007;11(48):iii ix-105.
Watson JM, Torgerson DJ. Increasing recruitment to randomised trials: a review of randomised controlled trials. BMC Med Res Methodol. 2006;6:34.
Gul RB, Ali PA. Clinical trials: the challenge of recruitment and retention of participants. J Clin Nurs. 2010;19(1–2):227–33.
Tudur Smith C, Hickey H, Clarke M, Blazeby J, Williamson P. The trials methodological research agenda: results from a priority setting exercise. Trials. 2014;15:32.
Treweek S, Lockhart P, Pitkethly M, Cook JA, Kjeldstrom M, Johansen M, et al. Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. BMJ Open. 2013;3(2):e002360.
Donovan JL, Rooshenas L, Jepson M, Elliott D, Wade J, Avery K, et al. Optimising recruitment and informed consent in randomised controlled trials: the development and implementation of the Quintet Recruitment Intervention (QRI). Trials. 2016;17(1):283.
Galbreath AD, Smith B, Wood P, Forkner E, Peters JI. Cumulative recruitment experience in two large single-center randomized, controlled clinical trials. Contemp Clin Trials. 2008;29(3):335–42.
Heinrichs N, Bertram H, Kuschel A, Hahlweg K. Parent recruitment and retention in a universal prevention program for child behavior and emotional problems: barriers to research and program participation. Prev Sci. 2005;6(4):275–86.
Adams M, Caffrey L, McKevitt C. Barriers and opportunities for enhancing patient recruitment and retention in clinical research: findings from an interview study in an NHS academic health science centre. Health Res Policy Syst. 2015;13:8.
Bower P, Brueton V, Gamble C, Treweek S, Smith CT, Young B, et al. Interventions to improve recruitment and retention in clinical trials: a survey and workshop to assess current practice and future priorities. Trials. 2014;15:399.
Kaur G, Smyth RL, Williamson P. Developing a survey of barriers and facilitators to recruitment in randomized controlled trials. Trials. 2012;13:218.
Treweek S, Altman DG, Bower P, Campbell M, Chalmers I, Cotton S, et al. Making randomised trials more efficient: report of the first meeting to discuss the Trial Forge platform. Trials. 2015;16:261.
Trial Forge. A systematic way to improve trial efficiency. 2019. Available from: https://www.trialforge.org/. Accessed February 2017.
Boots LM, de Vugt ME, Withagen HE, Kempen GI, Verhey FR. Development and initial evaluation of the web-based self-management program “Partner in Balance” for family caregivers of people with early stage dementia: an exploratory mixed-methods study. JMIR Res Protoc. 2016;5(1):e33.
Survey Monkey. Available from: https://www.surveymonkey.com/what-is-surveymonkey/?ut_source=footer. Accessed 31 May 2018.
Mead G, Hackett ML, Lundstrom E, Murray V, Hankey GJ, Dennis M. The FOCUS, AFFINITY and EFFECTS trials studying the effect(s) of fluoxetine in patients with a recent stroke: a study protocol for three multicentre randomised controlled trials. Trials. 2015;16:369.
Sullivan GM, Artino AR Jr. Analyzing and interpreting data from likert-type scales. J Grad Med Educ. 2013;5(4):541–2.
Likert R. A technique for the measurement of attitudes. Arch Psychol. 1932;22(140):55.
Treweek S, Pitkethly M, Cook J, Fraser C, Mitchell E, Sullivan F, et al. Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018;2:Mr000013.
Farrell B, Kenyon S, Shakur H. Managing clinical trials. Trials. 2010;11:78.
Treweek S, Littleford R. Trial management- building the evidence base for decision-making. Trials. 2018;19(1):11.
Ross S, Grant A, Counsell C, Gillespie W, Russell I, Prescott R. Barriers to participation in randomised controlled trials: a systematic review. J Clin Epidemiol. 1999;52(12):1143–56.
Wilson C, Rooshenas L, Paramasivan S, Elliott D, Jepson M, Strong S, et al. Development of a framework to improve the process of recruitment to randomised controlled trials (RCTs): the SEAR (Screened, Eligible, Approached, Randomised) framework. Trials. 2018;19(1):50.
Kaur G, Smyth RL, Powell CV, Williamson P. A survey of facilitators and barriers to recruitment to the MAGNETIC trial. Trials. 2016;17(1):607.
Berge E, Stapf C, Al-Shahi Salman R, Ford GA, Sandercock P, van der Worp HB, et al. Methods to improve patient recruitment and retention in stroke trials. Int J Stroke. 2016;11(6):663–76.
McDonald AM, Knight RC, Campbell MK, Entwistle VA, Grant AM, Cook JA, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7:9.
Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials? Stat Med. 1984;3(4):409–22.
Donovan JL, Lane JA, Peters TJ, Brindle L, Salter E, Gillatt D, et al. Development of a complex intervention improved randomization and informed consent in a randomized controlled trial. J Clin Epidemiol. 2009;62(1):29–36.
Berge E, Al-Shahi Salman R, van der Worp HB, Stapf C, Sandercock P, Sprigg N, et al. Increasing value and reducing waste in stroke research. Lancet Neurol. 2017;16(5):399–408.
Fletcher B, Gheorghe A, Moore D, Wilson S, Damery S. Improving the recruitment activity of clinicians in randomised controlled trials: a systematic review. BMJ Open. 2012;2(1):e000496.
Mapstone J, Elbourne D, Roberts I. Strategies to improve recruitment to research studies. Cochrane Database Syst Rev. 2007;(2):Mr000013.
Rooshenas L, Scott LJ, Blazeby JM, Rogers CA, Tilling KM, Husbands S, et al. The QuinteT Recruitment Intervention supported five randomized trials to recruit to target: a mixed-methods evaluation. J Clin Epidemiol. 2019;106:108–20.
SWAT store: The Northern Ireland network for trials methodology research. Identifying opinions on the features needed for making a study successful SWAT64: Queen's University Belfast. Available from: https://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/SWATSWARInformation/Repositories/SWATStore/. Accessed 23 May 2017.
We thank all the personnel in the EFFECTS study that responded to the survey as well as all the patients who participated in EFFECTS. We thank Professor Rustam Al-Shahi Salman, and Professor Martin Dennis, both from the University of Edinburgh, UK, for their valuable advice with the study.
This study was conducted within the funding for the EFFECTS study. Currently EFFECTS receives grants from the Swedish Research Council, the Swedish Brain Foundation, the Swedish Heart-Lung Foundation, King Gustav V and Queen Victoria’s Foundation of Freemasons, the Swedish Stroke Association and the Swedish Society of Medicine; all funders are non-commercial. None of the funders have any influence on the planning, conduct or interpretation of the study.
EFFECTS has approval from the ethics board 30th September 2013 (reg. no. 2013/1265–31/2); this study WIMSS-q 2017/1284–31/1 was approved 9th August 2017. The questionnaire was sent electronically by SurveyMonkey, an online survey tool, to all physicians and nurses who were actively involved in the EFFECTS trial, i.e., those listed in the delegation log at the time. Participation in the questionnaire was voluntary and the participants did not need to give any reason why they did not want to answer. None of the questions required mandatory answers.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional file 1.
Changes in the questionnaire after the think-aloud and the pilot phase.
Additional file 2.
Final version of the questionnaire.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Isaksson, E., Wester, P., Laska, A.C. et al. Identifying important barriers to recruitment of patients in randomised clinical studies using a questionnaire for study personnel. Trials 20, 618 (2019). https://doi.org/10.1186/s13063-019-3737-1
- Randomised controlled trials