Embedding qualitative research in randomised controlled trials to improve recruitment: findings from two recruitment optimisation studies of orthopaedic surgical trials

Abstract

Background

Recruitment of patients is one of the main challenges when designing and conducting randomised controlled trials (RCTs). Trials of rare injuries or those that include surgical interventions pose added challenges due to the small number of potentially eligible patients and issues with patient preferences and surgeon equipoise. We explore key issues to consider when recruiting to orthopaedic surgical trials from the perspective of staff and patients with the aim of informing the development of strategies to improve recruitment in future research.

Design

Two qualitative process evaluations of a UK-wide orthopaedic surgical RCT (ACTIVE) and mixed methods randomised feasibility study (PRESTO). Qualitative semi-structured interviews were conducted and data was analysed thematically.

Setting

NHS secondary care organisations throughout the UK. Interviews were undertaken via telephone.

Participants

Thirty-seven health professionals including UK-based spinal and orthopaedic surgeons and individuals involved in recruitment to the ACTIVE and PRESTO studies (e.g. research nurses, surgeons, physiotherapists). Twenty-two patients including patients who agreed to participate in the ACTIVE and PRESTO studies (n=15) and patients that declined participation in the ACTIVE study (n=7) were interviewed.

Results

We used a mixed methods systematic review of recruiting patients to randomised controlled trials as a framework for reporting and analysing our findings. Our findings mapped onto those identified in the systematic review and highlighted the importance of equipoise, randomisation, communication, patient’s circumstances, altruism and trust in clinical and research teams. Our findings also emphasised the importance of considering how eligibility criteria are operationalised and the impact of complex patient pathways when recruiting to surgical trials. In particular, the influence of health professionals, who are not involved in trial recruitment, on patients’ treatment preferences by suggesting they would receive a certain treatment ahead of recruitment consultations should not be underestimated.

Conclusions

A wealth of evidence exploring factors affecting recruitment to randomised controlled trials exists. A methodological shift is now required to ensure that this evidence is used by all those involved in recruitment and to ensure that existing knowledge is translated into methods for optimising recruitment to future trials.

Trial registration

ACTIVE: (ISRCTN98152560). Registered on 06/03/2018. PRESTO: (ISRCTN12094890). Registered on 22/02/2018,

Background

Participant recruitment is a key challenge for randomised controlled trials (RCTs). Failing to reach recruitment targets can have significant cost implications if funding is stopped or extensions are required and trials that do not recruit to target are underpowered to detect an effect [1]. Trials of surgical interventions, especially those that compare surgical and non-surgical interventions face recruitment challenges, due to the irreversibility of surgical treatments and issues with surgeon equipoise [2,3,4,5]. The stakes are particularly high for trials of rare fractures or injuries, where there are a limited number of eligible patients and so there is an added impetus to recruit as many eligible patients as possible.

There is growing evidence on the barriers and facilitators to surgical trial recruitment [3, 6]. A mixed methods systematic review synthesised the evidence on recruitment to surgical trials from the patient perspective [1]. The review, which included 34 papers (11 quantitative and 21 qualitative), concluded that whether a patient chooses to participate or not in a RCT is influenced by multiple inter-related factors such as patients’ perceptions of the risk and benefits of participation and their treatment preferences. The authors recommended that future trials consider a patient-centred approach to trial recruitment that tailors information to the individual patient. For recruitment to be effective, it is also important to take stock of staff’s experiences of recruitment to ensure that future strategies for improving recruitment consider not only what is important to patients, but the practicalities of recruiting patients in a complex healthcare system [1].

We present the findings of two qualitative process evaluations undertaken during the PRESTO (Pragmatic Randomised Evaluation of Stable Thoracolumbar fracture Treatment Outcomes) feasibility study [7] and ACTIVE (Articular type C pilon fracture Trial comparing Internal Versus External fixation) trial’s internal pilot. The main a priori research question for both process evaluations was how to best optimise recruitment by exploring patients, surgeons and local recruiting staffs’ views and experiences of the interventions and trial processes. In this paper, we draw on the work of Phelps et al. [1] and use the findings from our two process evaluations to highlight how recruitment to future orthopaedic RCTs may be improved from the perspective of trial recruiters and patients.

Methods

Design

Two qualitative evaluations of orthopaedic surgical trials were undertaken and consisted of interviews with patients who agreed to take part in the ACTIVE or PRESTO studies, interviews with patients who declined participation and interviews with ACTIVE and PRESTO trial recruiters (surgeons, physiotherapists, research nurses). Interviews with surgeons from across the UK (who were not part of recruiting teams) were conducted during PRESTO as one of the main aims of the study was to understand the current treatment of stable thoracolumbar fractures. Data collection took place during the PRESTO study’s 12-month recruitment phase and the ACTIVE study’s 12-month internal pilot recruitment phase (Table 1).

Table 1 ACTIVE and PRESTO study details

Sampling and recruitment

To obtain maximum variation, we sought to purposively sample patients on the basis of age, gender, recruitment site and treatment received. Given the lower than anticipated recruitment to the main PRESTO and ACTIVE studies and process evaluations, we adopted a convenience sampling frame with selection based on those who agreed to interview. In both studies, all patients, irrespective of whether they agreed to participate in the trial were invited to take part in the qualitative study and were provided with a separate participant information sheet and consent form. A contact details form was also completed and transferred to the qualitative team for those that declined to participate in the ACTIVE or PRESTO studies. Informed consent was obtained prior to each interview.

For the ACTIVE trial, initially, we aimed to only interview staff who had been directly involved in recruiting patients. Due to the staggered opening of sites and lower than anticipated recruitment of patients, we also invited staff at sites open for, but just prior to recruitment. ACTIVE trial recruiters were invited to interview via email.

For PRESTO, trial recruiters from the 3 participating sites were interviewed. In addition, snowballing techniques were used to ensure that a wide range of surgeons from across the UK were interviewed. This approach is appropriate when the number of experts in the field is relatively small and so many surgeons are known to each other [8]. However, we utilised a range of additional recruitment strategies to ensure interviewees represented a range of geographical locations, specialties (orthopaedics, neurosurgeons) and grades (consultants, registrars). We asked clinical co-applicants to forward recruitment emails to surgical colleagues, both in and outside of PRESTO sites; advertised for participants through the British Association of Spine Surgeons (BASS) newsletter and through a regional network of spinal surgeons; and by sending an invitation email to those who expressed an interest in the qualitative study through the PRESTO survey (n=19).

Participants

Thirty-seven surgeons and trial recruiters (research nurses, research physios) were interviewed during the PRESTO (n=19) and ACTIVE (n=18) process evaluations. Surgeons across both studies represented different grades (registrars and consultants) and specialties (orthopaedics, neurosurgery). Additionally, 15 patients who agreed to take part in the PRESTO (n=5) and ACTIVE (n=10) process evaluations were interviewed. ACTIVE interviewees represented a range of genders and treatment allocations. However, all but one of the PRESTO patient interviewees were male and received surgical fixation. Interviews were conducted with 7 patients that declined to participate in the ACTIVE trial, no patients that declined to participate in PRESTO were interviewed.

Semi-structured interviews with patients, surgeons and trial recruiters

All interviews were semi-structured, were conducted via telephone and followed topic guides that were developed by the research team and clinical co-applicants. Interviewees were asked a range of questions, which aimed to explore how patients were approached to take part in the two studies, trial procedures and current treatment. Further details of the topics covered during interviews can be found in Additional files 1, 2, 3, 4, 5 and 6.

Analysis

Data collection and analysis were undertaken by AS, a health services researcher with no clinical training or prior experience of research in orthopaedic surgery. All interviews and consultation recordings were audio-recorded and transcribed verbatim. For both studies, an initial thematic analysis was undertaken of all interviews and consultation recordings for each individual study [9]. Themes were largely deductive and derived from study topic guides, whilst also allowing for emergent themes. Analyses for both studies was also influenced by ‘initial impressions’ which were recorded immediately in an excel spreadsheet and involved documenting key issues under a priori headings that were devised by the research team, topic guide and clinical co-applicants. Documenting initial impressions allowed for early familiarisation and preliminary coding of our data and provided a mechanism for highlighting key issues with recruitment to the Trial Management Group and recruitment sites throughout data collection. A secondary analysis was then conducted where the findings from both studies were mapped onto the themes identified in Phelps’ systematic review [1], with particular attention given to identifying themes relating to optimising recruitment.

Results

First, we present the findings of both studies according to the three main themes that were identified by Phelps et al.’s [1] as influencing recruitment and retention to surgical trials—making sense, weighing up and trust. Given the significant amount of evidence that already exists around these themes [1, 10,11,12,13,14,15], we present our findings using summary tables to highlight the key issues that were identified during ACTIVE and PRESTO. We then provide a descriptive account of ‘eligibility criteria’ and ‘patient pathways’, two themes that were identified as influencing recruitment and retention during ACTIVE and PRESTO which are not featured in Phelps et al.’s review [1].

Making sense

Randomisation

For patients to agree to participate in an RCT, they first need to ‘make sense’ of randomisation and the influence it has on their treatment and involvement in their care (Table 2). Randomisation was a new concept for the majority of patients, which when described using trial jargon made the concept difficult to understand. In the most part, patients recalled randomisation; however, many had undergone significant trauma and so were heavily medicated when approached to take part in the study. As a result, patients were concerned that they lacked capacity at the time of recruitment and felt this restricted their ability to understand what was being proposed and make informed decisions. However, trial recruiters described various methods that were used during PRESTO to ensure capacity, for example, patients were asked to reiterate study aims before providing their consent.

Table 2 Key findings relating to randomisation

Communication

How the trial is communicated to patients is also a critical factor influencing a patient’s decision to take part in a RCT. In both studies, the perceived quality of information sheets, information about treatment and recovery and when patients are approached to take part were pivotal to ensuring effective communication and influenced whether patients chose to take part (Table 3).

Table 3 Key findings relating to communicating research to patients

Equipoise

Patients and trial recruiters provided various examples of situations where equipoise had been either deliberately, or unintentionally undermined (Table 4).

Table 4 Key findings relating to equipoise

Weighing up

Patients’ decisions to participate in RCTs are underpinned by being in equipoise and their understanding of randomisation. However, these decisions also require patients to trade off various perceived benefits of participation, such as altruism against treatment preferences, personal circumstances and the perceived risks of randomisation (Table 5).

Table 5 Factors that patients consider when ‘weighing up’ their participation in a RCT

Trust

If patients are to take part in a surgical trial, they need to feel confident and have trust in their surgeon and research team. When discussing how to optimise recruitment, it was considered crucial by the trial team to think about who is involved in recruitment consultations (Table 6). Whilst there was agreement that recruitment consultations should involve senior clinicians and the research team, the practicalities of involving both teams in large numbers of recruitment consultations were acknowledged, with some sites opting for the research team and clinicians to approach patients independently to fit around conflicting work schedules and time constraints.

Ensuring clinical teams are engaged throughout research studies was also considered important by both trial recruiters and patients not only for ensuring the trust of patients but to maximise recruitment, particularly in studies of rare conditions where there is pressure to recruit every eligible patient.

Table 6 Key findings relating to trust

Patient pathways

Patients are often admitted via complex referral pathways and as a result, may come into contact with clinicians from a range of specialties, depending on the specific injury (emergency department (ED), neurosurgery, orthopaedics) or from referring hospitals prior to being approached to take part in orthopaedic surgical trials. In PRESTO, the different ways that patients with thoracolumbar fractures could be referred to sites was a particular issue, with one site receiving referrals from up to 15 hospitals. The reduced awareness of on-going research studies, issues with equipoise and different treatment preferences of staff that are not involved in recruitment can mean that patients have multiple conversations about the treatment they are ‘likely’ to receive before official recruitment consultations and may lead to distrust in research and trial processes and/or patient’s being ‘biased’ towards specific treatments. The time between transfer of patients from local hospitals to Major Trauma Centres or hubs can mean that by the time patients are approached to take part in a research study they may no longer meet the eligibility criteria for the specific trial, for example they have already started to heal and in some cases receive non-operative treatment such as braces for stabilisation.

We generally want to see them within a week of them sustaining the injury and a few times it has not been possible to arrange transport from the local hospital up to our centre, and that has caused patients to miss the reasonable timeframe that we would like to follow these patients, for example if they reach five to six weeks following the injury by the time to come to the clinic, then it is a moot point to then discuss surgical or operative options. Because conservatively they have reached that point in their treatment, where the fracture could have reasonable been expected to heal and we wouldn’t be doing justice to either the patient or the trial to then discuss the two separate options for them. (PRESTO staff interview 06)

Communication between patients and specialties or clinicians that were not involved in trial recruitment was thought to have caused some patients across both studies to develop treatment preferences and view randomisation as unacceptable. Insufficient buy-in from clinical colleagues was seen by PRESTO recruiters to exacerbate this issue. Whilst attempts were made during PRESTO to increase engagement of neurosurgeons through multiple departmental presentations the difficulties of asking for a culture change ‘ahead of evidence’ was acknowledged particularly given the issues with surgical equipoise, the comparatively low levels of research activity in neurosurgical departments and the fact that neurosurgeons routinely provide the perceived less resource-intensive option of conservative management.

The neurosurgery team here are not the most helpful. (Laughter) I don’t know whether that's because we haven’t communicated the study very well or whether that’s just their mentality to research. We have decided that we will do a presentation to them again to make sure that it’s not a fault on our side. You’ll probably hear (name of staff) talk about this as well, but we did notice a couple of the patients that we were screening were being reviewed or presented in A&E and then neurosurgery registrars would be going down to review them on the request of A&E doctors. Then they would discharge them from there, so they weren’t getting admitted, which meant that those patients who have been told that they don’t need to be managed with an- they essentially just get discharged with a brace. A couple of those patients we, potentially, could have included in PRESTO, but because we missed them due to- I guess the registrars were completely unaware of the trial. (PRESTO staff interview, 10)

There were particular concerns amongst recruiters in both trials that treatment conversations between patients and EDs lead to patients being ‘biased’ towards a specific treatment. In addition to undermining equipoise, one patient described how receiving a call to return to hospital to discuss the possibility of surgery and participate in a trial after being discharged with either no or conservative treatment caused anxiety and provided a poor standard of care. Engaging ED doctors ahead of a full trial was considered a particular challenge given the pressure on ED to treat patients quickly to avoid affecting performance targets and the fact that patients are often admitted ‘out of hours’ which was perceived to increase the likelihood of patients being seen by doctors with a lower awareness of research studies based in other departments. High staff turnover, particularly of surgical registrars meant that Junior Doctors were also considered to be prone to inadvertently influencing patient expectations and be unaware of ongoing research protocols.

Unfortunately these fractures come in late at night or early in the morning and they’re already seen by a junior member of the team. The junior staff change over so frequently, they’re not aware of the trial, and they just go by…..meaning for stable fractures don’t fix, for unstable fractures, fix, because that’s what you’ve been taught traditionally. Once they’ve told them in A&E that they don’t need an operation, because this trial is all about stable fractures, then it is quite difficult to convince the patients otherwise. (PRESTO staff interview)

Variation in current practice and strong treatment preferences meant that the influence of ‘local logistics’, having strict eligibility criteria and carefully considering how to approach sites in future trials was considered crucial. Presenting a future trial, planned protocol and eligibility criteria at key clinical meetings was also suggested as a means to secure buy-in from the clinical community. Importance was placed on having centralised centres for recruitment with it considered unfeasible for referring hospitals to be involved in recruitment. Engaging staff from other specialties at recruiting sites was also considered important. Despite this, there were doubts as to the feasibility and implications on time and resources of involving multiple specialties in recruitment to a future trial, a number of suggestions were proposed and are displayed in Table 7.

Table 7 Involving multiple specialties in recruitment to future orthopaedic surgical trials

Eligibility criteria

Difficulties operationalising eligibility criteria were reported across both trials and were largely due to clinical teams interpreting clinical criteria differently. For example, in PRESTO, there was a lack of consensus as to what constitutes a stable thoracolumbar fracture, whilst in ACTIVE there was disagreement as to the definition of an open fracture. Other factors which were perceived to affect the number of eligible patients during PRESTO included: difficulties obtaining consent and completing study documentation for patients for whom English is not their first language; mental health; frailty; patients living in a different area to the treating hospital and so wanting to be followed up locally and patients at high risk of infection or of encountering wound problems and wound breakdown. It was however suggested that staff may be ‘hiding behind personal biases’ rather than excluding patients for ‘objective reasons’. Despite having screened approximately 80% of the predicted numbers of patients overall in the PRESTO study, surgeons reported that the target population was smaller than anticipated—the majority of patients were either considered ‘very stable’ or ‘very unstable’; therefore, far fewer patients met the eligibility criteria than was predicted.

I guess, perhaps, that will be much more useful to look at, at the end, when we see the trends really clearly and you can say, “Well, okay, this was a huge number of people we excluded for this reason. Was that actually justified, in terms of the science that we have out there at the moment, to say, “This group of patients should have been managed as they were, and excluded”? (PRESTO, staff interview)

Discussion

This paper reports the findings of two qualitative studies that were conducted during a large, multi-centre, UK-wide RCT’s internal pilot phase (ACTIVE) and a mixed methods feasibility study’s (PRESTO) recruitment period. We used a recent mixed methods review of patients’ experiences of being recruited to surgical trials as a framework for reporting our findings [1]. Our data mapped onto the key themes identified by Phelps [1] and suggest that a patient’s decision to take part in a RCT is determined by how they can ‘make sense’ of the trial and the impact of participation on patient care and daily life; a process of ‘weighing up’ where patients trade off the risks and benefits associated with participation and ‘trust’ in the treating clinicians and the research team.

We also identified two additional factors that affect recruitment, which are not reported by Phelps—the impact of patient pathways and how eligibility criteria are interpreted and applied by clinical staff [1]. Although recognised as barriers to trial recruitment [10, 11, 15], they have been given less attention in the recruitment optimisation literature than factors such as equipoise, randomisation and communication. The relative lack of attention paid to patient pathways may in part be due to the differential impact this will have across trials. The complexity of patient pathways will vary according to the surgical area of interest and is likely to be site specific. Some approaches to addressing the issues would potentially require a large investment of resources for both trials units and site staff. For example, many of the strategies for addressing issues arising from complex patient pathways that we propose in Table 7 would require trial recruiters to ensure that staff from multiple specialties are aware of a trial and are prepared to support the identification and recruitment of patients throughout a trial’s recruitment period. The complexities of this should not be underestimated, particularly when considering: the implications of staff turnover and part-time workers; the number of sites and specialties that can be involved in recruitment, particularly in large national trials such as ACTIVE (n= 30), the difficulties of balancing recruitment against routine clinical work; and the number of RCTs and research studies that can be on-going at any given site. In addition to these practical issues, problems with equipoise and the strong and divided opinions of clinicians around current treatment often underpinned many of the problems we identified with patient pathways and operationalising eligibility criteria.

During PRESTO, despite trial recruiters using well established and routinely used methods for determining whether a patient has capacity to provide informed consent, some PRESTO patients perceived themselves to lack capacity at the time of recruitment. The discrepancy between how staff and patients perceived the informed consent process during the PRESTO trial suggests that current methods for establishing capacity may need to be improved. This finding speaks to a larger ongoing discussion relating to the consent process in research. Much of this discussion has focussed on moving away from our current emphasis on a single moment of consent to an ongoing consent dialogue with research participants [16, 17]. However, this is difficult to achieve in clinical practice and has significant implications for how we currently approach consent within trials as it would require all stakeholders (including researchers and ethics committees) to adapt to what might potentially become a less well-defined process. The CONSULT study [18] is an on-going NIHR funded study which is focussed on improving the inclusion of adults lacking capacity in research through a decision support intervention to help family members to make decisions about research. Whilst this could equally be applied to potential trial participants for whom lack of capacity is transient, it is not the only solution. A collective effort is required to develop multiple strategies and/or interventions to improve and optimise consent processes which consider the needs of different populations and contexts.

The mixed methods systematic review conducted by Phelps and a recent Cochrane qualitative evidence synthesis provide two comprehensive evaluations of the evidence on factors affecting recruitment to RCTs from the perspective of trial participants [1, 15]. When combined with our findings, which incorporated the perspective of both patients and trial recruiters, it is clear that the same issues relating to equipoise, how randomisation is communicated and the impact of patient preferences are repeatedly reported as barriers to recruitment across a range of RCTs. It could therefore be argued that we have reached a saturation point of evidence exploring barriers and facilitators to trial recruitment. We do not dispute the value of including recruitment optimisation interventions within RCTs, especially bespoke solutions which target trial and site-specific issues in detail, e.g. QuinteT (Qualitative research integrated within trials) [19,20,21]. However, we suggest that future work focusses on ensuring that existing evidence is disseminated, applied and used to inform the design of process evaluations and recruitment to future RCTs if we are to avoid further research waste [22]. More specifically, there is a need for trials units to ‘lead by example’ and ensure that their staff are up to date on the current evidence on recruitment optimisation and directly apply this to site set-up, development of trial documentation (e.g. participant information sheets), trial design and recruitment. Training such as ‘Granule’ [23] should be completed by all those involved in recruitment and research teams to maximise the efficiency of site initiation visits and to ensure that study documentation and recruitment consultations are in line with the latest evidence. Lastly, developing methods that apply existing evidence, but require minimal resources, should be a methodological priority if we are to make use of the existing evidence base and optimise recruitment to future trials.

Strengths and limitations

We used a recent mixed methods systematic review [1] as a framework for reporting our findings, which has enabled us to go beyond a descriptive account and place our findings in the context of existing evidence. Our findings are based on data that was collected from patients who agreed and declined to participate in a trial and the views of surgeons and trial recruiters. This enabled us to obtain a more detailed understanding of the complexities surrounding trial recruitment than would have been possible had we have only explored this issue from the perspective of one stakeholder group. Trial recruiters across both studies expressed uncertainties surrounding the appropriateness of interviewing patient decliners despite this being established practice. In the ACTIVE study, we were able to mitigate against this and provide reassurance through site initiation visits and additional, separate follow up phone calls with sites about the qualitative study and this may account for the discrepancy in recruitment of patient decliners between the two trials. As such, we would recommend undertaking this additional preliminary work when designing future process evaluations.

Conclusion

This study has highlighted the challenges of recruiting patients to RCTs. In this paper, we presented findings from two qualitative process evaluations, which were conducted to inform recruitment to two orthopaedic surgical trials. When considering our findings and those of two recent systematic reviews of the evidence on recruitment to RCTs, we argue that a shift is required in how process evaluations are conducted. We suggest that to optimise recruitment, the wealth of knowledge that has been accumulated is used to inform recruitment and that priority is given to developing less resource-intensive interventions, which use existing evidence to optimise recruitment.