Enhancing practitioners’ confidence in recruitment and consent in the EcLiPSE trial: a mixed-method evaluation of site training – a Paediatric Emergency Research in the United Kingdom and Ireland (PERUKI) study

Abstract

Background

EcLiPSE (Emergency treatment with Levetiracetam or Phenytoin in Status Epilepticus in children) is a randomised controlled trial (RCT) in the United Kingdom. Challenges to success include the need to immediately administer an intervention without informed consent and changes in staffing during trial conduct, mainly due to physician rotations. Using literature on parents’ perspectives and research without prior consent (RWPC) guidance, we developed an interactive training package (including videos, simulation and question and answer sessions) and evaluated its dissemination and impact upon on practitioners’ confidence in recruitment and consent.

Methods

Questionnaires were administered before and immediately after training followed by telephone interviews (mean 11 months later), focus groups (mean 14 months later) and an online questionnaire (8 months before trial closure).

Results

One hundred and twenty-five practitioners from 26/30 (87%) participating hospitals completed a questionnaire before and after training. We conducted 10 interviews and six focus groups (comprising 36 practitioners); 199 practitioners working in all recruiting hospitals completed the online questionnaire. Before training, practitioners were concerned about recruitment and consent. Confidence increased after training for explaining (all scale 0–5, 95% CIs above 0 and p values < 0.05): the study (66% improved mean score before 3.28 and after 4.52), randomisation (47% improvement, 3.86 to 4.63), RWPC (72% improvement, 2.98 to 4.39), and addressing parents’ objections to randomisation (51% improvement, 3.37 to 4.25). Practitioners rated highly the content and clarity of the training, which was successfully disseminated. Some concerns about staff availability for training and consent discussions remained.

Conclusions

Training improved practitioners’ confidence in recruitment and RWPC. Our findings highlight the value of using parents’ perspectives to inform training and to engage practitioners in trials that are at high risk of being too challenging to conduct.

Background

Many clinical trials experience difficulties in recruiting the desired number of participants, resulting in underpowered studies and continued use of healthcare treatments that are not informed by scientific evidence [1,2,3,4]. Trials involving time-critical interventions in paediatric emergency and critical care have additional practical and ethical challenges. These include, high staff turnover due to junior physician rotations, no time to seek prior informed consent, and practitioner anxiety about approaching families for consent after a trial intervention has already been administered [5].

Emergency Treatment with Levetiracetam or Phenytoin in Status Epilepticus in Children (EcLiPSE) was a 30-site, un-blinded, pragmatic and randomised controlled trial that explored second-line treatment (levetiracetam versus phenytoin) of convulsive status epilepticus in children [6]. EcLiPSE was one of the first UK paediatric clinical trials of an investigational medical product (CTIMP) to randomise and treat patients without seeking prior informed consent from parents. As there is no time to seek written informed consent in a life-threatening situation, practitioners approached parents as soon as possible after the child has stabilised to inform them that their child has already been entered into a clinical trial and discuss the use of their child’s data and continued follow-up [7, 8]. This is research without prior consent (RWPC, also known as deferred consent). Challenges to the success of EcLiPSE included: practitioner concerns and inexperience in RWPC; trained staff leaving due to junior physician rotations; use of an anti-epileptic medication (levetiracetam) that is not the standard anticonvulsant used in this clinical setting; and the likelihood that randomised patients are moved between departments or hospitals, which increases the size of trial team and complexity of trial set up.

The CONseNt methods in paediatric Emergency and urgent Care Trials (CONNECT) study [7, 9] explored parent and practitioner perceptions and experiences of RWPC in paediatric and neonatal trials. CONNECT found that practitioners with no experience of RWPC might have negative perceptions of this consent process [10]. Parents and experienced practitioners who participated in CONNECT indicated support for this approach to consent, although some parents raised concerns about trials which involved interventions not commonly used in routine clinical practice. CONNECT interviews also explored parents’ perspectives on the EcLiPSE trial design, including approach to recruitment, consent, and patient information materials [11]. Parents made specific recommendations on potential approaches to recruitment and consent in EcLiPSE. This included the need to appropriately time the EcLiPSE consent discussions and to discuss the safety of the trial interventions, as well as how both EcLiPSE interventions are used in routine clinical practice [11].

A recent systematic review highlighted the need to develop practitioner training to improve recruitment and consent in trials [12]. Site initiation visits (SIVs) are used to engage practitioners with the study aims and procedures and how to deliver the clinical trial protocol [5]. In the early design stage, we recognised the need for a comprehensive training package to educate and help provide practitioners with confidence in recruitment and consent. The SIV training package included specific training on RWPC and recommendations made by parents to inform approaches to recruitment and consent seeking in EcLiPSE using CONNECT study findings and associated guidance [7, 9, 11, 13]. In this embedded study (called the Consent study) we evaluated the effectiveness of the EcLiPSE SIV training on practitioners’ confidence in recruitment and consent seeking as well as the effectiveness of training dissemination.

Methods

Study design and setting

We chose a mixed-method longitudinal design [14], to provide us with different forms of data and insights from multiple practitioner perspectives throughout the trial as part of an iterative process [15,16,17]. We designed a semi-structured questionnaire (see Additional file 1) administered immediately before (Part A) and after (Part B) the SIV training. This included four statements aimed to assess practitioners’ confidence in aspects of recruitment and consent with five response options (‘strongly disagree’ to ‘strongly agree’). We then conducted telephone interviews with recruitment and training leads at the first sites open and conducted focus groups drawn from high and low recruiting sites at the end of the first year of recruitment with a mix of nurses, research nurses and consultants. We also sent an online questionnaire to all sites in last phase of the trial (approximately 8 months before trial closure). The online questionnaire was sent to lead practitioners and research nurses at the study sites and they were asked to distribute it to staff at their site who were trained in EcLiPSE. Topic guides (see Additional file 2) and the online questionnaire were designed to explore recruitment, RWPC experience, barriers to training and any problems and potential solutions to trial conduct.

Selection of participating sites

EcLiPSE sites were part of the Paediatric Emergency Research in the United Kingdom and Ireland (PERUKI), a paediatric emergency medicine (PEM) research collaborative, which includes tertiary and district general hospitals, with varying levels of research experience [5]. In 26 of the 30 EcLiPSE sites KW, LR or AH provided a brief description of the evaluation before the opening presentation (see Table 1) and invited practitioners who intended to stay for the full training to participate in the evaluation by completing Part A of the questionnaire before and Part B at the end of training. The need for the SIV evaluation was identified after the first SIV. Three other sites were not included due to either their participation in a different training evaluation study (n = 2) or full SIV training adapted due to low attendance (n = 1). Questionnaire completion was taken as indication of consent. Personal details were not requested to ensure anonymity.

Table 1 Site training content and delivery

LR (female health psychologist, CPsychol) emailed lead practitioners at 16 high (exceeding recruitment target) and low (below recruitment target) recruiting sites inviting them and their colleagues to participate in a focus group at their site. Potential participants were not known to the qualitative team who had no prior experience of SIV training. Telephone interviews were conducted with lead practitioners at sites open in the first year of trial recruitment. Practitioners were eligible if they been involved in the recruitment and/or consent of at least two trial participants. Consent was sought for interview and focus group participation, including consent for audio recording. The trial co-ordinator (AH) emailed sites 8 months before the scheduled trial end date and invited staff to complete the online questionnaire. ML sent email reminders on behalf of the trial team and PERUKI. It was anticipated that some of the same staff who took part in a telephone interview or focus groups would also complete the online questionnaire.

Training package

We developed the recruitment and consent training, including a RWPC-scenario video, using CONNECT study guidance on RWPC, pre-trial feasibility work involving parents [7, 11] and Clinical Trials Research Centre (Clinical Trials Unit) Standard Operating Procedure guidance [18]. Printed materials were provided which included: guidance on RWPC [7]; published trial feasibility findings [11] and trial materials, such as participant information sheets, screening form and infusion guidelines (available on request). To assist ongoing training dissemination for new staff (e.g. after junior physician rotations) all materials were given to the site lead practitioner on a USB stick and made available to all practitioners via the study website.

Site training included presentation of protocol, screening and randomisation simulation (video and real-time), RWPC (presentation and consent discussion scenario video [https://www.liverpool.ac.uk/psychology-health-and-society/research/connect/resources/]), safety and reporting and question and answer sessions (see Table 1). Emergency department (ED) staff not expecting to be involved in recruitment discussions with families were given a brief overview of the approach to recruitment and RWPC in EcLiPSE (see Section 4, Table 1) and provided with the option of staying for the full RWPC training (see Section 5, Table 1).

Two to five (mean 3.2, mode 3) EcLiPSE team members delivered the EcLiPSE training; typically, the trial co-ordinator, a consultant-level physician (in PEM, paediatric neurology, or chief investigator) or research nurse and a member of the embedded study team (social scientist or health psychologist). Site training lasted for approximately 4 h and took place in a hospital meeting room.

Analysis

Questionnaire data were entered into SPSS. Descriptive statistics are presented with percentages and the chi-square test for trend, paired-samples t test and Wilcoxon signed-rank test (confidence interval 95%) used as appropriate. Questionnaires with recruitment and consent-related data missing were excluded from analysis. To investigate the presence of informative missing data, the results of those who completed only the Part A (before training) questionnaire were compared with those who completed the Part A and Part B (after training) questionnaire. This was also undertaken for those who completed only the ‘after’ questionnaire. LR and KW used NVivo 10 Software to assist in the organisation and coding of interview and focus group data and free-text questionnaire responses. Qualitative thematic analysis [17] was broadly interpretive and iterative [19, 20]. Interviews were conducted until a representative from each site open in the first year of recruitment had been interviewed. Focus groups were conducted until data saturation (no new major themes were discovered in analysis [17, 21]). Our approach to synthesising qualitative and quantitative data [22] drew on the constant comparative method [23, 24].

Results

Characteristics of the study subjects

A total of 333 practitioners received EcLiPSE training by attending SIVs. An average of 11 staff attended each SIV (range 3–18). Three hundred and twelve (94%) practitioners from 26 of the 30 (87%) sites anticipated being involved in consent processes. One hundred and forty-nine of 312 (48%) practitioners were eligible for inclusion as they anticipated staying for the full training session. Clinical commitments impacted attendees’ ability to attend the entire SIV; consequently, 24/149 (16%) were partially completed and excluded from analysis due to missing data. Of the 125 (45 nurses, 57 physicians, 23 other (e.g. pharmacist)) participants, many (84/125, 67%) had previous experience of conducting clinical trials (range 1–300 months, mean 54 months experience), whilst 24 (19%) had experience of RWPC in trials.

As shown in Fig. 1, LR telephone interviewed principal investigators (PIs) (n = 4) or lead research nurses (n = 6) and conducted six focus groups at six hospitals with 36 practitioners (20 nurses and 16 physicians, including three PIs). Focus groups took on average 60 min and interviews 40 min. Only the researcher and participants were present. Telephone interviews took place 8–18 months (mean = 11 months and 10 days, range 260–577 days) post SIV training and 5–16 months (mean 8 months and 21 days, range 168–490 days) after site opening. All (10/10, 100%) telephone interview participants had attended the SIV training. None of the PIs or research nurses had prior experience of RWPC in paediatric trials; one physician had comparable experience in an adult trial. Focus groups were held 13–18 months post SIV (mean = 14 months and 29 days, range 400–574 days) and mean 12 months and 9 days (range 329–420) after site opening. Half (18/36, 50%) of focus group participants had attended SIV training. A total of 199 practitioners from all 28 recruiting hospitals completed the online questionnaire 8 months before the end of the trial. Of these, 124/199, (62%) had attended SIV training. The other practitioners had attended subsequent EcLiPSE training sessions facilitated by their site.

Fig. 1

Study methods and participant characteristics. Figures are n (%). Abbreviation: SIV site initiation visit

Results

Practitioner concerns about recruitment and consent in EcLiPSE before site training

Thirty-three of 118 practitioners (28%) who completed a questionnaire cited concerns about recruiting patients to EcLiPSE; a slightly higher proportion (48/120, 40%) were concerned about seeking consent (RWPC) (see Table 2). Previous experience of RWPC was not associated with concerns about recruitment or seeking consent in EcLiPSE.

Table 2 Concerns about recruitment and consent in EcLiPSE by practitioner experience of research without prior consent (RWPC) (n = 125)

To explore this further, the questionnaire included four additional statements to specifically assess practitioners’ confidence in aspects of recruitment and RWPC in EcLiPSE before site training. The results grouped by experience of RWPC for statements 1 to 4 are presented in Table 3.

Table 3 Practitioner confidence in aspects of consent seeking by experience of research without prior consent (RWPC) before and after training (n = 125)

Just under half (57/125, 46%) responded positively to statement 1, indicating confidence in explaining the study to families. The majority (90/125, 72%) indicated confidence in explaining randomisation to families (statement 2). Previous experience of RWPC did not significantly improve levels of confidence in explaining either the study (p = 0.88), or the process of randomisation to families (p = 0.49).

Of the four statements, practitioners indicated they were least confident in explaining RWPC to families (statement 3). Thirty-three percent (41/125) responded positively. Although only two practitioners with previous experience of RWPC indicated a lack of confidence in the method, compared to 41% (39/69) of practitioners without experience of RWPC, there appeared to be some uncertainly about RWPC even amongst those with prior experience. Just under half of those with experience of RWPC (10/24, 42%) responded in the ‘neither agree not disagree’ option to statement 3.

In response to statement 4, just under half (60/125, 48%) of practitioners responded positively, indicating that they felt confident in dealing with parents who might object to their child being randomised. Prior experience of RWPC was also not associated with increased confidence in dealing with parents’ objections at the point of randomisation (p = 0.288). Themes identified in the analysis of interview and focus group data as well as qualitative free-text responses supported quantitative questionnaire findings. As shown in Table 4, many practitioners described how their concern arose from inexperience and lack of knowledge about RWPC. Some were ‘quite concerned how parents would take that (RWPC)’ (P1, focus group 4, female, nurse and about ‘How to approach it with the parents’ (P3, interview, female, nurse) after their child had been entered into EcLiPSE.

Table 4 Main themes related to practitioner concerns about recruitment, consent and the practicalities or logistics of conducting EcLiPSE before and after training identified in qualitative questionnaire free-text responses (n = 125), interviews (n = 10) and 6 focus groups (n = 36)

Logistical concerns about EcLiPSE before training

Over half (69/115, 60%) of practitioners anticipated that there would be practical or logistical difficulties in conducting EcLiPSE.

For many, EcLiPSE was their first ED-led paediatric clinical trial, which appeared to underpin many of the issues described by practitioners in free-text questionnaire responses (see Table 4). Practitioners were concerned about having adequate research support to conduct consent discussions with families, particularly at weekends, as well as the challenge of training all relevant staff across departments. Others discussed their concerns about following a trial protocol in an emergency resuscitation situation, whilst ensuring the clinical care of critically ill children was not compromised. As the number of eligible patients per site was expected to be low, practitioners referred to the anticipated challenge of maintaining trial awareness to ensure eligible patients were not missed. Principal investigators were most concerned about the importance of engaging and motivating all staff in EcLiPSE to maximise trial success.

Improved confidence in recruitment and consent after training

As shown in Table 3, improved levels of confidence were observed for all four statements regardless of whether practitioners had prior experience of RWPC. It was notable that following training, 82 (66%, 95% confidence intervals (CIs) 3.28 to 4.52) of practitioners felt their confidence in explaining the study to families was improved, whilst 90 (72%, CI 2.98 to 4.39) felt more confident in explaining RWPC to families. Approximately half of practitioners also indicated their confidence in explaining randomisation (47%, CI 3.86 to 4.63) and addressing parents’ objections to randomisation (51%, CI 3.37 to 4.25) had improved.

Questionnaire Part B (after training) free-text responses, as well as interview and focus group discussions also indicated that EcLiPSE training had addressed many of the practitioners’ concerns about recruitment and RWPC. After training, many described how the trial and its approach to consent seemed more ‘feasible’ (P128, SIV questionnaire, female, physician), ‘logical and straightforward’ (P30, SIV questionnaire, female, nurse). However, as shown in Table 4, themes identified in free-text questionnaire comments indicated that some practitioners remained anxious that parents would respond negatively to the RWPC discussion. Some practitioners involved in the follow-up interviews and focus groups reflected upon their post-training anxiety about RWPC before their site had opened to EcLiPSE recruitment. They described how despite finding the training videos useful, practitioners may need first-hand experience of RWPC to fully address anxieties about how parents will react to RWPC. Indeed they all reflected on how their EcLiPSE recruitment experience had addressed such anxieties as ‘It’s all been a really positive response (to RWPC). Like I say, we haven’t had one refusal yet’ (P8, interview, female, nurse). Some described how, when disseminating EcLiPSE training, they had highlighted the involvement of patients in the trial design and training package to help address concerns about the consent process:

‘It’s quite powerful to be able to say that parents have been involved in the process of approving deferred consent’ (P7, interview, male, nurse).

Valued aspects of the SIV

As shown in Table 5, practitioners rated highly the content and clarity of all sections of the training.

Table 5 Content and clarity of training (n = 125)

Practitioners were asked to rate two statements on a scale of 1 (strongly disagree) to 5 (strongly agree) to indicate whether the training videos had improved their confidence in (1) identifying eligible children and (2) seeking consent; they received mean ratings of 4.28 and 4.31, respectively.

In free-text responses, interviews and focus groups, many described how videos were useful to help visualise processes, including screening, randomisation and seeing a RWPC discussion ‘in action rather than theory’ (P21, SIV questionnaire, male, physician). Research nurses in particular valued the examples of tailored communication such as the ‘terminology used to explain this to families’ (P109, SIV questionnaire, female, nurse), and ‘there are some good sort of one-line quotes that you can take from it’ (P4, interview, female, nurse), as well as ‘see how nurse handled difficult questions’ from parents (P27, SIV questionnaire, female, nurse). Practitioners responsible for site training described how the videos and training slides provided had assisted learning and dissemination: ‘actually what really helped me is then when I was having to give the training to everybody else, I already had the package and I already had the PowerPoint, so that in actual fact helped me reinforce my own learning’ (P3, interview, female, physician).

During interviews, practitioners spoke of how the involvement of a number of the EcLiPSE team members, including the chief investigator, in delivering the SIV training had helped to create a sense of study importance, which appeared to help engage practitioners:

‘I think people were very impressed that the chief investigator had arrived… they saw that as a really, really good sign that people were taking this very seriously’(P1, interview 1, male, physician).

Logistical concerns after training and support

Although there were fewer logistical concerns described after training (Table 4), some practitioners re-stated concerns about staff availability to cover consent discussions with families 7 days a week. Questionnaire, interview and focus group participants described the anticipated challenge of successfully disseminating training to relevant staff including new physicians at the trainee rotational changeover, particularly over the busy winter period.

Practitioners suggested that the trial management team could provide recruitment and consent support through study updates, advice when required and recruitment-training tips from the ongoing embedded Consent study. This support was provided through regular contact and newsletter updates, which included recruitment tips in addition access to training materials online and on a USB provided to each site PI. Practitioners described how trial team support and access to training materials help facilitate the dissemination of training to new staff in the busy ED staff;

‘There is a study website and there’s an investigator-only section that you can log onto, and then there’s training resources on there, including the videos….when they’ve got 5 minutes, just to sit and watch one of those videos’ (P1, focus group 1, male, physician).

Staff views on recruitment, training and trial conduct prior to the final stages of the trial

In the online questionnaire 8 months before the end of the trial, practitioners involved in any element of EcLiPSE were asked to select the statements (see Table 6) which they felt were relevant to their site. The majority indicated that the trial was running well, which supported trial recruitment data (recruiting to target with a 95% consent rate). Only two practitioners (2/199, 1%) reported that anxieties about RWPC were a barrier to recruitment. The majority indicated that their site held regular (e.g. monthly) EcLiPSE training sessions (134/199, 67%) for new staff or as refresher training. Practitioners did not feel that they would benefit from additional training (132/199, 66%). The sample included 75 (38%) practitioners who had not attended the initial SIV training. Some reported that staff shortages had led to patients being missed, whilst some (6%) indicated that training was not frequent enough.

Table 6 Staff views on trial conduct and training dissemination 8 months before anticipated trial end date (n = 199)

Limitations

The study has some limitations. Although the majority of sites (87%) took part in the evaluation, only 47% of eligible practitioners anticipated staying for the full consent training and completed the questionnaires. Of these, 16% were excluded from analysis due to incomplete questionnaires. This attrition was because practitioners had to leave training early due to clinical commitments. This limitation reflects the challenge of delivering SIV training alongside ED clinical care commitments and highlights the importance of an effective, ongoing training dissemination strategy to ensure that all staff are able to access full training after a SIV. We were unable to re-administer the SIV questionnaire to the same participants at a later time point due to staff turnover. However, our study was strengthened by the conduct of interviews, focus groups and an online questionnaire throughout the trial with staff who had and had not attended the SIV. This mixed-method approach provided insight into multiple perspectives to assist understanding of the longer term uptake and impact of training upon practitioner confidence in recruitment and RWPC [16, 25] and potential barriers to trial success. Findings from the Consent study were used to inform support and feedback from the trial team to sites as part of an iterative approach. Finally, focus groups were conducted up to 18 months after SIV training due to the delays at sites that had been slow to open or recruit patients. This may have impacted upon practitioner recall about how they felt before SIV training and their views may have been influenced by trial recruitment experience.

Discussion

Our findings demonstrate how a 4-h interactive site training meeting can significantly alleviate practitioner concerns about recruitment and consent in a challenging trial in paediatric emergency medicine. Successful patient recruitment to EcLiPSE is dependent upon practitioners in many different departments, specifically the ED, general paediatrics, paediatric neurology and paediatric intensive care, being aware of, and comfortable with, the trial protocol. For many, this was the first time that teams would be working together to ensure that critically ill children were screened and randomised, whilst ensuring that families were appropriately approached to discuss the trial after the time-critical emergency situation had passed. Consequently, it was not surprising that before site training many practitioners had concerns about conducting EcLiPSE and lacked confidence in how to communicate some elements of the trial, including RWPC, to families.

Practitioners rated highly the clarity and content of SIV training. Significant improvements were observed in practitioners’ confidence in explaining the study, and randomisation, and RWPC to families, as well as how to respond to parents who might object to their child being randomised during an emergency resuscitation. A previous survey published by members of our group [10] showed how practitioners with previous experience of RWPC in a medical device trial (the CATCH trial) [26] had more positive perceptions of this method when compared to those without such experience. However, our questionnaire data show that before training, previous experience of RWPC was not associated with significantly reduced concerns or greater confidence in recruitment and consent in EcLiPSE. This may reflect the challenging nature of the trial, in that it was a CTIMP, involving a change in usual ED clinical practice and, for many, their first ED-led clinical trial.

Site training provides an opportunity to discuss and learn about the potential challenges and solutions to trial recruitment and conduct [27, 28]. Our findings suggest that the use of training videos complemented this process and helped practitioners to visualise potentially difficult trial processes, including screening in a resuscitation situation. Practitioners particularly valued the RWPC video, which had been informed by data on parents’ views and priorities for trial information from pre-trial research [11] and CONNECT study guidance on RWPC [7, 13]. The video provided practitioners with examples of how to communicate RWPC to parents [7, 13], as well as preparing them for the sorts of questions that parents might ask about the trial [11].

The involvement and commitment of the whole team is required in educational activities to facilitate successful trial recruitment [27]. Nevertheless, logistical concerns about having adequate numbers of staff to support the trial across departments could not be fully addressed through training. These concerns are currently relevant to all research conducted within the UK National Health Service; it is particularly pertinent to research led by EDs working under increasing pressures. Importantly, despite concerns over challenges in trial delivery, at the time of writing, EcLiPSE had completed recruited within the expected timeframe and achieving the recruitment target. Consent was provided for 385 of 404 (95.3%) randomised participants.

Although practitioner confidence in recruitment and consent and trial success are clearly important outcomes, arguably it is also important to establish whether EcLiPSE training improved the quality of consent discussions, and parental experiences of recruitment and consent in EcLiPSE. These questions will be explored as part of the ongoing Consent study [6].

Conclusion

Interactive SIV training can improve practitioners’ confidence in conducting research in a time-critical paediatric trial, which involves randomisation of children without prior informed consent. Our findings highlight the value of using parents’ perspectives to inform training and to engage practitioners in trials that are at high risk of being too challenging to conduct.