- Open Access
Implementation of a batched stepped wedge trial evaluating a quality improvement intervention for surgical teams to reduce anastomotic leak after right colectomy
Trials volume 24, Article number: 329 (2023)
Large-scale quality improvement interventions demand robust trial designs with flexibility for delivery in different contexts, particularly during a pandemic. We describe innovative features of a batched stepped wedge trial, ESCP sAfe Anastomosis proGramme in CoLorectal SurgEry (EAGLE), intended to reduce anastomotic leak following right colectomy, and reflect on lessons learned about the implementation of quality improvement programmes on an international scale.
Surgical units were recruited and randomised in batches to receive a hospital-level education intervention designed to reduce anastomotic leak, either before, during, or following data collection. All consecutive patients undergoing right colectomy were included. Online learning, patient risk stratification and an in-theatre checklist constituted the intervention. The study was powered to detect an absolute risk reduction of anastomotic leak from 8.1 to 5.6%. Statistical efficiency was optimised using an incomplete stepped wedge trial design and study batches analysed separately then meta-analysed to calculate the intervention effect. An established collaborative group helped nurture strong working relationships between units/countries and a prospectively designed process evaluation will enable evaluation of both the intervention and its implementation.
The batched trial design allowed sequential entry of clusters, targeted research training and proved to be robust to pandemic interruptions. Staggered start times in the incomplete stepped wedge design with long lead-in times can reduce motivation and engagement and require careful administration.
EAGLE’s robust but flexible study design allowed completion of the study across globally distributed geographical locations in spite of the pandemic. The primary outcome analysed in conjunction with the process evaluation will ensure a rich understanding of the intervention and the effects of the study design.
National Institute of Health Research Clinical Research Network portfolio IRAS ID: 272,250. Health Research Authority approval 18 October 2019. ClinicalTrials.gov, identifier NCT04270721, protocol ID RG_19196.
Improvement of surgical safety is a global healthcare priority with significant potential to reduce morbidity and mortality. The most important complication of colorectal surgery is anastomotic leak [1,2,3]. This can occur when a joint made between two ends of the bowel, e.g. following the removal of part of the bowel, fails to seal properly resulting in leakage of bowel contents into the peritoneal cavity, infection, sepsis or even death. The ESCP sAfe Anastomosis proGramme in CoLorectal SurgEry (EAGLE) study is an international, pragmatic, cluster randomised trial of a quality improvement intervention designed to reduce anastomotic leak following a common operation called right colectomy (removal of the right side of the large bowel, to treat bowel cancer or benign disease, and connect small bowel to remaining colon), by implementation of an education programme .
Previous research has shown that specific risk factors that may be identified pre- and intra-operatively increase the risk of anastomotic leak after right colectomy, and variation in surgical practice can harm patient outcomes [1, 5]. The EAGLE intervention promotes pre- and intra-operative operative risk calculation and harmonisation of surgical practice.
Large-scale quality improvement interventions generate a multitude of challenges that demand a robust study design and the ability for the design to adapt for delivery in different environments and social contexts. The EAGLE trial adopted an innovative batched stepped wedge design aimed at achieving good uptake and statistical efficiency. In this paper, we expand on the features of this design and reflect on lessons learned about the implementation of randomised evaluations of quality improvement programmes on an international scale.
Summary of the EAGLE trial
Any surgical units that routinely perform both elective and emergency right colectomy in adult patients were eligible to enrol on the study. A surgical consultant, surgical trainee, an anaesthetist and a nurse principle investigator were required as a minimum in each local team (cluster). The primary objective was to reduce 30-day anastomotic leak rate following right colectomy. Clusters were randomised in a series of batches or phases approximately every 2 months, provided at least 18 clusters were ready (batches varied in size). In each batch, clusters were randomised 1:1:1 to three different sequences, with data collection in two distinct 2-month calendar periods (Fig. 1). The sequences differed in terms of the timing of implementation of the intervention, and in the periods when data were collected. In the terminology of cluster randomised trial design, each batch consisted of an incomplete, three-sequence stepped wedge design , and the trial as a whole was an example of a batched (incomplete) stepped wedge design . We expand further on these concepts below. The plan for analysis, briefly, was to analyse each batch separately and pool (meta-analyse) the estimates of intervention effect (further details of the analysis plan are provided in the published protocol) .
The hospital-level intervention required all surgeons to complete online education modules and then with their surgical teams to implement the ‘ESCP Safe Anastomosis Intervention’. The intervention was comprised of three components: pre-operative risk stratification, harmonisation of surgical technique and an intra-operative ‘ESCP Safe Anastomosis checklist’. Data collection involved identification and enrolment of all consecutive patients ≥ 18 years undergoing right colectomy with or without anastomosis in the defined data collection period, and follow-up until 30 postoperative days. As this was a hospital-level educational intervention, patient-level consent was not necessary in most jurisdictions. EAGLE was prospectively registered, IRAS ID:272,250; ClinicalTrials.gov, identifier NCT04270721 and the full protocol is published elsewhere .
Choosing elements for quality improvement
EAGLE was tailored specifically to tackle differences identified in patient outcomes and improve overall anastomotic leak rate. Previous research including European Society of Coloproctology (ESCP) prospective audits demonstrated patient characteristics and intra-operative factors that increased the likelihood of anastomotic leak as well as differences in patient outcomes depending on whether they were operated by a general or colorectal surgeon [1, 5, 8]. Directly addressing the lessons learned in existing studies, the EAGLE Protocol Working Group created the EAGLE programme to ‘level up’ care by quality improvement. The EAGLE intervention adopts first, an objective patient risk calculator  to identify patients at increased risk and enable surgeons to reconsider anastomosis and/or a defunctioning ileostomy (to divert bowel contents away from the join and into a bag on the abdominal wall, to prevent internal contamination while an anastomosis is healing). Second, the training modules ensure the best evidence is presented to all surgeons; general or colorectal, to harmonise operative practice and reduce differences in technique. Third, the intra-operative checklist enables factors, identified by the anaesthetist, theatre staff or surgeons, that arise during the operation to be incorporated in final decision-making for the anastomosis.
The EAGLE study deliberately selected routinely collected data that would be simple to locate for any colorectal patient in any hospital regardless of geography or socio-economic situation. This promotes a high level of data completeness and reduces missing data. Full details of data collection and the case report form can be viewed via the published protocol .
The ESCP’s programme of snapshot audits (multicentre prospective audits to generate large datasets about colorectal surgery and its outcomes) [1, 10, 11], created an established international family of surgical researchers familiar with obtaining local governance approvals, collecting and uploading data, and working remotely with a central team. This network helped promote and extend the study as the group had already delivered results and publications from previous work and retained the collaboration of many reputable establishments. In addition, the ESCP network allowed EAGLE to access a patient population, in which rates of anastomotic leak are established and would be comparable; notably, the 2015 snapshot audit reported an 8.1% anastomotic leak rate following right colectomy in > 3000 patients .
Batched stepped wedge design
Large clinical trials are notoriously complex to deliver and can have prolonged timelines. Surgical researchers are increasingly interested in methodological developments in randomised trials . EAGLE planned to recruit 333 clusters (surgical units) and needed an approach that would allow clusters to on-board the trial as soon as they were ready, to avoid delays during which enthusiasm could start to wane. Stepped wedge trials typically involve all clusters commencing at the same time, though the possibility of separating clusters in a stepped wedge trial into distinct batches has recently been proposed and formalised to mitigate the problem of delays . Each batch in a batched stepped wedge design can commence at different times, and effects of calendar time are adjusted separately within each batch, but with all batches contributing to the estimation of the intervention effect. In practical terms this allows investigators to focus recruitment of clusters in any given batch, taking optimal advantage of clusters’ availability to participate, while still accruing the overall statistical power needed to answer the research question.
The EAGLE protocol suggested indicatively that 7 batches of around 48 clusters each might be run, but the design and approach to analysis (meta-analysis across batches) gave us the flexibility to have batches of varying sizes. The make-up of a batch was determined largely by the order in which clusters enrolled, but again the design and meta-analytic approach gave us the flexibility, if we wanted it, to focus the recruitment of clusters to different batches in different global regions, for example. The sample size target (see below) was based on the total number of clusters and total number of surgical procedures across all batches.
An incomplete stepped wedge design was chosen in order to improve statistical efficiency by reducing both the number of clusters and the burden of individual-level data collection over all clusters necessary to achieve the required power . The particular incomplete design adopted for each batch of the EAGLE trial was a “dog-leg” design with three randomised sequences and two periods of data collection  as demonstrated in Fig. 1. In the first sequence the intervention is implemented before any data collection takes place, and data are collected in the first period only. In the second sequence data are collected in both periods, and the intervention is implemented between these two periods. In the third sequence the intervention is not implemented until after data collection, and data are collected in the second period only. All clusters receive the intervention within 28 weeks.
The EAGLE trial was designed to detect what was judged to be a clinically important, absolute reduction in anastomotic leak rate from 8.1 to 5.6% (relative risk reduction 30%), based on the leak rate in previous research as above . In our sample size calculation we assumed an intra-cluster correlation coefficient of 0.05 and a mean identification rate of 10 procedures per 8-week data collection period, and on this basis calculated an overall “design effect” of 0.9 for our cluster-randomised dog-leg design relative to an individually-randomised, single-period, parallel-groups design . This design effect is an inflation factor (or in this case a deflation factor) that can be applied to the number of clusters required to achieve the specified power . The design effect is less than 1.0 in this case because of the efficiency offered by having both pre- and post-intervention data collection from the same clusters in the second sequence. Thus we determined that we needed 333 clusters with data on a total of 4440 surgical procedures in order to achieve 80% power at the 5% significance level. This calculation included a correction to allow for variable cluster size (for further details of the sample size calculation, see the published protocol) .
We also calculated that if we had used a more conventional, parallel groups cluster randomised design, with data collection in both periods in all clusters, we would have needed 358 clusters with data on 7,160 surgical procedures in total, thus demonstrating the superior efficiency of the dog-leg design.
Flexibility for clusters and participants
The EAGLE protocol and resource packs offered a flexible approach to data collection to respect differences in local practice, with hard copy or electronic case report forms and the option for contemporaneous or retrospective database data entry. Password-protected e-modules were available throughout a four-week intervention period and thereafter to enable participants to access them at their convenience rather than fixed, timed lectures.
Every surgeon consultant or trainee who could be the primary operator for a right colectomy in emergency or planned cases was encouraged to complete the e-modules in every collaborating hospital. Access was enabled for the start of the EAGLE intervention period at each cluster. Module completion generated Continuing Professional Development points, accredited by the Royal College of Surgeons of England, free of charge. Even clusters (hospitals) randomised to submit only pre-intervention patient data later received the intervention, such that no clusters completed the study without the opportunity for all surgeons to access the evidence-based resources. Module access continues beyond the timeframe of the study for users to refer back.
EAGLE’s global invitation introduced some challenges for the Central Operations Committee to ensure relevant resources were accessible to all participants. Theatre team presentations and the ESCP Safe Anastomosis Checklist were translated to thirteen languages (independently and checked by national coordinators) and hosted at a single hub. A working level of English was assumed however for the surgical team, with the e-modules for surgeons only available in English.
Case ascertainment and data accuracy
Data validation was addressed with two main strategies. Stepped wedge cluster randomised trials demand a particularly rigorous approach to case ascertainment . All participating EAGLE clusters were contacted after batch completion to report the number of sequential eligible patients in a randomly-selected 2-week sample of their data collection periods. Figures were compared with the cases reported electronically and any inconsistencies were investigated with the whole period being re-checked, any missed cases added or erroneous cases removed. Additionally, data accuracy was assessed at three clusters per randomisation sequence (approximately 30% clusters in each batch), selected using a random number generator. Each cluster nominated someone to complete the data accuracy exercise, resubmitting ten key data points for up to the first ten consecutive patients in a single data collection period.
Trial steering committee and data monitoring committee
EAGLE constituted a Trial Steering Committee (TSC) and Data Monitoring Committee (DMC) to provide ongoing oversight and independent decision-making in response to challenges. The DMC had access to unblinded, interim data summaries from wholly and partially completed batches of the trial, and their remit included monitoring the assumptions of the original sample size calculation. They also had the power to recommend to the TSC that the entire trial be stopped early, or that a single ongoing batch be terminated. The TSC acted in the usual role of “critical friend” , had access to blinded, interim summaries of data, and advised on when the trial should be considered to be complete.
In the following sections, we reflect further on the evolution of the trial processes as the trial unfolded.
Effects of and resilience to COVID-19
EAGLE recruited and randomised the first batch of hospitals in December 2019, as ‘phase 1’, and the second in February 2020, ‘phase 2’, prior to the start of the COVID-19 pandemic. The pandemic rapidly shut down global surgical services and a decision was made to suspend the EAGLE study, just five weeks after launch, on 13th March 2020. As pandemic waves reached different countries at different times, some clusters continued to progress local approvals and as the first wave retreated, clusters were asked their readiness and capacity to restart the EAGLE programme. ‘Phase 1’ resumed on 27th July 2020 with a mix of hospitals derived from phases 1 and 2 that had indicated readiness. Clusters who could not yet participate joined later batches.
Clusters in ‘phase 1’ that had implemented the intervention but then paused due to COVID-19 were offered a 2-week ‘refresher’ intervention period for teams to review e-modules and intervention components before embarking on post-intervention data collection. Several clusters reported structural or organisational changes related to the pandemic and EAGLE adapted to ensure inclusivity despite these circumstances. One hospital that had previously performed both elective and emergency colorectal surgery was re-structured with its sister hospital to separate emergency operating at one site and elective operating at the other, but with a single, cross-site surgical team. On resumption of the study, local approvals were amended to include both sites as a single cluster. Conversely, in another site where there were two entirely separate surgical teams, each team was randomised as a separate cluster.
Although the interruption to the first batches impacted on the conduct of the trial and thus potentially the interpretation of the data in these batches, this impact was contained and did not compromise later batches, illustrating a more general robustness in the batched approach to design. When the trial data are analysed it may be necessary to perform a sensitivity analysis excluding phases 1 and 2.
The EAGLE team comprised separate Protocol Writing and Education Committee groups to drive forward the study design and the educational resources independently. The Operations Committee was convened to deliver the study with ‘Meta-coordinators’ liaising between the central team and ‘Coordinators’ who were responsible for day-to-day liaison with collaborators from individual clusters, loosely divided into geographical groups of hospitals or by language (Fig. 2).
EAGLE Coordinators worked with local Principal Investigators (PIs) to ensure that relevant local approvals were realised. Any regional or national ethical approvals were shared with other members of the same jurisdiction to minimise duplication of work. Resources and approvals were disseminated by close communication between PIs, coordinators and meta-coordinators.
EAGLE has proved a highly effective study design to deliver a complex quality improvement intervention in a wide range of settings across multiple study batches. Translation of resources and batching of clusters has enabled the study to navigate geographic, language and varied pandemic landscapes. The streamlined outcomes and design with incomplete cross sections offered high data completeness with low investigator burden. The trial was powered to efficiently detect reduction of a significant, measurable outcome. Finally, regardless of whether the primary outcome is achieved, the trial delivered a framework for education and operating team communication to hospital teams in all study sequences.
Implementing the design in batches had a number of practical benefits [7, 17], including enabling sequential participation of different clusters through the course of the trial. The use of an incomplete rather than a complete design in each batch had a theoretical advantage in terms of the quantity of data required for the trial, but there were also disadvantages.
The staggered start dates for data collection and introduction of the intervention meant that collaborators, e.g. Surgical Trainee Principal Investigators could miss participating in the study itself despite garnering the appropriate approvals, because the lead-in time after randomisation could be up to 20 weeks. This affected trainees or other collaborators moving between hospitals/departments, particularly in sequence 3. The risks with such a long lead in time include reduced motivation and engagement by the time study activities begin; however, this may also be seen in the context of studies that recruit many clusters to start at the same time, as those recruited first may have lost motivation by the time enough teams are prepared to start. Where long delays mean collaborators have changed departments or hospitals, the local team also needs to recruit new core team members who may not be as familiar with the study.
A further risk associated with long lead-in times before pre-intervention data collection is the risk that participants will, in the meantime, access educational materials from elsewhere. This will, at worst, attenuate our estimate of the effect of the intervention.
EAGLE sequence 3 collaborators must also rely on internal rather than external motivation to complete e-learning, (since there was no post-intervention data collection for this group therefore core EAGLE team members would have less reason to urge surgeon colleagues to complete the modules). While this would not affect the study’s primary objective (as no post-intervention data collection planned), it would reduce the dissemination of evidence-based education content which is designed to be delivered at every EAGLE participating cluster.
The e-modules contained extensive evidence-based content for surgeons that were presented in English. Most surgeons in the ESCP are accustomed to working in English however the resource was offered to all surgeons at participating hospitals and this monolingual resource may have excluded selected surgeon users. In some countries (South Korea, Romania), surgeons formed working groups to undertake the e-modules together. A lead surgeon could translate and interpret the modules, and cross the language barrier for colleagues with more limited English.
One further drawback is that this study did not examine the longer-term effects of the intervention. Individual surgeons or surgical teams could slip back into older less evidence-based practice, team members will inevitably change, and without deliberate maintenance of the practice change, any beneficial intervention effects may be lost. There is a strong case for a follow-up cohort study to assess longer-term adherence and patient outcomes, should the study show a patient benefit.
To ensure EAGLE’s results can be understood, a prospectively designed process evaluation is being undertaken. The parallel process evaluation will demonstrate how the education resources have reached learners (engagement), to what extent patients have received, or been exposed to, the EAGLE Safe Anastomosis Programme in the post-intervention groups (implementation), and help identify factors that have promoted or distracted intervention implementation. This will allow the project’s outcomes to be examined, to delineate motivators and barriers for the intervention and to decipher any intervention failure from an implementation failure. This analysis can enhance a study’s overall results and avoid drawing false conclusions of failure (or success) in either aspect of study delivery.
EAGLE’s robust but flexible trial design allowed sequential recruitment and completion of the study across geographical locations in spite of the pandemic. EAGLE’s collaborative group originating in the ESCP network, helped nurture strong working relationships with resource sharing to reduce duplication. The primary outcome analysed in conjunction with the process evaluation will ensure a rich understanding of the intervention and the effects of the study design.
Availability of data and materials
Data sharing not applicable to this article as no datasets were generated or analysed during the current manuscript.
Data Monitoring Committee
European Society of Coloproctology
- EAGLE :
ESCP sAfe Anastomosis proGramme in CoLorectal SurgEry
- PIs :
Trial Steering Committee
European Society of Coloproctology collaborating group. Relationship between method of anastomosis and anastomotic failure after right hemicolectomy and ileo-caecal resection: an international snapshot audit. Color Dis. 2017;19(8):e296-311.
Ashburn JH, Stocchi L, Kiran RP, Dietz DW, Remzi FH. Consequences of anastomotic leak after restorative proctectomy for cancer: Effect on long-term function and quality of life. Dis Colon Rectum. 2013;56(3):275–80.
Gessler B, Eriksson O, Angenete E. Diagnosis, treatment, and consequences of anastomotic leakage in colorectal surgery. Int J Color Dis. 2017;32:549–56.
ESCP EAGLE Safe Anastomosis Collaborative†. ESCP Safe Anastomosis ProGramme in CoLorectal SurgEry (EAGLE): Study protocol for an international cluster randomised trial of a quality improvement intervention to reduce anastomotic leak following right colectomy. Color Dis. 2021;23(10):2761–71.
European Society of Coloproctology collaborating group. The impact of stapling technique and surgeon specialism on anastomotic failure after right-sided colorectal resection: an international multicentre, prospective audit. Color Dis. 2018;20(11):1028–40.
Hooper R. Key concepts in clinical epidemiology: Stepped wedge trials. J Clin Epidemiol. 2021;1(137):159–62.
Kasza J, Bowden R, Hooper R, Forbes AB. The batched stepped wedge design: A design robust to delays in cluster recruitment. Stat Med. 2022;
2017 and 2015 European Society of Coloproctology (ESCP) collaborating groups. The impact of conversion on the risk of major complication following laparoscopic colonic surgery: an international, multicentre prospective audit. Color Dis. 2018;20:69–89.
Sammour T, Lewis M, Thomas ML, Lawrence MJ, Hunter A, Moore JW. A simple web-based risk calculator (www.anastomoticleak.com) is superior to the surgeon’s estimate of anastomotic leak after colon cancer resection. Tech Coloproctol. 2017;21(1):35–41. Available from: Published online 19 December 2016
European Society of Coloproctology (ESCP). Cohort Studies & Audits. Online. 2022. Available from: https://www.escp.eu.com/research/cohort-studies
The 2017 European Society of Coloproctology (ESCP) Collaborating Group. The. European Society of Coloproctology (ESCP) international snapshot audit of left colon, sigmoid and rectal resections - Executive Summary. Colorectal Dis. 2017;2018(20):13–4.
Augustinus S, van Goor IWJM, Berkhof J, Daamen LA, Groot Koerkamp B, Mackay TM, et al. Alternative Randomized Trial Designs in Surgery: Systematic Review. Ann Surg. 2022 Jul 22;0. https://doi.org/10.1097/SLA.0000000000005620
Hooper R, Bourke L. Cluster randomised trials with repeated cross sections: Alternatives to parallel group designs. BMJ. 2015;350.
Hooper R, Teerenstra S, de Hoop E, Eldridge S. Sample size calculation for stepped wedge and other longitudinal cluster randomised trials. Stat Med. 2016;35(26):4718–28. Available from: https://www.ncbi.nlm.nih.gov/pubmed/27350420
Hemming K, Taljaard M, McKenzie JE, Hooper R, Copas A, Thompson JA, et al. Reporting of stepped wedge cluster randomised trials: extension of the CONSORT 2010 statement with explanation and elaboration. BMJ. 2018 Nov 9;363:k1614. https://doi.org/10.1136/bmj.k1614
Daykin A, Selman LE, Cramer H, McCann S, Shorter GW, Sydes MR, et al. What are the roles and valued attributes of a Trial Steering Committee? Ethnographic study of eight clinical trials facing challenges. Trials. 2016 Dec 1;17(1):307. https://doi.org/10.1186/s13063-016-1425-y
Hastings SN, Stechuchak KM, Choate A, Mahanna EP, Van Houtven C, Allen KD, et al. Implementation of a stepped wedge cluster randomized trial to evaluate a hospital mobility program. Trials. 2020;21(1):1–9.
James Keatley, ESCP Project Manager (European Society of Coloproctology).
This paper received no direct funding. EAGLE is an investigator-initiated and investigator-led study. The overall funding for study conduct has been provided by the European Society of Coloproctology (ESCP). Ethicon provided an unrestricted educational grant to the ESCP, used to support development of the online education material.
Ethics approval and consent to participate
The EAGLE study was approved in the UK as research without patient level consent by the Health Research Authority; IRAS project ID 272250; protocol number RG_19-196; REC reference 19/HRA/5656. Outside the UK ethical approvals were attained as mandated by each participating country.
Consent for publication
The authors declare that they have no competing interests and study sites were not paid for their contributions to the manuscript.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Venn, M.L., Knowles, C.H., Li, E. et al. Implementation of a batched stepped wedge trial evaluating a quality improvement intervention for surgical teams to reduce anastomotic leak after right colectomy. Trials 24, 329 (2023). https://doi.org/10.1186/s13063-023-07318-9
- Stepped wedge trial
- Batched stepped wedge
- Incomplete trial design
- Cluster randomised trial
- Group randomised trial
- Colorectal surgery
- Anastomotic leak prevention