Skip to main content
  • Study protocol
  • Open access
  • Published:

Assessment of implementation methods in sepsis: study protocol for a cluster-randomized hybrid type 2 trial



Sepsis is the leading cause of intensive care unit (ICU) admission and ICU death. In recognition of the burden of sepsis, the Surviving Sepsis Campaign (SSC) and the Institute for Healthcare Improvement developed sepsis “bundles” (goals to accomplish over a specific time period) to facilitate SSC guideline implementation in clinical practice. Using the SSC 3-h bundle as a base, the Centers for Medicare and Medicaid Services developed a 3-h sepsis bundle that has become the national standard for early management of sepsis. Emerging observational data, from an analysis conducted for the AIMS grant application, suggest there may be additional mortality benefit from even earlier implementation of the 3-h bundle, i.e., the 1-h bundle.


The primary aims of this randomized controlled trial are to: (1) examine the effect on clinical outcomes of Emergency Department initiation of the elements of the 3-h bundle within the traditional 3 h versus initiating within 1 h of sepsis recognition and (2) examine the extent to which a rigorous implementation strategy will improve implementation and compliance with both the 1-h bundle and the 3-h bundle. This study will be entirely conducted in the Emergency Department at 18 sites. A secondary aim is to identify clinical sepsis phenotypes and their impact on treatment outcomes.


This cluster-randomized trial, employing implementation science methodology, is timely and important to the field. The hybrid effectiveness-implementation design is likely to have an impact on clinical practice in sepsis management by providing a rigorous evaluation of the 1- and 3-h bundles.


NHLBI R01HL162954.

Trial registration NCT05491941. Registered on August 8, 2022.

Peer Review reports


Sepsis is the leading cause of admission to intensive care units (ICUs), the leading cause of death in ICUs, and the most common cause of hospital readmissions [1,2,3,4,5,6]. Sepsis is also the most expensive condition treated in the USA [7]. The Surviving Sepsis Campaign (SSC) was formed in 2002 to reduce sepsis mortality. SSC developed evidence-based guidelines in 2004, and updated guidelines in 2008, 2013, 2017, and 2021 [8,9,10,11]. Since simply publishing guidelines seldom leads to changes in clinical behavior [12], the SSC in partnership with the Institute for Healthcare Improvement [13] developed sepsis “bundles” to change clinical practice. A bundle aggregates evidence-based practices into a discrete management approach. Initial 6- and 24-h SSC bundles were revised to 3- and 6-h bundles, and subsequently to the 1-h bundle [14].

Since the advent of sepsis bundles, multiple observational studies have demonstrated an association between implementation of sepsis bundles and improved survival [15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35]. Adoption has been widespread [16, 36,37,38]. Higher bundle compliance is associated with lower mortality [39,40,41]. Based on these data and since sepsis is a “medical emergency” [7], all 3-h bundle elements were included in the 1-h bundle in recognition that earlier management might improve survival [14]. The 1-h bundle was named to differentiate the importance of initiating all the elements within that hour as opposed to completion. Although some hospitals have adopted the 1-h bundle, widespread adoption has not occurred.

In 2015 the Centers for Medicare and Medicaid Services (CMS) introduced “The Early Management Bundle: Severe Sepsis/Septic Shock” (SEP-1), which requires hospitals to report compliance with certain clinical processes related to sepsis care. SEP-1 focuses on timely recognition and early intervention. Despite the CMS reporting requirement, hospital compliance remains moderate at 60% [42].

Although bundle compliance would appear essential to improve sepsis outcomes, implementation science approaches to improve bundle compliance have not been evaluated. In particular, there is no widely accepted implementation strategy (i.e., a method to enhance adoption of identified practices) [43] to promote compliance with sepsis bundles. Application of rigorous implementation science techniques, identifying both barriers and facilitators, may improve bundle compliance [44].

This pragmatic, cluster-randomized hybrid type 2 effectiveness-implementation study will compare the effectiveness of the 1-h bundle to the 3-h bundle. The implementation strategy will be guided by the EPIS (Exploration, Preparation, Implementation, Sustainment) Framework [45], a commonly used approach that delineates phases of implementation and associated constructs. This study will explicitly test rigorous implementation strategies and incorporate formative, process, and summative evaluations throughout. In addition, sepsis phenotypes will be identified to probe how sepsis bundles may interact with a specific clinical presentation.


Study design

This study is a pragmatic, cluster-randomized trial of 18 hospitals (9 in each arm) comparing the 1- and 3-h bundles in patients presenting to emergency departments (ED) with sepsis. We will use a hybrid type 2 effectiveness-implementation design [46] to evaluate (1) effectiveness outcomes, including mortality and respiratory failure, and (2) implementation outcomes, including compliance (Fig. 1). The multi-faceted implementation strategy will apply to both trial arms. Application of the Pragmatic Explanatory Continuum Indicator Summary [47,48,49,50] (PRECIS) tool identifies this trial as pragmatic (vs. explanatory) [51]. A cluster-randomized trial is appropriate for analyzing the comparative effectiveness of the 1-h bundle versus the 3-h bundle because of the impossibility of randomizing patients to a particular approach and the importance of implementation of bundles at the hospital level. This protocol adheres to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidelines [52]. Figure 2 provides a summary of the enrollment schedule, interventions, and assessments using the SPIRIT flow diagram.

Fig. 1
figure 1

Study timeline

Fig. 2
figure 2

SPIRIT checklist: Schedule of enrollment, interventions and assessment

Study objectives

The study aims are as follows: (Aim 1) Examine comparative effectiveness of the 1-h versus 3-h sepsis bundle on the primary outcome of hospital mortality and secondary outcomes of length of stay, ventilator-free days at 28 days, and rate of acute respiratory failure requiring mechanical ventilation. Hypothesis: The 1-h bundle will have lower hospital mortality. (Aim 2): Conduct a mixed methods evaluation of a multi-faceted implementation strategy to enhance compliance with the 1 and the 3-h bundles. Hypothesis: 1 and 3-h bundle implementation compliance will be similar as assessed by (a) the electronic health record (EHR), (b) surveys of key informants regarding implementation climate, organizational readiness for change, and implementation leadership, (c) recordings of learning collaborative meetings, and (d) interviews with participants. (Aim 3): Determine if clinical sepsis phenotypes determined by characteristics measured at presentation modify the treatment effect of the 1 or 3-h bundle. Hypothesis: Routine clinical information available at ED presentation will identify phenotypes more likely to benefit from the 1 or 3-h bundle.

Study setting

We will conduct this study at 18 diverse acute care hospitals in the USA. Hospital selection was based upon responses of ED leaders to a baseline survey to ensure a diverse and representative sample of hospitals. Considerations included hospital size, teaching status, and known improvement strategies. Discovery, the Society of Critical Care Medicine’s (SCCM) critical care research network, will serve as clinical and data coordinating center for the study. Sites were recruited from Discovery and via polling emergency medicine members of SCCM as to suitability of their hospital as a site. Both academic and community hospitals are included to support the pragmatic nature of the study and ensure generalizability. To account for baseline implementation efforts, prior to randomization, sites that reported less than 30% or more than 70% compliance with SEP-1 were not included for study participation. Patient inclusion criteria include suspected severe sepsis and/or septic shock in the ED with 2 out of 4 Systematic Inflammatory Response Syndrome (SIRS) criteria, evidence of organ dysfunction, and suspected infection—consistent with the criteria established for SEP-1 by CMS. Time zero (the beginning of measuring compliance with the 3-h bundle) is defined in the same manner as described by CMS—all three criteria have to be met within 6 h in order for the definition of time zero to be met. Patients will be excluded if, within 24 h of admission, they are made do-not-resuscitate/do-not-intubate, comfort measures only, or considered not eligible to full aggressive care because of patient or family wishes. For consistency, only hospitals utilizing Epic as their EHR were included. In addition, the decision was made by the investigators that if patients do not present with hypotension, but develop hypotension after the criteria for time zero are met, then no fluids will be given. This does not mean that no fluids will be given to a patient, but rather that, for the purposes of the AIMS study compliance with fluids will not be measured.

Study committees

In addition to coordinating responsibilities overseen by the project manager (LH) at SCCM, the conduct of the study will be overseen by a steering committee and a Data and Safety Monitoring Board (DSMB). The DSMB is independent of the steering committee and study sponsor. It will meet annually to assess the safety and efficacy of study procedures, monitor the overall conduct of the study, and ensure that data is collected reliably. The steering committee will meet every other week to discuss trial management and will be comprised of the principal investigator (ML), the head of implementation science (HEF), the heads of the subcommittees (CS, DP, NS, and LE), a study consultant with experience from the New York State sepsis initiative (KMT), and the research associate (JG). The subcommittees include (1) the Data Subcommittee (chaired by CS): Charged with building the clinical report form, managing data entry, producing videos to describe data entry procedures, and reviewing missing data; (2) the Education Subcommittee (chaired by DP and NS): Charged with producing educational algorithms for posting in emergency departments and developing educational slide sets for distribution to all sites; and (3) the Publication Subcommittee (chaired by LE): Charged with developing policies for all study publications, including for authorship and manuscript proposals. All subcommittees will report to and receive input from the steering committee.


Hospitals will be randomized to one of two arms (1- or 3-h bundle) in a 1:1 ratio. Randomization will be accomplished using randomly permuted blocks of variable size and controlling for academic or community status. Hospital physician and nurse dyads will be told which condition they were randomized to during in-person meetings at SCCM immediately prior to the Implementation Phase; separate meetings will take place for each study arm. Each arm will undergo a 30-month structured, collaborative implementation approach to operationalize compliance with their bundle assignment. The elements of both bundles are identical—only the timing is different (see Table 1). In this cluster-randomized clinical trial, the hospital is the unit of randomization. Hospitals will be informed as to their randomized arm after the baseline collection period. The randomization will be done by the study statistician and held only in the statistician’s computer until the reveal meeting at the end of the baseline period. Due to the pragmatic, open-label design of this trial and its objective measurement of study outcomes, the study staff, investigators, sites, and biostatisticians will not be masked to treatment condition once randomization occurs. Patients will receive standard of care for sepsis and their data will be collected retrospectively by the hospital each month.

Table 1 1- and 3-h bundle elements: quality and outcome indicator definitions and specifications

Implementation strategies

The implementation package consists of six strategies (see Table 2) from the Expert Recommendations for Implementing Change (ERIC) [53] strategy taxonomy and carried out by the study team according to the Exploration, Preparation, Implementation, Sustainment (EPIS) [54] phase. The only difference in the implementation package for the two arms is training specific to implement the bundle assignment.

Table 2 Overview of implementation strategies by study phase

Exploration phase (6 months) (Fig. 1 )

Assess for readiness, organizational culture, and barriers and facilitators

Prior to implementation, sites will participate in formative evaluations [55], which will involve: First, surveys with ED leadership (physician and nurse dyads) to explore leadership and team attitudes and engagement, assess their needs related to sepsis bundle adoption, explore anticipated barriers to implementation, and identify potential facilitators of change. We will use several scales during the exploration phase, and then again at the conclusion of the Implementation phase: (a) Implementation Leadership Scale (ILS) [56]; (b) Evidence-Based Practices Attitudes Scale (EBPAS-15) [57]; (c) Organizational Readiness for Implementing Change (ORIC); and (d) Implementation Climate Scale (ICS) [58]. Secondly, semi-structured interviews with key ED informants will occur during the formative, process, and summative evaluation phases [59]. Leaders, physicians, nurses, and other hospital staff will be asked about their openness/resistance to change generally and to the use of sepsis bundles, beliefs about the evidence supporting sepsis bundles, perceptions of the utility of the bundles, fiscal obstacles (or facilitators), and beliefs about the importance of timely treatment.

Adoption-preparation (baseline data collection) phase (6 months)

Staff at each site will be trained to enter data into SCCM’s research electronic data capture (REDCap) [60, 61]. These data will serve as a baseline for evaluating the change in bundle compliance at the end of the 30-month implementation phase associated with the implementation strategy.

Identify and train champions

Sites will choose a nurse and physician as site champions (i.e., clinicians supportive of implementation [39, 62]). Aggregate results from exploration phase surveys will be shared with site investigators at an in-person meeting hosted at SCCM.

Create a learning collaborative

Each site will develop a change team consisting of an ED physician, ED nurse, hospital quality improvement advisor, and data collector. A monthly virtual meeting will be conducted by the study team for each arm with the site teams, including review of educational approaches and materials.

Implementation phase (30 months)

Conduct separate ongoing training virtual meetings for each arm

Following randomization, site teams will participate in virtual monthly learning collaborative meeting, focusing on content of the sepsis bundles, creating a team atmosphere and how to review and overcome barriers to implementation. Each learning session will include additional instruction on facilitating change, developing an information infrastructure to measure progress and quality improvement and implementation science theory. Sites in each arm will have access to a distinct on-line discussion forum to share ideas, questions, challenges, and solutions.

Develop and implement tools for quality monitoring (reporting tool)

Sites will (1) provide ongoing feedback and training of data collectors, (2) develop automated EHR alerts for identification of sepsis in the ED, (3) review order sets, and (4) assess physician and nurse engagement.

Audit, feedback and process evaluation of bundle implementation

Hospital sites in both arms will receive feedback on compliance every month [62, 63] including (1) review of successes and failures of implementation, (2) ongoing discussion of compliance barriers, and (3) methods for overcoming barriers. Learning collaborative meetings will be recorded for analysis in formative process evaluations.

Sustainment phase (12 months)

Outcome assessment and summative evaluation (6 months)

At the end of the 30-month implementation period, sites will continue to implement the 1 and 3-h bundles. Data collection and reporting will continue post-intervention to assess sustainability. Data collected during the 6-month outcome assessment phase will serve as the primary data set for analysis and for comparison to the 6-month period of baseline data. A mixed methods summative evaluation to measure change in barriers and facilitators identified in the exploration phase and bundle sustainment will be completed. We will administer the ILS [56], EBPAS-15 [57], ORIC, and ICS [58] measures, as well as the 40-item Program Sustainability Assessment Tool [64]. Semi-structured interviews will be repeated with key ED team members evaluating the EPIS framework and most effective implementation strategies.

Dissemination activities (6 months)

Following the data analysis, investigators will develop educational materials that summarize the findings of the trial as well as “how to guides” for generalizing the implementation strategy to other hospitals [65,66,67,68]. Investigators will disseminate results to national stakeholders (i.e., SCCM, Institute for Healthcare Improvement, professional societies, and hospital associations). Specifically, as indicated via letters of support, SCCM and the American College of Emergency Physicians (ACEP) will disseminate trial results through their communication channels, including discussion groups and social media accounts. In addition, trial findings will be published in peer-reviewed journals, presented at national and international conferences, and shared by study investigators via social media. Furthermore, de-identified data will be made available in a data repository in adherence with NIH’s Data Sharing policy.

Recruitment and power

The study sample size will (4070 in each arm) provide adequate power to detect a difference-in-difference in hospital mortality between the pre-intervention period and the post-intervention period. This estimate is based upon the number of patients who receive care compliant with the respective bundles and associated hospital mortality. Analyzing ED patients included in the New York Sepsis Database, pre-intervention hospital mortality was 19.1% for those compliant with the 3-h bundle. Analyzing the data to create cohorts compliant with therapies at 1 h and at 3 h, the post-intervention hospital mortality of those receiving care compliant with the 1-h bundle was 16.2% and hospital mortality for those receiving care compliant with the 3-h bundle was 18.9% (preliminary data submitted to NHLBI R01 HL153268). This is based on the assumption that in the post-intervention period, the median time to antibiotics will be reduced by a quarter of an hour (25% reduction) in the 1-h arm. This quarter of an hour reduction in time to antibiotics translates to a 0.2% reduction in hospital mortality. Table 3 summarizes the assumptions used in the power calculation.

Table 3 Assumptions for Donner and Klar’s cluster-randomized power calculation using a mixed model test for two proportions in a 2-level hierarchical design. An intraclass correlation coefficient of 0.025 is based on ED patients in the New York Sepsis Clinical Database

Using the power estimate methods of Donner and Klar, nine hospitals per arm with a minimum of 105 patients in each hospital will give 80% power to detect a 2.7% difference-in-difference in mortality, with α = 0.05, ICC = 0.025, and an equal number of hospitals per arm. To account for limitations in preliminary data, we increased the target number of patients by 50% to 158 patients. Since we expect bundle compliance in the 1-h arm to be 70%, 230 total subjects per hospital are needed. Table 4 shows the expected overall hospital mortality in the intent-to-treat analysis based on these assumptions. Multiple imputation will be used to handle missing data.

Table 4 Estimated pre- and post-intervention overall mortality used in the intent-to-treat analysis

Analysis plan

Aim 1: 1-h bundle will have significantly lower patient mortality and improve all secondary outcomes

Primary outcome analysis

An intention to treat analysis of patient mortality in each arm will be evaluated in 2 ways: (1) the difference, in primary outcomes (hospital mortality), between the two arms during the 6-month baseline assessment and the 6-month outcomes assessment, and (2) the comparison between the first two differences (i.e., difference-in-difference estimation). This difference-in-difference analysis will be the primary analysis to assess the primary hypothesis of Aim 1: the 1-h bundle will result in lower mortality than the 3-h bundle. Both absolute and risk-adjusted hospital mortality will be reported. Random-effects logistic regression will evaluate all the mortality differences. The random term will be the hospital. This model will contain three binary risk factors: (1) 1-h arm vs. 3-h arm, (2) 6-month baseline period prior to randomization vs. 6-month outcome assessment phase, and (3) the interaction of these two terms. The effect of the intervention and the effect of the two time periods will be quantified by odds ratios while the differences in hospital mortality will be quantified by model probabilities. Additionally, the model will be adjusted for hospital level 3-h bundle compliance prior to intervention along with patient and hospital characteristics.

Compliance definition

This study will collect the date and time associated with each of these four measures: (1) when serum lactate level was collected, (2) when blood culture was collected, (3) when antibiotics were started, and (4) when fluids (30 ml/kg crystalloids) were started. Compliance is defined as to whether or not these date and time stamps were within 1 or 3 h (inclusive) of time zero depending on hospital randomization.

Subgroup analyses

A second sensitivity analysis will follow the statistical plan for the primary intention to treat analysis but use triage time as the time of presentation (“time zero”), and a third sensitivity analysis will also follow the intention to treat analysis strategy but count discharge to hospice as a death. A fourth sensitivity analysis will examine only patients who received therapies after the time of presentation to more properly evaluate the impact of therapies initiated in the hospital (e.g., excluding patients assigned credit for fluids or antibiotics administered prior to arrival to the hospital). A fifth subgroup analysis will examine the impact of time to treatment on mortality in both study arms using time as a continuous variable.

Secondary outcomes

Four secondary outcomes will be investigated: respiratory failure requiring mechanical ventilation, the number of ventilator-free days, and hospital and ICU length of stay. The same statistical approach described in the primary analysis will be used here; however, the analysis for the count of ventilator-free days will be based on random-effects negative binomial regression or a zero inflated negative binomial regression, depending on the distribution of the count of ventilator-free days. Similar to hospital mortality, the requirement for mechanical ventilation will be based on random-effects logistic regression. Hospital length of stay analysis will be based on a random-effects linear regression. However, prior to the analysis, length of stay will be normalized using the natural logarithm since this variable is not normally distributed. After running the random-effects linear regression, the results will be back transformed to the original units producing the ratio of the geometric mean length of stay for patients in one arm compared to the other arm.

Aim 2: Quantitative analysis will demonstrate improved compliance in both arms through the implementation strategy and qualitative analyses will provide insights into implementation strategy effectiveness

Surveys assessing implementation-related constructs will be completed at baseline (Exploration phase) and then repeated during the Sustainment phase. We will conduct within-subjects Wilcoxon signed rank test with the survey data, examining changes over time.

Quantitative analysis of bundle compliance

Bundle compliance will be defined as all-or-nothing for the individual elements of the bundles. The statistical analysis will be similar to the primary analysis for hospital mortality.

Qualitative analyses of bundle compliance

We will supplement quantitative analyses with qualitative interviews with key informants (ED leaders, physicians, and nurses) during the Exploration and Sustainment Phases and qualitative process evaluation data from the Implementation Phase learning collaborative meetings. For the Exploration and Sustainment Phases, we will recruit participants until thematic saturation is reached [69], anticipating up to 5 participants per site during each Phase (N = 90). For the Implementation Phase, learning collaborative meetings will be recorded and coded as process evaluations by the study team. Learning collaborative meetings will be recorded but will not be transcribed; instead, detailed notes will be taken during interviews and collaborative meetings. Qualitative interviews will be coded using a rapid analytic approach described by Hamilton [70], in which a template is developed to summarize transcripts. The deductive domains for this summary template will be based on the interview guides with additional space for other observations and reflections. The qualitative team will also be able to add inductive domains to the template in response to the data collected [71]. The data entered into the summary template will be analyzed using matrix analysis [72].

Aim 3: Identify four discrete, previously validated, sepsis phenotypes, two of which may identify patients who are significantly more likely to benefit from the 1-h bundle (γ and ∂)

Recently, the Sepsis ENdotyping in Emergency CAre (SENECA) project derived and evaluated sepsis phenotypes using clinical data from presentation in the ED. Using a variety of unsupervised clustering methods, 4 sepsis phenotypes (α, ß, y, and ∂) were derived, using 29 clinical variables such as vital signs, laboratory values, and demographics [73]. These phenotypes were validated in 6 studies across > 60,000 patients and found to have differences in clinical features, host response biomarkers, and clinical outcomes. In the present study, routine clinical variables will be extracted through EHR data or manual entry within the first 24 h of sepsis onset. Sepsis phenotypes will be determined using multivariate modeling. To understand if treatment effectiveness of the 1-h bundle are differentiated by phenotype, we will use unadjusted random-effects logistic regression with the following risk factors: phenotype, intervention arm, and the interaction of these two variables.


This study represents a critical step in assessing sepsis bundle effectiveness and implementation strategies to support compliance. It will be the first rigorous cluster-randomized design used to evaluate sepsis bundles and will compare two different bundle timelines for accomplishing early metrics. In addition, a tailored implementation strategy to enhance the implementation of sepsis bundles represents a novel approach. Rigorous formative, process, and summative evaluations using a concurrent mixed methods approach will provide important information to sites to correct site-level implementation strategy problems. In addition to our pragmatic and hybrid effectiveness-implementation aims, we will also explore heterogeneity of treatment effects in recently described clinical phenotypes. These phenotypes may respond differently to the interventions, informing future studies of bundle effectiveness.

Design considerations

Several key design considerations for this study warrant discussion. First, we chose to compare the 1-h to the 3-h bundle. We could have proposed separate trials for 1- and 3-h bundles, comparing each to “usual care.” Because most United States hospitals now have sepsis protocols and are mandated to report compliance with the 3-h bundle [15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35], the 3-h bundle is a de facto proxy for usual care. In addition, because hospitals are mandated to report compliance with the 3-h bundle and because nationally compliance is moderate at 60% [42], the implementation strategy will be applied to both arms. We hypothesize that this strategy will increase compliance in both arms. Increased compliance should in turn assist sites in their efforts to comply with SEP-1.

Second, it is possible that hospitals in the 3-h arm may adopt features of the 1-h bundle, shortening the time to bundle completion and diluting the difference between the two study arms. We believe this is unlikely to be the case as the implementation strategy is designed to improve compliance with the 3-h bundle, not specifically to reduce time to treatment. In addition, it is unlikely that hospitals in the 3-h arm will develop strategies to approach the time requirements of the 1-h bundle. If 3-h hospitals do adopt implementation strategies that would target the 1-h bundle, Aim 2 analyses will allow assessment of this possibility.

Finally, by conducting the study among a diverse sample of hospitals from the Discovery Network and SCCM, we have considered issues related to generalizability. We have identified 18 hospitals and verified that none are outliers in terms of baseline compliance with the 3-h bundle. Conducting the study at hospitals of varying size and in both academic and community settings will enhance generalizability of the results.

Anticipated findings and impact

We hypothesize that the 1-h bundle will be more effective than the 3-h bundle for reducing hospital mortality, respiratory failure, increasing ventilator-free days, and hospital (and ICU) length of stay. We additionally anticipate that use of the implementation strategy will improve compliance with both bundles. If these hypotheses are correct, improvement efforts should shift to adopt the 1-h bundle employing a rigorous implementation strategy. If there are no differences between the bundles, efforts should return to enhancing compliance with the 3-h bundle. A negative finding would be particularly important, since some hospitals are already focusing on the 1-h bundle in the ED. In addition, identification of sepsis phenotypes using routinely available clinical data may identify a specific groups of patients that benefit from an appropriately timed bundle.

Trial status

Baseline data collection for the trial began December 1, 2022. Sites will be randomized to the 1- or 3-h bundle at the end of June 2023. The trial is funded through June 2027.

Availability of data and materials

The study’s Publication Subcommittee policy stipulates that investigators who contributed data to this study will have first priority for a period of 12 months after publication of primary manuscript for ancillary studies that require the analysis and use of data collected as part of the study. After that time, de-identified study data will be available in a data repository, consistent with NIH policy.



Centers for Medicare and Medicaid Services


Data and Safety Monitoring Board


Early Management Bundle: Severe Sepsis/Septic Shock


Electronic health record


Emergency departments


Evidence-Based Practices Attitudes Scale


Expert Recommendations for Implementing Change


Exploration, Preparation, Implementation, Sustainment Framework


Implementation Climate Scale


Implementation Leadership Scale


Intensive care unit


Organizational Readiness for Implementing Change


Pragmatic Explanatory Continuum Indicator Summary


Sepsis ENdotyping in Emergency CAre


Systematic Inflammatory Response Syndrome


Society of Critical Care Medicine


Standard Protocol Items: Recommendations for Interventional Trials


Surviving Sepsis Campaign


  1. Genga KR, Russell JA. Update of Sepsis in the Intensive Care Unit. J Innate Immun. 2017;9(5):441–55.

    PubMed  PubMed Central  Google Scholar 

  2. Gobatto AL, Besen BA, Azevedo LC. How Can We Estimate Sepsis Incidence and Mortality? Shock. 2017;47(1S Suppl 1):6–11.

    PubMed  Google Scholar 

  3. Lagu T, Rothberg MB, Shieh MS, Pekow PS, Steingrub JS, Lindenauer PK. What is the best method for estimating the burden of severe sepsis in the United States? J Crit Care. 2012;27(4):414e1-9.

    Google Scholar 

  4. Magill SS, Edwards JR, Bamberg W, et al. Multistate point-prevalence survey of health care-associated infections. N Engl J Med. 2014;370(13):1198–208.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  5. Melamed A, Sorvillo FJ. The burden of sepsis-associated mortality in the United States from 1999 to 2005: an analysis of multiple-cause-of-death data. Crit Care. 2009;13(1):R28.

    PubMed  PubMed Central  Google Scholar 

  6. Vincent JL, Sakr Y, Sprung CL, et al. Sepsis in European intensive care units: results of the SOAP study. Crit Care Med. 2006;34(2):344–53.

    Article  PubMed  Google Scholar 

  7. website C. ( last accessed 9 September 2023).

  8. Dellinger RP, Carlet JM, Masur H, et al. Surviving Sepsis Campaign guidelines for management of severe sepsis and septic shock. Crit Care Med. 2004;32(3):858–73.

    PubMed  Google Scholar 

  9. Dellinger RP, Levy MM, Carlet JM, et al. Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock: 2008. Crit Care Med. 2008;36(1):296–327.

    PubMed  Google Scholar 

  10. Dellinger RP, Levy MM, Rhodes A, et al. Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock, 2012. Intensive Care Med. 2013;39(2):165–228.

    PubMed  PubMed Central  CAS  Google Scholar 

  11. Evans L, Rhodes A, Alhazzani W, et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock 2021. Crit Care Med. 2021;49(11):e1063–143.

    PubMed  Google Scholar 

  12. Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement Jama. 1999;282(15):1458–65.

    PubMed  CAS  Google Scholar 

  13. Levy MM, Pronovost PJ, Dellinger RP, et al. Sepsis change bundles: converting guidelines into meaningful change in behavior and clinical outcome. Crit Care Med. 2004;32(11 Suppl):S595–7.

    PubMed  Google Scholar 

  14. Levy MM, Evans LE, Rhodes A. The Surviving Sepsis Campaign Bundle: 2018 Update. Crit Care Med. 2018;46(6):997–1000.

    PubMed  Google Scholar 

  15. Chen QH, Zheng RQ, Lin H, et al. The impact of sepsis bundles on mortality in patients with sepsis shock: a prospective clinical study. Zhongguo Wei Zhong Bing Ji Jiu Yi Xue 2008;20(9):534–7.

  16. Cronshaw HL, Daniels R, Bleetman A, Joynes E, Sheils M. Impact of the Surviving Sepsis Campaign on the recognition and management of severe sepsis in the emergency department: are we failing? Emerg Med J. 2011;28(8):670–5.

    Article  PubMed  Google Scholar 

  17. Daniels R. Surviving the first hours in sepsis: getting the basics right (an intensivist's perspective). J Antimicrob Chemother 2011;66 Suppl 2:ii11–23. DOI:

  18. Daniels R, Nutbeam T, McNamara G, Galvin C. The sepsis six and the severe sepsis resuscitation bundle: a prospective observational cohort study. Emerg Med J. 2011;28(6):507–12.

    Article  PubMed  Google Scholar 

  19. El Solh AA, Akinnusi ME, Alsawalha LN, Pineda LA. Outcome of septic shock in older adults after implementation of the sepsis “bundle.” J Am Geriatr Soc. 2008;56(2):272–8.

    Article  PubMed  Google Scholar 

  20. Ferrer R, Artigas A, Levy MM, et al. Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain. JAMA. 2008;299(19):2294–303.

    Article  PubMed  CAS  Google Scholar 

  21. Giuliano KK, Lecardo M, Staul L. Impact of protocol watch on compliance with the surviving sepsis campaign. Am J Crit Care. 2011;20(4):313–21.

    Article  PubMed  Google Scholar 

  22. Hoo WE, Muehlberg K, Ferraro RG, Jumaoas MC. Successes and lessons learned implementing the sepsis bundle. J Healthc Qual 2009;31(4):9–14; quiz 14–5. (

  23. Jeon K, Shin TG, Sim MS, et al. Improvements in compliance with resuscitation bundles and achievement of end points after an educational program on the management of severe sepsis and septic shock. Shock. 2012;37(5):463–7.

    Article  PubMed  Google Scholar 

  24. Johnson EE, Sterba KR, Goodwin AJ, et al. Implementation of an Academic-to-Community Hospital Intensive Care Unit Quality Improvement Program. Qualitative Analysis of Multilevel Facilitators and Barriers. Ann Am Thorac Soc 2019;16(7):877–885. DOI:

  25. Kortgen A, Niederprum P, Bauer M. Implementation of an evidence-based “standard operating procedure” and outcome in septic shock. Crit Care Med. 2006;34(4):943–9.

    Article  PubMed  Google Scholar 

  26. Memon JI, Rehmani RS, Alaithan AM, et al. Impact of 6-hour sepsis resuscitation bundle compliance on hospital mortality in a saudi hospital. Crit Care Res Pract 2012;2012:

  27. Menendez R, Torres A, Reyes S, et al. Compliance with guidelines-recommended processes in pneumonia: impact of health status and initial signs. PLoS ONE. 2012;7(5): e37570.

    PubMed  PubMed Central  CAS  Google Scholar 

  28. Miller RR 3rd, Dong L, Nelson NC, et al. Multicenter implementation of a severe sepsis and septic shock treatment bundle. Am J Respir Crit Care Med. 2013;188(1):77–82.

    PubMed  PubMed Central  Google Scholar 

  29. Nguyen HB, Corbett SW, Steele R, et al. Implementation of a bundle of quality indicators for the early management of severe sepsis and septic shock is associated with decreased mortality. Crit Care Med. 2007;35(4):1105–12.

    PubMed  Google Scholar 

  30. Nguyen HB, Lynch EL, Mou JA, Lyon K, Wittlake WA, Corbett SW. The utility of a quality improvement bundle in bridging the gap between research and standard care in the management of severe sepsis and septic shock in the emergency department. Acad Emerg Med. 2007;14(11):1079–86.

    PubMed  Google Scholar 

  31. Pestana D, Espinosa E, Sanguesa-Molina JR, et al. Compliance with a sepsis bundle and its effect on intensive care unit mortality in surgical septic shock patients. J Trauma. 2010;69(5):1282–7.

    PubMed  Google Scholar 

  32. Phua J, Ho BC, Tee A, et al. The impact of clinical protocols in the management of severe sepsis: a prospective cohort study. Anaesth Intensive Care. 2012;40(4):663–74.

    PubMed  CAS  Google Scholar 

  33. Phua J, Koh Y, Du B, et al. Management of severe sepsis in patients admitted to Asian intensive care units: prospective cohort study. BMJ. 2011;342: d3245.

    PubMed  PubMed Central  Google Scholar 

  34. Pruinelli L, Westra BL, Yadav P, et al. Delay Within the 3-Hour Surviving Sepsis Campaign Guideline on Mortality for Patients With Severe Sepsis and Septic Shock. Crit Care Med. 2018;46(4):500–5.

    PubMed  PubMed Central  Google Scholar 

  35. Wang Z, Xiong Y, Schorr C, Dellinger RP. Impact of sepsis bundle strategy on outcomes of patients suffering from severe sepsis and septic shock in china. J Emerg Med. 2013;44(4):735–41.

    PubMed  Google Scholar 

  36. Dellinger RP, Vincent JL. The Surviving Sepsis Campaign sepsis change bundles and clinical practice. Crit Care. 2005;9(6):653–4.

    PubMed  PubMed Central  Google Scholar 

  37. Lagu T, Higgins TL, Nathanson BH, Lindenauer PK. Incorporating initial treatments improves the performance of a mortality prediction model based on administrative data. Crit Care Med. 2012;40(7):2268–9 author reply 2269–70.

    PubMed  Google Scholar 

  38. Lagu T, Rothberg MB, Nathanson BH, Pekow PS, Steingrub JS, Lindenauer PK. Variation in the care of septic shock: the impact of patient and hospital characteristics. J Crit Care. 2012;27(4):329–36.

    PubMed  Google Scholar 

  39. Levy MM, Rhodes A, Phillips GS, et al. Surviving Sepsis Campaign: association between performance metrics and outcomes in a 7.5-year study. Crit Care Med. 2015;43(1):3–12.

    PubMed  Google Scholar 

  40. Levy MM, Gesten FC, Phillips GS, et al. Mortality Changes Associated with Mandated Public Reporting for Sepsis. The Results of the New York State Initiative. Am J Respir Crit Care Med. 2018;198(11):1406–12.

    PubMed  PubMed Central  Google Scholar 

  41. Seymour CW, Gesten F, Prescott HC, et al. Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. N Engl J Med. 2017;376(23):2235–44.

    PubMed  PubMed Central  Google Scholar 

  42. Barbash IJ, Davis B, Kahn JM. National Performance on the Medicare SEP-1 Sepsis Quality Measure. Crit Care Med. 2019;47(8):1026–32.

    PubMed  PubMed Central  Google Scholar 

  43. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implementation science : IS. 2013;8:139.

    PubMed  PubMed Central  Google Scholar 

  44. Bauer MS, Kirchner J. Implementation science: What is it and why should I care? Psychiatry Res. 2020;283: 112376.

    PubMed  Google Scholar 

  45. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38(1):4–23.

    PubMed  Google Scholar 

  46. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation Hybrid Designs. Med Care. 2012;50(3):217–26.

    PubMed  PubMed Central  Google Scholar 

  47. Steinmo SH, Michie S, Fuller C, Stanley S, Stapleton C, Stone SP. Bridging the gap between pragmatic intervention design and theory: using behavioural science tools to modify an existing quality improvement programme to implement “Sepsis Six.” Implement Sci. 2016;11:14.

    PubMed  PubMed Central  Google Scholar 

  48. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350: h2147.

    PubMed  Google Scholar 

  49. Loudon K, Zwarenstein M, Sullivan F, Donnan P, Treweek S. Making clinical trials more relevant: improving and validating the PRECIS tool for matching trial design decisions to trial purpose. Trials. 2013;14:115.

    PubMed  PubMed Central  Google Scholar 

  50. Loudon K, Zwarenstein M, Sullivan FM, et al. The PRECIS-2 tool has good interrater reliability and modest discriminant validity. J Clin Epidemiol. 2017;88:113–21.

    PubMed  Google Scholar 

  51. Mentz RJ, Hernandez AF, Berdan LG, et al. Good Clinical Practice Guidance and Pragmatic Clinical Trials: Balancing the Best of Both Worlds. Circulation. 2016;133(9):872–80.

    PubMed  PubMed Central  Google Scholar 

  52. Chan A-W, Tetzlaff JM, Gøtzsche PC, SPIRIT, et al. explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;2013:346.

    Google Scholar 

  53. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.

    PubMed  PubMed Central  Google Scholar 

  54. Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implement Sci. 2019;14(1):1.

    PubMed  PubMed Central  Google Scholar 

  55. Elwy AR, Wasan AD, Gillman AG, et al. Using formative evaluation methods to improve clinical implementation efforts: Description and an example. Psychiatry Res. 2019;283:112532.

    PubMed  Google Scholar 

  56. Aarons GA, Ehrhart MG, Farahnak LR, Sklar M, Horowitz J. Discrepancies in Leader and Follower Ratings of Transformational Leadership: Relationship with Organizational Culture in Mental Health. Adm Policy Ment Health. 2017;44(4):480–91.

    PubMed  PubMed Central  Google Scholar 

  57. Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res. 2004;6(2):61–74.

    PubMed  PubMed Central  Google Scholar 

  58. Weiner BJ, Belden CM, Bergmire DM, Johnston M. The meaning and measurement of implementation climate. Implement Sci. 2011;6:78.

    PubMed  PubMed Central  Google Scholar 

  59. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

    PubMed  Google Scholar 

  60. Harris PA, Taylor R, Minor BL, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019;95: 103208.

    PubMed  PubMed Central  Google Scholar 

  61. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.

    PubMed  Google Scholar 

  62. Pun BT, Balas MC, Barnes-Daly MA, et al. Caring for Critically Ill Patients with the ABCDEF Bundle: Results of the ICU Liberation Collaborative in Over 15,000 Adults. Crit Care Med. 2019;47(1):3–14.

    PubMed  PubMed Central  Google Scholar 

  63. REDCap. ( accessed September 9, 2023).

  64. PSAT. Tool. ( (last accessed September 89, 2023).

  65. Kleinpell R, Zimmerman J, Vermoch KL, et al. Promoting Family Engagement in the ICU: Experience From a National Collaborative of 63 ICUs. Crit Care Med. 2019;47(12):1692–8.

    PubMed  Google Scholar 

  66. VIRUS. ( (last accessed September 9, 2023).

  67. Manual Si. ( accessed September 9, 2023).

  68. SARI-PREP. ( (last accessed September 9, 2023).

  69. Saunders B, Sim J, Kingstone T, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. 2018;52(4):1893–907.

    PubMed  Google Scholar 

  70. Hamilton AB. Qualitative methods in rapid turn-around health services research. U.S. Department of Veterans Affairs. ( (last accessed September 9, 2023).

  71. Taylor B, Henshall C, Kenyon S, Litchfield I, Greenfield S. Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open. 2018;8(10): e019993.

    PubMed  PubMed Central  Google Scholar 

  72. Palinkas LA, Mendon SJ, Hamilton AB. Innovations in Mixed Methods Evaluations. Annu Rev Public Health. 2019;40(1):423–42.

    PubMed  PubMed Central  Google Scholar 

  73. Seymour CW, Kennedy JN, Wang S, et al. Derivation, Validation, and Potential Treatment Implications of Novel Clinical Phenotypes for Sepsis. JAMA. 2019;321(20):2003–17.

    PubMed  PubMed Central  CAS  Google Scholar 

Download references


Research reported in this manuscript was supported by the National Institutes of Health Heart, Lung and Blood Institute under award number RO1HL162954.


This study is funded by the National Heart, Lung, and Blood Institute (NHLBI R01HL162954). The study’s Program Official is Mihaela Stefan ( The funding body did not have a role in the design of the study; collection, analysis, and interpretation of the data; or writing the manuscript.

Author information

Authors and Affiliations



HEF is a study co-investigator who contributed to study design and wrote the initial draft of the manuscript. JG is a research assistant who assisted with drafting the initial version of the manuscript. LE, GP, DP, NS, CS, and KT are co-investigators and/or consultants and have made substantial contributions to the study design and methods. RPD and SRT contributed to the study design and methods. LH is the study project manager and has contributed to trial planning and coordination of study activities, including management of multi-site ethics approvals. MML is the Principal Investigator; he conceived of the study and led the proposal and protocol development. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mitchell M. Levy.

Ethics declarations

Ethics approval and consent to participate

All sites recruited for this trial will be instructed to follow their hospital’s research policies for local IRB approval. In addition and as is required by the National Institutes of Health, a central IRB (wcg IRB) has been established for full study review with each site submitting for central approval (Protection of Human Subjects #35170744.0; Protocol Version 2.0). The study and protocol (including the Data Safety Monitoring Plan and DSMB charter) was approved by the IRB at Rhode Island Hospital (CMTT/PROJ #218420). Any changes to study procedures or the study protocol will be reviewed by the central IRB. The central IRB approved a waiver of consent for patients enrolled in this study given that both interventions are now considered as the standard of care in alignment with the Centers for Medicare and Medicaid Services SEP-1 core measures. Because of the challenge to define “unanticipated events” for sepsis, the investigators have been advised of the following: “If any investigator observes what they believe to be a significant adverse event that is definitely or possibly related to the study procedure, please report this to the Project Manager (Lori Harmon) and the Principal Investigator (Mitchell Levy) within 24 h of the event.”

Consent for publication

Not applicable.

Competing interests

Dr. Evans is co-chair Adult Surviving Sepsis Guidelines, co-author of SSC 1-h bundle, and member of Scientific Advisory Board for Endpoint Health (USA). Dr. Dellinger has held a leadership position Surviving Sepsis Campaign 2002–2016. Dr. Schorr has been in a leadership position for the Surviving Sepsis Campaign from 2006 to present, and is a paid consultant on the current grant (R01HL162954). Dr. Townsend is a measure steward for the Centers for Medicare and Medicaid Services sepsis quality measure. Dr. Levy receives funding from T32 HL134625, 1RO1HL162954-0, is co-author of the SSC 1-h bundle, and is on scientific advisory boards for Inotrem (France) and Endpoint Health (USA). All other authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Frank, H.E., Evans, L., Phillips, G. et al. Assessment of implementation methods in sepsis: study protocol for a cluster-randomized hybrid type 2 trial. Trials 24, 620 (2023).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: