- Study protocol
- Open Access
- Open Peer Review
Evaluating processes of care and outcomes of children in hospital (EPOCH): study protocol for a randomized controlled trial
Trials volume 16, Article number: 245 (2015)
The prevention of near and actual cardiopulmonary arrest in hospitalized children is a patient safety imperative. Prevention is contingent upon the timely identification, referral and treatment of children who are deteriorating clinically. We designed and validated a documentation-based system of care to permit identification and referral as well as facilitate provision of timely treatment. We called it the Bedside Paediatric Early Warning System (BedsidePEWS). Here we describe the rationale for the design, intervention and outcomes of the study entitled Evaluating Processes and Outcomes of Children in Hospital (EPOCH).
EPOCH is a cluster-randomized trial of the BedsidePEWS. The unit of randomization is the participating hospital. Eligible hospitals have a Pediatric Intensive Care Unit (PICU), are anticipated to have organizational stability throughout the study, are not using a severity of illness score in hospital wards and are willing to be randomized. Patients are >37 weeks gestational age and <18 years and are hospitalized in inpatient ward areas during all or part of their hospital admission.
Randomization is to either BedsidePEWS or control (no severity of illness score) in a 1:1 ratio within two strata (<200, ≥200 hospital beds). All-cause hospital mortality is the selected primary outcome. It is objective, independent of do-not-resuscitate status and can be reliably measured. The secondary outcomes include (1) clinical outcomes: clinical deterioration, severity of illness at and during ICU admission, and potentially preventable cardiac arrest; (2) processes of care outcomes: immediate calls for assistance, hospital and ICU readmission, and perceptions of healthcare professionals; and (3) resource utilization: ICU days and use of ICU therapies.
Following funding by the Canadian Institutes of Health Research and local ethical approvals, site enrollment started in 2010 and was closed in February 2014. Patient enrollment is anticipated to be complete in July 2015. The results of EPOCH will strengthen the scientific basis for local, regional, provincial and national decision-making and for the recommendations of national and international bodies. If negative, the costs of hospital-wide implementation can be avoided. If positive, EPOCH will have provided a scientific justification for the major system-level changes required for implementation.
Trial registration: NCT01260831 ClinicalTrials.gov date: 14 December 2010.
The Evaluation of Processes of care and the Outcomes of Children in Hospital (EPOCH) study is a cluster-randomized trial comparing the effect of the Bedside Paediatric Early Warning System (BedsidePEWS) with standard care. The outcomes are clinical, processes of care and resource utilization. Here we describe the rationale for the trial, the intervention and the outcomes used, conflict of interest management and the potential impact of the study.
The problem and current treatments
Cardiopulmonary arrest occurs in 0.1-20/1000 children in hospital wards [1-4], with hospital survival of 27-50% [5-7]. Despite optimal resuscitation and post-arrest care, in-hospital cardiac arrest is associated with significant mortality and acquired morbidity in survivors [3,6,8-11], making prevention the best strategy [6,8,12-23].
Prevention of near and actual cardiopulmonary arrest
Prevention of near and actual cardiopulmonary arrest in hospitalized children is a patient safety imperative. Prevention is contingent upon the timely identification, referral and treatment of children whose conditions are clinically deteriorating [24,25]. Preventative strategies have focused on the latter portion of this sequence, primarily through improved access to ICU expertise through deployment of Medical Emergency Teams-Rapid Response Teams (MET-RRT). However, studies addressing methods to identify those in need have been few and of limited quality .
Emergency response teams
To date, MET-RRT have not fulfilled their promise. In adult inpatients, following several positive [27-29] and negative [30-33] single-center studies, a 23-hospital, cluster-randomized clinical trial compared implementation of un-validated calling criteria plus MET-RRT with standard care (no MET-RRT, no explicit calling criteria). The primary outcome was a composite of unexpected death, unplanned ICU admission and cardiac arrest. This negative trial  provides several important lessons: (1) calling criteria were met in <50% of patients with urgent ICU admission, suggesting that the criteria did not identify patients at risk. (2) The number of patients without unexpected death, unplanned ICU admission or cardiac arrest, who met one or more of these calling criteria is unknown (similar to pediatric studies) [35-39]. (3) Approximately 90% of the patients who met criteria were referred to ICU teams in both MET and control hospitals, suggesting that the calling criteria added little to the standard model.
In children, only before-and-after studies have been published [35-39], of which one is multi-center . The two studies showing reduced all-cause hospital mortality were performed over 6  and 7½ years , influenced the findings of a systematic review of pediatric MET-RRT , which include no RCTs, and reinforced the potential for description of time-related improvement [42,43] to be ascribed to MET-RRT. Furthermore, the additional benefit from MET-RRT without appropriate ‘afferent limb’ identification mechanisms may be questionable in hospitals where the efferent limb response-ICU expertise is readily available .
Mechanisms to identify children at risk for near and actual cardiopulmonary arrest
There are eight published pediatric mechanisms to identify children at risk for cardiac arrest [1,35,37,44-47]. The development of most has been methodologically limited . There are four published calling criteria [35,37,46,47]. The Cincinnati criteria include subjective elements and have been modified [35,49]. The Baltimore criteria are subjective and/or triggered by acute medical diagnoses . The Bristol tool is subjective and uses thresholds for the airway (nebulized epinephrine, ‘tiring’) and disability . The Melbourne criteria identified 10/24 (42%) patients who had cardiac arrests in their hospital  and similar proportions in other hospitals [40,50].
There are four published pediatric scores [1,44,45,48]. The Brighton score uses behavior, circulatory (color, relative tachycardia) and respiratory (relative tachypnea) domains and ‘persistent vomiting following surgery’ with a range of 0 –11 . Evaluation of this score in a cohort of 2979 children in Cincinnati Children’s Hospital found a sensitivity of 78% and specificity of 82% . In the development cohort, the Cardiff and Vale score had a sensitivity of 69% and specificity of 90%. Inspection of the score reveals that children in the first months of life may be expected to routinely score at or above the threshold (2 points for systolic blood pressure <70 mmHg and respiratory rate >50 breaths/min) .
The BedsidePEWS score differentiated between children who were urgently admitted to PICU and ‘well’ hospitalized children, with at least 1 h notice with a sensitivity of 83% and specificity of 95% . In a multi-center validation study of 2074 patients the score performed well in sub-populations . Among ‘case patients’ the score increased over time leading up to an urgent PICU admission or code-blue event, and was independent of the number of risk factors for cardiac arrest. .
The score was reliably calculated with an intraclass correlation coefficient of 0.92 and was associated with statistically significant improvements in clinical and process of care outcomes . The quality of development, validation and evidence of reliability supports the conclusion that the BedsidePEWS score is equivalent or superior to others and provides a rationale for its use as the intervention in EPOCH (Table 1).
The objectives of EPOCH are to evaluate the impact of the BedsidePEWS on early identification of children at risk for near and actual cardiopulmonary arrest, hospital mortality, processes of care and ICU resource utilization. We hypothesize that the BedsidePEWS improves early detection of critical illness, reduces mortality, improves processes of care and does not increase healthcare resource utilization.
EPOCH is a cluster-randomized trial comparing the BedsidePEWS with standard care (no severity of illness score) in hospitals with a pediatric intensive care unit (ICU). The unit of randomization is the participating hospital. Eligible hospitals provide care for more than a total of 200 eligible inpatient admissions in eligible inpatient wards each year, have specialized pediatric physicians and have at least one PICU. Eligible inpatient wards are defined as areas where care is provided to children who are admitted to the hospital, other than the NICU, PICU, operating rooms and other designated areas where anesthetist-supervised procedures are performed. Emergency departments that care for admitted patients may be regarded as an eligible ‘ward’ (Table 2). Eligible patients are >37 weeks gestational age and <18 years, and they have received care in an eligible inpatient ward. To ensure that major system changes do not introduce bias, we will exclude hospitals that plan to introduce or discontinue an MET-RRT during the study, are already using a severity of illness score in wards or consider randomization unacceptable. Hospitals with a pre-existing MET-RRT are eligible to participate.
The BedsidePEWS documentation-based system of care replaces existing documentation systems for vital signs in inpatient ward areas in hospitals randomized to the intervention. There are four elements in the system: (1) the BedsidePEWS score; (2) the BedsidePEWS documentation record; (3) score-matched care recommendations, developed using the responses of 280 healthcare professionals (80 community, 200 referral) surveyed to determine ‘reasonable’ care in each of the domains of vital sign assessment, secondary review, continuous monitoring and ICU consultation, and customized for each implementation; (4) the education program, developed by two nurse educators to support implementation and maintain expertise, including the BedsidePEWS Instructor course and frontline education program.
Participating hospitals are randomized in a 1:1 ratio to intervention or control. Computerized randomization is balanced by the use of two strata of hospital size: <200 vs. ≥200 eligible inpatient ward beds. Allocation is concealed until the start of the study measurements and is revealed in the 2nd week after the start of data collection at each site (Additional file 1).
Primary outcome: all-cause hospital mortality
The primary outcome is all-cause hospital mortality in eligible inpatients who were cared for in an eligible inpatient ward at some point during their hospital stay. This includes anticipated deaths in children with ‘do not resuscitate’ orders. Mortality events in children cared for exclusively in the PICU, NICU or emergency department, or combinations thereof, are excluded.
Secondary outcomes: clinical
The main secondary outcome is the significant clinical deterioration event (SCDE), a composite outcome comprised of the treatment(s) provided or death prior to transfer from an inpatient ward. An SCDE is defined as the provision of significant pre-specified respiratory or circulatory therapies or cardiopulmonary resuscitation in the 12 h before transfer from the inpatient ward or during the hour after transfer, or death without a DNR order in an inpatient ward (Table 2).
EPOCH includes eight additional clinical outcomes (Table 3): (1) the nature of SCDE rated on the seven-point Children’s Resuscitation Intensity Scale (Table 4), which measures the timing of interventions in children for whom active resuscitation is anticipated; (2) potentially preventable cardiac arrest in patients experiencing a cardiac arrest while in an eligible inpatient ward. This excludes patients with a preceding DNR order. Potentially preventable is defined as the degree to which ‘events may have been avoided given the application of reasonable current (2011-13) standards of practice by an average practitioner and system anticipated to manage the condition in question’ [53-57]. Events with a consensus rating from two expert reviewers of ≥4/6 are defined as potentially preventable cardiac arrests (Table 5) [55,56]. (3) Unplanned hospital re-admission; (4) urgent PICU re-admission, both within 48 h of discharge. Finally, in all patients admitted urgently to the PICU from a hospital inpatient ward measurement was made of: (5) the predicted risk of mortality (PIM2 score) [58,59], (6) PICU mortality, (7) organ dysfunction (PELOD score)  and (8) ventilator-free days to 28 days. Urgent PICU admission is defined as departure from the inpatient ward within 6 h of the decision to admit to the ICU (Table 2). This definition reflects the need to pragmatically distinguish between urgent and elective admission and the recognition that most urgent admission occurs within 1-2 h of the decision to transfer to an ICU.
Secondary outcomes: process of care
The study includes five processes of care assessments: (1) ‘Stat’ calls. This was formally defined (Table 3), and a consistent approach to data collection was applied throughout the study, recognizing that cultural and other factors may contribute to considerable inter-hospital variability. (2) Code Blue calls for immediate medical assistance of the resuscitation team and equipment. (3) Urgent consultations (within 15 min) to the ICU or MET-RRT. (4) Using documentation from five randomly (central computerized randomization) selected patients each week, physician visits, the nurse:patient ratio and the use of continuous monitoring are described. (5) Frontline staff complete the documentation and interaction survey (2 pages, <10 min to complete) that describes their perceptions of the documentation system and the nature of interactions with physicians (Additional file 2: CRF). This custom-built survey was developed and used in our pilot evaluations of BedsidePEWS. It has not been formally validated.
Secondary outcomes: resource utilization and decision-maker perceptions
Resource utilization is assessed in all hospitals by measurement of hospital length of stay and for patients undergoing urgent ICU admission, ICU length of stay and the days of use of the ICU technologies: mechanical ventilation, hemodialysis, ECMO and nitric oxide. A purpose-built survey of the perceptions of hospital decision-makers will be conducted 3 months after the end of the 12-month intervention period (Table 3) to describe the experience of the decision-maker in participating in a hospital-wide cluster RCT and their expectations of its results.
All outcomes are assessed prospectively. The first 26 weeks are baseline data. Then following a 5-week run-in period in intervention hospitals or break from data collection in control hospitals, outcomes are assessed prospectively for 52 weeks (Additional file 1).
Sample size and assumptions
Power calculations are based on population estimates derived from 14 local hospitals’ data using a published method for cluster RCTs . For all-cause mortality we found a baseline rate of 5.1/1000. The steering committee agreed that a mortality reduction of less than 1/1000 admissions would not be a compelling reason to modify practice. A study with 20 hospitals can show an 18% relative risk reduction in mortality (absolute risk reduction 0.09%), given alpha = 0.05 (2-sided), power = 80%, mean of 119 pediatric beds, length of stay = 4 days, with 0.90 average occupancy, k = 0.15, n = 20, baseline rate = 5.1/1000. The quantity k is the coefficient of variation for mortality between hospitals. Assuming attrition of 1-2 hospitals, we will include a sample of 22 hospitals.
Data from 4 pediatric hospitals in Ontario indicate there are 1052 urgent ICU admissions/year, of which we estimate 40% involve SCDE, for a rate of 2 per thousand patient days. Thus, with 20 sites we will be able to show a 31% reduction in SCDE [alpha = 0.05 (2-sided), power = 80%, k = 0.15, where k is the coefficient of variation for SCDE rate between hospitals]. The value of k allows for a moderate amount of clustering within hospital.
Demographic and unadjusted outcomes data will be reported using means, median, variances, interquartile ranges or as proportions with 95% confidence intervals. Outcomes will be reported for the baseline and intervention periods for each hospital consistent with published recommendations for cluster randomized trials .
All-cause hospital mortality will be evaluated using a logit regression model. The dependent variable will be the logit of the proportion dying in each hospital. The independent variables will include a dummy indicator for treatment arm, the baseline mortality logit and the hospital size stratification variable. The analysis will be weighted by the size of the hospital.
An identical logit model will be used for ICU mortality after urgent ICU admission where the PICU admission was initiated in a hospital ward, ICU mortality after urgent ICU admission initiated in the OR, unplanned hospital re-admission within 2 days of hospital discharge and mortality following DNR orders.
Significant clinical deterioration events, Code Blue events, stat calls and urgent PICU consultations per 1000 patient-days will be evaluated using Poisson regression using hospital-level aggregated count data. The independent variables will include a dummy indicator for treatment arm, the baseline event rate and hospital size stratification variable. A linear regression model, in which the within-hospital mean is the dependent variable, will be used to evaluate the nature of SCDE, inpatient ward patient-days, and for patients admitted urgently to the PICU, PIM2 score, PELOD score, ventilator-free days, ICU patient-days and ICU therapy-days. The independent variables will include a dummy indicator for treatment arm, the baseline means and hospital size stratification variable. The analysis will be weighted by the size of the hospital.
An identical linear regression model will be used to evaluate the frequency of vital sign documentation from the randomly selected patient records. Since the number of records abstracted will be the same in each hospital, weighting will not be necessary. Documentation will be evaluated as follows: the number of heart rate, respiratory rate, systolic blood pressure, trans-cutaneous oxygen saturation, capillary refill, oxygen therapy and respiratory effort measurements, and the number of times that all seven items were documented will be compared for patients in hospitals with and without BedsidePEWS. All analyses will adjust for baseline event rates and hospital strata. Linear regression will be used to compare the numbers of (1) vital signs documented, (2) documented physician visits, (3) the nurse: patient ratio, the use of (4) continuous ECG monitoring and (5) continuous pulse oximeter monitoring with the BedsidePEWS scores calculated from the abstracted clinical data.
Analysis of the documentation and interaction survey and the post-study decision-maker survey data will compare groups using linear regression weighted by the size of the hospital.
The following four pre-specified subgroup analyses will be performed. (1) Hospital size. Hospitals will be classified on the basis of the number of eligible inpatient ward beds. Hospitals with ≥200 eligible inpatient ward beds will constitute one group and those with <200 eligible inpatient ward beds the other. This is consistent with the stratification method used for randomization. (2) Hospitals with and without MET-RRT. (3) Hospitals with ECMO for children. (4) Patients with urgent PICU admission initiated in an inpatient ward.
Initial application for funding was made in September 2009, accompanied by 18 letters from hospital CEOs or other senior administrators indicating willingness for their hospital to participate. Successful re-submission in March of 2010 preceded announcement of funding in July 2010. Study resources were obtained from Canadian Institutes of Health Research (CIHR). After approval of $4 million by the reviewing panel, a budget of $3.3 million was granted. Funds were released by the CIHR in December 2010 following recruitment and approval of the three-person data safety and monitoring board, clinical trial registration (CTN#1000018562) and Research Ethics Board (REB-IRB) approval at the sponsoring institution. The study coordinating center is at the Center for Safety Research at the Hospital for Sick Children, Toronto. The study staff is supported by the infrastructure of the Child Health Evaluative Sciences Program of the Research Institute of the Hospital for Sick Children.
EPOCH REB-IRB applications sought and obtained waiver of patient-level consent (Table 6). The local documentation system is used as a part of routine hospital practice, patient-level data are retrospectively obtained and precedent exists in Canada and the UK [51,52]. Furthermore, consent for the participation of 100,000 patents is not feasible, and if required, would significantly undermine the scientific validity of the study. REB-IRB applications also seek approval of implied consent from healthcare professionals for staff surveys and the post-study survey of administrative perspectives . Hospital recruitment though the networks of the CCCTG preceded the enrollment of hospitals with an identified site investigator, administrative and research ethics board approval, and it was contingent on the number of sites previously enrolled in the study. The enrolled sites and associated ethical approvals are provided in Additional file 2.
Conflict of interest management
There are two parties with conflict of interest that are involved in EPOCH. These are the PI (CSP) and the host institution, The Hospital for Sick Children, who are both shareholders in a computerized decision support company (Bedside Clinical Systems). BedsidePEWS is an FDA-approved product of Bedside Clinical Systems. The study PI (CSP) is a named inventor of the BedsidePEWS (US patent 12/669,896). The patent is owned by the host institution (SickKids).
Study processes to ensure transparency, independence and oversight are in place to manage these conflicts of interest. First, EPOCH employs routine quality assurance processes including Data Safety Monitoring Board Review, annual reporting to CIHR, and scientific and operational oversight from the Executive Steering Committee and the Canadian Critical Care Trials Group (CCCTG).
Second, the PI is separated from key study processes including randomization, allocation concealment, data inspection and the conduct of analyses. The study statistician (AW) is responsible for overseeing the random allocation of sites to intervention or control arms, supervises the project manager and uses a password-protected account on randomizedtrials.net. The password is known only to the project manager and statistician, who complete the randomization process and inform the PI and site investigators of the outcome during the first 2 weeks of data collection at each site.
Third, review of site progress in sites randomized to implementation of the BedsidePEWS is conducted by the Executive Steering Committee in the 4th week of the run-in phase. The decision to complete or prolong the run-in phase is made by the majority vote of the Executive Steering Committee. The PI does not vote.
Fourth, data integrity is ensured by three methods. (1) Separation of data collection from randomization status will be achieved by training site coordinators before randomization, using standardized materials. (2) Inspection of study data during site visits is conducted by the Center for Safety Research staff. If present, the PI will not review primary study data. (3) Following creation of the ‘locked’ data set, each site investigator receives a data report to review, reconcile with the data from their site and provide written confirmation that the data are correct or require reconciliation. Once confirmed by all site investigators the final data set will undergo planned analyses.
Fifth, once completed the analyses will be reviewed by an independent statistician based at another university.
Sixth, the interpretation of the data and their reporting in iterative drafts of the main study manuscript will be a shared activity of the executive steering committee and writing committee and will be complimented by pre-submission review by independent CCCTG reviewers.
Preliminary data suggest that the BedsidePEWS score can identify patients at risk for near and actual cardiopulmonary arrest, can be calculated both rapidly and reliably and that its clinical implementation is associated with potential improvements in patient and process of care outcomes. The resource implications of system-wide implementation, the previous lessons from the MERIT study, and the uncertain effect on important clinical outcomes underscore the relevance of the cluster-randomized design, the selection of the intervention and the outcomes used in EPOCH. EPOCH will be conducted in 22 hospitals, involve an estimated 100,000 patients admitted for about 400,000 patient-days and describe approximately 1900 urgent ICU admissions, 200 cardiac arrest events and 500 deaths. Mortality in hospitalized children is an objective measure of quality of care and is used extensively to benchmark the performance of PICUs [58,63-66], surgical programs and hospitals [67-69].
All-cause hospital mortality was selected as the primary outcome in EPOCH for the following reasons. First, mortality is objective and can be reliably assessed. Second, a reduction in all-cause hospital mortality is the ultimate goal of any intervention to improve the outcomes of care of hospitalized children. Third, observational studies of MET-RRT have reported reduced all-cause mortality [36,38]. Fourth, several studies show that when children who have been hospitalized die, most do so in hospital [70-73]. In a 6-year study of in-ICU cardiac arrest, we found only one (1.4%) additional death after index-hospitalization discharge and within 12 months of the index cardiac arrest. Fifth, the effect of pediatric palliative care services on displacing place of death to home or hospice is small [72,74-77]. One large US study found the proportion of deaths in the hospital decreased by less than 6% (from 85.7 to 80.1%) over a 10-year period . Thus, measuring hospital mortality provides a reasonably accurate measure of mortality in children who have been or are currently hospitalized.
As well, all-cause mortality is an established quality metric in Canadian and British adult hospitals and is publicly reported as the standardized mortality ratio in Ontario Hospitals [68,69,78-80]. Finally, while death ‘with DNR’ vs. ‘unexpected’ deaths (no DNR) has been used to distinguish ‘preventable’ from ‘unpreventable’ deaths following acute events in adult patients [34,81], in hospitalized children there are significant limitations to this distinction: (1) DNR orders reflect current expectations of outcome and not the ‘preventability’ of the preceding clinical events; (2) the majority of deaths in hospitalized children, especially in the ICU, occur some days after a clinical deterioration event [3,7,10,18,21,82], but within a relatively short time (hours) between DNR order and death [83-85]. Before confirming that a death in a patient with a DNR order was truly unpreventable, the clinical context and timeline of clinical deterioration prior to the DNR order must be known. It must also be recognized that medically ‘futile’ resuscitation may be performed on hospitalized children  for whom no DNR order is in place.
EPOCH outcomes are assessed for 1 year after hospital-wide implementation of intervention or continuation of standard care. This duration was chosen to reduce the effects of seasonal variation, but also to reduce the burden of a multi-year intervention period on each hospital, thus increasing feasibility, and to support timely completion of the study to support the decision-making needs of administrators. Continued eligibility will be confirmed by intermittent reporting of the MET-RRT and Early Warning Score use during the course of the study.
Limitations of EPOCH
EPOCH has some limitations. First, by excluding the use other severity of illness scores, EPOCH will not provide data comparing alternate scores. This more refined question will need to be evaluated separately, preceded by appropriate studies to support selection of best comparators. Evaluation of the ‘best available’ system is important for proof of concept, needed before these more refined questions are addressed. Second, EPOCH does not assess quality of life, parental perceptions or neuro-cognitive outcome. All are relevant measures of the process and outcome of care . In other prospective work we have shown that urgent ICU admission is associated with reduced health-related quality of life and acquired neuro-cognitive deficits ; thus, it would be reasonable to include these measures as outcomes in EPOCH. Exclusion of these elements was due to logistic constraints; acquisition of these data for all admissions would not be feasible, and following these urgent ICU admissions would be both resource intense and would require prospective consent/assent and follow-up. Finally, mortality may be both too coarse a measure and too infrequent an event to demonstrate clinically important differences.
EPOCH is a 22-hospital cluster-randomized trial evaluating the impact of the BedsidePEWS as a mechanism to improve the identification of children with evolving critical illness, reduce near and actual cardiopulmonary arrest, and reduce all-cause hospital mortality in hospitalized children. The results of this trial will provide a scientific basis for local, regional, provincial and national decision-making and for recommendations to national and international bodies , such as the AHA and European Resuscitation Council  about cardiac arrest prevention and institutional best practices.
If negative, the costs of hospital-wide implementation can be avoided, resources more appropriately allocated and the collaborative multi-site data set from the study used to develop and evaluate new hypotheses about cardiac arrest prevention and processes of care. If positive, then EPOCH will have provided a scientific justification for the major system-level changes required for implementation and supported the creation of evidence-based policy.
At time of manuscript submission (October 2014) ethical approvals had been obtained at each participating site, study enrolment was complete, data collection had ended in 19 hospitals and was anticipated to be completed by July 2015 in remaining sites.
American Heart Association
Bedside Paediatric Early Warning System
Canadian Critical Care Trials Group
Canadian Institutes of Health Research
Evaluation of processes of care and the outcomes of children in hospital
Intensive care unit
Medical emergency team - rapid response team
Duncan H, Hutchison J, Parshuram CS. The Pediatric Early Warning System score: a severity of illness score to predict urgent medical need in hospitalized children. J Crit Care. 2006;21:271–8.
Suominen P, Olkkola KT, Voipio V, Korpela R, Palo R, Rasanen J. Utstein style reporting of in-hospital paediatric cardiopulmonary resuscitation. Resuscitation. 2000;45:17–25.
Reis AG, Nadkarni V, Perondi MB, Grisi S, Berg RA. A prospective investigation into the epidemiology of in- hospital pediatric cardiopulmonary resuscitation using the international Utstein reporting style. Pediatrics. 2002;109:200–9.
Tibballs J, Kinney S. A prospective study of outcome of in-patient paediatric cardiopulmonary arrest. Resuscitation. 2006;71:310–8.
Gillis J, Dickson D, Rieder M, Steward D, Edmonds J. Results of inpatient pediatric resuscitation. Crit Care Med. 1986;14:469–71.
Nadkarni VM, Larkin GL, Peberdy MA, Carey SM, Kaye W, Mancini ME, et al. First documented rhythm and clinical outcome from in-hospital cardiac arrest among children and adults. JAMA. 2006;295:50–7.
Meert KL, Donaldson A, Nadkarni V, Tieves KS, Schleien CL, Brilli RJ, et al. Multicenter cohort study of in-hospital pediatric cardiac arrest. Pediatr Crit Care Med. 2009;10(5):544–53.
Doherty DR, Parshuram CS, Gaboury I, Hoskote A, Lacroix J, Tucci M, et al. Hypothermia therapy after pediatric cardiac arrest. Circulation. 2009;119:1492–500.
van Litsenburg R, de Mos N, Edgell D, Gruenwald C, Bohn DJ, Parshuram CS. Resource use and health outcomes of paediatric extracorporeal membrane oxygenation. Arch Dis Child Fetal Neonatal Ed. 2005;90:F176–F7.
de Mos N, van Litsenburg RR, McCrindle B, Bohn DJ, Parshuram CS. Pediatric in-intensive-care-unit cardiac arrest: incidence, survival, and predictive factors. Crit Care Med. 2006;34:1209–15.
Ebrahim S, Hutchison J, Parshuram C. Neurocognitive and quality of life outcomes following in-hospital resuscitation. In: Pediatric Academic Societies Annual Meeting. Baltimore, MD: Pediatric Academic Societies; 2009.
Lacroix J, Hebert PC, Hutchison JS, Hume HA, Tucci M, Ducruet T, et al. Transfusion strategies for patients in pediatric intensive care units. N Engl J Med. 2007;356:1609–19.
Hutchison JS, Ward RE, Lacroix J, Hebert PC, Barnes MA, Bohn DJ, et al. Hypothermia therapy after traumatic brain injury in children. N Engl J Med. 2008;358:2447–56.
Zaritsky A, Nadkarni V, Getson P, Kuehl K. CPR in children. Ann Emerg Med. 1987;16:1107–11.
Morris MC, Wernovsky G, Nadkarni VM. Survival outcomes after extracorporeal cardiopulmonary resuscitation instituted during active chest compressions following refractory in-hospital pediatric cardiac arrest. Pediatr Crit Care Med. 2004;5:440–6.
Samson RA, Nadkarni VM, Meaney PA, Carey SM, Berg MD, Berg RA. Outcomes of in-hospital ventricular fibrillation in children. N Engl J Med. 2006;354:2328–39.
Berg MD, Nadkarni VM, Berg RA. Cardiopulmonary resuscitation in children. Curr Opin Crit Care. 2008;14:254–60.
Moler FW, Meert K, Donaldson AE, Nadkarni V, Brilli RJ, Dalton HJ, et al. In-hospital versus out-of-hospital pediatric cardiac arrest: a multicenter cohort study. Crit Care Med. 2009;37:2259–67.
Nance ML, Nadkarni VM, Hedrick HL, Cullen JA, Wiebe DJ. Effect of preextracorporeal membrane oxygenation ventilation days and age on extracorporeal membrane oxygenation survival in critically ill children. J Pediatr Surg. 2009;44:1606–10.
Sutton RM, Niles D, Nysaether J, Abella BS, Arbogast KB, Nishisaki A, et al. Pediatrics: quantitative analysis of CPR quality during in-hospital resuscitation of older children and adolescents; 2009. Quantitative analysis of CPR quality during in-hospital resuscitation of older children and adolescents. Pediatrics. 2009;124(2):494–9.
Topjian AA, Nadkarni VM, Berg RA. Cardiopulmonary resuscitation in children. Curr Opin Crit Care. 2009;15:203–8.
Perondi MB, Reis AG, Paiva EF, Nadkarni VM, Berg RA. A comparison of high-dose and standard-dose epinephrine in children with cardiac arrest. N Engl J Med. 2004;350:1722–30.
Srinivasan V, Morris MC, Helfaer MA, Berg RA, Nadkarni VM. Calcium use during in-hospital pediatric cardiopulmonary resuscitation: a report from the National Registry of Cardiopulmonary Resuscitation. Pediatrics. 2008;121:e1144–51.
Parshuram CS. Risk-adjusted intensive care unit outcomes and admission from our backyard*. Pediatr Crit Care Med. 2008;9:118–9.
VandenBerg SD, Hutchison JS, Parshuram CS. A cross-sectional survey of levels of care and response mechanisms for evolving critical illness in hospitalized children. Pediatrics. 2007;119:e940–6.
Chapman SM, Grocott MP, Franck LS. Systematic review of paediatric alert criteria for identifying hospitalised children at risk of critical deterioration. Intensive Care Med. 2009;36:600–11.
Bellomo R, Goldsmith D, Uchino S, Buckmaster J, Hart GK, Opdam H, et al. A prospective before-and-after trial of a medical emergency team. Med J Aust. 2003;179:283–7.
Buist MD, Moore GE, Bernard SA, Waxman BP, Anderson JN, Nguyen TV. Effects of a medical emergency team on reduction of incidence of and mortality from unexpected cardiac arrests in hospital: preliminary study. BMJ. 2002;324:387–90.
DeVita MA, Braithwaite RS, Mahidhara R, Stuart S, Foraida M, Simmons RL. Use of medical emergency team responses to reduce hospital cardiopulmonary arrests. Qual Saf Health Care. 2004;13:251–4.
Daly FF, Sidney KL, Fatovich DM. The Medical Emergency Team (MET): a model for the district general hospital. Aust N Z J Med. 1998;28:795–8.
Bristow PJ, Hillman KM, Chey T, Daffurn K, Jacques TC, Norman SL, et al. Rates of in-hospital arrests, deaths and intensive care admissions: the effect of a medical emergency team. Med J Aust. 2000;173:236–40.
Salamonson Y, Kariyawasam A, van Heere B, O’Connor C. The evolutionary process of Medical Emergency Team (MET) implementation: reduction in unanticipated ICU transfers. Resuscitation. 2001;49:135–41.
Parr MJ, Hadfield JH, Flabouris A, Bishop G, Hillman K. The Medical Emergency Team: 12 month analysis of reasons for activation, immediate outcome and not-for-resuscitation orders. Resuscitation. 2001;50:39–44.
Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet. 2005;365:2091–7.
Brilli RJ, Gibson R, Luria JW, Wheeler TA, Shaw J, Linam M, et al. Implementation of a medical emergency team in a large pediatric teaching hospital prevents respiratory and cardiopulmonary arrests outside the intensive care unit. Pediatr Crit Care Med. 2007;8:236–46. quiz 47.
Sharek PJ, Parast LM, Leong K, Coombs J, Earnest K, Sullivan J, et al. Effect of a rapid response team on hospital-wide mortality and code rates outside the ICU in a Children’s Hospital. JAMA. 2007;298:2267–74.
Hunt EA, Zimmer KP, Rinke ML, Shilkofski NA, Matlin C, Garger C, et al. Transition from a traditional code team to a medical emergency team and categorization of cardiopulmonary arrests in a children’s center. Arch Pediatr Adolesc Med. 2008;162:117–22.
Tibballs J, Kinney S. Reduction of hospital mortality and of preventable cardiac arrest and death on introduction of a pediatric medical emergency team. Pediatr Crit Care Med. 2009;10:306–12.
Zenker P, Schlesinger A, Hauck M, Spencer S, Hellmich T, Finkelstein M, et al. Implementation and impact of a rapid response team in a children’s hospital. Jt Comm J Qual Patient Saf. 2007;33:418–25.
Kotsakis A, Lobos AT, Parshuram C, Gilleland J, Gaiteiro R, Mohseni-Bod H, et al. Implementation of a multicenter rapid response system in pediatric academic hospitals is effective. Pediatrics. 2011;128:72–8.
Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid response teams: a systematic review and meta-analysis. Arch Intern Med. 2010;170:18–26.
Feudtner C, Hays RM, Haynes G, Geyer JR, Neff JM, Koepsell TD. Deaths attributed to pediatric complex chronic conditions: national trends and implications for supportive care services. Pediatrics. 2001;107, E99.
Joffe AR, Anton NR, Burkholder SC. Reduction in hospital mortality over time in a hospital without a pediatric medical emergency team: limitations of before-and-after study designs. Arch Pediatr Adolesc Med. 2011;165:419–23.
Monaghan A. Detecting and managing deterioration in children. Paediatr Nurs. 2005;17:32–5.
Edwards ED, Powell CV, Mason BW, Oliver A. Prospective cohort study to test the predictability of the Cardiff and Vale paediatric early warning system. Arch Dis Child. 2009;94:602–6.
Haines C, Perrott M, Weir P. Promoting care for acutely ill children-development and evaluation of a paediatric early warning tool. Intensive Crit Care Nurs. 2006;22:73–81.
Tibballs J, Kinney S, Duke T, Oakley E, Hennessy M. Reduction of paediatric in-patient cardiac arrest and death with a medical emergency team: preliminary results. Arch Dis Child. 2005;90:1148–52.
Parshuram CS, Hutchison J, Middaugh K. Development and initial validation of the Bedside Paediatric Early Warning System score. Crit Care. 2009;13:R135.
Tucker KM, Brewer TL, Baker RB, Demeritt B, Vossmeyer MT. Prospective evaluation of a pediatric inpatient early warning scoring system. J Spec Pediatr Nurs. 2009;14:79–85.
Edwards ED, Mason BW, Oliver A, Powell CV. Cohort study to test the predictability of the Melbourne criteria for activation of the medical emergency team. Arch Dis Child. 2011;96:174–9.
Parshuram CS, Duncan HP, Joffe AR, Farrell CA, Lacroix JR, Middaugh KL, et al. Multicentre validation of the bedside paediatric early warning system score: a severity of illness score to detect evolving critical illness in hospitalised children. Crit Care. 2011;15:R184.
Parshuram CS, Bayliss A, Reimer J, Middaugh K, Blanchard N. Implementing the Bedside Paediatric Early Warning System in a community hospital: a prospective observational study. Paediatr Child Health. 2011;16:e18–22.
Brennan TA, Leape LL. Adverse events, negligence in hospitalized patients: results from the Harvard Medical Practice Study. Perspect Healthc Risk Manage. 1991;11:2–8.
Thomas EJ, Studdert DM, Burstin HR, Orav EJ, Zeena T, Williams EJ, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care. 2000;38:261–71.
Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J, et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ. 2004;170:1678–86.
Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The Quality in Australian Health Care Study. Med J Aust. 1995;163:458–71.
Davis P, Lay-Yee R, Briant R, Ali W, Scott A, Schug S. Adverse events in New Zealand public hospitals I: occurrence and impact. N Z Med J. 2002;115:U271.
Slater A, Shann F, Pearson G. PIM2: a revised version of the Paediatric Index of Mortality. Intensive Care Med. 2003;29:278–85.
Slater A. Monitoring outcome in paediatric intensive care. Paediatr Anaesth. 2004;14:113–6.
Leteurtre S, Duhamel A, Grandbastien B, Lacroix J, Leclerc F. Paediatric logistic organ dysfunction (PELOD) score. Lancet. 2006;367:897. author reply 900-2.
Hayes RJ, Bennett S. Simple sample size calculation for cluster-randomized trials. Int J Epidemiol. 1999;28:319–26.
Campbell MK, Elbourne DR, Altman DG. CONSORT statement: extension to cluster randomised trials. BMJ. 2004;328:702–8.
Ruttimann UE, Patel KM, Pollack MM. Length of stay and efficiency in pediatric intensive care units. J Pediatr. 1998;133:79–85.
Pollack MM, Patel KM, Ruttimann UE. The Pediatric Risk of Mortality III-Acute Physiology Score (PRISM III-APS): a method of assessing physiologic instability for pediatric intensive care unit patients. J Pediatr. 1997;131:575–81.
Slater A, Shann F. The suitability of the Pediatric Index of Mortality (PIM), PIM2, the Pediatric Risk of Mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in Australia and New Zealand. Pediatr Crit Care Med. 2004;5:447–54.
Pearson GA, Stickley J, Shann F. Calibration of the paediatric index of mortality in UK paediatric intensive care units. Arch Dis Child. 2001;84:125–8.
Aylin P, Bottle A, Elliott P, Jarman B. Surgical mortality: Hospital episode statistics v central cardiac audit database. BMJ. 2007;335:839. author reply -40.
Jarman B, Gault S, Alves B, Hider A, Dolan S, Cook A, et al. Explaining differences in English hospital death rates using routinely collected data. BMJ. 1999;318:1515–20.
Canadian Institute for Health Information. HSMR: a New approach for measuring hospital mortality trends in Canada. Ottawa: Canadian Institute for Health Information; 2007.
Postovsky S, Levenzon A, Ofir R, Ben Arush MW. “Do not resuscitate” orders among children with solid tumors at the end of life. Pediatr Hematol Oncol. 2004;21:661–8.
Yun YH, Lim MK, Choi KS, Rhee YS. Predictors associated with the place of death in a country with increasing hospital deaths. Palliat Med. 2006;20:455–61.
Higginson IJ, Thompson M. Children and young people who die from cancer: epidemiology and place of death in England (1995-9). BMJ. 2003;327:478–9.
Feudtner C, Hexem KR, Shabbout M, Feinstein JA, Sochalski J, Silber JH. Prediction of pediatric death in the year after hospitalization: a population-level retrospective cohort study. J Palliat Med. 2009;12:160–9.
Hays RM, Valentine J, Haynes G, Geyer JR, Villareale N, McKinstry B, et al. The Seattle Pediatric Palliative Care Project: effects on family satisfaction and health-related quality of life. J Palliat Med. 2006;9:716–28.
Feudtner C, Feinstein JA, Satchell M, Zhao H, Kang TI. Shifting place of death among children with complex chronic conditions in the United States, 1989-2003. JAMA. 2007;297:2725–32.
Vickers J, Thompson A, Collins GS, Childs M, Hain R. Place and provision of palliative care for children with progressive cancer: a study by the Paediatric Oncology Nurses’ Forum/United Kingdom Children’s Cancer Study Group Palliative Care Working Group. J Clin Oncol. 2007;25:4472–6.
Gries CJ, Curtis JR, Wall RJ, Engelberg RA. Family member satisfaction with end-of-life decision making in the ICU. Chest. 2008;133:704–12.
Jarman B, Bottle A, Aylin P, Browne M. Monitoring changes in hospital standardised mortality ratios. BMJ. 2005;330:329.
Aylin P, Bottle A, Jarman B, Elliott P. Paediatric cardiac surgical mortality in England after Bristol: descriptive analysis of hospital episode statistics 1991-2002. BMJ. 2004;329:825.
Jarman B, Aylin P, Bottle A. Trends in admissions and deaths in English NHS hospitals. BMJ. 2004;328:855.
Chen J, Flabouris A, Bellomo R, Hillman K, Finfer S. The Medical Emergency Team System and not-for-resuscitation orders: results from the MERIT study. Resuscitation. 2008;79:391–7.
Berg MD, Nadkarni VM, Zuercher M, Berg RA. In-hospital pediatric cardiac arrest. Pediatr Clin North Am. 2008;55:589–604.
Garros D, Rosychuk RJ, Cox PN. Circumstances surrounding end of life in a pediatric intensive care unit. Pediatrics. 2003;112, e371.
Pierucci RL, Kirby RS, Leuthner SR. End-of-life care for neonates and infants: the experience and effects of a palliative care consultation service. Pediatrics. 2001;108:653–60.
Devictor D, Latour JM, Tissieres P. Forgoing life-sustaining or death-prolonging therapy in the pediatric ICU. Pediatr Clin North Am. 2008;55:791–804.
Truog RD. Is it always wrong to perform futile CPR? N Engl J Med. 2010;362:477–9.
Your Guide to the International Liaison Committee on Resuscitation (ILCOR). 2009. (Accessed March 28 2015, at http://www.ilcor.org/home/).
Soar J, Deakin CD, Nolan JP, Abbas G, Alfonzo A, Handley AJ, et al. European Resuscitation Council guidelines for resuscitation 2005. Section 7. Cardiac arrest in special circumstances. Resuscitation. 2005;67 Suppl 1:S135–70.
The EPOCH Executive Steering Committee gratefully acknowledges Aarthi Kamath, Kate Byrne, Sarah Ashley, Simran Singh, Susan Ferri, Jessica Grillo, Nida Shahid and Kristen Middaugh for their efforts in operationalizing the protocol into the multicenter EPOCH study. We also thank Karen Choong and Andrew Seely for providing suggestions about the manuscript as part of the Canadian Critical Care Trials Group manuscript review process.
This work was supported by the Canadian Institutes of Health Research (CIHR).
In the past 5 years, C. Parshuram, J. Hutchison, P. Parkin, K. Dryden-Palmer and A. Willan have received a salary from the Hospital for Sick Children. None of the other authors has received reimbursements, fees, funding or salary from an organization that may in any way gain or lose financially by the publication of this manuscript. The Hospital for Sick Children did not have any role in financing of this manuscript nor the decision to publish this manuscript.
C. Parshuram holds stocks in Bedside Clinical Systems, a company in part owned by the Hospital for Sick Children. Bedside Clinical Systems is a clinical decision support company that holds FDA- and Health Canada-approved electronic versions of BedsidePEWS. No other authors have relevant stocks or shares to declare.
C. Parshuram is a named inventor of a Patent for the Bedside Paediatric Early Warning System. This patent is owned by the Hospital for Sick Children, and Parshuram has received a salary from the Hospital for Sick Children as described above. No other author has a patent related to the content of the manuscript.
The authors have no other conflicts of interest to declare.
The study concept was first articulated by CSP and was developed by CSP, KDP, CF, RG, MG, JSH, MH, EH, AJ, JL, VN, PP, DW, AW and CCCTG. The protocol was drafted and submitted for funding by CSP, KDP, CF, RG, MG, JSH, MH, EH, AJ, JL, VN, PP, DW and AW; the current manuscript was drafted by CSP, reviewed and revised by CSP, KDP, CF, RG, MG, JSH, MH, EH, AJ, JL, VN, PP, DW, AW and CCCTG. All authors read and approved the final manuscript.
EPOCH study flow diagram. EPOCH timelines and study flow diagram. Three periods are described: a 26-week baseline period, a 5-week run-in phase during which hospitals randomized to implement BedsidePEWS can become familiar with the implementation and a 52-week post-randomization phase. The blue stars represent the administration of the Documentation and Interaction Survey, and the ‘Q’ represents administration of the decision-maker survey. Randomization occurs in the second week of data collection, and allocation is revealed to the primary investigator and relevant site investigator during the second study week.
We list the participating Hospital, the approving Approving Research Committee and the associated reference number for participating sites in the EPOCH cluster randomized trial.