Early warning scoring systems versus standard observations charts for wards in South Africa: a cluster randomized controlled trial
Trials volume 16, Article number: 103 (2015)
On South African public hospital wards, observation charts do not incorporate early warning scoring (EWS) systems to inform nurses when to summon assistance. The aim of this trial was to test the impact of a new chart incorporating a modified EWS (MEWS) system and a linked training program on nurses’ responses to clinical deterioration (primary outcome). Secondary outcomes were: numbers of patients with vital signs recordings in the first eight postoperative hours; number of times each vital sign was recorded; and nurses’ knowledge.
A pragmatic, parallel-group, cluster randomized, controlled clinical trial of intervention versus standard care was conducted in three intervention and three control adult surgical wards in an 867-bed public hospital in Cape Town, between March and July 2010; thereafter the MEWS chart was withdrawn. A total of 50 out of 122 nurses in full-time employment participated. From 1,427 case notes, 114 were selected by randomization for assessment.
The MEWS chart was implemented in intervention wards. Control wards delivered standard care, without training. Case notes were reviewed two weeks after the trial’s completion. Knowledge was assessed in both trial arms by blinded independent marking of written tests before and after training of nurses in intervention wards. Analyses were undertaken with IBM SPSS software on an intention-to-treat basis.
Patients in trial arms were similar. Introduction of the MEWS was not associated with statistically significant changes in responses to clinical deterioration (50 of 57 received no assistance versus 55 of 57, odds ratio (OR): 0.26, 95% confidence interval (CI): 0.05 to 1.31), despite improvement in nurses’ knowledge in intervention wards. More patients in intervention than control wards had recordings of respiratory rate (27 of 57 versus 2 of 57, OR: 24.75, 95% CI: 5.5 to 111.3) and recordings of all seven parameters (5 of 57 versus 0 of 57 patients, risk estimate: 1.10, 95% CI: 1.01 to 1.2).
A MEWS chart and training program enhanced recording of respiratory rate and of all parameters, and nurses’ knowledge, but not nurses’ responses to patients who triggered the MEWS reporting algorithm.
This trial was registered with the Pan African Clinical Trials Registry (identifier: PACTR201309000626545) on 9 September 2013.
Critically ill patients are increasingly being nursed on general wards, where it is reported that monitoring of vital signs is infrequent and inadequate [1,2]. Interpretation of signs of clinical deterioration is poor, and responses to clinical deterioration are inappropriate . Clinical and physiological deterioration, including changes in respiratory rates, occurs six to eight hours  before cardiopulmonary arrest. Arrest often occurs after a period of unrecognized, slow, and progressive physiological deterioration .
Multi-professional in-hospital courses for life support after a catastrophic event are well established . However, there are few examples of training programs for early recognition and management of adult patients with impending critical illness. One such program, the ‘Acute Life-threatening Events - Recognition and Treatment’ (ALERT) course , is based on the assumption of pre-existing knowledge of the biosciences, but previous research  suggests that this underpinning knowledge may be suboptimal. Traditional assumptions that mortality and determinants of survival fall within the domain of medical care may be inhibiting nurse-led research in this area, but there is increasing evidence that these outcomes are ‘nursing sensitive’  .
Introducing an early warning scoring (EWS) or modified early warning scoring (MEWS) system is complex . However, observational studies [4,10] and before and after evaluation studies  indicate that EWS systems improve detection of clinical deterioration. This paper describes a pragmatic, parallel-group, cluster randomized, controlled trial evaluating the impact of a MEWS training program and published consensus-derived MEWS observations chart (Additional file 1: Figure S1) [12,13] on the proportion of patients with physiological variables recorded and nurses’ responses.
To our knowledge, there are no published randomized controlled trials or other experimental studies on the implementation and evaluation of MEWS training programs and recording systems. The purpose of this trial of intervention versus standard care was to test the impact of a MEWS system and a linked training program.
We designed a prospective, pragmatic, cluster randomized trial with two arms (intervention versus no intervention), using hospital surgical wards as the unit of randomization, testing the hypothesis that nurse training plus introduction of a locally developed MEWS chart would improve knowledge, recording of vital signs, and nurses’ responses to predetermined physiological thresholds. Randomization was at two levels: wards and patient records. Our study satisfied the six criteria for pragmatic trials . There were no deviations from the protocol.
A pragmatic, cluster randomized trial design was chosen to minimize contamination, as nurses might use information for all patients, consciously or subconsciously . A permanent nurse was unlikely to be assigned to both an intervention and control ward, therefore allocation of all nurses in respective clusters to the intervention or control arm, rather than individual nurses, limited the threat of contamination. Secondly, monitoring, recording, and interpreting patients’ vital signs and responding to abnormal physiology is the responsibility of the ward nursing team, and evaluation of the quality of such care is best assessed at ward or cluster level. The Consolidated Standards of Reporting Trials (CONSORT) guidelines [14,16-18] were used to report the study (Additional file 2).
The study was conducted between 1 March and 31 July 2010 in six purposively sampled adult wards for general, vascular, and orthopedic surgery in an 867-bed academic public (government) hospital in Cape Town, South Africa. The traditional ‘cardiac arrest team’ , comprising ICU nurses and doctors, had been replaced by ward response teams more than two decades previously. There was no hospital-wide emergency response system for predefined thresholds for deterioration in physiological variables and no EWS system in place on any general wards. Approval for implementation of the MEWS chart was limited to the study period; usual care was resumed at the conclusion of the study.
Patient case notes from intervention and control wards provided clinical and demographic data.
Inclusion criteria for records from intervention and control wards
All patients identified from case notes who were aged 14 years or more , had had a general anaesthetic, and were admitted to one of the six research wards between 1 May and 31 July 2010, were eligible for inclusion.
Exclusion criteria for patient records
Those with absent, incomplete or unavailable records (for example due to medico-legal review) were excluded from the study. Incomplete records were defined as not including observations charts and patient progress notes. If either was absent, case notes were discarded, and replaced by an alternative random number. Patients designated ‘not for resuscitation’ in their case notes were excluded from the study as this might have affected nurses’ responses to changes in vital signs. Any case notes of patients transferred to high dependency care from the operating room, or from the ward, within eight hours of their operation (including patients who had died there) were excluded from the study because the study focus was ward-based vital sign monitoring.
Sample size determination for record review
We are unaware of any randomized controlled trials examining quality of recording of postoperative vital signs on general wards, or nurses’ responses to abnormal physiology, as defined by MEWS scores; this might be attributed to the relative novelty of this method of monitoring vital signs. However, the United Kingdom Health Foundation’s Safer Patients Initiative (SPI) controlled before and after study  indicates that a multi-component organizational intervention increases respiratory rate monitoring frequency from 40 (93 out of 233) to 69% (165 out of 239) in control hospitals, and from 37 (141 out of 381) to 78% (296 out of 379) in intervention hospitals. Preparatory work  indicated that patients’ respiratory rate was the parameter monitored least frequently (1.8%, one out of 55) in these wards. We felt an increase in respiratory rate monitoring frequency from 1.8 (one out of 55) to at least 20.0% (11 out of 55), would be clinically important and, based on the results of the study above, achievable.
A sample of 114 records (57 from each trial arm) was calculated to be sufficient to detect a difference of 18.2% (from 1.8 to 20.0%) between arms in the frequency of monitoring respiratory rates, with 80% power and 5% significance . No information was available on the impact of clustering on MEWS system use, and no intra-cluster correlation coefficient could be calculated.
All nurses who were permanently employed full time (professional, staff, and auxiliary) on the six study wards, and covering day and night shifts, were invited to participate in the training evaluation (n = 62 on intervention wards, n = 60 on control wards). Part-time staff were temporary and recruited from agencies. These individuals fluctuated daily, so they could not be included. Student nurses were excluded as they were allocated on a temporary, ad hoc basis. Unlike agency staff, permanent nurses were assigned to single wards, reducing the threat of contamination.
Randomization and sequence generation of wards
Six wards were randomized to either the intervention or control trial arms by the drawing of sealed lots .
Randomization and sequence generation of patient records
Following conclusion of the intervention, patients’ records were obtained for review. To select a total of 114 records, 19 records were randomly selected from all six wards. For patients who met the inclusion criteria, random numbers were generated using SPSS (IBM Corp., IBM SPSS Statistics for Windows, Version 19.0, Armonk, NY): each record was screened and, if incomplete, unavailable, or meeting other exclusion criteria, an alternative random number was generated and a new record was selected by further randomization. Records of all patients who met inclusion criteria, including those who died on the ward, were examined.
Recruitment of nurses
Post-randomization, all nurses in the intervention arm were approached to seek voluntary participation in the training program (two declined). Nurses in the control arm were recruited for voluntary participation in the pre- and post-intervention knowledge testing only (none declined) (Additional file: 3 Pre- and post- intervention test).
Allocation concealment and blinding
Allocation concealment was managed by an independent person (administrator, NL) drawing sealed lots, being blinded to the ward names on the lots and to outcome (intervention and control trial arms).
Patients were unaware of their allocation. Nurses who participated voluntarily in the training program and knowledge testing were not blinded to allocation, as the purpose of the training was to implement the MEWS chart.
Data collectors (JO and TW) were blinded to the allocation and implementation of the intervention. Data analysts for the knowledge tests (JO) and case notes (JJ and SJ) were blinded to the codes identifying trial arms. The first author (UK) was the Principal Investigator (PI) and was not blinded for two reasons: 1) the PI conducted nurses’ knowledge tests and trained nurses in the intervention wards in the use of the MEWS chart; and 2) the hospital database and operating room register identified patients by name and folder number, and both were searched manually for inclusion criteria for participants. Once identified, the PI requested case notes by name and folder number from the Records Department and signed a confidentiality clause. The CONSORT extended pragmatic checklist allows for an explanation of the absence of blinding.
The Cape Town modified early warning score training program
The Cape Town MEWS training program and manual  included basic physiology and literature on the advantages of using patients’ early warning scores as part of the handover process, thereby focusing shift handover meetings on patient safety . Three experts validated the content and face validity of the knowledge test, training manual, and power point presentation, using a checklist and scale of one to four to establish the Index of Content Validity (CVI) , giving a rating of three to four (high content-validity) for each. Recommended changes were made before administration.
Attendance at the two-hour interactive training sessions was dependent on staffing and workload as there were no resources to replace nurses on training. Wards released one to three nurses at a time, so five repeat training sessions were needed for day staff. Nurses on night duty were trained individually or in pairs when they were next on day duty. Nurses’ knowledge pre-and post-intervention was tested in all six wards (Additional file 3). Of the 11 official languages, English is used for recordings in patient notes, and was the medium of instruction and testing, although it is a second or third language for most nurses.
Staggered pre-intervention knowledge testing was conducted between 12 March and 23 March for nurses in both trial arms. The MEWS training program was conducted between 12 March and 22 April. The MEWS chart was introduced on 1 May and removed at midnight on 31 July. Post-intervention knowledge testing was conducted between 3 August and 13 August.
Implementing the Cape Town modified early warning score observation chart
While standard care (existing observations chart ) continued in three control wards, the Cape Town MEWS observations chart  (Additional file 1: Figure S1) was implemented in the three intervention wards. This MEWS chart indicates which scores should be reported, the degree of urgency needed, and where a total score should be entered.
During training, nurses were instructed to assign the correct score for each vital sign recorded, total the scores, and respond as follows: for a single parameter MEWS of one, to recheck the measurement after half an hour and report if there was no improvement; for a single parameter MEWS of two, to recheck the measurement after five minutes and report immediately if there was no improvement; for a critical single parameter MEWS of three, to report urgently; and for a total MEWS of three or higher, to report urgently. Senior ward nurses who had received training were recruited as MEWS Project Leaders, identifiable by a lanyard bearing a card with this title. Their task was to keep the project on track, order sufficient copies of the chart from the first author, and to ensure that new staff received training. Training manuals were available on all intervention wards. Patients’ records (n = 114) were reviewed two weeks after the completion of the study, on 31 July 2010. Data were captured electronically on an Excel criterion-based record review form. Researchers recoded vital signs from existing observations charts onto MEWS charts for all patients in the control arm and patients in the intervention arm where the MEWS charts were blank (36.8%, 21 out of 57). Data were examined for nurses’ responses to signs of disturbed physiology by reviewing patient progress notes for recorded interventions (above).
Primary outcome measure
The primary outcome measure was nurses’ documented responses to abnormal vital signs measures, as determined by rechecking vital signs or calling for assistance from a more senior nurse or medical doctor, for patients who met the pre-selected criteria for the intervention (above).
Secondary outcome measures
Secondary outcome measures were:
the number of patients with physiological variables (respiratory and heart rate, oxygen saturation, systolic blood pressure, temperature, level of consciousness, and urine output) recorded on the MEWS chart in intervention wards and on existing vital signs charts in control wards in the first eight postoperative hours;
the number of times each vital sign was recorded in the first eight postoperative hours in both trial arms; and
pre- and post-intervention test scores of nurses’ knowledge by blinded independent marking.
Statistical analyses were undertaken with IBM SPSS Statistics version 19 on an intention-to-treat basis . A per protocol analysis was undertaken  as a secondary sensitivity analysis . Records from the intervention arm where the MEWS was not used were excluded in the per protocol analysis. Tests of normality for distribution of data followed the convention of using the Shapiro-Wilk test for a sample size smaller than 50 , and we used the Kolmogorov-Smirnov test for a sample size greater than 50.
Bivariate associations were explored by comparisons of means or ranks (t tests, Mann-Whitney U, or equivalents for paired data), as appropriate. The χ2 test was used with Yates’ continuity correction for 2 × 2 tables. Fisher’s exact test was substituted for low numbers. Haldane’s correction was taken to calculate a risk estimate, when odds ratios could not be calculated. Statistical significance was set at P <0.05. Low numbers in outcome variables precluded adjusted analyses.
The study was approved by the University of Cape Town, Faculty of Health Sciences Human Research Ethics Committee (HREC REC reference 192/2009), the South African National Clinical Trial Registry (SANCTR DOH 27-0713-4483), and the Pan African Clinical Trial Registry (PACTR201309000626545), hospital management, and clinical structures. The South African National Health Research Ethics Committee (NHREC) was notified of the study in 2009 (application 2009/3483). The study was conducted according to the principles of the Declaration of Helsinki .
Written informed consent was obtained from nurse participants on intervention wards for the training program and knowledge testing, and on control wards for knowledge testing only . Cluster randomized controlled trials raise important ethical issues in relation to the nature and practice of informed consent [15,32]. When the MEWS observations chart was implemented on the three intervention wards, consent was not obtained from individual nurses  or patients, as the institutional consent (guardianship)  was considered to be the ‘community consent’  on behalf of individuals in the intervention clusters. The University of Cape Town, Faculty of Health Sciences Human Research Ethics Committee waived the need for written informed consent from individual patients whose records were reviewed. In terms of South African legislation at that time (National Health Act No. 61 of 2003, Section 16 (2))  a health care provider may examine a user’s health records for the purposes of research without authorization if the research will not obtain information relating to the identity of the user.
Of 1,427 patients undergoing surgery between 1 May and 31 July 2010 (Figure 1), 493 were aged 14 years or more and had received a general anaesthetic. The 1,427 included 11 patients who had died during this time (n = 8 in the intervention arm; n = 3 in the control arm). Of these 11, five had received a general anaesthetic, but records were unavailable for two of the five. The other six records were ineligible for analysis because three deaths occurred outside the wards and three patients who died were ‘not for resuscitation’. This left three records for analysis, all of which were from the same ward in the intervention arm. Figure 1 details the selection of records. Amongst the 482 survivors, 33 records were unavailable, six were incomplete, 0 patients were ‘not for resuscitation’ , and 0 patients were transferred to a high dependency unit (HDU) or ICU within eight hours of operation.
Baseline demographic and clinical characteristics of patients
Baseline demographic data and clinical characteristics of patients are presented in Tables 1 and 2, indicating that patients in trial arms were similar. The difference in proportion of patients undergoing different surgery reflected the wards’ surgical specialties (Table 1).
Nurses’ responses to high and low threshold modified early warning scores for vital sign recordings (intention-to-treat analysis)
Nurses had used the MEWS chart as a single parameter tracking tool; no-one had calculated a total MEWS for each observation time-point despite training. The numbers of abnormal vital signs recordings, summarized in Table 3, were similar in trial arms. Responses to MEWS triggers that should have been reported for patients who met pre-selected criteria for intervention are summarized in Table 4. Nurses in the intervention arm did not record responses to 121 out of 128 (94.5%) MEWS triggers that should have been reported for 50 out of 57 (87.7%) patients. Nurses in the control arm failed to record responses to 91 out of 93 (97.8%) deranged physiological parameters recoded into MEWS triggers that should have been reported for 55 out of 57 (96.5%) patients. These differences in response rates between the two trial arms did not reach statistical significance (odds ratio (OR): 2.63, 95% confidence interval (CI): 0.53 to 12.97) or the number of patients affected in each trial arm (OR: 0.26, 95% CI: 0.05 to 1.31). Overall, nurses in both trial arms reported nine out of 221 (4.1%) deranged physiological parameters for nine out of 105 (8.6%) patients who needed to be assessed. Nurses did not report two abnormal signs (respiratory rate, systolic blood pressure (BP), and/or heart rate) respectively for two of the three patients who died; the other patient had two out of two abnormal signs (heart rate and temperature) reported.
Patients with recorded postoperative parameters by trial arm
The MEWS chart was intended for all patients in the intervention arm but was only used with 63.2% (36 out of 57) patients. The numbers of patients with vital sign recordings within the first eight hours following surgery are shown in Table 5. All patients had recordings for blood pressure and heart rate. Patients in the intervention arm were more likely to have respiratory rate recorded (OR: 24.8, 95% CI: 5.50 to 111.32), and all seven vital signs recorded (Haldane’s estimator OR: 12.05, 95% CI: 0.650 to 223.19, P = 0.02).
Per protocol analysis of nurses’ responses to high and low threshold MEWS vital sign recordings
Per protocol analysis (Table 6) of nurses’ responses to MEWS triggers for 63.2% (36 out of 57) of patients in the intervention arm who had the MEWS chart showed that they did not record responses to 85 out of 88 (96.6%) MEWS triggers that should have been reported. Nurses in the control ward failed to record responses to 89 out of 90 (98.9%) deranged physiological parameters recoded into MEWS triggers that should have been reported for 57 patients. Only systolic BP recordings triggered reporting: three of 15 (20.0%) abnormal recordings using the MEWS and one of 25 (4.0%) recordings using the standard chart. These differences did not reach statistical significance. Overall, nurses in both trial arms reported a combined total of four out of 178 (2.2%) deranged physiological parameters for four out of 93 (4.3%) patients who needed to be assessed. Patients with the MEWS chart in the intervention arm were more likely to have recordings for respiratory rate (OR: 62.5, 95% CI: 12.89 to 303.15); oxygen saturation (OR: 5.5, 95% CI: 1.05 to 28.95), level of consciousness (OR: 5.95, 95% CI: 1.62 to 21.84), and all seven parameters (OR: 20.1, 95% CI: 1.08 to 375.09, using Haldane’s estimator).
Number of times each vital sign was recorded in the first eight postoperative hours by trial arm
The number of recordings (Table 7) was significantly different between the intervention and control arms for respiratory rate (Z = −5.42, P <0.001), heart rate (Z = −2.09, P = 0.036), systolic blood pressure (Z = −3.03, P = 0.002), temperature (Z = −2.742, P = 0.006), and level of consciousness (Z = −4.44, P <0.001). However, the rates of documented ‘abnormal’ vital signs (by MEWS criteria) were similar, and the incidence of serious adverse events (SAEs) was low.
The flow of nurses recruited for the training intervention on an intention-to-treat basis is shown in Figure 2 [16-18]. It was anecdotally reported, after the trial conclusions, that a small number of two to three nurses from control wards were working occasional overtime shifts in intervention wards. A total of 50 nurses participated in pre- and post-knowledge testing (25 in each arm) (Additional file 3). There were 13 Registered Professional Nurses (RPNs) in each trial arm, five Registered Staff Nurses (RSNs) in the intervention arm versus four in the control arm, and seven Registered Nursing Auxiliaries (RNAs) in the intervention arm versus eight in the control arm. One nurse in each of two intervention wards declined to participate in the study.
Comparison of pre- and post-intervention knowledge scores within and between trial arms
The mean difference (four out of 23, 19.5%) between pre- (10 out of 23, 41.9%) and post-intervention scores (14 out of 23, 61.4%) in the intervention arm reached statistical significance (paired t-value 3.8, 95% CI: −30.0 to 8.9, P = 0.001) but not in the control arm (Table 8). The independent t-test result (Table 9) indicated that improvement in knowledge scores between trial arms was significantly better in the intervention arm than the control arm (mean difference: 15.5% (three out of 23), 95% CI: 3.8 to 27.2, P = 0.01, t-value: 2.69 (35.9 degrees of freedom), equal variances not assumed).
To our knowledge, this is the first report of a randomized controlled trial of a MEWS. A pragmatic, cluster randomized, parallel group, controlled clinical trial of a training program and a MEWS chart for the detection of early signs of clinical deterioration made significant differences to knowledge, recording of respiratory rate, and recording of all seven physiological parameters during the first eight postoperative hours. However, there was little evidence that the intervention improved nurses’ responses to patients who triggered a critical MEWS reporting algorithm.
Difficulties with the conduct, analysis, and reporting of cluster trials have been reported  in both developed countries and sub-Saharan Africa [37,38]. Approval for implementation and testing of the MEWS in this study was limited to the period of study and the researchers had no authority to plan for its further use in practice thereafter. Although a cluster trial is intended to minimize contamination, professionals in the control arm may be sensitized to the problems under investigation by their involvement with researchers or communication with colleagues in the intervention arm . Any nurses from control wards, untrained in the MEWS, voluntarily working overtime shifts in intervention wards might have diluted the impact of the intervention ; this may account for some, but not all, of the non-use of 37% of MEWS charts and the improvement in nurses’ knowledge in control wards. Unusually for a randomized controlled trial, this work was conducted in a real-world setting in a resource-poor environment in a developing country. As in most cluster trials, we were unable to link individual nurses and patients, and most patients would have received care from several nurses. MEWS charts are not problem-free , and the complexity of introducing a EWS system and educational program suggest that pragmatic cluster randomized controlled trials or stepped wedge trials might be preferable  to patient randomized controlled trials [43-45].
Suboptimal compliance with the MEWS affected the trial; no scores were totaled. Complex systems such as EWS are prone to human calculation errors, when ‘pen and paper’ is used , rather than electronic systems . Not all patients in both trial arms had recordings of all seven vital signs. In our previous work  in the same six wards, results showed a similar pattern of recording of vital signs, and no patients’ records contained recordings for all seven parameters displayed on the MEWS. The absence of a total MEWS or a MEWS for one or more (of six) vital signs in 47% (n = 509) of patients in general surgical and medical wards in an observational data analysis pre- and post-intervention study in Belgium  resonates with our findings. There were differences in recordings between trial arms but similar rates of ‘abnormal’ vital signs (by MEWS criteria). The absence of catastrophic clinical consequences in both trial arms is noteworthy, but not unusual in our clinical areas , and we have no evidence that the MEWS improved clinical outcomes.
Our previous work  in the same wards indicated a similar uniform absence of responses to abnormal vital signs. Per protocol analyses indicated that, where used, the MEWS increased recordings of oxygen saturation and level of consciousness, in addition to respiratory rate. In both intention-to-treat and per protocol analyses, more patients in the intervention arm had abnormalities reported. However, differences did not reach statistical significance. Those that comply with trial protocols are not a random sample and may be different in some ways. Selectively excluding those who violate protocols may introduce bias, necessitating cautious interpretation of per protocol analyses. We acknowledge a paucity of conservatism in our sample size calculation, most particularly that we were unable to take account of clustering; however, data collection was limited by available resources. A larger sample is needed to test these findings. Our findings are presented without correction for multiple testing [48,49] in order to delineate leads for larger trials, and interpretation rests with readers.
Traditionally, nursing work that is not recorded is regarded as incomplete or even ‘not done’. However, the absence of recorded interventions for patients meeting pre-determined criteria for intervention, although of deep concern, does not mean necessarily that there were no interventions. Nevertheless, without a documented record, nurses’ reporting is incomplete, and counted as such in this study. From single site research we cannot assume that respondents and response patterns are representative of other populations. The trial location may have influenced recruitment, retention, and clinical outcomes. A new vital signs chart may improve recording due to increased interest [9,50].
No nurse observation protocol existed at our research site. Intervals between manual monitoring of vital signs on general wards vary substantially, for example between eight and 12 hours . Our study might have been strengthened by implementing and evaluating compliance with a standard protocol for frequency of physiological observations of at least every 12 hours for every patient, as recommended in the United Kingdom’s National Institute for Health and Care Excellence (NICE) Clinical Practice Guideline 50 . There were few deaths and too many confounding factors in a clinical setting to attribute these outcomes to inadequate recording and reporting. A much larger sample would be needed to examine the effect of the MEWS on SAEs.
The realities of practice
Incomplete recording of all seven vital signs in the postoperative period was high in the intervention arm and worse in the control arm. In the control arm, poor recording might be attributed to the absence of guidelines for appropriate frequency  and number of vital signs to be recorded. Not all MEWS charts include monitoring of urine output . Poor recording of oxygen saturation following the administration of a general anaesthetic and opioids  is of concern.
The improvement in scores between pre- and post-intervention knowledge tests of nurses in the intervention arm from 41.9 to 61.4% did not translate into improved documented reporting of patients who met the pre-determined criteria for intervention. The MEWS does not replace clinical judgment in detecting deteriorating patients . Even so, nurses failed to report deranged physiological parameters for 87.7% (intervention) and 96.5% (control) patients. This is deeply concerning and raises questions about the specificity of the MEWS to identify ‘at risk’ patients. However, in our previous work  in the same wards, there were no reported interventions for 10 out of 11 (90.9%) patients who died and had abnormal physiology, and for 38 (86.4%) control patients with abnormal physiology. The specificity of the Cape Town MEWS had been determined at between 77.3 and 81.4% . Other studies also report nurses’ failure to communicate concerns and inappropriate responses when patients showed signs of physiological deterioration [3,9,54]. There are too many confounding factors in a clinical setting to attribute these outcomes to inadequate recording and reporting. Strategies for implementing and monitoring patient safety initiatives in large tertiary hospitals need to accommodate the complexity of such organizations. Patients admitted to this level of care have complex conditions requiring treatment not available at secondary level hospitals, increasing their risk of adverse events. Other factors have to be considered.
Problems arise when nurses are competent in using technology for the monitoring of vital sign but lack clinical knowledge in interpreting data and intervening appropriately to ensure optimum and safe patient care. Misinterpretation of clinical data is associated with poor clinical reasoning skills and, in studies using computer-based clinical scenarios, nurses have been found to overestimate risks and the need to intervene . An Australian survey found that nurses may not always follow predetermined medical emergency team (MET) calling criteria and may not recognize when assistance is required .
Poor recording of vital signs and little evidence of appropriate responses to signs of physiological deterioration may be linked to ‘task allocation’, the method of patient care employed in the research wards, in preference to ‘patient allocation’. It is standard practice in South African wards to delegate monitoring, recording, and interpretation of patient observations to nursing auxiliaries and enrolled nurses, and the quality of recording should be viewed within this context. The core function of the nurse in avoiding SAEs should go beyond the recording of patients’ physiological vital signs . It is the nurse’s professional responsibility to understand the significance of patient observations [58,59].
Benefits of the intervention
Disappointingly, the improved knowledge scores only reached 61.4%, and may have been impacted by language difficulties and the inclusion of non-professional nurses who have a non-academic qualification. The MEWS chart resulted in significantly more patients in the intervention wards than control wards having recordings of respiratory rate, the best discriminator for clinical outcomes . Not all existing observation charts  included respiratory rate monitoring. This trial showed that, if included, respiratory rate is more likely to be monitored. As respiratory rate monitoring is the best discriminator of SAEs, it has replaced recording of oxygen saturation on some charts . Nevertheless, the improved scores suggest that a hospital-wide training program for early recognition and management of patients with impending critical illness is needed, in order to complement in-service education programs for late rescue techniques, such as cardiopulmonary resuscitation (CPR). Low pre-intervention bioscience knowledge scores in both trial arms confirm previous research  that suggests that this underpinning knowledge may be suboptimal.
Improved recording did not translate into an improvement in responses to deranged physiology. The Cape Town ward MEWS has a wider range of cut points (thresholds) for respiratory rate, oxygen saturation, heart rate, systolic BP, and temperature than other published MEWS. This might have made interpretation of parameter readings more difficult, and may have accounted for the low response rate to the large number of triggers, despite training. This raises questions regarding the clinical effectiveness of the MEWS chart deployed to identify patients at risk and the user friendliness of the chart. To address the latter possibility, a revised version of the Cape Town ward MEWS observations chart (Additional file 4: Figure S2: Revised Cape Town MEWS chart) is presented. The United Kingdom National Early Warning Score (NEWS)  was not available when the study was undertaken. Since completion of this trial, we have substituted the values from the NEWS into the Cape Town ward MEWS, with permission from the United Kingdom Royal College of Physicians, as the NEWS values retain the range of trigger cut points (thresholds) (heart rate: 111 to 129; systolic BP: 81 to 100) we found to be associated with mortality . User friendliness of the chart might be improved by reducing the number of cut points (thresholds) for physiological parameters that were not associated with mortality. To assist those responding to patients’ physiological deterioration, the United Kingdom NEWS algorithm for a total MEWS  has been included on the revised chart, in addition to guidelines for reporting a single parameter trigger.
The association between recordings of vital sign parameters (including respiratory rates) and mortality [61-63] challenges traditional assumptions that mortality outcomes and determinants of survival fall solely within the domain of medical care, and provides further evidence that these outcomes are ‘nursing sensitive’ . Nurses are concerned that patients on general wards are having increasingly more complex surgery, increasing their dependence and morbidity which, in the face of understaffing, results in increased workload and suboptimal quality of care, leaving less time to apply learning in practice . Specialist nurses, increased registered nurse (RN)-to-patient ratios and a richer RN skill mix (more RNs than other categories of nurses), is inversely related to hospital mortality rates (n = 10,184 nurses in 168 hospitals)  and to most adverse events (n = 124,204 patients) . These results should be generalized cautiously as hospital characteristics vary . A total of 19 of 27 studies that were systematically reviewed found an association between one or more unfavorable nursing environmental attributes and higher mortality. Despite extensive variability in attribute and outcome measures, settings, and research quality across studies, there is evidence that social and environmental characteristics of hospital nursing practice affect the outcomes of care, but more research is needed to link the nursing environment, including patient monitoring, to patient outcomes .
The monitoring of vital signs should be central to patient safety initiatives and hospital-wide morbidity and mortality reviews. Recording and analyzing trends in the monitoring of vital signs may alert managers to changing standards of care. The Cape Town MEWS observations chart as a combined (aggregated and single parameter) track-and-trigger tool  has been revised to indicate more clearly when abnormalities should be reported, and requires further testing. We recommend improved training for appropriate reporting of deranged physiology.
Acute Life-threatening Events - Recognition and Treatment
- EWS system:
Early warning scoring system
Early warning score
High dependency unit
Human Research Ethics Committee
Intensive Care Unit
- MEWS system:
Modified early warning scoring system
Modified early warning score
National Early Warning Score
National Institute for Health and Care Excellence
National Health Research Ethics Committee
Registered Professional Nurse
Registered Staff Nurse
Registered Nursing Auxiliary
Serious adverse event
Safer Patients Initiative
Zimlichman E, Szyper-Kravitz M, Unterman A, Goldman A, Levkovich S, Shoenfeld Y. How is my patient doing? evaluating hospitalized patients using continuous vital signs monitoring. Israeli Med Assoc J. 2009;11(June):382–4.
Rattray JE, Lauder W, Ludwick R, Johnstone C, Zeller R, Winchell J, et al. Indicators of acute deterioration in adult patients nursed in acute wards: a factorial survey. J Clin Nurs. 2011;20(5/6):723–32.
Thomson R, Luettel D, Healey F. Safer care for the acutely ill patient: learning from serious incident. United Kingdom: NHS National Patient Safety Agency; 2007. Report No.: 0559. http://www.npsa.nhs.uk/patientsafety/alerts-and-directives/directives-guidance/acutely-ill-patient/. Accessed 8 March 2015.
Subbe CP, Davies RG, Williams E, Rutherford P, Gemmell L. Effect of introducing the Modified Early Warning score on clinical outcomes, cardio-pulmonary arrests and intensive care utilisation in acute medical admissions. Anaesthesia. 2003;58:797–802.
Smith GB. In-hospital cardiac arrest: is it time for an in-hospital ‘chain of prevention’? Resuscitation. 2010;81(9):1209–11.
Smith GB, Osgood VM, Crane S. ALERT – a multiprofessional training course in the care of the acutely ill adult patient. Resuscitation. 2002;52(3):281–6.
Kyriacos U, Jordan S, van den Heever J. The biological sciences in nursing: a developing country perspective. J Adv Nurs. 2005;52(1):91–103.
Tourangeau AE. A theoretical model of the determinants of mortality. Adv Nurs Sci. 2005;28(1):58–69.
Robb G, Seddon M. A multi-faceted approach to the physiologically unstable patient. Qual Saf Health Care. 2010;19:e47.
Van Leuvan C, Mitchell I. Missed opportunities? An observational study of vital sign measurements. Crit Care Resusc. 2008;10(2):111–5.
Chatterjee MT, Moon JC, Murphy R, McCrea D. The “OBS” chart: an evidence based approach to re-design of the patient observation chart in a district general hospital setting. Postgrad Med J. 2005;81(960):663–6.
Kyriacos U, Jelsma J, James M, Jordan S. Monitoring vital signs: development of a modified early warning scoring (MEWS) system for general wards in a developing country. PLoS One. 2014;9(1):e87073.
Kyriacos U, Jelsma J, Jordan S. Record review to explore the adequacy of post-operative vital signs monitoring using a local modified early warning score (mews) chart to evaluate outcomes. PLoS One. 2014;9(1):e87320.
Zwarenstein M, Treweek S, Gagnier J, Altman D, Tunis S, Haynes B, et al. Pragmatic Trials in Healthcare (Practihc) group. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008;11(337):a2390.
Hutton JL. Are distinctive ethical principles required for cluster randomized controlled trials? Stat Med. 2001;20:473–88.
Moher D, Schultz K, Altman D, The Consort Group. The CONSORT statement: revised recommendation for improving the quality of reports of parallel-group randomised trials. Lancet. 2001;357:1191–4.
Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al.. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340.
Schulz K, Altman D, Moher D, the CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. Trials. 2010;11(1):32.
Bristow PJ, Hillman KM, Chey T, Daffurn K, Jacques TC, Norman SL, et al. Rates of in-hospital arrests, deaths and intensive care admissions: the effect of a medical emergency team. Med J Aust. 2000;173(5):236–40.
Benning A, Ghaleb M, Suokas A, Dixon-Woods M, Dawson J, Barber N, et al. Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ. 2011;342(d195):53–67.
Kyriacos U. The development, validation and testing of a vital signs monitoring tool for early identification of deterioration in adult surgical patients, PhD thesis. Cape Town: University of Cape Town; 2011.
Uitenbroek, Daan G, Binomial. SISA. 1997. http://www.quantitativeskills.com/sisa/distributions/binomial.htm. (1 Jan 2014). Accessed 8 March 2015.
Higgins JPT, Green S (eds.): Cochrane handbook for systematic reviews of interventions. Version 5.1.0; updated March 2011. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org. Accessed 10 March 2015.
Boulanger C, Toghill M. How to measure and record vital signs to ensure detection of deteriorating patients. Nurs Times. 2009;105(47):10–2.
Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6 November/December):382–5.
Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet. 2005;365(9477):2091–7.
Sussman JB, Hayward RA. Using instrumental variables to adjust for treatment contamination in randomised controlled trials. BMJ. 2010;340(c2073):1181–4.
Nϋesch E, Trelle S, Reichenbach S, Rutjes AWS, Bϋrgi E, Scherer M, et al. The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study. BMJ. 2009;339:b3244.
Razalt NM, Wah YB. Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests. Journal of Statistical Modeling and Analytics. 2011;2(1):21–33.
World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects. http://www.wma.net/en/30publications/10policies/b3/17c.pdf. Accessed 9 March 2015.
Eldridge S, Kerry S, Torgerson DJ. Bias in identifying and recruiting participants in cluster randomised trials: what can be done? BMJ. 2010;340:b4006.
McRae AD, Weijer C, Binik A, Grimshaw JM, Boruch R, Brehaut JC, et al. When is informed consent required in cluster randomized trials in health research? Trials. 2011;12:202.
Brierley G, Brabyn S, Torgerson D, Watson J. Bias in recruitment to cluster randomized trials: a review of recent publications. J Eval Clin Pract. 2012;18(4):878–86.
Edwards S, Braunholtz D, Lilford R, Stevens A. Ethical issues in the design and conduct of cluster randomised controlled trials. BMJ. 1999;318(7195):1407–9.
South African Government. National Health Act. vol. 26595, 23 July 2004. South Africa: Government Gazette; 2003: s. 48.
Campbell M, Elbourne D, Altman D. CONSORT statement: extension to cluster randomised trials. BMJ. 2004;328:702–8.
Isaakidis P, Ioannidis JPA. Evaluation of cluster randomized controlled trials in Sub-Saharan Africa. Am J Epidemiol. 2003;158(9):921–6.
MacLennan G, Ramsay C, Mollison J, Campbell M, Grimshaw J, Thomas R. Room for improvement in the reporting of cluster randomised trials in behaviour change research [abstract]. Control Clin Trials. 2003;24:69S–70.
Fletcher SJ, Cuthbertson BH. Outreach, epistemology and the evolution of critical care. Anaesthesia. 2010;65(2):115–8.
Keogh-Brown MR, Bachmann MO, Shepstone L, Hewitt C, Howe A, Ramsay CR, et al. Contamination in trials of educational interventions. Health Technol Assess. 2007;11(43 Oct):iii. ix–107.
Kyriacos U, Jelsma J, Jordan S. Monitoring vital signs using early warning scoring systems: a review of the literature. J Nurs Manag. 2011;19(3):311–30.
Brown C, Lilford R. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6(54):1–9.
Quarterman CP, Thomas AN, McKenna M, McNamee R. Use of a patient information system to audit the introduction of modified early warning scoring. J Eval Clin Pract. 2005;11(2):133–8.
Grypdonck MHF. Qualitative health research in the era of evidence-based practice. Qual Health Res. 2006;16(10):1371–85.
Hudson K, Duke G, Haas B, Varnell G. Navigating the evidence-based practice maze. J Nurs Manag. 2008;16(4):409–16.
Schmidt PE, Meredith P, Prytherch DR, Watson D, Watson V, Killen RM, et al. Impact of introducing an electronic physiological surveillance system on hospital mortality. BMJ Qual Saf. 2015;24(1):10–20.
De Meester K, Das T, Hellemans K, Verbrugghe W, Jorens PG, Verpooten GA, et al. Impact of a standardized nurse observation protocol including MEWS after Intensive Care Unit discharge. Resuscitation. 2013;84(2):184–8.
Streiner D, Norman G. Correction for multiple testing: is there a resolution? Chest. 2011;140(1):16–8.
Rothman K. No adjustments are needed for multiple comparisons. Epidemiology. 1990;1:43–6.
McBride J, Knight D, Piper J, Smith GB. Long-term effect of introducing an early warning score on respiratory rate charting on general wards. Resuscitation. 2005;65:41–4.
Bates DW, Zimlichman E. Finding patients before they crash: the next major opportunity to improve patient safety. BMJ Qual Saf. 2015;24(1):1–3.
National Institute for Health and Care Excellence. Acutely ill patients in hospital: recognition of and response to acute illness in adults in hospital. United Kingdom: NICE clinical guideline 50; 2007. http://www.nice.org.uk/guidance/cg50. Accessed 10 March 2015.
Jordan S. The Prescription Drug Guide for Nurses. Berkshire, England: Open University Press; 2008.
Luettel D, Beaumont K, Healey F. Recognising and responding appropriately to early signs of deterioration in hospitalised patients. United Kingdom: NHS National Patient Safety Agency; 2007. Report No.: 0683. http://www.npsa.nhs.uk/patientsafety/alerts-and-directives/directives-guidance/acutely-ill-patient/deterioration-in-hospitalised-patients/. Accessed 8 March 2015.
Thompson C, Bucknall T, Estabrookes CA, Huthinson A, Fraser K, de Vos R, et al. Nurses’ critical event risk assessments: a judgement analysis. J Clin Nurs. 2009;18(4):601–12.
Crispin C, Daffurn K. Nurses’ responses to acute severe illness. Aust Crit Care. 1998;11(4):131–3.
Smith S, Fraser J, Plowright C, Dennington L, Seymour P, Oliver G, et al. An audit of nursing observations on ward patients. Nurs Times. 2008;104(30):28–9. http://www.nursingtimes.net/an-audit-of-nursing-observations-on-ward-patients/1744564.article.
Kisiel M, Perkins C. Nursing observations: knowledge to help prevent critical illness. Br J Nurs. 2006;15(19):1052–6.
Hogan J. Why don’t nurses monitor the respiratory rates of patients? Br J Nurs. 2006;15(9):489–92.
Royal College of Physicians. National Early Warning Score (NEWS): standardising the assessment of acute-illness severity in the NHS. In: Report of a working party. London: Royal College of Physicians; 2012.
Gao H, McDonnell A, Harrison DA, Moore T, Adam S, Daly K, et al. Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward. Intensive Care Med. 2007;33(4):667–79.
Hartman ND, Mehring BB, Brady WJ. Clinical predictors of physiological deterioration and subsequent cardiorespiratory arrest among hospitalized patients. In: Vincent J-L, editor. Intensive Care Medicine Annual Update 2006. New York: Springer Science and Business Media Inc; 2006. p. 309–15.
Harrison GA, Jacques T, McLaws ML, Kilborn G. Combinations of early signs of critical illness predict in-hospital death-the SOCCER study (signs of critical conditions and emergency responses). Resuscitation. 2006;71(3):327–34.
Cutler LR. From ward-based critical care to educational curriculum 2: a focussed ethnographic case study. Intensive Crit Care Nurs. 2002;18(5):280–91.
Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002;288(16):1987–93.
Cho S, Ketefian S, Barkauskas VH, Smith DG. The effects of nurse staffing on adverse events, morbidity, mortality, and medical costs. Nurs Res. 2003;52(2):71–9.
Stanton MW, Rutherford MK. Hospital nurse staffing and quality of care. Rockville (MD): Agency for Health Research and Healthcare Quality; 2004.
Ae K, Green C, Wong J, Reid R. Effect of the hospital nursing environment on patient mortality: a systematic review. J Health Serv Res Policy. 2005;10:111–7.
The authors acknowledge the experts who validated the training program and materials. We are indebted to researcher assistants Terry Wulff, for recruiting participants for the training program, and Johann Olivier, for independent blind marking of participants’ knowledge tests, and to both for assisting with record review. We thank Naomi Lenders for assisting with allocation concealment.
Source of funding
University of Cape Town Research Development Grants including a Carnegie PhD Award, the Department of Health and Rehabilitation Sciences Research Committee Award and the Faculty of Health Sciences Research Committee Award.
The authors declare that they have no competing interests.
Conceived and designed the experiments: UK in consultation with JJ, MJ, and SJ. Performed the experiments: UK. Analyzed the data: UK, JJ, and SJ. Wrote the paper: UK. Revised and critically reviewed the article for important intellectual content and gave final approval for the manuscript: SJ, JJ, and MJ. All authors read and approved the final manuscript.
Consensus derived Cape Town MEWS observations chart and reporting algorithm.
CONSORT Statement extended to pragmatic cluster trials: checklist.
Pre- and post-intervention test.
Revised Cape Town MEWS chart.
About this article
Cite this article
Kyriacos, U., Jelsma, J., James, M. et al. Early warning scoring systems versus standard observations charts for wards in South Africa: a cluster randomized controlled trial. Trials 16, 103 (2015). https://doi.org/10.1186/s13063-015-0624-2
- Cluster randomized trial
- Developing country
- In-service training
- Monitoring (physiological)
- Pragmatic randomized controlled trial
- Vital signs
- Scoring systems (MeSH checked 3 April 2013)