- Open Access
Testing effectiveness of the revised Cape Town modified early warning and SBAR systems: a pilot pragmatic parallel group randomised controlled trial
Trials volume 20, Article number: 809 (2019)
Nurses’ recognition of clinical deterioration is crucial for patient survival. Evidence for the effectiveness of modified early warning scores (MEWS) is derived from large observation studies in developed countries.
We tested the effectiveness of the paper-based Cape Town (CT) MEWS vital signs observation chart and situation-background-assessment-recommendation (SBAR) communication guide. Outcomes were: proportion of appropriate responses to deterioration, differences in recording of clinical parameters and serious adverse events (SAEs) in intervention and control trial arms. Public teaching hospitals for adult patients in Cape Town were randomised to implementation of the CT MEWS/SBAR guide or usual care (observation chart without track-and-trigger information) for 31 days on general medical and surgical wards. Nurses in intervention wards received training, as they had no prior knowledge of early warning systems. Identification and reporting of patient deterioration in intervention and control wards were compared. In the intervention arm, 24 day-shift and 23 night-shift nurses received training. Clinical records were reviewed retrospectively at trial end. Only records of patients who had given signed consent were reviewed.
We recruited two of six CT general hospitals. We consented 363 patients and analysed 292 (80.4%) patient records (n = 150, 51.4% intervention, n = 142, 48.6% control arm). Assistance was summoned for fewer patients with abnormal vital signs in the intervention arm (2/45, 4.4% versus (vs) 11/81, 13.6%, OR 0.29 (0.06–1.39)), particularly low systolic blood pressure. There was a significant difference in recording between trial arms for parameters listed on the MEWS chart but omitted from the standard observations chart: oxygen saturation, level of consciousness, pallor/cyanosis, pain, sweating, wound oozing, pedal pulses, glucose concentration, haemoglobin concentration, and “looks unwell”. SBAR was used twice. There was no statistically significant difference in SAEs (5/150, 3.3% vs 3/143, 2.1% P = 0.72, OR 1.61 (0.38–6.86)).
The revised CT MEWS observations chart improved recording of certain parameters, but did not improve nurses’ ability to identify early signs of clinical deterioration and to summon assistance. Recruitment of only two hospitals and exclusion of patients too ill to consent limits generalisation of results. Further work is needed on educational preparation for the CT MEWS/SBAR and its impact on nurses’ reporting behaviour.
Nurses are responsible for monitoring patients for signs of clinical deterioration. These might be most effectively detected using bedside early warning scoring (EWS) systems and a standardised reporting system such as the situation-background-assessment-recommendation (SBAR) guide. In 2010 we conducted our first pragmatic, cluster randomised controlled trial (RCT) of the original, consensus-derived, Cape Town (CT) modified EWS (MEWS) observations chart in three intervention and three control adult surgical wards in a single research site [1,2,3]. At the time, electronic track-and-trigger MEWS and reporting systems were increasingly widely used in healthcare settings in developed countries [4,5,6], whereas pen-and-paper systems were slowly being implemented in developing countries [1, 7]. As in our trial, government sector hospital wards in low-income and middle-income countries in Africa used pen-and-paper graphic charts to plot temperature, blood pressure, heart and respiratory rates, without information on normal or expected values . Paper charting of observations takes longer than electronic systems, increasing nursing workload  and risk of human error , particularly inaccuracy of calculations .
From our trial, we concluded that the original CT MEWS chart and training programme enhanced recording of respiratory rate and all parameters, and nurses’ knowledge, but not nurses’ responses to patients who triggered the MEWS reporting algorithm. At the conclusion of the trial we were required to withdraw the CT MEWS because this was an independent rather than a government-endorsed study. Nevertheless, we were inspired by the Royal College of Physicians’ National Early Warning Score (NEWS)  to revise the CT MEWS chart under the Creative Commons Attribution-No Derivatives 4.0 International Licence (Additional file 1), and to undertake a further RCT.
Changes in clinical parameters, such as pallor and reduced level of consciousness, and abnormal physiological vital signs herald patient deterioration, often 6–8 h before the onset of a serious adverse event (SAE) . To help nurses recognise deterioration, physiological values are recorded on MEWS charts within partitioned coloured bands (cut points or thresholds) with corresponding colour-coded weighted trigger points (scores) (0, upper and lower 1–3) (Additional file 1). If nurses are not adequately trained in recognition of deterioration, absence of trigger points may restrict nurses’ capacity to summon clinical assistance when needed.
Educational programmes normally accompany implementation of EWS/MEWS . A systematic review reported effectiveness of educational interventions (mainly simulation) in recognition and management of deteriorating patients, improved patient outcomes and organisational systems. The review identified 20 quantitative studies, 2 mixed methods studies and 1 qualitative study (total n = 23): 11 of 20 (55.0%) studies indicated a statistically significant improvement in knowledge, clinical performance, use of medical emergency teams (METs), frequency of observations, and patient outcomes . Improvement in confidence in approaching senior staff for advice and working as a team were also reported .
A systematic review of 11 articles (10 studies) on the effect of EWS systems training reported a statistically significant improvement in either nurses’ knowledge or confidence or clinical performance . All studies (but one) were conducted in countries with high human-development indices .
It is not surprising, therefore, that we were unable to locate reports of the introduction of EWS in government sector hospitals in developing countries. At the inception of this study, the public healthcare sector in South Africa had commenced transitioning to a National Health Insurance (NHI) system . The outcome is a health financing system designed to pool funds and actively purchase services to provide access to high quality, affordable personal health services for all South Africans, irrespective of socioeconomic status. The final phase is scheduled for 2022. A pen-and-paper recording system will make it difficult to achieve objective evaluation of the quality of nursing care: like all evaluations, this will require data capture, storage and analysis to guide a standardised approach to clinical care .
In the present study, we aimed to test the effectiveness of the revised instrument (Additional file 1) and a locally validated MEWS-linked SBAR guide (DB) (Additional file 2) on hospital nurses’ performance in early identification and paper reporting of clinical and physiological deterioration in adult patients in public sector teaching hospitals in Cape Town, South Africa. As in our first RCT of the original CT MEWS chart, the second RCT of the revised MEWS was conducted de novo without the nurses having any prior knowledge of EWS for monitoring patients’ vital signs. To our knowledge this is the first multi-site RCT on the effectiveness of a MEWS vital signs observations chart for early recognition and response to signs of deterioration .
This was a pilot, pragmatic, parallel-group RCT with two arms (intervention versus usual care). Pragmatic clinical trials (PCTs) are randomised trials designed to compare the effectiveness of interventions in real-world settings  for the primary purpose of informing decision-makers of the balance of benefits, burdens and risks . Here, hospitals were the unit of randomization, that is, random allocation of all participating general medical and surgical wards in the hospital to the same trial arm. There was contamination bias in our previous single-site trial due to nurses working across wards .
We regarded our study as being of minimal risk, limited to “interventions already used in routine clinical care” (:486). The local University of Cate Town (UCT) Clinical Research Centre (CRC) was the trial sponsor. The trial protocol received ethical approval from the Local Human Research Ethics Committee (HREC 337/2014), the local Provincial Department of Health (2014RP069), and the National Department of Health (DOH-27-0614-4779). They determined that patients would be recruited by informed, written consent for use of the revised MEWS chart and record review (after the intervention). In the earlier trial (HREC 192/2009) , individual patient consent for these processes had been deemed unnecessary.
In this second RCT, recruitment by individual patient consent soon proved impractical and hindered the delivery of the trial. We then submitted a revised protocol, requesting a waiver of individual written consent. Subsequent revised HREC approval (Ref. 825/2014) required written permission from gatekeepers for such a waiver and hospital notices to this effect in A3 size. This was not acceptable to some hospitals. For example, one academic tertiary hospital declined participation, stating that the display of A3 posters in research wards, replacing individual patient consent, would breach their value framework and might result in family or staff unhappiness or public media inquiries or complaints. We then used the earlier approval (HREC 337/2014), requiring individual patient consent, without posters.
All six acute, government-sector hospitals for adult patients in the Western Cape, South Africa were matched for level of acuity and patient numbers (three pairs) within a predetermined distance range (124 km) from UCT, and approached (recruitment success is described under “Results”). Previously, in our first single-site study , we observed that some staff worked across several wards in their hospitals, often as relief or bank nurses. Therefore, to limit contamination in this RCT, each hospital was either an intervention or a control site. Before the study we obtained, with permission, encrypted, fully anonymised data from the Western Cape provincial Department of Health database for February 2014 (admission numbers, death rate, length of stay). We compared data to ensure that we were comparing wards and hospitals with similar reporting levels for these parameters. The trial was conducted on general medical and surgical wards. Specialist units such as intensive care were outside the scope of the study. Data were obtained by retrospective review of patient records for both trial arms, following the educational intervention and trial use of the MEWS chart and SBAR guide in the intervention arm.
Screening and randomisation
The Swansea Clinical Trials Unit randomly allocated hospitals electronically to either the intervention or control trial arm before commencement of the study. As hospitals became available they were randomised in pairs (pre-matched by acuity and type, as above). There were no statistically significant differences in mortality rates between the hospitals paired for randomisation. General medical and surgical wards were matched, resulting in the selection of two matched surgical and one medical ward in each hospital allocated to their hospital’s trial arm. The process of consenting nurses to training or no training in the respective trial arms (hospitals) precluded blinding.
The primary outcome was nurses’ reporting behaviour, defined as proportion of responses to early signs of clinical and physiological deterioration judged as appropriate, using the CT MEWS criteria for critical deterioration labelled as “red MEWS” and scoring a 3. Secondary outcomes were the differences between intervention and control arms in (1) number of patients with recordings in the order that these appear on the MEWS chart (respiratory rate, oxygen saturation, temperature, heart rate, systolic blood pressure (BP), level of consciousness, urine output), (2) recordings of all seven vital signs, (3) nurses’ responses to MEWS, and (4) the proportion of patients who developed serious adverse events (SAEs) . We recorded patients’ demographic and clinical characteristics, and prescription of opiates, sedatives, diuretics and anti-hypertensives received on the day of vital sign recordings.
Patient records for retrospective review
We aimed to evaluate effectiveness of the revised MEWS chart by reviewing records of consented patients aged ≥ 18 years, admitted in August 2014 to the study wards, with the following exclusions: noted as “Not for Resuscitation”, transferred out of the ward to another department within 12 h following admission, or pregnancy. Clinical records of consented patients were excluded if:
Unavailable: in the medico-legal department (for example after death); in the x-ray department; removed for teaching purposes or transcription; in active use in association with re-admission of the patient; or
Incomplete: without a vital signs chart (in the intervention wards, the MEWS chart) or the nursing record/progress notes; or
Missing for unknown reasons.
Patients with multiple drug-resistant pulmonary tuberculosis were included only after data collectors were fitted with the correct masks.
Sample size for record review
Our original sample size calculation was based on our previous RCT of the CT MEWS (Trial registration PACTR201309000626545) , which detected a difference of 9% between the two arms in the numbers of patients whose abnormal vital signs triggered calls for assistance (7/53, 13.2% vs 2/52, 3.8%). An effective sample of 300 records (150 from each arm) would have been sufficient to detect a difference of 9% (4–13% increase) between arms in summoning appropriate assistance with 80% power and 5% significance. This calculation took no account of clustering [20, 21]. The intra-cluster correlation coefficient for numbers of patients for whom responses were triggered appropriately was calculated to be 0.02 (based on comparison of variance between and within clusters) . With 12 wards potentially available and 51 patients in each ward, the design effect is 2.0. Therefore, to give an effective sample size of 300, 612 patients had to be recruited, 51 in each of 12 wards .
Ward nurses’ training and implementation of MEWS and SBAR
All nurses in full-time employment in the three intervention wards (31 nurses on day floor duty plus 17 on night duty, n = 48), were eligible for participation. There were 26 beds in each of two wards, and 30 in the third ward (n = 82 beds). The 31 “day staff” across the three wards comprised 9, 10 and 12 nurses, respectively, split into two teams, working on different days of the week. External agency nurses were hired (4 for day duty and up to 10 for night duty depending on patient load) to cover staffing deficits. Agency nurses on day duty did not receive training, even though they would be using the research instruments, because they were allocated to different hospitals from day to day.
The eight hospital nurse managers, who did not do floor duty, did not participate in the training sessions. We were given a training schedule allowing 24/31 (77.4%) ward nurses on day floor duty in the three intervention wards to attend on-site training; 3 clinical nurse ward operational managers and 4 ward nurses were excluded from the training schedule. No reasons were given for their exclusion. Six groups of 3–5 informed, consenting nurses per session were trained (n = 24: 8 Registered Professional Nurses (RPNs), 3 RPNs on a compulsory 2-year community service post-training programme, 6 Enrolled (Staff) Nurses and 7 Nursing Auxiliaries).
Informed, consented nurses on night duty (n = 23) from the three intervention wards were trained on site in the evenings over the same 6 days. Of these, 17 were in full-time employment and 6 were external agency nurses consistently stationed on night duty in the intervention wards.
Intervention (1): training programme for intervention wards
An interactive 8-h classroom training programme on the CT Mews chart and MEWS-linked SBAR communication guide was reduced by hospital managers to 2 h face-to-face teaching. Day duty nurses were trained over 6 days in July 2014, a month before testing the instruments (1–31 August 2014). The training programme (by UK, DB) consisted of a MEWS and SBAR pocket guide, revision of basic anatomy and physiology, followed by recording vital signs and clinical data on blank MEWS charts from hypothetical patient scenarios (DB). Data on MEWS charts were then transcribed onto SBAR forms and used for role play for calling the doctor. All nurses were offered additional training for up to 8 h but no-one accepted. Overall, 24/31 (77.4%) day staff and all the night staff received training. In the control arm, nurses were informed and consented to record review, but received no training. Nurses in both trial arms gave written informed consent to participate. We would have excluded those unwilling to give signed, informed consent, but this was unnecessary, as all those approached gave written consent to participate.
Intervention (2): 31-day use of MEWS/SBAR on intervention wards
In both trial arms, we presented the study to senior nurses, doctors and hospital managers. Nurses in both trial arms received a written and verbal explanation in English, the medium of instruction in local nursing educational institutions. In the intervention wards, nurses, managers and doctors agreed to use the CT MEWS chart and SBAR guide for monitoring patients’ vital signs for 1 month. In control wards, nurses used the usual observation chart (Additional file 3), and, if a patient’s condition deteriorated, they followed their usual procedure for calling for help. When patients were discharged, researchers reviewed patients’ charts for the quality of vital signs recordings in both trial arms.
Neither the MEWS chart nor the SBAR guide were in use in government sector hospitals at the time of the trial nor are they currently in use, so the concepts were new to the nurses. The MEWS chart replaced the existing observation chart (Additional file 3) for consented patients for the duration of the 31-day trial. A condition of ethical approval of the study by the Western Cape Department of Health was that hospital staff would not be burdened with the process of obtaining consent from patients. During the trial, two researchers (UK, DB) interviewed patients on 11 occasions on the three intervention wards.
On these occasions, written and verbal explanations of the study were given to eligible patients in any of three languages (English, Afrikaans and isiXhosa). For cultural reasons, some patients requested time for discussion with their families before giving written consent. If patients were illiterate or if they customarily deferred to health professionals, the researchers or research assistants fluent in their language read, translated and explained the consent form. In this way, we took care to ensure that patients understood the implications of their signed consent. Since most patients were unfamiliar with the various forms of nursing documentation, this involved explaining that researchers wanted to see the quality of recordings of their blood pressure for example, and nurses’ responses to any abnormal recordings. Signed consent or a witnessed ink thumb print was taken from every patient participating: no consent by patient consultee was authorised. No identifiable patient data were extracted from patients’ records. The same consenting process for record review was followed for patients in control trial arms on 11 occasions but with the assistance of more research assistants (SG, RR and VvH in addition to UK) because of geographical proximity.
Data collectors (RR, SG, FB, VvH) were not informed of the study aims and objectives, to reduce observer bias . They were required to treat the data as confidential and undertook in writing not to publish the data or any portion thereof or any documentation relating to the data, other than for the purpose of carrying out their mandate.
Raw data for each patient (vital signs recordings on the MEWS chart and data recorded on the SBAR communication guide) were captured directly on IBM SPSS datasheets (version 22 for Windows, IBM Corp 2011) on the hard drive of a personal computer secured by password-protected access (UK, DB). Baseline patient clinical and demographic characteristics were located on each patient’s personal folder and entered into the SPSS datasheets.
Trial outcomes were assessed by criterion-based clinical record review , for example, assistance summoned: yes = 1, no = 0, and did not require subjective interpretation. Vital signs were coded according to MEWS criteria (lower 1, lower 2, lower 3, 0, upper 1, upper 2, and upper 3). These criteria were used to assess the appropriateness of responses (summoning or not summoning assistance). Double data-entry was undertaken by an independent investigator (SJ) on 28 (9.6%) records randomly selected using SPSS. Following discussion on interpretations, two further random samples of 7 and 34 records (total 69/292, 23.6%) were checked . Data, labelled as trial arms A and B to allow analysis to be blinded, were passed to the analyst (SJ). We followed Consolidated standards of reporting trials (CONSORT) guidelines extended to pragmatic RCTs  (Additional file 4) and the local University of Cape Town (UCT) (CRC) Standard Operating Procedures (SOPs).
Demographic and clinical data were described for both arms. Frequency counts and cross-tabulations were used to compare the prevalence of documentation and responses in both arms during the study period. Where numbers were sufficient, associations were explored in bivariate analyses. Pearson’s chi-squared test was used to compare proportions. Where cells contained fewer than the minimum expected count, Fisher’s exact test was substituted. Findings for binary outcomes were expressed in odds ratios with 95% confidence intervals (CIs) . Small numbers of values for outcome variables precluded multivariate analyses .
Response rate and follow up
Following provisional assurances, three of the six acute hospitals declined to participate, leaving two matching hospitals and a third that could not be matched. Six wards were purposively recruited (two surgical, one medical), respectively, in the two participating hospitals (three wards in each hospital). The hospitals declined to participate for various reasons (one after an 8-month delay): recent introduction of a pilot electronic hospital record system which included the standard vital signs observations chart (the MEWS was paper-based), “too busy”, and the HREC’s approved patient consenting system by poster was deemed unacceptable. We resorted to individual written patient consent. This effectively excluded patients too ill to be approached and to sign fully informed consent and those most likely to experience SAEs, and precluded recruitment of sufficient patients to achieve the required sample size of 612 patients, with the expected prevalence of SAEs. We obtained written consent from 363 patients for their records to be reviewed: 292 (80.4%) were available for analysis, 150 (51.4%) in the intervention arm (n = 101 surgical; n = 49 medical) and 142 (48.6%) in the control arm (n = 98 surgical; n = 44 medical) as shown in the CONSORT flow diagram (Fig. 1). Three participants in the intervention arm had no MEWS chart and were excluded, and a further nine had no total MEWS recorded but were retained.
Patients were not individually randomised, and we compared patients in intervention and control wards. Demographic details and the prescription of opiates, sedatives, diuretics and anti-hypertensives received on the day of abnormal vital signs recordings are described in Table 1.
Baseline differences in recordings of the numbers of patients with each of respiratory rate, oxygen saturation, temperature, heart rate, systolic BP, level of consciousness, urine output and with each of the clinical signs of deterioration are presented in Table 2.
There was a statistically significant difference in recording between the trial arms for physiological parameters listed on the MEWS observations chart but omitted from the standard ward observations chart: oxygen saturation (81/150, 54.0% vs 25/142, 17.6%, OR 5.49 [3.21–9.41], P < 0.001) and level of consciousness (135/150, 90.0% vs 108/142, 76.1%, OR 2.83 [1.47–5.47], P = 0.001). All seven vital signs were recorded more often on the MEWS observations chart in the intervention arm than on the standard chart in the control arm and the difference was close to statistical significance (33/150, 22.0% vs 19/142, 13.4%, OR 1.83 [0.98–3.40]).
There was a statistically significant difference between the trial arms for reporting clinical parameters listed on the MEWS but not the standard chart (Table 2). Skin colour (pallor/cyanosis), pain, sweating, wound oozing, pedal pulses and “looks unwell” were more frequently recorded on the MEWS. Glucose and haemoglobin levels were more frequently recorded on standard charts (Table 2).
Assistance summoned for a red MEWS warning - score of 3
The primary outcome, nurses’ appropriate reporting behaviour, assessed from documentation in patients’ records, was disappointing in both arms. Assessment of a patient is required immediately when a single physiological parameter falls outside predetermined ranges printed in red on the MEWS chart with a score of 3. The number of patients with 0–4 recordings of level-3 MEWS is shown in Table 3: no patients had more than 4 such recordings. The difference between the two trial arms in the number of patients with any vital signs recordings of level-3 red MEWS was statistically significant (45/150, 30.0% vs 81/142, 57.0%, P = < 0.001, OR 0.32 [0.20–0.52]), indicating that the control-arm participants were more likely to have disturbed physiology. Some patients had more than one recording at red level 3. More patients in the control arm had one, two or four such abnormal recordings. Fewer patients with abnormal vital signs at red level 3 in the intervention arm were referred for medical attention, that is, they had assistance summoned (2/45, 4.4% vs 11/81, 13.6%, OR 0.29 [0.06–1.39]) (Table 3).
Physiological parameters with MEWS red (3) and yellow (2) warning scores
While assessment of a patient is required immediately when a single physiological parameter triggers at a score of 3 (red) on the MEWS chart, review of a patient is required 5 min after a single physiological parameter triggers at a MEWS of 2 (yellow). All abnormal recordings of vital signs reaching the trigger thresholds of a MEWS level 2 or 3 for respiratory rate, oxygen saturation, heart rate, systolic BP and level of consciousness for each patient are detailed in Table 4. Some patients had more than one recording at a MEWS level 2 or 3, but this is not indicated in Table 4. The number of recordings of abnormal vital signs that should have triggered at a MEWS level 2 or 3 for these parameters reached 165 in the intervention arm (but did not in 159/165, 96.4%) and 178 in the control arm (but did not in 158/178, 88.8%) (Table 4).
More patients in the control arm (n = 70, 49.3%) needed review of respiratory rate than in the intervention arm (n = 43, 28.7%). The difference between trial arms in the number of patients whose abnormal respiratory rates triggered assistance was not statistically significant (intervention arm 1/43, 2.3% vs 1/70, 1.4% in the control arm).
There were 9 of 150 patients in the intervention arm (6.0%) and 61 of 142 patients (42.9%) in the control arm with a rapid respiratory rate (≥ 25 bpm) recorded, meeting the criterion for a single red MEWS of 3, suggesting more acute illness in the control arm. None of these triggered a response in either arm. There were no recordings in either the intervention or control arm of a single red MEWS for a low respiratory rate (≤ 8 bpm).
In the intervention arm, the SBAR tool was used appropriately for one patient with a respiratory rate of 22 breaths per minute (bpm) (MEWS of 2). In the intervention arm most patients with abnormal respirations (34/43, 22.7%) had respiratory rates between 21 and 24 bpm (MEWS of 2). In the control arm most (61/70, 42.9%) abnormal respiratory rates were > 25 bpm. Assistance was summoned for two patients (one in each arm) with rates of 21–24 per minute (yellow MEWS of 2) (Table 4).
There was a statistically significant difference between trial arms in the number of patients whose abnormal oxygen saturations triggered assistance (intervention arm 0/19 vs 5/9, 55.6% in the control arm; P = 0.003, Fisher’s exact test, Table 4). Patients in the intervention arm were less likely to have assistance summoned. There were 12 of 150 patients (8.0%) in the intervention arm and 6 of 142 patients (4.2%) in the control arm with oxygen saturation levels < 92%, indicating acute illness. Of the five responses in the control arm, three (60.0%) were for a MEWS level 3 and two (40.0%) for a MEWS level 2. The SBAR tool was not used to communicate low saturation levels for 19 patients in the intervention arm: 12 were patients with saturations < 92% (MEWS of 3) and 7 were patients with 92–93% saturations (MEWS of 2) (Table 4).
More patients in the intervention arm (n = 44, 29.3%) than in the control arm (n = 37, 26.1%) required review for an abnormal heart rate (Table 4). In the intervention arm none of the eight recordings in the red MEWS range of 3 for tachycardia at ≥ 131 beats per minute triggered a response but there was one response for MEWS level 2. The single recording of bradycardia at a MEWS level 3 triggered a response. In the control arm only one reported incidence of tachycardia at MEWS level 2 elicited a response.
Systolic blood pressure
In the intervention trial arm, 57 patients required review for abnormal systolic blood pressure (Table 4). The lower response rate in the intervention arm (1/57, 1.8% vs 11/59, 18.6%) was statistically significant (P = 0.003, OR 0.08 (0.01–0.63)). No low systolic blood pressure (SBP) (≤ 90 mmHg) recordings in the red range triggered a response, and neither did the single recording for high SBP (≥ 220 mmHg). The single response was for MEWS level 2 (92 mmHg) and the SBAR tool was used appropriately.
In the control arm 23 of 142 (16.2%) patients had a red MEWS value recorded for low SBP (≤ 90 mmHg), and 4 of 23 (17.4%) were actioned comprising 4/11 (36.4%) of the total responses. There were 4 of 11 (36.4%) responses for recordings of high SBP (≥ 220 mmHg) and 3 of 11 (27.3%) responses from 32 recordings for MEWS level 2 (91–100 mmHg).
Level of consciousness
There were few reported incidents: in the intervention arm both reported incidents of a disturbed level of consciousness were actioned, as were two of three reports (66.7%) in the control arm.
At a critical MEWS level of 3, there were 65/150 patients (43.6%) in the intervention arm with abnormally low temperature recordings (≤ 35 °C) as their only abnormal sign; none of these elicited a response. Response rate was similarly low in the control arm. Ward digital thermometers were not calibrated regularly and the procedure for taking axillary temperatures was questionable: nurses were observed placing the digital thermometer between layers of clothing.
We regarded recordings of temperature readings as unreliable and omitted these from Table 4. Had these recordings been accurate, recordings would have triggered calls for assistance for 45 of 150 (30.0%) patients in the intervention arm and for 81 of 142 (57.0%) patients in the control arm.
For many patients in both trial arms, no urine output was recorded (intervention arm: n = 107, 71.3%; control arm: n = 69, 48.6%) and no responses were recorded. In both trial arms a fluid balance (intake/output) chart for recording urine output was used, and this was completed in preference to the MEWS chart in the intervention arm. Most were recorded as passed urine in toilet (PUIT). Volume was seldom measured on MEWS or fluid balance charts (5/142, 3.5% in the control arm and 13/150, 8.7% in the intervention arm).
Serious adverse events (SAEs)
Few SAEs were recorded and there were no statistically significant differences between trial arms (Table 5).
It is reported that delayed recognition of deterioration of patients on general wards can be attributed to human-related monitoring failures . Early detection of at-risk patients requires regular and systematic assessment . The primary outcome, nurses’ responses to early signs of clinical and physiological deterioration, was disappointing in both arms and the SBAR communication guide for reporting clinical deterioration in intervention wards was rarely used. Importantly, and for the first time, this trial offers limited support for the assumed superiority of EWS/MEWS. This “no benefit” conclusion can only be extended to the same or similar settings and conditions pertaining to this trial: evaluation of paper-based EWS instruments not previously used routinely, gatekeepers declining access to researchers for ethically approved studies, and the requirement for patient consent for testing new nursing documentation posing low/no risk to patient care. No similar studies were identified.
Nurses in the control arm were more likely to respond to abnormal physiology, except for level of consciousness. Other than (likely spurious) low temperature recordings, no assistance was summoned for a single red MEWS for critical illness for 96% of affected patients in the intervention arm and 86% of patients in the control arm. Nurses largely relied on SBP, oxygen saturation and level of consciousness when deciding on whether to summon assistance, and tended to disregard respiration rate. Nurses’ measurement of respiratory rate was found to be unreliable .
Although axillary temperature is not considered accurate  and is about 1 °F lower than oral temperature , it was the conventional method of measurement on public sector hospital wards in Cape Town at the time. It is unusual for nurses to report low temperature recordings.
There was a statistically significant difference between the trial arms for recording of physiological parameters listed on the MEWS observations chart but not on the standard ward observations chart: oxygen saturation (OR 5.49 (3.21–9.41), P < 0.001) and level of consciousness (OR 2.83 (1.47–5.47), P = 0.001) (Table 2).
As reported elsewhere  clinical signs such as skin tone, sweating, nausea or nurses’ intuitive assessment of the patient being “just not right” and “looking unwell”  should be monitored regularly to limit avoidable, serious adverse events (SAEs) such as cardiac arrest, urgent and unanticipated admission to an intensive care unit (ICU) or even death. There was a statistically significant difference between the trial arms for clinical parameters listed on the MEWS observations chart but not on the standard ward observations chart: skin colour (OR 9.32 (4.40–19.74), P < 0.001), pain (OR 4.48 (2.73–7.36), P < 0.001), sweating (OR 144.81 (19.74–1062.35), P < 0.001), wound oozing (OR 3.76 (2.23–6.34), P < 0.001), pedal pulses (OR 4.68 (2.73–8.01), P < 0.001), and looks unwell (OR 69.0 (24.13–197.27), P < 0.001). The MEWS ensured that such measurements were more comprehensive, with two exceptions for glucose (OR 0.17 (0.10–0.28), P < 0.001), and haemoglobin (OR 0.48 (0.30–0.77), P <0.002) levels. There were too few SAEs to draw any conclusions and the higher numbers of SAEs in the intervention arm, despite fewer abnormal vital signs, may have been due to complexities and intra-hospital factors other than the type of observation chart.
Limitations, challenges and strengths
Our pilot study was not delivered as intended. However, this trial has raised important questions about ethical and regulatory challenges related to PCTs , including the role of gatekeepers , and determination of what constitutes “no more than minimal risk” research .
Gatekeeping recruitment and delivery
This was a two-hospital comparison, where, for administrative and practical considerations, and to avoid contamination , trial arm was dictated by hospital. Whilst we acknowledge the limitation of this design , this was the only available option after three hospitals declined involvement. With only one hospital in each arm, we were unable to adjust for hospital characteristics, overt and tacit, and we acknowledge this limitation. Preliminary data had indicated that patients were similar in the two participating hospitals, and there were no marked dissimilarities between the recruited samples. However, on data collection, consenting patients in the intervention hospital were slightly older and more likely to have been medicated. Similarly, recordings of vital signs (unavailable before the study) indicated that the patients in the control arm were probably sicker (Tables 1, 2).
Gatekeepers critically shaped our trial . During recruitment, we were denied access to a hospital after an 8-month delay, justified by the commencement of a pilot study on an electronic document system that included the standard vital signs observations chart. The duration of the training programme was reduced by hospital managers, from a planned 8 h to 2 h, and this fell to 30–60 min on night duty, due to staff shortages. We do not know the number of relief, agency or part-time nurses who completed the MEWS charts without any training from full-time staff. Failure to train agency nurses may have affected study findings: nurse managers did not give these nurses the opportunity to receive the training on the intervention, because they were not considered to be the hospital’s stable workforce. The clinical impact of professional training courses varies : whilst some authors report the success of 1-day multi-media multi-professional training courses [37, 38], others report no significant changes . Our teaching was limited to nurses in full-time employment and the strategy included a didactic approach, followed by clinical scenarios to guide filling in blank MEWS charts, but might have been enhanced by simulation . No simulation laboratory was available in the hospital, and, at the time, there were no facilities for online teaching and learning .
“No more than minimal risk” research
The study was less representative and smaller than planned. Patient records could not be randomly selected as the individual patient consenting process effectively excluded ill patients and proved to be a barrier to recruiting sufficient patients to achieve the required sample size. We had aimed to recruit 51 patients in each of 12 wards across 6 hospitals. We were able to recruit 292 eligible patients across 6 wards in 2 hospitals. We recognise that this lower effective sample size detracts from our trial and subsequent decision-making. Although the previous Research Ethics Committee (REC) in 2009 and funders (National Research Foundation, South Africa) had agreed that consent would be obtained at cluster (ward) level, restrictions were imposed by the REC in 2014 on the consenting process for collection of anonymised data from hospital records .
As this was a trial of nursing documentation, without patient contact, and patients are not normally consulted about the design of nursing charts for recording vital signs, or other nursing documentation, we had anticipated compliance with the Ottawa Statement, integral to the CONSORT statement [41,42,43] and recommendations for PCTs [16, 18]. The limited available resources to consent patients are reflected in the reduced number of participants, indicating that findings must be interpreted with caution and regarded as preliminary. Largely due to the exclusion of the patients who were the most ill, there were small numbers in outcome variables (two subjects in the key outcome in the intervention arm), precluding adjusted analyses . We made no adjustment for multiple testing. A larger sample would have been unlikely to reverse our findings; however, we can only speculate as to whether the original multi-site study with an unbiased patient sample would have yielded similar findings.
HIV status was not a consideration for this study. Zambia, a developing country, with a significant HIV burden, demonstrated no significant difference in early warning scores when comparing patients with different HIV status (HIV positive, negative or unknown), P = 0.51 .
From single-centre research, we cannot assume that findings can necessarily be generalised to settings where the prevalence of the conditions under consideration or nurse-to-patient ratio or nurses’ educational levels or communication ethos may differ [45, 46]. Furthermore, resources did not extend to qualitative analyses of nurses’ perceptions of the MEWS charts . However, the staffing and case mix are representative of hospitals in Cape Town. The non-participation of three hospitals and patients unwilling or too ill to consent to the review of records by researchers introduces a potential for volunteer bias. We cannot generalise our findings to sicker patients or settings less willing to participate in research , and clinical trial volunteers are often of higher socioeconomic status than those who decline . Had all patients been included, the prevalence of the “summoning assistance” outcome variables might have been sufficient for a more robust analysis.
Sources of bias
Blinding: it is rarely possible to blind staff participants in trials of documentation, but patients were unlikely to have been aware of their allocation. As the MEWS charts were in patients’ records in the intervention wards, it was not possible to blind the independent record reviewers to trial arm. Blinding of staff during administration of the intervention, data capture and outcome assessment in non-pharmacological , cluster  or pragmatic trials may be impossible , increasing the risk of bias  in line with observers’ expectations . Bias in the detection of outcomes was minimised, but not removed, by a pre-arranged data collection template and the low subjectivity of outcome data extracted from standardised documentation, such as vital signs’ records . There were no obvious differences between the two arms in outcomes; therefore data analysts could not guess whether A or B was the intervention arm. Findings favouring the control arm offer no evidence that the study team were influenced by the Rosenthal effect  and entrapment by prior expectation . Systematic review indicates that some 3% unblinded assessments are likely to be misclassified , which would not materially affect interpretation of our study.
Our findings are dependent on the reliability and completeness of patient records. There were several instances where summoning senior assistance would have been warranted, but this was not documented: we could only conclude that this was not done . We have no reason to suppose that this and the Hawthorne effect  would have affected the arms unequally.
Despite shortcomings, our pilot trial can contribute to a “learning healthcare system”  in which continuous learning takes place in the context of routine patient care towards providing answers to clinical questions . The revised CT MEWS observations chart improved recording of certain physiological and all clinical parameters when compared to the standard chart but did not improve nurses’ ability to identify early signs of clinical deterioration and summon assistance. Recruitment of only two hospitals limits generalisation of these findings. Further work is needed on educational preparation for the CT MEWS/SBAR and testing impact on nurses’ reporting behaviour.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Clinical Research Centre
Department of Health
Early warning score
Human Research Ethics Committee
Level of consciousness
Medical Emergency Team
Modified Early Warning Score
National early warning score
National Health Insurance
Pan African Clinical Trials Registry
Pragmatic clinical trial
Registered Professional Nurse
Serious adverse event
Standard operating procedures
University of Cape Town
Kyriacos U, Jelsma J, James M, Jordan S. Monitoring vital signs: development of a modified early warning scoring (Mews) system for general wards in a developing country. PLoS One. 2014;9:e87073.
Kyriacos U, Jelsma J, Jordan S. Record review to explore the adequacy of post-operative vital signs monitoring using a local modified early warning score (Mews) chart to evaluate outcomes. PLoS One. 2014;9:e87320.
Kyriacos U, Jelsma J, James M, Jordan S. Early warning scoring systems versus standard observations charts for wards in South Africa: a cluster randomized controlled trial. Trials. 2015;16:103.
Robb G, Seddon M. A multi-faceted approach to the physiologically unstable patient. Qual Saf Health Care. 2010;19:e47.
De Meester K, Verspuy M, Monsieurs KG, Van Bogaert P. SBAR improves nurse–physician communication and reduces unexpected death: a pre and post intervention study. Resuscitation. 2013;84:1192–6.
Ludikhuize J, de Jonge E, Goossens A. Measuring adherence among nurses one year after training in applying the modified early warning score and situation-background-assessment-recommendation instruments. Resuscitation. 2011;82:1428–33.
Rylance J, Baker T, Mushi E, Mashaga D. Use of an early warning score and ability to walk predicts mortality in medical patients admitted to hospitals in Tanzania. Trans R SocTrop Med Hyg. 2009;103:790–4.
Wong D, Bonnici T, Knight J, Gerry S, Turton J, Watkinson P. A ward-based time study of paper and electronic documentation for recording vital sign observations. J Am Med Inform Assoc. 2017;24:717–21.
Sefton G, Lane S, Killen R, Black S, Lyon M, Ampah P, Sproule C, Loren-Gosling D, Richards C, Spinty J, et al. Accuracy and efficiency of recording pediatric early warning scores using an electronic physiological surveillance system compared with traditional paper-based documentation. Comput Inform Nurs. 2017;35:228–36.
Mohammed M, Hayton R, Clements G, Smith G, Prytherch D. Improving accuracy and efficiency of early warning scores in acute care. Br J Nurs. 2009;18:18–24.
Royal College of Physicians. National early warning score (NEWS): standardising the assessment of acute-illness severity in the NHS. In: Report of a working party. London: Royal College of Physicians; 2012.
Subbe CP, Davies RG, Williams E, Rutherford P, Gemmell L. Effect of introducing the modified early warning score on clinical outcomes, cardio-pulmonary arrests and intensive care utilisation in acute medical admissions. Anaesthesia. 2003;58:797–802.
Connell CJ, Endacott R, Jackman JA, Kiprillis NR, Sparkes LM, Cooper SJ. The effectiveness of education in the recognition and management of deteriorating patients: a systematic review. Nurse Educ Today. 2016;44:133–45.
Saab MM, McCarthy B, Andrews T, Savage E, Drummond FJ, Walshe N, Forde M, Breen D, Henn P, Drennan J, Hegarty J. The effect of adult early warning systems education on nurses’ knowledge, confidence and clinical performance: a systematic review. J Adv Nurs. 2017;73:2506–21.
National Health Insurance Bill. In Government Gazette No. 42598 of 26 July. ISBN 978–1–4850-0609-1; 2019. Retrieved from: https://www.gov.za/sites/default/files/gcis_document/201908/national-health-insurance-bill-b-11-2019.pdf. Accessed 6 Dec 2019.
Califf RM, Sugarman J. Exploring the ethical and regulatory issues in pragmatic clinical trials. Clin Trials. 2015;12:436–41.
Whicher DM, Miller JE, Dunham KM, Joffe S. Gatekeepers for pragmatic clinical trials. Clin Trials. 2015;12:442–8.
Lantos JD, Wendler D, Septimus E, Wahba S, Madigan R, Bliss G. Considerations in the evaluation and determination of minimal risk in pragmatic clinical trials. Clin Trials. 2015;12:485–93.
ICH Harmonised Tripartite Guideline for Good Clinical Practice. J Postgrad Med. 2001;47:199–203.
SISA-binomial (simple interactive statistical analysis) [http://www.quantitativeskills.com/sisa/calculations/sampshlp.htm]. Accessed 6 Dec 2019.
Killip S, Mahfoud Z, Pearce K. What Is an Intracluster Correlation Coefficient? Crucial Concepts for Primary Care Researchers. Ann Fam Med. 2004;2:204–8.
Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. Bmj. 2012;344:e1119.
European Clinical Research Infrastructure Network (ECRIN) In Requirements for certification of ECRIN data centres with explanation and elaboration of standards; 2018.
Zwarenstein M, Treweek S, Gagnier J, Altman D, Tunis S, Haynes B, Oxman A, Moher D. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008;337:1223–6.
Altman D. Practical Statistics for Medical Research. London: Chapman & Hall; 1991.
Ottenbacher KJ, Ottenbacher HR, Tooth L, Ostir GV. A review of two journals found that articles using multivariable logistic regression frequently did not report commonly recommended assumptions. J Clin Epidemiol. 2004;57:1147–52.
New manuscripts, reporting guidelines [https://www.nejm.org/author-center/new-manuscripts]. Accessed 6 Dec 2019.
van Galen LS, Struik PW, Driesen BE, Merten H, Ludikhuize J, van der Spoel JI, Kramer MH, Nanayakkara PW. Delayed recognition of deterioration of patients in general wards is mostly caused by human related monitoring failures: a root cause analysis of unplanned ICU admissions. PLoS One. 2016;11:e0161393.
Petersen JA. Early warning score. Challenges and opportunities in the care of deteriorating patients. Dan Med J. 2018;65:1–13.
Brabrand M, Hallas P, Folkestad L, Lautrup-Larsen CH, Brodersen JB. Measurement of respiratory rate by multiple raters in a clinical setting is unreliable: a cross-sectional simulation study. J Crit Care. 2018;44:404–6.
Nonose Y, Sato Y, Kabayama H, Arisawa A, Onodera M, Imanaka H, Nishimura M. Accuracy of recorded body temperature of critically ill patients related to measurement site: a prospective observational study. Anaesth Intensive Care. 2012;40:820–4.
Singh V, Sharma A, Khandelwal R, Kothari K. Variation of axillary temperature and its correlation with oral temperature. J Assoc Physicians India. 2000;48:898–900.
Cioffi J. Nurses’ experiences of making decisions to call emergency assistance to their patients. J Adv Nurs. 2000;32:108–14.
Keogh-Brown M, Bachmann M, Shepstone L, Hewitt C, Howe A, Ramsay C. Contamination in trials of educational interventions. Health Technol Assess. 2007;11:iii.
Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions Version 5.1.0 edition; 2011.
Jordan S. Educational input and patient outcomes: exploring the gap. J Adv Nurs. 2000;31:461–71.
Featherstone P, Smith GB, Linnell M, Easton S, Osgood VM. Impact of a one-day inter-professional course (ALERT) on attitudes and confidence in managing critically ill adult patients. Resuscitation. 2005;65:329–36.
Tume L. The deterioration of children in ward areas in a specialist children's hospital. Nurs Crit Care. 2007;12:12–9.
Fuhrmann L, Perner A, Klausen TW, Ostergaard D, Lippert A. The effect of multi-professional education on the recognition and outcome of patients at risk on general wards. Resuscitation. 2009;80:1357–60.
Boutron I, Moher D, Altman D, Schulz K, Ravaud P. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008;148:295–309.
Weijer C, Grimshaw JM, Eccles MP, McRae AD, White A, Brehaut JC, Taljaard M. The Ottawa statement on the ethical design and conduct of cluster randomized trials. PLoS Med. 2012;9:e1001346.
DiazOrdaz K, Slowther A-M, Potter R, Eldridge S. Consent processes in cluster-randomised trials in residential facilities for older adults: a systematic review of reporting practices and proposed guidelines. BMJ Open. 2013;3:e003057.
Taljaard M, Weijer C, Grimshaw JM, Eccles MP. The Ottawa statement on the ethical design and conduct of cluster randomised trials: précis for researchers and research ethics committees. BMJ. 2013;346:f2838.
Dart PJ, Kinnear J, Bould MD, Mwansa SL, Rakhda Z, Snell D. An evaluation of inpatient morbidity and critical care provision in Zambia. Anaesthesia. 2017;72:172–80.
Aiken L, Clarke S, Sloane D, Sochalski J, Silber J. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002;288:1987–93.
Aiken LH, Clarke SP, Cheung RB, Sloane DM, Silber JH. Educational levels of hospital nurses and surgical patient mortality. JAMA. 2003;290:1617–23.
Burns KA, Reber T, Theodore K, Welch B, Roy D, Siedlecki SL. Enhanced early warning system impact on nursing practice: a phenomenological study. J Adv Nurs. 2018;74:1150–6.
Ozdemir B, Karthikesalingam A, Sinha S, Poloniecki J, Hinchliffe R, Thompson M, Gower J, Boaz A, Holt P. Research activity and the association with mortality. PLoS One. 2015;10:e0118253.
Jordan S, Watkins A, Storey M, Allen S, Brooks C, Garaiova I, Heaven M, Jones R, Plummer S, Russell I, et al. Volunteer bias in recruitment, retention, and blood sample donation in a randomised controlled trial involving mothers and their children at six months and two years: a longitudinal analysis. PLoS One. 2013;8:e67912.
Campbell M, Piaggio G, Elbourne D, Altman D. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012;345:19.
Rosenthal R, Jacobson L. Teachers’ expectancies: determinants of pupils’ IQ gains. Psychol Rep. 1963;19:115–8.
Sackett D, Haynes R, Guyatt G, Tugwell P. Clinical epidemiology: a basic science for clinical medicine. 2nd ed. Boston: Little, Brown; 1991.
Schulz K, Grimes D. The Lancet handbook of essential concepts in clinical research. Edinburgh: Elsevier; 2006.
McIntosh J, Stellenberg E. Effect of a staffing strategy based on voluntary increase in working hours on quality of patient care in a hospital in KwaZulu-Natal. Curationis. 2009;32:11–20.
Roethlisberger F, Dickson W. Management and the worker. Cambridge: Harvard University Press; 1939.
Olsen L, Aisner D, McGinnis JM. Institute of Medicine (US). Roundtable on evidence-based medicine. the learning healthcare system: workshop summary. Washington, DC: National Academies Press; 2007.
We are indebted to all patients and staff involved in this study for their time and support. We are grateful to researcher assistants Sandra Grünewald (SG), Reinette Roziers (RR) and Filomena Borrageira (FB) for assisting with patient consent and record review and Ventie van Heerden (VvH) for assisting with patient consent. We acknowledge with grateful thanks the contribution of the UCT Faculty of Health Sciences’ Clinical Research Centre (CRC) and the Swansea University Data Monitoring committee.
This study was funded by the South African National Research Foundation Grant No. 90295. The funder was not involved in the design, conduct or interpretation of the study.
Ethics approval and consent to participate
The study received ethical approval from the Local University of Cape Town Human Research Ethics Committee (HREC ref. 337/2014; HREC ref. 825/2014), the local Provincial Department of Health (ref. 2014RP069) and the National Department of Health. The trial sponsor was the local CRC, UCT. Permissions from hospital gatekeepers were obtained.
Written consent was obtained from participants (patients for record review following the intervention and nurses for training and record review (intervention wards) and nurses for record review and no training (control wards).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Kyriacos, U., Burger, D. & Jordan, S. Testing effectiveness of the revised Cape Town modified early warning and SBAR systems: a pilot pragmatic parallel group randomised controlled trial. Trials 20, 809 (2019). https://doi.org/10.1186/s13063-019-3916-0
- Early warning score
- Decision making
- Healthcare systems
- Nurse training (MESH terms: early diagnosis, monitoring – physiologic/nursing, vital signs)