When enough is enough: how the decision was made to stop the FEAST trial: data and safety monitoring in an African trial of Fluid Expansion As Supportive Therapy (FEAST) for critically ill children
© Todd et al.; licensee BioMed Central Ltd. 2013
Received: 14 October 2012
Accepted: 13 March 2013
Published: 26 March 2013
In resource-rich countries, bolus fluid expansion is routinely used for the treatment of poor perfusion and shock, but is less commonly used in many African settings. Controversial results from the recently completed FEAST (Fluid Expansion As Supportive Therapy) trial in African children have raised questions about the use of intravenous bolus fluid for the treatment of shock. Prior to the start of the trial, the Independent data monitoring committee (IDMC) developed stopping rules for the proof of benefit that bolus fluid resuscitation would bring. Although careful safety monitoring was put in place, there was less expectation that bolus fluid expansion would be harmful and differential stopping rules for harm were not formulated.
In July 2010, two protocol amendments were agreed to increase the sample size from 2,880 to 3,600 children, and to increase bolus fluid administration. There was a non-significant trend against bolus treatment, but although the implications were discussed, the IDMC did not comment on the results, or on the amendments, in order to avoid inadvertent partial unblinding of the study.
In January 2011, the trial was stopped for futility, as the combined intervention arms had significantly higher mortality (relative risk 1.46, 95% CI 1.13 to 1.90, P = 0.004) than the control arm. The stopping rule for proof of benefit was not achieved, and the IDMC stopped the trial with a lower level of significance (P = 0.01) due to futility and an increased risk of mortality from bolus fluid expansion in children enrolled in the trial. The basis for this decision was that the local standard of care was not to use bolus fluid for the care of children with shock in these African countries, and this was a different standard of care to that used in the UK. These decisions emphasize two important principles: firstly, the IDMC should avoid inadvertent unblinding of the trial by commenting on amendments, and secondly, when considering stopping a trial, the IDMC should be guided by the local standard of care rather than standards of care in other parts of the world.
The FEAST (Fluid Expansion As Supportive Therapy) trial was a multicenter randomized controlled trial of rapid fluid resuscitation in African children with features of shock . Fluid expansion is routine emergency therapy in the management of shock in resource-rich countries , however, the use of fluid resuscitation in severely ill children is less common in sub-Saharan Africa. Moreover, the international evidence for fluid expansion is largely subjective, and the most recent review graded the pediatric recommendation (20mls/kg boluses over 5 to 10 minutes up to 60mls/kg) as 2C, indicating a weak recommendation with low quality of evidence . Prior to the start of the FEAST trial, ethical concerns expressed by some experts about the control arm were raised as children in the control arm would only receive maintenance fluids, in line with the current standard of care in Kenya, Tanzania and Uganda. The trial was stopped early by the Independent data and safety monitoring committee (IDMC) because of lack of benefit and potential harm from bolus fluid compared to children in the control arm.
IDMCs are required to ensure the safety of trial participants and guarantee integrity of the trial data  but the deliberations of the IDMC are not often published. Previous papers have reported on the conduct of the IDMCs after study results have been published [5, 6]. The FEAST trial results were surprising, since they question decades of practice in resource-rich countries and have produced controversy in subsequent commentaries, discussion forums and correspondence [7–11]. Two questions have arisen: firstly, should the IDMC have approved a protocol amendment increasing the bolus dose ; and secondly, was the trial stopped at the right time? This paper highlights the challenges faced in making these key decisions.
The FEAST trial
There were two separate studies within the FEAST trial (FEAST A and B). FEAST B was restricted to children with severe febrile illness complicated by severe hypotension (defined as a systolic blood pressure < 50 mmHg if < 12 months old; < 60 mmHg if 1 to 5 years old; < 70 mmHg if > 5 years old) who were randomized to albumin or saline boluses of 40 to 60mls/kg only. The numbers enrolled were too small (N = 29) to be assessed and FEAST B will not be considered further. In FEAST A, children had severe febrile illness and impaired perfusion but without severe hypotension. Children were randomized to one of three arms: saline bolus 20 ml/kg (increased to 40 ml/kg after protocol amendment), albumin bolus 20 ml/kg (increased to 40 ml/kg after protocol amendment), or control (no bolus: maintenance fluids 4 ml/kg/hour), with the two primary comparisons between saline and control arms (to address whether fluid boluses resulted in a better outcome than standard of care), and between albumin and saline arms (examining whether a colloid resuscitation fluid (5% human albumin solution) was superior to an inexpensive crystalloid solution (0.9% saline)). In addition, children in all three arms received the current standard of care, which included intravenous antibiotics, antimalarial drugs (for those with Plasmodium falciparum malaria) and intravenous maintenance fluids (between 2.5 and 4mls/kg/hour as per national guidelines) until the child was able to retain oral fluids. Antipyretics, anticonvulsants and treatment for hypoglycemia (blood sugar < 2.5mmols/L) were administered according to nationally agreed protocols. Children with a hemoglobin < 5 g/dl were transfused with 20mls/kg of whole blood over four hours. Children were managed on general pediatric wards; mechanical ventilation (other than short-term ‘bag-and-mask’ support) was not available. Basic infrastructural support for emergency care, oxygen saturation and automatic blood pressure monitors were provided for each site. Training in triage and emergency pediatric life support was given prior to and throughout the trial to optimize case recognition, implement supportive management and ensure protocol adherence.
Structure and role of the FEAST IDMC
Possible recommendations from the FEAST charter for the IDMC included
No action needed, trial continues
Early stopping due to clear benefit, or harm of a treatment
Stopping recruitment within a pre-defined subgroup
Stopping one or more arms of the trial and recommendation to amend trial
Proposing or commenting on proposed protocol changes
Commenting on the Statistical Analysis Plan
The FEAST trial was designed to provide evidence that inform policy makers about the benefits of using a fluid bolus in children admitted to hospital with severe febrile illness. The IDMC charter did not specify explicit stopping rules for the trial, but the protocol required ‘proof beyond reasonable doubt’ that one treatment arm was better than its comparator [12, 13]. In the first meetings of the IDMC, the committee discussed the protocol requirements and agreed that the trial should be stopped if proof of benefit for the fluid bolus arms achieved the Haybittle-Peto criteria with a P-value < 0.002. However, although the harm from fluid overload in children receiving the bolus was discussed, a different stopping rule for proof of harm was not formulated. The adoption of a single P-value implied a symmetric two-sided stopping rule that the trial would be stopped if the results unambiguously identified best clinical practice. In later discussions, the IDMC agreed that an asymmetric stopping rule should be adopted with a lower threshold for stopping the trial if it was found that the new treatment (fluid bolus) was not as good as the local standard of care given in the control arm. However, there was no explicit decision on the P-value that would be used for stopping the trial if proof of harm was found.
The monitoring process
IDMC recommendations from first four meetings
First Meeting (25 February 2009):
Requested details of adverse event reporting and listings of all individual events by patient identification
Second Meeting (18 June 2009):
Requested information on children screened for the trial but not enrolled
Requested the number and quantity of additional fluids (other than boluses) given to all children
Asked the study teams to make extra effort to ensure that all children attend for neurological examination at 28 days
Third meeting (12 October 2009):
Endorsed the review procedure for SAE proposed, and stressed the importance of blinded review by the ERC
Accepted co-enrollment of children into other studies
Noted planned sub-studies
Fourth meeting (26 January 2010, third interim analysis):
Observed improvement in attendance for neurological follow up at 28 days
Referred the request by a local IRB for representation on the IDMC to the TSC
Requested more detailed information on all fluids received in each arm by time from randomization
Agreed with the proposed plan of analysis
Mortality in FEAST A presented in each IDMC report
Comparisons between arms for mortality at 48 hours
Mortality in each arm
Risk ratio for saline bolus versus no bolus
Risk ratio for albumin bolus versus saline bolus
Risk ratio bolus versus no bolus
18 June 2009 (data to 13 May 2009); N = 406
0.76 (0.39 to 1.47);P = 0.41
0.87 (0.42 to1.81); P = 0.71
0.70 (0.40 to 1.24); P = 0.22
12 October 2009 (data to 31 August 2009); N = 844
0.89 (0.06 to 1.45); P = 0.65
1.07 (0.66 to 1.74); P = 0.79
0.92 (0.61 to 1.39); P = 0.71
26 January 2010 (data to 31 December 2009); N = 1,337
1.07 (0.73 to 1.60); P = 0.71
1.09 (0.75 to 1.59); P = 0.66
1.13 (0.78 to 1.58); P = 0.50
26 July 2010 (data to 15 June 2010); N = 2,196
1.32 (0.95 to 1.84); P = 0.10
1.001 (0.74 to 1.36); P = 0.99
1.32 (0.98 to 1.77); P = 0.06
12 January 2011 (data to 15 November 2010); N = 2,987
1.46 (1.09 to 1.96);P = 0.01
1.00 (0.77 to 1.30); P = 1.0
1.46 (1.13 to 1.90);P = 0.004
Non-fatal SAEs and neurological sequelae by fluid arm
Non-fatal SAEs and neurological sequelaea in each arm
18 June 2009 (data to 13 May 2009); N = 406
NS : 11
12 October 2009 (data to 31 August 2009); N = 844
26 January 2010 (data to 31 December 2009); N = 1,337
26 July 2010 (data to 15 June 2010); N = 2,196
12 January 2011 (data to 15 November 2010); N = 2,987
- RICP: 1
- AR: 4
- AR: 3
- RICP: 3
Following the fourth report, the TMG informed the IDMC of two proposed changes to the trial protocol. Data to June 2010 showed the global mortality was 9.3%, lower than the 11% assumed in the initial sample size calculations, so an increase in the sample size from 2,880 to 3,600 was proposed so that 350 deaths across the three trial arms would be observed. The IDMC noted the TMG request to increase the sample size, and agreed with the rationale for the increase, but did not comment on the increased sample size. The unblinded results showed that the differences between the arms was smaller than expected, and although not statistically significant, in the wrong direction. However, these results were not made available to the TSC and were not used to support the proposed increase in the sample size.
The second proposed change was to increase the fluid volume in both bolus arms during the first hour from 20 ml/kg to 40 ml/kg, which was the usual recommendation for management of children with shock in high-income countries . In the original design of the trial, owing to safety concerns about the possibility of pulmonary edema in children receiving a large initial fluid bolus, the initial protocol specified that children receive 20 ml/kg bolus in the first hour, followed by a further 20 ml/kg bolus in the second hour if shock was ongoing and there were no safety concerns. However, the open report indicated that only 16% of eligible children received a second bolus, only two children had non-fatal, fluid-specific SAEs, both with evidence of raised intracranial pressure (Table 4), and that only 22 children with severe hypotensive shock had been with enrolled in FEAST B. The TSC had three reasons for recommending an increase. Firstly, the bolus fluid volume (20 ml/kg) (rather than 20 ml/kg plus 20 ml/kg as anticipated in the trial design) may be insufficient to answer the study question (that is futility). Secondly, by not following international guidelines on fluid bolus volume, the results would not convince policymakers on the validity of findings. Finally, only 1% of trial participants had severe hypotensive shock (versus 5% predicted in the trial design), which may affect the saline versus albumin comparison (secondary endpoint).
The IDMC, satisfied that the protocol was being followed correctly, and that the data did not provide a conclusive result, recommended the trial continue. The report from the IDMC to the TSC had ‘no concerns about safety in any of the three arms’, and ‘noted the change in sample size, and in the fluid management proposed’ and had ‘no issues with the amendments’, but was careful to avoid unblinding of the study by consideration of unblinded results.
The fifth and final interim report (12 January 2011)
The IDMC reviewed data on 2,987 children. 48-hour mortality was 10.2% in albumin bolus, 10.2% saline bolus and 7.0% in control arms. The relative risk for saline bolus compared to control was 1.46 (95% CI: 1.09 to 1.96, P = 0.01) with no evidence of any difference between saline bolus and albumin bolus - relative risk 1.00 (95% CI: 0.74 to 1.36). Comparing the combined intervention arms against control (a secondary comparison) gave a highly significant result (P = 0.004), although it had not crossed the implicit threshold (P = 0.002) for proven benefit that was set at the beginning of the study. The IDMC discussed and agreed that a stopping rule for proven harm would require less evidence, and the results were adequate to show the futility of continuing with the bolus fluid intervention. The IDMC unanimously recommended immediate cessation of the study recruitment as there was no prospect that the intervention arms could be superior to the control arm (standard of care in Africa), despite knowing that the intervention arms were standard of care in high income countries.
The IDMC recommendation, but not the result of the trial, was communicated to the chief investigator and the TSC chair, who informed site leaders, and immediately stopped the trial. Confidentiality of the results (including primary outcome) was maintained so that the implications could be discussed at a TSC meeting convened on 19 January 2011 in Nairobi, Kenya before locking the database and unblinding. The manuscript was submitted for publication on 6 April 2011, and published on 26 May 2011 .
Results seen by the IDMC
The efficacy and safety results seen at IDMC meetings are summarized in Table 3, Table 4 and Figure 2. The 48-hour mortality fell from 13.5% to 7.0% in the control arm, but remained around 10% in the two intervention arms throughout the study.
An IDMC is an important safeguard for individuals participating in studies. Clinical trials with mortality or serious morbidity as their main outcome need a properly constituted IDMC to independently monitor the conduct of the trial and examine data on safety and efficacy in order to protect the integrity of the trial and the safety of the participants. The terms and conditions of the IDMC should be defined beforehand and should include guidance on the recommendations open to the IDMC and the criteria for making these recommendations. However, IDMC members need to use external evidence, and their expert judgment, in deciding whether or not to recommend changing the trial protocol or stopping the trial. In this trial the IDMC made recommendations to the TSC about the conduct of the study, including the issues on the data, the end point review, and the continuation of the trial.
Before the trial started, views on the use of bolus fluid for children admitted to hospital with signs of shock were polarized. Whereas in many high income countries the use of fluid bolus was routine and standard , the practice was not widespread in African hospitals . It was unclear whether the trial should be considered as a new intervention in hospitals where rapid fluid resuscitation was not generally used, or if this was the extension of a ‘proven’ intervention into a group of children with different characteristics. The trial had a null hypothesis of no effect and a two-sided alternative hypothesis, so the IDMC applied the same criteria to all three arms. This point was made to institutional review boards who were initially concerned about the ethics of including a control arm in the study. However, in this African setting, the standard of care was largely no bolus fluid, and the stopping rule for proof of benefit for the new intervention was set at 0.002 to ensure that the final significance would not be adversely affected by the interim analyses. Initially, although the potential for harm was discussed, there were no discussions about having a different stopping rule for proof of harm, and this was agreed later by the IDMC, when safety considerations for children receiving the intervention became apparent.
In July 2010, the IDMC was informed about protocol amendments to increase the sample size and to increase the bolus fluid volume. There was higher mortality in both intervention arms compared to the control arm, but there was no indication to stop the trial (Figure 2), and no ‘proof beyond reasonable doubt’ that one treatment arm was better than the others. Although the IDMC was careful not to comment on the unblinded data, some of the blinded investigators considered the IDMC approval as evidence in favor of the bolus treatment. Following the publication of the trial results, concerns were raised about allowing the amendment to increase the initial fluid bolus when interim results suggested the intervention may be harming children in those arms of the trial . The IDMC considered the data without revealing the results, and noted the proposed amendment without conveying approval for the increased initial bolus dose. The IDMC concluded that the benefits of continuing the trial, to reach a statistically significant result, were more important than stopping the trial when there was no definitive proof-of-harm to children receiving the new interventions.
In January 2011, the trial was stopped early with significantly higher mortality (P = 0.01) in children in both bolus arms compared to the control arm (combined endpoints P = 0.004) although primary comparisons did not cross the boundary for early stopping according to the guidelines in the protocol. The trial was not stopped because of proven superiority of the control versus intervention arms, as this would have required ‘evidence beyond reasonable doubt’. The IDMC stopped the trial for futility, because the standard of care in Africa was the control arm, and there was no prospect that the intervention arms could ever be superior to the control arm. Furthermore, there was good evidence that children may be harmed by continuation of the trial. The IDMC realized that they were applying asymmetric stopping rules but considered this appropriate for the local standard of care. The IDMC may have continued the trial if the standard of care had been bolus fluid administration as stronger evidence is required to establish ‘beyond reasonable doubt’, that a standard of care is inferior, as opposed to showing that a new treatment is inferior.
The FEAST trial emphasized two important principles that underpin the role of the IDMC in the conduct of trials. Firstly, the IDMC should not comment on protocol amendments that might lead to inadvertent partial unblinding of the trial and was correct in noting the protocol amendment increasing the bolus dose without conveying information about the results of the trial. Secondly, early termination of the trial should be considered when there is good evidence that continuation would put participants at unnecessary risk and there is no realistic possibility that the new treatment could be superior to current standard of care. Additionally, the implications of the results from the FEAST trial may differ between countries and IDMC applied local standards of care to stop the study.
endpoint review committee
Fluid Expansion As Supportive Therapy
independent data monitoring committee
institutional review board
non-fatal serious adverse event
raised intracranial pressure
serious adverse event
trial management group
trial steering committee.
We would like to thank the FEAST trial leadership and TSC for the opportunity to monitor this landmark study and congratulate them on its excellent conduct. We have special appreciation for the support given by Kathryn Maitland (chief investigator), for the help Abdel Babiker and Liz Russell of the MRC CTU have given in preparing the reports for the IDMC, and for allowing the use of confidential papers for this report. TP is a NIHR senior investigator.
- Maitland K, Kiguli S, Opoka RO: Mortality after fluid bolus in African children with severe infection. N Engl J Med. 2011, 364 (26): 2483-2495. 10.1056/NEJMoa1101549.View ArticlePubMed
- Hospital Care for Children: guidelines for the management of common illnesses with limited resources. 2005, Geneva, Switzerland: World Health Organization
- Dellinger RP, Levy MM, Carlet JM: Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock: 2008. Intensive Care Med. 2008 Jan, 34 (1): 17-60. 10.1007/s00134-007-0934-2.PubMed CentralView ArticlePubMed
- Ellenberg S: Data Monitoring Committees in Clinical Trials: A Practical Perspective. 2002, WileyView Article
- Armitage P: Data and safety monitoring in the Delta trial. Control Clin Trials. 1999, 20: 229-241. 10.1016/S0197-2456(99)00003-3.View ArticlePubMed
- Armitage P: Data and safety monitoring in the Concorde and Alpha trials. Control Clin Trials. 1999, 20: 207-238. 10.1016/S0197-2456(99)00005-7.View ArticlePubMed
- Duke T: What the African fluid-bolus trial means. Lancet. 2011, 378 (9804): 1685-1687. 10.1016/S0140-6736(11)60881-7.View ArticlePubMed
- Myburgh JA: Fluid resuscitation in acute illness - time to reappraise the basics. N Engl J Med. 2011, 364 (26): 2543-2544. 10.1056/NEJMe1105490.View ArticlePubMed
- Hilton AK, Bellomo R: Totem and taboo: fluids in sepsis. Crit Care. 2011, 15 (3): 164-10.1186/cc10247.PubMed CentralView ArticlePubMed
- Bates I: Detrimental effect of fluid resuscitation in the initial management of severely ill children in Africa. Transfus Med Oct. 2011, 21 (5): 289-290. 10.1111/j.1365-3148.2011.01108.x.View Article
- Southall DP, Samuels MP: Treating the wrong children with fluids will cause harm: response to 'mortality after fluid bolus in African children with severe infection'. Arch Dis Child Oct. 2011, 96 (10): 905-906. 10.1136/archdischild-2011-300436.View Article
- Boag JW, Haybittle JL, Fowler JF, Emery EW: The number of patients required in a clinical trial. Br J Radiol. 1971, 44 (518): 122-125. 10.1259/0007-1285-44-518-122.View ArticlePubMed
- Peto R, Pike MC, Armitage P: Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br J Cancer. 1976, 34 (6): 585-612. 10.1038/bjc.1976.220.PubMed CentralView ArticlePubMed
- Brierley J, Carcillo JA, Choong K: Clinical practice parameters for hemodynamic support of pediatric and neonatal septic shock: 2007 update from the American College of Critical Care Medicine. Crit Care Med. 2009, 37 (2): 666-688. 10.1097/CCM.0b013e31819323c6.PubMed CentralView ArticlePubMed
- Robertson MA, Molyneux EM: Description of cause of serious illness and outcome in patients identified using ETAT guidelines in urban Malawi. Arch Dis Child. 2001, 85 (3): 214-217. 10.1136/adc.85.3.214.PubMed CentralView ArticlePubMed
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.