Improving oxygen therapy for children and neonates in secondary hospitals in Nigeria: study protocol for a stepped-wedge cluster randomised trial

Abstract

Background

Oxygen is a life-saving, essential medicine that is important for the treatment of many common childhood conditions. Improved oxygen systems can reduce childhood pneumonia mortality substantially. However, providing oxygen to children is challenging, especially in small hospitals with weak infrastructure and low human resource capacity.

Methods/design

This trial will evaluate the implementation of improved oxygen systems at secondary-level hospitals in southwest Nigeria. The improved oxygen system includes: a standardised equipment package; training of clinical and technical staff; infrastructure support (including improved power supply); and quality improvement activities such as supportive supervision. Phase 1 will involve the introduction of pulse oximetry alone; phase 2 will involve the introduction of the full, improved oxygen system package. We have based the intervention design on a theory-based analysis of previous oxygen projects, and used quality improvement principles, evidence-based teaching methods, and behaviour-change strategies.

We are using a stepped-wedge cluster randomised design with participating hospitals randomised to receive an improved oxygen system at 4-month steps (three hospitals per step). Our mixed-methods evaluation will evaluate effectiveness, impact, sustainability, process and fidelity. Our primary outcome measures are childhood pneumonia case fatality rate and inpatient neonatal mortality rate. Secondary outcome measures include a range of clinical, quality of care, technical, and health systems outcomes. The planned study duration is from 2015 to 2018.

Discussion

Our study will provide quality evidence on the effectiveness of improved oxygen systems, and how to better implement and scale-up oxygen systems in resource-limited settings. Our results should have important implications for policy-makers, hospital administrators, and child health organisations in Africa and globally.

Trial registration

Australian New Zealand Clinical Trials Registry: ACTRN12617000341325. Retrospectively registered on 6 March 2017.

Background

Oxygen therapy is an essential medication that is poorly available to sick children globally, despite clear evidence of benefit and wide clinical application [1]. As the primary treatment for hypoxaemia, oxygen therapy is relevant for a variety of clinical conditions including pneumonia and other lung conditions, sepsis, malaria, trauma, and neonatal conditions (including respiratory distress syndrome, asphyxia, and neonatal sepsis). Together, these conditions are responsible for 50% of child deaths globally [2], and hypoxaemia is both a common fatal complication and a significant independent risk factor for death. A recent systematic review estimated that hypoxaemia affects approximately 13% of children admitted to hospital with severe pneumonia, 20% of sick neonates and 10–15% of children with conditions such as malaria, meningitis, or convulsions [3].

Despite the enormous need for oxygen, its availability and use in hospitals globally is poor. Oxygen assessment from various countries shows that access to oxygen equipment, clinical guidelines, training, and technical support is lacking [4,5,6,7,8]. Improved oxygen systems can reduce hospital case fatality rates from childhood pneumonia by 35% [9], when implemented as part of a broad-based quality improvement package. However, experience from four decades of ‘oxygen projects’ shows that effective improvement of oxygen systems in low-resource settings is complex, requiring context-specific technical, clinical, and managerial solutions [10].

Data from various countries shows that improved oxygen use requires optimising both oxygen availability and the use of oxygen by healthcare workers. Achieving good oxygen access requires provision of quality, user-friendly equipment that functions in harsh conditions (e.g. high heat, humidity and dust) and systems to ensure equipment can be maintained, repaired, and replaced (including a system for sustained financing or cost-recovery) [10]. Projects in Egypt [11], the Gambia [12, 13] and Papua New Guinea [9, 14] provide successful examples of improving oxygen access through the use of appropriate equipment and supported by maintenance systems. Achieving good clinical use of oxygen requires skilled, motivated healthcare workers and a work environment that makes it easy to use oxygen well. Projects in Laos [15] and Papua New Guinea [9, 14] provide successful examples of improving healthcare worker use of oxygen through practical training, supportive supervision, and the implementation of evidence-based clinical guidelines. These and other oxygen projects demonstrate that improving oxygen therapy for children in hospitals requires addressing broader case management and care processes, and managerial and policy support [9, 14,15,16,17].

Nigerian context

Nigeria is a populous middle-income country in sub-Saharan Africa that contributes to global child mortality disproportionately. Despite a decrease in child mortality since 1990 (U/5 mortality 213 to 117, 1990 to 2012), child, infant, and neonatal mortality remains high (117, 69 and 34 per 1000 live births, respectively) [18]. The biggest childhood killers are pneumonia (18%), malaria (14%), complications of prematurity (12%), birth asphyxia and trauma (11%), diarrhoeal diseases (10%), and neonatal sepsis (5%) [19].

Nigeria has a decentralised healthcare system with three levels of healthcare delivery: primary, secondary, and tertiary (plus alternative and traditional healthcare providers). Government health expenditure is minimal (7% of total government expenditure, among the lowest in the world) and 66% of total health expenditure is out-of-pocket expenditure, accounting for 9% of household expenditure on average [20]. Public secondary health facilities generally are managed by state government, which provide salary support; however, most day-to-day service activities are funded by patient fees [21]. Private and mission-based health services are common [21]. Nigeria has a large number of healthcare providers, comprising the largest human resource for health in Africa [22]. However, their distribution is weighted heavily towards large urban areas and there is a strong financial incentive to work at federally funded tertiary institutions over state-funded secondary health facilities [22]. Migration of healthcare workers out of Nigeria, and out of the Africa region, is a persistent challenge [22].

Limited data from southwest Nigeria suggests that hypoxaemia is common in children admitted to hospital [23], and access to oxygen therapy for children and newborns is a major challenge [24]. At one tertiary hospital, 29% of children under 5 years of age were hypoxaemic on admission, including 49% of pneumonia cases, 41% of sick neonates, and 14–30% of malaria cases [23, 25].

Theoretical basis

We use a realist approach to project design, implementation, and evaluation. The realist approach seeks to understand how intervention(s) work to produce particular outcomes in a particular context [26]. As such, the realist approach is similar to other theory-based approaches in seeking to develop and test theories of how interventions work in particular contexts in order to inform future implementation in challenging real-world environments [27]. In planning this project, we reviewed experience from previous oxygen projects, including interviewing project coordinators about what they had done, why they chose that approach, and how it had worked (processes and outcomes) [10]. From this, we obtained an understanding of how various strategies have succeeded and failed in different contexts, and could make more informed decisions about how we would approach it in the Nigerian hospital context.

In designing the various components of our intervention, we drew heavily on theories of behaviour change, in particular Michie’s Theoretical Domains Framework (TDF) [28, 29] and the Behaviour Change Wheel (BCW) [30]. The TDF (and BCW) provides a framework for choosing particular behaviour-change strategies based on the behaviour being targeted and the barriers that need to be addressed. For the training activities, we used modern educational theories, particularly Merrill’s ‘first principles’ approach [31]. Merrill promotes a task-based, active learning approach that is structured around ‘demonstration’ (showing), ‘application’ (practice and feedback), ‘activation’ (builds on prior knowledge), and ‘integration’ (application in their particular context).

Objectives

Given the clinical benefits of oxygen therapy, and the current poor access and use of oxygen therapy in Nigerian hospitals, we aim to implement an improved oxygen system in 12 Nigerian hospitals. We anticipate benefits in improved quality of care, and improved clinical outcomes for children (particularly children with pneumonia) and neonates. Our primary objective is to understand how to most effectively implement oxygen systems in complex, resource-limited environments. Our mixed-methods evaluation will evaluate effectiveness, impact, sustainability, process and fidelity.

Methods/design

Study design

This is a stepped-wedge cluster randomised field trial (cross-sectional). The study design and characteristics are summarised in Table 1 and Fig. 1 (see Additional file 1 for the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Checklist).

Table 1 Characteristics of the Nigeria Oxygen Implementation Project stepped-wedge cluster randomised trial

Fig. 1

Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Figure of stepped-wedge study design, illustrating stepped roll-out of the intervention, enrolment and data collection activities over time

We chose a stepped-wedge design as the most efficient and pragmatic trial design for our purposes, using methodology described by Hemming et al. [32,33,34]. Stepped-wedge designs can be useful for investigating the effects of an intervention that is likely to be beneficial in real-world environments. For this reason, it has been used increasingly for evaluating service delivery type interventions in developing countries, and is regarded as a pragmatic design suited to real-world settings [32]. We wanted all hospitals to receive the intervention and we had preliminary data suggesting that it was an effective intervention. We wanted to roll the intervention out sequentially, so it was not feasible to do parallel randomisation. We anticipated moderate intra-cluster correlation meaning that the stepped-wedge design would be relatively efficient. The pulse oximetry provision and training will be done before the full intervention as a ‘phase I’, as hypoxaemia data is necessary for evaluation of the full intervention.

We made a pragmatic decision to define clusters at the hospital level, as this was the natural level at which the intervention would be implemented and geographic separation between hospitals meant there was minimal risk of contamination. We considered clustering at the sub-hospital level (e.g. by ward), but rejected this due to the heterogeneity of ward arrangements, the large cross-over between wards, and the logistical complications that would have resulted. We randomised groups of three hospitals to receive the intervention at four different time-points (steps), spaced 4 months apart. This was based on practical considerations (i.e. how many hospitals could be practically set-up at a particular point in time), effect considerations (i.e. the effect should be realised within this time period), and statistical considerations (i.e. balancing the number of steps with the number of clusters within each step). We considered pairing clusters (hospitals), but rejected this due to the low number of included hospitals. We nominated a 3-week cross-over time for each hospital as it ‘stepped’ into the intervention, as this would be adequate time for the intervention to be fully implemented and pre-intervention patients to be discharged (to avoid contamination). We determined that data collection would be cross-sectional, measured at multiple points before and after the intervention at the level of individual patients (and anticipating a low rate of re-admission during the trial period).

Study setting

This study is based in 12 secondary-level hospitals across four states (Oyo, Ondo, Ogun, and Osun) in southwest Nigeria (Table 2). In a three-tier health system, secondary-level hospitals have an important role in providing first-contact hospital care for the population. We wanted to conduct this study in hospitals that would be representative of secondary-level hospitals treating children in a challenging, high-mortality environment, to ensure that findings would be most relevant for future scale-up in Nigeria and sub-Saharan Africa more broadly.

Table 2 Characteristics of 12 secondary-level hospitals in southwest Nigeria: paediatric and neonatal wards (adapted from baseline needs assessment [8])

In collaboration with the Oyo State Hospitals Management Board, we screened all government secondary health facilities in Oyo state for inclusion based on the whether they admit children routinely. We aimed to identify 10–15 hospitals for inclusion, considering this to be a feasible number of hospitals to effectively implement the project and evaluate the implementation issues involved in oxygen scale-up. After we could not identify sufficient government hospitals in Oyo state alone, we modified our criteria to include government or mission secondary health facilities (i.e. excluding private hospitals) that admitted at least 150 children per year [8]. This provided a representative sample of secondary health facilities providing hospital care for children in southwest Nigeria.

Participant selection and consent

We used a participatory approach to select and request consent of participating hospitals. This included site visits, discussions with clinicians and administrators at individual hospitals, and a central meeting bringing together representatives from all hospitals to discuss the project in more detail. This culminated in signed agreements with each hospital to participate in the study. Hospitals were free to withdraw consent at any time.

We will not obtain consent from individual patients within the hospitals, as we are administering the intervention at the hospital level (children’s and neonatal wards), collecting de-identified data from case records without direct contact with patients and without interfering with their clinical care.. The Ethics Committee agreed that consent at the level of hospitals was appropriate. The unit of analysis will be at the individual patient level, taking into account hospital clustering. The population of interest are sick children (age 28 days to under 18 years) who are admitted to the children’s wards, and sick newborns (aged under 28 days of age) who are admitted to the neonatal ward (or require additional treatment on the maternity ward).

We will invite our project nurses, and key hospital staff at particular hospitals to participate in focus groups and/or interviews. We will obtain individual informed consent from participants of focus groups and interviews using written plain language summaries and verbal explanation in Yoruba or English (as preferred).

Intervention

The intervention will involve the implementation of a comprehensive oxygen delivery system in the children’s and neonatal wards of 12 secondary-level hospitals in southwest Nigeria.

We used a participatory planning approach to develop the intervention and plan implementation. This involved the coordination team visiting all potential sites, conducting a mixed-methods needs assessment [8], bringing key stakeholders from each hospital together for a planning workshop, and meeting with state health authorities. Following this, the general design and components of the intervention were reviewed by revisiting the programme theory derived from past experience, results of the needs assessment, and feedback from healthcare workers, technicians, and administrators at participating hospitals. We based the intervention design on a theory-based analysis of previous oxygen projects, and used quality improvement principles, evidence-based teaching methods, and behaviour-change strategies.

Phase I involves the introduction of pulse oximeters to all participating hospitals, and the commencement of prospective data collection. The early introduction of pulse oximetry will enable accurate classification of severity of illness and indication for oxygen (which is needed for primary analysis) and will give all hospitals a tangible sign of involvement even while they are waiting for full oxygen system installation and training. During phase I, each participating hospital will be visited by the coordination team at least once to work out the site-specific approach to implementing the comprehensive oxygen system (phase II).

Phase II involves the introduction of the comprehensive oxygen system, including (1) standardised oxygen equipment package, (2) clinical education and support, (3) technical training and support, (4) procurement, installation and maintenance structures, (5) infrastructure support (including improved electricity supply using solar technology where needed), (6) strengthening disease surveillance and health information systems, and (7) strengthening quality improvement processes. This is described in detail in Table 3.

Table 3 ‘Improved oxygen system’ intervention components, purpose, and description

During phase II, the coordination team will visit at least every 3 months, to provide supportive supervision, feedback, and collect user feedback. In keeping with the ideals of quality improvement, we will review ongoing user feedback and interim data to monitor implementation and enable timely response to problems in the field. Feedback to the hospitals may include written reports to congratulate them for their progress, and to inform them of how other sites are going and what the next steps in implementation may be.

Hospitals will step into phase II at different times according to pre-defined random allocation. As such, hospitals will be in phase I for between 4 and 16 months, and in phase II for between 8 and 20 months, with a total period of 24 months of prospective data collection.

Outcome measures

To achieve our primary objective of understanding how to most effectively implement oxygen systems, our mixed-methods evaluation will assess effectiveness, process and fidelity and measure a range of clinical, quality of care, technical, and health systems outcomes (including cost, sustainability, and managerial/policy implications) (Table 4).

Table 4 Outcome measures used to evaluate the Nigeria Oxygen Implementation Project with associated research question and data source

Clinical outcomes

We will collect data on clinical outcomes from case notes to evaluate clinical effectiveness and to enable exploration of variability of effect between hospitals. The primary clinical outcome will be in-hospital case fatality rates for the target population. Secondary clinical outcomes will include a range of quality of care measures relating to oxygen therapy, and supportive care. We will assess quality of care based on key clinical activities at various stages of care (assessment, diagnosis/classification, management, monitoring) based on WHO guidelines for the hospital care for children [35]. This approach is similar to the recently described Paediatric Admission Quality of Care (PAQC) score [36].

Implementation outcomes

Our broader approach for evaluating the Nigeria Oxygen Implementation Project is guided by recommendations on the evaluation of complex interventions, particularly the theory-based realist evaluation approach [37,38,39,40,41,42,43]. Our programme theory articulates a series of ‘mechanisms’ through which particular parts of our intervention will impact on clinical outcomes. As such, our evaluation seeks to answer three broad questions. First, was the intervention implemented as we intended? Second, did the intervention affect the identified mechanism as we expected? Third, did the mechanisms activated by the intervention result in change clinical outcomes?

We will collect quantitative and qualitative data to answer these implementation questions. The outcome measures are described in Table 4, together with the corresponding research question and data source. The timeline of data collection is illustrated in Fig. 1.

Sample size

We made sample size and power estimates using the Stata menu-driven programme steppedwedge, which is based on the Hussey and Hughes’ method for calculating sample size for a stepped-wedge trial [44, 45]. We based our assumptions on admission, prevalence, and case-fatality rate data from our needs-analysis, and informed by correlation estimates from previous studies [46]. We estimated that we would enrol approximately 16,770 children and 7670 neonates (1290 preterm) over the course of the study. Based on an alpha of 5%, we calculated that we would have approximately 80% power to detect a 35% reduction in pneumonia case fatality rate (6.1% to 4%), and 90% power to detect a 20% reduction in neonatal case fatality rate (8.2% to 6.6%). We would have greater than 90% power to detect 10% change in quality of care scores.

Allocation

We used pre-randomisation stratification to ensure each step included one ‘large’ (>1500 child admissions per year) and two ‘small/medium’ hospitals (<1500 admissions), as we supposed that the size of hospital (and associated staffing and expertise) may influence the outcomes. We randomised hospitals to receive the intervention in steps 2, 3, 4, or 5, using a computer-generated random sequence generator, resulting in one ‘large’ and two ‘small’ hospitals allocated to receive the intervention at equally spaced 4-month intervals. This stepped-wedge randomised design enabled all hospitals to receive the intervention, while maintaining a robust controlled design (individual hospitals acting as their own historical control, and each other’s contemporaneous control).

We conducted randomisation with four members of the research team in attendance, and shared the randomisation order with hospitals as part of the participatory planning process. It was not possible to blind patients, healthcare-providers, or hospital administrators from the intervention due to the nature of the intervention.

Data management and analysis

Clinical data will be collected from individual case notes by trained nurse data collectors. These data collectors will complete standardised Case Report Forms (CRF1) (see Additional file 2), which will be collated and returned to the project manager before being handed to the data manager. Trained data-entry clerks will double-enter the data into EpiData (EpiData Association, Odense, Denmark). The data manager will supervise the data entry and verification process, and will export the raw data file into.dta format for analysis using Stata v14 (StataCorp, College Station, TX, USA). All data will be de-identified before analysis to preserve confidentiality. The primary investigators will have full access to the data, and summary data will be made available for external validation purposes.

We will report characteristics of the individuals and clusters descriptively to illustrate potential imbalance between exposed and unexposed and allow for discussion of potential biases. We will illustrate the number of observations within each cluster by overlaying the summary numbers onto a stepped-wedge diagram. We will analyse effectiveness using intention-to-treat, according to the time hospitals were randomised to crossover. We will report any differences between intended crossover time and actual crossover time. We will adjust for the effect of calendar time, clustering, and other significant confounders using generalised linear mixed models (GLMM) or generalised estimating equations (GEE) [32, 44]. We will examine heterogeneity in effects between different hospitals using within-cluster comparisons of exposed and unexposed periods. This will contribute to better understanding of how various contextual factors influence effectiveness. We will conduct 6-monthly reviews of data for presentation to a supervisory committee, and to ensure that there is no evidence of significant unintended harm. Given the anticipated benefits and safety of this intervention, we do not have a formal data monitoring committee.

Equipment data will be collected by trained biomedical technicians during routine preventive maintenance visits. These technicians will complete standardised Case Report Forms (CRF2), and return them to the project manager. They will be entered and verified using the same procedures as for CRF1. We will report equipment outcome data descriptively, allowing for comparison between different hospitals.

Healthcare worker knowledge will be assessed using individual written tests (TEST) before and after each training, and at 6-month periods post implementation. Trainers will administer these tests, collate the completed tests, and return them to the project manager. They will be entered and verified using the same procedures as for CRF1. We will report before and after test scores in relation to the basic pulse oximetry training, full oxygen training, and at follow-up. We will report confidence intervals for all estimates, and assess for statistical significance using a t test for (normally distributed) continuous outcomes, and chi-squared for binary outcomes.

Qualitative data from interviews and focus groups will be transcribed and verified for accuracy by participants. We will import these to NVivo v11 (QSR International, Doncaster, VIC, Australia) for data management and analysis. We will use the qualitative data to explore the reasons behind the observed intervention effects (the ‘mechanisms’) and variation between hospitals (the contextual influences). To achieve this we will use a Grounded Theory approach to data analysis, using mixed inductive and deductive content analysis approaches.

Other process data will be collected from administrative records, finance ledgers, and correspondence between investigators and hospitals. We will use this data to understand how the intervention was implemented, identify new avenues of inquiry, and collate practical lessons for future application.

We will report results using the CONSORT 2010 extension to cluster randomised trials, with modifications for the stepped-wedge design as proposed by Hemming et al. [32]. We will publish our results in an open-access journal.

Ethics

This study obtained ethics approval from the University of Melbourne (1543797.1) and University of Ibadan/University College Hospital Ethics Committee, Ibadan, Nigeria (UI/EC/16/0413). We registered the trial on the Australian and New Zealand Clinical Trials Registry (ACTRN12617000341325). Data monitoring will be done by the lead author, reporting to an advisory committee every 6 months.

Discussion

This project is part of a multi-country implementation research project based in Nigeria and Papua New Guinea [47], which seeks to test and build upon the implementation lessons learnt from oxygen projects in Papua New Guinea [9, 14] and elsewhere [10]. By using a realist approach to project design and evaluation, this project will provide information not only on what works, but how it works in different contexts. We consider this information critical to meaningfully scale up oxygen therapy globally and ensure that this basic medical therapy is available to those who need it the most.

Two previous oxygen projects have demonstrated that improved oxygen systems can improve paediatric pneumonia outcomes for hospitalised children [9, 15]. This project will advance the evidence on effectiveness using a rigorous, randomised control design to evaluate clinical outcomes in admitted children and neonates in a challenging sub-Saharan African setting.

At the time of writing we have commenced activities in all 12 participating hospitals. We completed a needs assessment and participatory planning process in 2015, and implemented the full oxygen system in the first three hospitals in April 2016. The final three hospitals have commenced intervention activities in March 2017, as scheduled. The main implementation challenges encountered so far have related to the development and testing of the solar power systems, necessitating a complete redesign and pilot testing of the system. We took alternative measures in the interim to improve and stabilise the power systems for the early intervention hospitals and were able to complete full implementation of the improved oxygen systems according to the planned stepped-wedge timeline.

As expected, different hospitals have taken slightly different approaches to particular aspects of the intervention. For example, while hospital administrators have been open to alternative financing structures for oxygen, the actual implementation of this has been difficult in some hospitals due to administrative and governance requirements. We have been impressed with the ingenuity and problem-solving capacity of the participating hospitals, but informal feedback shows significant variability in the functioning of teams at different hospitals. This highlights the importance of measuring and reporting fidelity and other process indicators, in order to better understand this variability and how improved oxygen systems work in different contexts. Overall, the intervention appears to be achieving very high levels of fidelity, and early data shows large improvements in oxygen-related clinical behaviours (e.g. the use of pulse oximetry, administration of oxygen to hypoxaemic patients, etc.). Interim safety data suggests that the intervention is not causing harm and probably causing benefit, and we anticipate having adequate power to detect a meaningful difference by the end of data collection despite lower than expected admission numbers.

Oxygen therapy is a basic, life-saving therapy that is relevant for many paediatric and adult health conditions, but has limited global availability. Our study should provide quality evidence on the effectiveness of improved oxygen systems, and how to better implement and scale up oxygen systems in resource-limited settings. Our results will have important implications for policy-makers, hospital administrators, and child health organisations in Africa and globally. Our findings may have broader applicability to other healthcare interventions that are worthy of scale-up but are proving challenging to implement in the places that can benefit the most.

Trial status

This trial is underway. Recruitment, clinical data collection, project team visits, and interviews are all ongoing.