Table 4 Outcome measures used to evaluate the Nigeria Oxygen Implementation Project with associated research question and data source
Category | Research question(s) | Outcome measures | Data source |
|---|---|---|---|
Clinical effectiveness | Does the intervention improve outcomes for the target population (specifically child pneumonia, pre-term/small neonates)? | Under-five case fatality rate (all-cause) Under-five pneumonia case fatality rate Neonatal case fatality rate Pre-term/small neonatal case fatality rate | Case notes |
Epidemiological | What are the characteristics of patients with hypoxaemia in these hospitals? | Mean duration of hypoxaemia, oxygen therapy Descriptive statistics regarding hypoxaemia and oxygen therapy (age, condition, illness severity, etc.) | Case notes |
Quality of care – oxygen practices | Can healthcare works correctly identify, treat, and monitor children and neonates who need oxygen therapy? Does the intervention improve healthcare workers’ oxygen care practices? How is the intervention adopted by healthcare workers? What training and supervisions is required to change practice? | Proportion of hypoxaemic children who correctly received oxygen therapy Proportion of admitted children who have pulse oximetry done correctly during admission Proportion of children receiving oxygen who had a documented clinical indication for oxygen Mean quality of care score for oxygen therapy Mean knowledge/skill scores Attitudes, behaviours, and feedback from healthcare workers (qualitative) | Case notes Knowledge/skill test Interviews/focus groups |
Quality of care – broader care | What is the quality of inpatient paediatric hospital care? Does the intervention improve healthcare workers’ broader care practices? | Mean quality of care score for target conditions (pneumonia, malaria, pre-term/small neonate) Proportion of admitted children discharged against medical advice Mean length of stay Healthcare worker motivation, satisfaction Structural and process determinants of care quality (medical supplies, hygiene, guidelines, clinical review meetings, record keeping, etc.) | Case notes Interviews |
Technical | Can hospital staff reliably maintain the oxygen equipment in working order? What technical problems are reported, how common, what solution, what cost? | Proportion of oxygen concentrators clean and in working order (producing > 85% oxygen, adequate flow, etc.) Proportion of pulse oximeters clean and in working order Proportion of time when concentrators were not functional, or not available for use (and reasons for non-availability) Proportion of solar power (or other improved power) systems in working order (and reasons for failure) Proportion of maintenance checks actually completed Descriptive statistics regarding technical problems identified, solutions, and costs | Technician log book (includes Standardised Report Form) |
Fidelity | Was the intervention implemented as planned? | Actual timing of intervention vs intended (including training, installation, and supervision activities) Actual composition of intervention vs intended (including training, installation, and supervision activities) | Administrative records Technician log book |
Cost | What is the cost of the intervention? | Equipment and installation costs Training and supervision costs Mean cost per life saved Mean cost per disability-adjusted life year (DALY) saved | Administrative records Case notes |
Managerial and policy | Can the intervention be integrated in hospital managerial and state policy structures? | Hospital implementation of oxygen financing recommendations Hospital implementation of ‘oxygen team’ structure State commitment and funding for oxygen programme National impact on oxygen policy and guidelines AFASS (Acceptability, Feasibility, Affordability, Sustainability, Safety) Unintended effects Barriers and enablers | Administrative records Interviews |
Sustainability | Can the intervention be sustained in the medium to long term? | Assess at 1, 2 and 5 years (using indicators above): • Technical function and maintenance processes • Healthcare worker oxygen practices • Cost recovery and ongoing financing • Hospital-level scale-up • Managerial and policy integration and scale-up | Technician log book Interviews |