Oxygen systems strengthening as an intervention to prevent childhood deaths due to pneumonia in low-resource settings: systematic review, meta-analysis and cost-effectiveness

Objectives Increasing access to oxygen services may improve outcomes among children with pneumonia living in low-resource settings. We conducted a systematic review to estimate the impact and cost-effectiveness of strengthening oxygen services in low-income and middle-income countries with the objective of including oxygen as an intervention in the Lives Saved Tool. Design We searched EMBASE and PubMed on 31 March 2021 using keywords and MeSH terms related to ‘oxygen’, ‘pneumonia’ and ‘child’ without restrictions on language or date. The risk of bias was assessed for all included studies using the quality assessment tool for quantitative studies, and we assessed the overall certainty of the evidence using Grading of Recommendations, Assessment, Development and Evaluations. Meta-analysis methods using random effects with inverse-variance weights was used to calculate a pooled OR and 95% CIs. Programme cost data were extracted from full study reports and correspondence with study authors, and we estimated cost-effectiveness in US dollar per disability-adjusted life-year (DALY) averted. Results Our search identified 665 studies. Four studies were included in the review involving 75 hospitals and 34 485 study participants. We calculated a pooled OR of 0.52 (95% CI 0.39 to 0.70) in favour of oxygen systems reducing childhood pneumonia mortality. The median cost-effectiveness of oxygen systems strengthening was $US62 per DALY averted (range: US$44–US$225). We graded the risk of bias as moderate and the overall certainty of the evidence as low due to the non-randomised design of the studies. Conclusion Our findings suggest that strengthening oxygen systems is likely to reduce hospital-based pneumonia mortality and may be cost-effective in low-resource settings. Additional implementation trials using more rigorous designs are needed to strengthen the certainty in the effect estimate.


Rating
Explanation (A) Selection bias (Q1) Are the individuals selected to participate in the study likely to be representative of the target population?
Somewhat likely Five hospitals were purposely selected but represented both highland and lowland areas. Page 2.
(Q2) What percentage of selected individuals agreed to participate? 60 -79% agreement All pneumonia admissions were included through retrospective register reviews though some of the registers were not available. Page 3. (B) Study design (Q1) Indicate the study design Cohort analytic (two group pre + post) Pre + Post in the same facilities. (Q2) Was the study described as randomized? If NO, go to Component C.
No (Q3) If Yes, was the method of randomization described? (See dictionary) (Q4) If Yes, was the method appropriate? (See dictionary) (C) Confounders (Q1) Were there important differences between groups prior to the intervention?
The following are examples of confounders: 1 Race 2 Sex 3 Marital status/family 4 Age 5 SES (income or class) 6 Education 7 Health status 8 Pre-intervention score on outcome measure Can't tell No description of patient demographics presented. (Q2) If yes, indicate the percentage of relevant confounders that were controlled (either in the design (e.g. stratification, matching) or analysis)?
Can't tell (D) Blinding (Q1) Was (were) the outcome assessor(s) aware of the intervention or exposure status of participants?
Can't tell Registers are filled by the hospital staff who knew the assessment plan but did not know the outcome data were coming from admission records books. Page 3-4.
(Q2) Were the study participants aware of the research question? No Unlikely as individual consent not sought and data collection relied on retrospective register reviews. Patient numbers, age, length of stay and pneumonia severity were similar between hospital cohorts both preintervention and postintervention. There was a reduction in the proportion of severe/very severe pneumonia cases in the postintervention era. Page 5. (Q2) If yes, indicate the percentage of relevant confounders that were controlled (either in the design (e.g. stratification, matching) or analysis)?
Can't tell Analysis stratified by severity of pneumonia but did not discuss any other methods used to account for confounders.

(D) Blinding
(Q1) Was (were) the outcome assessor(s) aware of the intervention or exposure status of participants?
Can't tell Data were collected by a paediatric resident or paediatrician familiar with WHO classification of pneumonia and trained to identify signs or symptoms of pneumonia from records. Page 3.
Likely that data collectors knew that facility had received oxygen intervention or not but not explicitly addressed.
(Q2) Were the study participants aware of the research question? No Unlikely as individual consent not sought and data collection relied on retrospective case note reviews. (E) Data collection methods (Q1) Were data collection tools shown to be valid? Yes Case note reviews seem to be valid and reliable and data collectors were trained pediatricians.
(Q2) Were data collection tools shown to be reliable? No The accuracy and detail of medical records limited our ability to accurately determine pneumonia severity or patient outcomes in some instances. Medical records represented those we could locate; we cannot be sure how many children were admitted with pneumonia in the 20 hospitals in each of the two time periods. Page 8. (F) Withdrawals and drop outs (Q1) Were withdrawals and drop-outs reported in terms of numbers and/or reasons per group?
Can't tell The accuracy and detail of medical records limited our ability to accurately determine pneumonia severity or patient outcomes in some instances. Medical records represented those we could locate; we cannot be sure how many children were admitted with pneumonia in the 20 hospitals in each of the two time periods. Page 8.

COMPONENT RATINGS Gray 2017
BMJ Publishing Group Limited (BMJ) disclaims all liability and responsibility arising from any reliance Data collection method used retrospective case note reviews. (G) Intervention integrity (Q1) What percentage of participants received the allocated intervention or exposure of interest?
60 -79% (some) Documentation of SpO2 and O2 administration increased in the intervention arm but overall was still quite low. Less than 40% for documentation of SpO2 on all pneumonia and <50% for documentation of oxygen for hypoxemic patients. This may not reflect true intervention exposure but overall figures are low. Pages 6-7. (Q2) Was the consistency of the intervention measured? No (Q3) Is it likely that subjects received an unintended intervention (contamination or cointervention) that may influence the results?
No (H) Analyses (Q1) Indicate the unit of allocation (circle one) organization/institutio n Hospitals were selected to receive the intervention.
(Q2) Indicate the unit of analysis (circle one) organization/institutio n Summary data of mortality rates at hospital collected. Some individual data collected on severity of disease and age but outcome data was aggregated at facility level for analysis. (Q3) Are the statistical methods appropriate for the study design? Yes (Q4) Is the analysis performed by intervention allocation status (i.e. intention to treat) rather than the actual intervention received?
Yes All pneumonia patients in post-intervention arm included in the analysis (not only the ones that received oxygen Can't tell Patient register books used so no individual data on patient demographics available to compare pre + post. (Q2) If yes, indicate the percentage of relevant confounders that were controlled (either in the design (e.g. stratification, matching) or analysis)?
Can't tell (D) Blinding (Q1) Was (were) the outcome assessor(s) aware of the intervention or exposure status of participants?
Can't tell Clinical staff knew about the evaluation of the solar oxygen system; however, they did not know that outcome data were derived from ward admission record books. The accuracy with which data were recorded did not change during the study. There were significantly fewer children with pneumonia admitted in the postintervention era, and this requires some consideration. Page 5. (Q2) Were the study participants aware of the research question?
No (E) Data collection methods (Q1) Were data collection tools shown to be valid? Yes (Q2) Were data collection tools shown to be reliable? No The data were collected from the health facility admission and discharge record books, which are generally kept meticulously by senior nursing staff in PNG. Each facility has a record book, and details of every admission is entered manually; the data include patient name, contact address, diagnosis at admission and discharge, and outcome. Page 2.
Some facilities and periods excluded from analysis due to low quality data. (F) Withdrawals and drop outs (Q1) Were withdrawals and drop-outs reported in terms of numbers and/or reasons per group?
Can't tell See above on case note reviews. Some facilities and periods excluded due to missing or low quality registers.