Inequalities in effective coverage measures: are we asking too much of the data?
•,.
...
Summary box
The need to shift to effective coverage measures has gained widespread acknowledgement. Effective coverage combines need, use and quality of care into a single metric to estimate the proportion of a population in need of a service that resulted in a positive health outcome from that service.
To support efforts towards universal health coverage, effective coverage measures should assess inequalities. At present, direct measures of equity, such as wealth, age, ethnicity, gender, education, place of residence, are only available in household data. However, population-level data alone do not provide information on all components of quality of care and may have poor validity.
Many measure of effective coverage require linking household data with information from health facilities on the quality of care provided. Health facility data provide a summary of quality of care at the facility-level and consequently ignores variation that may exist between service users with different characteristics.
Inequalities in effective coverage may be larger than we are able to demonstrate using existing data sources most commonly used to construct effective coverage measures.
Effective coverage measures combine need, use and quality of care into a single metric to estimate the benefit of a service or intervention. Effective coverage is defined as the proportion of the population in need of a service that resulted in a positive health outcome from that service.1 For reproductive, maternal, newborn, child health and nutrition (RMNCH+N) services and interventions, effective coverage can be defined using a cascade (see figure 1). Effective coverage is represented by the final step of the cascade, while the full cascade can be used to identify bottlenecks in implementation.
Overview of the data used to measure each step of the coverage cascade* and the stratification possible for a complementary feeding intervention and childbirth care. *Adapted from Marsh et al.1 DHS, Demographic and Health Surveys; MICS, Multiple Indicator Cluster Surveys; SARA, Service Availability and Readiness Assessments; SPA, Service Provision Assessments.
Universal health coverage means that high-quality interventions and services are available to all.2–4 Inequalities in the availability and quality of health services exist at all levels: between geographic regions, within geographic regions, and even within individual health facilities and families.5 To address inequalities effective coverage measures should be disaggregated by key sociodemographic and economic variables1—such as wealth, age, ethnicity, gender, education, place of residence.6
The potential to investigate inequalities in effective coverage is dependent on the data used to construct each step in the cascade. Here, we illustrate two methodological constraints that limit measuring inequalities in effective coverage when using: (1) only population-based data such as Demographic and Health Surveys (DHS) or Multiple Indicator Cluster Surveys (MICS) (eg, complementary feeding interventions) and (2) linked population and health facility data such as Service Provision Assessments (SPA) or Service Availability and Readiness Assessments (SARA) (eg, high-quality childbirth care), summarised in figure 1.
1. Population-level data alone do not provide information on all quality of care steps of the cascade and may have low validity.
A literature review of effective coverage measures revealed 14 studies that used only population-level data.7 A common example was treatment for malnutrition that typically reflected caregiver reports of whether nutritional interventions were received, whether children were ever given nutritional interventions, and whether the interventions were used appropriately in the household (see figure 1). Quality dimensions of health provider practise were not incorporated.
Since information on sociodemographic and economic variables is typically captured in household surveys, it is possible to stratify each of the relevant steps of the cascade by the desired measure of equity. However, household data provide no information on inputs and evidence on the validity of coverage data collected through household surveys suggest that while it can provide accurate coverage measures for some interventions, for many interventions household respondents cannot accurately report on quality of care dimensions.8–10 For the latter, alternative measurement approaches that link multiple data sources have been recommended.
2. Facility-level data do not include the individual data needed to track inequalities.
For many services (such as childbirth care), effective coverage measurement relies on linking data on access to care, derived from household surveys, where measures of inequality are incorporated, with information on the quality of care (inputs, interventions, process and experience of care) from health facility datasets.11–13 Health facility data, for example, nationally representative surveys such as SPA or SARA or indeed routine data sources such as District Health Information Software 2 (DHIS-2, do not report individual-level data but instead provide a summary of a facility’s capacity to provide high-quality care. Applying a facility-level score to each step of the cascade derived from health data (see figure 1), assumes that there are no systematic differences in the quality of care between individuals attending the same facility. However, evidence demonstrates that this is not the case; individuals with different characteristics receive different quality of care.5 14 Estimates of inequalities in effective coverage measures that are derived from linked household and facility data are driven only by the access to care measure.
There are further implications depending on the method applied for linking household and health facility data, whether: (1) individual or exact-match linking or (2) ecological linking.11–13
Exact-match linking of individuals in population data to the exact health facility they attended will capture systematic differences in care-seeking behaviour between individuals with different characteristics—for example, that wealthier individuals are more likely to bypass their nearest sources of care to seek higher quality care—either outside of their catchment area or at a higher level facility.5 15 Ecological linking—in which individuals from population data are linked to an average quality score across multiple health facilities—takes us a step further away, since it assumes there are no systematic differences in care-seeking behaviour between individuals with different characteristics. Adjusting for the type of facility that people report receiving care from has been demonstrated to generate valid measures of effective coverage, as likely accounts for some difference in care-seeking behaviour.12 13 Even so the approach ignores intersectionality and assumes that the quality and experience of care is homogeneous across facilities included in the average score, that is, that the average quality of primary healthcare facilities accessed by the wealthiest people is the same as the average quality of primary healthcare facilities accessed by the poorest people.
Herein lies the measurement dilemma. Relying on summary facility measures for linked effective coverage ignores variation in quality of care both within and (where using ecological linking approaches) between facilities. While generating effective coverage measures using only household data limits the adjustment made for quality and introduces issues with the reliability and validity of measures. In both scenarios, inequalities in effective coverage are driven only by the steps that use population data and are likely to be underestimated as a result. It is important to be mindful of which stratified analyses are feasible and what they are able to tell us about inequalities in effective coverage and refrain from asking too much of the data.
Effective coverage measures remain a crucial tool as we move towards universal access to high-quality care; we need to adjust coverage measures for the process and experience of care for individuals. Alongside continuing to promote effective coverage, we need to support the adoption of unique health identifiers that would allow us to link information on individuals’ care-seeking with information on the quality of care received. In the meantime, greater advocacy and investment in health information systems is needed to shift from reporting aggregated-level to individual-level data and to capture information on individual patients that would enable examination of inequalities within facilities.
Handling editor: Seye Abimbola
Contributors: JE and TM conceptualised the commentary. JE wrote the initial draft, and TM reviewed and edited the manuscript.
Funding: JE and TM work on the IDEAS (Informed Decisions for Actions to improve maternal and newborn health) project, funded through a grant from the Bill & Melinda Gates Foundation to the London School of Hygiene & Tropical Medicine. Gates Global Health Grant Number: OPP1149259/INV-007644.
Disclaimer: The funder played no role in the conceptualisation, the decision to publish or preparation of the commentary.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data availability statement
There are no data in this work.
Ethics statements
Patient consent for publication:
Not applicable.
Ethics approval:
Not applicable.
Acknowledgements
We would like to thank Joanna Schellenberg (LSHTM) and James Hargreaves (LSHTM) for their review of an earlier draft of this commentary.
Marsh AD, Muzigaba M, Diaz T, et al. Effective coverage measurement in maternal, newborn, child, and adolescent health and nutrition: progress, future prospects, and implications for quality health systems. Lancet Glob Health2020; 8:e730–6. doi:10.1016/S2214-109X(20)30104-2•Google Scholar•PubMed
Kruk ME, Gage AD, Arsenault C, et al. High-quality health systems in the sustainable development goals era: time for a revolution. Lancet Glob Health2018; 6:e1196–252. doi:10.1016/S2214-109X(18)30386-3•Google Scholar•PubMed
Requejo J, Diaz T, Park L, et al. Assessing coverage of interventions for reproductive, maternal, newborn, child, and adolescent health and nutrition. BMJ2020; 368:l6915. doi:10.1136/bmj.l6915•Google Scholar•PubMed
World Health Organization. Inequality monitoring in sexual, reproductive, maternal, newborn, child and adolescent health: a step-by-step manual. Geneva, World Health Organization2022; Google Scholar
Fink G, Kandpal E, Shapira G, et al. Inequality in the Quality of Health Services: Wealth, Content of Care, and Price of Antenatal Consultations in the Democratic Republic of Congo, Policy research working papers. Washington, DC, World Bank2019; Google Scholar
Barros AJD, Victora CG. Measuring coverage in MNCH: determining and interpreting inequalities in coverage of maternal, newborn, and child health interventions. PLoS Med2013; 10. doi:10.1371/journal.pmed.1001390•Google Scholar•PubMed
Exley J, Gupta PA, Schellenberg J, et al. A rapid systematic review and evidence synthesis of effective coverage measures and cascades for childbirth, newborn and child health in low- and middle-income countries. J Glob Health2022; 12. doi:10.7189/jogh.12.04001•Google Scholar•PubMed
Munos MK, Stanton CK, Bryce J, et al. Improving coverage measurement for reproductive, maternal, neonatal and child health: gaps and opportunities. J Glob Health2017; 7. doi:10.7189/jogh.07.010801•Google Scholar•PubMed
Bhattacharya AA, Allen E, Umar N, et al. Monitoring childbirth care in primary health facilities: a validity study in Gombe state, northeastern Nigeria. J Glob Health2019; 9. doi:10.7189/jogh.09.020411•Google Scholar•PubMed
Do M, Micah A, Brondi L, et al. Linking household and facility data for better coverage measures in reproductive, maternal, newborn, and child health care: systematic review. J Glob Health2016; 6. doi:10.7189/jogh.06.020501•Google Scholar•PubMed
Munos MK, Maiga A, Do M, et al. Linking household survey and health facility data for effective coverage measures: a comparison of ecological and individual linking methods using the multiple indicator cluster survey in Côte d'Ivoire. J Glob Health2018; 8. doi:10.7189/jogh.08.020803•Google Scholar•PubMed
Willey B, Waiswa P, Kajjo D, et al. Linking data sources for measurement of effective coverage in maternal and newborn health: what do we learn from individual- vs ecological-linking methods? J Glob Health2018; 8. doi:10.7189/jogh.06.0207028.010601•Google Scholar•PubMed
Mubiri P, Kajjo D, Okuga M, et al. Bypassing or successful referral? a population-based study of reasons why women travel far for childbirth in eastern Uganda. BMC Pregnancy Childbirth2020; 20. doi:10.1186/s12884-020-03194-2•Google Scholar•PubMed