Elsevier

The Lancet

Volume 362, Issue 9389, 27 September 2003, Pages 1022-1027
The Lancet

Articles
Validity of reported vaccination coverage in 45 countries

https://doi.org/10.1016/S0140-6736(03)14411-XGet rights and content

Summary

Background

Monitoring and assessment of coverage rates in national health programmes is becoming increasingly important. We aimed to assess the accuracy of officially reported coverage rates of vaccination with diphtheria-tetanus-pertussis vaccine (DTP3), which is commonly used to monitor child health interventions.

Methods

We compared officially reported national data for DTP3 coverage with those from the household Demographic and Health Surveys (DHS) in 45 countries between 1990 and 2000. We adjusted survey data to reflect the number of valid vaccinations (ie, those administered in accordance with the schedule recommended by WHO) using a probit model with sample selection. The model predicted the probability of valid vaccinations for children, including those without documented vaccinations, after correcting for bias from differences between the children with and without documented information on vaccination. We then assessed the extent of survey bias and differences between officially reported data and those from DHS estimates.

Findings

Our results suggest that officially reported DTP3 coverage is higher than that reported from household surveys. This size of the difference increases with the rate of reported coverage of DTP3. Results of time-trend analysis show that changes in reported coverage are not correlated with changes reported from household surveys.

Interpretation

Although reported data might be the most widely available information for assessment of vaccination coverage, their validity for measuring changes in coverage over time is highly questionable. Household surveys can be used to validate data collected by service providers. Strategies for measurement of the coverage of all health interventions should be grounded in careful assessments of the validity of data derived from various sources.

Introduction

Worldwide interest in the assessment of outcomes of health programmes is growing. For donor-supported initiatives like the Global Alliance for Vaccines and Immunizations (GAVI), which provides aid to strengthen countries' immunisation programmes, and the Global Fund to fight AIDS, Tuberculosis and Malaria, monitoring of results is essential. GAVI gives financial rewards to countries that increase the absolute number of children vaccinated and is testing methodological approaches to improve this process.1, 2, 3 With an increased focus on the monitoring of outcomes and their important implications for policy and operations, verification of the quality of the estimates used to assess performance of health programmes is necessary.4 We address the quality of data used to monitor outcomes of immunisation programmes.

There are two main sources of data used to assess coverage of immunisation programmes worldwide: health service delivery records and household-based surveys.5, 6 Countries are requested to report their vaccination coverage estimates every year to WHO and the United Nations Children's Fund (UNICEF) using the WHO/UNICEF joint reporting form on vaccine preventable diseases. Here, we refer to data from these forms as officially reported data.7, 8 Methods and strategies for collection and reporting of these data are specific to each country. The source of data for official reports can include service registries, surveys, or a combination of both. The target population in which vaccination coverage is assessed can also vary between countries, taking into account either yearly number of births, number of infants that survive their first year of life, or the number of children within a specific age range. Further, a country might change its methods for obtaining estimates from year to year.3, 9 The absence of standardisation in data sources and methods of collection decreases the comparability of officially reported data between countries and over time. Officially reported data tend to be the primary source of information for assessment of vaccination coverage,3, 8 and thus it is essential to analyse their validity.

Immunisation is also an important model for many other health programmes. There has been much investment in many countries to develop information systems to support programme implementation, monitoring, and evaluation. A careful assessment of the validity of coverage estimates collected from service providers will have important implications for improving the accuracy of health information systems.

The main objective of our analysis is to assess the validity of officially reported vaccination coverage data by comparing them with the best available gold standard. Nationally representative household-based surveys such as the Demographic and Health Surveys (DHS), one of the largest programmes for collection of quantitative data on population, health, and nutrition in the developing world, can serve as such a gold standard. Use of data from the DHS allows comparative analyses, since the surveys use standardised instruments, training, data collection, and data processing.10 Results of studies, which have validated DHS methodology and assessed the quality of DHS vaccination data, have shown little evidence of systematic bias.11, 12 Furthermore, since data compiled through the DHS are generally nationally representative, they capture vaccinations delivered by both the private and public sectors, whereas officially reported data often report only vaccinations given via public programmes.3

Diphtheria-tetanus-pertussis (DTP3) and measles vaccinations are most often used to monitor childhood vaccination coverage rates and trends.3, 5 We focus on DTP3 and compare the officially reported DTP3 coverage with that from the DHS.

Section snippets

Methods

We used vaccination data from 67 DHS done in 45 countries (panel) from 1990 to 2000. We excluded six surveys because data were not nationally representative, officially reported data were not available for the corresponding years, or the surveys followed a local calendar system for date-related information.

The DHS used a two-stage sampling scheme, with selection at the first stage from primary sampling units or clusters, followed by the second stage of random selection of households within each

Role of the funding source

The survey data used in the analysis were obtained from a secondary source, Macro International, compiled under financial support of the United States Agency for International Development.

Results

The total valid DTP3 rates range from 11% to 77% between countries (figure 1). Total valid vaccination rates by birth cohort were between the crude rates, which ranged from 17% to 92%, and documented valid rates, which range from 2% to 74%. The large variation in vaccination coverage was not surprising in view of the wide differences between countries in sociodemographic characteristics and degrees of health system development. For example, the rate of secondary education in mothers ranged from

Discussion

Our results show that officially reported data could be misleading in assessment of changes in vaccinations over time.

Rates of DTP3 coverage as reported via WHO/ UNICEF joint reporting form on vaccine preventable diseases are higher than those derived from a household survey, the DHS. Officially reported rates of vaccination coverage are highly informative if they are low. However, if reported coverage is high, total valid coverage in the population might, in fact, be rather low. More

References (29)

  • MW Borgdoff et al.

    Estimating vaccination coverage: routine information or sample survey?

    J Trop Med Hygiene

    (1988)
  • Model “A” questionnaire. MEASURE DHS + Basic Documentation series

    (2001)
  • JT Boerma et al.

    The quality of data on child immunisation in DHS-I surveys: an assessment of the quality of health data in DHS-I surveys. DHS Methodological Reports

    (1994)
  • J Brown et al.

    Assessment of the quality of estimates of child immunization coverage from population-based surveys: MEASURE Evaluation Working Paper series

    (2002)
  • Cited by (0)

    View full text