ArticlesValidity of reported vaccination coverage in 45 countries
Introduction
Worldwide interest in the assessment of outcomes of health programmes is growing. For donor-supported initiatives like the Global Alliance for Vaccines and Immunizations (GAVI), which provides aid to strengthen countries' immunisation programmes, and the Global Fund to fight AIDS, Tuberculosis and Malaria, monitoring of results is essential. GAVI gives financial rewards to countries that increase the absolute number of children vaccinated and is testing methodological approaches to improve this process.1, 2, 3 With an increased focus on the monitoring of outcomes and their important implications for policy and operations, verification of the quality of the estimates used to assess performance of health programmes is necessary.4 We address the quality of data used to monitor outcomes of immunisation programmes.
There are two main sources of data used to assess coverage of immunisation programmes worldwide: health service delivery records and household-based surveys.5, 6 Countries are requested to report their vaccination coverage estimates every year to WHO and the United Nations Children's Fund (UNICEF) using the WHO/UNICEF joint reporting form on vaccine preventable diseases. Here, we refer to data from these forms as officially reported data.7, 8 Methods and strategies for collection and reporting of these data are specific to each country. The source of data for official reports can include service registries, surveys, or a combination of both. The target population in which vaccination coverage is assessed can also vary between countries, taking into account either yearly number of births, number of infants that survive their first year of life, or the number of children within a specific age range. Further, a country might change its methods for obtaining estimates from year to year.3, 9 The absence of standardisation in data sources and methods of collection decreases the comparability of officially reported data between countries and over time. Officially reported data tend to be the primary source of information for assessment of vaccination coverage,3, 8 and thus it is essential to analyse their validity.
Immunisation is also an important model for many other health programmes. There has been much investment in many countries to develop information systems to support programme implementation, monitoring, and evaluation. A careful assessment of the validity of coverage estimates collected from service providers will have important implications for improving the accuracy of health information systems.
The main objective of our analysis is to assess the validity of officially reported vaccination coverage data by comparing them with the best available gold standard. Nationally representative household-based surveys such as the Demographic and Health Surveys (DHS), one of the largest programmes for collection of quantitative data on population, health, and nutrition in the developing world, can serve as such a gold standard. Use of data from the DHS allows comparative analyses, since the surveys use standardised instruments, training, data collection, and data processing.10 Results of studies, which have validated DHS methodology and assessed the quality of DHS vaccination data, have shown little evidence of systematic bias.11, 12 Furthermore, since data compiled through the DHS are generally nationally representative, they capture vaccinations delivered by both the private and public sectors, whereas officially reported data often report only vaccinations given via public programmes.3
Diphtheria-tetanus-pertussis (DTP3) and measles vaccinations are most often used to monitor childhood vaccination coverage rates and trends.3, 5 We focus on DTP3 and compare the officially reported DTP3 coverage with that from the DHS.
Section snippets
Methods
We used vaccination data from 67 DHS done in 45 countries (panel) from 1990 to 2000. We excluded six surveys because data were not nationally representative, officially reported data were not available for the corresponding years, or the surveys followed a local calendar system for date-related information.
The DHS used a two-stage sampling scheme, with selection at the first stage from primary sampling units or clusters, followed by the second stage of random selection of households within each
Role of the funding source
The survey data used in the analysis were obtained from a secondary source, Macro International, compiled under financial support of the United States Agency for International Development.
Results
The total valid DTP3 rates range from 11% to 77% between countries (figure 1). Total valid vaccination rates by birth cohort were between the crude rates, which ranged from 17% to 92%, and documented valid rates, which range from 2% to 74%. The large variation in vaccination coverage was not surprising in view of the wide differences between countries in sociodemographic characteristics and degrees of health system development. For example, the rate of secondary education in mothers ranged from
Discussion
Our results show that officially reported data could be misleading in assessment of changes in vaccinations over time.
Rates of DTP3 coverage as reported via WHO/ UNICEF joint reporting form on vaccine preventable diseases are higher than those derived from a household survey, the DHS. Officially reported rates of vaccination coverage are highly informative if they are low. However, if reported coverage is high, total valid coverage in the population might, in fact, be rather low. More
References (29)
Introducing GAVI and the Global Fund for Children's Vaccines
Vaccine
(2000)- et al.
GAVI, the first steps: lessons for the Global Fund
Lancet
(2002) Assessing immunization coverage: how and why?
Vaccine
(1998)- et al.
The demand for deductibles in private health insurance: a probit model with sample selection
J Econometrics
(1981) - et al.
The accuracy of mother's reports of child vaccination: evidence from rural Egypt
Soc Sci Med
(1998) Immunization data quality audit evaluation report: final report from Deloitte Touche Tohmatsu Emerging Markets, Euro Health Group, and Liverpool Associates in Tropical Health
(2001)Routine immunization: estimates on immunization coverage 1980–1999
- et al.
Surveillance for the expanded programme on immunization
Bull World Health Organ
(1993) Immunization policy: publication WHOGPV/GEN/95.03 REV 1
(1996)Informal meeting on issues of immunization data and monitoring/management of adverse events: publication WHO/V&B/99.06
(1999)