Design options for sample-based mortality surveillance

Abstract

Information on causes of death in a population is fundamental to health policy development, implementation, and evaluation.¹ The best source of such information is vital registration in which every death in the population is medically certified as to its causal antecedents. Recent assessments indicate that globally only about one-third of countries have registration systems that yield adequate data on causes of death.² While the number of countries meeting this criterion has increased from 50 to 115 in the five decades the World Health Organization (WHO) has been monitoring national mortality data,³ there is still no information regarding causes of death in many countries in Africa, South-east Asia, and the Pacific region, and only limited information from certain countries in the Middle East and Latin America. Existing policies for health gain in these regions would undoubtedly benefit from improvements in the availability and quality of cause of death information.

Although the development of high-quality population-wide vital registration systems takes decades to achieve and requires ongoing commitment in terms of resources, it is increasingly being recognized that the attainment and maintenance of vital registration systems is imperative for all governments.⁴ However, countries currently without such systems will require substantial external assistance in order to realize this vision.

It is fortunate, therefore, that models other than population-wide vital registration exist. Experience from the Sample Registration System in India has shown that continuous mortality surveillance in a nationally representative sample of the population is a useful method for monitoring mortality trends over time and differentials between subgroups.⁵ Similarly, the Disease Surveillance Point system in China generates useful data on causes of death from a 1% representative sample of the national population.⁶ Such approaches demonstrate the feasibility of sample-based approaches to mortality surveillance for generating valid and reliable information on causes of death, particularly in situations where the establishment of a registration system for an entire population is unlikely to occur in the short to medium term. Sample-based mortality surveillance has the added benefit of providing the basis for more complete registration over the longer term.

Clearly, sample-based mortality surveillance can only yield useful information when good design principles are followed. Hauser identified this issue >50 years ago in his discussion of sample-based approaches to all-cause mortality estimation.⁷ More recently, Kaufman and colleagues discuss sample size estimation with reference to cause-specific mortality.⁸ They take an approach that assumes randomness in the variability of cause of death information but do not provide methods for deriving prior information on cause composition in a population, which is a prerequisite for the application of their methods.

Chandramohan et al.⁹ on the other hand, articulate the principle of sufficient numbers of deaths for the rarest cause of interest. This observation, made in relation to validating verbal autopsy instruments (a technique whereby cause of death is determined by a physician based on the responses of family members to a structured set of questions about symptoms experienced by the deceased in the period prior to death), is pertinent to the design of sample registration systems: sampling necessarily increases uncertainty; it is prudent to know a priori by how much and whether or not this matters. Our review of the literature suggests that those faced with designing sample-based systems would benefit from a systematic approach to these issues. In this paper, we provide practical methods for determining efficient sizes for sample-based mortality surveillance systems, particularly in situations where prior information on the cause composition of mortality is lacking.

Methods

We begin by observing that one purpose of sample-based mortality surveillance is to generate robust age-specific and sex-specific estimates of important causes of death from a representative subset of a population. To achieve this purpose efficiently, a system need only accumulate enough person-years of observation to enable the generation of robust estimates for the least frequently occurring cause of death for which certainty is important (i.e. the rarest cause of interest). Requirements for the design of such a system are: a simple measure of uncertainty; prior information on the frequency of mortality by age, sex, and cause in the population; and some knowledge about which causes of death are important and at what ages.

We demonstrate the implications of these observations by designing systems for three hypothetical populations, each representing different levels of health development according to the WHO classification of national mortality characteristics.¹⁰ Population B represents countries with low child mortality and low adult mortality, Population D represents countries with high child mortality and high adult mortality, and Population E represents countries with high child mortality and very high adult mortality (i.e. countries in which HIV/AIDS and possibly malaria are endemic). Additional information on the health and economic development of each population is provided in Table 1.

Our objective is to estimate the total person-years of observation needed in each of these populations in order to derive a predetermined set of age-specific and sex-specific estimates with tolerable margins of error. In addition, we are interested in knowing the effect on person-years of pragmatic but, in our view, acceptable compromises to the comprehensiveness of this set in terms of the number of age and sex groups for which error is tolerable.

Uncertainty

The most robust estimates of cause-specific mortality in a population are derived from population-wide registration systems in which the main sources of uncertainty are causal attribution and stochastic processes (i.e. random variability). Uncertainty in the former is largely a function of the thoroughness with which physicians record the sequence of events prior to death, as well as the availability and quality of evidence available at the time of certification. While undoubtedly critical to the reliability of any source of data on causes of death, this is unrelated to statistical aspects of uncertainty and is not within the scope of this paper. Uncertainty in the latter exists even in population-wide systems, but sample-based approaches increase this by reducing the opportunity for observing deaths. Sampling also introduces uncertainty with regard to the extent to which the population being observed (i.e. the sample) is representative of the total population.

If we assume the observed population is representative, then random variability in the occurrence of events within that population over time can be quantified using simple statistical probability models. The Poisson distribution assumes events occur independently of each other and randomly in time, and is commonly used to describe stochastic uncertainty in mortality rates.¹¹ A defining characteristic of the Poisson distribution is that its mean is equal to its variance. When the rate of events in a population fluctuates randomly, the variance exceeds the mean, in which case other distributions (e.g. the negative binomial distribution) may be more appropriate. Given it is reasonable to assume mortality as an underlying force in a population is relatively stable, particularly at the levels we consider in this paper, we see compelling theoretical grounds for choosing the Poisson over other distributions to quantify random variability in the occurrence of deaths.

Figure 1

Open in new tabDownload slide

Relative Standard Errors for different frequencies of a Poisson parameter

The area below the bold line in Figure 2 represents event rates for which there is insufficient exposure to achieve an RSE of ≤15%. For a given exposure of 100 000 person-years, Figure 2 shows that an event rate of 10 per 100 000 person-years would yield an RSE >15%, whereas an event rate of 100 per 100 000 person-years would yield an RSE of ≤15%. Since we assume the underlying event rate is invariant in a population, only exposure will influence RSE. In mortality surveillance, exposure has two dimensions; the number of people being observed and the period of observation. Adjusting either dimension until the intersection between the rate for the rarest cause of interest and exposure falls exactly on this line will result in a design for which efficiency is maximized. We adopt this algorithm to determine the required person-years in each of our populations.

Figure 2

Open in new tabDownload slide

Combinations of exposure and event rates (and corresonding 95% exact confidence inervals) for which the Relative Standard Error (RSE) is 15%

Prior estimates of all-cause mortality

Ideally, prior estimates of age-specific and sex-specific all-cause mortality should come from empirical sources. In countries without reliable vital registration systems, observed mortality rates must first be assessed for completeness and corrected accordingly before being used as estimates of all-cause mortality. In the absence of usable information from such sources, we advocate the use of indirect methods. Model life table systems provide indirect means for deriving complete schedules of age-specific and sex-specific rates from, at a minimum, estimates of childhood mortality.¹³ Most countries are able to derive this information from local sources (e.g. Demographic and Health Surveys). To demonstrate the feasibility of this approach, we use model life tables based on the probabilities of dying during childhood and at adult ages in Table 1 to derive age-specific and sex-specific all-cause mortality rates in each of our example countries. The specific approach we adopt is the Modified Logit Life Table System currently used by WHO, a two-parameter model based on empirical mortality data from mostly developed countries over the period from 1950 to 2002.¹⁴

Prior estimates of cause-specific mortality

Prior estimates of the cause structure of mortality by age and sex should also come from empirical sources (e.g. research studies, population laboratories, or neighbouring countries). While such information is generally available for most countries, it is typically less reliable than data on all-cause mortality and should be thoroughly checked for plausibility. (Guidelines for assessing the quality of cause-specific information are available from the authors.) In the absence of plausible local cause-specific information, we again advocate the use of model-based methods, as discussed below.

Models relating all-cause to cause-specific mortality enable the cause structure of mortality by age and sex to be predicted for a given level of all-cause mortality. Work by Preston¹⁵ and Lopez and Hull¹⁶ builds on the observation that economic development is associated with a shift from infectious to chronic diseases as leading causes of death, a phenomenon referred to as the 'epidemiologic transition'.¹⁷ More recently, Murray and Lopez¹⁸ and Salomon and Murray¹⁹ have proposed cause of death models (e.g. CODMOD) [Cause Of Death MODel (CODMOD) is a statistical model for predicting the proportionate mortality distribution in populations across broad causes (communicable, non-communicable, and injuries) as a function of total mortality and level of development (income)] that include national per capita income as well as all-cause mortality as predictors. These models typically generate estimates by age and sex across very broad cause groupings and are consistent with the philosophy of the original Global Burden of Disease Study that stressed the importance of first getting broad cause of death categories correct so that estimation errors were constrained to diseases with roughly similar public health policy implications.¹⁸ This approach avoids the bias inherent in many cause of death estimates, towards over-emphasis of communicable diseases at the expense of non-communicable diseases and, especially, injuries.²⁰

To demonstrate the feasibility of model-based approaches to estimating causes of death, we use the age-specific and sex-specific all-cause mortality rates from our model-derived life tables and the estimates of Gross Domestic Product (GDP) from Table 1 as inputs into the Salomon and Murray cause of death model so as to estimate expected cause-specific rates by age and sex in our example countries. These rates are then applied to UN population structures²¹ (also shown in Table 1) so as to derive the expected number of deaths in each age, sex, and cause stratum for any given value of person-time accumulated during surveillance.

Age, sex, and cause groups of interest

Decisions regarding the age, sex, and cause groups for which estimates are to be derived with certainty through sample-based methods might justifiably be regarded as contingent upon local considerations. At a minimum, we advocate adopting the broad-level objectives outlined in Table 2, which are intended to address basic policy concerns in populations with similar mortality structures as our example countries. Apart from Group I causes in Population E, we chose broad cause groupings over more specific causes for describing a population in terms of its progress through the epidemiological transition. HIV/AIDS and malaria are identified separately in Population E because of the particular policy relevance of these causes in populations with high child mortality and very high adult mortality. We used WHO sources to derive the expected proportions of mortality associated with these causes.³ As a general principle, we strongly advocate consideration of broad-level priorities over more detailed priorities in the initial stages of sample-based mortality surveillance, unless prior information on the expected frequency of important specific causes is compelling. This is not to deny that information on more detailed causes of death is critical for most health planning purposes, but for general policy and priority setting in health, knowledge about the comparative magnitude of broader causes is probably sufficient.

Results

Modelled life expectancy at birth increased as the probability of dying during childhood and at ages 15–60 decreased across our three example populations (Table 3) and was within expected ranges based on published estimates for countries with comparable mortality.¹⁰ This increase was associated with a decline in the prominence of Group I causes, and to a lesser extent Group III causes, compared with Group II causes, reflecting the expected impact on causes of death of overall improvements in health (Table 3).

Table 4 presents the total person-years of observation required in each population in order to achieve an RSE of ≤15% across the set of age, sex, and cause groups listed in Table 2. We refer to this as an 'optimal' design solution for sample-based mortality surveillance in these populations. In Populations D and E, the 'optimal' approach achieves tolerable uncertainty in more groups than specified in Table 2 owing to the minimum exposure needed to achieve an RSE of 15% in the rarest group of interest for these populations. If a limited number of age and sex groups are excluded from the set, the total exposure required in each of the three populations is more than halved to between 600 000 and 800 000 people, annually. This is typically only a minor fraction (2–3%) of the population of most developing countries, and in many cases, represents <1% of the total population. Moreover, it is unlikely that annual data are strictly necessary for the control of major endemic diseases, and hence the required population could be more than halved by grouping observations over a period of (breve)2 years. Experience in India,⁵ China,²² and Tanzania²³ suggests that these numbers are achievable.

Discussion

Health systems must meet numerous demands for health care services, often with limited resources. The economic and political constraints surrounding the provision of health care are well known, and health systems are often stretched to provide even essential services, with little or no capacity to address endemic diseases at the population level. The framework proposed in this paper is consistent with the philosophy of establishing priorities across broad clusters of causes for which the public health response implications are essentially similar, and the correct determination of which will avoid serious misunderstanding of the extent of epidemiological transition in populations. Its application should provide guidance to policy makers as to the minimum person-years necessary for mortality surveillance systems to yield useful information at the population rather than health service delivery level. Once these priorities have been established with adequate certainty, the measurement of more detailed causes of relevance to specific disease control initiatives is likely to be feasible using disease-specific surveillance data, disease modelling, or more extensive verbal autopsy with medical certification and review.^18,20,24,25

Recent advances in the development of verbal autopsy procedures^24,26–28 will further increase the viability of this approach as a cost-effective means of cause ascertainment over complete medical certification for population health monitoring purposes. Regardless of the orientation of a particular surveillance system with respect to methods for reducing uncertainty in causal attribution, it is the underlying frequency of events, not population size, which remains fundamental to stochastic uncertainty. Mortality surveillance systems to date have generally been determined by the size of a population within a given administrative area, and have not taken into account the number of deaths needed to yield information that is sufficiently robust. The framework we propose in this paper addresses this basic requirement.

The importance of monitoring health development in populations, and in particular, the effectiveness of disease control strategies, is clear from the rapid expansion of HIV/AIDS, particularly in Southern Africa, in the early 1980s. After almost a quarter of a century, there remains vast uncertainty about the pace and extent of mortality associated with this epidemic, with implications for targeting control strategies, largely because of the continued absence of adequate mortality information in this region. Similarly, there is great uncertainty about whether malaria mortality in Africa is rising, or not, and little is reliably known about the pattern of injury deaths, or indeed, the emergence of chronic diseases.

Most countries are already investing resources into mortality surveillance activities of one sort or another. Our framework could readily be used to reorient this infrastructure so as to greatly enhance its efficiency and relevance for health development. The average population size of the 20 or so demographic surveillance sites, largely in Africa, of INDEPTH (International Network for the continuous Demographic Evaluation of Populations and Their Health), for example, is ∼40 000.²⁹ Notwithstanding the fact that there may be complete enumeration in many of these sites, our analyses suggest that these numbers would need to be increased 10-fold in order to adequately monitor changes in causes of death. This would vastly reduce persistent ignorance about health conditions throughout Africa and could be achieved with increased and longer-term commitments from the global donor community. Our focus, in the short-term, is the Asia-Pacific region, where we intend applying the framework to the design of surveillance sites as part of collaborative capacity building efforts already underway in the Philippines (Bohol), Bangladesh (MATLAB), and Indonesia.

Large-scale WHO programme such as the 3 by 5 Initiative against HIV/AIDS, the Stop TB programme, Roll Back Malaria, and the Safe Motherhood Initiative urgently require mortality information to evaluate progress and guide planning and implementation in developing countries. Major health information initiatives, such as the WHO Health Metrics Network and the United States Census Bureau-sponsored Sample Vital Registration with Verbal Autopsy (SAVVY) Programme, are currently being oriented towards establishing sample-based registration in a number of countries in response to this need. We have shown in this paper how such approaches can be designed to maximize efficiency. Competent, scientifically based surveillance that yields sufficiently reliable and relevant information for programmes action is well within the means of many developing countries. Indeed, such systems represent the only useful alternative to establish the evidence base for health policy and programme delivery for the foreseeable future in much of the developing world.

This work has been supported by the United States National Institute of Ageing Grant PO1-AG17625. We are grateful to Gail Williams, Professor of International Health Statistics at the School of Population Health, University of Queensland, for her contributions to the conceptual stages of this work.

References