Elsevier

Health Policy

Volume 91, Issue 3, August 2009, Pages 297-305
Health Policy

Assessing the reliability of household expenditure data: Results of the World Health Survey

https://doi.org/10.1016/j.healthpol.2009.01.002Get rights and content

Abstract

The World Health Survey (WHS) which has been implemented in more than 70 countries with standardized questionnaires opens a great opportunity for research on health care financing issues. This study examines the household expenditures and health expenditure collected in the WHS in terms of reliability, consistency between different ways of data collection within the survey and with other types of household surveys.

Data used in this study include 50 WHS and 37 other type of surveys, namely the Living Standard Measurement Survey, Household Budget Survey and Income and Expenditure Survey. The analysis consists of comparison of test–retest results; the aggregated and reported total household expenditure and health expenditure; the expenditures from the WHS and other type of surveys.

The results from test–retest are fairly similar in the WHS. For health expenditure the average of reported total is lower than the aggregated total while for household total expenditure the estimate is similar from the two measures. Finally the WHS was found to report lower total household expenditure but higher out-of-pocket expenditure comparing with other types of surveys. The study suggests further efforts to standardize the questions in collecting expenditure data in household surveys for the purpose of cross-country and over time comparison.

Introduction

Household expenditure data has been used extensively for monitoring general household living standards, well being and consumption patterns [1]. More recently, considerable attention has been paid for monitoring household expenditures on health with a view for determining if the need to pay for services prevents some people from seeking or continuing care, and results in severe financial hardship or impoverishment for others [2], [3], [4]. This literature has grown considerably over the last five years, with analysts using available expenditure data, including the Living Standard Measurement Survey (LSMS) supported by the World Bank, Household Budget Surveys (HBS), Income and Expenditure Surveys (IES) and Socio-Economic Surveys (SES) [5], [6], [7], [8].

There has long been a concern with the accuracy of expenditure data reported in household surveys, often linked to concerns about the abilities of households to remember a multitude of different types of expenditures accurately [9], [10], [11]. Measurement error can be introduced at any stage of a survey: design of the survey instrument, data collection, or data entry [12]. This is partly because the household expenditure surveys are among the most difficult and expensive surveys to field [13].

While these concerns are well established, there has been little attempt to understand the extent to which phrasing questions in different ways can influence the response to health expenditure questions, and whether different types of surveys produce consistent results.

We contribute to this literature by comparing two ways of seeking information on health expenditure developed in the World Health Survey (WHS), and then also consider the comparability of the estimated expenditures derived from other surveys undertaken in the same countries.

The WHS was launched by the World Health Organization to strengthen national capacity to monitor critical health inputs, outputs and outcomes [14]. They collected information on total household expenditure with a breakdown that included health expenditures, together with a wide range of indicators on health status, health service utilization, risk factors, and the perceived responsiveness of the health system. This makes the WHS appealing to policy makers and researchers seeking information on diverse topics including the assessment of inequality of health and in intervention coverage across different socio-economic groups. World Health Surveys have been implemented in 72 countries using standard questionnaires and many of the country data sets have recently been put into the public domain (http://www.who.int/healthinfo/survey/en/index.html).

Section snippets

Analysis framework

Reliability refers to the repeatability or consistency of a set of measurements or measuring instrument [15]. A measure is considered reliable if it would give us the same result over and over assuming that what we are measuring is not changing. Reliability could be characterized as either internal or external. Internal reliability compares two sets of data on the same subject using different measures. In World Health Survey we compared the difference between the total reported in response to

External reliability: results from test–retest in the WHS

Fig. 1 reports the ICCs for the test–retest responses for total household expenditure and expenditures on education, food and health. Each vertical bar depicts a country, and the range shows the 95% confidence intervals around the mean estimate of the ICC. For most countries, the average value of the ICC is above 0.6 for all items, which is generally considered to imply good external reliability [19], [20]. The lowest for household expenditure is 0.28, for food is 0.19, for education is 0.39

Discussion

The WHS will be a major source for health and health system related studies. In the area of health financing studies, including out-of-pocket health expenditure, financial catastrophe and impoverishment by health payment, the WHS has great potential to fill in the gaps where no appropriate household surveys exist or where the existing surveys are not up to date. Information on quality and the comparability of the WHS is crucial for researchers in analysing the data and interpreting the results.

Conclusion

The WHS has great applicability to a range of health care financing studies. Countries need timely information to evaluate their health policies, manage their health systems and monitor progress. The WHS may be best viewed as another source of survey data to supplement the information provided by routine national information systems.

In this study we found that the expenditure data in the WHS are reliable based on the test–retest estimates. The aggregated total gives higher non-zero response

Conflict of interest

No conflict of interest declared.

Acknowledgements

We thank Somnath Chatterji for his valuable comments in the early draft. The authors are also grateful to Nirmala Naidoo for her assistance on various questions related to the procedure in conducting the World Health Survey.

References (25)

  • P. Visaria

    Poverty and living standards in Asia

    Population and Development Review

    (1980)
  • S. Anand et al.

    Choosing a welfare indicator

    American Economic Review

    (1994)
  • Cited by (0)

    View full text