The reliability of self-assessed health status

https://doi.org/10.1016/S0167-6296(02)00007-3Get rights and content

Abstract

The use of self-assessed health status (SAHS) as a measure of health is common in empirical research. We analyse a unique Australian survey in which a random sub-sample of respondents answer a standard self-assessed health question twice—before and after an additional set of health related questions. A total of 28% of respondents change their reported health status. Response reliability is related to age, income and occupation. We also compare the responses of these individuals to other respondents who are queried only once, to isolate effects of question order and mode of administration.

Introduction

Self-assessed health status (SAHS) is an increasingly common measure of health in empirical research (e.g. Smith, 1999, Kennedy et al., 1998, Deaton and Paxson, 1998, Schofield, 1996, Ettner, 1996, Saunders, 1996). This is supported by a literature that shows that SAHS predicts mortality and morbidity (e.g. Idler and Kasl, 1995, McCallum et al., 1994, Connelly et al., 1989, Okun et al., 1984). Furthermore, Gerdtham et al. (1999) have demonstrated that a continuous health status measure constructed from a categorical response by the method of Wagstaff and van Doorslair (1994) is highly correlated with other continuous measures of health.1

The 1995 Australian National Health Survey provides a unique opportunity to examine SAHS measures in a different way. The following standard SAHS question was asked of all respondents and twice of a random subsample:

In general, would you say that your health is

  • excellent?

  • very good?

  • good?

  • fair?

  • poor?

For the “treatment” group, this question is first asked at the beginning of a general health and well being questionnaire.2 The question is asked again after the respondent has completed the general health and well being questionnaire and answered some other non-health related questions. The distributions of responses to these two questions are statistically different. In addition, both distributions are statistically different from the distribution of responses by the group that was asked only once.

Among respondents who were asked the self-reported health status question twice, approximately 28% change their response, though, only 3% change their response by more than one category. Some socio-economic groups are more likely than others to revise their SAHS on repeated questioning. For example, a higher proportion of older than younger persons change their SAHS.

These patterns of responses and changes in responses admit several interpretations and may have implications for empirical research employing self-assessed health measures. Before turning to those interpretations and implications, we provide further detail on the data, and the patterns therein.

Section snippets

The 1995 Australian National Health Survey

The Australian Bureau of Statistics (1995) National Health Survey3 was conducted over 12 months period from January 1995 to January 1996 and based on a sample of private and non-private dwellings. Approximately 23,800 dwelling households were surveyed and the overall response rate for households was 91.5%. An important feature of the data is that they are representative of the entire Australian adult population which allows us to compare the

The distribution of responses by question and group

The distribution of responses to both self-assessed health questions for the treatment group and to the single self-assessed health question in the case of the control group are presented in Table 2. In Table 3, we present tests for differences in distribution across questions within the treatment group, and across the treatment and control groups.

Comparing the distributions of responses to the two questions for the treatment group, we cannot reject the null that the means are the same (at a 5%

Who revises their self-assessed health status?

As noted in the introduction, 28% of the treatment group change their response between the two SAHS questions. A total of 13.6% reported a higher level of health whilst 14.8% reported a lower level of health. These gross flows are large relative to the net changes discussed above (in Table 2, responses in the top two categories combined went up by 1% point between the first and second question, while responses to the bottom to categories increased by a combined 2.7% points). There is evidently

Interpretations

To summarise, we find that the distribution of SAHS responses differs between:

  • 1.

    a form-based questionnaire and a face-to-face question separated by the SF-36 form-based questions asked of the same individuals;

  • 2.

    form-based and face-to-face questions asked of two randomly allocated groups;

  • 3.

    face-to-face questions asked of a control group and a randomly selected treatment group which was “pre-treated” with form-based questions.

We also find that for the treatment group, changes in self-assessed health

Implications for empirical research

Our results suggest that individuals’ responses to a self-assessed health question depend on both the nature of the survey (particularly whether responses are written or verbal) and the sequence of preceding questions. These are important factors to keep in mind when comparing the distribution of SAHS across different surveys. Many health surveys contain multiple measures of health status. Our results suggest that it is necessary to consider the order in which health status questions are asked

Acknowledgements

The first author gratefully acknowledges the financial support of the Social Sciences and Humanities Research Council of Canada (SSHRC), the Social and Economic Dimensions of an Ageing Population research program at McMaster University, and the Social Policy Evaluation, Analysis and Research Centre at The Australian National University. We thank the Australian Bureau of Statistics and seminar participants at The Australian National University and particularly Jeff Borland, Deborah Cobb-Clark,

References (22)

  • S.L. Ettner

    New evidence on the relationship between income and health

    Journal of Health Economics

    (1996)
  • Australian Bureau of Statistics, 1995. National Health Survey Users’ Guide. ABS Catalogue No. 4363.0, Australian...
  • J. Bound

    Self-reported versus objective measures of health in retirement models

    Journal of Human Resources

    (1991)
  • J. Cohen

    A coefficient of agreement for nominal scales

    Educational and Psychological Measurement

    (1960)
  • J. Cohen

    Weighted kappa: nominal scale agreement with provision for scales disagreement or partial credit

    Psychological Bulletin

    (1968)
  • J.E. Connelly et al.

    Health perceptions of primary care patients and the influence on health care utilisation

    Medical Care

    (1989)
  • Currie, J., Madrian, B.C., 1999. Health, health insurance and the labor market. In: Ashenfelter, O., Card, D. (Eds.),...
  • A.S. Deaton et al.

    Ageing and inequality in income and health

    American Economic Review, Papers and Proceedings

    (1998)
  • U.G. Gerdtham et al.

    A note on validating Wagstaff and van Dooslair’s health measure in the analysis of inequality in health

    Journal of Health Economics

    (1999)
  • P. Grootendorst et al.

    Does it matter whom and how you ask? Inter and intra-rater agreement in the Ontario health survey

    Journal of Clinical Epidemiology

    (1997)
  • E.L. Idler et al.

    Self-ratings of health: do they also predict change in functional ability?

    Journal of Gerontology

    (1995)
  • Cited by (290)

    • Is inconsistent reporting of self-assessed health persistent and systematic? Evidence from the UKHLS

      2023, Economics and Human Biology
      Citation Excerpt :

      Because older individuals expect to face health problems, younger individuals tend to interpret the information about their own health differently (see Jylhä, 2009). Nonetheless, it has been shown that males are more likely to inconsistently report their SAH when compared to women (see Black, Johnston, Suziedelyte, 2017; Black, Johnston, Shields, Suziedelyte, 2017; Clarke and Ryan, 2006; Crossley and Kennedy, 2002). Two measures are used to capture the socioeconomic status of the respondents: highest educational attainment (degree, other higher qualification, A-level, GCSE, other lower qualification, no qualification) and household income (equivalized using the modified OECD scale and deflated).

    View all citing articles on Scopus
    View full text