Review Article
Critical appraisal of health literacy indices revealed variable underlying constructs, narrow content and psychometric weaknesses

https://doi.org/10.1016/j.jclinepi.2010.04.005Get rights and content

Abstract

Objective

Health literacy refers to an individual's ability to seek, understand, and use health information. A range of indices exist that purport to measure health literacy across individuals and populations. This study aimed to review the development and content of existing indices and to critically appraise their properties.

Study Design and Setting

Using standardized search terms, published generic health literacy indices (1990–2008) were identified. Using a methodological framework, each was evaluated for purpose, validity (face, content, construct), reliability, responsiveness, feasibility, and generalizability.

Results

Nineteen instruments were evaluated. Three measurement approaches were identified: direct testing of individual abilities, self-report of abilities, and population-based proxy measures. Composition of underlying constructs and content varied widely across instruments, and none appeared to fully measure a person's ability to seek, understand, and use health information. The content was focused primarily on reading comprehension and numeracy; scoring categories were poorly defined and may not be mutually exclusive, and few indices had been assessed for reliability.

Conclusion

Health literacy is not consistently measured, making it difficult to interpret and compare health literacy at individual and population levels. Empirical evidence demonstrating validity and reliability of existing indices is required, and more comprehensive health literacy instruments need to be developed.

Introduction

What is new?

Key finding

  1. A key finding of this critical appraisal was that limited empirical evidence exists on the reliability and construct validity of health literacy measures. This raises uncertainty about the accuracy of data being produced in relation to health literacy levels at an individual and population level. Furthermore this research demonstrates that great variation exists across indices in terms of content across domains, methods of item selection and scoring and how individuals are categorised or classified in terms of health literacy levels. The variation and weak psychometric data makes it difficult to compare or pool results across studies and hinders the establishment of clear benchmarks for policy and program development aimed at addressing suboptimal health literacy.

What this adds to what is known?
  1. This appraisal, using a structured methodological framework, provides new and synthesised information for researchers and clinicians on the strengths and limitations of current indices of health literacy. Although previous literature acknowledges the narrow approach taken to the measurement of health literacy, the systematic presentation of the validity, reliability and applicability of these instruments provides more detail relating to psychometric properties of existing instruments.

What is the implication, what should change now?
  1. This review will support clinicians, researchers and policymakers to qualify the findings and recommendations derived from the use of health literacy indices. Further research is required to obtain empirical evidence across different populations and settings of the construct validity and reliability of existing measures. New measures which incorporate broader constructs of health literacy are required and are needed to advance this field.

As a result of health care changes toward a more patient-centered care approach [1], [2], there is an expectation that patients take an increased role in decisions about their health. For patients to function effectively within this shifting paradigm, they require a basic set of skills to seek, understand, and use health information, a concept referred to as “health literacy” [3], [4]. Suboptimal health literacy skills reduce the likelihood of maintaining good health, independent of other sociodemographic factors [5], [6], and are associated with increased health care costs [7].

A range of indices have been developed to measure health literacy [8], [9]. To provide credible information to inform clinical practice and health policies and programs, it is imperative that indices have sound psychometric properties. These include the following: validity—the extent to which an instrument measures what it purports to measure when properly administered [10]; reliability—the extent to which the obtained scores are free from measurement error [11]; feasibility—ease of administration; generalizability—use across and within fields; and in some circumstance, responsiveness—the ability of indices to detect change [12]. The purpose of this work was to review the literature on existing health literacy indices and perform a critical appraisal of their concept, content, and psychometric properties.

Section snippets

Method

Medline, PubMed, and PsychInfo databases were searched for publications of generic health literacy indices between 1990 and 2008 inclusive, using the following search terms: health, literacy, health literacy, instrument, tool, and assessment. The search was limited to publications in English language. Instruments that were not generic, that is, specific to particular fields or groups, were excluded. Additional instruments were identified through manual searching of the references of relevant

Questionnaires

Nineteen health literacy instruments met the prespecified selection criteria and were included in the appraisal (Fig. 1) (Tables 1, A1, and A2). There were 12 original instruments [8], [9], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23] and seven derivatives that were short-form versions or adaptations of original instruments [24], [25], [26], [27], [28], [29], [30]. There were also Spanish versions of two of the instruments [9], [16] and a Chinese version of a short-form instrument

Discussion

We have critically appraised the development and psychometric properties of 19 health literacy indices. Three approaches for measuring health literacy were identified: direct testing of an individual's abilities, self-report of abilities, and population-based proxy measures. The underlying constructs assessed varied widely across instruments and most were not based on a specific conceptual framework. The content focused primarily on reading, comprehension, and numeracy skills; scoring

Acknowledgments

Richard H. Osborne is supported in part by a National Health and Medical Research Council (NHMRC) Population Health Fellowship (Career Development Award), and Rachelle Buchbinder is supported in part by an NHMRC Practitioner Fellowship. Joanne E. Jordan is supported by an Australian Postgraduate Award.

References (72)

  • J.E. Jordan et al.

    Conceptualising health literacy from the patient perspective

    Patient Educ Couns

    (2010)
  • G. Eysenbach et al.

    How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews

    BMJ

    (2002)
  • S.K. Simonds

    Health education as social policy

    Health Educ Monogr

    (1974)
  • D. Nutbeam

    Health promotion glossary

    Health Promot Int

    (1998)
  • B.D. Weiss et al.

    Health status of illiterate adults: relation between literacy and health status among persons with low literacy skills

    J Am Board Fam Pract

    (1992)
  • D.A. Dewalt et al.

    Literacy and health outcomes: a systematic review of the literature

    J Gen Intern Med

    (2004)
  • USA National Academy on an Aging Society. Low health literacy skills Increase Annual Health Care Expenditures by $73...
  • T.C. Davis et al.

    Rapid assessment of literacy levels of adult primary care patients

    Fam Med

    (1991)
  • R.M. Parker et al.

    The test of functional health literacy in adults: a new instrument for measuring patients' literacy skills

    J Gen Intern Med

    (1995)
  • L. Del Greco et al.

    Questionnaire development: validity and reliability

    CMAJ

    (1987)
  • J. Nunnally

    Psychometric theory

    (1978)
  • J.C. Nunally et al.

    Psychometric theory

    (1994)
  • E.C. Hanson-Divers

    Developing a medical achievement reading test to evaluate patient literacy skills: a preliminary study

    J Health Care Poor Underserved

    (1997)
  • S.Y. Lee et al.

    Development of an easy-to-use Spanish Health Literacy test

    Health Serv Res

    (2006)
  • B.D. Weiss et al.

    Quick assessment of literacy in primary care: the Newest Vital Sign

    Ann Fam Med

    (2005)
  • L.D. Chew et al.

    Brief questions to identify patients with inadequate health literacy

    Fam Med

    (2004)
  • C. Norman et al.

    eHEALS: The eHealth Literacy Scale

    J Med Internet Res

    (2006)
  • H. Ishikawa et al.

    Measuring functional, communicative, and critical health literacy among diabetic patients

    Diabetes Care

    (2008)
  • A.D. Hanchate et al.

    The Demographic Assessment for Health Literacy (DAHL): a new tool for estimating associations between health literacy and outcomes in national surveys

    J Gen Intern Med

    (2008)
  • M. Kutner et al.

    The health literacy of America's adults: results from the 2003 National Assessment of Adult Literacy (NCES 2006-483)

    (2006)
  • R.E. Rudd

    Health literacy skills of U.S. adults

    Am J Health Behav

    (2007)
  • Health literacy in Canada: initial results from the International Adult Literacy and Skills Survey 2007

    (2007)
  • T.C. Davis et al.

    Rapid estimate of adult literacy in medicine: a shortened screening instrument

    Fam Med

    (1993)
  • T.C. Davis et al.

    Development and validation of the Rapid Estimate of Adolescent Literacy in Medicine (REALM-Teen): a tool to screen adolescents for below-grade reading in health care settings

    Pediatrics

    (2006)
  • P.F. Bass et al.

    A shortened instrument for literacy screening

    J Gen Intern Med

    (2003)
  • A.M. Arozullah et al.

    Development and validation of a short-form, rapid estimate of adult literacy in medicine

    Med Care

    (2007)
  • Cited by (247)

    • Development and initial validation of the falls health literacy scale

      2022, Maturitas
      Citation Excerpt :

      However, the questionnaire was developed to evaluate behaviour change and was insufficient to capture the relevant dimensions of fall-related health literacy. Similarly, earlier versions of health literacy tools have several limitations, including testing only one dimension of health literacy (i.e. reading and writing skills), not assessing one's ability to improve health and not being subjected to vigorous psychometric analysis [12, 13]. To fill these gaps, we developed a comprehensive health literacy tool specific to the fall context, called the Falls Health Literacy Scale (FHLS).

    • Exploring health literacy and self-management after kidney transplantation: A prospective cohort study

      2022, Patient Education and Counseling
      Citation Excerpt :

      Given this, we believe research on health literacy and self-management during the pre-transplant work-up is needed. Furthermore, previous research shows that health literacy is indeed modifiable and suggests that professionals should focus to a greater extent on tailoring the delivery of information and organization of care offered to promote better understanding among all patients [27]. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

    • A process-knowledge approach to supporting self-care among older adults

      2022, Psychology of Learning and Motivation - Advances in Research and Theory
      Citation Excerpt :

      It is unclear whether the same concept should be applied to so many patient factors (Chinn, 2011; Wills, 2009). A related challenge is the fact that different health literacy tests, which presumably measure the same construct, vary in the abilities that they tap into (Jordan et al., 2011). For example, performance on the Rapid Estimate of Adult Literacy in Medicine (REALM, Davis et al., 1993), a common measure of health literacy, is better predicted by measures of crystalized mental abilities (general knowledge) than by measures of fluid mental abilities (working memory, processing speed, or other aspects of processing capacity).

    View all citing articles on Scopus
    View full text