The mathematical relationship among different forms of responsiveness coefficients

Abstract

Background

Little consensus exists regarding the most appropriate measure of responsiveness. While most indices are variants on Cohen’s effect size, the mathematical relationships among these indices have not been elucidated. Consequently, the health-related quality of life (HRQL) literature contains many publications in which a variety of different indices are computed and differences among them noted. These differences are completely predictable when the underlying analytical form of each coefficient is explicated.

Methods

In this paper, we begin with a mathematical analysis of the variance components underlying an observed change score. From this, we determine analytically the relationships among the more commonly used indices of responsiveness.

Conclusions

Based on this analysis, we conclude that Cohen’s effect size and the Standardized Response Mean are the two most appropriate measures, as each provides unique information and each best captures an important relation between treatment effect and variability in response. However, the latter should be interpreted with caution, as under some circumstances, any measure based on variability in change scores can give misleading information. On this basis, we recommend that future analysis of responsiveness be restricted to the Cohen effect size to ensure interpretability and comparability with treatment effects in other domains.

Appendix 1: Derivation of variance components

As described in the paper, from Classical Test Theory, any observed score is considered to have two components, a true score and an error:

$$ O_{ij} =p_i +e_{ij} $$

(24)

where O _ij is the observed score, p _i represents the true score of patient i, and the error term, e _ij is associated with the jth observation on patient I. By definition, errors have a mean of 0, and a standard deviation, σ ² _e . From this equation, the variance of baseline scores is a sum of variances due to differences between subjects and measurement error.

$$ \sigma_{\rm baseline}^2=\sigma_p^2+\sigma_e^2 $$

(25)

If we now consider a pretest–posttest situation, where there is no treatment effect, the difference between observed pretest and posttest, from Eq. 1, is:

$$ D_i=O_{i2}-O_{i1}=(p_i-p_i)+(e_{i2}-e_{i1})=(e_{i2}-e_{i1}) $$

(26)

where the pretest corresponds to j = 1 and the posttest to j = 2. From this, the variance of the difference scores is:

$$ \sigma _{{\rm pre}\hbox{-}{\rm post}}^2 =\sigma _e^2 +\sigma _e^2 =2\sigma _e^2 $$

(27)

If there is a treatment between the pretest and the posttest, Appendix Eq. 3 contains an additional term corresponding to the effect of treatment on patient i, which we will call t _i . This can be viewed in turn as the sum of the overall treatment effect, T, and the difference between the overall treatment and the response of patient, i, which amounts to a (p  ×  T) interaction.

$$ \hbox{Change}_i=T+(t_i-T)+(e_{i2}-e_{i1}) $$

(28)

Since the variance of the overall change, T, is zero, the variance of the change score is then:

$$ \sigma _{\rm change}^2 =\sigma _e^2 +\sigma _e^2+\sigma _{p\times T}^2=2\sigma _e^2+\sigma _{p\times T}^2 $$

(29)