Article Text

Priority setting in early childhood development: an analytical framework for economic evaluation of interventions
1. Stéphane Verguet1,
2. Sarah Bolongaita1,
3. Anthony Morgan1,
4. Nandita Perumal1,
5. Christopher R Sudfeld1,
6. Aisha K Yousafzai1,
7. Günther Fink2,3
1. 1Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Boston, MA, USA
2. 2University of Basel, Basel, Switzerland
3. 3Swiss Tropical and Public Health Institute, Allschwil, Switzerland
1. Correspondence to Professor Stéphane Verguet; verguet{at}hsph.harvard.edu

## Abstract

Background Early childhood development (ECD) sets the foundation for healthy and successful lives with important ramifications for education, labour market outcomes and other domains of well-being. Even though a large number of interventions that promote ECD have been implemented and evaluated globally, there is currently no standardised framework that allows a comparison of the relative cost-effectiveness of these interventions.

Methods We first reviewed the existing literature to document the main approaches that have been used to assess the relative effectiveness of interventions that promote ECD, including early parenting and at-home psychosocial stimulation interventions. We then present an economic evaluation framework that builds on these reviewed approaches and focuses on the immediate impact of interventions on motor, cognitive, language and socioemotional skills. Last, we apply our framework to compute the relative cost-effectiveness of interventions for which recent effectiveness and costing data were published. For this last part, we relied on a recently published review to obtain effect sizes documented in a consistent manner across interventions.

### Drawing from the scoping review: assessing value for money of interventions that promote ECD

With the scoping review of the literature, two important features specific to ECD emerged that need to be considered when assessing the intrinsic impact of interventions. First, ECD is multidimensional. There is a great variety of domains of ECD that have been researched in the literature. Most developmental assessments of children under the age of 3 years focus on four domains: (gross and fine) motor skills; cognitive skills, (expressive and receptive) language skills and socioemotional skills.29 While these domains of development can be affected by a single intervention in principle, interventions may have specific focal areas (such as shared book reading on early language or socioemotional development) that are explicitly targeted. Second, improvements in any domain are continuous. Universal interventions designed to promote ECD for all children, such as parenting and stimulation suggest benefits are possible for all children. There is no ‘maximum’ level of development and population-level future improvements in these outcomes should be expected.30

With these two critical considerations in mind, we can define the most effective intervention that promotes ECD as one that achieves the largest possible improvements across the four domains of development (denoted d’s). To identify the most cost-effective interventions, we can then compare the overall improvements in ECD achieved by an intervention with the required cost. Following traditional cost-effectiveness analysis guidelines,25 26 we can compute ‘incremental cost-effectiveness ratios’ (or ICERs) for interventions as: , where Ci and C0 are the costs of intervention i and of the status quo (ie, the control or standard-of-care group), respectively, and Bi and B0 are the benefits of intervention i and of the status quo, respectively.

The incremental nature of the cost calculations (in the ICER computation above) is important because potential intervention programmes can either be implemented on top of existing programmes or replace them, with very different budgetary implications. Benefits can be computed across all four domains in the same way, that is, by comparing the average development of children with the average development in the four domains with status quo. As a result, the full ICER can be expressed as , where Zd,i is the average standardised developmental outcome in domain d with intervention i, Zd,0 is the average developmental outcome in the same domain without intervention (status quo) and wd is the specific weight given to each domain d. While all four domains have been associated with improved later life outcomes, policymakers may wish to give higher priority to some domains and rank interventions accordingly. If all domains are given a uniform weight of 1, the denominator of this formula simply becomes the sum of Z-score differences in developmental outcomes with and without the intervention across the four domains. A smaller ICER means a more cost-effective intervention; interventions can thus be rated directly with respect to the ICER computed.

### Review of Aboud and Yousafzai (2015): impact and value for money of interventions that promote ECD

Figure 1 details our study selection process including all the interventions reviewed and retained based off the original review by Aboud and Yousafzai.19 Ultimately, 15 studies (out of the 34 studies initially selected)31 32 were selected for subsequent analyses. These 15 studies were psychosocial stimulation interventions with direct impact on cognitive and language skills and targeting children between ages 0 and 24 months.

Figure 1

Flow diagram of each study selection process. CEA, cost-effectiveness analysis.

Online supplemental table S1 details the interventions reviewed and provides selected information including a brief description of the intervention arms, the workforce implications, the impact and effectiveness estimates for the two domains of cognitive and language skills (among the four domains identified above) and other benefits and reported cost (if any). The main interventions identified for which we could obtain a developmental impact (estimated in Cohen’s d units here) in at least one of the domains included integrated enhanced nutrition and responsive stimulation, child development messages (eg, short films, play materials), home visiting programmes, community-based parenting group sessions and psychosocial stimulation (online supplemental table S1). Only two studies31 32 reported cost estimates in addition to a developmental impact.

### Supplemental material

We then report on the summarised effectiveness measures and the corresponding estimates of costs using our ingredients-based approach (table 1). We observe a range of effectiveness estimates from −0.41 (95% uncertainty range (UR): −0.82 to 0.00) to 1.80 (95% UR: 1.21 to 2.18), in this case in the cognitive domain from the intervention arms in Lozoff and colleagues (2010) (psychosocial and nutrition intervention with professional educators).19 33 As for costs, we computed unit costs varying from $1 per child from Aboud and colleagues (for a psychosocial intervention with home visits delivered by already employed community health workers in Bangladesh)31 to$ 2172 per child from Lozoff and colleagues (for an integrated psychosocial and nutrition intervention delivered by professional educators in Chile);33 when using standardised costs for LMICs, the unit costs per child were $24 and$3519, respectively.

Table 1

Summary of effectiveness (intervention effect that is, Cohen’s d effect estimates; 95% uncertainty ranges given in parentheses) and estimated cost (in total and per child targeted) for each intervention that promotes ECD included in this study

### Applying the economic evaluation framework

Combining cost and effectiveness measures, we were able to assess value for money, that is, to compute ICERs for 12 interventions (table 2). These value-for-money (or cost-effectiveness) estimates enable ranking and possible prioritisation of interventions: in other words, having positive returns to a given intervention does not directly imply that such an intervention should be prioritised if other interventions can achieve similar outcomes at a lower cost. These cost-effectiveness estimates can help prioritise those interventions with greater returns on ECD outcomes for similar budget expenditure impact. When using an averaged effect estimate (mean of the effect estimates across the domains for which an effect size was reported), the two most cost-effective interventions were the psychosocial interventions by Aboud and colleagues (home visits) and Aboud and colleagues (group sessions),31 with ICERs of $29 (95% UR:$23–$37) and$49 ($40–$63) per SD increase in domain-specific ECD, respectively (using LMIC-standardised costs), while the least cost-effective interventions would be the psychosocial intervention by Hamadani and colleagues34 and the psychosocial and nutrition intervention with professional educators by Lozoff and colleagues,33 with ICERs of $4112 ($2225–33 925) and $5063 ($3228–18 046) per SD increase in domain-specific ECD, respectively. When using the combined effect estimate (sum of the effect estimates across the domains), the most cost-effective interventions remained the same, with ICERs between $10 ($8–$12) and$16 ($13–$21) per SD increase in domain-specific ECD. The least cost-effective interventions were also the same, with ICERs between $2532 ($1614–$9023) and$5063 ($3228–$18 046) per SD increase in domain-specific ECD, respectively.

Table 2

Summary of estimated cost-effectiveness (ICER, 95% uncertainty ranges given in parentheses) for each intervention that promotes ECD included in this study

## Discussion

With increased attention and global funding towards ECD,35 36 an analytical framework for ranking interventions that promote ECD across settings will be needed. In this paper, we have introduced a basic value-for-money approach designed to make interventions directly comparable across settings, and we show that the computed ICERs would vary widely across settings.

As it is often the case for health interventions, cost-effectiveness rankings are dominated by costing aspects. Rather remarkably, we found the cost ratio of the most expensive relative to the cheapest intervention exceeded 100:1, while impacts on ECD across a rather diverse set of interventions were relatively similar. Unsurprisingly, the most cost-effective interventions were those with the lowest cost of implementation; the most cost-effective intervention analysed was a psychosocial stimulation intervention implemented with close to zero additional staff resources in Bangladesh.31 (In this study, group sessions were delivered by already employed (uncompensated) village peer educators, while home visits were delivered by community health workers who would normally visit households to talk about health. In this respect, the use of volunteers in this study might face challenges at scale such as staff retention.) Lower ICERs associated with higher costs of implementation do not necessarily mean that the returns to these programmes will not be high (see the case for instance of the psychosocial and nutrition intervention in Chile33). In addition, many analyses have shown that both private and social returns to ECD programmes in high-income countries (where implementation costs are higher) are rather substantial.

This is to our knowledge the first attempt to rank different interventions across countries, and evidently, it comes with a number of important limitations. First, we only could source a few studies of interventions that would comprehensively report on intervention impact in a consistent manner across our choice of ECD outcomes (figure 1; online supplemental table S1). As a result, we had little coverage of all possible interventions and settings in which they could be implemented. Second, we used a standardised measure for intervention effect (using Cohen’s d) across studies, while there is some variety in the measures used to report ECD outcomes per intervention (eg, Bayley scales of infant development, McCarthy score). While the Bayley scales for infant and toddler development37 are considered a gold standard by many researchers, the tool was developed originally for high-income settings and requires formative work to ensure appropriate adaptation, reliability and validity for application in diverse cultural settings (often the tool is not adapted and the psychometric properties are not commonly reported). A large number of tools have been used that measure developmental outcomes in different ways (eg, direct child assessment, caregiver report) that may not be directly comparable. Efforts to develop a measure of development for children less than 3 years of age that can be relatively less of a burden on time and open access are underway, including the global scale for early development tool that may make these comparisons easier in the future.38 39 Third, aggregation of ECD outcomes (eg, cognitive skills added to language skills) within one study could well be limited by potentially high correlations across ECD outcomes in the study, and beyond mere additions, there may be reinforcing multiplicative effects between ECD outcomes. Fourth, the likely long-term effects of interventions were not included in our modelling: for example, interventions that promote ECD may improve human capital outcomes including greater educational attainment and potentially higher wages into future adulthood (with very significant returns). Also, we did not discount ECD benefits and costs in our computed ICERs, even though this could be further done using standard economic evaluation guidelines.25 26 Likewise, we focused on ECD-specific interventions, that is, interventions that primarily target ECD outcomes. As such, we did not include in our review ‘ECD-sensitive’ interventions, that is, interventions that promote ECD indirectly, say, for example, infant and young child feeding promotion, or child multiple micronutrient supplementation (which is designed to reduce anaemia and micronutrient deficiencies), for which a recent meta-analysis of the effects on ECD was published by Prado and colleagues.40 Fifth, given the extreme scarcity of data, our quantitative findings should be interpreted with caution before any recommendation can be drawn. In this respect, important considerations including the distribution in ECD outcomes at the population level or across socioeconomic status (eg, wealth and education levels) as well as more generally any distributional or equity aspects of the ECD interventions were not taken into account; neither were possible mediating effects of the environment (eg, classroom quality) for ECD.41 Last, we envision future work would strengthen consistent and comparable measurement on data collection on the effectiveness side, tied with rigorous economic data on the implementation cost side so that value-for-money arguments for priority setting can be developed. We intend that the work presented will be a first step towards a unified analytical framework that will stimulate conversations on assessment of the relative importance and cost-effectiveness of ECD interventions at the national and global levels. In this respect, fully including ECD outcomes into priority setting is now an urgent necessity.

## Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

## Acknowledgments

We would like to thank Frances Aboud, Michael Boivin, Waldemar Carlo, Sophie Eickmann, Jena Hamadani, Betsy Lozoff, Sylvia Fernandez Rao and Susan Walker for generously providing inputs for the costing of studies.

• ## Supplementary Data

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

## Footnotes

• Handling editor Lei Si

• Contributors SV and GF conceptualised, designed and supervised the study; contributed to the methodology, investigation, analysis and interpretation of data. SV wrote the original draft of the manuscript. SB led the acquisition of data, the analysis and investigation and contributed to the methodology and interpretation of data. AM contributed to data acquisition, preliminary analyses and investigation. NP, CS and AY contributed to acquisition and interpretation of data. All authors critically reviewed and edited the manuscript, approved the final manuscript as submitted and agreed to be accountable for all aspects of the work.

• Funding SV and SB acknowledge funding from the Trond Mohn Foundation and NORAD through BCEPS (#813596). NP, CS and GF received support from the Bill & Melinda Gates Foundation (OPP1198520).

• Competing interests None declared.

• Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

• Provenance and peer review Not commissioned; externally peer reviewed.

• Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.