What is new?
Key findings- •
There are a range of tools available for appraising applicability of findings from systematic reviews; however, there is a lack of consensus regarding terminology and tool content. There are challenges in appraising applicability for systematic reviews of complex interventions and a lack of guidance on how to do this. Review authors can use three main approaches to address these challenges: (1) increased use of both qualitative and quantitative methods to explore causal pathways and assess variations in effect across important characteristics; (2) improved description of studies and their context; and (3) appraisal of applicability for at least one primary target population, setting, and context.
What this adds to what was known?- •
The study summarizes controversies in existing guidance for assessing applicability in systematic reviews.
- •
It also provides recommendations for assessing applicability in systematic reviews of complex interventions using examples from existing reviews.
What is the implication and what should change now?- •
There is a need for further guidance to establish consensus on transparent reporting of applicability judgments, improved description of contextual factors, and assessment of causes for variations in effect.
Carefully assessing applicability is important in the use of all research, whether single studies or systematic reviews. Applicability can be considered in terms of individuals and their specific circumstances or can be extended to include populations and settings [1]. In the context of systematic reviews of effects, applicability can be understood as an assessment of whether the findings of a review can be applied in a particular context or population [2]. A number of related terms are used in the literature (e.g., generalizability, external validity, directness, transferability), and there is a lack of standardization in how these terms are used. We have summarized and defined the most widely used terms in Table 1.
There is increasing focus in systematic reviews on judging the applicability of results to a target (or intended) population and setting. For example, The Cochrane Collaboration now requires every review to include a section that discusses the applicability of the evidence and a summary of findings table, which is tailored to a specific population and setting. Although systematic reviews cannot possibly judge applicability to all possible settings and populations, the authors of systematic reviews are well placed to provide information needed for judging applicability, such as characteristics of included populations and settings, and an assessment of both relative and absolute effects.
Assessing the applicability of evidence may be challenging for users of systematic reviews of complex interventions. As described in this series [3] and elsewhere, the features that make an intervention more complex include number of interacting components within experimental and control interventions, number and difficulty of behaviors required by those delivering or receiving the intervention, number of groups or organizational levels targeted by the intervention, number and variability of outcomes, and the degree of flexibility or tailoring of the intervention permitted [4], [5]. Assessing applicability is more challenging as interventions become more complex because of greater diversity of, and interactions within and between, elements of the complex intervention and other characteristics such as the setting and context.
Numerous checklists and items have been developed for appraising applicability and other related concepts such as transferability. These checklists are mostly focused on attributes of patients, providers, and setting, with very little attention to characteristics of the intervention and intervention complexity.
In this article, we summarize the issues faced in assessing the applicability of research findings from systematic reviews of the effects of complex interventions, assess existing guidance on applicability, provide additional guidance for systematic review authors in documenting relevant considerations to inform applicability assessments, and make suggestions regarding where efforts to develop additional guidance could be directed.