Introduction
What is new?
Key findings- •
Almost half of the overviews (53.3%) mentioned overlaps, whereas the remaining overviews did not.
What this adds to what was known?- •
This is the first systematic analysis of overlaps in reviews of reviews.
- •
Development and validation of a measure (corrected covered area [CCA]) to calculate the actual degree of overlap in overviews.
What is the implication and what should change now?- •
Insufficient reporting of the quality of systematic reviews complicates the production of overviews, in particular with respect to overlaps.
- •
Overlaps must be reported in well-conducted overviews, and this can comprehensively be done using the CCA method.
Overviews (reviews of reviews), as a new type of evidence synthesis, have recently gained more interest, such that the number of published overviews is steadily increasing [1]. It is possible that overviews are becoming more prevalent because overviews have potential advantages over systematic reviews (SRs). For example, overviews enable data obtained from different interventions or conditions to be compared, which provides decision makers with a broader summary of the current information available. This is a limitation of SRs, which may be overcome by using overviews [2]. Furthermore, overviews can compare the findings of several reviews and determine the reasons for conflicting results. By identifying the reasons for discordance, users are able to base their decisions on the most current, reliable, and suitable data for their situation [3], [4].
It has been stated that many of the methodological standards for SRs can also be applied to overviews [5]. However, little guidance is available for authors on how to conduct methodologically sound overviews. Interestingly, a descriptive analysis concluded that overviews often have limited rigor [4].
Decisions in health care should be based on all of the available evidence to draw reliable conclusions and to support policy making. Therefore, we most often rely on the SRs [6]. When conducting an overview, one might argue that a decision should be based on an enormous body of evidence. Even if this holds to be true, it may be difficult and challenging to survey all of the available evidence that is gained from primary studies mainly because they are often included in more than one review. Additionally, a meta-analysis of meta-analyses may also be difficult to conduct because many of the primary studies will usually be included in more than one meta-analysis. Therefore, pooling the results of all of the reviews would give disproportionate statistical power to multiple primary studies [2]. An informal analysis that sums the results of the reviews could also introduce significant overlap and result in many primary studies being included more than once, which would lead to biased results. This problem should be addressed by developing standard methods for authors of overviews to follow [7].
However, to the best of our knowledge, the degree of overlap in overviews has not been examined systematically. Thus, the amount of overlapping data in overviews remains unknown. First, we aimed to determine whether authors mentioned overlaps in their overviews, and if so, we examined how the authors dealt with these overlaps. Second, we examined the actual overlap in published overviews and suggested potential measures for handling these overlaps.