You are here

Article Text

Article menu

Download PDFPDF

Research
Considerations of complexity in rating certainty of evidence in systematic reviews: a primer on using the GRADE approach in global health
  1. Paul Montgomery1,
  2. Ani Movsisyan2,
  3. Sean P Grant3,
  4. Geraldine Macdonald4,
  5. Eva Annette Rehfuess5
  1. 1School of Social Policy, University of Birmingham, Birmingham, UK
  2. 2Centre for Evidence-Based Intervention, Department of Social Policy and Intervention, University of Oxford, Oxford, UK
  3. 3Pardee RAND Graduate School, RAND Corporation, Santa Monica, California, USA
  4. 4School for Policy Studies, University of Bristol, Bristol, UK
  5. 5Institute for Medical Information Processing, Biometry and Epidemiology, Pettenkofer School of Public Health, Ludwig Maximilian University, Munich, Germany
  1. Correspondence to Dr Paul Montgomery; p.x.montgomery{at}bham.ac.uk

Abstract

Public health interventions and health technologies are commonly described as ‘complex’, as they involve multiple interacting components and outcomes, and their effects are largely influenced by contextual interactions and system-level processes. Systematic reviewers and guideline developers evaluating the effects of these complex interventions and technologies report difficulties in using existing methods and frameworks, such as the Grading of Recommendations Assessment, Development and Evaluation (GRADE). As part of a special series of papers on implications of complexity in the WHO guideline development, this paper serves as a primer on how to consider sources of complexity when using the GRADE approach to rate certainty of evidence. Relevant sources of complexity in systematic reviews, health technology assessments and guidelines of public health are outlined and mapped onto the reported difficulties in rating the estimates of the effect of these interventions. Recommendations on how to address these difficulties are further outlined, and the need for an integrated use of GRADE from the beginning of the review or guideline development is emphasised. The content of this paper is informed by the existing GRADE guidance, an ongoing research project on considering sources of complexity when applying the GRADE approach to rate certainty of evidence in systematic reviews and the review authors’ own experiences with using GRADE.

  • systematic review
  • public health

This is an open access article distributed under the terms of the Creative Commons Attribution-Non commercial IGO License (CC BY 3.0 IGO), which permits use, distribution,and reproduction for non-commercial purposes in any medium, provided the original work is properly cited. In any reproduction of this article there should not be any suggestion that WHO or this article endorse any specific organization or products. The use of the WHO logo is not permitted. This notice should be preserved along with the article’s original URL.

https://doi.org/10.1136/bmjgh-2018-000848

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Footnotes

    • Handling editor Seye Abimbola

    • Contributors Conceptualisation: all authors. Data curation: AM. Formal analysis: PM, AM, SPG and EAR. Funding acquisition: PM. Investigation: PM, AM and SPG. Methodology: all authors. Project administration: PM. Supervision: PM and EAR. Validation: all authors. Visualisation: PM, AM and SPG. Writing and original draft preparation: PM and AM. Writing, review and editing: all authors.

    • Funding This project was prepared as part of the GRADE Guidance for Complex Interventions, funded by the Economic and Social Research Council (ES/N012267/1). Additional funding was also provided by the World Health Organization Department of Maternal, Newborn, Child and Adolescent Health through grants received from the United States Agency for International Development and the Norwegian Agency for Development Cooperation.

    • Competing interests All authors are current members of the GRADE Working Group. SPG’s spouse is a salaried employee of Eli Lilly and Company and owns stock. SPG has accompanied his spouse on company-sponsored travels.

    • Patient consent Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Data will be made available when the entire study is completed on the University of Birmingham repository.