Table 1

Major themes and corresponding qualitative codes

ThemesLevel 2 codesLevel 1 codes
WHO’s guideline development reforms represented a transformational shift in its approach to producing clinical and public health recommendationsTriggers of the reformDominance of expert opinion
Impact of the reformHandbook with comprehensive guidance
Transformed WHO’s guideline development process
Institutionalisation of evidence-based principles
More consistent use of systematic reviews
Independent evaluation and feedback by the GRC has strengthened the legitimacy of the decision-making processes underlying WHO’s recommendationsIndependent evaluation strengthens legitimacyGRC process has helped recommendations stand up to criticism
WHO guideline development efforts are not yet designed to systematically make use of all relevant research evidence needed to inform decisions about complex interventionsChallenges with retrieving and appraising evidence to inform complex interventionsNature of WHO guidelines becoming more complex
Evidence from beyond RCTs needed to inform recommendations
Challenges with formulating systematic review questions that capture broader range of evidence
Challenges with and perceptions about GRADEGRADE struggling with qualitative evidence
Dissatisfaction with how GRADE evaluate non-randomised study designs
Challenging with rigid application of GRADE
GRC process perceived rigid/complicated
Misperceptions about GRADE only being applicable to evidence from RCTs
Need for more sophisticated understanding of GRADE
Dialogues to address challengesIncreasing awareness within GRC about difficulties
Constructive dialogue with GRC and methodologists crucial
GRADE approach evolving to become more applicable to broader range of evidence
WHO’s guideline development reforms do not currently apply to all outputs published from all of WHO’s technical units and scientific advisory committeesBypassing of formal guideline development processGuidance being issued outside process overseen by GRC
Tempting to circumvent GRC process
All issued guidance could benefit from independent evaluation, monitoring and feedbackDisorganised approach to managing guidance produced outside GRC requirements
Similar quality assurance needed for other guidance
  • GRADE, Grading of Recommendations Assessment, Development and Evaluation; GRC, Guidelines Review Commitee; RCT, randomised controlled trial.