Major themes and corresponding qualitative codes
| Themes | Level 2 codes | Level 1 codes |
| WHO’s guideline development reforms represented a transformational shift in its approach to producing clinical and public health recommendations | Triggers of the reform | Dominance of expert opinion |
| Impact of the reform | Handbook with comprehensive guidance | |
| Transformed WHO’s guideline development process | ||
| Institutionalisation of evidence-based principles | ||
| More consistent use of systematic reviews | ||
| Independent evaluation and feedback by the GRC has strengthened the legitimacy of the decision-making processes underlying WHO’s recommendations | Independent evaluation strengthens legitimacy | GRC process has helped recommendations stand up to criticism |
| WHO guideline development efforts are not yet designed to systematically make use of all relevant research evidence needed to inform decisions about complex interventions | Challenges with retrieving and appraising evidence to inform complex interventions | Nature of WHO guidelines becoming more complex |
| Evidence from beyond RCTs needed to inform recommendations | ||
| Challenges with formulating systematic review questions that capture broader range of evidence | ||
| Challenges with and perceptions about GRADE | GRADE struggling with qualitative evidence | |
| Dissatisfaction with how GRADE evaluate non-randomised study designs | ||
| Challenging with rigid application of GRADE | ||
| GRC process perceived rigid/complicated | ||
| Misperceptions about GRADE only being applicable to evidence from RCTs | ||
| Need for more sophisticated understanding of GRADE | ||
| Dialogues to address challenges | Increasing awareness within GRC about difficulties | |
| Constructive dialogue with GRC and methodologists crucial | ||
| GRADE approach evolving to become more applicable to broader range of evidence | ||
| WHO’s guideline development reforms do not currently apply to all outputs published from all of WHO’s technical units and scientific advisory committees | Bypassing of formal guideline development process | Guidance being issued outside process overseen by GRC |
| Tempting to circumvent GRC process | ||
| All issued guidance could benefit from independent evaluation, monitoring and feedback | Disorganised approach to managing guidance produced outside GRC requirements | |
| Similar quality assurance needed for other guidance |
GRADE, Grading of Recommendations Assessment, Development and Evaluation; GRC, Guidelines Review Commitee; RCT, randomised controlled trial.