Step 1: Establishing a WHO steering group and defining the scope of work

A WHO steering group (composed of staff from the Departments of Reproductive Health and Research and Maternal, Newborn, Child and Adolescent Health) was established to lead the development of this new ‘living guidelines’ approach to prioritising, updating and developing individual WHO MPH recommendations.25 As existing WHO recommendations in this area are largely (but not exclusively) derived from Cochrane systematic reviews, editorial staff from Cochrane Pregnancy and Childbirth were co-opted to form the Technical Working Group for this approach, working closely with the WHO steering group to collate, and appraise the status of evidence. The processes and methods used by the WHO steering group are consistent with international standards and best practices for guideline development, as recommended and implemented by the WHO GRC.20

The scope of work included several priority MPH topics, including health conditions (hypertensive disorders of pregnancy, postpartum haemorrhage (PPH), peripartum infections, preterm birth), interventions (induction of labour, augmentation of labour, caesarean section) and packages of care (antenatal care, intrapartum care, postnatal care and health promotion interventions). As of December 2016, WHO had published 10 guidelines (with 177 individual recommendations) on these topics.1–10 For each included topic, the working group considered both existing recommendation questions (ie, questions where a WHO recommendation already exists) and new questions (ie, questions that have been identified as critical to improving clinical practice or health programmes, but for which there have been no WHO recommendations previously). By question, we mean research questions structured using the Participant, Intervention, Comparison, Outcome format, however in a few instances we identified different types of research questions (such as the definition of a health condition).

Step 2: Defining a framework and establishing an independent advisory group

From March to October 2016, we conducted a focused literature search to identify existing frameworks and approaches relating to ‘living’ systematic reviews and guidelines.26–30 The WHO steering group drafted a framework (figure 1) as part of the standard operating procedures (SOPs) for operationalising the ‘living guidelines’ approach. These SOPs included description of the roles and responsibilities for the groups involved (online supplementary table 1) and how questions would be prioritised and updated. It was designed to be congruent with the WHO guideline development internal procedures and will be updated on an ongoing basis (available on request). The first draft of the document was prepared in October 2016 by the WHO steering group. Initial feedback was sought from the WHO GRC secretariat, WHO guideline developers in other thematic areas, guideline methodologists and members of the Cochrane Pregnancy and Childbirth editorial team. The SOPs were revised based on their input.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Framework for assessing priority for updating (adapted from Garner et al [29]). *By ‘revalidate recommendation’, we mean reaffirming the existing recommendation in terms of direction and technical content. The wording of the recommendation may be revised to improve clarity.

In December 2016, we established a new, independent Executive Guideline Steering Group on Maternal and Perinatal Health Recommendations (the Executive GSG) to play an advisory role in helping WHO maintain an up-to-date and balanced portfolio of recommendations.31 The 15 GSG members were selected to ensure a balance of geographical location, gender and expertise. The GSG includes maternal, newborn and health systems experts from the six WHO regions with no significant conflicts of interests. GSG members are content experts, skilled in the conduct and critical appraisal of evidence and in implementation of evidence-based recommendations at different health system levels. The SOPs were circulated to the GSG for feedback and comments were incorporated. We continued to develop the SOPs through its application to this first cycle of prioritising and updating (described below). The approach was also discussed and formally endorsed by the WHO GRC.

Figure 1 shows the framework underpinning the prioritisation and updating process as described in the SOPs, adapted from the framework for prioritising systematic reviews for update developed by Garner et al.29 It is composed of three critical decision points that the Executive GSG would use to assess whether an update is warranted (in step 4):

1. Does the published recommendation still address a current topic for clinical practice or programmes?

2. Are there any new studies or new information relating to the recommendation question?

3. Will any new studies, information or data substantively change the evidence base for the recommendation or the recommendation’s credibility?

Step 3: Prioritising questions and mapping evidence

In February 2017, the WHO steering group conducted an international stakeholder prioritisation exercise via an online survey. The group developed a long list of stakeholders in maternal and newborn health across a range of disciplines, organisations and settings for potential participation, including clinicians, researchers, implementing organisations, professional associations, United Nations agency staff and consumer advocacy organisations. The survey was also circulated to the executive boards and national society presidents of two professional associations (the International Federation of Obstetricians and Gynaecologists and the International Confederation of Midwives). Some, but not all, of the individuals invited to participate had experience on WHO MPH guideline panels. The survey included the list of existing WHO MPH recommendations, and participants were asked to prioritise all questions as ‘low’, ‘medium’, ‘high’ priority for updating the recommendation. They were also asked to nominate any new questions they considered a priority. An ‘unknown’ option was available for situations where the survey participant felt they did not have sufficient expertise or information to make a ranking. Participants were asked to prioritise based on the following criteria which were developed to align with the framework in figure 1:

• The question is of public health importance (burden of suffering is high and expected effectiveness of the intervention in the question has the potential to reduce that burden).

• There is new evidence that has the potential to change the existing WHO recommendation.

• There is new evidence that could impact on the credibility of the existing recommendation.

• There is potential for an updated WHO recommendation to significantly change clinical practice globally.

­

Using the responses, a prioritisation score was calculated for each question (ie, percentage of survey respondents who ranked the question as a medium-priority or high-priority). For example, one question was ranked by 91 participants: 36, 37 and 16 participants ranked this question as low, medium and high priority, respectively (2 participants responded ‘unknown’). The prioritisation score ((37+16) / 91) was 58.2% for this question.

The online survey was conducted over a 6-week period in English only. A total of 264 stakeholders was invited to participate via email, with reminders at 2 weeks and 4 weeks. In total, 124 stakeholders participated (62% response rate), with all WHO regions represented (ranging from 5.6% of participants from South-East Asia to 28.2% participants each from Europe and the Americas) (table 1). Sixty-two participants (50.0%) described themselves as primarily an obstetrician/gynaecologist, 18 participants (14.5%) as a researcher, 15 participants (12.1%) as a nurse or midwife, and 10 (8.1%) as an epidemiologist. The remainder included neonatologists (8, 6.5%), policymakers (3, 2.4%), clients/patients (2, 1.6%), a reproductive health programme manager (1, 0.8%), funding agency staff (3, 2.4%) and staff of a professional association (2, 1.6%). Across all 177 questions included in the survey, the median of all prioritisation scores was 36.2% (range 14.1% to 73.6%). A total of 27 questions had a prioritisation score greater than 50%, and 6 questions had a prioritisation score greater than 60%.

In parallel with the survey, the Technical Working Group (ie, the group performing mapping and surveillance of evidence for prioritisation) and the WHO steering group conducted an evidence mapping exercise for all recommendations underpinned by Cochrane reviews. The groups mapped the existing WHO MPH recommendations to their corresponding question and systematic review(s) of benefits and harms, and conducted new searches of the Cochrane Pregnancy and Childbirth’s Specialised Register of Controlled Trials to identify the number of new trial reports that had not yet been incorporated into the existing systematic review. For questions where a large number of new trial reports were identified, a brief, narrative overview of the findings of the trials with the largest sample sizes was prepared. These findings were summarised in a spreadsheet (along with the corresponding prioritisation scores) and shared with the GSG members ahead of the face-to-face meeting described below.

Step 4: Selecting the high-priority questions for updating or developing WHO recommendations

In March 2017, a face-to-face meeting of the Executive GSG was held. Five observers from partner organisations were also invited to attend (representatives of professional associations, maternal and newborn health programmes, and funding agencies). The objective was to reach consensus on a short list of questions that should be prioritised for recommendation development or updating over the next 2 years.

At the meeting, the new approach was discussed and the GSG agreed on the definitions of priority levels they would assign to each question (table 2). Over the course of 2 days, the Chair facilitated a question-by-question assessment, informed by the prioritisation scores and evidence mapping. Questions often differed in the volume of evidence available, as well as the population, setting and health worker cadres they applied to. A set of ‘decision rules’ for identifying high-priority questions emerged from these discussions (box 1) and were used to operationalise the prioritisation framework presented in figure 1. The group reached consensus on a list of 36 high-priority questions (of which 6 were new questions) across different topics—intrapartum care (6 questions), peripartum infections,2 hypertensive disorders of pregnancy,8 induction of labour,4 PPH,8 antenatal care3 and health promotion.5 A spreadsheet detailing the 36 high-priority questions was published online on a WHO web page.31 A separate web page was also published, where stakeholders can nominate new priority MPH questions for the GSG to consider for future recommendations.32

Step 5: Developing recommendations with a ‘living’ guideline development group

After the meeting, the Cochrane Pregnancy and Childbirth editorial team initiated the creation or updating of Cochrane reviews on the effectiveness of interventions related to the high-priority questions and developed the corresponding evidence profiles. In order to help guideline panels make evidence-informed decisions, WHO staff developed GRADE evidence-to-decision (EtD) frameworks for each high-priority question.33 These frameworks have been used previously in WHO MPH guidelines, and allow explicit and transparent assessment of prespecified criteria (including desirable and undesirable effects, values and preferences, resource requirements, equity, acceptability and feasibility).34 Assessments of how stakeholders value the outcomes, and on equity, acceptability and feasibility, were informed by recent qualitative evidence syntheses on antenatal and intrapartum care.35–37 Where needed, new qualitative evidence syntheses were conducted.

In June 2017, the WHO steering group identified approximately 50 experts and stakeholders from the six WHO regions to constitute a WHO MPH ‘living’ guideline development group (GDG). This is a diverse pool of stakeholders skilled in critical appraisal, guideline development, implementation, clinical practice, policy and programmes relating to maternal and newborn health. The mix of experts and stakeholders was based on considerations of previous GDG membership for existing WHO MPH guidelines, while allowing for introduction of new members with specific expertise relating to content, programmatic area or guideline development methods. ‘Living guidelines’ panels were drawn from this pool by matching recommendations under consideration with individual expertise. Thus a group of approximately 15–20 individuals from the MPH GDG (with appropriate geographical and gender balance, and thematic expertise) was convened virtually on an as-needed basis.25 Partner organisations are invited to observe and contribute to guideline panel meetings, but are not permitted to vote (in the event a formal vote is required).20

In accordance with WHO guideline development standards, the GDG reviews the EtD frameworks, including the evidence profiles for benefits and harms, how stakeholders value outcomes, resource requirements, cost-effectiveness, acceptability and feasibility and the intervention’s impacts on equity. During the ‘living guidelines’ panel meeting, the GDG formulates the updated or new recommendations, and prepares clarifying remarks, implementation considerations and research priorities. These recommendations are peer reviewed by external experts, and then reviewed by the WHO GRC as part of the organisation’s quality assurance process prior to publication.

Between April 2017 and December 2018, 25 new or updated recommendations had been published based on the questions prioritised by the process described above.12 21 38–42 We briefly describe the updating of prioritised recommendations on PPH prevention and management as illustrative examples in boxes 2 and 3.38 All steps were conducted with a view to a sustained ‘living guidelines’ process of literature surveillance, prioritisation and updating of WHO maternal and perinatal recommendations beyond 2019.

Operationalising the living guideline approach

Guideline developers worldwide are facing the challenge of ensuring that recommendations are up to date in the context of a rapidly evolving or growing evidence base.16 17 Through this new ‘living guidelines’ approach, WHO can respond rapidly to new and important evidence without compromising the quality and rigour of its guideline development process. Implementing this approach adapts techniques already used in development of rapid advice guidelines (such as for public health emergencies), including early engagement with a standing guideline panel members and peer reviewers, and the use of virtual guideline panel meetings. The most compelling advantages of this approach are more rapid availability of updated recommendations and reduced costs through a process that maintains rigour and transparency. The average WHO MPH guideline (containing multiple recommendations) typically takes 1–2 years and considerable resources to develop. The efficiencies of the living guidelines approach are largely driven by, first, focusing efforts only on those individual recommendations where an update is warranted. This avoids the substantial costs associated with updating reviews and convening guideline panels, even where the strength or direction of a recommendation is unlikely to change. Second, the use of more frequent online GDG meetings, rather than in-person GDG meetings requiring international travels, also reduces costs and is more time-efficient for GDG members.

This new approach to dynamically updating individual WHO recommendations brings several new challenges. There is not yet clear consensus on the most efficient and effective methods for prioritising the recommendation questions that should be updated, and such standards are needed.27 43 While the prioritisation survey of international stakeholders helped inform the GSG’s discussions, the judgements of survey participants on a recommendation’s credibility or need for updating may be subjective. A systematic review by Vernooij et al reported that many organisations that issue guidelines do not provide adequate guidance on methods and procedures for updating.44 While we aimed to engage with a range of stakeholders in prioritising questions for updating, better engagement with those who will be affected by the recommendations (ie, women and their families) and elicitation of their perspectives and preferences are needed. The WHO steering group is exploring ways to better engage women in future prioritisation cycles.

Qualitative evidence syntheses on the views and perspectives of all stakeholders and consumers are being used increasingly in the development of WHO guidelines.45 These syntheses have proved critical to ensuring that the perspectives of women and their families are explicitly considered in WHO MPH recommendations.36 However, the scope of evidence surveillance and prioritisation of systematic reviews for rapid updating in this project has been largely informed by findings from quantitative systematic reviews on the effects of interventions. Further innovation is needed in order to apply a ‘living’ systematic review approach to qualitative evidence syntheses46 that may inform the EtD framework criteria on whether an intervention is acceptable to key stakeholders and whether an intervention is feasible to implement. Further work is also needed to apply a ‘living’ systematic review approach to other types of evidence that contribute to guideline development and decision-making in public health, such as cost-effectiveness studies, economic analyses or surveys on the views of key stakeholders.

Evidence mapping is itself a time-consuming process, particularly with a large number of recommendations. For a ‘living guidelines’ approach to work, guideline developers need to implement systems for continuous, systematic literature surveillance, rapid appraisal of the potential impact of new evidence and (where appropriate) rapid updating of systematic reviews.25 47 Use of machine learning for literature surveillance and evidence synthesis is being investigated which could make these processes more efficient in the near future.48 49 We plan to explore methods on how to rapidly assess whether systematic review update is warranted. Some authors have explored the use of statistical tests to assess the likelihood of whether new evidence will change the conclusions of a systematic review; however further research is required.50–52 The collaboration between WHO and Cochrane Pregnancy and Childbirth has proven essential to mapping of new trial reports in relation to Cochrane reviews, coordinating updates and preparing evidence profiles, as well as ensuring an agile response to new priorities. It should be acknowledged, however, that such collaboration does not exist across all WHO guideline priorities and technical departments and other mechanisms of maintaining a robust repository of evidence supporting WHO recommendations will be required to support ‘living guidelines’.

As Akl et al have emphasised, ‘living’ recommendations require ‘living’ systematic reviews, evidence profiles, EtD frameworks, guideline panels and budgets.25 To this we would add the need for a ‘living’ prioritisation process, in addition to ensuring that institutional policies and approval mechanisms can respond flexibly to updated recommendations. A 2017 systematic review by Martinez Garcia et al identified considerable variability in methods for prioritising updates of systematic reviews and guidelines; a prioritisation tool is under development.53 54 In the context of a large guideline portfolio and evidence base as in this case, a ‘living’ prioritisation process is critical to determining which systematic reviews demand resources for more frequent surveillance and possible updating using new technologies comprising both human and machine effort. It is important, however, to carefully examine the need for and implication of investing in living systematic reviews, even for a subset of prioritised recommendations. Switching systematic reviews to a ‘living’ mode is only worthwhile for questions where research remains active or there are lingering controversies, but less so in situations where questions have been prioritised for update based on considerations other than research evidence. While we maintained surveillance of the Cochrane Pregnancy and Childbirth database for new, impactful evidence, none of the Cochrane reviews underpinning this group of recommendations have yet transitioned to continuous evidence surveillance or to an updating model that combines human and machine effort.47 We plan to explore such a model to shorten the time required to prepare the evidence base for consideration by a guideline panel.

Updating individual recommendations also presents a challenge to dissemination, communication and subsequent implementation. WHO has a responsibility to ensure that recommendations on interventions that can impact health are made available to WHOMember States as soon as possible. However, it is likely that national healthcare decision-makers can only maintain a limited number of adoption and adaptation processes simultaneously, and these have not been developed with ‘living’ recommendations in mind. Prioritisation and updating processes would benefit from explicit consideration of the likely impact of updating recommendations on these stakeholders.

MPH recommendations have historically been packaged within a thematic guideline as a static paper or electronic document. Digital tools or platforms that can reflect the latest, up-to-date recommendations while linking to the broader guideline will be needed to disseminate those recommendations developed or updated within this new approach. For example, the WHO Reproductive Health Library now hosts the latest version of all WHO’s MPH recommendations, with a brief history of the recommendation’s previous iterations and planned updates.55 Other innovations (such as the MAGIC project’s interactive guideline platform, the BMJ’s Rapid Recommendations project and GRADEpro GDT) are also aimed at providing flexible, interactive digital platforms that can be easily updated if required.56–58

Policymakers and clinicians will need to be kept informed of important changes in individual recommendations on a more continuous basis, and implementation tools (such as job aids, checklists, decision support algorithms and the like) will also need to become more responsive to recommendation updates. Digital platforms that can connect guideline development and publication through to dissemination and implementation should be developed, to facilitate knowledge transfer to and uptake by end users.